RU2792223C1 - Information coding regarding a set of transformation kernels - Google Patents

Abstract

FIELD: image coding technology.

SUBSTANCE: invention relates to image coding technology, specifically, to information coding regarding a set of transformation kernels. The technical result is achieved by the fact that image decoding comprises a step for obtaining a multiple transformation selection (MTS) index and residual information from the bit stream; a step for extracting transformation coefficients for the block regarding residual information and a step for forming residual selection of the block regarding MTS index and transformation coefficient, where the MTS index indicates the set of transformation kernels, which shall be applied to the block, from possible variants of the conversion kernels sets at least one resolution element from the MTS index resolution element string is extracted based on context coding, while context coding is performed based on the increase and decrease values in the context index related to the MTS index, and one increase and decrease value in the context index can be used to context encode the first resolution element from the resolution elements from the resolution element string.

EFFECT: invention improves an image/video compression gain.

9 cl, 13 dwg, 6 tbl

Description

State of the art

The technical field to which the invention belongs

[1] The present disclosure relates to image coding technology, and more specifically, to encoding information regarding a set of transformation kernels.

Description of the Prior Art

[2] Recently, the demand for high-quality high-definition image/video, such as 4K, 8K, or even more ultra-high definition (UHD) images/video, has been growing in various fields of technology. As the resolution or quality of images/video becomes higher, a relatively larger amount of information or number of bits is transmitted than for conventional image/video data. Therefore, if image/video data is transmitted via a medium such as an existing wired/wireless broadband line, or stored in a legacy storage medium, transmission and storage costs are greatly increased.

[3] In addition, there is growing interest and demand for virtual reality (VR) and artificial reality (AR) content and immersive multimedia such as holograms; and the broadcasting of images/videos showing image/video characteristics different from those of the actual image/video, such as game images/videos, is also on the rise.

[4] Therefore, a highly efficient image/video compression technology is required in order to efficiently compress and transmit, store, or reproduce high-quality, high-resolution images/videos exhibiting various characteristics as described above.

The essence of the invention

[5] According to an embodiment of the present document, a method and equipment for improving image/video coding efficiency is provided.

[6] According to an embodiment of the present document, a method and equipment for encoding an MTS index in video encoding are provided.

[7] According to an embodiment of the present document, a method and equipment for signaling MTS index information is provided.

[8] According to an embodiment of the present document, a method and equipment is provided for signaling information representing a set of transform kernels to be applied to a current block from a plurality of transform kernel sets.

[9] According to an embodiment of the present document, a method and equipment for context encoding or bypass encoding regarding bins for an MTS index is provided.

[10] According to an embodiment of the present document, a video/image decoding method by means of decoding equipment is provided.

[11] According to an embodiment of the present document, decoding equipment is provided to perform video/image decoding.

[12] According to an embodiment of the present document, there is provided a video/image encoding method performed by encoding equipment.

[13] According to an embodiment of the present document, encoding equipment for performing video/image encoding is provided.

[14] According to an embodiment of the present document, a computer-readable digital storage medium is provided that stores encoded video/image information generated according to the video/image coding method disclosed in at least one embodiment of this document.

[15] According to an embodiment of the present document, a computer-readable digital storage medium is provided that stores encoded information or encoded video/image information instructing decoding equipment to perform the video/image decoding method disclosed in at least one embodiment of this document.

[16] According to the present document, the overall efficiency of image/video compression can be improved.

[17] According to the present document, MTS index information can be efficiently signaled.

[18] According to the present document, the complexity of the encoding system can be reduced by efficiently encoding the MTS index information.

[19] The benefits that can be obtained through the detailed example of this document are not limited to the benefits listed above. For example, various technical effects may be envisaged, which may be understood or inferred by those skilled in the art from this document. Accordingly, the detailed benefits of this document are not limited to those expressly stated herein and may include various benefits that may be understood or inferred from the technical features of this document.

Brief description of the drawings

[20] FIG. 1 schematically illustrates an example of a video/image coding system to which this document is applicable.

[21] FIG. 2 is a diagram schematically explaining the configuration of video/image coding equipment to which this document is applicable.

[22] FIG. 3 is a diagram schematically explaining the configuration of video/image decoding equipment to which this document is applicable.

[23] FIG. 4 schematically illustrates a multiple conversion technology according to an embodiment of the present document.

[24] FIG. 5 roughly illustrates the intra-directional modes in 65 prediction directions.

[25] FIG. 6 and 7 are diagrams explaining RST according to an embodiment of the present document.

[26] FIG. 8 exemplifies context-adaptive binary arithmetic coding (CABAC) for encoding syntax elements.

[27] FIG. 9 and 10 schematically illustrate a video/image coding method and an example of related components according to embodiment(s) of the present document.

[28] FIG. 11 and 12 schematically illustrate a video/image decoding method and an example of related components according to the embodiment(s) of the present document.

[29] FIG. 13 illustrates an example of a content streaming system to which the embodiments disclosed herein are applicable.

Detailed description of embodiments

[30] The present disclosure may be modified in various forms, and specific embodiments thereof are described and illustrated in the drawings. However, these embodiments are not intended to limit the disclosure. The terms used in the description below are used to simply describe specific embodiments, but are not intended to limit the disclosure. The singular expression includes the plural expression as long as they are clearly interpreted differently. Terms such as "include" and "have" are intended to indicate that there are features, numbers, steps, operations, elements, components, or combinations of the foregoing used in the description below, and as such should be understood , which does not exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, elements, components, or combinations of the above.

[31] In addition, each configuration of the drawings described herein is an independent illustration for explaining functions as features that differ from each other, and does not mean that each configuration is implemented by mutually different hardware or different software. For example, two or more patterns may be combined to form one pattern, and one pattern may also be split into multiple patterns. Without departing from the spirit of this document, embodiments in which configurations are combined and/or separated are included within the scope of the claims.

[32] Hereinafter, examples of the present embodiment are described in detail with reference to the accompanying drawings. In addition, like reference numerals are used to refer to like elements throughout the drawings, and like descriptions of like elements are omitted.

[33] This document relates to video/image coding. For example, the methods/embodiments disclosed in this document may relate to the Universal Video Coding (VVC) (ITU-T (Rec. (H.266)) standard, the next generation video/picture coding standard after VVC, or other related video coding standards (eg, High Efficiency Video Coding (HEVC) standard (ITU-T (Rec. (H.265), Fundamental Video Coding (EVC) standard, AVS2 standard, etc.).

[34] This document proposes various video/image coding embodiments, and the above embodiments may also be performed in combination with each other unless otherwise noted.

[35] In this document, video can mean a sequence of images over time. A frame generally means a unit representing one image in a particular temporal movie frame, and a slice/tile means a unit that is part of a frame from an encoding point of view. A slice/tile may include one or more coding tree units (CTUs). One frame may consist of one or more slices/tiles. One frame may consist of one or more tile groups. One group of tiles may include one or more tiles.

[36] A pixel or pel can mean the smallest unit that makes up one frame (or image). In addition, "sample" can be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel/pixel value of a luma component, or only a pixel/pixel value of a chroma component.

[37] The unit may represent a basic image processing unit. The unit may include at least one of a specific area of the frame and information associated with the area. One unit may include one luminance block and two chrominance blocks (eg, Cb, Cr). The unit can be used interchangeably with terms such as block or zone in some cases. In general, an MxN block may include samples (or arrays of samples) or a set (or array) of transform coefficients of M columns and N rows. Alternatively, the sample may mean a pixel value in the spatial domain, and when such a pixel value is converted to the frequency domain, it can mean a transform factor in the frequency domain.

[38] In this document, the terms "/" and "," are to be interpreted as indicating "and/or". For example, the expression "A/B" can mean "A and/or B". Additionally, "A, B" may mean "A and/or B". Additionally, "A/B/C" may mean "at least one of A, B and/or C". In addition, "A/B/C" may mean "at least one of A, B and/or C".

[39] Additionally, in the document, the term "or" should be interpreted as indicating "and/or". For example, the expression "A or B" may contain 1) only A, 2) only B, and/or 3) both A and B. In other words, the term "or" in this document is to be interpreted as indicating "in addition or alternatively" .

[40] In the present description of the invention, "at least one of A and B" can mean "only A", "only B", or "both A and B". In addition, in the present description of the invention, the expression "at least one of A or B" or "at least one of A and/or B" can be interpreted identically to "at least one of A and B".

[41] Additionally, in the present description of the invention, "at least one of A, B and C" can mean "only A", "only B", "only C", or "any combination of A, B and C". In addition, "at least one of A, B or C" or "at least one of A, B and/or C" can mean "at least one of A, B and C".

[42] Additionally, the parentheses used in the present description of the invention may mean "for example". In particular, in the case where "prediction (internal prediction)" is expressed, it may be indicated that "internal prediction" is proposed as an example of "prediction". In other words, the term "prediction" in the present description of the invention is not limited to "intra prediction", and it may be indicated that "intra prediction" is offered as an example of "prediction". Additionally, even if "prediction (ie, intra prediction)" is expressed, it may be indicated that "intra prediction" is offered as an example of "prediction".

[43] In the present description of the invention, the technical features, separately explained in one drawing, may be implemented separately or may be implemented simultaneously.

[44] FIG. 1 illustrates an example of a video/image coding system to which the disclosure of this document may apply.

[45] Referring to FIG. 1, a video/image coding system may include a source device and a destination device. The source device may transmit the encoded information or video/image data to the receiving device via a digital storage medium or a network in the form of a file or streaming.

[46] The source device may include a video source, encoding equipment, and a transmitter. The receiver may include a receiver, decoding hardware, and a renderer. The encoding equipment may be referred to as "video/image coding equipment", and the decoding equipment may be referred to as "video/image decoding equipment". The transmitter may be included in the encoding equipment. The receiver may be included in the decoding equipment. The renderer may include a display, and the display may be configured as a separate device or an external component.

[47] A video source may acquire a video/image through a process of capturing, synthesizing, or generating a video/image. The video source may include a video/image capture device and/or a video/image generation device. The video/image capture device may include, for example, one or more cameras, video/image archives including previously captured videos/images, and the like. The video/image generation apparatus may include, for example, computers, tablet computers, and smartphones, and may (electronically) generate video/images. For example, the virtual video/image may be generated via a computer or the like. In this case, the video/image capturing process may be replaced by a linked data generation process.

[48] The encoding equipment may encode the input video/image. The encoding equipment may perform a series of procedures such as prediction, transformation, and quantization for compression and encoding efficiency. Encoded data (encoded video/picture information) may be output in the form of a bit stream.

[49] The transmitter may transmit information or coded image/image data output in the form of a bit stream to the receiver of the receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. The transmitter may include an element for generating a media file via a predetermined file format and may include an element for transmission via a broadcast/communication network. The receiver may receive/retrieve the bitstream and transmit the received bitstream to the decoding equipment.

[50] The decoding equipment can decode the video/image by performing a sequence of procedures such as dequantization, inverse transform, and prediction corresponding to the operation of the encoding equipment.

[51] The rendering module may prepare a decoded video/image by rendering. The rendered video/image can be displayed through the display.

[52] FIG. 2 is a diagram principally illustrating the configuration of video/image coding equipment to which the disclosure of this document may apply. Hereinafter in this document, what is referred to as "video encoding equipment" may include image encoding equipment.

[53] Referring to FIG. 2, the encoding equipment 200 may include and be configured with an image segmentation module 210, a prediction module 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The prediction module 220 may include a mutual prediction and module 222 internal prediction. Residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. Residual processor 230 may further include a subtractor 231. Adder 250 may be referred to as a "rebuilder" or "recovered block generator". The image segmentation module 210, prediction module 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 as described above may be configured by one or more hardware components (e.g., chipsets or encoder processors) according to an embodiment. In addition, the storage device 270 may include a decoded frame buffer (DPB) and may also be configured by a digital storage medium. The hardware component may further include a storage device 270 as an internal/external component.

[54] The image segmentation module 210 may split the input image (or frame, movie frame) input to the encoding equipment 200 into one or more processing units. As an example, a processing unit may be referred to as a "coding unit (CU)". In this case, a coding unit may be recursively split according to a quadtree, binary tree, and ternary tree (QTBTTT) structure from a coding tree unit (CTU) or a largest coding unit (LCU). For example, one coding unit may be split into multiple deeper coding units based on a quadtree structure, a binary tree structure, and/or a ternary tree structure. In this case, for example, a quadtree structure is applied first, and a binary tree structure and/or a ternary tree structure may subsequently be applied. Alternatively, a binary tree structure may also be applied first. An encoding procedure according to the present disclosure may be performed based on a final coding unit that is no longer split. In this case, based on the coding efficiency according to the characteristics of images and the like, the maximum coding unit may be directly used as the final coding unit, or, if necessary, the coding unit may be recursively split into coding units of greater depth, so that a coding unit having optimal size, can be used as the final coding unit. Here, the encoding procedure may include a procedure such as prediction, transformation, and recovery, which is described below. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, each of the prediction unit and the transformation unit may be split or segmented from the above final coding unit. The prediction unit may be a selective prediction unit, and the transform unit may be a unit for inferring a transform coefficient and/or a unit for inferring a residual signal from a transform coefficient.

[55] A unit can be used interchangeably with a term such as block or zone in some cases. Typically, an MxN block may represent samples consisting of M columns and N rows, or a group of transform coefficients. A sample can generally represent a pixel or a pixel value, and can also represent only a pixel/pixel value of the luminance signal component, as well as represent only a pixel/pixel value of the chrominance signal component. Sample can be used as a term corresponding to a pixel or pel configuring one frame (or image).

[56] The subtractor 231 may generate a residual signal (residual block, residual samples, or residual sample array) by subtracting the prediction signal (predicted block, predictive samples, or predictive sample array), output from prediction unit 220 from the input image signal (original block, original samples or an array of original samples) and the generated residual signal is provided to the converter 232. Prediction module 220 may perform prediction on a target processing block (hereinafter referred to as "current block") and generate a prediction block including prediction samples for the current block. Prediction module 220 may determine whether or not intra prediction or inter prediction is applied, for the current block or in units of CUs. As described below in the description of each prediction mode, the prediction module may generate various kinds of prediction-related information, such as prediction mode information, and pass the generated information to the entropy encoder 240. The prediction information may be encoded in the entropy encoder 240 and output in the form bit stream.

[57] Intra prediction module 222 may predict the current block with reference to samples within the current frame. The referenced samples may be located as adjacent to the current block, or may also be located at a large distance from the current block according to the prediction mode. Prediction modes in intra prediction may include a plurality of non-directional modes and a plurality of directional modes. The non-directional mode may include, for example, a DC mode or a planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the fine prediction direction degree. However, this is illustrative, and directional prediction modes that are greater than or less than the above number may be used according to the setting. The intra prediction unit 222 may also determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[58] The inter-prediction unit 221 may infer a predicted block of the current block based on a reference block (reference sample array) indicated by a motion vector for the reference frame. At this time, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the adjacent block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0 prediction, L1 prediction, bi-prediction, and the like). In the case of inter prediction, the neighbor block may include a spatial neighbor block existing in the current frame and a temporal neighbor block existing in the reference frame. The reference frame including the reference block and the reference frame including the temporal neighbor block may be identical to each other or different from each other. The temporal neighbor block may be referred to as a "collocated reference block", "collocated CU (colCU)" or the like, and a reference frame including the temporal neighbor block may also be referred to as a "collocated frame ( colpic)". For example, the inter prediction unit 221 may configure a motion information candidate list based on neighboring blocks, and generate information indicating which candidate is used to derive the motion vector and/or the reference frame index of the current block. Inter prediction may be performed based on various prediction modes, and, for example, in the case of the skip mode and the join mode, the inter prediction unit 221 may use the neighboring block motion information as the current block motion information. In the case of the skip mode, the residual signal may not be transmitted, in contrast to the combine mode. The motion vector prediction (MVP) mode may indicate the motion vector of the current block by using the motion vector of an adjacent block as a motion vector predictor and signaling the motion vector difference.

[59] The prediction module 220 may generate a prediction signal based on various prediction methods described below. For example, the prediction module may not only apply intra prediction or inter prediction to predict one block, but also apply both intra prediction and inter prediction at the same time. This may be referred to as "combined mutual and intra prediction (CIIP)". In addition, the prediction module may perform intra-block copy (IBC) to predict a block. Intra-block copying can be used for encoding content images/moving images of a game and the like, such as screen content coding (SCC). The IBC essentially performs prediction in the current frame, but it can be performed similar to inter prediction in which the reference block is retrieved in the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described herein.

[60] The predictive signal generated through the inter prediction module 221 and/or the intra prediction module 222 may be used to generate a recovered signal or generate a residual signal. Converter 232 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of discrete cosine transform (DCT), discrete sine transform (DST), graph-based transform (GBT), or conditionally non-linear transform (CNT). Here, GBT means a transformation obtained from a graph when relationship information between pixels is represented by a graph. CNT means the transformation obtained based on the predictive signal generated using all previously reconstructed pixels. In addition, the transformation process may be applied to square pixel blocks having the same size, or may be applied to blocks having a variable size rather than square.

[61] The quantizer 233 may quantize the transform coefficients and pass them to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (information regarding the quantized transform coefficients) and output a bitstream. The information regarding the quantized transform coefficients may be referred to as "residual information". The quantizer 233 may repackage the block quantized transform coefficients into one-dimensional vector form based on the scan order of the coefficients, and generate information about the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form. Entropy encoder 240 may implement various coding methods such as exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like, for example. Entropy encoder 240 may encode information needed to reconstruct the video/images together with or separately from the quantized transform coefficients (eg, syntax element values, etc.). Encoded information (eg, encoded video/image information) may be transmitted or stored in units of a network abstraction layer (NAL) in the form of a bitstream. The video/image information may further include information regarding various parameter sets such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. Herein, signaling/transmitted information and/or syntax elements, which are described below, may be encoded via the above-described encoding procedure and included in a bitstream. The bitstream may be transmitted over a network or may be stored on a digital storage medium. Here, the network may include a broadcast network and/or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. . A transmitter (not illustrated), a transmitting signal output from the entropy encoder 240, and/or a storage module (not illustrated) storing the signal may be configured as an internal/external element of the encoding equipment 200, and alternatively, the transmitter may be included in the entropy encoder. 240.

[62] The quantized transform coefficients output from quantizer 233 may be used to generate a predictive signal. For example, the residual signal (residual block or residual samples) may be recovered by applying dequantization and inverse transform to the quantized transform coefficients via dequantizer 234 and inverse transform 235. Summer 250 adds the recovered residual signal to the predictive signal output from predictor 220 to generate a reconstructed signal (recovered frame, recovered samples, or array of recovered samples). If there is no remainder for the target block to be processed, such as when the skip mode is applied, the predicted block may be used as the reconstructed block. The generated reconstructed signal may be used to intra-predict the next target block for processing in the current frame, and may be used to inter-predict the next frame through filtering as described below.

[63] Meanwhile, luminance chrominance scaling (LMCS) transform may be applied during the encoding and/or frame reconstruction process.

[64] The filter 260 can improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 260 may generate a modified reconstructed frame by applying various filtering methods to the reconstructed frame and store the modified reconstructed frame in memory 270, namely the DPB of memory 270. Various filtering methods may include, for example, filtering to deblock , sampled adaptive offset (SAO), adaptive loop filter, bilateral filter, etc. The filter 260 may generate various kinds of information related to the filtering and pass the generated information to the entropy encoder 290 as described below in the description of each filtering method. Filtering related information may be encoded by an entropy encoder 290 and output in the form of a bit stream.

[65] The modified reconstructed frame transmitted to the memory 270 may be used as a reference frame in the inter prediction module 221 . When inter prediction is applied by the encoding equipment, prediction mismatch between the encoding equipment 200 and the decoding equipment can be eliminated, and encoding efficiency can be improved.

[66] The DPB of the memory 270 may store the modified reconstructed frame for use as a reference frame in the inter prediction module 221 . The memory 270 may store block motion information from which motion information in the current frame is extracted (or encoded) and/or motion information of already reconstructed blocks in the frame. The stored motion information may be transmitted to the inter-prediction unit 221 for use as spatial neighbor block motion information or temporal neighbor block motion information. The memory 270 may store the recovered samples of the recovered blocks in the current frame and may pass the recovered samples to the intra prediction module 222 .

[67] FIG. 3 is a diagram for schematically explaining the configuration of video/image decoding equipment to which the disclosure of the present document may apply.

[68] Referring to FIG. 3, the decoding equipment 300 may include and be configured with an entropy decoder 310, a residual processor 320, a prediction unit 330, an adder 340, a filter 350, and a memory 360. The prediction unit 330 may include an inter prediction unit 331 and an intra prediction unit 332. forecasting. Residual processor 320 may include a dequantizer 321 and an inverse converter 322. Entropy decoder 310, residual processor 320, predictor 330, summer 340, and filter 350, as described above, may be configured by one or more hardware components (e.g., chipsets or decoder processors) according to the embodiment. Additionally, memory 360 may include a decoded frame buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include a storage device 360 as an internal/external component.

[69] When a bitstream including video/image information is input, the decoding equipment 300 may reconstruct an image in response to a process in which the video/image information is processed in the encoding equipment illustrated in FIG. 2. For example, decoding equipment 300 may extract units/blocks based on blocking-related information obtained from the bitstream. The decoding equipment 300 may perform decoding using the processing unit used for the encoding equipment. Therefore, the processing unit for decoding, for example, may be a coding unit, and the coding unit may be partitioned according to a quadtree structure, a binary tree structure, and/or a ternary tree structure from a coding tree unit or a maximum coding unit. One or more transformation units may be derived from a coding unit. In addition, the reconstructed image signal decoded and output by the decoding equipment 300 can be reproduced by the playback equipment.

[70] The decoding equipment 300 may receive a signal output from the encoding equipment of FIG. 2 in the form of a bitstream, and the received signal may be decoded via entropy decoder 310. For example, entropy decoder 310 may parse the bitstream to extract information (eg, video/image information) needed for image recovery (or frame recovery). The video/image information may further include information regarding various parameter sets such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The decoding equipment may further decode the frame based on the parameter set information and/or general constraint information. Signaling/receiving information and/or syntax elements described later in this document may be decoded, may be decoded by a decoding procedure, and obtained from a bit stream. For example, entropy decoder 310 decodes information in the bitstream based on an encoding method such as Exponential Golomb, CAVLC, or CABAC coding and output syntax elements required for image reconstruction and quantized transform coefficient values for the remainder. More specifically, the CABAC entropy decoding method can receive a bin corresponding to each syntax element in the bit stream, determine a context model by using the decoding target syntax element information, the decoding information of the decoding target block, or the symbol/bin information decoded in the previous step, and perform arithmetic decoding on the permission element by predicting the occurrence probability of the permission element according to the determined context model, and generating a character corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method can update the context model by using the decoded symbol/bin information for the context model of the next symbol/bin after the context model is determined. Prediction-related information from the information decoded by the entropy decoder 310 may be provided to the prediction unit 330, and information regarding the residue for which entropy decoding is performed in the entropy decoder 310, i. quantized transform coefficients and related parameter information may be input to the dequantizer 321. In addition, information regarding filtering from the information decoded by the entropy decoder 310 may be provided to the filter 350. Meanwhile, a receiver (not illustrated) for receiving a signal output from the equipment encoding equipment may be further configured as an internal/external member of the decoding equipment 300, or a receiver may be a constituent element of the entropy decoder 310. Meanwhile, the decoding equipment according to the present document may be called "video/image/frame decoding equipment", and the decoding equipment can be classified into an information decoder (video/picture/frame information decoder) and a sampling decoder (video/picture/frame samples decoder). The information decoder may include an entropy decoder 310, and the selective decoder may include at least one of a dequantizer 321, an inverse transform 322, a predictor 330, an adder 340, a filter 350, and a memory 360.

[71] The dequantizer 321 may dequantize the quantized transform coefficients so as to output the transform coefficients. The dequantizer 321 may recompose the quantized transform coefficients in a two-dimensional block form. In this case, the reassembly may be performed based on the scan order of the coefficients performed by the encoding equipment. A dequantizer 321 may perform dequantization on the quantized transform coefficients using a quantization parameter (eg, quantization step size information) and obtain transform coefficients.

[72] The inverse transform 322 inversely transforms the transform coefficients to obtain a residual signal (residual block, array of residual samples).

[73] Prediction module 330 may perform prediction on the current block and generate a predicted block including the prediction samples of the current block. The prediction module may determine whether intra prediction is applied or inter prediction is applied to the current block based on the prediction information output from the entropy decoder 310 and determine a specific intra/inter prediction mode.

[74] The prediction module may generate a prediction signal based on various prediction methods described below. For example, the prediction module may not only apply intra prediction or inter prediction in order to predict one block, but also apply intra prediction and inter prediction at the same time. This may be referred to as "combined mutual and intra prediction (CIIP)". In addition, the prediction module may perform intra-block copy (IBC) to predict the block. Intra-block copying can be used for encoding content images/moving images of a game and the like, such as screen content coding (SCC). The IBC essentially performs prediction in the current frame, but it can be performed similar to inter prediction in which the reference block is retrieved in the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described herein.

[75] Intra prediction module 332 may predict the current block by referring to the samples in the current frame. The reference samples may be located in the surroundings of the current block, or may be spaced apart from the current block according to the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit 332 may determine the prediction mode to be applied to the current block by using the prediction mode applied to the adjacent block.

[76] The inter prediction unit 331 may derive a predicted block for the current block based on a reference block (reference sample array) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the adjacent block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include information regarding the direction of inter-prediction (L0 prediction, L1 prediction, bi-prediction, and the like). In the case of inter prediction, the neighbor block may include a spatial neighbor block existing in the current frame and a temporal neighbor block existing in the reference frame. For example, inter-prediction unit 331 may construct a list of motion information candidates based on adjacent blocks, and extract the current block's motion vector and/or reference frame index based on the received candidate selection information. Inter prediction may be performed based on various prediction modes, and the prediction information may include information indicative of the inter prediction mode for the current block.

[77] The adder 340 may generate a reconstructed signal (recovered frame, reconstructed block, or array of reconstructed samples) by summing the obtained residual signal with the predictive signal (predicted block or array of predicted samples) output from prediction module 330. If there is no remainder for the target processing block, such as when the skip mode is applied, the predicted block may be used as the reconstructed block.

[78] The adder 340 may be referred to as a "recovery module" or "recovered block generator". The generated recovered signal may be used for intra-prediction of the next block to be processed in the current frame, and as described below, may also be filtered out, or may also be used for inter-prediction of the next frame.

[79] Meanwhile, luma signal scaling (LMCS) transformation can also be applied in the frame decoding process.

[80] The filter 350 can improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 350 may generate a modified reconstructed frame by applying various filtering methods to the reconstructed frame and store the modified reconstructed frame in memory 360, namely the DPB of memory 360. Various filtering methods may include, for example, filtering to deblock , sampled adaptive bias, adaptive loop filter, bilateral filter, etc.

[81] The (modified) reconstructed frame stored in the DPB of the memory 360 may be used as a reference frame in the inter prediction unit 331 . The memory 360 may store block motion information from which motion information in the current frame is extracted (or decoded) and/or motion information of already reconstructed blocks in the frame. The stored motion information may be transmitted to the inter-prediction unit 331 so that it is used as spatial neighbor block motion information or temporal neighbor block motion information. The memory 360 may store the reconstructed samples of the reconstructed blocks in the current frame and pass the reconstructed samples to the intra prediction module 332 .

[82] In the present description of the invention, the embodiments described in the predictor 330, the dequantizer 321, the inverse transform 322, and the filter 350 of the decoding equipment 300 can also be applied in the same way or respectively to the predictor 220, the dequantizer 234, the inverse transform 235, and the filter 260 equipment 200 encoding.

[83] Meanwhile, as described above, when video encoding is performed, prediction is performed in order to improve compression efficiency. Through this, a predicted block including prediction samples for the current block can be generated as a block to be encoded (ie, an encoding target block). Here, the prediction block includes prediction samples in the spatial domain (or pixel domain). The prediction block is extracted identically in the encoding equipment and the decoding equipment, and the encoding equipment can signal information (residual information) about the residual between the original block and the predicted block, rather than the value of the original sample of the original block, to the decoding equipment, thereby improving efficiency. image encoding. The decoding equipment can extract a residual block including residual samples based on the residual information, sum the residual block and the predicted block to generate reconstructed blocks including the reconstructed samples, and generate a reconstructed frame including the reconstructed blocks.

[84] The residual information may be generated through a transformation and quantization procedure. For example, the encoding equipment may extract the residual block between the original block and the predicted block, perform a transform procedure on the residual samples (array of residual samples) included in the residual block to extract transform coefficients, perform a quantization procedure on the transform coefficients to extract the quantized transform coefficients, and signal-related residual information to the decoding equipment (via the bitstream). Here, the residual information may include quantized transform coefficient value information, location information, transform technology, transform kernel, quantization parameter, and the like. The decoding equipment may perform a dequantization/inverse transform procedure based on the residual information and extract residual samples (or residual blocks). The decoding equipment may generate a reconstructed frame based on the predicted block and the residual block. In addition, for reference for image inter-prediction later, the encoding equipment may also dequantize/inversely transform the quantized transform coefficients to extract the residual block and generate a reconstructed frame based on it.

[85] FIG. 4 schematically illustrates the multiple transform technology of the present document.

[86] Referring to FIG. 4, the converter may correspond to the converter in the encoding equipment of FIG. 2 as described above, and the inverse converter may correspond to the inverse converter in the encoding equipment of FIG. 2 or the inverse converter in the decoding equipment of FIG. 3 as described above.

[87] The converter can extract (primary) transform coefficients by performing primary transform based on the residual sample (residual sample array) in the residual block (S410). Such a primary transformation may be referred to as a "basic transformation". Here, the primary transform may be based on multiple transform selection (MTS), and in case the multiple transform is applied as the primary transform, it may be referred to as "multiple basic transform".

[88] For example, the multiple base transform may represent a transform method by further using discrete cosine transform (DCT) type 2 (DCT-II), discrete sine transform (DST) type 7 (DST-VII), DCT type 8 (DCT- VIII) and/or DST-type 1 (DST-I). Thus, the multiple base transform may represent a transform method for converting a spatial domain residual signal (or residual block) into frequency domain transform coefficients (or primary transform coefficients) based on a plurality of transform kernels selected from DCT type 2, DST type 7, DCT type 8 and DST type 1. Here, the primary transform coefficients may be referred to as "temporal transducer-side transform coefficients".

[89] In other words, if the existing transform method is applied, the spatial domain transform for the residual signal (or residual block) into the frequency domain can be applied based on DCT type 2, and transform coefficients can be generated. However, in contrast, in case a multiple base transform is applied, the spatial domain transform for the residual signal (or residual block) to the frequency domain may be applied based on DCT type 2, DST type 7, DCT type 8, and /or DST type 1, and transform coefficients (or primary transform coefficients) may be generated. Here, DCT type 2, DST type 7, DCT type 8, and DST type 1 may be referred to as "transformation type", "transformation core", or "transformation base". Types of DCT/DST transformations can be specified based on basis functions.

[90] In case multiple base transform is performed, the vertical transform kernel and/or the horizontal transform kernel for the target block may be selected from among the transform kernels, the vertical transform for the target block may be performed based on the vertical transform kernel, and the horizontal transform for the target block can be performed on the basis of the horizontal transformation kernel. Here, the horizontal transform may represent a transform for the horizontal components of the target box, and the vertical transform may represent the transform for the vertical components of the target box. The vertical transform kernel/horizontal transform kernel may be adaptively determined based on the prediction mode and/or transform index of the target block (CU or subblock) including the residual block.

[91] Further, for example, in the case of performing primary transformation by applying the MTS, specific basis functions can be configured as the specified values, and in the case of vertical transformation or horizontal transformation, the transformation relationship for the transformation kernel can be configured by combining which basis functions apply. For example, in the case where the horizontal direction transformation kernel is represented by trTypeHor and the vertical direction transformation kernel is represented by trTypeVer, trTypeHor or trTypeVer having a value of 0 can be configured as DCT2, and trTypeHor or trTypeVer having a value of 1 can be configured as DCT7 ; trTypeHor or trTypeVer having a value of 2 may be configured as DCT8.

[92] Alternatively, for example, to indicate any of the plurality of transform core sets, the MTS index may be encoded, and the MTS index information may be signaled to the decoding equipment. Here, the MTS index may be represented as a tu_mts_idx syntax element or an mts_idx syntax element. For example, if the MTS index is 0, the values of trTypeHor and trTypeVer may appear to be 0, and (trTypeHor, trTypeVer)=(DCT2, DCT2). If the MTS index is 1, the values of trTypeHor and trTypeVer may appear to be 1, and (trTypeHor, trTypeVer)=(DST7, DST7). If the MTS index is 2, the trTypeHor value is 2 and the trTypeVer value is 1, and (trTypeHor, trTypeVer)=(DCT8, DST7) may be represented. If the MTS index is 3, the value of trTypeHor is 1 and the value of trTypeVer is 2, and (trTypeHor, trTypeVer)=(DST7, DCT8). If the MTS index is 4, the values of trTypeHor and trTypeVer may appear to be 2, and (trTypeHor, trTypeVer)=(DCT8, DCT8). For example, the set of transformation kernels according to the MTS index may be represented as indicated in the following table.

[93] Tab. 1

[94] The converter can extract modified (secondary) transform coefficients by performing secondary transform based on the (primary) transform coefficients (S420). The primary transform may be a spatial-to-frequency domain transform, and the secondary transform may be a transform to a more compressive expression by using the correlation existing between the (primary) transform coefficients.

[95] For example, the secondary transform may include an inseparable transform. In this case, the secondary transform may be referred to as "non-separable secondary transform (NSST)" or "mode-dependent non-separable secondary transform (MDNSST)". The non-separable second transform may represent a transform for generating modified transform coefficients (or secondary transform coefficients) for the residual signal by re-transforming the (primary) transform coefficients derived through the primary transform based on the non-separable transform matrix. Here, the vertical transform and the horizontal transform may not be applied separately (or independently) with respect to the (primary) transform coefficients based on the nonseparable transform matrix, but may be applied simultaneously.

[96] In other words, an inseparable secondary transform may be a transform method for recomposing, for example, two-dimensional signals (transform coefficients) into a one-dimensional signal via a specifically defined direction (for example, a row first direction or a column first direction), without separating the coefficients (primary ) transforming into vertical components and horizontal components, and then generating modified transform coefficients (or secondary transform coefficients) based on an inseparable transform matrix.

[97] For example, a direction (or order) row first may represent the layout of an MxN box in a line in the order "first row - Nth row", and a direction (or order) column first may represent the layout of an MxN box in a line in the order "first column - Mth column". Here, M and N may represent the width (W) and height (H) of the block, and may be positive integers.

[98] For example, an inseparable secondary transform may be applied to the top left region of a block consisting of (primary) transform coefficients (hereinafter, the transform coefficient block). For example, if the width (W) and height (H) of the transform coefficient block are equal to or greater than 8, an 8×8 inseparable secondary transform may be applied to the upper left 8×8 region of the transform coefficient block. Additionally, if the width (W) and height (H) of the transform coefficient block are equal to or greater than 4 and less than 8, a 4x4 inseparable secondary transform may be applied to the upper left zone min(8, W) x min(8, H) of the coefficient block transformations. However, the embodiments are not limited to this, and for example, even if such a condition that the width (W) and height (H) of the transform coefficient block is equal to or greater than 4 is satisfied, a 4×4 inseparable secondary transform can be applied to the left upper zone min(8, W) x min(8, H) of the transform coefficient block.

[99] Specifically, for example, in the case where a 4×4 input block is used, the non-separable secondary transform may be performed as follows.

[100] The input block X 4×4 can be represented as follows.

[101] Equation 1

[102] For example, a vector shape X can be represented as follows.

[103] Equation 2

может представлять вектор X, и двумерный блок X в уравнении 1 может перекомпоновываться и представляться в качестве одномерного вектора в соответствии с порядком сначала по строкам.[104] Referring to Equation 2,

may represent a vector X, and the two-dimensional block X in Equation 1 may be rearranged and represented as a one-dimensional vector according to row first order.

[105] In this case, the secondary inseparable transformation can be calculated as follows.

[106] Equation 3

может представлять вектор коэффициентов преобразования, и T может представлять матрицу (неразделимого) преобразования 16×16.[107] Here,

may represent a vector of transform coefficients, and T may represent a 16×16 (inseparable) transform matrix.

, имеющий размер 16×1, может извлекаться, и

может реорганизовываться в качестве блока 4×4 через порядок сканирования (горизонтальный, вертикальный или диагональный). Тем не менее, вышеописанное вычисление является примерным, и чтобы уменьшать вычислительную сложность неразделимого вторичного преобразования, гиперкубическое преобразование Гивенса (HyGT) и т.п. может использоваться для того, чтобы вычислять неразделимое вторичное преобразование.[108] Based on Equation 3,

, having a size of 16×1, can be extracted, and

can be reorganized as a 4×4 block through the scan order (horizontal, vertical or diagonal). However, the above calculation is exemplary, and in order to reduce the computational complexity of the inseparable secondary transform, the Givens Hypercubic Transform (HyGT) and the like. can be used for computing an inseparable secondary transformation.

[109] Meanwhile, in an inseparable secondary transform, a transform kernel (either a transform base or a transform type) may be selected in a mode-dependent manner. Here, the mode may include an intra prediction mode and/or an inter prediction mode.

[110] For example, as described above, the NSST may be performed based on an 8×8 transform or a 4×4 transform determined based on the width (W) and height (H) of the transform coefficient block. For example, if W and H are equal to or greater than 8, the 8×8 transform may represent a transform that can be applied to the 8×8 zone included in the corresponding transform coefficient block, and the 8×8 zone may represent the top left 8×8 zone in corresponding block of transformation coefficients. Further, similarly, if W and H are equal to or greater than 4, the 4×4 transform may represent a transform that can be applied to the 4×4 zone included in the corresponding transform coefficient block, and the 4×4 zone may represent the upper left 4× zone. 4 in the corresponding transform coefficient block. For example, an 8x8 transform kernel matrix may be a 64x64/16x64 matrix, and a 4x4 transform kernel matrix may be a 16x16/8x16 matrix.

[111] In this case, for mode-based transform core selection, two non-separable sub-transform cores per transform set for non-separable secondary transform can be configured with respect to all of 8×8 transform and 4×4 transform, and four can be provided. conversion set. Thus, four transform sets may be configured with respect to the 8×8 transform, and four transform sets may be configured with respect to the 4×4 transform. In this case, each of the four transform sets for 8×8 transform may include two 8×8 transform kernels, and each of the four transform sets for 4×4 transform may include two 4×4 transform kernels.

[112] However, the size of the transform sub-block, the number of sets, and the number of transform cores in the set are exemplary, and a size other than 8×8 or 4×4 may be used, or n sets may be configured, and k transform cores may be included in each set. Here, n and k can be positive integers.

[113] For example, a transform set may be referred to as an "NSST set", and a transform core in an NSST set may be referred to as an "NSSAT core". For example, selection of a particular set of sets for transformation may be performed based on the intra prediction mode of the target block (CU or sub-block).

[114] For example, the intra prediction mode may include two non-directional or non-angular intra prediction modes and 65 directional or angular intra prediction modes. The non-directional intra prediction modes may include a planar intra prediction mode number 0 and a DC intra prediction mode number 1, and the directional intra prediction modes may include 65 intra prediction modes (numbers 2-66). However, this is exemplary, and the embodiment according to the present document may be applied even to the case in which a different number of intra prediction modes are provided. Meanwhile, in some cases, the intra prediction mode number 67 may additionally be used, and the intra prediction mode number 67 may represent a linear model (LM) mode.

[115] FIG. 5 roughly illustrates the intra-directional modes in 65 prediction directions.

[116] Referring to FIG. 5, the modes can be divided into intra prediction modes having a horizontal directionality and intra prediction modes having a vertical directionality with respect to the intra prediction mode number 34 having a left upper diagonal prediction direction. In FIG. 5, H and V may denote horizontal orientation and vertical orientation, respectively, and the numbers -32 to 32 may represent offsets in units of 1/32 for a position on the sample grid. It may represent an offset for the mode index value.

[117] For example, intra prediction modes 2-33 may be horizontally directed, and intra prediction modes 34-66 may be vertically oriented. Meanwhile, from a technical point of view, the intra prediction mode number 34 can be considered to have neither horizontal directionality nor vertical directionality, but can be classified as belonging to horizontal directionality from the point of view of defining a transform set for secondary transformation. This is because the input data is transposed and used with respect to the vertical directional modes symmetrical with respect to the intra prediction mode number 34, and the layout method of the input data for the horizontal directional mode is used with respect to the intra prediction mode number 34. Here, the transposition of the input data may mean the NxM data configuration in such a way that rows become columns and columns become rows with respect to MxN two-dimensional block data.

[118] Additionally, the intra prediction mode number 18 and the intra prediction mode number 50 may represent the horizontal intra prediction mode and the vertical intra prediction mode, respectively, and the intra prediction mode number 2 may be referred to as the "right upper diagonal intra prediction mode" because the prediction is performed in the direction up and to the right with the left reference pixel. In the same context, intra prediction mode number 34 may be referred to as "right lower diagonal intra prediction mode", and intra prediction mode number 66 may be referred to as "left lower diagonal intra prediction mode".

[119] Meanwhile, if it is determined that a specific set is used for a non-separable transform, one of the k transform kernels in the specific set may be selected through the non-separable secondary transform index. For example, the encoding equipment may derive a non-separable second transform index representing a particular transform kernel based on a rate-distortion (RD) test, and may signal the non-separable second transform index to the decoding equipment. For example, the decoding hardware may select one of the k transform cores in a particular set based on the non-separable secondary transform index. For example, an NSST index having a value of 0 may represent a first non-separable subtransform kernel, an NSST index of 1 may represent a second nonseparable secondary transform kernel, and an NSST index of 2 may represent a third nonseparable secondary transform kernel. . Alternatively, an NSST index having a value of 0 may represent that the first non-separable secondary transform is not applied to the target block, and an NSST index having a value of 1-3 may indicate three transformation kernels as described above.

[120] The transformer may perform an inseparable secondary transform based on the selected transform kernels and may obtain modified (secondary) transform coefficients. The modified transform coefficients may be extracted as quantized transform coefficients via the above-described quantizer, and may be signal-coded to the decoding equipment, and may be transmitted to a dequantizer/inverse converter in the encoding equipment.

[121] Meanwhile, if the secondary transform is omitted as described above, the (primary) transform coefficients, which are outputs of the primary (separable) transform, can be extracted as quantized transform coefficients via a quantizer as described above, and can be encoded with the possibility be signaled to the decoding equipment and may be sent to a dequantizer/inverter in the encoding equipment.

[122] Referring again to FIG. 4, the reverse converter may perform the sequence of procedures in reverse order of the procedures performed by the above-described converter. The inverse transform can receive (dequantized) transform coefficients, extract (primary) transform coefficients by performing a secondary (inverse) transform (S450), and obtain a residual block (residual samples) by performing a primary (inverse) transform with respect to the (primary) transform coefficients (S460) . Here, the primary transform coefficients may be referred to as "modified transform coefficients on the inverter side". As described above, the encoding equipment and/or the decoding equipment may generate a reconstructed block based on the residual block and the predicted block, and may generate a reconstructed frame based on them.

[123] Meanwhile, the decoding equipment may further include a secondary inverse transform application/non-application determination unit (or an element for determining whether or not to apply the secondary inverse transform), and a secondary inverse transformation determination unit (or an element for determining the secondary inverse transform). transformations). For example, the secondary inverse transform application/non-application determination module may determine whether or not the secondary inverse transformation should be applied. For example, the secondary inverse transform may be NSST or RST, and the secondary inverse transform apply/non-apply determination module may determine whether or not the secondary inverse transform should be applied based on the secondary transform flag parsed or obtained from the bit stream. Alternatively, for example, the second inverse transform application/non-application determination module may determine whether or not the secondary inverse transform should be applied based on the transform coefficient of the residual block.

[124] The secondary inverse transform determination module may determine the secondary inverse transform. In this case, the secondary inverse transform determination module may determine the secondary inverse transform applied to the current block based on the set for the NSST (or RST) transform specified according to the intra prediction mode. Alternatively, the method for determining the secondary transform may be determined depending on the method for determining the primary transform. Alternatively, different combinations of the primary transform and the secondary transform may be determined according to the intra prediction mode. For example, the secondary inverse transform determination module may determine the area to which the secondary inverse transform is applied based on the size of the current block.

[125] Meanwhile, if the secondary (inverse) transform is omitted as described above, the residual block (residual samples) can be obtained by receiving the (dequantized) transform coefficients and performing the primary (separable) inverse transform. As described above, the encoding equipment and/or the decoding equipment may generate a reconstructed block based on the residual block and the predicted block, and may generate a reconstructed frame based on them.

[126] Meanwhile, herein, in order to reduce the computational amount and the required storage space caused by the inseparable secondary transform, a reduced secondary transform (RST) having a reduced transform matrix (kernel) size can be applied to the NSST concept.

[127] As used herein, RST may mean a (simplified) transformation performed on the residual samples for a target block based on a transformation matrix whose size is reduced in accordance with the simplification factor. If this is done, the amount of computation required during the transformation may be reduced due to the reduction in the size of the transformation matrix. Thus, RST can be used to solve the problem of computational complexity arising during the transformation of a block having a large size or non-separable transformation.

[128] For example, RST may be referred to by various terms such as "reduced transformation", "reduced secondary transformation", "reduced transformation", "simplified transformation", or "simple transformation", and the names by which the RST is called are not limited to the examples listed. Further, RST is mainly performed in the low frequency region including coefficients that are not 0 in the transform block, and thus may be referred to as "low frequency inseparable transform (LFNST)".

[129] Meanwhile, in the case where the secondary inverse transform is performed based on the RST, the inverse transformer 235 of the encoding equipment 200 and the inverse transformer 322 of the decoding equipment 300 may include an inverse RST module extracting modified transform coefficients based on the inverse RST of the transform coefficients. , and an inverse primer extracting residual samples for the target block based on the inverse primer for the modified transform coefficients. Reverse primary means the reverse of the primary transform applied to the remainder. As used herein, extracting transform coefficients based on a transform may mean extracting transform coefficients by applying an appropriate transform.

[130] FIG. 6 and 7 are diagrams explaining RST according to an embodiment of the present document.

[131] For example, FIG. 6 may be a drawing explaining that a direct shorthand transformation is applied, and FIG. 7 may be a drawing explaining that an inverse shorthand transformation is applied. Here, a target block may represent a current block, a residual block, or a transform block, depending on what is being encoded.

[132] For example, in RST, an N-dimensional vector may be transformed into an R-dimensional vector located in another space, and a shorthand transformation matrix may be determined. Here, N and R may be positive integers, and R may be less than N. N may mean the square of the length of one side of the block to which the transform is applied, or the total number of transform coefficients corresponding to the block to which the transform is applied, and the simplification factor may mean R/N value. The simplification factor may be referred to by various terms such as "reduced factor", "reduction factor", "simplified factor", or "simple factor". Meanwhile, R may be referred to as a "reduced factor", and in some cases, the simplification factor may mean R. Further, in some cases, the simplification factor may mean an N/R value.

[133] For example, the simplification factor or reduced factor may be signaled via the bitstream, but is not limited to this. For example, preset values for the simplification factor or the reduced factor may be stored in the encoding equipment 200 and the decoding equipment 300, in which case, the simplification factor or the reduced factor may not be signaled separately.

[134] For example, the size (RxN) of the simplified transformation matrix may be smaller than the size (NxN) of the regular transformation matrix, and may be given in the following equation.

[135] Equation 4

[136] For example, the matrix T in the reduced transform block illustrated in FIG. 6 may represent the TRxN matrix of Equation 4. As shown in FIG. 6, in case the residual samples for the target block are multiplied by the simplified transform matrix TRxN, the transform coefficients for the target block can be extracted.

[137] For example, in the case where the block size to which the transform is applied is 8×8 and R is 16 (i.e., R/N=16/64=1/4), the RST of FIG. 6 may be expressed by a matrix operation as indicated in Equation 5 below. In this case, the memory and the multiplication operation may be reduced to approximately 1/4 by the simplification factor.

[138] As used herein, a matrix operation can be understood as an operation to obtain a column vector by placing a matrix to the left of the column vector and multiplying the matrix and the column vector.

[139] Equation 5

[140] In Equation 5, r1-r64 may represent the residual samples for the target block. Alternatively, for example, they may be transform coefficients generated by applying a primary transform. Based on the operation result of Equation 5, the conversion coefficients ci for the target block can be retrieved.

[141] For example, if R is 16, the transform coefficients c1-c16 for the target block may be extracted. If a transformation matrix having a size of 64×64 (NxN) is multiplied by the residual samples having a size of 64×1 (Nx1) through applying a regular transform rather than RST, 64 (N) transformation coefficients for the target block can be extracted, but since RST is applied, only 16 (N) transform coefficients for the target block can be extracted. Since the total number of transform coefficients for a target block is reduced from N to R, the amount of data that the encoding equipment 200 transmits to the decoding equipment 300 can be reduced, and therefore the transmission efficiency between the encoding equipment 200 and the decoding equipment 300 can be improved.

[142] Considering the size of the transformation matrix, since the size of the regular transformation matrix is 64×64 (NxN) and the size of the simplified transformation matrix is reduced to 16×64 (RxN), the memory usage when performing RST can be reduced in terms of R/N by compared to the case in which a regular transformation is performed. Additionally, compared to the number (NxN) of multiplications when using a regular transformation matrix, the use of a simplified transformation matrix can reduce the number of multiplications (RxN) with respect to R/N.

[143] In an embodiment, the transformer 232 of the encoding equipment 200 may derive transform coefficients for a target block by performing a primary transform and a secondary transform based on RST on the residual samples for the target block. The transform coefficients may be passed to an inverse transform of the decoding equipment 300, and the inverse transformer 322 of the decoding equipment 300 may derive modified transform coefficients based on an inverse reduced secondary transform (RST) for the transform coefficients, and may derive residual samples for a target block based on an inverse primary transform of the modified coefficients. transformations.

[144] The size of the TNxR inverse RST matrix according to an embodiment may be NxR, which is smaller than the NxN size of the regular inverse transform matrix, and may have a transposed relationship with the simplified transform matrix TRxN illustrated in Equation 4.

[145] The Tt matrix in the reduced inverse transform block illustrated in FIG. 7 may represent an inverse RST matrix of TRxNT. Here, the superscript T may represent a transposition. As shown in FIG. 7, in case the transform coefficients for the target block are multiplied by the inverse RST matrix TRxNT, modified transform coefficients for the target block or residual samples for the target block can be extracted. The inverse RST matrix TRxNT can be expressed as (TRxN)TNxR.

[146] More specifically, in the case where an inverse RST is applied as a secondary inverse transform, the modified transform coefficients for the target block may be derived by multiplying the transform coefficients for the target block by the inverse RST matrix TRxNT. Meanwhile, the inverse RST may be applied as an inverse primary transform, in which case, the residual samples for the target block may be extracted when the inverse RST matrix TRxNT is multiplied by the transform coefficients for the target block.

[147] In an embodiment, in the case where the block size to which the inverse transform is applied is 8x8 and R is 16 (i.e., R/N=16/64=1/4), the RST of FIG. 7 can be expressed by a matrix operation as indicated in Equation 6 below.

[148] Equation 6

[149] In Equation 6, c1-c16 may represent transform coefficients for the target block; rj representing the modified transform coefficients for the target block or residual samples for the target block may be derived based on the result of the operation of Equation 6. Thus, r1-rN representing the modified transform coefficients for the target block or residual samples for the target block may be derived.

[150] Considering the size of the inverse transform matrix, since the size of the regular inverse transform matrix is 64×64 (NxN) and the size of the simplified inverse transform matrix is reduced to 64×16 (NxR), the memory usage when performing the inverse RST can be reduced in relation to R/N compared to the case in which a regular inverse transformation is performed. Additionally, compared to the number (NxN) of multiplications when using a regular inverse transform matrix, using a simplified inverse transform matrix can reduce the number (NxR) of multiplications in terms of R/N.

[151] Meanwhile, conversion sets can be configured and applied even with respect to RST 8×8. Thus, the corresponding RST 8x8 can be applied according to the conversion set. Since one transform set is composed of two or three transform kernels according to the intra prediction mode, it can be configured to select one of four transforms, which at most includes even the case in which the secondary transform is not applied. In a transformation where no secondary transformation is applied, the identity matrix can be considered to be applied. If it is assumed that index 0, 1, 2, or 3 is given for four transformations (for example, index number 0 may be allocated to the case in which the identity matrix, i.e., the secondary transformation, is not applied), the transformation to be applied may be denoted by signaling a syntax element that is an NSST index to each block of transform coefficients. Thus, through the NSST index, NSST 8x8 can be denoted for the top left 8x8 block, and in the RST configuration, RST 8x8 can be denoted. NSST 8×8 and RST 8×8 may represent transforms capable of being applied to the 8×8 zone included in the corresponding transform coefficient block in case the W and H of the target block that becomes the transform target is equal to or greater than 8, and the zone is 8 x8 may represent the top left 8x8 zone in the corresponding transform coefficient block. Similarly, NSST 4x4 and RST 4x4 may represent transforms capable of being applied to a 4x4 zone included in the corresponding transform coefficient block in case W and H of the target block are equal to or greater than 4, and the 4x4 zone may represent is the top left 4×4 zone in the corresponding transform coefficient block.

[152] Meanwhile, for example, the encoding equipment can extract a bit stream by encoding a syntax element value or quantized transform coefficient values for a residue based on various encoding methods such as exponential Golomb code, context adaptive variable length coding (CAVLC), context-sensitive adaptive binary arithmetic coding (CABAC), and the like. Additionally, the decoding equipment can extract the value of the syntax element or the quantized transform coefficient values for the residue based on various encoding methods such as Exponential Golomb encoding, CAVLC, CABAC, and the like.

[153] For example, the above-described encoding methods can be performed as content, which is described below.

[154] FIG. 8 exemplifies context-adaptive binary arithmetic coding (CABAC) for encoding a syntax element.

[155] For example, in the CABAC encoding process, if the input signal is a syntax element that is not a binary value, the value of the input signal can be converted to a binary value through binarization. Additionally, if the input signal is already a binary value (ie, if the value of the input signal is a binary value), binarization may not be performed and the input signal may be used as is. Here, each binary number 0 or 1 constituting a binary value may be referred to as a "permission element". For example, if the binary string after binarization is 110, each of 1, 1, and 0 can be represented as one permission element. The permission element(s) for one syntax element may represent the value of the syntax element. The binarization may be based on a different binarization method such as a truncated Rice code binarization process or a fixed length code binarization process, and the binarization method for the target syntax element may be preset. The binarization procedure may be performed by a binarization module in the entropy encoder.

[156] Thereafter, the binarized syntax element sample elements may be input to a regular encoding engine or a bypass encoding engine. The regular coding engine of the encoding equipment may extract a context model that reflects a probability value with respect to a corresponding bin, and encode the corresponding bin based on the extracted context model. The encoding hardware's regular encoding engine may update the context model for the corresponding permission element after encoding is performed on the corresponding permission elements. Permission elements encoded as the content described above may be represented as context-encoded permission elements.

[157] Meanwhile, in case the binarized syntax element sample elements are input to the bypass encoding mechanism, they may be encoded as follows. For example, the bypass encoding mechanism of the encoding equipment may omit a procedure for estimating a probability with respect to an input bin and a procedure for updating a probability model applied to a bin after encoding. In the case where bypass coding is applied, the coding equipment can encode the input bin by applying a regular probability distribution instead of extracting a context model, and thereby the coding speed can be improved. The permission element encoded as the content described above may be presented as a bypass permission element.

[158] Entropy decoding may represent a process for performing a process identical to the above-described entropy encoding in reverse order.

[159] The decoding equipment (entropy decoder) can decode the encoded image/video information. The image/video information may include segmentation related information, prediction related information (e.g., inter/intra prediction separation information, intra prediction mode information, inter prediction mode information, and the like), residual information, or in-loop filtering related information. information or may include various syntactic elements from it. Entropy encoding may be performed in units of a syntax element.

[160] The decoding hardware may binarize the target syntax elements. Here, the binarization may be based on various binarization methods such as a truncated Rice code binarization process or a fixed length code binarization process, and the binarization method for the target syntax element may be previously set. The decoding hardware can extract the available bin strings (possible bin strings) for the available target syntax element values via a binarization procedure. The binarization procedure may be performed by a binarization module in the entropy decoder.

[161] The decoding hardware may compare the extracted bin string with the available bin strings for the corresponding syntax elements when sequentially decoding or parsing the corresponding bins for target syntax elements from the input bit(s) in the bitstream. If the extracted permission element string is equal to one of the available permission element strings, the value corresponding to the corresponding permission element string is retrieved as the value of the corresponding syntax element. If not, the decoding hardware may re-execute the above procedure after further parsing the next bit in the bitstream. Through such a process, it is possible to signal specific information (or a specific syntax element) in a bit stream using a variable length bit even without using a start bit or end bit of the corresponding information. Through this, a relatively smaller bit can be allocated with respect to a smaller value, and thereby the overall coding efficiency can be improved.

[162] The decoding equipment may perform context model-based decoding or bypass decoding of the corresponding bins in a bin string from a bitstream based on an entropy coding technology such as CABAC or CAVLC.

[163] In case the syntax element is decoded based on the context model, the decoding equipment may receive a permission element corresponding to the syntax element via the bit stream, may determine the context model by using the syntax element, and decoding the decoding target block or neighboring block information or character information / bins decoded in the previous step, and can extract the value of the syntax element by performing arithmetic decoding of the bin by predicting the occurrence probability of the received bin according to the determined context model. Thereafter, the context model of the next decodable permission element may be updated based on the determined context model.

[164] The context model may be allocated and updated by context-encoded (regularly encoded) permission elements, and the context model may be specified based on a context index (ctxIdx) or a context index increment (ctxInc). CtxIdx can be derived based on ctxInc. In particular, for example, a ctxIdx representing the context model for each of the regularly encoded permission elements can be extracted by the sum of ctxInc and the context index offset (ctxIdxOffset). For example, ctxInc may be retrieved differently via permission elements. ctxIdxOffset can be represented as the smallest value of ctxIdx. Typically, ctxIdxOffset may be a value used to distinguish it from context models for other syntax elements, and the context model for one syntax element may be separated or derived based on ctxInc.

[165] In the entropy coding procedure, it may be determined whether to perform coding through a regular coding mechanism or to perform coding through a bypass coding mechanism, and the coding path may switch accordingly. Entropy decoding can perform the same process as entropy encoding in reverse order.

[166] Meanwhile, for example, in the case where the bypass decoding of the syntax element is performed, the decoding equipment may receive a bin corresponding to the syntax element via the bitstream, and may decode the input bin by applying a regular probability distribution. In this case, the decoding hardware may omit the syntax element context model extraction procedure and the context model update procedure applied to the permission element after decoding.

[167] Meanwhile, an embodiment of the present document may propose a scheme for signaling an MTS index. Here, the MTS index as described above may represent any of a plurality of transform kernel sets, and the MTS index may be encoded, and the MTS index information may be signaled to the decoding equipment. The decoding equipment may obtain the MTS index by decoding the MTS index information, and may determine the set of transform kernels to be applied based on the MTS index. The MTS index may be represented as a tu_mts_idx syntax element or an mts_idx syntax element. For example, an MTS index may be binarized using zero order Rice-Golomb parameters, but may be binarized based on a truncated Rice code. In the case of binarization based on the truncated Rice code, the input parameter cMax may have the value 4 and cRiceParam may have the value 0. For example, the encoding equipment may extract the permission element(s) for the MTS index by binarizing the MTS index , extract the MTS index information (bit(s) for the MTS index) by encoding the extracted permission element(s), and signal the MTS index information to the decoding equipment. The decoding equipment can extract the MTS index bin(s) by decoding the MTS index information, and extract the MTS index by comparing the extracted MTS index bin(s) with candidate MTS index bins.

[168] For example, an MTS index (eg, the tu_mts_idx syntax element or the mts_idx syntax element) may be context-encoded based on a context model or a context index relative to all permission elements. In this case, the increment of the context index (ctxInc) for the context encoding of the MTS index or ctxInc according to the location of the permission element may be allocated or determined as indicated in Table 2. Additionally, the context model may be selected according to the location of the permission element as indicated in the table. 2.

[169] Tab. 2

[170] Referring to Table 2, ctxInc for permission element number 0 (the first permission element) may be allocated based on cqtDepth. Here, cqtDepth may represent the depth of the quadtree for the current block, and may be retrieved as a single value in 0-5. Thus, ctxInc for a null permission element can allocate one value in 0-5 according to cqtDepth. Additionally, ctxInc for permission element number 1 (second permission element) may be allocated 6, ctxInc for permission element number 2 (third permission element) may be allocated 7, and ctxInc for permission element number 3 (fourth permission element) may be allocated 8. Thus , permission element number 0 - permission element number 3 can be allocated ctxInc, which has different values. Here, a different value of ctxInc may represent different context models, in which case, 9 context models for encoding the MTS index may be provided.

[171] Additionally, for example, the MTS index (for example, the tu_mts_idx syntax element or the mts_idx syntax element) may be bypass-encoded with respect to all permission elements as indicated in Table 3. In this case, 0 context models for encoding the MTS index may be provided.

[172] Tab. 3

[173] Additionally, for example, an MTS index (e.g., tu_mts_idx syntax element or mts_idx syntax element) may be context-encoded based on a context model or a context index relative to permission element number 0 (the first permission element) as indicated in Table 4, and may be bypassed with respect to the remaining permission elements. Thus, ctxInc for permission element number 0 (the first permission element) may be allocated 0. In this case, one context model for encoding the MTS index may be provided.

[174] Tab. 4

[175] Additionally, for example, an MTS index (e.g., the tu_mts_idx syntax element or the mts_idx syntax element) may be context-encoded based on a context model or context index relative to permission element number 0 (the first permission element) and permission element number 1 (the second permission element ) as specified in Table 5 and may be bypassed with respect to the remaining bins. Thus, ctxInc for permission element number 0 (first permission element) may be allocated 0, and ctxInc for permission element number 1 (second permission element) may be allocated 1. In this case, 2 context models for encoding the MTS index may be provided.

[176] Tab. 5

[177] Additionally, for example, an MTS index (e.g., the tu_mts_idx syntax element or the mts_idx syntax element) may be context-encoded based on a context model or a context index relative to all permission elements as indicated in Table 6, and may allocate one ctxInc relative to each permission element. Thus, ctxInc for permission element number 0 (the first permission element) may be allocated 0, and ctxInc for permission element number 1 (the second permission element) may be allocated 1; ctxInc for permission element number 2 (the third permission element) may be allocated 2, and ctxInc for permission element number 3 (the fourth permission element) may be allocated 2. In this case, 4 context models for encoding the MTS index may be provided.

[178] Tab. 6

[179] As described above, according to an embodiment, since the number of context models is reduced by applying bypass coding to all or some of the MTS index bins or by applying a specific value to ctxInc while context coding is applied, the complexity can be reduced and the amount of decoder output can be increase. Additionally, in an embodiment, in the case of using context models as described above, the initial value and/or multi-window size may be variable based on the occurrence statistics for the position of each bin.

[180] FIG. 9 and 10 schematically illustrate a video/image coding method and an example of related components according to embodiment(s) of the present document.

[181] The method disclosed in FIG. 9 may be performed by the encoding equipment disclosed in FIG. 2 or fig. 10. In particular, for example, S900-S920 of FIG. 9 may be performed by the encoding equipment residual processor 230 of FIG. 10 and S930 of FIG. 9 may be performed by the entropy encoder 240 of the encoding equipment of FIG. 10. Additionally, although not illustrated in FIG. 9, the prediction samples or prediction-related information may be retrieved by the prediction module 220 of the encoding equipment of FIG. 10, residual information may be extracted from the original samples or predictive samples by the encoding equipment residual processor 230, and a bitstream may be generated from the residual information or prediction-related information by the encoding equipment entropy encoder 240. The method disclosed in FIG. 9 may include the embodiments described above herein.

[182] Referring to FIG. 9, the encoding equipment extracts residual samples for the current block (S900). For example, the encoding equipment may derive residual samples based on the predictive samples and the original samples. Although not illustrated in FIG. 9, in order to generate prediction samples for the current block, the encoding equipment may perform intra prediction or inter prediction with respect to the current block considering the rate-based distortion cost function RD, and may generate prediction-related information including prediction mode/type information. .

[183] The encoding equipment extracts transform coefficients for the current block based on the residual samples (S910). For example, the encoding equipment may derive transform coefficients by performing a transform on the residual samples. Here, the transformation may be performed based on a transformation kernel or a set of transformation kernels. For example, the set of transformation kernels may include a transformation kernel in the horizontal direction and a transformation kernel in the vertical direction. For example, the encoding equipment may derive transform coefficients by performing a primary transform on the residual samples. Further, for example, the encoding equipment may extract temporal transform coefficients by performing a primary transform on the residual samples, and may extract the transform coefficients by performing a secondary transform on the temporal transform coefficients. For example, a transformation that is performed based on a set of transformation kernels may be represented as a primary transformation.

[184] The encoding equipment generates an MTS index and residual information based on the transform coefficients (S920). In other words, the encoding equipment may generate an MTS index and/or residual information based on the transform coefficients.

[185] The MTS index may represent a set of transform kernels applied to the (transform coefficients) of the current block, among the possible set of transform kernels. Here, the MTS index may be represented as a tu_mts_idx syntax element or an mts_idx syntax element. As described above, the set of transformation kernels may include a horizontal direction transformation kernel and a vertical direction transformation kernel, and the horizontal direction transformation kernel may be represented as trTypeHor, and the vertical direction transformation kernel may be represented as trTypeVer.

[186] For example, the values of trTypeHor and trTypeVer may be represented by a transform kernel in the horizontal direction and a transform kernel in the vertical direction applied to the (transform coefficients) of the current block, and the MTS index may be represented as one of the options including 0- 4 through the values trTypeHor and trTypeVer.

[187] For example, if the MTS index is 0, it may represent that trTypeHor and trTypeVer are 0. Additionally, if the MTS index is 1, it may represent that trTypeHor and trTypeVer are equal to 1. Additionally, if the MTS index is is 2, it may represent that trTypeHor is 2 and trTypeVer is 1. If MTS index is 3, it may represent that trTypeHor is 1 and trTypeVer is 1. Additionally, if MTS index is 4, it may represent that trTypeHor and trTypeVer are 2. For example, if the value of trTypeHor or trTypeVer is 0, it may represent that DCT2 is applied to the (transformation factors) of the current block in the horizontal direction, or in the vertical direction, if 1, it may represent that DST7 applies, and if 2, it may represent that DCT8 applies. Thus, each of a transformation kernel to be applied in the horizontal direction and a transformation kernel to be applied in the vertical direction can be presented as one of the options including DCT2, DST7, and DCT8 based on the MTS index.

[188] The MTS index may be represented based on the permission elements from the permission element row of the MTS index. In other words, the MTS index may be binarized and represented as (lexi) from the MTS index le element string, and (leni) from the MTS index le row may be entropy encoded.

[189] In other words, at least one of the permission elements from the permission element row of the MTS index may be represented based on the context encoding. Here, the context encoding may be performed based on the increment value of the context index (ctxInc). Additionally, context encoding may be performed based on a context index (ctxIdx) or a context model. Here, the context index may be represented based on the increment value of the context index. Additionally, the context index may be represented based on the value of the context index increment and the context index offset (ctxIdxOffset).

[190] For example, all permission elements from the permission element row of the MTS index may be represented based on the context encoding. For example, ctxInc for the first permission element or permission element zero (permission element 0) of the permission elements in the MTS-index permission element row can be represented based on cqtDepth. Here, cqtDepth may represent the depth of the quadtree for the current block, and may be represented as one value from 0-5. Additionally, ctxInc for the second permission element or permission element number 1 (permission element 1) may be represented as 6, ctxInc for the third permission element or permission element number 2 (permission element 2) may be represented as 7, and ctxInc for the fourth permission element or permission element permission number 3 (permission element 3) may be represented as 8. Additionally, for example, ctxInc for the first permission element or permission element zero (permission element 0) of the permission elements from the MTS-index permission element row may be represented as 0, ctxInc for the second permission element permission or permission element number 1 (permission element 1) can be represented as 1, ctxInc for the third permission element or permission element number 2 (permission element 2) can be represented as 2, and ctxInc for the fourth permission element or permission element number 3 (element 3 permission) can be represented as 3. Thus, the number o the context index increment values that can be used to context-encode the first of the permission elements from the permission element string may be equal to one.

[191] Additionally, for example, some permission elements from the MTS index permission element row may be represented based on the context encoding, and the remaining ones may be represented based on the bypass encoding. For example, ctxInc for the first permission element or permission element number 0 (permission element 0) of the permission elements from the MTS-index permission element row may be represented as 0, and the remaining permission elements may be represented based on bypass encoding. Additionally, for example, ctxInc for the first permission element or permission element number 0 (permission element 0) of the permission elements from the MTS-index permission element row may be represented as 0, ctxInc for the second permission element or permission element number 1 (permission element 1) may be represented as 1, and the remaining bins may be represented based on bypass coding. Thus, the number of context index increment values that can be used to context-encode the first permission element of the permission elements of the permission element string may be one.

[192] Additionally, all permission elements from the permission element row of the MTS index may be represented based on bypass coding. Here, bypass encoding may represent performing context encoding based on a uniform probability distribution, and since the context encoding update procedure is omitted, encoding efficiency may be improved.

[193] The residual information may represent information that is used to extract residual samples. Additionally, for example, the encoding equipment may perform quantization for the transform coefficients, and the residual information may include information regarding the residual samples, transform-related information, and/or quantization-related information. For example, the residual information may include information regarding quantized transform coefficients.

[194] The encoding equipment encodes image information including an MTS index and residual information (S930). For example, the image information may further include prediction related information. For example, the encoding equipment may generate a bit stream by encoding image information. Here, the bit stream may be referred to as "encoded information (pictures)".

[195] Additionally, although not illustrated in FIG. 9, for example, the encoding equipment may generate reconstructed samples based on residual samples and predictive samples. Additionally, the reconstructed block and the reconstructed frame may be derived based on the reconstructed samples.

[196] For example, the encoding equipment may generate bitstream information or encoded information by encoding image information including all or parts of the above-described pieces of information (or syntax elements). Additionally, the information may be output in the form of a bitstream. Additionally, the bitstream or encoded information may be transmitted to the decoding equipment via a network or a storage medium. Further, the bitstream or encoded information may be stored in a computer-readable storage medium, and the bitstream or encoded information may be generated by the above-described image encoding method.

[197] FIG. 11 and 12 schematically illustrate a video/image decoding method and an example of related components according to the embodiment(s) of the present document.

[198] The method disclosed in FIG. 11 may be performed by the decoding equipment disclosed in FIG. 3 or fig. 12. In particular, for example, S1100 of FIG. 11 may be performed by the entropy decoder 310 of the decoding equipment of FIG. 12 and S1110 and S1120 of FIG. 11 may be performed by the decoding hardware residual processor 320 of FIG. 12. Additionally, although not illustrated in FIG. 11, prediction related information or residual information can be extracted from the bitstream by the entropy decoder 310 of the decoding equipment of FIG. 12 and the residual samples can be extracted from the residual information by the residual processor 320 of the decoding hardware. Predictive samples may be extracted from prediction-related information by decoding equipment predictor 330, and a reconstructed block or reconstructed frame may be extracted from residual samples or predictive samples by decoding equipment adder 340 . The method disclosed in FIG. 11 may include the embodiments described above herein.

[199] Referring to FIG. 11, the decoding equipment obtains the MTS index and residual information from the bit stream (S1100). For example, the decoding equipment may obtain the MTS index and/or residual information by parsing or decoding the bitstream. Here, the bitstream may be referred to as "encoded information (pictures)".

[200] The MTS index may represent a set of transformation cores to be applied to the current block from among the possible sets of transformation cores. Here, the MTS index may be represented as a tu_mts_idx syntax element or an mts_idx syntax element. Additionally, the set of transformation kernels may include a transformation kernel to be applied to the current block in the horizontal direction and a transformation kernel to be applied to the current block in the vertical direction. Here, a transformation kernel to be applied in the horizontal direction may be represented as trTypeHor, and a transformation kernel to be applied in the vertical direction may be represented as trTypeVer.

[201] For example, the MTS index can be extracted as one of the options including 0-4, and according to the MTS index, each of trTypeHor and trTypeVer can be extracted as one of 0-2. For example, if the MTS index is 0, trTypeHor and trTypeVer may be 0. Additionally, if the MTS index is 1, trTypeHor and trTypeVer may be 1. Additionally, if the MTS index is 2, trTypeHor may be 2, and trTypeVer may be 1. If the MTS index is 3, trTypeHor may be 1, and trTypeVer may be 1. Additionally, if the MTS index is 4, trTypeHor and trTypeVer may be 2. For example, the value of trTypeHor or trTypeVer may be represent a transform kernel, and if 0, it may represent DCT2, if 1, it may represent DST7, and if 2, it may represent DCT8. Thus, each of a transformation kernel to be applied in the horizontal direction and a transformation kernel to be applied in the vertical direction can be extracted as one of the options including DCT2, DST7, and DCT8 based on the MTS index.

[202] The MTS index may be derived based on the permission elements from the permission element row of the MTS index. In other words, the MTS index information can be entropy decoded and can be retrieved as a binarized MTS index, and the binarized MTS index can be represented as (bins) from the bin row of the MTS index.

[203] In other words, at least one of the permission elements from the permission element row of the MTS index can be extracted based on the context encoding. Here, the context encoding may be performed based on the increment value of the context index (ctxInc). Additionally, context encoding may be performed based on a context index (ctxIdx) or a context model. Here, the context index may be derived based on the increment value of the context index. Additionally, the context index may be derived based on the value of the context index increment and the context index offset (ctxIdxOffset).

[204] For example, all permission elements from the permission element row of the MTS index can be extracted based on the encoding context. For example, ctxInc for the first permission element or permission element zero (permission element 0) of the permission elements in the MTS-index permission element row can be allocated based on cqtDepth. Here, cqtDepth may represent the depth of the quadtree for the current block, and may be retrieved as a single value from 0-5. Additionally, ctxInc for the second permission element or permission element number 1 (permission element 1) may be allocated 6, ctxInc for the third permission element or permission element number 2 (permission element 2) may be allocated 7, and ctxInc for the fourth permission element or permission element number 3 (permission element 3) can be allocated 8. Additionally, for example, ctxInc for the first permission element or permission element zero (permission element 0) of the permission elements from the MTS-index permission element string, 0 can be allocated, ctxInc for the second permission element or permission element number 1 (permission element 1) may be allocated 1, ctxInc for the third permission element or permission element number 2 (permission element 2) may be allocated 2, and ctxInc for the fourth permission element or permission element number 3 (permission element 3) may be allocated 3. Thus, the number of context index increment values that can and be used to context-encode the first permission element of the permission elements in the permission element string, may be equal to one.

[205] Additionally, for example, some permission elements from the MTS index permission element row may be extracted based on the context encoding, and the remaining ones may be extracted based on the bypass encoding. For example, ctxInc for the first permission element or permission element number 0 (permission element 0) of the permission elements from the MTS-index permission element row can be allocated 0, and the remaining permission elements can be extracted based on bypass encoding. Additionally, for example, ctxInc for the first permission element or permission element number 0 (permission element 0) of the permission elements from the MTS index permission element row can be allocated 0, ctxInc for the second permission element or permission element number 1 (permission element 1) can be allocated 1 and the remaining permission elements may be extracted based on the bypass coding. Thus, the number of context index increment values that can be used to context-encode the first permission element of the permission elements of the permission element string may be one.

[206] Additionally, all permission elements from the permission element row of the MTS index may be extracted based on the bypass encoding. Here, bypass encoding may represent performing context encoding based on a uniform probability distribution, and since the context encoding update procedure is omitted, encoding efficiency may be improved.

[207] The residual information may represent information that is used to extract residual samples and may include information on residual samples, inverse transform related information, and/or dequantization related information. For example, the residual information may include information regarding quantized transform coefficients.

[208] The decoding equipment extracts transform coefficients for the current block based on the residual information (S1110). For example, the decoding equipment may derive quantized transform coefficients for the current block based on information regarding quantized transform coefficients included in the residual information. For example, the decoding equipment may derive transform coefficients for the current block by performing dequantization with respect to the quantized transform coefficients.

[209] The decoding equipment generates residual samples of the current block based on the MTS index and transform coefficients (S1120). For example, the residual samples may be generated based on a set of transform kernels represented by transform coefficients and an MTS index. Thus, the decoding equipment can generate residual samples from the transform coefficients via an inverse transform using the set of transform kernels represented by the MTS index. Here, the inverse transform using the set of transform kernels represented by the MTS index may be included in the primary inverse transform. Additionally, the decoding equipment may use not only a primary inverse transform but also a secondary inverse transform when generating residual samples from the transform coefficients. In this case, the decoding equipment may extract the modified transform coefficients by performing a secondary inverse transform on the transform coefficients, and may generate residual samples by performing a primary inverse transform on the modified transform coefficients.

[210] Although not illustrated in FIG. 11, for example, the decoding equipment may obtain prediction-related information including prediction mode/type from the bit stream and may generate prediction samples for the current block by performing intra prediction or inter prediction based on the prediction mode/type information. Additionally, for example, the decoding hardware may generate reconstructed samples based on the predictive samples and the residual samples. Additionally, for example, a reconstructed block or a reconstructed frame may be derived based on the reconstructed samples.

[211] For example, the decoding equipment can obtain image information including all or parts of the above-described pieces of information (or syntax elements) by decoding the bit stream or encoded information. Additionally, the bit stream or encoded information may be stored in a computer-readable storage medium and may cause the decoding method described above to be performed.

[212] Although the methods are described based on a flowchart in which the steps or blocks are listed in sequence in the above-described embodiments, the steps of this document are not limited to a certain order, and a certain step may be performed in a different step or in a different order or simultaneously. regarding what is described above. Additionally, those skilled in the art will appreciate that the steps in the flowchart are not the only ones possible, and another step may be included, or one or more steps in the flowchart may be removed without affecting the scope of the present disclosure.

[213] The above method according to the present disclosure may be implemented in the form of software, and the encoding equipment and/or decoding equipment according to the present disclosure may be included in an apparatus for performing image processing, such as a television, computer, smartphone, set-top box, display device, and etc.

[214] When embodiments of the present disclosure are implemented by software, the above method may be implemented by a module (process or function) that performs the above function. The module may be stored in a storage device and executed by a processor. The storage device may be installed inside or outside the processor and may be connected to the processor through various known means. The processor may include an application specific integrated circuit (ASIC), other chipsets, logic, and/or a data processing device. The storage device may include Read Only Memory (ROM), Random Access Memory (RAM), flash memory, a memory card, a storage medium, and/or other storage device. In other words, embodiments according to the present disclosure may be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional modules illustrated in the respective drawings may be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, implementation information (eg, information regarding instructions) or algorithms may be stored on a digital storage medium.

[215] In addition, the decoding equipment and encoding equipment to which the embodiment(s) of this document applies may be included in a broadcast multimedia transceiver, a mobile communication terminal, a home theater system video device, a digital cinema system video device, a surveillance camera, video chatting device and real-time communication device, such as video communication, mobile streaming device, storage media, video recording camera, video-on-demand (VoD) service provider, video device over networks (OTT), service provider Internet streaming device, 3D video device, virtual reality (VR) style device, augmented reality (AR) style device, telephone video device, in-vehicle terminal (e.g., in-vehicle terminal (including autonomous vehicle), air terminal udne or terminal in a sea vessel) and a medical video device; and may be used to process the image signal or data. For example, an OTT video device may include a game console, a Blu-ray player, an Internet TV, a home theater system, a smartphone, a tablet PC, and a digital video recorder (DVR).

[216] In addition, the processing method to which the embodiment(s) of the present document is applied may be formed in the form of a program executable by a computer and may be stored in a computer-readable recording medium. The multimedia data having the data structure according to the embodiment(s) of the present document may also be stored in a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices that store computer-readable data. The computer-readable recording medium may include, for example, a Blu-ray Disc (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical device. data storage. The computer-readable recording medium also includes media implemented in the form of a carrier wave (eg, transmission over the Internet). In addition, the bit stream generated by the encoding method may be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network.

[217] In addition, the embodiment(s) of the present document may be implemented as a computer program product based on the program code, and the program code may be executed on the computer according to the embodiment(s) of the present document. The program code may be stored on a computer readable medium.

[218] FIG. 13 illustrates an example of a content streaming system to which the embodiments disclosed herein are applicable.

[219] Referring to FIG. 13, a content streaming system to which the embodiments disclosed herein are applicable may generally include an encoding server, a streaming server, a web server, a media storage, a user device, and a media input device.

[220] The encoding server functions to compress content input from media input devices such as a smartphone, camera, camcorder, and the like into digital data to generate a bitstream and transmit it to the streaming server. As another example, in the case where a media input device such as a smartphone, a camera, a camcorder or the like directly generates a bitstream, the encoding server may be omitted.

[221] The bit stream may be generated by an encoding method or a bit stream generation method to which the embodiments of the present document apply. In addition, the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[222] The streaming server transmits media data to the user equipment based on a user request via a web server that functions as a tool that informs the user as to which service is provided. When the user requests a service that the user wants, the web server passes the request to the streaming server, and the streaming server transmits the media data to the user. In this regard, the content streaming system may include a separate control server, in which case, the control server functions to manage commands/responses between respective equipments in the content streaming system.

[223] The streaming server may receive content from a media store and/or an encoding server. For example, in case the content is received from the encoding server, the content may be received in real time. In this case, the streaming server may keep the bit stream for a predetermined period of time to smoothly provide the streaming service.

[224] For example, the subscriber equipment may include a mobile phone, a smartphone, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable media player (PMP), a navigation device, a stylus tablet PC, a tablet PC, an ultrabook, a wearable device (e.g., watch terminal (smart watch), glasses terminal (smart glasses), head-mounted display (HMD)), digital TV, desktop computer, digital signage system, etc.

[225] Each of the servers in the content streaming system may operate as a distributed server, in which case, data received by each server may be processed in a distributed manner.

[226] the Claims in the present description can be combined in various ways. For example, the technical features in the method claims of the present disclosure can be combined with the ability to be implemented or performed in equipment, and the technical features in the equipment claims can be combined with the ability to be implemented or performed in the method. Additionally, the technical features in the method claim(s) and the equipment claim(s) can be combined with the ability to be implemented or performed in the equipment. Additionally, the technical features in the method claim(s) and the equipment claim(s) can be combined with the ability to be implemented or performed in the method.

Claims (39)

1. An image decoding method carried out by means of decoding equipment, the method comprising the steps of: obtaining a multiple choice transform index (MTS) and residual information by parsing the bit stream; extracting transform coefficients for the current block based on the residual information; And generating residual samples of the current block based on the MTS index and transform coefficients, at the same time, the MTS index represents a set of transformation kernels that should be applied to the current block, from the possible variants of sets of transformation kernels, wherein the permission elements from the MTS index permission element string are extracted based on the context encoding, the context encoding is performed based on the context index increment values for the permission elements from the MTS index permission element string, wherein the permission elements from the permission element row of the MTS index include the first, second, third and fourth permission elements, and wherein the context index increment value for the first permit element is fixed at 0, the context index increment value for the second permit element is fixed at 1, the context index increment value for the third permit element is fixed at 2, and the context index increment value for the fourth permit element is fixed at 3. 2. The image decoding method according to claim 1, wherein the set of transformation kernels comprises a transformation kernel to be applied to the current block in the horizontal direction and a transformation kernel to be applied to the current block in the vertical direction. 3. The image decoding method of claim 1, wherein each of a transformation kernel to be applied in the horizontal direction and a transformation kernel to be applied in the vertical direction is extracted as one of the options including DCT2, DST7, and DCT8, based on the MTS index. 4. The image decoding method according to claim 1, wherein the MTS index is extracted as one of the options including 0-4, and The MTS index is retrieved based on the permission entries from the MTS index's permission entries row. 5. An image encoding method carried out by encoding equipment, the method comprising the steps of: extracting residual samples for the current block; extracting transform coefficients for the current block based on the residual samples; generating a multiple choice transform index (MTS) and residual information based on the transform coefficients; And encoding image information including MTS index and residual information, at the same time, the MTS index represents a set of transformation kernels that should be applied to the current block, from the possible variants of sets of transformation kernels, wherein the permission elements from the MTS index permission element string are represented based on the context encoding, the context encoding is performed based on the context index increment values for the permission elements from the MTS index permission element string, wherein the permission elements from the permission element row of the MTS-index include the first, second, third and fourth permission elements, and wherein the context index increment value for the first permit element is fixed at 0, the context index increment value for the second permit element is fixed at 1, the context index increment value for the third permit element is fixed at 2, and the context index increment value for the fourth permit element is fixed at 3. 6. The image encoding method according to claim 5, wherein the set of transformation kernels comprises a transformation kernel to be applied to the current block in the horizontal direction and a transformation kernel to be applied to the current block in the vertical direction. 7. The image encoding method according to claim 5, wherein each of a transformation kernel to be applied in the horizontal direction and a transformation kernel to be applied in the vertical direction is represented as one of the options including DCT2, DST7, and DCT8 , based on the MTS index. 8. A non-volatile computer-readable digital storage medium that stores a bit stream generated by an image encoding method, the method comprising: extracting residual samples for the current block; extracting transform coefficients for the current block based on the residual samples; generating a multiple choice transform index (MTS) and residual information based on the transform coefficients; And encoding image information to form a bit stream, wherein the image information includes an MTS index and residual information, at the same time, the MTS-index represents a set of transformation kernels that should be applied to the current block, from the possible options for sets of transformation kernels, wherein the permission elements from the MTS index permission element string are represented based on the context encoding, the context encoding is performed based on the context index increment values for the permission elements from the MTS index permission element string, wherein the permission elements from the permission element row of the MTS-index include the first, second, third and fourth permission elements, and the context index increment value for the first permit element is fixed at 0, the context index increment value for the second permit element is fixed at 1, the context index increment value for the third permit element is fixed at 2, and the context index increment value for the fourth permit element is fixed at 3. 9. A method for transmitting data for an image to decoding equipment, the method comprising the steps of: a bitstream for an image is obtained, wherein the bitstream is generated based on extracting the residual samples for the current block, extracting transform coefficients for the current block based on the residual samples, generating a multiple choice transform index (MTS) and residual information based on the transform coefficients, and encoding image information , which includes the MTS index and residual information; And transmitting the data containing the bit stream to the decoding equipment to perform decoding and generate a reconstructed image based on the encoded image information, at the same time, the MTS-index represents a set of transformation kernels that should be applied to the current block, from the possible options for sets of transformation kernels, wherein the permission elements from the MTS index permission element string are represented based on the context encoding, the context encoding is performed based on the context index increment values for the permission elements from the MTS index permission element string, wherein the permission elements from the permission element row of the MTS-index include the first, second, third and fourth permission elements, and the context index increment value for the first permit element is fixed at 0, the context index increment value for the second permit element is fixed at 1, the context index increment value for the third permit element is fixed at 2, and the context index increment value for the fourth permit element is fixed at 3.

Images