RU2815810C2 - Encoding information on set of transformation kernels - Google Patents

Abstract

FIELD: image processing means.

SUBSTANCE: invention relates to image encoding technology. Result is achieved by obtaining image information including a multiple transformation selection (MTS) index and residual information from the bit stream, based on which quantised transformation coefficients are extracted to obtain transformation coefficients for the current block by performing dequantization over quantised transformation coefficients. Residual samples of the current block are formed by performing an inverse transformation on the transformation coefficients based on the set of transformation kernels associated with the MTS index, which is a set of conversion kernels, which must be applied to the current unit, from possible versions of sets of conversion kernels. Further, resolution elements from the MTS index resolution elements row are extracted based on context encoding, executed based on context index increment values for resolution elements from MTS-index resolution elements row, wherein resolution elements from the row of resolution elements of the MTS index include four resolution elements.

EFFECT: high efficiency of encoding images/video.

4 cl, 13 dwg, 6 tbl

Description

State of the art

Field of technology to which the invention relates

[1] The present disclosure relates to image encoding technology, and more particularly to encoding information regarding a set of transform kernels.

Description of the Prior Art

[2] Recently, the demand for high-quality high-definition images/videos, for example, 4K, 8K or even higher ultra-high definition (UHD) images/videos, has been growing in various fields of technology. As the resolution or quality of images/videos becomes higher, a relatively larger amount of information or number of bits is transmitted than for traditional image/video data. Therefore, if image/video data is transmitted through a medium such as an existing wired/wireless broadband line, or stored on a legacy storage medium, transmission and storage costs increase significantly.

[3] In addition, there is growing interest and demand for virtual reality (VR) and artificial reality (AR) content and immersive multimedia, such as holograms; and there is also an increase in broadcasting of images/videos exhibiting characteristics of the images/videos that differ from those of the actual image/video, for example, game images/videos.

[4] Therefore, a highly efficient image/video compression technology is required in order to efficiently compress and transmit, store or play back high-quality, high-resolution images/videos exhibiting various characteristics as described above.

The essence of the invention

[5] According to an embodiment of the present document, a method and equipment for improving the efficiency of image/video encoding are provided.

[6] According to an embodiment of the present document, a method and equipment for encoding an MTS index in video encoding are provided.

[7] According to an embodiment of the present document, a method and equipment for transmitting MTS index information in signaling signals are provided.

[8] According to an embodiment of the present document, a method and equipment are provided for signaling information representing a set of transformation kernels to be applied to a current block from a plurality of sets of transformation kernels.

[9] According to an embodiment of the present document, a method and equipment for context encoding or bypass encoding with respect to resolution elements for an MTS index are provided.

[10] According to an embodiment of the present document, a video/image decoding method performed by decoding equipment is provided.

[11] According to an embodiment of the present document, decoding equipment for performing video/image decoding is provided.

[12] According to an embodiment of the present document, a video/image encoding method performed by encoding equipment is provided.

[13] According to an embodiment of the present document, encoding equipment for performing video/image encoding is provided.

[14] According to an embodiment of the present document, a computer-readable digital storage medium storing encoded video/image information generated according to a video/image encoding method disclosed in at least one embodiment of this document is provided.

[15] According to an embodiment of the present document, a computer-readable digital storage medium storing encoded information or encoded video/image information causing decoding equipment to perform a video/image decoding method disclosed in at least one embodiment of this document is provided.

[16] According to the present document, the overall efficiency of image/video compression can be improved.

[17] According to the present document, MTS index information can be efficiently transmitted in signaling signals.

[18] According to the present document, the complexity of the encoding system can be reduced by efficiently encoding the MTS index information.

[19] The benefits that can be obtained through the detailed example of this document are not limited to the benefits listed above. For example, various technical effects may be contemplated that may be understood or inferred by those skilled in the art from the present document. Accordingly, the detailed advantages of this document are not limited to those explicitly stated herein, and may include various advantages that may be understood or inferred from the technical features of this document.

Brief description of drawings

[20] FIG. 1 schematically illustrates an example of a video/image coding system to which this document is applicable.

[21] FIG. 2 is a diagram schematically explaining the configuration of video/image encoding equipment to which this document is applicable.

[22] FIG. 3 is a diagram schematically explaining the configuration of video/image decoding equipment to which this document is applicable.

[23] FIG. 4 schematically illustrates a multiple conversion technology according to an embodiment of the present document.

[24] FIG. 5 roughly illustrates the internal directional modes in the 65 forecast directions.

[25] FIG. 6 and 7 are diagrams for explaining RST according to an embodiment of the present document.

[26] FIG. 8 approximately illustrates context-adaptive binary arithmetic coding (CABAC) for encoding syntactic elements.

[27] FIG. 9 and 10 schematically illustrate a video/image encoding method and an example of related components according to embodiment(s) of the present document.

[28] FIG. 11 and 12 schematically illustrate a video/image decoding method and an example of related components according to embodiment(s) of the present document.

[29] FIG. 13 illustrates an example of a content streaming system to which the embodiments disclosed herein are applicable.

Detailed Description of Embodiments

[30] The present disclosure may be modified in various forms, and specific embodiments thereof are described and illustrated in the drawings. However, these embodiments are not intended to limit the disclosure. The terms used in the following description are used to simply describe specific embodiments, but are not intended to limit the disclosure. Singular expressions include plural expressions as long as they are clearly interpreted differently. Terms such as “include” and “have” are intended to indicate that there are features, numbers, steps, operations, elements, components, or combinations thereof used in the following description, and as such are to be understood that the possibility of the presence or addition of one or more other features, numbers, steps, operations, elements, components or combinations of the above is not excluded.

[31] In addition, each configuration of the drawings described in this document is an independent illustration for explaining functions as features that are different from each other, and does not mean that each configuration is implemented by mutually different hardware or different software. For example, two or more configurations may be combined to form one configuration, and one configuration may also be divided into multiple configurations. Without departing from the spirit of this document, embodiments in which configurations are combined and/or separated are included within the scope of the claims.

[32] Hereinafter, examples of the present embodiment will be described in detail with reference to the accompanying drawings. In addition, like reference numerals are used to identify like elements throughout the drawings, and identical descriptions of like elements are omitted.

[33] This document relates to video/image coding. For example, the methods/embodiments disclosed herein may relate to the Universal Video Coding (VVC) standard (ITU-T (Rec. (H.266), the next generation video/image coding standard after VVC, or other related video coding standards (for example, High Efficiency Video Coding (HEVC) standard (ITU-T (Rec. (H.265), EVC), AVS2 standard, etc.).

[34] This document proposes various embodiments of video/image encoding, and the above embodiments may also be performed in combination with each other unless otherwise noted.

[35] In this document, video may refer to a sequence of images over time. A frame generally means a unit representing one image in a particular time frame, and a slice/tile means a unit constituting a part of a frame from an encoding point of view. A slice/tile may include one or more coding tree units (CTUs). One frame can consist of one or more slices/tiles. One frame can consist of one or more groups of tiles. One tile group may include one or more tiles.

[36] Pixel or pel can mean the smallest unit that makes up one frame (or image). Additionally, "sample" can be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel/pixel value of a luma component or only a pixel/pixel value of a chroma component.

[37] The unit may represent a basic image processing unit. The unit may include at least one of a specific frame area and information associated with the area. One unit may include one block of luma signals and two blocks of chrominance signals (eg, Cb, Cr). Unit may be used interchangeably with terms such as block or zone in some cases. In general, an MxN block may include samples (or arrays of samples) or a set (or array) of transform coefficients of M columns and N rows. Alternatively, a sample may refer to a pixel value in the spatial domain, and when such a pixel value is converted to a frequency domain, it may refer to a frequency domain conversion factor.

[38] In this document, the terms "/" and "," shall be interpreted to indicate "and/or". For example, the expression "A/B" could mean "A and/or B". Additionally, "A, B" may mean "A and/or B". Additionally, "A/B/C" may mean "at least one of A, B and/or C". In addition, "A/B/C" may mean "at least one of A, B and/or C".

[39] Additionally, as used herein, the term “or” should be interpreted to indicate “and/or.” For example, the expression "A or B" may contain 1) only A, 2) only B, and/or 3) both A and B. In other words, the term "or" in this document should be interpreted to indicate "in addition or alternatively" .

[40] In the present specification, “at least one of A and B” may mean “A only,” “B only,” or “both A and B.” Moreover, in the present specification, the expression “at least one of A or B” or “at least one of A and/or B” may be interpreted identically to “at least one of A and B”.

[41] Additionally, as used herein, “at least one of A, B, and C” may mean “A only,” “B only,” “C only,” or “any combination of A, B, and C.” Additionally, “at least one of A, B, or C” or “at least one of A, B, and/or C” may mean “at least one of A, B, and C.”

[42] Additionally, parentheses used in the present description of the invention may mean "for example." In particular, in the case where "forecasting (internal forecasting)" is expressed, it may be indicated that "internal forecasting" is offered as an example of "forecasting". In other words, the term "prediction" in the present specification is not limited to "internal prediction", and it may be stated that "internal prediction" is offered as an example of "prediction". Additionally, even if "forecasting (ie, internal forecasting)" is expressed, it may be indicated that "internal forecasting" is offered as an example of "forecasting".

[43] In the present specification, technical features separately explained in one drawing may be implemented separately or may be implemented simultaneously.

[44] FIG. 1 illustrates an example of a video/image coding system to which the disclosure herein may apply.

[45] Referring to FIG. 1, the video/image encoding system may include a source device and a receiving device. A source device may transmit encoded information or video/image data to a receiving device via a digital storage medium or network in the form of a file or stream.

[46] The source device may include a video source, encoding equipment, and a transmitter. The receiving device may include a receiver, decoding equipment, and a rendering module. The encoding equipment may be referred to as "video/image encoding equipment" and the decoding equipment may be referred to as "video/image decoding equipment". The transmitter may be included in the encoding equipment. The receiver may be included in the decoding equipment. The rendering module may include a display, and the display may be configured as a separate device or an external component.

[47] A video source may obtain video/image through a video/image capture, synthesis, or shaping process. The video source may include a video/image capture device and/or a video/image generation device. The video/image capturing device may include, for example, one or more cameras, video/image archives including previously captured video/images, and the like. The video/image generating apparatus may include, for example, computers, tablet computers and smartphones and may (electronically) generate video/images. For example, a virtual video/image may be generated via a computer or the like. In this case, the video/image capturing process can be replaced by the associated data generation process.

[48] Encoding equipment can encode the input video/image. Encoding hardware may perform a sequence of procedures such as prediction, transform, and quantization to achieve compression and encoding efficiency. Encoded data (encoded video/image information) may be output in the form of a bit stream.

[49] A transmitter may transmit information or encoded image/image data output in the form of a bitstream to a receiver of a receiving device via a digital storage medium or network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and the like. The transmitter may include an element for generating a multimedia file through a predefined file format and may include an element for transmission through a broadcast/communication network. The receiver may receive/extract the bit stream and transmit the received bit stream to decoding equipment.

[50] The decoding equipment may decode a video/image by performing a sequence of procedures such as dequantization, deconversion, and prediction corresponding to the operation of the encoding equipment.

[51] The rendering module may prepare a decoded video/image by rendering. The video/image prepared by rendering can be displayed through the display.

[52] FIG. 2 is a diagram fundamentally illustrating the configuration of video/image encoding equipment to which the disclosure of this document may be applied. Hereinafter in this document, what is referred to as “video encoding equipment” may include image encoding equipment.

[53] Referring to FIG. 2, encoding equipment 200 may include and be configured with an image segmentation module 210, a prediction module 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The prediction module 220 may include a mutual module 221. forecasting and internal forecasting module 222. Residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. Residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a “recovery module” or a “recovered block generator.” The image segmentation module 210, prediction module 220, residual processor 230, entropy encoder 240, adder 250, and filter 260, which are described above, may be configured by one or more hardware components (eg, encoder chipsets or processors) according to an embodiment. In addition, storage device 270 may include a decoded frame buffer (DPB) and may also be configured by a digital storage medium. The hardware component may further include a storage device 270 as an internal/external component.

[54] The image segmentation module 210 may split an input image (or frame) input to the encoding equipment 200 into one or more processing units. As an example, the processing unit may be called a "coding unit (CU)". In this case, the coding unit may be recursively partitioned according to a quadtree, binary tree, and ternary tree (QTBTTT) structure from a coding tree unit (CTU) or largest coding unit (LCU). For example, one encoding unit may be decomposed into multiple encoding units of greater depth based on a quadtree structure, a binary tree structure, and/or a ternary tree structure. In this case, for example, a quadtree structure is first applied, and a binary tree structure and/or a ternary tree structure may subsequently be applied. Alternatively, a binary tree structure can also be used first. The encoding procedure of the present disclosure may be performed based on a final encoding unit that is no longer split. In this case, based on the coding efficiency according to the characteristics of images and the like, the maximum coding unit can be directly used as the final coding unit, or if necessary, the coding unit can be recursively divided into coding units of greater depth, so that a coding unit having optimal size, can be used as the final coding unit. Here, the encoding procedure may include a procedure such as prediction, transformation and reconstruction, which is described below. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, each of the prediction unit and the transformation unit may be split or segmented from the above final encoding unit. The prediction unit may be a sample prediction unit, and the transformation unit may be a unit for inferring a transformation coefficient and/or a unit for inferring a residual signal from the transformation coefficient.

[55] Unit may be used interchangeably with a term such as block or zone in some cases. Typically, an MxN block may represent samples consisting of M columns and N rows or a group of transform coefficients. A sample may generally represent a pixel or a pixel value, and may also represent only a pixel/pixel value of a luma component, and may also represent only a pixel/pixel value of a chrominance signal component. Sample can be used as a term corresponding to a pixel or pel configuring one frame (or image).

[56] The subtractor 231 may generate a residual signal (a residual block, residual samples, or an array of residual samples) by subtracting a prediction signal (a predicted block, prediction samples, or an array of prediction samples) output from prediction module 220 from an input image signal (original block, original samples or an array of original samples) and the generated residual signal is transmitted to converter 232. Prediction module 220 may perform prediction on a target processing block (hereinafter referred to as a “current block”) and generate a predicted block including prediction samples for the current block. Prediction module 220 may determine whether intra-prediction or inter-prediction is applied or not, for the current block or in CU units. As described below in the description of each prediction mode, the prediction module may generate various kinds of prediction-related information, for example, prediction mode information, and transmit the generated information to the entropy encoder 240. Information regarding the prediction may be encoded in the entropy encoder 240 and output in the form bit stream.

[57] Intra prediction module 222 may predict the current block with reference to samples within the current frame. The referenced samples may be located as adjacent to the current block or may also be located at a large distance from the current block according to the prediction mode. Prediction modes in intra prediction may include a plurality of non-directional modes and a plurality of directed modes. The omnidirectional mode may include, for example, DC mode or planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the precise degree of the prediction direction. However, this is exemplary, and directional prediction modes that are greater or less than the above number may be used as configured. The intra prediction module 222 can also determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[58] The inter-prediction unit 221 may logically output a predicted block of the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. At this time, in order to reduce the amount of motion information transmitted in the inter-prediction mode, motion information may be predicted in units of blocks, sub-blocks or samples based on the correlation of motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0-prediction, L1-prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block existing in the current frame and a temporal neighbor block existing in the reference frame. The reference frame including the reference block and the reference frame including the temporary adjacent block may be identical to each other or different from each other. A temporary neighbor block may be referred to by a name such as a "co-located reference block", a "co-located CU (colCU)" and the like, and a reference frame including a temporary neighboring block may also be called a "co-located frame ( colPic)". For example, the inter-prediction module 221 may configure a list of motion information candidates based on neighboring blocks and generate information indicating which candidate is used to retrieve the motion vector and/or reference frame index of the current block. Inter prediction can be performed based on various prediction modes, and, for example, in the case of the skip mode and the merge mode, the inter prediction unit 221 can use the motion information of a neighboring block as the motion information of the current block. In the case of the skip mode, the residual signal may not be transmitted, unlike the combine mode. A motion vector prediction (MVP) mode may indicate the motion vector of a current block by using the motion vector of an adjacent block as a motion vector predictor and transmitting motion vector difference signals.

[59] The prediction module 220 may generate a prediction signal based on various prediction methods described below. For example, the prediction module may not only apply intra-prediction or inter-prediction to predict one block, but also simultaneously apply both intra-prediction and inter-prediction. This may be referred to as "combined inter-intraprediction (CIIP)". In addition, the prediction module can perform intra-block copying (IBC) to predict a block. In-block copying can be used for encoding content images/game moving images and the like, such as screen content encoding (SCC). IBC essentially performs prediction in the current frame, but it can be performed similarly to inter-prediction in which the reference block is retrieved in the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described herein.

[60] The prediction signal generated through the inter prediction module 221 and/or the intra prediction module 222 can be used to generate a reconstructed signal or generate a residual signal. Converter 232 may generate conversion coefficients by applying conversion technology to the residual signal. For example, the transform technology may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). Here, GBT means a graph-derived transformation when the relationship information between pixels is represented by the graph. CNT means the transformation obtained from the prediction signal generated using all the previously reconstructed pixels. In addition, the conversion process can be applied to square pixel blocks that are identical in size, or can be applied to blocks that are variable in size rather than square.

[61] Quantizer 233 may quantize the transform coefficients and transmit them to entropy encoder 240, and entropy encoder 240 may encode the quantized signal (information regarding the quantized transform coefficients) and output a bit stream. Information regarding the quantized transform coefficients may be referred to as "residual information". Quantizer 233 may re-arrange the block quantized transform coefficients into one-dimensional vector form based on the scanning order of the coefficients, and generate information regarding the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form. The entropy encoder 240 may implement various encoding techniques, such as, for example, exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like. Entropy encoder 240 may encode information needed to reconstruct video/images together with or separately from quantized transform coefficients (eg, syntax element values, etc.). Encoded information (eg, encoded video/image information) may be transmitted or stored in Network Abstraction Layer (NAL) units in the form of a bitstream. The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. Herein, the signaling/transmitted information and/or syntax elements that are described below may be encoded through the above-described encoding procedure and included in the bitstream. The bit stream may be transmitted over a network or may be stored on a digital storage medium. Here, the network may include a broadcast network and/or a communications network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD and the like. . A transmitter (not illustrated) transmitting the signal output from the entropy encoder 240 and/or a storage module (not illustrated) storing the signal may be configured as an internal/external element of the encoding equipment 200, and alternatively, the transmitter may be included in the entropy encoder 240.

[62] The quantized transform coefficients output from quantizer 233 can be used to generate a predictive signal. For example, a residual signal (residual block or residual samples) may be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients through dequantizer 234 and inverter 235. Adder 250 adds the reconstructed residual signal with the predicted signal output from prediction module 220 to generate a reconstructed signal (reconstructed frame, reconstructed samples, or array of reconstructed samples). If there is no remainder for the target block to process, for example in the case where the skip mode is applied, the predicted block can be used as the reconstructed block. The generated reconstructed signal can be used to internally predict the next target block for processing in the current frame and can be used to inter-predict the next frame through filtering, as described below.

[63] Meanwhile, luma conversion with chrominance scaling (LMCS) can be applied during the encoding and/or frame restoration process.

[64] Filter 260 may enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, filter 260 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and store the modified reconstructed frame in storage device 270, namely, a DPB of storage device 270. Various filtering techniques may include, for example, deblocking filtering. , sampled adaptive offset (SAO), adaptive loop filter, bilateral filter, etc. The filter 260 may generate various types of filtering-related information and transmit the generated information to the entropy encoder 290, as described below in the description of each filtering method. Information associated with filtering may be encoded by entropy encoder 290 and output in the form of a bit stream.

[65] The modified reconstructed frame transferred to the storage device 270 can be used as a reference frame in the inter-prediction unit 221. When inter-prediction is applied by the encoding equipment, prediction mismatch between the encoding equipment 200 and the decoding equipment can be eliminated, and encoding efficiency can be improved.

[66] The DPB of the storage device 270 may store the modified reconstructed frame for use as a reference frame in the inter-prediction module 221. Memory 270 may store block motion information from which motion information in the current frame is retrieved (or encoded) and/or motion information of already reconstructed blocks in the frame. The stored motion information may be transmitted to the inter-prediction unit 221 to be used as spatial neighbor block motion information or temporal neighbor block motion information. Memory 270 may store reconstructed samples of reconstructed blocks in the current frame and may transmit reconstructed samples to intra prediction module 222.

[67] FIG. 3 is a diagram for schematically explaining the configuration of video/image decoding equipment to which the disclosure of this document may be applied.

[68] Referring to FIG. 3, decoding equipment 300 may include and be configured with an entropy decoder 310, a residual processor 320, a prediction module 330, an adder 340, a filter 350, and a memory 360. The prediction module 330 may include an inter-prediction module 331 and an internal prediction module 332. forecasting. Residual processor 320 may include a dequantizer 321 and an inverse converter 322. Entropy decoder 310, residual processor 320, prediction module 330, adder 340, and filter 350, which are described above, may be configured by one or more hardware components (e.g., chipsets or decoder processors) according to an embodiment. Additionally, storage device 360 may include a decoded frame buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include storage device 360 as an internal/external component.

[69] When a bit stream including video/image information is input, the decoding equipment 300 can reconstruct the image in response to the process in which the video/image information is processed in the encoding equipment illustrated in FIG. 2. For example, decoding equipment 300 may extract units/blocks based on blocking-related information obtained from the bit stream. The decoding equipment 300 may perform decoding using a processing unit applied to the encoding equipment. Therefore, a processing unit for decoding, for example, may be an encoding unit, and the encoding unit may be partitioned according to a quadtree structure, a binary tree structure, and/or a ternary tree structure from an encoding tree unit or a maximum encoding unit. One or more transformation units may be derived from a coding unit. In addition, the reconstructed image signal decoded and output by the decoding equipment 300 can be reproduced by the playback equipment.

[70] The decoding equipment 300 may receive a signal output from the encoding equipment of FIG. 2 in the form of a bitstream, and the received signal may be decoded through entropy decoder 310. For example, entropy decoder 310 may parse the bitstream to extract information (eg, video/image information) needed to reconstruct images (or reconstruct frames). The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The decoding equipment may further decode the frame based on information regarding the parameter set and/or general constraint information. The signaling/receiving information and/or syntax elements described later in this document may be decoded, may be decoded by a decoding procedure, and obtained from the bit stream. For example, entropy decoder 310 decodes information in a bitstream based on an encoding method, such as exponential Golomb, CAVLC, or CABAC encoding, and the output syntax elements required for image reconstruction and the quantized values of the transform coefficients for the remainder. More specifically, the entropy CABAC decoding method may receive a resolution element corresponding to each syntax element in the bitstream, determine a context model by using information of a target decoding syntax element, decoding information of a target decoding block, or information of a symbol/resolution element decoded in a previous stage, and perform arithmetic decoding for the permission element by predicting the probability of occurrence of the permission element according to the determined context model and generating a symbol corresponding to the meaning of each syntactic element. In this case, the entropy CABAC decoding method can update the context model by using the decoded symbol/grant element information for the context model of the next symbol/grant element after determining the context model. Prediction-related information from the information decoded by the entropy decoder 310 may be provided to the prediction module 330, and information regarding the remainder for which entropy decoding is performed in the entropy decoder 310, i.e. quantized conversion coefficients and associated parameter information may be input to the dequantizer 321. In addition, information regarding filtering from information decoded by the entropy decoder 310 may be provided to the filter 350. Meanwhile, a receiver (not illustrated) for receiving a signal output from the equipment encoding may be further configured as an internal/external element of the decoding equipment 300, or the receiver may be a constituent element of the entropy decoder 310. Meanwhile, the decoding equipment herein may be referred to as "video/image/frame decoding equipment", and the decoding equipment can be classified into information decoder (video/image/frame information decoder) and sample decoder (video/image/frame sample decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverter 322, a predictor 330, an adder 340, a filter 350, and a memory 360.

[71] Dequantizer 321 may dequantize the quantized transform coefficients to output transform coefficients. Dequantizer 321 may re-arrange the quantized transform coefficients in two-dimensional block form. In this case, re-arrangement can be performed based on the scanning order of the coefficients performed by the encoding equipment. The dequantizer 321 may perform dequantization on the quantized transform coefficients using a quantization parameter (eg, quantization step size information) and obtain the transform coefficients.

[72] An inverse converter 322 inversely converts the transform coefficients to obtain a residual signal (residual block, residual sample array).

[73] Prediction module 330 may perform prediction on the current block and generate a predicted block including prediction samples of the current block. The prediction module may determine whether intra prediction is applied or inter prediction is applied to the current block based on the prediction information output from the entropy decoder 310, and determine a particular intra/inter prediction mode.

[74] The prediction module may generate a prediction signal based on various prediction methods described below. For example, the prediction module may not only apply intra-prediction or inter-prediction to predict one block, but also simultaneously apply intra-prediction and inter-prediction. This may be referred to as "combined inter-intraprediction (CIIP)". In addition, the prediction module can perform intra-block copying (IBC) to predict a block. In-block copying can be used for encoding content images/game moving images and the like, such as screen content encoding (SCC). IBC essentially performs prediction in the current frame, but it can be performed similarly to inter-prediction in which the reference block is retrieved in the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described herein.

[75] Intra prediction module 332 may predict the current block by reference to samples in the current frame. Samples for reference may be located in the surroundings of the current block or may be located spaced apart from the current block according to a prediction mode. In intraprediction, prediction modes may include a plurality of non-directional modes and a plurality of directed modes. The intra prediction module 332 may determine the prediction mode to be applied to the current block by using the prediction mode applied to the adjacent block.

[76] The inter-prediction unit 331 can extract a predicted block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include information regarding the direction of the inter-prediction (L0-prediction, L1-prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block existing in the current frame and a temporal neighbor block existing in the reference frame. For example, the inter-prediction module 331 may construct a motion information candidate list based on neighboring blocks and extract the current block's motion vector and/or reference frame index based on the received candidate selection information. The inter-prediction may be performed based on various prediction modes, and the information regarding the prediction may include information indicating the inter-prediction mode for the current block.

[77] The adder 340 may generate a reconstructed signal (a reconstructed frame, a reconstructed block, or an array of reconstructed samples) by adding the resulting residual signal with a prediction signal (a predicted block or an array of predicted samples) output from prediction module 330. If there is no remainder for the target processing block, for example in the case where the skip mode is applied, the predicted block can be used as the reconstructed block.

[78] The adder 340 may be referred to as a “recovery module” or a “recovered block generator.” The generated reconstructed signal may be used to internally predict the next block to be processed in the current frame, and, as described below, may also be output by filtering, or may also be used to inter-predict the next frame.

[79] Meanwhile, luma conversion with chrominance scaling (LMCS) can also be applied in the frame decoding process.

[80] Filter 350 may enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, filter 350 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and store the modified reconstructed frame in storage device 360, namely, a DPB of storage device 360. Various filtering techniques may include, for example, deblocking filtering. , sampled adaptive bias, adaptive loop filter, bilateral filter, etc.

[81] The (modified) reconstructed frame stored in the DPB of the storage device 360 can be used as a reference frame in the inter-prediction unit 331. Storage device 360 may store block motion information from which motion information in the current frame is retrieved (or decoded) and/or motion information of already reconstructed blocks in the frame. The stored motion information may be transmitted to the inter-prediction unit 331 so that it is used as spatial neighbor block motion information or temporal neighbor block motion information. Storage device 360 may store reconstructed samples of reconstructed blocks in the current frame and transmit the reconstructed samples to intra prediction module 332.

[82] In the present specification, the embodiments described in prediction module 330, dequantizer 321, inverse converter 322, and filter 350 of decoding equipment 300 may also be applied in an identical manner or corresponding to prediction module 220, dequantizer 234, inverse converter 235, and filter 260 equipment 200 coding.

[83] Meanwhile, as described above, when performing video encoding, prediction is performed in order to improve compression efficiency. Through this, a predicted block including predictive samples for the current block can be generated as a block to be encoded (ie, an encoding target block). Here, the predicted block includes predictive samples in a spatial domain (or pixel domain). The predicted block is extracted identically in the encoding equipment and the decoding equipment, and the encoding equipment can signal information (residual information) regarding the remainder between the original block and the predicted block, rather than the original sample value of the original block, to the decoding equipment, thereby improving efficiency image coding. The decoding equipment may extract a residual block including residual samples based on the residual information, add the residual block and the predicted block to generate reconstructed blocks including the reconstructed samples, and generate a reconstructed frame including the reconstructed blocks.

[84] Residual information can be generated through a transformation and quantization procedure. For example, the encoding equipment may extract a residual block between an original block and a predicted block, perform a transform procedure on the residual samples (an array of residual samples) included in the residual block to extract transform coefficients, perform a quantization procedure on the transform coefficients to extract quantized transform coefficients, and residual information associated with the signals to the decoding equipment (via the bit stream). Here, the residual information may include quantized transform coefficient value information, location information, transform technology, transform kernel, quantization parameter, and the like. The decoding equipment may perform a dequantization/deconversion procedure based on the residual information and extract the residual samples (or residual blocks). The decoding equipment may generate a reconstructed frame based on the predicted block and the residual block. In addition, for reference to inter-picture prediction later, the encoding equipment may also dequantize/de-convert the quantized transform coefficients to extract a residual block and generate a reconstructed frame based on it.

[85] FIG. 4 schematically illustrates the multiple conversion technology according to the present document.

[86] Referring to FIG. 4, the converter may correspond to the converter in the encoding equipment of FIG. 2 as described above, and the inverter may correspond to the inverter in the encoding equipment of FIG. 2 or the inverse converter in the decoding equipment of FIG. 3 as described above.

[87] The converter can extract (primary) transform coefficients by performing a primary transform based on the residual sample (residual sample array) in the residual block (S410). Such a primary transformation may be called a "base transformation". Here, the primary transform may be based on a multiple transform selection (MTS), and in case the multiple transform is applied as the primary transform, it may be called a "multiple base transform".

[88] For example, a multiple base transform may represent a transform method by additionally using discrete cosine transform (DCT) type 2 (DCT-II), discrete sine transform (DST) type 7 (DST-VII), DCT type 8 (DCT- VIII) and/or DST-type 1 (DST-I). Thus, a multiple core transform may represent a transform method for converting a spatial domain residual signal (or residual block) into frequency domain transform coefficients (or primary transform coefficients) based on a plurality of transform kernels selected from DCT-type 2, DST-type 7, DCT-type 8 and DST-type 1. Here, the primary conversion coefficients may be called "converter-side temporary conversion coefficients".

[89] In other words, in the case where an existing conversion method is applied, spatial domain conversion of a residual signal (or residual block) to frequency domain can be applied based on DCT type 2, and transform coefficients can be generated. However, in contrast, in case a multiple base transform is applied, the spatial domain transform for the residual signal (or residual block) to the frequency domain can be applied based on DCT type 2, DST type 7, DCT type 8 and /or DST type 1, and transform coefficients (or primary transform coefficients) can be generated. Here, DCT type 2, DST type 7, DCT type 8 and DST type 1 may be referred to as a “transformation type”, a “transformation core” or a “transformation base”. DCT/DST transformation types can be specified based on basis functions.

[90] In the case where multiple base transforms are performed, a vertical transform kernel and/or a horizontal transform kernel for the target block may be selected from among the transform kernels, a vertical transform for the target block may be performed based on the vertical transform kernel, and a horizontal transform for the target block can be executed based on the horizontal transformation kernel. Here, a horizontal transform may represent a transform for the horizontal components of the target block, and a vertical transform may represent a transform for the vertical components of the target block. The vertical transform kernel/horizontal transform kernel may be adaptively determined based on the prediction mode and/or transform index of the target block (CU or subblock) including the residual block.

[91] Additionally, for example, in the case of performing primary transformation by applying MTS, specific basis functions can be configured as specified values, and in the case of vertical transformation or horizontal transformation, the transformation relationship for the transformation core can be configured by combining which basis functions apply. For example, in the case where the horizontal direction transformation kernel is represented by trTypeHor, and the vertical direction transformation kernel is represented by trTypeVer, trTypeHor or trTypeVer having a value of 0 can be configured as DCT2, and trTypeHor or trTypeVer having a value of 1 can be configured as DCT7 ; trTypeHor or trTypeVer having a value of 2 can be configured as DCT8.

[92] Alternatively, for example, to indicate any of a plurality of sets of transform kernels, the MTS index may be encoded and the MTS index information may be signaled to decoding equipment. Here, the MTS index may be represented as a tu_mts_idx syntax element or an mts_idx syntax element. For example, if the MTS index is 0, it may be represented that the values of trTypeHor and trTypeVer are 0, and (trTypeHor, trTypeVer)=(DCT2, DCT2). If the MTS index is 1, it may be represented that the values of trTypeHor and trTypeVer are 1, and (trTypeHor, trTypeVer)=(DST7, DST7). If the MTS index is 2, it may be represented that the value of trTypeHor is 2, and the value of trTypeVer is 1, and (trTypeHor, trTypeVer)=(DCT8, DST7). If the MTS index is 3, it may be represented that the value of trTypeHor is 1, and the value of trTypeVer is 2, and (trTypeHor, trTypeVer)=(DST7, DCT8). If the MTS index is 4, it may be represented that the values of trTypeHor and trTypeVer are 2, and (trTypeHor, trTypeVer)=(DCT8, DCT8). For example, the set of transformation kernels according to the MTS index can be represented as shown in the following table.

[93] Tab. 1

[94] The converter can extract modified (secondary) conversion coefficients by performing secondary conversion based on the (primary) conversion coefficients (S420). The primary transform may be a transformation from the spatial domain to the frequency domain, and the secondary transform may be a transformation to a more compressive expression by exploiting the correlation that exists between the (primary) transform coefficients.

[95] For example, a secondary transformation may include an inseparable transformation. In this case, the secondary transform may be called a "non-separable secondary transform (NSST)" or a "mode-dependent non-separable secondary transform (MDNSST)". The inseparable secondary transform may represent a transform for generating modified transform coefficients (or secondary transform coefficients) for the residual signal by secondary transforming the (primary) transform coefficients extracted through the primary transform based on the inseparable transform matrix. Here, the vertical transformation and the horizontal transformation may not be applied separately (or independently) with respect to the (primary) transformation coefficients based on the inseparable transformation matrix, but may be applied simultaneously.

[96] In other words, an inseparable secondary transform may represent a transform method for rearranging, for example, two-dimensional signals (transform coefficients) into a one-dimensional signal through a specifically defined direction (for example, row-first direction or column-first direction), without separating the coefficients (primary ) transformation into vertical components and horizontal components, and subsequent generation of modified transformation coefficients (or secondary transformation coefficients) based on the inseparable transformation matrix.

[97] For example, a row-first direction (or order) may represent a layout of an MxN block in a line in first-row-Nth-row order, and a column-first direction (or order) may represent a layout of an MxN block in a line in the order "the first column is the Mth column". Here, M and N can represent the width (W) and height (H) of the block and can be positive integers.

[98] For example, an inseparable secondary transform may be applied to the upper left region of a block consisting of (primary) transform coefficients (hereinafter referred to as a transform coefficient block). For example, if the width (W) and height (H) of the transform coefficient block are equal to or greater than 8, an inseparable secondary 8×8 transform may be applied to the upper left 8×8 zone of the transform coefficient block. Additionally, if the width (W) and height (H) of the transform coefficient block are equal to or greater than 4 and less than 8, an inseparable 4x4 secondary transform can be applied to the upper left min(8, W) x min(8, H) zone of the coefficient block transformations. However, the embodiments are not limited to this, and for example, even if the condition that the width (W) and height (H) of the transform coefficient block are equal to or greater than 4 is satisfied, a non-separable 4x4 secondary transform can be applied to the left upper zone min(8, W) x min(8, H) of the block of conversion coefficients.

[99] In particular, for example, in the case where a 4x4 input block is used, the non-separable secondary transform can be performed as follows.

[100] The X 4x4 input block can be represented as follows.

[101] equation 1

[102] For example, the vector shape of X can be represented as follows.

[103] equation 2

[104] Referring to equation 2, can represent a vector X, and the two-dimensional block X in Equation 1 can be rearranged and represented as a one-dimensional vector according to row-first order.

[105] In this case, the secondary inseparable transform can be calculated as follows.

[106] equation 3

[107] Here, may represent a vector of transform coefficients, and T may represent a 16x16 (non-separable) transform matrix.

[108] Based on Equation 3, , having a size of 16×1, can be removed, and can be reorganized as a 4x4 block via scan order (horizontal, vertical or diagonal). However, the above calculation is exemplary, and in order to reduce the computational complexity of the inseparable secondary transform, hypercubic Givens transform (HyGT), etc. can be used for computing an inseparable secondary transform.

[109] Meanwhile, with an inseparable secondary transform, the transform kernel (either transform base or transform type) can be selected in a mode-dependent manner. Here, the mode may include an intra prediction mode and/or an inter prediction mode.

[110] For example, as described above, NSST may be performed based on an 8×8 transform or a 4×4 transform determined based on the width (W) and height (H) of the transform coefficient block. For example, if W and H are equal to or greater than 8, the 8×8 transform may represent a transform that may be applied to the 8×8 zone included in the corresponding transform coefficient block, and the 8×8 zone may represent the left upper 8×8 zone in the corresponding block of conversion coefficients. Additionally, similarly, if W and H are equal to or greater than 4, the 4x4 transform may represent a transform that can be applied to a 4x4 zone included in the corresponding transform coefficient block, and the 4x4 zone may represent the left upper 4x zone 4 in the corresponding block of conversion coefficients. For example, an 8×8 transform kernel matrix may be a 64×64/16×64 matrix, and a 4×4 transform kernel matrix may be a 16×16/8×16 matrix.

[111] In this case, to select a transformation core based on mode, two non-separable secondary transformation cores per transformation set for the non-separable secondary transformation can be configured with respect to all of the 8×8 transformation and the 4×4 transformation, and four can be provided conversion kit. Thus, four transform sets can be configured with respect to an 8×8 transform, and four transform sets can be configured with respect to a 4×4 transform. In this case, each of the four transform sets for the 8×8 transform may include two 8×8 transform cores, and each of the four transform sets for the 4×4 transform may include two 4×4 transform cores.

[112] However, the size of the translation subunit, the number of sets, and the number of translation cores per set are approximate, and a size other than 8x8 or 4x4 may be used, or n sets may be configured and k translation cores may be included in every set. Here, n and k can be positive integers.

[113] For example, a transformation set may be referred to as an "NSST set" and a transformation core in an NSST set may be referred to as a "NSSAT core". For example, the selection of a particular set of sets for transformation may be performed based on the intra prediction mode of the target block (CU or sub-block).

[114] For example, an intra-prediction mode may include two non-directional or non-angular intra-prediction modes and 65 directional or angular intra-prediction modes. The non-directional intra-prediction modes may include planar intra-prediction mode number 0 and intra-DC prediction mode number 1, and the directed intra-prediction modes may include 65 intra-prediction modes (number 2-66). However, this is exemplary, and the embodiment according to the present document may be applied even to the case in which a different number of intra prediction modes is provided. Meanwhile, in some cases, the intra prediction mode number 67 may be further used, and the intra prediction mode number 67 may represent a linear model (LM) based mode.

[115] FIG. 5 roughly illustrates the internal directional modes in the 65 forecast directions.

[116] Referring to FIG. 5, the modes can be divided into intra-prediction modes having a horizontal direction and intra-prediction modes having a vertical direction with respect to the intra-prediction mode number 34 having a left upper diagonal prediction direction. In fig. 5, H and V may represent horizontal directionality and vertical directionality, respectively, and numbers -32 to 32 may represent offsets in units of 1/32 for a position on the sampling grid. It may represent an offset for the mode index value.

[117] For example, intra prediction modes number 2-33 may have a horizontal orientation, and intra prediction modes number 34-66 may have a vertical orientation. Meanwhile, from a technical point of view, the intra prediction mode number 34 can be considered as having neither horizontal directionality nor vertical directionality, but can be classified as belonging to horizontal directionality in terms of determining the transformation set for secondary transformation. This is because the input data is transposed and used with respect to the vertical directional modes symmetrical with respect to the intra prediction mode number 34, and the arrangement method of the input data for the horizontal directional mode is used with respect to the intra prediction mode number 34. Here, the transposition of the input data may mean an NxM data configuration in such a way that rows become columns and columns become rows relative to two-dimensional MxN block data.

[118] Additionally, the intra prediction mode number 18 and the intra prediction mode number 50 may represent a horizontal intra prediction mode and a vertical intra prediction mode, respectively, and the intra prediction mode number 2 may be called a "right upper diagonal intra prediction mode" since prediction is performed in the upward and right direction with the left reference pixel. In the same context, the intra-prediction mode number 34 may be referred to as the “right lower diagonal intra-prediction mode”, and the intra-prediction mode number 66 may be referred to as the “left lower diagonal intra-prediction mode”.

[119] Meanwhile, if it is determined that a particular set is used for a non-separable transformation, one of the k transformation kernels in the specific set can be selected through the index of the non-separable secondary transformation. For example, the encoding equipment may extract a non-separable secondary transform index representing a particular transform kernel based on a rate-dependent distortion (RD) check, and may signal the non-separable secondary transform index to the decoding equipment. For example, the decoding equipment may select one of the k transform kernels in a particular set based on the inseparable secondary transform index. For example, an NSST index having a value of 0 may represent a first inseparable secondary transform kernel, an NSST index having a value of 1 may represent a second inseparable secondary transform kernel, and an NSST index having a value of 2 may represent a third inseparable secondary transform kernel . Alternatively, an NSST index having a value of 0 may represent that the first non-separable secondary transform is not applied to the target block, and an NSST index having a value of 1-3 may indicate three transform cores, as described above.

[120] The transformer may perform an inseparable secondary transform based on the selected transform kernels and may obtain modified (secondary) transform coefficients. The modified transform coefficients may be extracted as quantized transform coefficients through the above-described quantizer, and may be encoded so as to be signaled to the decoding equipment, and may be transmitted to the dequantizer/inverse converter in the encoding equipment.

[121] Meanwhile, if the secondary transform is omitted as described above, the (primary) transform coefficients, which are the outputs of the primary (separable) transform, can be extracted as quantized transform coefficients through a quantizer as described above, and can be encoded with the ability signaled to the decoding equipment and may be transmitted to the dequantizer/inverter in the encoding equipment.

[122] Referring again to FIG. 4, the inverse converter can perform a sequence of procedures in reverse order with respect to the procedures performed by the above-described converter. The inverse transformer may receive (dequantized) transform coefficients, extract (primary) transform coefficients by performing a secondary (inverse) transform (S450), and obtain a residual block (residual samples) by performing a primary (inverse) transform with respect to the (primary) transform coefficients (S460) . Here, the primary conversion coefficients may be called "modified inverter-side conversion coefficients". As described above, the encoding equipment and/or decoding equipment may generate a reconstructed block based on the residual block and the predicted block, and may generate a reconstructed frame based thereon.

[123] Meanwhile, the decoding equipment may further include a secondary inverse transform application/non-application determining unit (or an element for determining whether or not to apply the secondary inverse transform), and a secondary inverse transform determining unit (or an element for determining secondary inverse transform transformations). For example, the secondary inverse transform application/non-application determination module may determine whether or not the secondary inverse transform should be applied. For example, the secondary inverse transform may be NSST or RST, and the secondary inverse transform application/non-apply determination module can determine whether or not to apply the secondary inverse transform based on the secondary transform flag parsed or obtained from the bit stream. Alternatively, for example, the secondary inverse transform application/non-application determination unit may determine whether or not to apply the secondary inverse transform based on the transform coefficient of the residual block.

[124] The secondary inverse transform determination module may determine the secondary inverse transform. In this case, the secondary inverse transform determining unit may determine the secondary inverse transform applied to the current block based on the set for the NSST (or RST) transform specified in accordance with the intra prediction mode. Alternatively, the method for determining the secondary transformation may be determined depending on the method for determining the primary transformation. Alternatively, various combinations of the primary transform and the secondary transform may be determined in accordance with the intra prediction mode. For example, the secondary inverse transform determination module may determine the area to which the secondary inverse transform is applied based on the size of the current block.

[125] Meanwhile, if the secondary (inverse) transform is omitted as described above, the residual block (residual samples) can be obtained by receiving the (dequantized) transform coefficients and performing the primary (separable) inverse transform. As described above, the encoding equipment and/or decoding equipment may generate a reconstructed block based on the residual block and the predicted block, and may generate a reconstructed frame based thereon.

[126] Meanwhile, herein, in order to reduce the computational amount and required storage space caused by the non-separable secondary transform, a reduced secondary transform (RST) having a reduced size of the transform matrix (kernel) can be applied to the concept of NSST.

[127] As used herein, RST may refer to a (simplified) transformation performed on the residual samples for a target block, based on a transformation matrix whose size is reduced according to a simplification factor. If this is done, the amount of computation required during the transformation may be reduced due to the reduction in the size of the transformation matrix. Thus, RST can be used to resolve the problem of computational complexity encountered during the conversion of a block having a large size or a non-separable conversion.

[128] For example, an RST may be referred to by various terms such as "reduced transform", "reduced secondary transform", "reduced transform", "simplified transform" or "simple transform", and the names by which the RST is called , are not limited to the examples listed. Additionally, RST is mainly performed in the low-frequency region including coefficients that are not equal to 0 in the transform block, and therefore may be called “low-frequency non-separable transform (LFNST)”.

[129] Meanwhile, in the case where the secondary inverse transform is performed based on the RST, the inverse transformer 235 of the encoding equipment 200 and the inverse transformer 322 of the decoding equipment 300 may include an inverse RST module that extracts modified transform coefficients based on the inverse RST for the transform coefficients , and an inverse primary converter that extracts residual samples for the target block based on the inverse primary transform for the modified transform coefficients. Inverse primary transformation means the inverse transformation of the primary transformation applied to the remainder. As used herein, extracting transform coefficients based on a transform may mean extracting transform coefficients by applying an appropriate transform.

[130] FIG. 6 and 7 are diagrams for explaining RST according to an embodiment of the present document.

[131] For example, FIG. 6 may be a drawing explaining that direct shortening is applied, and FIG. 7 may be a drawing for explaining that inverse shortening is applied. As used herein, a target block may represent a current block, a residual block, or a transform block depending on what is being encoded.

[132] For example, in RST, an N-dimensional vector can be transformed into an R-dimensional vector located in another space, and a reduced transformation matrix can be determined. Here, N and R can be positive integers, and R can be less than N. N can mean the square of the length of one side of the block to which the transformation is applied, or the total number of transformation coefficients corresponding to the block to which the transformation is applied, and the simplification factor can mean R/N value. The simplification factor may be referred to by various terms such as "reduced factor", "reduction factor", "simplified factor" or "simple factor". Meanwhile, R may be referred to as a "reduced coefficient", and in some cases, the simplification factor may mean R. Additionally, in some cases, the simplification factor may mean the N/R value.

[133] For example, a simplification factor or a reduced factor may be signaled through the bit stream, but is not limited to it. For example, preset values for the simplification factor or the reduced coefficient may be stored in the encoding equipment 200 and the decoding equipment 300, in which case, the simplification factor or the reduced coefficient may not be signaled separately.

[134] For example, the size (RxN) of the simplified transform matrix may be smaller than the size (NxN) of the regular transform matrix and may be given by the following equation.

[135] equation 4

[136] For example, the matrix T in the short transform block illustrated in FIG. 6 may represent the TRxN matrix of Equation 4. As shown in FIG. 6, in the case that the residual samples for the target block are multiplied by the simplified transform matrix TRxN, the transform coefficients for the target block can be extracted.

[137] For example, in the case where the block size to which the transformation is applied is 8×8, and R is 16 (ie, R/N=16/64=1/4), RST according to FIG. 6 can be expressed by a matrix operation as indicated in Equation 5 below. In this case, the storage and multiplication operation can be reduced to approximately 1/4 by a simplification factor.

[138] As used herein, a matrix operation can be understood as the operation of obtaining a column vector by placing a matrix to the left of the column vector and multiplying the matrix and the column vector.

[139] equation 5

[140] In Equation 5, r1-r64 may represent residual samples for the target block. Alternatively, for example, they may be transform coefficients generated by applying a primary transform. Based on the operation result of Equation 5, the transformation coefficients ci for the target block can be retrieved.

[141] For example, in the case where R is 16, transform coefficients c1-c16 for the target block can be extracted. If a transform matrix having a size of 64x64 (NxN), through the application of a regular transform rather than an RST, is multiplied by residual samples having a size of 64x1 (Nx1), 64 (N) transform coefficients for the target block can be extracted, but since RST is applied, only 16 (N) transform coefficients for the target block can be extracted. As the total number of transform coefficients for a target block is reduced from N to R, the amount of data that the encoding equipment 200 transmits to the decoding equipment 300 can be reduced, and thereby the transmission efficiency between the encoding equipment 200 and the decoding equipment 300 can be improved.

[142] Considering the size of the transformation matrix, since the size of the regular transformation matrix is 64×64 (NxN), and the size of the simplified transformation matrix is reduced to 16×64 (RxN), the storage usage when performing RST can be reduced in the ratio R/N by compared to the case in which the regular conversion is performed. Additionally, compared with the number (NxN) of multiplication operations when using a regular transformation matrix, using a simplified transformation matrix can reduce the number of (RxN) multiplication operations in the ratio R/N.

[143] In an embodiment, transformer 232 of encoding equipment 200 may extract transform coefficients for a target block by performing a primary transform and a secondary RST-based transform on the residual samples for the target block. The transform coefficients may be transmitted to the inverse transformer of the decoding equipment 300, and the inverse transformer 322 of the decoding equipment 300 may extract the modified transform coefficients based on the inverse reduced secondary transform (RST) for the transform coefficients, and may extract the residual samples for the target block based on the inverse primary transform of the modified coefficients. transformations.

[144] The size of the inverse RST matrix TNxR according to an embodiment may be NxR, which is smaller than the NxN size of the regular inverse transform matrix, and may have a transposed relationship with the simplified transform TRxN matrix illustrated in Equation 4.

[145] The Tt matrix in the reduced inverse transform block illustrated in FIG. 7 may represent the inverse RST matrix TRxNT. Here, the superscript T may represent transpose. As shown in FIG. 7, in the case that the transform coefficients for the target block are multiplied by the inverse RST matrix TRxNT, the modified transform coefficients for the target block or residual samples for the target block can be extracted. The inverse RST matrix TRxNT can be expressed as (TRxN)TNxR.

[146] More specifically, in the case where the inverse RST is used as a secondary inverse transform, the modified transform coefficients for the target block can be extracted by multiplying the transform coefficients for the target block by the inverse RST matrix TRxNT. Meanwhile, the inverse RST can be used as the inverse primary transform, in which case, the residual samples for the target block can be extracted when the inverse RST matrix TRxNT is multiplied by the transform coefficients for the target block.

[147] In the embodiment, in case the block size to which the inverse transform is applied is 8×8, and R is 16 (ie, R/N=16/64=1/4), RST according to FIG. 7 can be expressed through a matrix operation as indicated in Equation 6 below.

[148] equation 6

[149] In Equation 6, c1-c16 may represent conversion coefficients for the target block; rj representing the modified transform coefficients for the target block or residual samples for the target block may be extracted based on the operation result of Equation 6. Thus, r1-rN representing the modified transform coefficients for the target block or residual samples for the target block may be extracted.

[150] Considering the size of the inverse transform matrix, since the size of the regular inverse transform matrix is 64×64 (NxN), and the size of the simplified inverse transform matrix is reduced to 64×16 (NxR), the memory usage when performing the inverse RST can be reduced in the ratio R/N compared to the case in which the regular inverse transform is performed. Additionally, compared with the number (NxN) of multiplication operations when using a regular inverse transform matrix, using a simplified inverse transform matrix can reduce the number (NxR) of multiplication operations in the ratio R/N.

[151] Meanwhile, conversion sets can be configured and applied even relative to RST 8x8. Thus, the corresponding 8x8 RST can be applied according to the conversion set. Since one transformation set consists of two or three transformation kernels in accordance with the intra prediction mode, it can be configured to select one of four transformations, which at most includes even the case in which the secondary transformation is not applied. In a transformation where a secondary transformation is not applied, the identity matrix can be considered to be applied. If it is intended that index 0, 1, 2, or 3 be specified for four transformations (for example, index number 0 may be allocated to the case in which the identity matrix, i.e., the secondary transformation, is not applied), the transformation that should be applied may be indicated by signaling a syntax element that represents the NSST index to each transform coefficient block. Thus, through the NSST index, NSST 8x8 can be designated for the top left block of 8x8, and in the RST configuration, RST 8x8 can be designated. NSST 8x8 and RST 8x8 may represent transformations that can be applied to an 8x8 zone included in the corresponding block of transform coefficients in the event that the W and H of the target block that becomes the target of the transformation are equal to or greater than 8, and the zone is 8 ×8 may represent the upper left 8×8 zone in the corresponding transform coefficient block. Likewise, NSST 4×4 and RST 4×4 may represent transformations that can be applied to a 4×4 zone included in the corresponding block of transform coefficients in case the W and H of the target block are equal to or greater than 4, and the 4×4 zone may represent is the upper left 4x4 zone in the corresponding block of transformation coefficients.

[152] Meanwhile, for example, the encoding equipment may extract the bit stream by encoding the syntax element value or quantized transform coefficient values for the remainder based on various encoding methods such as exponential Golomb code, context-adaptive variable-length coding (CAVLC), context-adaptive variable-length coding (CAVLC), adaptive binary arithmetic coding (CABAC), etc. Additionally, the decoding equipment may extract the syntax element value or quantized transform coefficient values for the remainder based on various encoding methods such as exponential Golomb encoding, CAVLC, CABAC, and the like.

[153] For example, the above-described encoding methods can be performed as content that is described below.

[154] FIG. 8 approximately illustrates context-adaptive binary arithmetic coding (CABAC) for encoding a syntactic element.

[155] For example, in a CABAC encoding process, if the input signal is a syntax element that is not a binary value, the value of the input signal may be converted to a binary value through binarization. Additionally, if the input signal is already a binary value (ie, if the input signal value is a binary value), binarization may not be performed and the input signal may be used as is. Here, each binary number 0 or 1 constituting the binary value may be called a "permission element". For example, if the binary string, when converted to binary form, is 110, each of the 1s, 1s, and 0s can be represented as one resolution element. The permission element(s) for one syntax element may represent the value of the syntax element. The binarization may be based on a different binarization method, such as a truncated Rice code binarization process or a fixed-length code binarization process, and the binarization method for the target syntax element may be preset. The binarization procedure may be performed by a binarization module in the entropy encoder.

[156] Thereafter, the binarized syntax element sample elements can be input to the regular encoding engine or the bypass encoding engine. The regular encoding engine of the encoding equipment may extract a context model that reflects a probability value relative to a corresponding resolution element, and encode the corresponding resolution element based on the extracted context model. The regular encoding mechanism of the encoding equipment may update the context model for the corresponding permission element after encoding has been performed regarding the corresponding permission elements. The permission elements encoded as the above-described content may be represented as context-encoded permission elements.

[157] Meanwhile, in case the binarized syntax element sample elements are input into the bypass encoding mechanism, they can be encoded as follows. For example, the encoding hardware's bypass encoding mechanism may omit a procedure for estimating a probability relative to an input sample element and a procedure for updating the probability model applied to the resolution element after encoding. In case bypass encoding is used, the encoding equipment can encode the input sample element by applying a regular probability distribution instead of extracting the context model, and thereby the encoding speed can be improved. The permission element encoded as the above-described content may be represented as a bypass permission element.

[158] Entropy decoding may be a process for performing a process identical to the entropy encoding described above, in reverse order.

[159] Decoding equipment (entropy decoder) can decode encoded image/video information. The image/video information may include segmentation-related information, prediction-related information (eg, inter/intra prediction separation information, intra-prediction mode information, inter-prediction mode information, etc.), residual information, or in-loop filtering-related information. information or may include various syntactic elements from it. Entropy encoding can be performed in units of a syntactic element.

[160] Decoding equipment may perform binarization of target syntax elements. Here, the binarization may be based on various binarization methods such as a truncated Rice code binarization process or a fixed-length code binarization process, and the binarization method for the target syntactic element may be previously given The decoding hardware may extract available permission element strings (possible permission element strings) for available target syntax element values through a binarization procedure. The binarization procedure may be performed by a binarization module in the entropy decoder.

[161] The decoding equipment may compare the extracted permission element string with the available permission element strings for the corresponding syntax elements while sequentially decoding or parsing the corresponding permission elements for target syntax elements from the input bit(s) in the bitstream. If the retrieved permission element string is equal to one of the available permission element strings, the value corresponding to the corresponding permission element string is retrieved as the value of the corresponding syntax element. If not, the decoding equipment may repeat the above procedure after further parsing the next bit in the bitstream. Through such a process, it is possible to signal a specific information (or a specific syntax element) in a bit stream using a variable length bit even without using a start bit or a stop bit of the corresponding information. Through this, a relatively smaller bit can be allocated with a relatively smaller value, and thereby the overall encoding efficiency can be improved.

[162] The decoding equipment may perform decoding based on context models or bypass decoding of corresponding bins in a string of bins from the bitstream based on an entropy coding technology such as CABAC or CAVLC.

[163] In the case where a syntax element is decoded based on a context model, the decoding equipment may receive a resolution element corresponding to the syntax element via a bit stream, may determine the context model by using the syntax element and decoding the information of a target decoding block or neighboring block or symbol information / of the permission elements decoded in the previous stage, and can extract the value of the syntactic element by performing arithmetic decoding of the permission element by predicting the probability of occurrence of the received permission element in accordance with the determined context model. Thereafter, the context model of the next permission element to be decoded can be updated based on the determined context model.

[164] The context model may be allocated and updated via context-encoded (regularly encoded) permission elements, and the context model may be indicated based on a context index (ctxIdx) or a context index increment (ctxInc). CtxIdx can be retrieved based on ctxInc. In particular, for example, ctxIdx representing the context model for each of the regularly encoded resolution elements may be retrieved by the sum of ctxInc and the context index offset (ctxIdxOffset). For example, ctxInc may be retrieved differently through permission elements. ctxIdxOffset can be represented as the smallest value of ctxIdx. Typically, ctxIdxOffset may be a value used to distinguish it from context models for other syntax elements, and the context model for one syntax element may be shared or retrieved based on ctxInc.

[165] In the entropy encoding procedure, it can be determined whether encoding should be performed through a regular encoding mechanism or encoding should be performed through a bypass encoding mechanism, and accordingly, the encoding path can be switched. Entropy decoding can perform the same process as entropy encoding in reverse order.

[166] Meanwhile, for example, in the case where bypass decoding of a syntax element is performed, the decoding equipment may receive a resolution element corresponding to the syntax element via a bitstream, and can decode the input sampling element by applying a regular probability distribution. In this case, the decoding equipment may omit the syntax element context model retrieval procedure and the context model updating procedure applied to the permission element after decoding.

[167] Meanwhile, an embodiment of the present document may propose a scheme for signaling an MTS index. Here, the MTS index, as described above, may represent any of a plurality of sets of transform cores, and the MTS index may be encoded, and the MTS index information may be signaled to the decoding equipment. The decoding equipment may obtain the MTS index by decoding the MTS index information, and may determine a set of transform kernels to be applied based on the MTS index. The MTS index can be represented as a tu_mts_idx syntax element or an mts_idx syntax element. For example, the MTS index may be converted to binary form using zero order Rice-Golomb parameters, but may be converted to binary form using a truncated Rice code. In the case of truncated Rice code binarization, the input parameter cMax may have a value of 4 and cRiceParam may have a value of 0. For example, encoding equipment may extract the resolution element(s) for the MTS index by binarizing the MTS index , extract the MTS index information (bit(s) for the MTS index) by encoding the extracted resolution element(s), and signal the MTS index information to the decoding equipment. The decoding equipment may extract the MTS index grant element(s) by decoding the MTS index information and extract the MTS index by comparing the extracted MTS index grant element(s) with candidate MTS index grant elements.

[168] For example, an MTS index (eg, a tu_mts_idx syntax element or an mts_idx syntax element) may be context-encoded based on a context model or context index relative to all permission elements. In this case, the context index increment (ctxInc) for the context encoding MTS index or ctxInc according to the location of the permission element may be allocated or determined as specified in Table 2. Additionally, the context model may be selected according to the location of the permission element as specified in the table 2.

[169] Tab. 2

[170] Referring to Table 2, the ctxInc for permission element number 0 (the first permission element) may be allocated based on cqtDepth. Here, cqtDepth can represent the quadtree depth for the current block and can be retrieved as a single value of 0-5. Thus, ctxInc for the null resolution element can be allocated one value of 0-5 according to cqtDepth. Additionally, the ctxInc for permission element number 1 (the second permission element) may be allocated 6, the ctxInc for permission element number 2 (the third permission element) may be allocated 7, and the ctxInc for permission element number 3 (the fourth permission element) may be allocated 8. Thus , permission element number 0 - permission element number 3 can be allocated ctxInc having different values. Here, different values of ctxInc may represent different context models, and in this case, 9 context models for encoding the MTS index may be provided.

[171] Additionally, for example, the MTS index (e.g., tu_mts_idx syntax element or mts_idx syntax element) may be bypass encoded with respect to all resolution elements as specified in Table 3. In this case, 0 context models for encoding the MTS index may be provided.

[172] Tab. 3

[173] Additionally, for example, an MTS index (e.g., a tu_mts_idx syntax element or an mts_idx syntax element) may be context-encoded based on a context model or context index relative to permission element number 0 (the first permission element) as specified in Table 4, and may undergo bypass coding with respect to the remaining resolution elements. Thus, ctxInc for permission element number 0 (the first permission element) may be allocated 0. In this case, one context model for encoding the MTS index may be provided.

[174] Tab. 4

[175] Additionally, for example, an MTS index (e.g., a tu_mts_idx syntax element or an mts_idx syntax element) may be context-encoded based on a context model or context index relative to permission element number 0 (the first permission element) and permission element number 1 (the second permission element ), as indicated in Table 5, and may be bypassed with respect to the remaining resolution elements. Thus, the ctxInc for permission element number 0 (the first permission element) may be allocated 0, and the ctxInc for permission element number 1 (the second permission element) may be allocated 1. In this case, 2 context models for encoding the MTS index may be provided.

[176] Tab. 5

[177] Additionally, for example, an MTS index (e.g., a tu_mts_idx syntax element or a mts_idx syntax element) may be context-encoded based on a context model or context index relative to all permission elements as specified in Table 6, and may allocate one ctxInc relative to each permission element. Thus, the ctxInc for permission element number 0 (the first permission element) may be allocated 0, and the ctxInc for permission element number 1 (the second permission element) may be allocated 1; The ctxInc for permission element number 2 (the third permission element) may be allocated 2, and the ctxInc for permission element number 3 (the fourth permission element) may be allocated 2. In this case, 4 context models for encoding the MTS index may be provided.

[178] Tab. 6

[179] As described above, according to an embodiment, since the number of context models is reduced by applying bypass encoding to all or some of the MTS index resolution elements or applying a specific value to ctxInc, although context encoding is applied, the complexity may be reduced and the output volume of the decoder may increase. Additionally, in an embodiment, when using context models as described above, the initial value and/or multi-window size may be variable based on occurrence statistics for the position of each resolution element.

[180] FIG. 9 and 10 schematically illustrate a video/image encoding method and an example of related components according to embodiment(s) of the present document.

[181] The method disclosed in FIG. 9 may be performed by the encoding equipment disclosed in FIG. 2 or fig. 10. In particular, for example, S900-S920 of FIG. 9 may be performed by the residual encoding equipment processor 230 of FIG. 10, and S930 of FIG. 9 may be performed by the entropy encoder 240 of the encoding equipment of FIG. 10. Additionally, although not illustrated in FIG. 9, prediction samples or prediction-related information may be retrieved by the encoding equipment prediction unit 220 of FIG. 10, residual information may be extracted from the original samples or prediction samples by the residual processor 230 of the encoding equipment, and a bitstream may be generated from the residual information or prediction-related information by the entropy encoder 240 of the encoding equipment. The method disclosed in FIG. 9 may include the embodiments described above herein.

[182] Referring to FIG. 9, the encoding equipment extracts residual samples for the current block (S900). For example, the encoding equipment may extract residual samples based on the predictive samples and the original samples. Although not illustrated in FIG. 9, to generate prediction samples for the current block, the encoding equipment can perform intra prediction or inter prediction with respect to the current block taking into account the rate-dependent distortion cost function RD, and can generate prediction related information including prediction mode/type information .

[183] The encoding equipment extracts transform coefficients for the current block based on the residual samples (S910). For example, encoding equipment may extract transform coefficients by performing a transform on the residual samples. Here, the transformation may be performed based on a transformation kernel or a set of transformation kernels. For example, a set of transformation kernels may include a transformation kernel in the horizontal direction and a transformation kernel in the vertical direction. For example, encoding equipment may extract transform coefficients by performing a primary transform on the residual samples. Additionally, for example, the encoding equipment may extract temporal transform coefficients by performing a primary transform on the residual samples, and may extract transform coefficients by performing a secondary transform on the temporal transform coefficients. For example, a transformation that is performed based on a set of transformation kernels may be represented as a primary transformation.

[184] The encoding equipment generates the MTS index and residual information based on the transform coefficients (S920). In other words, the encoding equipment may generate the MTS index and/or residual information based on the transform coefficients.

[185] The MTS index may represent a set of transform kernels applied to the current block from among possible sets of transform kernels. Here, the MTS index may be represented as a tu_mts_idx syntax element or an mts_idx syntax element. As described above, the set of transformation kernels may include a transformation kernel in the horizontal direction and a transformation kernel in the vertical direction, and the transformation kernel in the horizontal direction may be represented by trTypeHor, and the transformation kernel in the vertical direction may be represented by trTypeVer.

[186] For example, the values trTypeHor and trTypeVer may be represented by a horizontal transformation kernel and a vertical transformation kernel applied to the current block, and the MTS index may be represented as one of the options including 0- 4, through the values trTypeHor and trTypeVer.

[187] For example, if the MTS index is 0, it may represent that trTypeHor and trTypeVer are equal to 0. Additionally, if the MTS index is 1, it may represent that trTypeHor and trTypeVer are equal to 1. Additionally, if the MTS index is 2, it can represent that trTypeHor is 2 and trTypeVer is 1. If the MTS index is 3, it can represent that trTypeHor is 1 and trTypeVer is 1. Additionally, if the MTS index is 4, it can represent that trTypeHor and trTypeVer are equal to 2. For example, if the value of trTypeHor or trTypeVer is 0, it can represent that DCT2 is applied to (the transform coefficients of) the current block in the horizontal direction or in the vertical direction, if 1, it can represent that DST7 is applied, and if 2, it may represent that DCT8 is applied. Thus, each of the transformation kernel to be applied in the horizontal direction and the transformation kernel to be applied in the vertical direction can be represented as one of the candidates including DCT2, DST7 and DCT8 based on the MTS index.

[188] The MTS index may be represented based on the permission elements from the MTS index permission element string. In other words, the MTS index can be converted to binary form and represented as (resolution elements) from the MTS index permission element string, and (resolution elements) from the MTS index permission element string can be entropy encoded.

[189] In other words, at least one of the permission elements from the MTS index permission element string may be represented based on context encoding. Here, context encoding may be performed based on the context index increment value (ctxInc). Additionally, context encoding may be performed based on the context index (ctxIdx) or the context model. Here, the context index may be represented based on the increment value of the context index. Additionally, the context index may be represented based on the value of the context index increment and the context index offset (ctxIdxOffset).

[190] For example, all permission elements from a string of permission elements in an MTS index may be represented based on context encoding. For example, the ctxInc for the first permission element or the zero permission element (permission element 0) of the permission elements from the MTS index permission element row may be represented based on cqtDepth. Here, cqtDepth can represent the depth of the quadtree for the current block and can be represented as a single value from 0-5. Additionally, ctxInc for the second permission element or permission element number 1 (permission element 1) may be represented as 6, ctxInc for the third permission element or permission element number 2 (permission element 2) may be represented as 7, and ctxInc for the fourth permission element or element permission number 3 (permission element 3) may be represented as 8. Additionally, for example, ctxInc for the first permission element or permission element zero (permission element 0) of the permission elements from the MTS index permission element row may be represented as 0, ctxInc for the second element permission or permission element number 1 (permission element 1) may be represented as 1, ctxInc for the third permission element or permission element number 2 (permission element 2) may be represented as 2, and ctxInc for the fourth permission element or permission element number 3 (permission element 3 permissions) may be represented as 3. Thus, the number of context index increment values that can be used to context encode the first permission element of the permission elements from a string of permission elements may be one.

[191] Additionally, for example, some permission elements from a string of MTS index permission elements may be represented based on context encoding, and the remaining ones may be represented based on bypass encoding. For example, ctxInc for the first permission element or permission element number 0 (permission element 0) of the permission elements from the MTS index permission element row may be represented as 0, and the remaining permission elements may be represented based on bypass encoding. Additionally, for example, ctxInc for the first permission element or permission element number 0 (permission element 0) of the permission elements from the MTS index permission element row may be represented as 0, ctxInc for the second permission element or permission element number 1 (permission element 1) may be represented as 1, and the remaining resolution elements may be represented based on bypass encoding. Thus, the number of context index increment values that can be used to context encode the first permission element of the permission elements from a string of permission elements may be one.

[192] Additionally, all permission elements from the MTS index permission element string may be represented based on bypass encoding. Here, bypass encoding may represent performing context encoding based on a uniform probability distribution, and since the context encoding update procedure is omitted, encoding efficiency can be improved.

[193] Residual information may represent information that is used to extract residual samples. Additionally, for example, the encoding equipment may perform quantization on the transform coefficients, and the residual information may include information regarding residual samples, transform-related information, and/or quantization-related information. For example, the residual information may include information regarding quantized transform coefficients.

[194] The encoding equipment encodes image information including the MTS index and residual information (S930). For example, the image information may further include prediction-related information. For example, encoding equipment may generate a bitstream by encoding image information. Here, the bit stream may be called "encoded information (images)".

[195] Additionally, although not illustrated in FIG. 9, for example, the encoding equipment may generate reconstructed samples based on residual samples and predictive samples. Additionally, the reconstructed block and the reconstructed frame can be retrieved based on the reconstructed samples.

[196] For example, encoding equipment may generate bitstream information or encoded information by encoding image information including all or parts of the above-described pieces of information (or syntax elements). Additionally, the information may be output in the form of a bit stream. Additionally, the bit stream or encoded information may be transmitted to the decoding equipment via a network or storage medium. Further, the bitstream or encoded information may be stored in a computer-readable storage medium, and the bitstream or encoded information may be generated by the above-described image encoding method.

[197] FIG. 11 and 12 schematically illustrate a video/image decoding method and an example of related components according to embodiment(s) of the present document.

[198] The method disclosed in FIG. 11 may be performed by the decoding equipment disclosed in FIG. 3 or fig. 12. In particular, for example, S1100 of FIG. 11 may be performed by the entropy decoder 310 of the decoding equipment of FIG. 12, and S1110 and S1120 of FIG. 11 may be performed by the residual decoding equipment processor 320 of FIG. 12. Additionally, although not illustrated in FIG. 11, prediction-related information or residual information can be extracted from the bit stream by the entropy decoder 310 of the decoding equipment of FIG. 12, and residual samples can be extracted from the residual information by the residual processor 320 of the decoding equipment. Prediction samples may be extracted from the prediction-related information by the decoding equipment prediction unit 330, and a reconstructed block or reconstructed frame may be extracted from the residual samples or prediction samples by the decoding equipment adder 340. The method disclosed in FIG. 11 may include the embodiments described above herein.

[199] Referring to FIG. 11, the decoding equipment obtains the MTS index and residual information from the bit stream (S1100). For example, decoding equipment may obtain the MTS index and/or residual information by parsing or decoding the bit stream. Here, the bit stream may be called "encoded information (images)".

[200] The MTS index may represent a set of transformation kernels to be applied to the current block from among the possible sets of transformation kernels. Here, the MTS index may be represented as a tu_mts_idx syntax element or an mts_idx syntax element. Additionally, the set of transformation kernels may include a transformation kernel that is to be applied to the current block in the horizontal direction and a transformation kernel that is to be applied to the current block in the vertical direction. Here, a transformation kernel that should be applied in the horizontal direction can be represented as trTypeHor, and a transformation kernel that should be applied in the vertical direction can be represented as trTypeVer.

[201] For example, the MTS index may be retrieved as one of the options including 0-4, and according to the MTS index, each of trTypeHor and trTypeVer may be retrieved as one of 0-2. For example, if the MTS index is 0, trTypeHor and trTypeVer may be 0. Additionally, if the MTS index is 1, trTypeHor and trTypeVer may be 1. Additionally, if the MTS index is 2, trTypeHor may be 2, and trTypeVer may be equal to 1. If the MTS index is equal to 3, trTypeHor may be equal to 1, and trTypeVer may be equal to 1. Additionally, if the MTS index is equal to 4, trTypeHor and trTypeVer may be equal to 2. For example, the value of trTypeHor or trTypeVer may be equal to represent the transformation kernel, and if 0 it can represent DCT2, if 1 it can represent DST7, and if 2 it can represent DCT8. Thus, each of the transformation kernel to be applied in the horizontal direction and the transformation kernel to be applied in the vertical direction can be extracted as one of the candidates including DCT2, DST7 and DCT8 based on the MTS index.

[202] The MTS index may be retrieved based on the permission elements from the MTS index permission element string. In other words, the MTS index information can be entropy decoded and can be retrieved as a binarized MTS index, and the binarized MTS index can be represented as (resolution elements) from a string of MTS index resolution elements.

[203] In other words, at least one of the permission elements from the MTS index permission element string may be retrieved based on context encoding. Here, context encoding may be performed based on the context index increment value (ctxInc). Additionally, context encoding may be performed based on the context index (ctxIdx) or the context model. Here, the context index may be retrieved based on the increment value of the context index. Additionally, the context index may be retrieved based on the context index increment value and the context index offset value (ctxIdxOffset).

[204] For example, all permission elements from the MTS index permission element string may be retrieved based on the encoding context. For example, the ctxInc for the first permission element or the zero permission element (permission element 0) of the permission elements from the MTS index permission element row may be allocated based on cqtDepth. Here, cqtDepth can represent the quadtree depth for the current block and can be retrieved as a single value from 0-5. Additionally, the ctxInc for the second permission element or permission element number 1 (permission element 1) may be allocated 6, the ctxInc for the third permission element or permission element number 2 (permission element 2) may be allocated 7, and the ctxInc for the fourth permission element or permission element number 3 (permission element 3) may be allocated 8. Additionally, for example, ctxInc for the first permission element or permission element zero (permission element 0) of the permission elements from the MTS index permission element row may be allocated 0, ctxInc for the second permission element or permission element number 1 (permission element 1) may be allocated 1, ctxInc for the third permission element or permission element number 2 (permission element 2) may be allocated 2, and ctxInc for the fourth permission element or permission element number 3 (permission element 3) may be allocated 3. Thus, the number of context index increment values that can be used to context encode the first permission element of the permission elements from a string of permission elements may be one.

[205] Additionally, for example, some permission elements from the MTS index permission element string may be extracted based on context encoding, and the remainder may be extracted based on bypass encoding. For example, ctxInc for the first permission element or permission element number 0 (permission element 0) of the permission elements from the MTS index permission element row may be allocated 0, and the remaining permission elements may be retrieved based on bypass encoding. Additionally, for example, ctxInc for the first permission element or permission element number 0 (permission element 0) of the permission elements from the MTS index permission element row may be allocated 0, ctxInc for the second permission element or permission element number 1 (permission element 1) may be allocated 1, and the remaining resolution elements can be extracted based on bypass encoding. Thus, the number of context index increment values that can be used to context encode the first permission element of the permission elements from a string of permission elements may be one.

[206] Additionally, all permission elements from the MTS index permission element string may be retrieved based on bypass encoding. Here, bypass encoding may represent performing context encoding based on a uniform probability distribution, and since the context encoding update procedure is omitted, encoding efficiency can be improved.

[207] Residual information may represent information that is used to extract residual samples, and may include information regarding residual samples, deconversion-related information, and/or dequantization-related information. For example, the residual information may include information regarding quantized transform coefficients.

[208] The decoding equipment extracts transform coefficients for the current block based on the residual information (S1110). For example, the decoding equipment may extract quantized transform coefficients for the current block based on information regarding the quantized transform coefficients included in the residual information. For example, decoding equipment may extract transform coefficients for the current block by performing dequantization with respect to the quantized transform coefficients.

[209] The decoding equipment generates residual samples of the current block based on the MTS index and transform coefficients (S1120). For example, residual samples may be generated based on a set of transform kernels represented by transform coefficients and an MTS index. Thus, the decoding equipment can generate residual samples from the transform coefficients through an inverse transform using a set of transform kernels represented by the MTS index. Here, an inverse transform using a set of transform kernels represented by an MTS index may be included in the primary inverse transform. Additionally, the decoding equipment may use not only the primary inverse transform, but also the secondary inverse transform when generating residual samples from the transform coefficients. In this case, the decoding equipment can extract the modified transform coefficients by performing a secondary inverse transform on the transform coefficients, and can generate residual samples by performing a primary inverse transform on the modified transform coefficients.

[210] Although not illustrated in FIG. 11, for example, the decoding equipment may obtain prediction-related information including prediction mode/type from the bit stream, and may generate prediction samples for the current block by performing intra prediction or inter prediction based on the prediction mode/type information. Additionally, for example, the decoding equipment may generate reconstructed samples based on the predictive samples and residual samples. Additionally, for example, a reconstructed block or a reconstructed frame may be extracted based on the reconstructed samples.

[211] For example, decoding equipment may obtain image information including all or parts of the above-described pieces of information (or syntax elements) by decoding a bit stream or encoded information. Additionally, the bit stream or encoded information may be stored on a computer-readable storage medium and may cause the above-described decoding method to be performed.

[212] Although the methods are described on the basis of a flowchart in which steps or blocks are listed in sequence in the above-described embodiments, the steps herein are not limited to a particular order, and a certain step may be performed in another step or in a different order or simultaneously regarding what is described above. Additionally, those skilled in the art will appreciate that steps in the flowcharts are not the only possible steps, and another step may be included or one or more steps in the flowchart may be deleted without affecting the scope of the present disclosure.

[213] The above method according to the present disclosure may be implemented in the form of software, and the encoding equipment and/or decoding equipment according to the present disclosure may be included in an apparatus for performing image processing, for example, a television, a computer, a smartphone, a set-top box, a display device, and etc.

[214] When embodiments of the present disclosure are implemented by software, the above method may be implemented by a module (process or function) that performs the above function. The module may be stored in a storage device and executed by a processor. The storage device may be mounted inside or outside the processor and may be coupled to the processor through various known means. The processor may include an application specific integrated circuit (ASIC), other chipsets, logic circuitry, and/or data processing apparatus. The storage device may include read-only memory (ROM), random access memory (RAM), flash memory, memory card, storage medium, and/or other data storage device. In other words, embodiments of the present disclosure may be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional modules illustrated in the respective drawings may be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, implementation information (eg, information regarding instructions) or algorithms may be stored on a digital storage medium.

[215] In addition, the decoding equipment and encoding equipment to which the embodiment(s) of this document applies may be included in a broadcast multimedia transceiver device, a mobile communication terminal, a home theater system video device, a digital cinema system video device, a surveillance camera, video chat device and real-time communication device, e.g., video communication device, mobile streaming device, storage medium, video recording camera, video-on-demand (VoD) service provider, video-over-the-top (OTT) device, service provider Internet streaming device, 3D video device, virtual reality (VR) style device, augmented reality (AR) style device, telephone video device, in-vehicle terminal (e.g., in-vehicle terminal (including autonomous vehicle), aircraft terminal or ship terminal) and medical video device; and can be used to process an image signal or data. For example, an OTT video device may include a gaming console, Blu-ray player, Internet-enabled television, home theater system, smartphone, tablet PC, and digital video recorder (DVR).

[216] In addition, the processing method to which the embodiment(s) of the present document is applied may be formed in the form of a program executed by a computer and may be stored on a computer-readable recording medium. Multimedia data having a data structure according to the embodiment(s) of the present document may also be stored on a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices on which computer-readable data is stored. The computer-readable recording medium may include, for example, a Blu-ray disc (BD), a universal serial bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical device. data storage. The computer-readable recording medium also includes media implemented in the form of a carrier wave (eg, Internet transmissions). In addition, the bitstream generated by the encoding method may be stored in a computer-readable recording medium or transmitted through a wired or wireless communication network.

[217] In addition, the embodiment(s) of the present document may be implemented as a computer program product based on program code, and the program code may be executed on a computer according to the embodiment(s) of the present document. The program code may be stored on a computer-readable medium.

[218] FIG. 13 illustrates an example of a content streaming system to which the embodiments disclosed herein are applicable.

[219] Referring to FIG. 13, a content streaming system to which the embodiments disclosed herein are applicable may generally include an encoding server, a streaming server, a web server, a media storage device, a user device, and a media input device.

[220] The encoding server functions to digitally compress content input from media input devices such as a smartphone, camera, video recorder, and the like to form a bitstream and transmit it to the streaming server. As another example, in case a media input device such as a smartphone, camera, video recorder, etc. directly generates a bitstream, the encoding server may be omitted.

[221] The bitstream may be generated by an encoding method or a bitstream generation method to which embodiments of this document apply. In addition, the streaming server may temporarily store the bitstream while transmitting or receiving the bitstream.

[222] The streaming server transmits multimedia data to the subscriber device based on the user's request through a web server, which functions as a tool that informs the user as to what service is provided. When a user requests a service that the user wants, the web server passes the request to the streaming server and the streaming server streams the media data to the user. In this regard, the content streaming system may include a separate control server, in which case, the control server functions to manage commands/responses between associated equipment in the content streaming system.

[223] The streaming server may receive content from a media storage and/or encoding server. For example, if the content is received from an encoding server, the content may be received in real time. In this case, the streaming server can store the bit stream for a predetermined period of time to smoothly provide the streaming service.

[224] For example, a subscriber device may include a mobile phone, a smartphone, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable media player (PMP), a navigation device, a tablet PC, a tablet PC, an ultrabook, a wearable device (for example, a watch terminal (smart watch), a glasses terminal (smart glasses), a head-mounted display (HMD)), a digital TV, a desktop computer, a digital signage system, etc.

[225] Each of the servers in the content streaming system may operate as a distributed server, in which case, data received through each server may be processed in a distributed manner.

[226] The claims herein may be combined in various ways. For example, the technical features in the claims for the method of the present description can be combined with the ability to be implemented or performed in equipment, and the technical features in the claims for equipment can be combined with the ability to be implemented or performed in the method. Additionally, the technical features in the method claim(s) and equipment claim(s) may be combined with the ability to be implemented or performed in the equipment. Additionally, the technical features in the method claim(s) and equipment claim(s) may be combined with the ability to be implemented or performed in the method.

Claims (28)

1. A method for decoding images, carried out by means of decoding equipment, the method comprising the steps of: obtaining image information including a multiple transform selection (MTS) index and residual information from the bit stream; extract quantized transform coefficients for the current block based on the residual information, obtaining transform coefficients for the current block by performing dequantization on the quantized transform coefficients; And generate residual samples of the current block by performing an inverse transform on the transform coefficients based on a set of transform kernels related to the MTS index, wherein the MTS index represents the set of transformation kernels that should be applied to the current block, from the possible sets of transformation kernels, wherein the permission elements from the MTS index permission element string are retrieved based on context encoding, the context encoding is performed based on the context index increment values for the permission elements from the MTS index permission element string, wherein the permission elements from the MTS index permission element string include first, second, third and fourth permission elements, and wherein the context index increment value for the first permission element is fixed to 0, the context index increment value for the second permission element is fixed to 1, the context index increment value for the third permission element is fixed to 2, and the context index increment value for the fourth permission element is fixed to 3 . 2. A method for encoding images, carried out by means of encoding equipment, the method comprising the steps of: extracting residual samples for the current block; extract transformation coefficients for the current block by performing a transformation on the residual samples based on a set of transformation kernels; obtaining quantized transform coefficients by performing quantization on the transform coefficients; generating a multiple transform selection (MTS) index related to the set of transform kernels and residual information related to the quantized transform coefficients; And encode image information including an MTS index and residual information for outputting a bit stream, wherein the MTS index represents the set of transformation kernels that should be applied to the current block, from the possible sets of transformation kernels, wherein the permission elements from the MTS index permission element string are represented based on context encoding, the context encoding is performed based on the context index increment values for the permission elements from the MTS index permission element string, wherein the permission elements from the MTS index permission element string include first, second, third and fourth permission elements, and wherein the context index increment value for the first permission element is fixed to 0, the context index increment value for the second permission element is fixed to 1, the context index increment value for the third permission element is fixed to 2, and the context index increment value for the fourth permission element is fixed to 3 . 3. A non-volatile computer-readable digital storage medium storing instructions directing the encoding equipment to carry out the image encoding method of claim 2. 4. A method for transmitting image data to decoding equipment, the method comprising the steps of: obtain a bit stream for the image, the bit stream is generated based on extracting the residual samples for the current block, extracting the transform coefficients for the current block by performing a transform on the residual samples based on a set of transform kernels, obtaining the quantized transform coefficients by performing quantization on the transform coefficients, generating a multiple index selecting a transform (MTS) related to a set of transform kernels and residual information related to the quantized transform coefficients, and encoding image information including an MTS index and residual information for outputting a bitstream; And transmitting data containing a bit stream for an image to decoding equipment to perform decoding and generate a reconstructed image based on the encoded data, wherein the MTS index represents the set of transformation kernels that should be applied to the current block, from the possible sets of transformation kernels, wherein the permission elements from the MTS index permission element string are represented based on context encoding, the context encoding is performed based on the context index increment values for the permission elements from the MTS index permission element string, wherein the permission elements from the MTS index permission element string include first, second, third and fourth permission elements, and wherein the context index increment value for the first permission element is fixed to 0, the context index increment value for the second permission element is fixed to 1, the context index increment value for the third permission element is fixed to 2, and the context index increment value for the fourth permission element is fixed to 3 .