RU2806813C2 - Method and device for image coding based on bdpcm - Google Patents

Abstract

FIELD: means for image encoding and decoding.

SUBSTANCE: predictive samples for the current block are retrieved based on the BDPCM. The residual samples of the current block are formed based on the predictive samples. Perform quantization on the residual samples. The quantized residual information based on the BDPCM is extracted, and the quantized residual information and flag information related to the BDPCM for the current block are encoded. Here, based on the fact that the width of the current block is less than or equal to the first threshold value, and the height of the current block is less than or equal to the second threshold value, the flag information is encoded. However, based on the fact that BDPCM is applied to the current block, the inseparable transformation is not applied to the current block. Based on the fact that BDPCM is applied to the current block, the transform index for the non-separable transform is not encoded.

EFFECT: increase in efficiency of image encoding.

14 cl, 11 dwg

Description

Field of technology to which the invention relates

[1] The present disclosure generally relates to image encoding technology, and more particularly to a BDPCM (Block Differential Pulse Code Modulation) based image encoding method in an image encoding system and equipment therefor.

Description of the Prior Art

[2] Nowadays, the demand for high-quality high-definition images/videos, for example, 4K, 8K or even higher ultra-high definition (UHD) images/videos, is growing in various fields of technology. As image/video data becomes increasingly higher resolution and higher quality, the amount of information transmitted, or bit capacity, increases compared to traditional image data. Therefore, when image data is transmitted using a medium such as a traditional wired/wireless broadband line, or image/video data is stored using an existing storage medium, transmission costs and storage costs for them increase.

[3] Additionally, nowadays, interest and demand for immersive multimedia such as virtual reality (VR), artificial reality (AR) or hologram content, etc. is growing, and broadcasting for images/videos having features images different from the features of real images, such as a game image, are increased.

[4] Accordingly, there is a need for high-performance image/video compression technology for efficiently compressing and transmitting or storing and reproducing information of high-quality, high-resolution images/videos having various features as described above.

The essence of the invention

[5] The technical aspect of the present disclosure is to provide a method and equipment for improving the efficiency of image encoding.

[6] Another technical aspect of the present disclosure is to provide a method and equipment for improving the efficiency of transform index coding when encoding BDPCM-based images.

[7] Another technical aspect of the present disclosure is to provide a method and equipment for improving coding efficiency based on transform skip flags when encoding BDPCM-based images.

[8] Yet another technical aspect of the present disclosure is to provide a method and equipment for performing BDPCM encoding for each luma signal component or chrominance signal component is provided.

[9] According to an embodiment of the present disclosure, an image decoding method performed by decoding equipment is provided. The method may include extracting quantized transform coefficients for a current block based on BDPCM; extracting transform coefficients by performing dequantization on the quantized transform coefficients; and extracting residual samples based on transformation coefficients; in which when BDPCM is applied to the current block, the inverse non-separable transform is not applied to the transform coefficients.

[10] When BDPCM is applied to the current block, the transform index value for the inverse non-separable transform that is applied to the current block is logically output to 0.

[11] When BDPCM is applied to the current block, the value of the conversion skip flag indicating whether or not conversion is skipped in the current block is logically output to 0.

[12] BDPCM is separately applied to the luma signal block of the current block or to the chrominance signal block of the current block, wherein when BDPCM is applied to the luma signal block, the transform index for the luma signal block is not received, and also when BDPCM is applied to the signal block chrominance, the transform index for the chrominance signal block is not accepted.

[13] When the width of the current block is less than or equal to the first threshold value, and the height of the current block is less than or equal to the second threshold value, BDPCM is applied to the current block.

[14] Quantized transform coefficients are extracted based on directional information relative to the direction in which BDPCM is performed.

[15] The image decoding method further comprises performing intra prediction for the current block based on the direction in which BDPCM is performed.

[16] Direction information indicates horizontal direction or vertical direction

[17] According to another embodiment of the present disclosure, an image encoding method performed by encoding equipment is provided. The method may include: retrieving predictive samples for the current block based on the BDPCM; extracting residual samples for the current block based on the predicted samples; perform quantization on residual samples; extracting quantized residual information based on BDPCM; and encoding the quantized residual information and encoding information for the current block; in which when BDPCM is applied to the current block, the non-separable transformation is not applied to the current block.

[18] According to yet another embodiment of the present disclosure, a digital storage medium that stores image data including encoded image information and a bitstream generated according to an image encoding method carried out by encoding equipment may be provided.

[19] According to yet another embodiment of the present disclosure, a digital storage medium that stores image data including encoded image information and a bit stream may be provided to instruct decoding equipment to perform an image decoding method.

[20] According to the present disclosure, it is possible to improve the overall efficiency of image/video compression.

[21] According to the present disclosure, it is possible to improve the overall compression efficiency of images/videos in index transform coding.

[22] According to the present disclosure, it is possible to improve the efficiency of transform index coding when encoding BDPCM-based images.

[23] According to the present disclosure, it is possible to improve coding efficiency based on transform skip flags when encoding an image based on BDPCM.

[24] According to the present disclosure, a method and equipment are provided for performing BDPCM encoding for each luma signal component or chrominance signal component.

[25] The benefits that may be obtained by specific examples of the present disclosure are not limited to the benefits mentioned above. For example, there may be various technical effects that those skilled in the art may understand or learn from the present disclosure. Accordingly, the specific advantages of the present disclosure are not limited to those explicitly described in the present disclosure, and may include various advantages that may be understood or derived from the technical features of the present disclosure.

Brief description of drawings

[26] FIG. 1 schematically illustrates an example of a video/image coding system to which the present disclosure is applicable.

[27] FIG. 2 is a diagram fundamentally illustrating the configuration of video/image encoding equipment to which the present disclosure is applicable.

[28] FIG. 3 is a diagram fundamentally illustrating the configuration of video/image decoding equipment to which the present disclosure is applicable.

[29] FIG. 4 schematically illustrates a multiple conversion technology according to an embodiment of the present disclosure.

[30] FIG. 5 illustrates the directional internal modes of 65 prediction directions.

[31] FIG. 6 is a diagram illustrating an RST according to an embodiment of the present disclosure.

[32] FIG. 7 is a flowchart illustrating operation of image decoding equipment according to an embodiment of the present disclosure.

[33] FIG. 8 is a flowchart illustrating a method for decoding images according to an embodiment of the present disclosure.

[34] FIG. 9 is a flowchart illustrating operation of image encoding equipment according to an embodiment of the present disclosure.

[35] FIG. 10 is a flowchart illustrating an image encoding method according to an embodiment of the present disclosure.

[36] FIG. 11 illustrates the structure of a content streaming system to which the present disclosure applies.

Detailed Description of Embodiments

[37] Although the present disclosure is subject to various modifications and includes various embodiments, specific embodiments thereof are shown in the drawings by way of example and are described in detail below. However, it is not intended to limit the present disclosure to the specific embodiments disclosed herein. The terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the technical intent of the present disclosure. Singular forms may include plural forms unless the context clearly indicates otherwise. Terms such as “include” and “have” are intended to indicate that there are features, figures, steps, operations, elements, components, or combinations thereof used in the description below, and as such should not understand that the possibility of the presence or addition of one or more other features, numbers, steps, operations, elements, components or combinations of the above is excluded in advance.

[38] Meanwhile, each component in the drawings described herein is illustrated independently for convenience of description with respect to characteristic functions different from each other, and, however, it is not intended that each component is implemented by separate hardware or software. For example, any two or more of these components may be combined to form one component, and any one component may be divided into multiple components. Embodiments in which components are combined and/or separated are intended to fall within the scope of the patent rights of this disclosure, provided that they do not depart from the spirit of this disclosure.

[39] Hereinafter, preferred embodiments of the present disclosure are explained in more detail with reference to the accompanying drawings. In addition, identical reference numbers are used for identical components in the drawings, and repeated descriptions for identical components are omitted.

[40] This document relates to video/image coding. For example, the method/example disclosed in this document may relate to the VVC (Universal Video Coding) standard (ITU-T-(H.266 recommendation), the next generation video/image coding standard after VVC, or other related video coding standards (for example, the HEVC (High Efficiency Video Coding) standard (ITU-T (H.265 recommendation), the EVC (fundamental video coding) standard, the AVS2 standard, etc.).

[41] Herein, a variety of embodiments related to video/image coding may be provided, and unless otherwise noted, embodiments may be combined and executed.

[42] In this document, video can mean a collection of sequences of images over time. In general, a frame means a unit representing an image in a particular time zone, and a slice/tile is a unit constituting a part of a frame. A slice/tile may include one or more coding tree units (CTUs). One frame can consist of one or more slices/tiles. One frame can consist of one or more groups of tiles. One tile group may include one or more tiles.

[43] Pixel or pel can mean the smallest unit that makes up one frame (or image). Additionally, "sample" can be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel/pixel value of a luma component or only a pixel/pixel value of a chrominance signal component. Alternatively, sample can mean a pixel value in the spatial domain, or when that pixel value is converted to the frequency domain, it can mean a conversion factor in the frequency domain.

[44] The unit may represent a basic image processing unit. The unit may include at least one of a specific area and information associated with the area. One unit may include one block of luma signals and two blocks of chrominance signals (eg, Cb, Cr). "Unit" and a term such as "block", "zone", etc. may be used instead of each other according to circumstances. In general, an MxN block may include a set (or array) of samples (or arrays of samples) or transform coefficients, consisting of M columns and N rows.

[45] In this document, the terms "/" and "," shall be interpreted to indicate "and/or". For example, the expression "A/B" could mean "A and/or B". Additionally, "A, B" may mean "A and/or B". Additionally, "A/B/C" may mean "at least one of A, B and/or C". In addition, "A/B/C" may mean "at least one of A, B and/or C".

[46] Additionally, as used herein, the term “or” should be interpreted to indicate “and/or.” For example, the expression "A or B" may include 1) A only, 2) B only, and/or 3) both A and B. In other words, the term "or" in this document should be interpreted to indicate "in addition or alternatively."

[47] In the present disclosure, “at least one of A and B” may mean “A only,” “B only,” or “both A and B.” In addition, in the disclosure, the expression “at least one of A or B” or “at least one of A and/or B” can be interpreted as “at least one of A and B”.

[48] Additionally, in the present disclosure, “at least one of A, B, and C” may mean “A only,” “B only,” “C only,” or “any combination of A, B, and C.” . In addition, "at least one of A, B or C" or "at least one of A, B and/or C" may mean "at least one of A, B and C".

[49] In addition, the parenthesis used in this disclosure may mean "for example." In particular, when specified as "forecasting (internal forecasting)", it may mean that "internal forecasting" is proposed as an example of "forecasting". In other words, the “prediction” of the present disclosure is not limited to “internal prediction”, and “internal prediction” may be offered as an example of “prediction”. In addition, when referred to as "forecasting (ie, internal forecasting)", it may also mean that "internal forecasting" is proposed as an example of "forecasting".

[50] Technical features separately described in a single drawing in this disclosure may be implemented separately or may be implemented simultaneously.

[51] FIG. 1 schematically illustrates an example of a video/image coding system to which the present disclosure is applicable.

[52] Referring to FIG. 1, a video/image encoding system may include a first device (source device) and a second device (receiver device). A source device may deliver encoded information or video/image data in file or streaming form to a receiving device via a digital storage medium or a network.

[53] The source device may include a video source, encoding equipment, and a transmitter. The receiving device may include a receiver, decoding equipment, and a rendering module. The encoding equipment may be referred to as "video/image encoding equipment" and the decoding equipment may be referred to as "video/image decoding equipment". The transmitter may be included in the encoding equipment. The receiver may be included in the decoding equipment. The rendering module may include a display, and the display may be configured as a separate device or an external component.

[54] A video source may obtain video/image through a video/image capture, synthesis, or shaping process. The video source may include a video/image capture device and/or a video/image generation device. The video/image capturing device may include, for example, one or more cameras, video/image archives including previously captured video/images, and the like. The video/image generating apparatus may include, for example, a computer, a tablet computer, and a smartphone, and may (electronically) generate the video/image. For example, a virtual video/image may be generated via a computer or the like. In this case, the video/image capturing process can be replaced by the associated data generation process.

[55] Encoding equipment can encode the input video/image. Encoding hardware may perform a sequence of procedures such as prediction, transform, and quantization to achieve compression and encoding efficiency. Encoded data (encoded video/image information) may be output in the form of a bitstream.

[56] The transmitter may transmit information or encoded video/image data output in the form of a bitstream to a receiver of a receiving device via a digital storage medium or network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and the like. The transmitter may include an element for generating a multimedia file through a predefined file format and may include an element for transmission through a broadcast/communication network. The receiver may receive/extract the bit stream and transmit the received/extracted bit stream to decoding equipment.

[57] The decoding equipment may decode the video/image by performing a sequence of procedures such as dequantization, deconversion, prediction, and the like, corresponding to the operation of the encoding equipment.

[58] The rendering module may prepare a decoded video/image by rendering. The video/image prepared by rendering can be displayed through the display.

[59] FIG. 2 is a diagram fundamentally illustrating the configuration of video/image encoding equipment to which the present disclosure is applicable. Hereinafter in this document, what is referred to as “video encoding equipment” may include image encoding equipment.

[60] Referring to FIG. 2, the encoding equipment 200 may include an image segmentation module 210, a prediction module 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The prediction module 220 may include an inter-prediction module 221 and a cross-prediction module 222. internal forecasting. Residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a “recovery module” or a “recovery block generator.” The image segmentation module 210, prediction module 220, residual processor 230, entropy encoder 240, adder 250, and filter 260, which are described above, may be composed of one or more hardware components (eg, encoder chipsets or processors) according to an embodiment. Additionally, storage device 270 may include a decoded frame buffer (DPB) and may be configured with a digital storage medium. The hardware component may further include a storage device 270 as an internal/external component.

[61] The image segmentation module 210 may segment an input image (either a frame or a movie frame) input to the encoding equipment 200 into one more processing unit. As one example, the processing unit may be referred to as a "coding unit (CU)". In this case, starting from a coding tree unit (CTU) or largest coding unit (LCU), the coding unit can be recursively segmented according to a quadtree, binary tree, and ternary tree (QTBTTT) structure. For example, one coding unit may be divided into multiple coding units of greater depth based on a quadtree structure, a binary tree structure, and/or a ternary structure. In this case, for example, a quadtree structure may be applied first, and a binary tree structure and/or a ternary structure may be subsequently applied. Alternatively, a binary tree structure may be used first. The encoding procedure according to the present disclosure may be performed based on the final encoding unit, which is not further segmented. In this case, the maximum encoding unit can be used directly as the final encoding unit based on the encoding efficiency according to the characteristic of the image. Alternatively, the coding unit can be recursively segmented into coding units of additional greater depth as needed, so that the optimally sized coding unit can be used as the final coding unit. Here, the encoding procedure may include procedures such as prediction, transformation and reconstruction, which are described below. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit may be split or segmented from the above-described final encoding unit. The prediction unit may be a sample prediction unit, and the transformation unit may be a unit for extracting a transformation coefficient and/or a unit for extracting a residual signal from the transformation coefficient.

[62] "Unit" and a term such as "block", "zone", etc. may be used instead of each other according to circumstances. In general, an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally represent a pixel or a pixel value, and may represent only a pixel/pixel value of a luma component or only a pixel/pixel value of a chrominance signal component. Sampling can be used as a term corresponding to a pixel or pel of a single frame (or image).

[63] The subtractor 223 subtracts the prediction signal (predicted block, prediction sample array) output from the inter-prediction module 221 or the intra-prediction module 222 from the input image signal (source block, source sample array) to generate a residual signal (residual block, residual sample array), and the generated residual signal is transmitted to converter 232. In this case, as shown, a module that subtracts the prediction signal (predicted block, prediction sample array) from the input image signal (source block, source sample array) at encoder 200 , may be called "subtractor 231". The prediction module may perform prediction for a target processing block (hereinafter referred to as a "current block") and may generate a prediction block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied based on the current block or CU. As explained below in the description of each prediction mode, the prediction module may generate various prediction-related information, such as prediction mode information, and transmit the generated information to the entropy encoder 240. Information regarding the prediction may be encoded in the entropy encoder 240 and output in the form of a stream bits

[64] The intra prediction module 222 may predict the current block by reference to the samples in the current frame. Samples for the link can be located in an adjacent node or spaced from the current block according to the prediction mode. In intraprediction, prediction modes may include a plurality of non-directional modes and a plurality of directed modes. Omnidirectional modes may include, for example, DC mode and planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the granularity of the prediction direction. However, this is merely an example, and more or less directional prediction modes may be used depending on the setting. The intra prediction module 222 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[65] The inter-prediction unit 221 can extract a predicted block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. At this time, in order to reduce the amount of motion information transmitted in the inter-prediction mode, motion information may be predicted on a block, sub-block, or sample basis based on the correlation of motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0-prediction, L1-prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block existing in the current frame and a temporal neighbor block existing in the reference frame. The reference frame including the reference block and the reference frame including the temporary adjacent block may be identical to each other or different from each other. A temporary neighbor block may be called a “co-located reference block”, a “col-located CU (colCU)”, etc., and a reference frame including a temporary neighboring block may be called a “col-located frame (colPic)”. For example, the inter-prediction module 221 may configure a list of motion information candidates based on neighboring blocks and generate information indicating which candidate is used to retrieve the motion vector and/or reference frame index of the current block. Mutual prediction can be performed based on different prediction modes. For example, in the case of the skip mode and the merge mode, the inter-prediction unit 221 may use the motion information of the adjacent block as the motion information of the current block. In the skip mode, unlike the combine mode, the residual signal may not be transmitted. In the case of a motion vector information prediction mode (motion vector prediction, MVP), the motion vector of an adjacent block may be used as a motion vector predictor, and the motion vector of the current block may be indicated by signaling a motion vector difference.

[66] The prediction module 220 may generate a prediction signal based on various prediction methods. For example, the prediction module may apply intra-prediction or inter-prediction to make predictions regarding one block, and may also apply intra-prediction and inter-prediction simultaneously. This may be referred to as "combined inter-intraprediction (CIIP)". Additionally, the prediction module may be based on an intra-block copy (IBC) prediction mode or a palette mode to perform prediction relative to a block. The IBC prediction mode or palette mode can be used for encoding image/video content of a game and the like, for example, screen content encoding (SCC). Although IBC essentially performs prediction in the current block, it can perform similar to inter prediction in that it retrieves the reference block in the current block. Thus, the IBC may use at least one of the inter-prediction technologies described in the present disclosure.

[67] The prediction signal generated through the inter prediction module 221 and/or the intra prediction module 222 can be used to generate a reconstructed signal or generate a residual signal. Converter 232 may generate conversion coefficients by applying conversion technology to the residual signal. For example, the transform technology may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally nonlinear transform (CNT). ). Here, GBT means a graph-derived transformation when the relationship information between pixels is represented by the graph. CNT means the transformation obtained from the prediction signal generated using all the previously reconstructed pixels. In addition, the conversion process can be applied to square pixel blocks that are identical in size, or can be applied to blocks that are variable in size rather than square.

[68] Quantizer 233 may quantize transform coefficients and transmit them to entropy encoder 240, and entropy encoder 240 may encode the quantized signal (information regarding the quantized transform coefficients) and output the encoded signal in a bit stream. Information regarding the quantized transform coefficients may be referred to as "residual information". Quantizer 233 may re-arrange the block quantized transform coefficients into one-dimensional vector form based on the scanning order of the coefficients, and generate information regarding the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form. The entropy encoder 240 may implement various encoding techniques, such as, for example, exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like. Entropy encoder 240 may encode information needed to reconstruct video/images other than quantized transform coefficients (eg, syntax element values, etc.), together or separately. Encoded information (eg, encoded video/image information) may be transmitted or stored based on Network Abstraction Layer (NAL) units in the form of a bit stream. The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS), and the like. Additionally, the video/image information may further include general restriction information. In the present disclosure, information and/or syntax elements that are/are signaled to decoding equipment from encoding equipment may be included in video/image information. The video/image information may be encoded through the encoding procedure described above and included in the bit stream. The bit stream may be transmitted over a network or stored on a digital storage medium. Here, the network may include a broadcast network, a communication network, and the like, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and etc. A transmitter (not shown) that transmits the signal output from entropy encoder 240 and/or a storage module (not shown) that stores it may be configured as an internal/external element of encoding equipment 200, or the transmitter may be included in entropy encoder 240 .

[69] The quantized transform coefficients output from quantizer 233 can be used to generate a predictive signal. For example, by applying dequantization and inverse transform to the quantized transform coefficients through dequantizer 234 and inverter 235, the residual signal (residual block or residual samples) can be recovered. The adder 155 adds the reconstructed residual signal with the prediction signal output from the inter-prediction module 221 or the intra-prediction module 222, so that a reconstructed signal (reconstructed frame, reconstructed block, reconstructed sample array) can be generated. When there is no remainder for the target processing block, as is the case when the skip mode is applied, the predicted block can be used as the reconstructed block. The adder 250 may be referred to as a "recovery module" or a "recovered block generator." The generated reconstructed signal can be used to internally predict the next target processing block in the current block and, as described below, can be used to inter-predict the next frame through filtering.

[70] Meanwhile, in the process of encoding and/or restoring frames, luma conversion with chrominance scaling (LMCS) may be applied.

[71] Filter 260 may enhance the subjective/objective quality of the video by applying filtering to the reconstructed signal. For example, filter 260 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and may store the modified reconstructed frame in storage device 270, particularly in the DPB of storage device 270. Various filtering techniques may include, for example, deletion filtering blocking, sampled adaptive offset, adaptive contour filter, bilateral filter, etc. As explained below in the description of each filtering method, the filter 260 may generate various filtering-related information and transmit the generated information to the entropy encoder 240. The filtering information may be encoded in the entropy encoder 240 and output in the form of a bit stream.

[72] The modified reconstructed frame, which is transferred to the storage device 270, can be used as a reference frame in the inter-prediction unit 221. Through this, the encoding equipment can prevent prediction mismatch in the encoding equipment 100 and the decoding equipment when inter-prediction is applied, and can also improve encoding efficiency.

[73] The DPB of the storage device 270 may store the modified reconstructed frame to use as a reference frame in the inter-prediction module 221. Memory 270 may store block motion information in the current frame from which motion information is extracted (or encoded), and/or block motion information in an already reconstructed frame. The stored motion information may be transmitted to the inter-prediction unit 221 for use as neighboring block motion information or temporary neighbor block motion information. Memory 270 may store reconstructed samples of reconstructed blocks in the current frame and transmit them to intra prediction module 222.

[74] FIG. 3 is a diagram fundamentally illustrating the configuration of video/image decoding equipment to which the present disclosure is applicable.

[75] Referring to FIG. 3, the image decoding equipment 300 may include an entropy decoder 310, a residual processor 320, a prediction unit 330, an adder 340, a filter 350, and a memory 360. The prediction unit 330 may include an inter-prediction unit 331 and an intra-prediction unit 332. Residual processor 320 may include a dequantizer 321 and an inverse converter 321. Entropy decoder 310, residual processor 320, prediction module 330, adder 340, and filter 350, which are described above, may be comprised of one or more hardware components (e.g., chipsets or decoder processors) according to an embodiment. Additionally, storage device 360 may include a decoded frame buffer (DPB) and may be configured with a digital storage medium. The hardware component may further include storage device 360 as an internal/external component.

[76] When a bit stream including video/image information is input, the decoding equipment 300 can reconstruct an image corresponding to the process by which the video/image information is processed in the encoding equipment of FIG. 2. For example, decoding equipment 300 may extract units/blocks based on block segmentation information obtained from the bit stream. The decoding equipment 300 may perform decoding by using a processing unit applied in the encoding equipment. Therefore, a decoding processing unit, for example, may be an encoding unit, which may be segmented along a quadtree structure, a binary tree structure, and/or a ternary tree structure from an encoding tree unit or a largest encoding unit. One or more transformation units may be derived from a coding unit. Also, the reconstructed image signal decoded and output through the decoding equipment 300 can be reproduced through the playback unit.

[77] The decoding equipment 300 may receive a signal output from the encoding equipment of FIG. 2 in the form of a bit stream, and the received signal can be decoded through an entropy decoder 310. For example, an entropy decoder 310 can parse the bit stream to extract information (eg, video/image information) required for image reconstruction (or frame reconstruction). The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS), and the like. Additionally, the video/image information may further include general restriction information. The decoding equipment may further decode the frame based on information regarding the parameter set and/or general constraint information. In the present disclosure, the signaling/receiving information and/or syntax elements that are described below can be decoded by a decoding procedure and obtained from the bitstream. For example, entropy decoder 310 may decode information in a bit stream based on an encoding method such as exponential Golomb coding, CAVLC, CABAC, and the like, and may output the value of a syntax element required for image reconstruction and quantized transform coefficient values with respect to remainder. More specifically, the entropy CABAC decoding method may take a resolution element corresponding to each syntax element in the bitstream, determine a context model using decoding the information of target syntax elements and decoding the information of adjacent and decodable target blocks or the symbol/resolution element information decoded in a previous step. , predict the probability of generating a resolution element according to a certain context model, and perform arithmetic decoding of the permission element to generate a symbol corresponding to each value of the syntactic element. Here, the entropy CABAC decoding method can update the context model using the symbol/entry element information decoded for the context model of the next symbol/entry element after determining the context model. Information regarding prediction from the information decoded in the entropy decoder 310 may be provided to the prediction module (inter prediction module 332 and intra prediction module 331), and the residual values, i.e. quantized transform coefficients for which entropy decoding is performed in entropy decoder 310 and associated parameter information may be input to residual processor 320. Residual processor 320 may extract a residual signal (residual block, residual samples, array of residual samples). Additionally, information regarding filtering from information decoded in the entropy decoder 310 may be provided to the filter 350. Meanwhile, a receiver (not shown) that receives a signal output from the encoding equipment may further constitute the decoding equipment 300 as an internal/external element. , and the receiver may be a component of the entropy decoder 310. Meanwhile, the decoding equipment according to the present disclosure may be called "video/image/frame encoding equipment", and the decoding equipment may be classified into information decoder (video/image/frame information decoder) and selective decoder (video/image/frame sampling decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverter 322, an adder 340, a filter 350, a memory 360, an inter-prediction module 332, and an intra-prediction module 331 .

[78] The dequantizer 321 may output transform coefficients by dequantizing the quantized transform coefficients. The dequantizer 321 may re-arrange the quantized transform coefficients in the form of a two-dimensional block. In this case, the re-arrangement may perform the re-arrangement based on the scanning order of the coefficients, which is performed in the encoding equipment. The dequantizer 321 may perform dequantization on the quantized transform coefficients using a quantization parameter (eg, quantization step size information) and obtain the transform coefficients.

[79] The dequantizer 322 obtains the residual signal (residual block, array of residual samples) by inversely transforming the transform coefficients.

[80] Prediction module 230 may perform prediction for the current block and may generate a predicted block including prediction samples for the current block. The prediction module can determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoder 310, and in particular can determine the intra/inter prediction mode.

[81] The prediction module can generate a prediction signal based on various prediction methods. For example, the prediction module may apply intra-prediction or inter-prediction to make predictions regarding one block, and may also apply intra-prediction and inter-prediction simultaneously. This may be referred to as "combined inter-intraprediction (CIIP)". In addition, the prediction module can perform intra-block copy (IBC) to predict for a block. In-block copying can be used to encode game image/video content and the like, for example, screen content encoding (SCC). Although IBC essentially performs prediction in the current block, it can perform similar to inter prediction in that it retrieves the reference block in the current block. Thus, the IBC may use at least one of the inter-prediction technologies described in the present disclosure.

[82] The intra prediction module 331 may predict the current block by reference to the samples in the current frame. Samples for reference can be located in the vicinity of or spaced from the current block according to the prediction mode. In intraprediction, prediction modes may include a plurality of non-directional modes and a plurality of directed modes. The intra prediction unit 331 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[83] The inter-prediction unit 332 may extract a predicted block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. At this time, in order to reduce the amount of motion information transmitted in the inter-prediction mode, motion information may be predicted on a block, sub-block, or sample basis based on the correlation of motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0-prediction, L1-prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block existing in the current frame and a temporal neighbor block existing in the reference frame. For example, the inter-prediction module 332 may configure a motion information candidate list based on neighboring blocks and extract the motion vector and/or reference frame index of the current block based on the received candidate selection information. The inter-prediction may be performed based on various prediction modes, and the information regarding the prediction may include information indicating the inter-prediction mode for the current block.

[84] The adder 340 can generate a reconstructed signal (reconstructed frame, reconstructed block, reconstructed sample array) by adding the resulting residual signal with a predicted signal (predicted block, predicted sample array) output from prediction unit 330. When there is no remainder for the target processing block, as is the case when the skip mode is applied, the predicted block can be used as the reconstructed block.

[85] The adder 340 may be referred to as a "recovery module" or a "recovered block generator." The generated reconstructed signal may be used to internally predict the next target processing block in the current block and, as described below, may be output by filtering or used to inter-predict the next frame.

[86] Meanwhile, in the process of decoding frames, luma conversion with chrominance scaling (LMCS) may be applied.

[87] Filter 350 may enhance the subjective/objective quality of video by applying filtering to the reconstructed signal. For example, filter 350 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and may transfer the modified reconstructed frame to storage device 360, particularly to a DPB of storage device 360. Various filtering techniques may include, for example, deletion filtering blocking, sampled adaptive offset, adaptive contour filter, bilateral filter, etc.

[88] The (modified) reconstructed frame, which is stored in the DPB of the storage device 360, can be used as a reference frame in the inter-prediction unit 332. Storage device 360 may store block motion information in the current frame from which motion information is extracted (or decoded), and/or block motion information in an already reconstructed frame. The stored motion information may be transmitted to the inter-prediction unit 260 for use as neighbor block motion information or temporary neighbor block motion information. Storage device 360 may store reconstructed samples of reconstructed blocks in the current frame and transmit them to intra prediction module 331.

[89] In this detailed description, the examples described in prediction module 330, dequantizer 321, inverse converter 322, and filter 350 of decoding equipment 300 may be similarly or correspondingly applied to prediction module 220, dequantizer 234, inverse converter 235, and filter 260 encoding equipment 200, respectively.

[90] As described above, prediction is performed in order to improve compression efficiency when performing image encoding. Due to this, a predicted block can be generated including predictive samples for the current block, which is the target encoding block. Here, the predicted block includes predictive samples in a spatial domain (or pixel domain). The predicted block can be identically extracted in the encoding equipment and the decoding equipment, and the encoding equipment can improve the efficiency of image encoding by signaling to the decoding equipment not the original sample value of the original block itself, but information regarding the remainder (residual information) between the original block and the predicted block . The decoding equipment may extract a residual block including residual samples based on the residual information, generate a reconstructed block including the reconstructed samples by adding the residual block with the predicted block, and generate a reconstructed frame including the reconstructed blocks.

[91] Residual information can be generated through transformation and quantization procedures. For example, the encoding equipment may extract a residual block between an original block and a predicted block, extract transform coefficients by performing a transform procedure on the residual samples (an array of residual samples) included in the residual block, and extract quantized transform coefficients by performing a quantization procedure on the transform coefficients, such as that it can signal associated residual information to decoding equipment (via a bitstream). Here, the residual information may include value information, position information, conversion technology, conversion kernel, quantization parameter, and the like. quantized conversion coefficients. The decoding equipment may perform a quantization/dequantization procedure and extract residual samples (or a block of residual samples) based on the residual information. The decoding equipment can generate a reconstructed block based on the predicted block and the residual block. The encoding equipment may extract the residual block by dequantizing/de-converting the quantized transform coefficients for the next frame inter-prediction reference and may generate a reconstructed frame based on it.

[92] FIG. 4 schematically illustrates a multiple conversion technology according to an embodiment of the present disclosure.

[93] Referring to FIG. 4, the converter may correspond to the converter in the encoding equipment of FIG. 2, and the inverter may correspond to the inverter in the encoding equipment of the above FIG. 2 or the inverse converter in the decoding equipment of FIG. 3.

[94] The converter can extract (primary) transform coefficients by performing a primary transform based on the residual samples (residual sample array) in the residual block (S410). This primary transformation may be called the "base transformation". Herein, the primary transform may be based on multiple transform selection (MTS), and when the multiple transform is applied as the primary transform, it may be referred to as a "multi-core transform".

[95] The multi-core transform may represent a transform method further using discrete cosine transform (DCT) type 2 and discrete sine transform (DST) type 7, DCT-type 8 and/or DST-type 1. Thus, the multi-core transform may represent the method transform for converting the spatial domain residual signal (or residual block) into frequency domain transform coefficients (or primary transform coefficients) based on a plurality of transform kernels selected from DCT-type 2, DST-type 7, DCT-type 8, and DST-type 1. In this document, the primary conversion coefficients may be referred to as “temporary conversion coefficients” from the point of view of the converter.

[96] In other words, when the conventional transform method is applied, transform coefficients can be generated by applying a spatial domain to frequency domain transform to the residual signal (or residual block) based on DCT type 2. In contrast, in this case, when multi-core transform is applied, transform coefficients (or primary transform coefficients) can be generated by applying a spatial domain to frequency domain transform to the residual signal (or residual block) based on DCT-type 2, DST-type 7, DCT-type 8 and/or DST -type 1. In this document, DCT type 2, DST type 7, DCT type 8 and DST type 1 may be referred to as "transformation type", "transform kernel" or "transform core" ". These types of DCT/DST transformations can be specified based on basis functions.

[97] If a multiple base transform is performed, a vertical transform kernel and a horizontal transform kernel for the target block may be selected from among the transform kernels, a vertical transform for the target block may be performed based on the vertical transform kernel, and a horizontal transform for the target block may be performed based on horizontal transformation kernels. Here, a horizontal transform may represent a transform for the horizontal components of the target block, and a vertical transform may represent a transform for the vertical components of the target block. The vertical transform kernel/horizontal transform kernel may be adaptively determined based on the prediction mode and/or transform index of the target block (CU or subblock) including the residual block.

[98] Additionally, according to an example, if the primary transformation is performed by applying the MTS, the transformation relationship for the transformation kernels can be specified by setting specific basis functions to predetermined values and combining the basis functions to be applied in the vertical transformation or horizontal transformation. For example, when the horizontal transformation kernel is expressed as trTypeHor and the vertical direction transformation kernel is expressed as trTypeVer, a trTypeHor or trTypeVer value of 0 may be specified for DCT2, a trTypeHor or trTypeVer value of 1 may be specified for DST7, and a trTypeHor or trTypeVer value of 2 may be specified for DCT8.

[99] In this case, the MTS index information may be encoded and signaled to decoding equipment to indicate any of a plurality of sets of transform cores. For example, MTS index 0 may indicate that trTypeHor and trTypeVer are equal to 0, MTS index 1 may indicate that trTypeHor and trTypeVer are equal to 1, MTS index 2 may indicate that trTypeHor is equal to 2 and the value trTypeVer. Equal to 1, MTS index 3 may indicate that trTypeHor is 1 and trTypeVer is 2, and MTS index 4 may indicate that trTypeHor and trTypeVer are both 2.

[100] In one example, sets of transformation kernels according to MTS index information are illustrated in the following table.

[101] Tab. 1

[102] The converter can extract modified (secondary) conversion coefficients by performing secondary conversion based on the (primary) conversion coefficients (S420). The primary transform is a conversion from the spatial domain to the frequency domain, and the secondary transform is a conversion to a more compressive expression by exploiting the correlation existing between the (primary) transform coefficients. The secondary transformation may include an inseparable transformation. In this case, the secondary transform may be called a "non-separable secondary transform (NSST)" or a "mode-dependent non-separable secondary transform (MDNSST)". The inseparable secondary transform may be a transform that generates modified transform coefficients (or secondary transform coefficients) for the residual signal by a secondary transform, based on the inseparable transform matrix of the (primary) transform coefficients extracted through the primary transform. At this time, the vertical transform and horizontal transform may not be applied separately (or the horizontal and vertical transforms may not be applied independently) to the (primary) transform coefficients, but the transforms may be applied immediately based on the inseparable transform matrix. In other words, an inseparable secondary transform may represent a transform method in which the vertical and horizontal components of the (primary) transform coefficients are not separated, and, for example, the two-dimensional signals (transform coefficients) are recomposed into a one-dimensional signal through a particular defined direction (for example, the row-first direction or column-first direction), and then modified transform coefficients (or secondary transform coefficients) are generated based on the inseparable transform matrix. For example, according to row first order, MxN blocks are arranged in a line in the order "first row, second row, ..., and Nth row". According to column first order, MxN blocks are arranged in a line in the order "first column, second column, ..., and Nth column". An inseparable secondary transform may be applied to the upper left region of a block configured with (primary) transform coefficients (hereinafter in this document, may be referred to as a “transform coefficient block”). For example, if the width (W) and height (H) of the transform coefficient block are equal to or greater than 8, an inseparable secondary 8×8 transform may be applied to the upper left 8×8 region of the transform coefficient block. Additionally, if the width (W) and height (H) of the transform coefficient block are equal to or greater than 4, and the width (W) or height (H) of the transform coefficient block is less than 8, then an inseparable 4x4 secondary transform can be applied to the left upper region min (8, W)xmin(8, H) block of transformation coefficients. However, the embodiment is not limited to this and, for example, even if only the condition that the width (W) or height (H) of the transform coefficient block is equal to or greater than 4 is satisfied, the inseparable 4x4 secondary transform can be applied to the left the upper region min(8, W)xmin(8, H) of the block of transformation coefficients.

[103] In particular, for example, if a 4x4 input block is used, the non-separable secondary transform can be performed as follows.

[104] The X 4x4 input block can be represented as follows.

[105] equation 1

[106] If X is represented in vector form, the vector may be presented as below.

[107] equation 2

[108] In Equation 2, the vector is a one-dimensional vector obtained by rearranging the two-dimensional block X of Equation 1 according to row-first order.

[109] In this case, the secondary inseparable transform can be calculated as follows.

[110] equation 3

[111] In this equation, represents a vector of transform coefficients, and T represents a 16x16 (non-separable) transform matrix.

[112] Through the above equation 3, the vector 16×1 transform coefficients can be extracted, and can be reorganized into a 4x4 block via scan order (horizontal, vertical, diagonal, etc.). However, the above calculation is an example, and the Hypercubic Givens Transform (HyGT) etc. can be used to compute the non-separable secondary transform to reduce the computational complexity of the non-separable secondary transform.

[113] Meanwhile, in the non-separable secondary transformation, the transformation kernel (or transformation base, transformation type) can be selected depending on the mode. In this case, the mode may include an intra prediction mode and/or an inter prediction mode.

[114] As described above, NSST may be performed based on an 8×8 transform or a 4×4 transform determined based on the width (W) and height (H) of the transform coefficient block. 8×8 transform means a transform that is applicable to an 8×8 region included in a block of transform coefficients when both W and H are equal to or greater than 8, and the 8×8 region may be the left upper 8×8 region in the block conversion factors. Likewise, a 4×4 transform means a transform that is applicable to a 4×4 region included in a block of transform coefficients when both W and H are equal to or greater than 4, and the 4×4 region may be the left upper 4×4 region in block of conversion coefficients. For example, an 8×8 transform kernel matrix may be a 64×64/16×64 matrix, and a 4×4 transform kernel matrix may be a 16×16/8×16 matrix.

[115] Here, in order to select a mode-dependent transformation kernel, two non-separable secondary transformation cores per transformation set for the non-separable secondary transformation can be configured for both 8×8 transformation and 4×4 transformation, and can be provided four conversion kits. Thus, four conversion sets can be configured for an 8×8 conversion, and four conversion sets can be configured for a 4×4 conversion. In this case, each of the four transform sets for the 8×8 transform may include two 8×8 transform cores, and each of the four transform sets for the 4×4 transform may include two 4×4 transform cores.

[116] The sizes of transformations, the number of sets, and the number of transformation kernels in each set mentioned above are merely for illustration. Instead, a size other than 8x8 or 4x4 may be used, n sets may be configured, and k translation cores may be included in each set.

[117] The transformation set may be referred to as an "NSST-set", and the transformation core in the NSST-set may be referred to as an "NSST-core". The selection of a particular set from among the sets for transformation may be performed, for example, based on the intra prediction mode of the target block (CU or sub-block).

[118] By way of reference, by way of example, an intra prediction mode may include two non-directional (or non-angular) intra prediction modes and 65 directional (or angular) intra prediction modes. The non-directional intra-prediction modes may include a planar intra-prediction mode number 0 and an intra-DC prediction mode number 1, and the directed intra-prediction modes may include 65 intra-prediction modes between the intra-prediction mode number 2 and the intra-prediction mode number 66. However, this is an example, and the present disclosure may be applied to a case in which a different number of intra prediction modes are provided. Meanwhile, according to circumstances, the intra prediction mode number 67 may further be used, and the intra prediction mode number 67 may represent a linear model (LM) based mode.

[119] FIG. 5 illustrates the directional internal modes of 65 prediction directions.

[120] Referring to FIG. 5, based on the intra prediction mode number 34 having a left up diagonal prediction direction, the intra prediction mode having a horizontal direction and the intra prediction mode having a vertical direction can be classified. H and V in Fig. 5 indicate horizontal directivity and vertical directivity, respectively, and numbers -32 to 32 indicate offsets of 1/32 units for the position on the sample grid. It may represent an offset for the mode index value. Intra prediction modes number 2-33 are horizontally directed, and intra prediction modes numbered 34-66 are vertically directed. Meanwhile, strictly speaking, the internal prediction mode number 34 can be considered as neither horizontal nor vertical, but it can be classified as belonging to the horizontal direction from the point of view of defining the secondary transform transform set. This is because the input data is transposed to be used for the vertical direction mode symmetrical based on the intra prediction mode number 34, and the input data combination method for the horizontal mode is used for the intra prediction mode number 34. Transposition of the input data means that the rows and columns of two-dimensional MxN block data is switched to NxM data. The intra-prediction mode number 18 and the intra-prediction mode number 50 may represent a horizontal intra-prediction mode and a vertical intra-prediction mode, respectively, and the intra-prediction mode number 2 may be called "right-up diagonal intra-prediction mode" because it has a left reference pixel and predicts in an upward-right direction. In an identical manner, the intra-prediction mode number 34 may be referred to as the "right-down-diagonal intra-prediction mode" and the intra-prediction mode number 66 may be referred to as the "left-down-diagonal intra-prediction mode".

[121] In one example, four transformation sets may be transformed according to an intra prediction mode, as indicated in the following table.

[122] Tab. 2

[123] As shown in Table 2, one of the four conversion sets, i.e. stTrSetIdx, can be converted to one of four values, i.e. 0-3, according to the internal forecasting mode.

[124] When it is determined that a particular set is used for an inseparable transformation, one of the k transformation kernels in the particular set may be selected via the inseparable secondary transformation index. The encoding equipment may extract an inseparable secondary transform index indicating a particular transform kernel based on a rate-dependent distortion check (RD), and may signal the inseparable secondary transform index to the decoding equipment. The decoding equipment may select one of the k transform cores in a particular set based on the inseparable secondary transform index. For example, an NSST index value of 0 may indicate a first non-separable secondary transform core, an NSST index value of 1 may indicate a second non-separable secondary transform core, and an NSST index value of 2 may indicate a third non-separable secondary transform core. Alternatively, an NSST index value of 0 may indicate that the first non-separable secondary transform is not applied to the target block, and NSST index values of 1-3 may indicate three transform cores.

[125] The transformer may perform an inseparable secondary transform based on the selected transform kernels and may obtain modified (secondary) transform coefficients. As described above, the modified transform coefficients can be extracted as transform coefficients quantized through a quantizer, and can be encoded and signaled to the decoding equipment and transmitted to a dequantizer/inverse converter in the encoding equipment.

[126] Meanwhile, as described above, if the secondary transform is omitted, the (primary) transform coefficients that represent the output of the primary (separable) transform can be extracted as transform coefficients quantized through a quantizer as described above, and can be encoded and transmitted in service signals to the decoding equipment and transmitted to the dequantizer/inverse converter in the encoding equipment.

[127] The inverse converter may perform a sequence of procedures in the reverse order of the order in which they are performed in the above-described converter. The inverse converter can receive (dequantized) transform coefficients and extract the (primary) transform coefficients by performing a secondary (inverse) transform (S450), and can obtain a residual block (residual samples) by performing a primary (inverse) transform on the (primary) transform coefficients (S460 ). In this regard, the primary conversion coefficients may be referred to as "modified conversion coefficients" from the point of view of the inverse converter. As described above, the encoding equipment and decoding equipment can generate a reconstructed block based on the residual block and the predicted block, and can generate a reconstructed image based on the reconstructed block.

[128] The decoding equipment may further include a secondary inverse transform application determining unit (or an element for determining whether or not the secondary inverse transform should be applied) and a secondary inverse transform determining unit (or an element for determining whether or not the secondary inverse transform is applied). transformation). The secondary inverse transform application determination module may determine whether or not to apply the secondary inverse transform. For example, the secondary inverse transform may be NSST or RST, and the secondary inverse transform application determination module may determine whether or not to apply the secondary inverse transform based on the secondary transform flag obtained by parsing the bitstream. In another example, the secondary inverse transform application determining unit may determine whether or not to apply the secondary inverse transform based on the transform coefficient of the residual block.

[129] The secondary inverse transform determination module may determine the secondary inverse transform. In this case, the secondary inverse transform determining unit may determine the secondary inverse transform applied to the current block based on the set for the NSST (or RST) transform specified according to the intra prediction mode. In an embodiment, the method for determining the secondary transformation may be determined depending on the method for determining the primary transformation. Various combinations of primary transformations and secondary transformations may be determined according to the internal prediction mode. Additionally, in the example, the secondary inverse transform determination module may determine the region to which the secondary inverse transform is applied based on the size of the current block.

[130] Meanwhile, as described above, if the secondary (inverse) transform is omitted, the (dequantized) transform coefficients can be received, the primary (separable) inverse transform can be performed, and the residual block (residual samples) can be obtained. As described above, the encoding equipment and the decoding equipment can generate a reconstructed block based on the residual block and the predicted block, and can generate a reconstructed image based on the reconstructed block.

[131] Meanwhile, in the present disclosure, reduced secondary transform (RST), in which the size of the transformation matrix (kernel) is reduced, can be applied in the concept of NSST to reduce the amount of computation and storage required for the inseparable secondary transform.

[132] Meanwhile, the transformation kernel, the transformation matrix and the coefficient constituting the transformation kernel matrix, i.e. the kernel coefficient or matrix coefficient described in the present disclosure may be expressed in 8 bits. This may be a condition for implementation in decoding equipment and encoding equipment, and allows the storage space required to store the conversion kernel to be reduced, with a performance penalty that can be accommodated sufficiently compared to the existing 9 bits or 10 bits. In addition, expressing the kernel matrix in 8 bits can enable the use of a small multiplier and may be more suitable for single instruction multiple data (SIMD) instructions used for optimal software implementation.

[133] As used herein, the term "RST" may refer to a transformation that is performed on the residual samples for a target block based on a transformation matrix whose size is reduced according to a reduction factor. When performing a reduced transformation, the amount of computation required for the transformation may be reduced due to the reduction in the size of the transformation matrix. Thus, RST can be used to solve the problem of computational complexity encountered in non-separable transformation or large block size transformation.

[134] RST may be referred to by various terms such as "reduced transform", "reduced secondary transform", "reduced transform", "simplified transform", "simple transform", etc., and named by which may be called RST, is not limited to the examples listed. Alternatively, since RST is primarily performed in the low frequency region including a non-zero coefficient in the transform block, it may be referred to as "low frequency non-separable transform (LFNST)". The transformation index may be referred to as the "LFNST index".

[135] Meanwhile, when the secondary inverse transform is performed based on the RST, the inverse transformer 235 of the encoding equipment 200 and the inverse transformer 322 of the decoding equipment 300 may include an inverse shortened secondary transformer that extracts modified transform coefficients based on the inverse RST transform coefficients, and an inverse primary transformer that extracts residual samples for the target block based on the inverse primary transform of the modified transform coefficients. Reverse the primary transformation means the inverse transformation of the primary transformation applied to the remainder. In the present disclosure, extracting a transform coefficient based on a transform may mean extracting a transform coefficient by applying a transform.

[136] FIG. 6 is a diagram illustrating an RST according to an embodiment of the present disclosure.

[137] In the present disclosure, a "target block" may mean a current block to be encoded, a residual block, or a transform block.

[138] In RST, by example, an N-dimensional vector can be transformed into an R-dimensional vector located in a different space, so that the reduced transformation matrix can be defined where R is less than N. N can mean the square of the side length of the block to which the transformation is applied , or the total number of transformation coefficients corresponding to the block to which the transformation is applied, and the shortened coefficient may indicate the R/N value. The shortened ratio may be referred to as a “shortened ratio,” “reduced ratio,” “simplified ratio,” “simple ratio,” or various other terms. Meanwhile, R may be called a "shortened coefficient", but according to circumstances, the shortened coefficient may mean R. Additionally, according to circumstances, the shortened coefficient may mean N/R value.

[139] In an example, a reduced factor or reduced coefficient may be signaled via a bit stream, but the example is not limited to this. For example, a predetermined value for a reduced factor or reduced coefficient may be stored in each of the encoding equipment 200 and the decoding equipment 300, in which case the reduced factor or reduced coefficient may not transmitted separately in service signals.

[140] The size of the reduced transformation matrix according to the example may be RxN, which is smaller than the NxN size of the conventional transformation matrix, and may be specified in Equation 4 below.

[141] equation 4

[142] The matrix T in the reduced transform block shown in FIG. 6A may mean the TRxN matrix of Equation 4. As shown in FIG. 6A, when the reduced transform matrix TRxN is multiplied by the residual samples for the target block, the transform coefficients for the target block can be extracted.

[143] In the example, if the block size to which the transformation is applied is 8×8, and R=16 (ie, R/N=16/64=1/4), then RST according to FIG. 6A can be expressed as a matrix operation as shown in Equation 5 below. In this case, the storage and multiplication calculation can be reduced to approximately 1/4 by the reduced factor.

[144] In the present disclosure, a matrix operation can be understood as an operation of multiplying a column vector by a matrix located to the left of the column vector to obtain a column vector.

[145] equation 5

[146] In Equation 5, r1-r64 may represent residual samples for the target block, and specifically may represent transform coefficients generated by applying the primary transform. As the result of the calculation of Equation 5, the transformation coefficients ci for the target block can be extracted, and the ci extraction process can be as specified in Equation 6.

[147] equation 6

[148] As a result of calculating Equation 6, the transformation coefficients (c1-cR) for the target block can be extracted. Thus, when R=16, the transformation coefficients c1-c16 for the target block can be extracted. If instead of RST, a regular transform is applied, and a 64x64 (NxN) transform matrix is multiplied by 64x1 (Nx1) size residual samples, then only 16 (R) transform coefficients are extracted for the target block as RST is applied, although 64 ( N) transform coefficients are extracted for the target block. Since the total number of transform coefficients for the target block is reduced from N to R, the amount of data transmitted by the encoding equipment 200 to the decoding equipment 300 is reduced, so that the transmission efficiency between the encoding equipment 200 and the decoding equipment 300 can be improved.

[149] When considered in terms of the size of the transform matrix, the size of the regular transform matrix is 64×64 (NxN), but the size of the reduced transform matrix is reduced to 16×64 (RxN), so that the storage usage in case of performing RST can be reduced by R/N ratio compared to the case of performing a regular transformation. In addition, compared with the number of NxN multiplication calculations in the case of using a regular transformation matrix, using a reduced transformation matrix can reduce the number of multiplication calculations by the ratio R/N (RxN).

[150] In an example, transformer 232 of encoding equipment 200 may extract transform coefficients for a target block by performing a primary transform and a secondary RST-based transform on the residual samples for the target block. These transform coefficients may be transmitted to the inverse transformer of the decoding equipment 300, and the inverse transformer 322 of the decoding equipment 300 may extract modified transform coefficients based on the inverse reduced secondary transform (RST) for the transform coefficients, and may extract residual samples for the target block based on the inverse primary transform for modified conversion factors.

[151] The size of the inverse RST matrix TNxR according to the example is NxR, which is smaller than the NxN size of the regular inverse transform matrix, and is in transposed relationship with the reduced transform matrix TRxN shown in Equation 4.

[152] The Tt matrix in the reduced inverse transform block shown in FIG. 6B may denote the inverse RST matrix TRxNT (superscript T denotes transpose). When the inverse RST matrix TRxNT is multiplied by the transform coefficients for the target block, as shown in FIG. 6B, modified transform coefficients for the target block or residual samples for the target block may be extracted. The inverse RST matrix TRxNT can be expressed as (TRxNT)NxR.

[153] More specifically, when the inverse RST is applied as a secondary inverse transform, modified transform coefficients for the target block can be extracted when the inverse RST matrix TRxNT is multiplied by the transform coefficients for the target block. Meanwhile, the inverse RST can be used as the inverse primary transform, in which case, the residual samples for the target block can be extracted when the inverse RST matrix TRxNT is multiplied by the transform coefficients for the target block.

[154] In the example, if the block size to which the inverse transform is applied is 8×8, and R=16 (ie, R/N=16/64=1/4), then RST according to FIG. 6B can be expressed as a matrix operation, as shown in Equation 7 below.

[155] equation 7

[156] In Equation 7, c1-c16 may represent conversion coefficients for the target block. As a result of the calculation of Equation 7, rj representing the modified transform coefficients for the target block or residual samples for the target block may be extracted, and the rj extraction process may be as specified in Equation 8.

[157] equation 8

[158] As a result of computing Equation 8, r1-rN representing modified transform coefficients for the target block or residual samples for the target block can be extracted. When considered in terms of the size of the inverse transform matrix, the size of the regular inverse transform matrix is 64×64 (NxN), but the size of the reduced inverse transform matrix is reduced to 64×16 (RxN), so that the storage usage in case of performing inverse RST may be reduced on the R/N ratio compared to the case of performing a regular inverse transformation. In addition, compared with the number of NxN multiplication calculations in the case of using a regular inverse transform matrix, using a reduced inverse transform matrix can reduce the number of multiplication calculations by the R/N ratio (NxR).

[159] Meanwhile, the configuration of conversion sets shown in Table 2 can also be applied to the 8x8 RST. Thus, RST 8x8 can be applied according to the transformation set in Table 2. Since one transformation set includes two or three transformations (kernels) according to the internal prediction mode, the ability to select one of up to four transformations, including including the case if the secondary transformation is not applied. In a transformation in which the secondary transformation is not applied, the possibility of using the identity matrix may be considered. Provided that indexes 0, 1, 2 and 3, respectively, are assigned to the four transformations (for example, index 0 may be allocated to the case in which the identity matrix is applied, i.e., the case in which the secondary transformation is not applied), the NSST index as a syntax element may be signaled for each block of transform coefficients, thereby indicating the transform that is to be applied. Thus, through the NSST index, one can denote the NSST 8x8 for the top left 8x8 block and denote the RST 8x8 in the RST configuration. NSST 8x8 and RST 8x8 mean transformations applicable to an 8x8 region included in a block of transform coefficients when both W and H of the target block to be transformed are equal to or greater than 8, and the 8x8 region can represent the top left 8x8 region in the transform coefficient block. Similarly, NSST 4x4 and RST 4x4 mean transformations applicable to a 4x4 region included in a block of transform coefficients when both W and H of the target block are equal to or greater than 4, and the 4x4 region may represent the upper left 4x4 region in the transform coefficient block.

[160] According to an embodiment of the present disclosure, for transformation in the encoding process, only 48 pieces of data can be selected, and a maximum matrix of transformation kernels of 16x48 can be applied to it, instead of applying a matrix of transformation kernels of 16x64 to 64 pieces of data forming 8x8 area. Here, "maximum" means that m has a maximum value of 16 in the mx48 matrix of transformation kernels to form m coefficients. Thus, when RST is performed by applying an mx48 transformation kernel matrix (m≤16) to an 8×8 area, 48 data pieces are input and m coefficients are generated. When m is 16, 48 pieces of data are input and 16 coefficients are generated. Thus, given that 48 pieces of data form a 48×1 vector, the 16×48 matrix and the 48×1 vector are sequentially multiplied, thereby forming a 16×1 vector. Here, 48 pieces of data forming an 8x8 area can be properly arranged, thereby forming a 48x1 vector. Here, when the matrix operation is performed by applying a maximum 16×48 transformation kernel matrix, 16 modified transformation coefficients are generated, and 16 modified transformation coefficients can be placed in the left upper 4×4 area according to the scanning order, and the right upper 4×4 area and the left lower the 4x4 area can be filled with zeros.

[161] For inverse transformation in the decoding process, a transposed matrix of the above transformation kernel matrix can be used. Thus, when the inverse RST or LFNST is performed in the inverse transform process performed by the decoding equipment, the input coefficient data to which the inverse RST is applied is configured in a one-dimensional vector according to a predetermined arrangement order, and the modified coefficient vector obtained by multiplying the one-dimensional vector and corresponding inverse RST matrix to the left of the one-dimensional vector, can be placed in a two-dimensional block according to a predetermined arrangement order.

[162] In general, in the transformation process, when RST or LFNST is applied to the 8×8 area, the matrix operation of 48 transformation coefficients in the upper left, upper right and lower left areas for the 8×8 area, excluding the lower right area from the coefficients transformations in an 8x8 domain and a 16x48 matrix of transformation kernels. For matrix operation, 48 transformation coefficients are entered in a one-dimensional array. When the matrix operation is performed, 16 modified transform coefficients are extracted, and the modified transform coefficients can be placed in the upper left area for the 8x8 area.

[163] Conversely, in the inverse transform process, when the inverse RST or LFNST is applied to the 8×8 region, 16 transform coefficients corresponding to the left upper region for the 8×8 region of the transform coefficients in the 8×8 region can be input in a one-dimensional array according to scan order and can undergo matrix operation with a 48x16 matrix of transformation kernels. Thus, the matrix operation can be expressed as (48×16 matrix) * (16×1 vector of transform coefficients)=(48×1 vector of modified transform coefficients). Here, the vector nx1 can be interpreted as having identical meaning to the matrix nx1 and can therefore be expressed as a column vector nx1. Additionally, * denotes matrix multiplication. When the matrix operation is performed, the 48 modified transform coefficients can be extracted, and the 48 modified transform coefficients can be placed in the upper left, upper right, and lower left areas of the 8x8 area excluding the lower right area.

[164] When the secondary inverse transform is based on RST, the inverse transformer 235 of the encoding equipment 200 and the inverse transformer 322 of the decoding equipment 300 may include an inverse shortened secondary transformer to extract modified transform coefficients based on the inverse RST for the transform coefficients, and an inverse primary transformer to extract residual samples for the target block based on the inverse primary transform for the modified transform coefficients. Reverse the primary transformation means the inverse transformation of the primary transformation applied to the remainder. In the present disclosure, extracting a transform coefficient based on a transform may mean extracting a transform coefficient by applying a transform.

[165] Meanwhile, according to an embodiment, block differential pulse code modulation (BDPCM) technology may be used. BDPCM may also be called "RDPCM (quantized remainder delta pulse code modulation)".

[166] When predicting a block by applying BDPCM, the reconstructed samples are used to predict the row or column of the block line by line. In this case, the reference pixels used may be unfiltered samples. The BDPCM direction may indicate whether prediction is used in the vertical direction or in the horizontal direction. The prediction error is quantized in the spatial domain, and the pixel is reconstructed by summing the dequantized prediction error with the prediction samples. As an alternative to this BDPCM, a BDPCM in the quantized residual domain may be proposed, and the prediction direction or signaling may be identical to the BDPCM applied to the spatial domain. In this way, the quantization coefficient itself can be accumulated, for example in DPCM (Delta Pulse Code Modulation) via BDPCM in the quantized residual region, and then the residual can be recovered through dequantization. Therefore, BDPCM in the quantized residual domain can be used in the sense of applying DPCM in residual coding. The quantized residual region used below is the quantized residuals without transformation in which the residuals extracted based on the prediction are quantized, and means the region for quantized residual samples.

[167] For a block of size M (row) x N (column), assume that the prediction residuals obtained by performing intra-prediction in the horizontal direction (copying the left adjacent pixel line into the prediction block line by line) or intra-prediction in the vertical direction (copying top adjacent line into a line-by-line prediction block) using unfiltered samples from the left or top edge samples, are r _{(i, j)} (0≤i≤M-1, 0≤j≤N-1). Also, suppose that the quantized version of the remainder r _{(i, j)} is Q(r _{(i, j)} ) (0≤i≤M-1, 0≤j≤N-1). Here, the remainder means the difference value between the value of the original block and the value of the prediction block.

[168] Thereafter, when BDPCM is applied to the quantized residual samples, the modified array MxN configured with , is extracted.

[169] When vertical BDPCM is transmitted in overhead signals, is as follows.

[170] equation 9

[171] In a similar application to horizontal forecasting, the residual quantized samples are as follows.

[172] equation 10

[173] Quantized residual sample ( ) is transmitted to the decoding device.

[174] In the decoding equipment, to extract Q (r _{(i, j)} ) (0≤i≤M-1, 0≤j≤N-1), the above operation is performed in reverse order.

[175] For vertical forecasting, the following equation can be applied.

[176] equation 11

[177] In addition, the following equation can be applied to horizontal forecasting.

[178] equation 12

[179] The dequantized quantized remainder Q-1(Q(r _{(i, j)} )) is added to the predicted value of the inner block to extract the reconstructed sample value.

[180] The main advantage of this technology is that inverse BDPCM can be performed immediately by simply adding predictors during coefficient parsing or after parsing during coefficient parsing.

[181] As described above, BDPCM may be applied to the quantized residual region, and the quantized residual region may include the quantized residuals (or quantized residual coefficients), at which time a skip transform may be applied to the residuals. Thus, for the residual sample, the transformation is skipped and quantization can be applied. Alternatively, the quantized residual region may include quantized transform coefficients. A flag for whether or not BDPCM is applicable may be signaled at the sequence level (SPS), and this flag may be signaled only when signaled that the skip conversion mode is activated in the SPS.

[182] When BDPCM is applied, intra prediction for the quantized residual region is performed for the entire block by selective copying according to a prediction direction similar to the intra prediction direction (eg, vertical prediction or horizontal prediction). The remainders are quantized, and the delta values, i.e. difference values ( ), are encoded between the quantized residuals and the predictors in the horizontal or vertical direction (i.e., the quantized residual in the horizontal or vertical direction).

[183] If BDPCM is applicable when the CU size is less than or equal to MaxTsSize (maximum transform skip size) for luminance signal samples, and the CU is encoded by intra prediction, flag information may be transmitted at the CU layer. Here, MaxTsSize means the maximum block size to enable skip conversion mode. This flag information indicates whether traditional intracoding or BDPCM is used. When BDPCM is applied, a BDPCM prediction direction flag indicating whether the prediction direction is a horizontal direction or a vertical direction may be transmitted. The block is then predicted through a traditional horizontal or vertical intra-forecasting process using unfiltered reference samples. The residues are quantized, and the difference value between each quantized residue and its predictor, for example, the already quantized residue of an adjacent position in the horizontal or vertical direction according to the BDPCM prediction direction, is encoded.

[184] The syntactic elements for the above-described content and their semantics are shown in the table as follows.

[185] Tab. 3

7.3.2.3. RBSP syntax for sequence parameter sets

7.4.3.3. RBSP semantics of sequence parameter sets

- sps_bdpcm_enabled_flag equal to 1 indicates that intra_bdpcm_luma_flag and intra_bdpcm_chroma_flag may be present in the coding unit syntax for intra coding units; sps_bdpcm_enabled_flag equal to 0 indicates that intra_bdpcm_luma_flag and intra_bdpcm_chroma_flag are not present in the coding unit syntax for intra coding units. When not present, the value of sps_bdpcm_enabled_flag is logically output to 0.

[186] Table 3 shows "sps_bdpcm_enabled_flag" signaled in the Sequence Parameter Set (SPS), and when the syntax element "sps_bdpcm_enabled_flag" is 1, flag information indicating whether or not BDPCM is applied to a coding unit in which intra prediction is executed, i.e. "intra_bdpcm_luma_flag", and "intra_bdpcm_chroma_flag", are present in the coding unit.

[187] If the syntax element "sps_bdpcm_enabled_flag" does not exist, its value is inferred to be 0.

[188] Tab. 4

7.3.10.5. Coding Unit Syntax

7.4.11.5. The semantics of an intra_bdpcm_luma_flag coding unit equal to 1 indicates that BDPCM is applied to the current luminance coding block at location (x0, y0), i.e. the conversion is skipped, the internal luminance prediction mode is indicated by intra_bdpcm_luma_dir_flag; intra_bdpcm_luma_flag equal to 0 indicates that BDPCM is not applied to the current luma coding block at location (x0, y0).

When intra_bdpcm_luma_flag is not present, it is logically set to 0.

The variable BdpcmFlag[x][y][cIdx] is set to intra_bdpcm_luma_flag for x=x0...x0+cbWidth-1, y=y0...y0+cbHeight-1 and cIdx=0.

[189] - intra_bdpcm_luma_dir_flag equal to 0 indicates that the BDPCM forecast direction is horizontal; intra_bdpcm_luma_dir_flag equal to 1 indicates that the BDPCM forecast direction is vertical. The variable BdpcmDir[x][y][cIdx] is set to intra_bdpcm_luma_dir_flag for x=x0...x0+cbWidth-1, y=y0...y0+cbHeight-1 and cIdx=0.

- intra_bdpcm_chroma_flag equal to 1 indicates that BDPCM is applied to the current chroma coding blocks at location (x0, y0), i.e. the conversion is skipped, the internal chroma prediction mode is indicated by intra_bdpcm_chroma_dir_flag; intra_bdpcm_chroma_flag equal to 0 indicates that BDPCM is not applied to the current chroma coding blocks at location (x0, y0).

When intra_bdpcm_chroma_flag is not present, it is logically set to 0.

The variable BdpcmFlag[x][y][cIdx] is set to intra_bdpcm_chroma_flag for x=x0...x0+cbWidth-1, y=y0...y0+cbHeight-1 and cIdx=1...2.

- intra_bdpcm_chroma_dir_flag equal to 0 indicates that the BDPCM prediction direction is horizontal; intra_bdpcm_chroma_dir_flag equal to 1 indicates that the BDPCM prediction direction is vertical. The variable BdpcmDir[x][y][cIdx] is set to intra_bdpcm_chroma_dir_flag for x=x0...x0+cbWidth-1, y=y0...y0+cbHeight-1 and cIdx=1...2.

[190] As described in Table 3, the syntax elements "intra_bdpcm_luma_flag" and "intra_bdpcm_chroma_flag" of Table 4 indicate whether or not BDPCM is applied to the current luma coding block or to the current chroma coding block. If the value of "intra_bdpcm_luma_flag" or "intra_bdpcm_chroma_flag" is 1, the transformation for the corresponding encoding block is skipped, and the prediction mode for the encoding block may be set to the horizontal or vertical direction according to "intra_bdpcm_luma_dir_flag" or "intra_bdpcm_chroma_dir_flag" indicating the prediction direction. If "intra_bdpcm_luma_flag" or "intra_bdpcm_chroma_flag" is not present, this value is logically output as 0.

[191] If intra_bdpcm_luma_dir_flag" or "intra_bdpcm_chroma_dir_flag" indicating the prediction direction is 0, it indicates that the BDPCM prediction direction is a horizontal direction, and if the value is 1, it indicates that the BDPCM prediction direction is a vertical direction direction.

[192] The intra prediction process based on the flag information is shown in the table as follows.

[193] Tab. 5

3.4.3. Extraction process for intraprediction mode of luminance signals

The inputs to this process are as follows:

- luminance signal location (xCb, yCb) indicating the upper left sample of the current luminance encoding block relative to the upper left luminance signal sample of the current frame,

- cbWidth variable indicating the width of the current encoding block in luminance signal samples,

- cbHeight variable indicating the height of the current encoding block in luma samples. In this process, the internal prediction mode of luminance signals IntraPredModeY[xCb][yCb] is extracted.

Table 19 indicates the value for the internal prediction mode IntraPredModeY[xCb][yCb] and associated names.

Table 19. Specification of internal forecasting mode and associated names

Internal prediction mode Associated title 0 INTRA_PLANAR 1 INTRA_DC 2...66 INTRA_ANGULAR2...INTRA_ANGULAR66 81...33 INTRA_LT_CCLM, INTRA_L_CCLM, INTRA_T_CCLM

Note. The INTRA_LT_CCLM, INTRA_L_CCLM and INTRA_T_CCLM intra prediction modes are applicable only to the chrominance signal component.

- IntraPredModeY[xCb][yCb] is retrieved as follows:

- If intra_luma_not_planar_flag[xCb][yCb] is 0, IntraPredModeY[xCb][yCb] is set to INTRA_PLANAR.

- Otherwise, if BdpcmFlag[xCb][yCb][0] is equal to 1, IntraPredModeY[xCb][yCb] is set to BdpcmDir[xCb][yCb][0]? INTRA_ANGULAR50:INTRA_ANGULAR18.

[194] Table 5 shows the process of extracting the intra prediction mode, and the intra prediction mode (IntraPredModeY[xCb][yCb]) is set to INTRA_PLANAR according to "Table 19" when intra_luma_not_planar_flag[xCb][yCb] is 0, and when intra_luma_not_planar_flag. If [xCb][yCb] is 1, it can be set to vertical mode (INTRA_ANGULAR50) or horizontal mode (INTRA_ANGULAR18) according to the variable BdpcmDir[xCb][yCb][0].

[195] The variable BdpcmDir[xCb][yCb][0] is set to the value intra_bdpcm_luma_dir_flag or intra_bdpcm_chroma_dir_flag, as shown in Table 4. Accordingly, the intra prediction mode can be set to the horizontal mode if the variable BdpcmDir[xCb][yCb][0] is 0, and vertical mode if it is 1.

[196] In addition, when BDPCM is applied, the dequantization process can be represented as shown in Table 6.

[197] Tab. 6

8.4.3. Scaling Process for Conversion Factors

The inputs to this process are as follows:

- luminance signal location (xTbY, yTbY) indicating the top left sample of the current luminance transform block relative to the top left sample of the luminance signal of the current frame,

- nTbW variable indicating the width of the conversion block,

- nTbH variable indicating the height of the transformation block,

- predMode variable indicating the coding unit prediction mode,

- cIdx variable indicating the color component of the current block.

The output of this process is an array d(nTbW)x(nTbH) of scaled transform factors with elements d[x][y].

To extract scaled transformation coefficients d[x][y] with x=0...nTbW-1, y=0...nTbH-1, the following applies:

- When BdpcmFlag[xTbY][yYbY][cIdx] is equal to 1, dz[x][y] is modified as follows:

- If BdpcmDir[xTbY][yYbY][cIdx] is 0 and x is greater than 0, the following applies:

- dz[x][y]=Clip3(CoeffMin, CoeffMas, dz[x-1][y]+dz[x][y]) (1174)

- Otherwise, if BdpcmDir[xTbY][yTbY][cIdx] is 1 and y is greater than 0, the following applies:

- dz[x][y]=Clip3(Coeff, CoeffMax, dz[x][y-1]+dz[s][y]) (1175)

- The dnc[x][y] value is retrieved as follows:

- dnc[x][y]=(dz[x][y] *ls[x][y]+bdOffeet)>>bdShift (1176)

- The scaled transformation factor d[x][y] is extracted as follows:

- d[x][y]=Clip3(CoeffMin, CoeffMax, dnc[x][y]) (1177)

[198] Table 6 shows the dequantization process for transform factors ("8.4.3. Scaling Process for Transform Factors"). If the value of BdpcmFlag[xTbY][yYbY][cIdx] is 1, the dequantized residual (d[x][y]) can be extracted based on the intermediate variable dz[x][y]. If BdpcmDir[xTbY][yYbY][cIdx] is equal to 0, i.e. when internal prediction is performed by horizontal mode, the variable dz[x][y] is "dz[x-1][y]+dz[x][y]". In addition, if BdpcmDir[xTbY][yYbY][cIdx] is equal to 1, i.e. when internal prediction is performed by vertical mode, the variable dz[x][y] is "dz[x][y-1]+dz[x][y]". Thus, the remainder at a specific location can be retrieved based on the sum of the remainder at the previous location in the horizontal or vertical direction and the value taken as the residual information at the specific location. This is because when BDPCM is applied, the difference between the residual sample value at a particular position (x, y) (x increases from left to right as the horizontal coordinate, y increases from top to bottom as the vertical coordinate, and the position in the two-dimensional block is expressed as ( x, y).In addition, the particular position represents the position (x, y) when the upper left position of the corresponding transform block is specified (0, 0)), and the residual sample value at the previous position ((x-1, y) or ( x, y-1)) in the horizontal or vertical direction is transmitted in service signals as residual information.

[199] Meanwhile, according to the example, when BDPCM is applied, the inverse secondary transform, which is an inseparable transform, for example, LFNST, may not be applied. Therefore, when BDPCM is applied, the LFNST index (transformation index) signaling may be omitted. As described above, it is possible to specify whether or not LFNST should be applied, and which transformation kernel matrix for LFNST should be applied, via the LFNST index. For example, if the LFNST index value is 0, it indicates that LFNST is not applied, and if the LFNST index value is 1 or 2, one of the two transformation kernel matrices constituting the LFNST transformation set, selected based on the intra prediction mode, may be indicated. More specific embodiments associated with BDPCM and LFNST may be used as follows.

[200] First embodiment

[201] BDPCM can only be applied to either the luma signal component or the chrominance signal component. In the case of separate encoding of the CTU split tree for the luma component and the CTU split tree for the chroma component (for example, a bitree structure in the VVC standard), provided that BDPCM is applied only to the luma component, the LFNST index may be transmitted only when BDPCM is not applied to the luma component and the LFNST index may be transmitted to all blocks in which LFNST may be applied to the chroma component. On the other hand, in a bitree structure, under the condition that BDPCM is applied only to the chroma component, the LFNST index can be transmitted only when BDPCM is not applied to the chrominance component, and LFNST indices can be transmitted for all blocks. to which LFNST can be applied for the luminance signal component.

[202] Second Embodiment

[203] When the luma component and the chroma component are encoded with an identical CTU split tree, i.e. When they share a partition type (eg, a single tree structure in the VVC standard), in a block to which BDPCM is applied, LFNST may not apply to both the luma and chroma components. Alternatively, the ability to apply LFNST to only one component (eg, luma component or chrominance component) for the block to which the BDPCM is applied may be configured. In this case, only the LFNST index for the corresponding component can be encoded and signaled.

[204] Third Embodiment

[205] When BDPCM is applied only to a specific type of image or sub-image (eg, internal projection image, internal slice, etc.), the ability to apply BDPCM only to the image or partial image of the corresponding type can be configured. For an image or partial image to which BDPCM applies, a LFNST index may be transmitted for each block to which BDPCM does not apply, and for a type or partial image to which BDPCM does not apply, the LFNST index for all blocks to which it may apply LFNST, transferable. Here, the block may be an encoding block or a transform block.

[206] Fourth Embodiment

[207] BDPCM can only be applied to blocks of a specific size or smaller. For example, the ability to apply BDPCM only when the block has a width W or less and a height H or less may be configured. Here, W and H can be set to 32, respectively. If the block width is W or less and the height is H or less so that BDPCM can be applied, the LFNST index can be transmitted only when the flag indicating whether or not BDPCM is applied is coded as 0 (when BDPCM is not applied) .

[208] On the other hand, when the block width exceeds W or the height exceeds H, since BDPCM is not applied, signaling a flag indicating whether or not BDPCM is applied is optional, and the LFNST index may be transmitted for all blocks. to which LFNST may apply.

[209] Fifth Embodiment

[210] Combinations of the above first to fourth embodiments may be used. For example, 1) BDPCM applies only to the luminance signal component, 2) BDPCM applies only to internal slices, 3) BDPCM is configured to apply only when both the width and height are 32 or less and the LFNST index can be encoded and be signaled for blocks to which BDPCM applies.

[211] The following drawings are provided to describe specific examples of the present disclosure. While specific device designations or designations of particular signals/messages/fields illustrated in the drawings are provided for illustrative purposes, the technical features of the present disclosure are not limited to the specific designations used in the drawings below.

[212] FIG. 7 is a flowchart illustrating operation of image decoding equipment according to an embodiment of the present disclosure.

[213] Each operation illustrated in FIG. 7 may be performed by the decoding equipment 300 illustrated in FIG. 3. Specifically, S710 may be performed by the entropy decoder 310 illustrated in FIG. 3, S720 may be performed by the dequantizer 321 illustrated in FIG. 3, S730 and S740 may be performed by the inverter 322 illustrated in FIG. 3, and S750 may be performed by the adder 340 illustrated in FIG. 3. The operations according to S710-S750 are based on some of the above details explained with reference to FIGS. 4 figs. 6. Therefore, a description of specific details overlapping with the details explained above with reference to FIG. 4-6, omitted or summarized.

[214] The decoding equipment 300 according to an embodiment can extract quantized transform coefficients for a target block from the bit stream (S710). In particular, the decoding equipment 300 can decode information regarding the quantized transform coefficients for the target block from the bit stream, and can extract the quantized transform coefficients for the target block based on the information regarding the quantized transform coefficients for the target block. Information regarding the quantized transform coefficients for the target block may be included in a sequence parameter set (SPS) or slice header and may include at least one of information regarding whether or not a shortened transform (RST) is applied, information regarding a shortened coefficient, information regarding the minimum transform size to which RST is applied, information regarding the maximum transform size to which RST is applied, information regarding the size of the reduced inverse transform, and information regarding a transform index indicating any of the transform kernel matrices included in the transform set.

[215] The decoding equipment 300 according to the embodiment can extract transform coefficients by dequantizing the quantized transform coefficients for the target block (S720).

[216] The decoding equipment 300 according to the embodiment can extract modified transform coefficients based on the inverse non-separable transform or inverse reduced secondary transform (RST) of the transform coefficients (S730).

[217] In an example, the inverse non-separable transform or inverse RST may be performed based on the inverse RST transform matrix, and the inverse RST transform matrix may be a non-square matrix in which the number of columns is less than the number of rows.

[218] In an embodiment, S730 may include decoding a transform index, determining whether or not a condition for applying the inverse RST is satisfied based on the transform index, selecting a matrix of transform kernels, and applying the inverse RST to the transform coefficients based on the matrix of the selected transform kernel and/or a reduced coefficient when the condition for applying the reverse RST is satisfied. In this case, the size of the inverse RST matrix can be determined based on the reduced coefficient.

[219] The decoding equipment 300 according to the embodiment can extract residual samples for the target block based on the inverse transform of the modified transform coefficients (S740).

[220] Decoding equipment 300 may perform an inverse transform on the modified transform coefficients for the target block, in which case a shortened inverse transform may be applied, or a traditional separable transform may be used as the inverse transform.

[221] The decoding equipment 300 according to the embodiment can generate reconstructed samples based on the residual samples for the target block and the prediction samples for the target block (S750).

[222] As for S730, it can be identified that the residual samples for the target block are extracted based on the inverse RST transform coefficients for the target block. In terms of the size of the inverse transform matrix, since the size of the regular inverse transform matrix is NxN, but the size of the inverse RST matrix is reduced to NxR, it is possible to reduce the storage usage in the case of performing the inverse RST by the R/N ratio compared with the storage usage in the case of perform regular conversion. Additionally, the use of an inverse RST matrix allows the number (NxR) of multiplications per R/N ratio to be reduced compared to the NxN number of multiplications when using a regular inverse transform matrix. In addition, since only R transform coefficients must be decoded when inverse RST is applied, the total number of transform coefficients for the target block may be reduced from N to R, compared to the number of transform coefficients in the case where N transform coefficients must be decoded when regular inverse transform is used, thereby increasing the efficiency of decoding. Thus, according to S730, the (inverse) conversion efficiency and decoding efficiency of the decoding equipment 300 can be improved through the inverse RST.

[223] FIG. 8 is a flowchart illustrating an image decoding method according to an embodiment of the present document.

[224] The decoding apparatus 300 receives encoding information such as BDPCM information from the bitstream (S810). In addition, the decoding equipment 300 may further receive transform skip flag information indicating whether or not transform skip is applied to the current block, transform index information for the inverse secondary transform, i.e. inverse inseparable transformation, i.e. LFNST index or MTS index information indicating the inverse primary transform transform kernel.

[225] The BDPCM information may include BDPCM flag information indicating whether or not BDPCM is applied to the current block, and direction information regarding the direction in which BDPCM is executed.

[226] If BDPCM is applied to the current block, the value of the BDPCM flag may be 1, and if BDPCM is not applied to the current block, the value of the BDPCM flag may be 0.

[227] When BDPCM is applied to the current block, the value of the skip conversion flag may be logically output as 1, and when the value of the skip conversion flag is 1, the value of the LFNST index may be logically output as 0 or not accepted. Thus, when BDPCM is applied to the current block, the transformation may not be applied to the current block.

[228] Meanwhile, the tree type of the current block may be divided into a single tree (SINGLE_TREE) or a dual tree (DUAL_TREE) depending on whether or not the luminance signal block and the corresponding luminance signal block have a separate segmentation structure. When a chroma block has a segmentation structure identical to that of a luma block, it may be represented as a single tree, and when a chrominance block has a segmentation structure different from the segmentation structure of a luma block, it may be represented as a dual tree. According to an example, BDPCM may be separately applied to a luma block or a chrominance block of the current block. If BDPCM is applied to a luma block, a transform index for the luma block may not be received, and if BDPCM is applied to a chrominance block, a transform index for the chrominance block may not be received.

[229] When the tree structure of the current block is a dual tree, BDPCM can be applied to only one component block, and even when the current block has a single tree structure, BDPCM can be applied to only one component block. In this case, the LFNST index can only be accepted for component blocks to which BDPCM does not apply.

[230] Alternatively, according to the example, BDPCM may be applied only when the width of the current block is less than or equal to the first threshold value, and the height of the current block is less than or equal to the second threshold value. The first threshold value and the second threshold value may be equal to 32 and may be set equal to the maximum height or maximum width of the transformation block in which the transformation is performed.

[231] Meanwhile, direction information for BDPCM may indicate a horizontal direction or a vertical direction, and quantization information can be extracted according to the direction information, and a prediction sample can be extracted according to the direction information.

[232] The decoding equipment 300 may extract quantized transform coefficients for the current block based on BDPCM (S820). Here, the transformation coefficients may represent the untransformed residual sample values.

[233] When BDPCM is applied to the current block, the residual information received by the decoding equipment 300 may be a quantized residual difference value. Depending on the BDPCM direction, a difference value between a quantized remainder in a previous vertical line or a previous horizontal direction line and a quantized remainder of a particular line may be received, and the decoding equipment 300 may extract the quantized remainder of a particular line by adding the quantized residual value of the previous vertical or horizontal line with difference value of the received quantized remainder. The quantized remainder can be extracted based on Equation 11 or Equation 12.

[234] The decoding equipment 300 can extract transform coefficients by performing dequantization on the quantized transform coefficients (S830) and can extract residual samples based on the transform coefficients (S840).

[235] As described above, when BDPCM is applied to the current block, the dequantized transform coefficient can be extracted as a residual sample without a transform process.

[236] The intra prediction unit 331 may perform intra prediction for the current block based on the direction in which the BDPCM is executed (S850).

[237] If the BDPCM is applied to the current block, intra prediction using it may be performed, which may mean that the BDPCM can only be applied to an intra slice or an intra block predicted in the intra mode.

[238] Intra prediction is performed based on the direction information for the BDPCM, and the intra prediction mode of the current block may be either a horizontal direction mode or a vertical direction mode.

[239] The decoding equipment 300 may generate a reconstructed frame based on the extracted residual samples and predicted samples, as indicated in S750 of FIG. 7 (S860).

[240] The following drawings are provided to describe specific examples of the present disclosure. While specific device designations or designations of particular signals/messages/fields illustrated in the drawings are provided for illustrative purposes, the technical features of the present disclosure are not limited to the specific designations used in the drawings below.

[241] FIG. 9 is a flowchart illustrating operation of video encoding equipment according to an embodiment of the present disclosure.

[242] Each operation illustrated in FIG. 9 may be performed by the encoding equipment 200 illustrated in FIG. 2. Specifically, S910 may be performed by the prediction unit 220 illustrated in FIG. 2, S820 may be performed by the subtractor 231 illustrated in FIG. 2, S930 and S940 may be performed by the converter 232 illustrated in FIG. 2, and S950 may be performed by the quantizer 233 and entropy encoder 240 illustrated in FIG. 2. Operations according to S910-S950 are based on the content portion described in FIG. 4-6. Therefore, a description of specific details overlapping with the details explained above with reference to FIG. 4-6, omitted or summarized.

[243] The encoding equipment 200 according to the embodiment can extract prediction samples based on the intra prediction mode applied to the target block (S910).

[244] The encoding equipment 200 according to the embodiment can extract residual samples for the target block (S920).

[245] The encoding equipment 200 according to the embodiment can extract transform coefficients for the target block based on the primary transform of the residual sample (S930). The primary transformation may be performed through a plurality of transformation kernels, and the transformation kernels may be selected based on the intra prediction mode.

[246] Decoding equipment 300 may perform a secondary transform or a non-separable transform, specifically an NSST, on the transform coefficients for a target block, in which case the NSST may be performed based on a reduced transform (RST) or not based on an RST. When NSST is performed based on the shortened transformation, the operation according to S940 can be performed.

[247] The encoding equipment 200 according to the embodiment can extract modified transform coefficients for a target block based on the RST transform coefficients (S940). In an example, the RST may be performed based on a reduced transform matrix or a matrix of transform kernels, and the reduced transform matrix may be a non-square matrix in which the number of rows is less than the number of columns.

[248] In an embodiment, S940 may include determining whether or not a condition for applying the RST is satisfied, generating and encoding a transform index based on the determination, selecting a transform kernel, and applying the RST to the residual samples based on a matrix of the selected transform kernel and/or reduced coefficient when the condition for applying RST is satisfied. At this time, the size of the shortened transformation matrix may be determined based on the shortened coefficient.

[249] The encoding equipment 200 according to the embodiment can extract quantized transform coefficients by performing quantization based on the modified transform coefficients for the target block, and can encode information regarding the quantized transform coefficients (S950).

[250] In particular, the encoding equipment 200 can generate information regarding the quantized transform coefficients, and can encode the generated information regarding the quantized transform coefficients.

[251] In an example, information regarding the quantized transform coefficients may include at least one of information regarding whether or not RST is applied, information regarding the shortened coefficient, information regarding the minimum transform size to which RST is applied, and information regarding the maximum size of the transform to which the RST is applied.

[252] As for S940, it can be identified that the transform coefficients for the target block are extracted based on the RST of the residual samples. In terms of the size of the transformation kernel matrix, since the size of the regular transformation kernel matrix is NxN, but the size of the reduced transformation matrix is reduced to RxN, it is possible to reduce the memory usage in the case of performing RST by the ratio R/N compared with the memory usage in the case of performing regular transformations. Additionally, the use of a matrix of reduced transform kernels allows the number (RxN) of multiplications to be reduced by the ratio R/N, compared to the number of NxN multiplications in the case of using a matrix of regular transform kernels. In addition, since only R transform coefficients are extracted when RST is applied, the total number of transform coefficients for the target block can be reduced from N to R, compared to the number of transform coefficients in the case where N transform coefficients are extracted when regular transform is applied, thereby reducing the amount of data transmitted from the encoding equipment 200 to the decoding equipment 300. Thus, according to S940, the conversion efficiency and encoding efficiency of the encoding equipment 320 can be improved through RST.

[253] FIG. 10 is a flowchart illustrating an image encoding method according to an embodiment of this document.

[254] The encoding equipment 200 may extract prediction samples for the current block based on the BDPCM (S1010).

[255] Encoding equipment 200 may extract internal prediction samples for the current block based on the particular direction in which BDPCM is performed. The specific direction may be a vertical direction or a horizontal direction, and prediction samples for the current block may be generated according to an intra prediction mode.

[256] Meanwhile, the tree type of the current block can be divided into a single tree (SINGLE_TREE) or a dual tree (DUAL_TREE) depending on whether or not the luminance signal block and the corresponding luminance signal block have a separate segmentation structure. When a chroma block has a segmentation structure identical to that of a luma block, it may be represented as a single tree, and when a chrominance block has a segmentation structure different from the segmentation structure of a luma block, it may be represented as a dual tree. According to an example, BDPCM may be separately applied to a luma block or a chrominance block of the current block.

[257] When the tree structure of the current block is a dual tree, BDPCM can be applied to only one component block, and even when the current block has a single tree structure, BDPCM can be applied to only one component block.

[258] Alternatively, according to the example, BDPCM may be applied only when the width of the current block is less than or equal to the first threshold value, and the height of the current block is less than or equal to the second threshold value. The first threshold value and the second threshold value may be equal to 32 and may be set equal to the maximum height or maximum width of the transformation block in which the transformation is performed.

[259] The encoding equipment 200 may extract residual samples for the current block based on the predicted samples (S1020) and perform quantization on the residual samples (S1030).

[260] Then, the encoding equipment 200 can extract the quantized residual information based on the BDPCM (S1040).

[261] The encoding equipment 200 may extract the quantized residual sample of a particular line and the difference value between the quantized residual sample of the previous vertical or horizontal line and the quantized residual sample of the particular line as quantized residual information. Thus, the difference value of the quantized residual rather than the normal residual is generated as residual information and can be extracted based on Equation 9 or Equation 10.

[262] The encoding equipment 200 may encode the quantized residual information and encoding information for the current block (S1050).

[263] The encoding apparatus 200 may encode BDPCM information, transform skip flag information indicating whether or not transform skip is applied to the current block, transform index information for inverse secondary transform, i.e. inverse inseparable transformation, i.e. LFNST index or MTS index information indicating the inverse primary transform transform kernel.

[264] The BDPCM information may include BDPCM flag information indicating whether or not BDPCM is applied to the current block, and direction information regarding the direction in which the BDPCM is executed.

[265] When BDPCM is applied to the current block, the value of the BDPCM flag may be encoded as 1, and when BDPCM is not applied to the current block, the value of the BDPCM flag may be encoded as 0.

[266] If BDPCM is applied to the current block, the value of the conversion skip flag may be logically output as 1 or encoded as 1. In addition, if the value of the conversion skip flag is 1, the LFNST index value may be logically output as 0 or may not be coded. Thus, when BDPCM is applied to the current block, the transformation may not be applied to the current block.

[267] In addition, as described above, when the tree structure of the current block is a dual tree, the BDPCM can be applied to only one component block, and even when the current block has a single tree structure, the BDPCM can be applied to only one component block. In this case, the LFNST index can only be encoded for component blocks to which BDPCM does not apply.

[268] The direction information for the BDPCM may indicate a horizontal direction or a vertical direction.

[269] In the present disclosure, at least one of quantization/dequantization and/or transform/de-transform may be omitted. When quantization/dequantization is omitted, the quantized conversion factor may be referred to as a "transformation factor". When the transform/inverse transform is omitted, the conversion factor may be called the "coefficient" or "residual coefficient" or may still be called the "transform factor" for consistency of expression.

[270] In addition, in the present disclosure, the quantized transform coefficient and the transform coefficient may be referred to as the “transform coefficient” and the “scaled transform coefficient,” respectively. In this case, the residual information may include information regarding the transform coefficient(s), and the information regarding the transform coefficient(s) may be signaled through a residual encoding syntax. Transform coefficients can be extracted based on residual information (or information regarding the transform coefficient(s)), and scaled transform coefficients can be extracted through inverse transform (scaling) of the transform coefficients. Residual samples can be extracted based on the inverse transform (transform) of the scaled transform coefficients. These details may also be applied/expressed in other parts of this disclosure.

[271] In the above-described embodiments, the methods are explained on the basis of flowcharts of the method through a sequence of steps or blocks, but the present disclosure is not limited to the order of the steps, and a certain step may be performed in an order or step different from the order or step described higher, or simultaneously with another stage. Additionally, those skilled in the art will appreciate that the steps shown in the flowchart are not the only ones possible, and that another step may be included or one or more steps of the flowchart may be deleted without affecting scope of this disclosure.

[272] The above-described methods according to the present disclosure may be implemented in software form, and encoding equipment and/or decoding equipment according to the disclosure may be included in an image processing device, for example, a television, computer, smartphone, set-top box, display device and so on.

[273] When the embodiments in the present disclosure are implemented by software, the above-described methods may be implemented as modules (processes, functions, etc.) to perform the above-described functions. The modules may be stored in a memory device and may be executed by a processor. The storage device may be located inside or outside the processor and may be coupled to the processor in a variety of known ways. The processor may include an application specific integrated circuit (ASIC), another chipset, logic circuit, and/or data processing device. The storage device may include read-only memory (ROM), random access memory (RAM), flash memory, memory card, storage medium, and/or other data storage device. Thus, the embodiments described in the present disclosure may be implemented and executed in a processor, microprocessor, controller, or chip. For example, the functional modules shown in each drawing may be implemented and executed on a computer, processor, microprocessor, controller, or chip.

[274] Additionally, the decoding equipment and encoding equipment to which the present disclosure applies may be included in a multimedia broadcast device, a mobile communication terminal, a home theater system video device, a digital cinema system video device, a surveillance camera, a video chat device, a real-time communications such as video communications, mobile streaming device, storage medium, video recording camera, video-on-demand (VoD) service delivery device, over-the-top (OTT) video device, over-the-top streaming service device Internet, a three-dimensional video device, a video telephony system video device and a medical video device, and can be used for processing a video signal or a data signal. For example, an over-the-top (OTT) video device may include a game console, Blu-ray player, Internet-enabled television, home theater system, smartphone, tablet PC, digital video recorder (DVR), etc.

[275] In addition, the processing method to which the present disclosure applies may be formed in the form of a program executed by a computer and stored on a computer-readable recording medium. Multimedia data having a data structure according to the present disclosure may also be stored on a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices on which computer-readable data is stored. The computer-readable recording medium may include, for example, a Blu-ray disc (BD), a universal serial bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical storage device. . Additionally, the computer-readable recording medium includes media implemented in the form of a carrier wave (eg, Internet transmissions). In addition, the bitstream generated by the encoding method may be stored in a computer-readable recording medium or transmitted through a wired or wireless communication network. Additionally, embodiments of the present disclosure may be implemented as a computer program product through program codes, and the program codes may be executed on a computer by embodiments of the present disclosure. The program codes may be stored on a machine-readable medium.

[276] FIG. 11 illustrates the structure of a content streaming system to which the present disclosure applies.

[277] Additionally, the content streaming system to which the present disclosure applies may mainly include an encoding server, a streaming server, a web server, a media storage, a subscriber device, and a media input device.

[278] The encoding server functions to digitally compress content input from media input devices such as a smartphone, camera, video recorder, etc. to form a bitstream and transmit it to the streaming server. As another example, in case a media input device such as a smartphone, camera, video recorder, etc. directly generates a bit stream, the encoding server may be omitted. The bitstream may be generated by an encoding method or a bitstream generation method to which the present disclosure applies. In addition, the streaming server may store the bitstream temporarily during a process to transmit or receive the bitstream.

[279] The streaming server transmits multimedia data to the subscriber device based on the user's request through a web server, which functions as a tool that informs the user as to what service is provided. When a user requests a service that the user wants, the web server passes the request to the streaming server and the streaming server streams the media data to the user. In this regard, the content streaming system may include a separate control server, in which case, the control server functions to manage commands/responses between associated equipment in the content streaming system.

[280] The streaming server may receive content from a media storage and/or encoding server. For example, if the content is received from an encoding server, the content may be received in real time. In this case, the streaming server can store the bit stream for a predetermined period of time to smoothly provide the streaming service.

[281] For example, a subscriber device may include a mobile phone, a smartphone, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable media player (PMP), a navigation device, a tablet PC, a tablet PC, an ultrabook, a wearable device (for example, a watch terminal (smart watch), a glasses terminal (smart glasses), a head-mounted display (HMD)), a digital TV, a desktop computer, a digital signage system, etc. Each of the servers in the content streaming system may operate as a distributed server, in which case, data received through each server may be processed in a distributed manner.

[282] The claims disclosed herein may be combined in various ways. For example, the technical features of the method claims of the present disclosure may be combined with the ability to be implemented or performed in equipment, and the technical features of the equipment claims may be combined with the ability to be implemented or performed in the method. Additionally, the technical features of the method claims and the equipment claims can be combined with the ability to be implemented or performed in the equipment, and the technical features of the method claims and the equipment claims can be combined with the ability to be implemented or performed in the method.

Claims (40)

1. A method for decoding images, carried out by means of decoding equipment, the method comprising the steps of: determining whether BDPCM (block-based delta pulse code modulation) is applied to the current block; extracting quantized transform coefficients for the current block based on the BDPCM; extracting transform coefficients by performing dequantization on the quantized transform coefficients; And extracting residual samples based on the transformation coefficients; wherein the determination of whether BDPCM is applied to the current block includes a determination that the width of the current block is less than or equal to the first threshold value, and the height of the current block is less than or equal to the second threshold value, wherein, based on the fact that the width of the current block is less than or equal to the first threshold value, and the height of the current block is less than or equal to the second threshold value, BDPCM is applied to the current block, wherein, based on the fact that BDPCM is applied to the current block, the inverse non-separable transform is not applied to the transform coefficients, and however, based on the fact that BDPCM is applied to the current block, the transform index value for the inverse non-separable transform is logically output to 0. 2. The image decoding method of claim 1, wherein BDPCM is separately applied to the luminance signal block of the current block or to the chrominance signal block of the current block, wherein, based on the fact that BDPCM is applied to the luma signal block, a transform index for the luma signal block is not received, and however, based on the fact that BDPCM is applied to the chrominance block, a transform index for the chrominance block is not received. 3. The image decoding method of claim 1, wherein when BDPCM is applied to the current block, the value of a conversion skip flag indicating whether conversion is skipped in the current block is logically output to 1. 4. The image decoding method according to claim 1, wherein the first threshold value and the second threshold value are 32. 5. The image decoding method of claim 1, wherein quantized transform coefficients are extracted based on direction information relative to the direction in which BDPCM is performed. 6. The image decoding method of claim 5, further comprising performing intra-prediction for the current block based on the direction in which BDPCM is performed. 7. The image decoding method according to claim 6, wherein the direction information indicates a horizontal direction or a vertical direction. 8. An image encoding method carried out by means of image encoding equipment, the method comprising the steps of: extracting prediction samples for the current block based on BDPCM (block-based delta pulse code modulation); extracting residual samples for the current block based on the predicted samples; perform quantization on the residual samples; extracting quantized residual information based on the BDPCM; And encoding quantized residual information and flag information related to the BDPCM for the current block; wherein, based on the fact that the width of the current block is less than or equal to the first threshold value and the height of the current block is less than or equal to the second threshold value, the flag information is encoded, wherein, based on the fact that BDPCM is applied to the current block, the non-separable transform is not applied to the current block, and however, based on the fact that BDPCM is applied to the current block, the transform index for the non-separable transform is not encoded. 9. The image encoding method of claim 8, wherein BDPCM is separately applied to the luminance signal block of the current block or to the chrominance signal block of the current block, wherein, based on the fact that BDPCM is applied to the luma signal block, the transform index for the luma signal block is not encoded, and however, based on the fact that BDPCM is applied to the chrominance block, the transform index for the chrominance block is not encoded. 10. The image encoding method according to claim 8, wherein the first threshold value and the second threshold value are 32. 11. The image encoding method of claim 8, wherein the internal prediction samples for the current block are retrieved based on a particular direction in which BDPCM is performed, and whereby quantization for the residual samples is performed based on a specific direction. 12. The image encoding method as set forth in claim 11, wherein the specific direction includes a horizontal direction or a vertical direction. 13. A non-transitory computer readable digital storage medium, wherein the instructions on the non-transitory computer readable digital storage medium, when executed by the at least one processor, cause the at least one processor to execute the image encoding method of claim 8. 14. A method for transmitting data for an image, comprising the steps of: obtain a bitstream for the image, the bitstream being generated based on extracting prediction samples for the current block based on BDPCM (block-based delta pulse code modulation), extracting residual samples for the current block based on the prediction samples, performing quantization on the residual samples, extracting quantized residual information based on the BDPCM, and encode the quantized residual information and flag information related to the BDPCM for the current block; And transmit data containing a stream of bits, wherein, based on the fact that the width of the current block is less than or equal to the first threshold value and the height of the current block is less than or equal to the second threshold value, the flag information is encoded, wherein, based on the fact that BDPCM is applied to the current block, the non-separable transform is not applied to the current block, and however, based on the fact that BDPCM is applied to the current block, the transform index for the non-separable transform is not encoded.

Images