RU2811759C2 - Method and device for image encoding/decoding using adaptive color conversion and method for bitstream transmission - Google Patents

Abstract

FIELD: image encoding.

SUBSTANCE: color space transform information for applying color space transformation to the residual sample of the current block is obtained based on the prediction mode being the intra prediction mode and the tree type being the single tree type. The first quantization parameter for the current coding unit is retrieved. The final quantization parameter is extracted based on adding the value of the first quantization parameter for the current coding unit to the quantization parameter offset value for the current coding unit. The transform coefficient for the current coding unit is extracted based on the final quantization parameter for the current coding unit. A residual sample for the current coding unit is extracted based on the transform factor for the current coding unit. The current coding unit contains a block of luminance signal components, a block of first color components and a block of second color components.

EFFECT: increase of efficiency of image encoding.

12 cl, 10 tbl, 54 dwg

Description

Field of technology to which the invention relates

[1] The present disclosure relates to an image encoding/decoding method and apparatus. More specifically, the present disclosure relates to an image encoding/decoding method and apparatus using adaptive color conversion and a method for transmitting a bitstream generated by the image encoding method/device of the present disclosure.

State of the art

[2] Recently, the demand for high-quality, high-definition images, such as high-definition (HD) images and ultra-high-definition (UHD) images, has been increasing in various fields of technology. As the resolution and quality of image data increases, the amount of transmitted information or bits increases relatively compared to existing image data. Increasing the amount of information or bits transmitted results in increased transmission costs and storage costs.

[3] Accordingly, there is a need for high-performance image compression technology for efficiently transmitting, storing and reproducing information regarding high-quality, high-resolution images.

The essence of the invention

Technical problem

[4] The present disclosure is directed to providing an image encoding/decoding method and apparatus with improved encoding/decoding efficiency.

[5] In addition, the present disclosure aims to provide an image encoding/decoding method and apparatus for improving encoding/decoding efficiency by performing adaptive color conversion.

[6] In addition, the present disclosure is directed to providing a method for transmitting a bit stream generated by an image encoding method or apparatus according to the present disclosure.

[7] In addition, the present disclosure is directed to providing a recording medium storing a bitstream generated by an image encoding method or apparatus according to the present disclosure.

[8] In addition, the present disclosure is directed to providing a recording medium storing a bitstream that is received, decoded, and used to reconstruct an image by an image decoding apparatus according to the present disclosure.

[9] Those skilled in the art should appreciate that the technical objectives to be achieved in the present disclosure are not limited to the above technical objectives, and other technical objectives that are not described herein will be clearly understood from the description below.

Technical solution

[10] According to an aspect of the present disclosure, there is provided an image decoding method performed by an image decoding apparatus, the method including: determining a prediction mode of a current block based on prediction information obtained from a bit stream; obtaining, based on the prediction mode of the current block, a flag indicating whether or not a color space transformation is applied to the residual sample of the current block; determining a quantization parameter of the current block based on the flag; and determining a conversion coefficient (transformation coefficient) of the current block based on the quantization parameter. Herein, determination of a quantization parameter may be performed by applying a cutoff to the quantization parameter such that the value of the quantization parameter has a value in a predetermined range.

[11] In addition, according to an aspect of the present disclosure, there is provided an image decoding apparatus including a memory device and at least one processor, wherein the at least one processor is configured to determine a prediction mode of a current block based on prediction information obtained from the bit stream, obtain, based on the prediction mode of the current block, a flag indicating whether or not color space conversion is applied to the residual sample of the current block, determine a quantization parameter of the current block based on the flag, and determine a transform coefficient of the current block based on quantization parameter. Herein, a processor may be configured to determine a quantization parameter by applying clipping to the quantization parameter such that the value of the quantization parameter has a value in a predetermined range.

[12] In addition, according to an aspect of the present disclosure, there is provided an image encoding method performed by an image encoding apparatus, the method including: determining a prediction mode of a current block; determining a quantization parameter of the current block depending on whether or not a color space transformation is applied to the residual sample of the current block; determining the conversion coefficient of the current block based on the quantization parameter; and encoding, based on the prediction mode of the current block, information indicating whether or not a color space transformation is applied to the residual sample of the current block. Herein, determining a quantization parameter may be performed by applying a cutoff to the quantization parameter such that the value of the quantization parameter has a value in a predetermined range.

[13] In addition, according to another aspect of the present disclosure, a transmission method is provided for transmitting a bit stream generated by an image encoding apparatus or an image encoding method of the present disclosure.

[14] In addition, according to another aspect of the present disclosure, a computer-readable recording medium storing a bitstream generated by an image encoding method or an image encoding apparatus of the present disclosure is provided.

[15] The features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the following detailed description of the present disclosure and do not limit the scope of the present disclosure.

Advantages of the invention

[16] According to the present disclosure, it is possible to provide an image encoding/decoding method and apparatus with improved encoding/decoding efficiency.

[17] In addition, according to the present disclosure, it is possible to provide an image encoding/decoding method and apparatus capable of improving encoding/decoding efficiency by performing adaptive color conversion.

[18] In addition, according to the present disclosure, it is possible to provide a method for transmitting a bit stream generated by an image encoding method or apparatus according to the present disclosure.

[19] In addition, according to the present disclosure, it is possible to provide a recording medium storing a bit stream generated by an image encoding method or apparatus according to the present disclosure.

[20] In addition, according to the present disclosure, it is possible to provide a recording medium storing a bit stream that is received, decoded, and used to reconstruct an image by an image decoding apparatus according to the present disclosure.

[21] Those skilled in the art will appreciate that the advantages that may be achieved through the present disclosure are not limited to those specifically described above, and other advantages of the present disclosure will be more clearly understood from the description below.

Brief description of drawings

[22] FIG. 1 is a view schematically showing a video encoding system to which an embodiment of the present disclosure is applicable.

[23] FIG. 2 is a view schematically showing an image encoding device to which an embodiment of the present disclosure is applicable.

[24] FIG. 3 is a view schematically showing an image decoding apparatus to which an embodiment of the present disclosure is applicable.

[25] FIG. 4 is a view showing an image segmentation structure according to the embodiment.

[26] FIG. 5 is a view showing an embodiment of a block segmentation type according to a multi-type tree structure.

[27] FIG. 6 is a view showing a signaling mechanism for blocking information in a quadtree structure with a nested multi-type tree according to the present disclosure.

[28] FIG. 7 is a view showing an embodiment in which a CTU is segmented into multiple CUs.

[29] FIG. 8 is a view showing adjacent reference samples according to the embodiment.

[30] FIG. 9 and 10 are views showing intra prediction according to the embodiment.

[31] FIG. 11 is a view showing an embodiment of a decoding process to which ACT is applied.

[32] FIG. 12 is a view showing an embodiment of a sequence parameter set syntax table in which a syntax element associated with an ACT is signaled.

[33] FIG. 13 to 19 are views sequentially showing an embodiment of an encoding basis syntax table in which a syntax element associated with an ACT is signaled.

[34] FIG. 20 is a view showing the syntax of a coding tree according to the embodiment.

[35] FIG. 21 is a view showing a CABAC block diagram according to an embodiment for encoding one syntax element.

[36] FIG. 22-25 are views showing entropy encoding and decoding according to the embodiment.

[37] FIG. 26 and 27 are views showing examples of frame encoding and decoding procedures according to the embodiment.

[38] FIG. 28 is a view showing a hierarchical structure for a coded image according to the embodiment.

[39] FIG. 29 is a view showing a BDPCM residual sample encoding method according to the embodiment.

[40] FIG. 30 is a view showing modified quantized residual blocks generated by executing BDPCM according to the embodiment.

[41] FIG. 31 is a flowchart showing a procedure for encoding a current block by applying BDPCM in an image encoding apparatus according to the embodiment.

[42] FIG. 32 is a flowchart showing a procedure for restoring a current block by applying BDPCM in an image decoding apparatus according to an embodiment.

[43] FIG. 33 to 35 are views schematically showing syntax for signaling information regarding BDPCM.

[44] FIG. 36 to 51 are views showing a syntax table for signaling a syntax ACT element according to each individual embodiment of the present disclosure.

[45] FIG. 52 is a view showing an image decoding method according to the embodiment.

[46] FIG. 53 is a view showing an image encoding method according to the embodiment.

[47] FIG. 54 is a view showing a content streaming system to which an embodiment of the present disclosure is applicable.

Optimal mode for carrying out the invention

[48] Embodiments of the present disclosure are described in detail hereinafter with reference to the accompanying drawings so that they can be easily implemented by those skilled in the art. However, the present disclosure may be implemented in a variety of forms and is not limited to the embodiments described herein.

[49] In describing the present disclosure, if it is determined that a detailed description of a related known function or construct would cause unnecessary ambiguity to the scope of the present disclosure, the detailed description thereof is omitted. In the drawings, parts not related to the description of the present disclosure are omitted, and like reference numbers are added to like parts.

[50] In the present disclosure, when a component is “connected,” “coupled,” or “linked” to another component, this may include not only a direct connection-based relationship, but also an indirect connection-based relationship in which an intermediate component is present. In addition, when a component “includes” or “has” other components, it means that other components may be additionally included, not the exclusion of other components, unless otherwise noted.

[51] In the present disclosure, the terms “first”, “second”, etc. may be used only for the purpose of distinguishing one component from other components and does not limit the order or importance of the components unless otherwise noted. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a “second component” in another embodiment, and likewise, a second component in one embodiment may be referred to as a “first component” in another embodiment.

[52] In the present disclosure, components that are different from each other are intended to clearly describe each feature and do not mean that the components are necessarily separate. Thus, multiple components may be integrated and implemented in a single hardware or software module, or a single component may be distributed and implemented in multiple hardware or software modules. Therefore, even if not stated otherwise, such embodiments in which components are integrated or a component is distributed are also included within the scope of the present disclosure.

[53] In the present disclosure, the components described in the various embodiments do not necessarily mean essential components, and some components may be optional components. Accordingly, an embodiment consisting of a subset of the components described in the embodiment is also included within the scope of the present disclosure. In addition, embodiments including other components in addition to the components described in the various embodiments are included within the scope of the present disclosure.

[54] The present disclosure relates to image encoding and decoding, and the terms used in the present disclosure may have a general meaning commonly used in the art to which the present disclosure pertains, unless defined for the first time in the present disclosure.

[55] In the present disclosure, "video" may mean a set of images in sequence according to the passage of time. "Frame" generally means a basis representing one image at a particular time period, and a slice/tile is an encoding basis constituting a portion of a frame during encoding. One frame can consist of one or more slices/tiles. In addition, a slice/tile may include one or more coding tree units (CTUs). One frame can consist of one or more slices/tiles. One frame can consist of one or more groups of tiles. One tile group may include one or more tiles. A brick can mean a quadrangular area of CTU lines in a tile in a frame. One tile may include one or more bricks. Brick can mean a quadrangular area of CTU lines in a tile. A single tile may be segmented into multiple bricks, and each brick may include one or more CTU rows belonging to the tile. Tiles that are not segmented into many bricks can also be treated as bricks.

[56] In the present disclosure, "pixel" or "pixel" can mean the smallest unit constituting one frame (or image). In addition, "sample" can be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel/pixel value of a luma component or only a pixel/pixel value of a chroma component.

[57] In the present disclosure, a "unit" may represent a basic unit of image processing. The unit may include at least one of a specific frame area and information associated with the area. One unit may include one block of luma signals and two blocks of chrominance signals (eg, Cb, Cr). The unit may be used interchangeably with terms such as "sample array", "block" or "zone" in some cases. In general, an MxN block may include samples (or arrays of samples) or a set (or array) of transform coefficients of M columns and N rows.

[58] In the present disclosure, "current block" may mean one of "current encoding block", "current encoding unit", "target encoding block", "target decoding block" or "target processing block". When prediction is performed, "current block" may mean "current prediction block" or "target prediction block". When transform (inverse transform)/quantization (dequantization) is performed, "current block" may mean "current transform block (current transform block)" or "transform target block (transform target block)". When filtering is performed, "current block" can mean "target filtering block".

[59] In addition, in the present disclosure, "current block" may mean "the luma block of the current block" unless explicitly referred to as a chrominance block. The "current block chroma block" may be expressed by including an explicit description of the chroma block, such as "chroma block" or "current chroma block".

[60] In the present disclosure, the terms “/” and “,” are to be interpreted to indicate “and/or.” For example, the expression "A/B" and "A, B" can mean "A and/or B". Additionally, "A/B/C" and "A, B, C" may mean "at least one of A, B and/or C".

[61] In the present disclosure, the term “or” should be interpreted to indicate “and/or.” For example, the expression "A or B" may contain 1) only "A", 2) only "B", and/or 3) "both A and B". In other words, in the present disclosure, the term “or” is to be interpreted as indicating “in addition or alternatively.”

[62] Video Coding System Overview

[63] FIG. 1 is a view showing a video encoding system according to the present disclosure.

[64] A video encoding system according to an embodiment may include a source device 10 and a receiver device 20. The source device 10 may deliver encoded video and/or image information or data to the receiver device 20 in the form of a file or stream via a digital storage medium. or network.

[65] The source device 10 according to an embodiment may include a video source generator 11, an encoding equipment 12, and a transmitter 13. A receiving device 20 according to an embodiment may include a receiver 21, a decoding equipment 12, and a rendering unit 23. The encoding equipment 12 may be referred to as “video/image encoding equipment,” and the decoding equipment 12 may be referred to as “video/image decoding equipment.” The transmitting device 13 may be included in the encoding equipment 12. The receiving device 21 may be included in the decoding equipment 12. The rendering module 23 may include a display, and the display may be configured as a separate device or an external component.

[66] The video source generator 11 may obtain video/image through a video/image capturing, synthesizing, or shaping process. The video source generator 11 may include a video/image capture device and/or a video/image generation device. The video/image capturing device may include, for example, one or more cameras, video/image archives including previously captured video/images, and the like. The video/image generating apparatus may include, for example, computers, tablet computers and smartphones and may (electronically) generate video/images. For example, a virtual video/image may be generated via a computer or the like. In this case, the video/image capturing process can be replaced by the associated data generation process.

[67] The encoding equipment 12 may encode the input video/image. The encoding equipment 12 may perform a series of procedures such as prediction, transform, and quantization for compression and encoding efficiency. The encoding equipment 12 may output encoded data (encoded video/image information) in the form of a bitstream.

[68] The transmitter 13 may transmit encoded information or video/image data output in the form of a bitstream to the receiver 21 of the decoding receiver 20 via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and the like. The transmitter 13 may include an element for generating a multimedia file through a predefined file format and may include an element for transmission through a broadcast/communications network. The receiver 21 may retrieve/receive a bit stream from a storage medium or network and transmit the bit stream to the decoding equipment 12.

[69] The decoding equipment 12 can decode a video/image by performing a sequence of procedures such as dequantization, deconversion, and prediction corresponding to the operation of the encoding equipment 12.

[70] The rendering unit 23 may render the decoded video/image. The video/image prepared by rendering can be displayed through the display.

[71] General introduction to image encoding equipment

[72] FIG. 2 is a view schematically showing image encoding equipment to which an embodiment of the present disclosure is applicable.

[73] As shown in FIG. 2, the image encoding equipment 100 may include an image segmentation unit 110, a subtractor 115, a converter 120, a quantizer 130, a dequantizer 140, an inverse converter 150, an adder 155, a filter 160, a memory 170, an inter-prediction unit 180, an intra-prediction unit 185 and an entropy encoder 190. The inter-prediction module 180 and the intra-prediction module 185 may be collectively referred to as a “prediction module”. Converter 120, quantizer 130, dequantizer 140, and inverse converter 150 may be included in the residual processor. The residual processor may further include a subtractor 115.

[74] All or at least a portion of the plurality of components configuring image encoding equipment 100 may be configured by a single hardware component (eg, an encoder or processor) in some embodiments. In addition, storage device 170 may include a decoded frame buffer (DPB) and may be configured with a digital storage medium.

[75] The image segmentation module 110 may segment an input image (either a frame or a movie frame) input to the image encoding equipment 100 into one more processing unit. For example, the processing unit may be called a "coding unit (CU)". A coding unit may be obtained by recursively segmenting a coding tree unit (CTU) or largest coding unit (LCU) according to a quadtree, binary tree, and ternary tree (QT/BT/TT) structure. For example, one coding unit may be segmented into multiple coding units of greater depth based on a quadtree structure, a binary tree structure, and/or a ternary structure. To segment a coding unit, a quadtree structure may first be applied, and subsequently a binary tree structure and/or a ternary structure may be applied. The encoding procedure of the present disclosure may be performed based on a final encoding unit that is no longer segmented. The largest coding unit may be used as the final coding unit, or a coding unit of greater depth obtained by segmenting the largest coding unit may be used as the final coding unit. Here, the encoding procedure may include a prediction, transformation and reconstruction procedure, which is described below. As another example, the processing unit of the encoding procedure may be a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transformation unit may be split or segmented from the final encoding unit. The prediction unit may be a sample prediction unit, and the transformation unit may be a unit for extracting a transformation coefficient and/or a unit for extracting a residual signal from the transformation coefficient.

[76] The prediction module (inter-prediction module 180 or intra-prediction module 185) may perform prediction regarding a block to be processed (the current block) and generate a predicted block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied based on the current block or CU. The prediction module may generate various information related to prediction of the current block and transmit the generated information to the entropy encoder 190. Information regarding the prediction may be encoded in the entropy encoder 190 and output in the form of a bit stream.

[77] The intra prediction module 185 may predict the current block by reference to the samples in the current frame. Samples for reference may be located in the surroundings of the current block or may be spaced apart according to an intra-prediction mode and/or intra-prediction technology. The intra prediction modes may include a plurality of non-directional modes and a plurality of directed modes. The omnidirectional mode may include, for example, DC mode and planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the granularity of the prediction direction. However, this is merely an example; more or less directional prediction modes may be used depending on the setting. The intra prediction module 185 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[78] The inter-prediction unit 180 may extract a predicted block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0-prediction, L1-prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block present in the current frame and a temporal neighbor block present in the reference frame. The reference frame including the reference block and the reference frame including the temporary adjacent block may be identical or different. A temporary neighbor block may be referred to as a "co-located reference block", "co-located CU (colCU)", or the like. A reference frame including a temporary adjacent block may be referred to as a "co-located frame (colPic)". For example, the inter-prediction module 180 may configure a list of motion information candidates based on neighboring blocks and generate information indicating which candidate is used to retrieve the motion vector and/or reference frame index of the current block. Mutual prediction can be performed based on different prediction modes. For example, in the case of the skip mode and the merge mode, the inter-prediction unit 180 may use the motion information of a neighboring block as the motion information of the current block. In the case of the skip mode, unlike the combining mode, the residual signal may not be transmitted. In the case of the motion vector prediction (MVP) mode, the motion vector of an adjacent block can be used as a motion vector predictor, and the motion vector of the current block can be signaled by encoding the motion vector difference and an indicator for the motion vector predictor. The motion vector difference may mean the difference between the motion vector of the current block and the motion vector predictor.

[79] The prediction module can generate a prediction signal based on various prediction methods and prediction technologies described below. For example, the prediction module may not only apply intra-prediction or inter-prediction, but also simultaneously apply both intra-prediction and inter-prediction to predict the current block. A prediction method that simultaneously uses both intra prediction and inter prediction to predict the current block may be called "combined inter and intra prediction (CIIP)". In addition, the prediction module can perform intra-block copying (IBC) to predict the current block. In-block copying can be used to encode game image/video content and the like, such as screen content encoding (SCC). IBC is a method of predicting the current frame using a previously reconstructed reference block in the current frame at a location spaced from the current block by a predetermined distance. When IBC is applied, the location of the reference block in the current frame can be encoded as a vector (block vector) corresponding to a predetermined distance. IBC essentially performs prediction within the current frame, but can be performed similarly to inter-prediction in that the reference block is retrieved within the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described in the present disclosure.

[80] The prediction signal generated by the prediction module can be used to generate a reconstructed signal or generate a residual signal. The subtractor 115 may generate a residual signal (a residual block or an array of residual samples) by subtracting a prediction signal (a prediction block or an array of prediction samples) output from the prediction module from an input image signal (a source block or an array of original samples). The generated residual signal can be transmitted to the converter 120.

[81] Converter 120 can generate conversion coefficients by applying conversion technology to the residual signal. For example, the transform technology may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally nonlinear transform (CNT). ). Here, GBT means a graph-derived transformation when the relationship information between pixels is represented by the graph. CNT means the transformation obtained from the prediction signal generated using all the previously reconstructed pixels. In addition, the conversion process can be applied to square pixel blocks that are identical in size, or can be applied to blocks that are variable in size rather than square.

[82] Quantizer 130 may quantize the transform coefficients and pass them to entropy encoder 190. Entropy encoder 190 may encode the quantized signal (information regarding the quantized transform coefficients) and output a bit stream. Information regarding the quantized transform coefficients may be referred to as "residual information". Quantizer 130 may re-arrange the quantized transform coefficients in block form into one-dimensional vector form based on the scan order of the coefficients, and generate information regarding the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form.

[83] Entropy encoder 190 may implement various encoding techniques, such as, for example, exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like. Entropy encoder 190 may encode information needed to reconstruct video/images other than quantized transform coefficients (eg, syntax element values, etc.), together or separately. Encoded information (eg, encoded video/image information) may be transmitted or stored in network abstraction layer (NAL) units in the form of a bit stream. The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The signaling information, transmission information, and/or syntax elements described in the present disclosure may be encoded through the encoding procedure described above and included in the bit stream.

[84] The bitstream may be transmitted over a network or may be stored on a digital storage medium. The network may include a broadcast network and/or a communications network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and the like. A transmitter (not shown) transmitting the signal output from the entropy encoder 190 and/or a storage module (not shown) storing the signal may be included as an internal/external element of the encoding equipment 100, and alternatively, the transmitter may be included in the entropy encoder 190 .

[85] The quantized transform coefficients output from quantizer 130 can be used to generate a residual signal. For example, a residual signal (residual block or residual samples) may be recovered by applying dequantization and inverse transform to the quantized transform coefficients through dequantizer 140 and inverse transformer 150.

[86] An adder 155 adds the reconstructed residual signal with the prediction signal output from the inter-prediction module 180 or the intra-prediction module 185 so as to generate a reconstructed signal (reconstructed frame, reconstructed block, reconstructed sample array). If there is no remainder for a block to be processed, for example in a case in which the skip mode is applied, the predicted block can be used as a reconstructed block. The adder 155 may be referred to as a "recovery module" or a "recovered block generator." The generated reconstructed signal can be used to internally predict the next block to be processed in the current frame, and can be used to inter-predict the next frame through filtering, as described below.

[87] Filter 160 may enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, filter 160 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and store the modified reconstructed frame in storage device 170, namely, a DPB of storage device 170. Various filtering techniques may include, for example, deblocking filtering. , filtering based on sampled adaptive offset, adaptive loop filter, bilateral filter, etc. The filter 160 may generate various filtering-related information and transmit the generated information to the entropy encoder 190, as described below in the description of each filtering method. Information associated with filtering may be encoded by entropy encoder 190 and output in the form of a bitstream.

[88] The modified reconstructed frame transferred to the storage device 170 can be used as a reference frame in the inter-prediction module 180. When mutual prediction is applied through the image encoding equipment 100, prediction mismatch between the image encoding equipment 100 and the image decoding equipment can be eliminated, and encoding efficiency can be improved.

[89] The DPB of the storage device 170 may store the modified reconstructed frame for use as a reference frame in the inter-prediction module 180 . The storage device 170 may store block motion information from which motion information in the current frame is retrieved (or encoded) and/or motion information of blocks in a frame that have already been reconstructed. The stored motion information may be transmitted to the inter-prediction unit 180 and used as spatial neighbor block motion information or temporal neighbor block motion information. Memory 170 may store reconstructed samples of reconstructed blocks in the current frame and may transmit reconstructed samples to intra prediction module 185.

[90] General introduction of image decoding equipment

[91] FIG. 3 is a view schematically showing image decoding equipment to which an embodiment of the present disclosure is applicable.

[92] As shown in FIG. 3, the image decoding equipment 200 may include an entropy decoder 210, a dequantizer 220, an inverse converter 230, an adder 235, a filter 240, a memory 250, an inter-prediction unit 260, and an intra-prediction unit 265. The inter-prediction module 260 and the intra-prediction module 265 may be collectively referred to as a “prediction module”. A dequantizer 220 and an inverse converter 230 may be included in the residual processor.

[93] All or at least a portion of the plurality of components configuring the image decoding equipment 200 may be configured by a hardware component (eg, a decoder or processor) according to an embodiment. In addition, storage device 250 may include a decoded frame buffer (DPB) or may be configured with a digital storage medium.

[94] The image decoding equipment 200, which receives the bitstream including video/image information, can reconstruct the image by performing a process corresponding to the process performed by the image encoding equipment 100 in FIG. 2. For example, the image decoding equipment 200 may perform decoding using a processing unit used in the image encoding equipment. Thus, a decoding processing unit, for example, may be an encoding unit. A coding unit may be obtained by segmenting a coding tree unit or a largest coding unit. The reconstructed image signal decoded and output by the decoding equipment 200 can be reproduced by the playback equipment.

[95] The image decoding equipment 200 may receive a signal output from the image encoding equipment of FIG. 2 in the form of a bit stream. The received signal may be decoded through entropy decoder 210. For example, entropy decoder 210 may parse the bitstream to extract information (eg, video/image information) needed to reconstruct images (or reconstruct frames). The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The image decoding equipment may further decode the frame based on information regarding the parameter set and/or general constraint information. The signaling/receiving information and/or syntax elements described in the present disclosure may be decoded by a decoding procedure and obtained from the bit stream. For example, entropy decoder 210 decodes information in a bitstream based on an encoding method, such as exponential Golomb, CAVLC, or CABAC encoding, and the output values of the syntax elements required for image reconstruction and the quantized values of the transform coefficients for the remainder. More specifically, the entropy CABAC decoding method may take a resolution element corresponding to each syntax element in the bitstream, determine a context model using information of a target decoding syntax element, decoding information of an adjacent block and a target decoding block, or information of a symbol/resolution element decoded on a previous one. stage, and perform arithmetic decoding for the resolution element by predicting the probability of occurrence of the permission element according to the determined context model and generating a symbol corresponding to the meaning of each syntactic element. In this case, the entropy CABAC decoding method can update the context model by using the decoded symbol/grant element information for the context model of the next symbol/grant element after determining the context model. The prediction-related information from the information decoded by the entropy decoder 210 may be provided to the prediction module (inter-prediction module 260 and intra-prediction module 265), and the residual value for which entropy decoding is performed in the entropy decoder 210, i.e. quantized conversion coefficients and related parameter information may be input to the dequantizer 220. In addition, information regarding filtering from information decoded by the entropy decoder 210 may be provided to the filter 240. Meanwhile, a receiver (not shown) for receiving a signal output from the equipment image encoding device may be further configured as an internal/external element of image decoding equipment 200, or the receiver may be a component of entropy decoder 210.

[96] Meanwhile, the image decoding equipment according to the present disclosure may be referred to as “video/image/frame decoding equipment.” The image decoding equipment can be classified into an information decoder (video/image/frame information decoder) and a sample decoder (video/image/frame sample decoder). The information decoder may include an entropy decoder 210. The sample decoder may include at least one of a dequantizer 220, an inverse converter 230, an adder 235, a filter 240, a memory 250, an inter-prediction module 260, or an intra-prediction module 265.

[97] Dequantizer 220 may dequantize the quantized transform coefficients and output the transform coefficients. Dequantizer 220 may re-arrange the quantized transform coefficients in the form of a two-dimensional block. In this case, rearrangement may be performed based on the order of coefficient scanning performed in the image encoding equipment. Dequantizer 220 may perform dequantization on the quantized transform coefficients by using a quantization parameter (eg, quantization step size information) and obtain the transform coefficients.

[98] The inverse converter 230 may inversely convert the transform coefficients to obtain a residual signal (residual block, residual sample array).

[99] The prediction module may perform prediction for the current block and may generate a predicted block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoder 210, and may determine a particular intra/inter prediction mode.

[100] Identical to that described in the prediction module of the image encoding equipment 100, the prediction module can generate a prediction signal based on various prediction methods (techniques), which are described below.

[101] Intra prediction module 265 may predict the current block by reference to samples in the current frame. The description of the intra prediction module 185 applies equally to the intra prediction module 265.

[102] The inter-prediction module 260 may extract a predicted block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks or samples based on the correlation of the motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0-prediction, L1-prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block present in the current frame and a temporal neighbor block present in the reference frame. For example, the inter-prediction module 260 may configure a motion information candidate list based on neighboring blocks and extract the current block's motion vector and/or reference frame index based on the received candidate selection information. The inter-prediction may be performed based on various prediction modes, and the information regarding the prediction may include information indicating the inter-prediction mode for the current block.

[103] The adder 235 can generate a reconstructed signal (recovered frame, reconstructed block, reconstructed sample array) by adding the resulting residual signal with a prediction signal (predicted block, predicted sample array) output from a prediction module (including inter prediction module 260 and /or internal prediction module 265). If there is no remainder for a block to be processed, for example in a case in which the skip mode is applied, the predicted block can be used as a reconstructed block. The description of adder 155 is equally applicable to adder 235. Adder 235 may be referred to as a “recovery module” or a “recovered block generator.” The generated reconstructed signal can be used to internally predict the next block to be processed in the current frame, and can be used to inter-predict the next frame through filtering, as described below.

[104] Filter 240 may enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, filter 240 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and store the modified reconstructed frame in storage device 250, namely, a DPB of storage device 250. Various filtering techniques may include, for example, deblocking filtering. , filtering based on sampled adaptive offset, adaptive loop filter, bilateral filter, etc.

[105] The (modified) reconstructed frame stored in the DPB of the storage device 250 can be used as a reference frame in the inter-prediction unit 260. Memory 250 may store block motion information from which motion information in the current frame is retrieved (or decoded) and/or motion information of blocks in a frame that have already been reconstructed. The stored motion information may be transmitted to the inter-prediction unit 260 so that it is used as spatial neighbor block motion information or temporal neighbor block motion information. Memory 250 may store reconstructed samples of reconstructed blocks in the current frame and transmit the reconstructed samples to intra prediction module 265.

[106] In the present disclosure, the embodiments described in the filter 160, the inter-prediction module 180, and the intra-prediction module 185 of the image encoding equipment 100 may be equally or correspondingly applied to the filter 240, the inter-prediction module 260, and the intra-prediction module 265 image decoding equipment 200.

[107] General introduction to image segmentation

[108] The video/image encoding method according to the present disclosure can be implemented based on an image segmentation structure as follows. In particular, the prediction, residual processing ((in)transformation, (de)-quantization, etc.), syntax element encoding and filtering procedures described below can be performed on a CTU, CU (and/or TU, PU) basis ), extracted based on the image segmentation structure. The image may be segmented in units of blocks, and the block segmentation procedure may be performed in the image segmentation unit 110 of the encoding equipment. The segmentation-related information may be encoded by the entropy encoder 190 and transmitted to the image decoding equipment in the form of a bit stream. The entropy decoder 210 of the image decoding equipment can extract the block segmentation structure of the current frame based on the segmentation-related information obtained from the bitstream, and based on this, can perform a series of procedures (eg, prediction, residual processing, block/frame reconstruction, in-loop filtering etc.) to decode images.

[109] Frames may be segmented into a sequence of coding tree units (CTUs). Fig. 4 shows an example in which a frame is segmented into CTUs. A CTU may correspond to a coding tree block (CTB). Alternatively, the CTU may include a luma sample coding tree block and two coding tree blocks of corresponding chrominance samples. For example, for a frame that contains three sample arrays, the CTU may include a block of NxN luma samples and two corresponding blocks of chroma samples.

[110] CTU segmentation overview

[111] As described above, a coding unit can be obtained by recursively segmenting a coding tree unit (CTU) or a largest coding unit (LCU) according to a quadtree/binary tree/ternary tree (QT/BT/TT) structure. For example, a CTU may first be segmented into quadtree structures. Thereafter, the leaf nodes of the quadtree structure can be further segmented by a multi-type tree structure.

[112] Quadtree segmentation means that the current CU (or CTU) is segmented equally into four. Through segmentation according to the quadtree, the current CU can be segmented into four CUs having identical width and identical height. When the current CU is no longer segmented into a quadtree structure, the current CU corresponds to a leaf node of the quadtree structure. The CU corresponding to the leaf node of the quadtree structure can no longer be segmented and can be used as the final encoding unit described above. Alternatively, the CU corresponding to a leaf node of the quadtree structure may further be segmented by a multi-type tree structure.

[113] FIG. 5 is a view showing a block segmentation type according to a multi-type tree structure. Partitioning according to a multi-type tree structure may include two types of partitioning according to a binary tree structure and two types of partitioning according to a ternary tree structure.

[114] Two types of partitioning according to the binary tree structure may include vertical binary partitioning (SPLIT_BT_VER) and horizontal binary partitioning (SPLIT_BT_HOR). Vertical binary splitting (SPLIT_BT_VER) means that the current CU is split equally in half in the vertical direction. As shown in FIG. 4, through vertical binary partitioning, two CUs having a height identical to the height of the current CU and having a width that is half the width of the current CU can be formed. Horizontal binary splitting (SPLIT_BT_HOR) means that the current CU is split equally in half in the horizontal direction. As shown in FIG. 5, through horizontal binary partitioning, two CUs having a height that is half the height of the current CU and having a width identical to the width of the current CU can be formed.

[115] Two types of partitioning according to the ternary tree structure may include vertical ternary partitioning (SPLIT_TT_VER) and horizontal ternary partitioning (SPLIT_TT_HOR). In vertical ternary splitting (SPLIT_TT_VER), the current CU is split in the vertical direction in a ratio of 1:2:1. As shown in FIG. 5, through vertical ternary partitioning, two CUs having a height identical to the height of the current CU and having a width that is 1/4 of the width of the current CU, and a CU having a height identical to the height of the current CU and having a width that is half the width of the current CU. In horizontal ternary splitting (SPLIT_TT_HOR), the current CU is split horizontally in a ratio of 1:2:1. As shown in FIG. 4, through horizontal ternary partitioning, two CUs having a height that is 1/4 of the height of the current CU and having a width identical to the width of the current CU, and a CU having a height that is half the height of the current CU and having width identical to the width of the current CU.

[116] FIG. 6 is a view showing a signaling mechanism for blocking information in a quadtree structure with a nested multi-type tree according to the present disclosure.

[117] Here, the CTU is treated as the root node of a quadtree and is segmented for the first time into a quadtree structure. Information (eg, qt_split_flag) indicating whether or not quadtree splitting is performed relative to the current CU (CTU or quadtree node (QT_node)) is signaled. For example, when qt_split_flag has the first value (for example, "1"), the current CU can be segmented into a quadtree. In addition, when qt_split_flag has a second value (for example, "0"), the current CU is not split into the quadtree, but becomes a leaf node (QT_leaf_node) of the quadtree. Each quadtree leaf node can then be further segmented into multi-type tree structures. Thus, a quadtree leaf node can become a node (MTT_node) of a multi-type tree. In a multi-type tree structure, the first flag (eg, Mtt_split_cu_flag) is signaled to indicate whether or not the current node is further segmented. If the corresponding node is further split (eg, if the first flag is 1), a second flag (eg, Mtt_split_cu_vertical_flag) may be signaled to indicate the direction of the split. For example, the split direction may be a vertical direction if the second flag is 1, and may be a horizontal direction if the second flag is 0. A third flag (e.g., mtt_split_cu_binary_flag) may then be signaled to indicate what represents the split type is a binary partition type or a ternary partition type. For example, the partition type may be a binary partition-based type when the third flag is 1, and may be a ternary partition-based type when the third flag is 0. A node of a multi-type tree obtained by a binary partition or a ternary partition may further be segmented into multi-type tree structures. However, a multi-type tree node may not be segmented into quadtree structures. If the first flag is 0, the corresponding multi-type tree node is no longer split, but becomes a leaf node (MTT_leaf_node) of the multi-type tree. The CU corresponding to a leaf node of a multi-type tree can be used as the final encoding unit described above.

[118] Based on mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-tree split mode (MttSplitMode) of the CU can be retrieved as shown in Table 1 below. In the following description, the multi-tree split mode may be called "multi-tree split type" or "split type " to be short.

[119]

Table 1 MttSplitMode mtt_split_cu_vertical_flag mtt_split_cu_binary_flag SPLIT_TT_HOR 0 0 SPLIT_BT_HOR 0 1 SPLIT_TT_VER 1 0 SPLIT_BT_VER 1 1

[120] FIG. 7 is a view showing an example in which a CTU is segmented into multiple CUs by applying a multi-type tree after applying a quadtree. In fig. 7, bold block edges 710 represent quadtree-based segmentation, and remaining block edges 720 represent multi-tree segmentation. The CU may correspond to a coding block (CB). In an embodiment, the CU may include a luma sample encoder and two chroma sample encoders corresponding to the luma samples. The CB or TB size of the chroma component samples may be extracted based on the CB or TB size of the luminance component samples according to the component ratio according to the color format (chrominance signal format, e.g., 4:4:4, 4:2 :2, 4:2:0, etc.) frame/image. In the case of a 4:4:4 color format, the CB/TB size of the chrominance signal components may be set to be equal to the CB/TB size of the luma signal components. In the case of a 4:2:2 color format, the width of the CB/TB chrominance components may be set equal to half the width of the CB/TB chrominance components, and the height of the CB/TB chrominance components may be set to be equal to the height of the CB/TB luminance components. In the case of a 4:2:0 color format, the width of the CB/TB chrominance components may be set to half the width of the CB/TB luminance components, and the height of the CB/TB chrominance components may be set to half the height of the CB/TB luminance components.

[121] In the embodiment, when the CTU size is 128 based on the luminance signal unit samples, the CU size may have a size from 128x128 to 4x4, which is the same size as the CTU. In one embodiment, in the case of a 4:2:0 color format (or chroma signal format), the CB size of the chroma signals may be from 64x64 to 2x2.

[122] Meanwhile, in an embodiment, the CU size and the TU size may be identical. Alternatively, a plurality of TUs may be provided in the CU area. The TU size generally represents the size of the transform unit (TB) of component (sampled) luminance signals.

[123] The TU size may be derived based on the largest valid TB size maxTbSize, which is a predetermined value. For example, when the CU size exceeds maxTbSize, a plurality of TUs (TB) having maxTbSize may be retrieved from the CU, and conversion/reconversion may be performed in TU units (TB). For example, the largest allowable TB size for luma signals may be 64×64, and the largest allowable TB size for chrominance signals may be 32×32. If the width or height of a CB segmented according to the tree structure is greater than the largest width or height of the transform, the CB may be automatically (or implicitly) segmented until the TB size limit in the horizontal and vertical directions is satisfied.

[124] In addition, for example, when intra prediction is applied, the intra prediction mode/type may be retrieved in CU (or CB) units, and the procedure for retrieving neighboring reference samples and generating prediction samples can be performed in TU (or TB) units. In this case, one or a plurality of TUs (or TBs) may be provided in one CU (or CB) area, and, in this case, the plurality of TUs or TBs may share the same intra prediction mode/type.

[125] Meanwhile, for the multi-type tree nested quadtree encoding scheme, the following parameters may be signaled as SPS syntax elements from the image encoding equipment to the decoding equipment. For example, at least one of CTU-size, which is a parameter representing the size of the quadtree root nodes, MinQTSize, which is a parameter representing the minimum allowed size of the quadtree leaf nodes, MaxBtSize, which is a parameter representing the maximum the allowed size of the root nodes of a binary tree, MaxTtSize, which is a parameter representing the maximum allowed size of the root nodes of a ternary tree, MaxMttDepth, which is a parameter representing the maximum allowed depth of a multi-tree based partition hierarchy from a quadtree leaf node, MinBtSize, which is a parameter representing the minimum allowed size of binary tree leaf nodes, or MinTtSize, which is a parameter representing the minimum allowed size of ternary tree leaf nodes, is signaled.

[126] As an embodiment of using a 4:2:0 chrominance signal format, the CTU size may be set to 128x128 luma blocks and two 64x64 chroma blocks corresponding to luma blocks. In this case, MinOTSize can be set to 16x16, MaxBtSize can be set to 128x128, MaxTtSzie can be set to 64x64, MinBtSize and MinTtSize can be set to 4x4, and MaxMttDepth can be set to 4. Quadtree Segmentation can be applied to a CTU to form quadtree leaf nodes. A quadtree leaf node may be called a "QT leaf node". Quadtree leaf nodes can range in size from 16x16 (for example, MinOTSize) to 128x128 (for example, CTU-size). If the QT leaf node is 128x128, it may additionally not be segmented into a binary tree/ternary tree. This is due to the fact that in this case, even with segmentation, it exceeds MaxBtsize and MaxTtszie (for example, 64x64). In other cases, QT leaf nodes may further be segmented into a multi-type tree. Therefore, a QT leaf node is the root node for a multi-type tree, and a QT leaf node can have a value of 0 for the multi-type tree depth (mttDepth). If the depth of the multi-type tree reaches MaxMttdepth (for example, 4), additional segmentation may no longer be considered. If the node width of a multi-type tree is MinBtSize and less than or equal to 2xMinTtSize, then additional horizontal segmentation may not be considered. If the node height of a multi-type tree is MinBtSize and less than or equal to 2xMinTtSize, additional vertical segmentation may not be considered. When segmentation is not considered, the image encoding equipment may omit the signaling of segmentation information. In this case, the image decoding equipment can extract segmentation information with a predetermined value.

[127] Meanwhile, one CTU may include a luminance signal sample encoding unit (hereinafter referred to as a "luminance signal unit") and two chrominance signal sample encoding units corresponding thereto (hereinafter referred to as "chrominance signal units"). The above-described coding tree-based scheme may be applied equally or separately to the luma block and the chrominance block of the current CU. In particular, luma and chroma blocks in the same CTU may be segmented into an identical block tree structure, in which case, the tree structure is represented as SINGLE_TREE. Alternatively, blocks of luma and chrominance signals in a single CTU may be segmented into separate block tree structures, in which case, the tree structure may be represented as DUAL_TREE. Thus, when a CTU is segmented into dual trees, a block tree structure for a luma block and a block tree for a chrominance block may be separately present. In this case, the block tree structure for the luminance signal block may be called "DUAL_TREE_LUMA", and the block tree structure for the chrominance signal component may be called "DUAL_TREE_CHROMA". For groups of P- and B-slices/tiles, luma and chrominance blocks in the same CTU may be limited to having identical encoding tree structure. However, for groups of I-slices/tiles, luma and chroma blocks may have a separate block tree structure relative to each other. If a separate block tree structure is used, the luma CTB may be segmented into CUs based on a particular coding tree structure, and the chroma CTB may be segmented into chrominance CUs based on a different coding tree structure. Thus, this means that a CU in an I-slice/tile group to which a separate block tree structure is applied may include a luma component encoding unit or two chrominance component encoding units, and CUs of a P- or B- group. slices/tiles may include blocks of three color components (a luma component and two chrominance components).

[128] Although a structure in the form of a quadtree-based coding tree with a nested multi-type tree is described, the structure in which the CU is segmented is not limited to this. For example, a BT structure and a TT structure may be interpreted as a concept included in a multi-partitioning tree (MPT) structure, and a CU may be interpreted as segmentable through a QT structure and an MPT structure. In an example in which the CU is segmented across a QT structure and an MPT structure, a syntax element (for example, MPT_split_type) including information regarding how many blocks the leaf node of the QT structure is segmented into, and a syntax element (for example, MPT_split_mode) , including information regarding in which of the vertical and horizontal directions the leaf node of the QT structure is segmented, may be signaled to determine the segmentation structure.

[129] In another example, the CU may be segmented in a manner different from the QT structure, BT structure, or TT structure. Thus, in contrast to a smaller CU being segmented 1/4 of a larger depth CU according to the QT structure, a shallower CU being segmented 1/2 of a larger depth CU according to the BT structure, or a shallower CU being segmented by 1 /4 or 1/2 of a deeper CU according to the TT structure, a shallower CU can be segmented to 1/5, 1/3, 3/8, 3/5, 2/3 or 5/8 of a deeper CU in some cases, and the CU segmentation method is not limited to this.

[130] The multi-type tree quadtree coding block structure can provide a very flexible block segmentation structure. Due to the segmentation types supported in a multi-type tree, different segmentation patterns can potentially result in an identical encoding block structure in some cases. In the image encoding equipment and image decoding equipment, by limiting the occurrence of such redundant segmentation patterns, the data amount of segmentation information can be reduced.

[131] In addition, when encoding and decoding a video/image according to the present document, the image processing basis may have a hierarchical structure. A single frame may be divided into one or more tiles, bricks, slices, and/or groups of tiles. One slice may include one or more bricks. One brick may include one or more CTU rows in a tile. The slice may include frame bricks, where the number of bricks is an integer. One tile group may include one or more tiles. One tile may include one or more CTUs. A CTU can be segmented into one or more CUs. A tile may be a quadrilateral area consisting of specific rows of tiles and specific columns of tiles consisting of multiple CTUs in a frame. The tile group may include tiles according to a raster scan of the tiles in the frame, wherein the number of tiles is an integer. A slice header may carry information/parameter applicable to the corresponding slice (blocks in the slice). When the encoding equipment or decoding equipment has a multi-core processor, encoding/decoding procedures for a tile, slice, brick, and/or group of tiles can be performed in parallel.

[132] In the present disclosure, the entity, name or concept of a slice or group of tiles may be used interchangeably. Thus, the header of a group of tiles can be called a "slice header". As used herein, a slice may have one of the slice types including an internal (I) slice, a predictive (P) slice, and a bipredictive (B) slice. For blocks in an I-slice, inter-prediction is not used for prediction, and only intra-prediction can be used. Even so, the original sample value may be encoded and signaled without prediction. For blocks in a P-slice, intra-prediction or inter-prediction can be used. When mutual forecasting is used, only uniforecasting can be used. Meanwhile, for blocks in a B-slice, intra prediction or inter prediction can be used. When cross forecasting is used, biforecasting, which is the maximum degree, can be used.

[133] According to the characteristic (eg, resolution) of the video image or taking into account the efficiency of encoding or parallel processing, the encoding equipment can determine the dimensions of a tile/group of tiles, a brick, a slice, and the largest and smallest encoding unit. In addition, information regarding the signified or information for retrieving the signified may be included in the bit stream.

[134] Decoding equipment may obtain information indicating whether a CTU in a tile/group of tiles, a brick, a slice, or a tile of the current frame is segmented into multiple encoding units. The encoding equipment and decoding equipment signal this information only under a specific condition, thereby increasing the encoding efficiency.

[135] A slice header (slice header syntax) may include information/parameter that applies to the slice collectively. APS (APS Syntax) or PPS (PPS Syntax) may include information/parameter applicable to one or more frames together. SPS (SPS syntax) may include information/parameter applicable to one or more sequences together. VPS (VPS syntax) may include information/parameter applicable to multiple layers together. DPS (DPS syntax) may include information/parameter applicable to the entire video collectively. The DPS may include information/parameter associated with a coded video sequence (CVS) combination.

[136] In addition, for example, information regarding the segmentation and configuration of a tile/group of tiles/brick/slice may be constructed in the encoding stage via high-level syntax and transmitted in the form of a bit stream to the decoding equipment.

[137] General presentation of internal forecasting

[138] Hereinafter, this document describes in detail the intra prediction performed by the encoding equipment and decoding equipment described above. Intra-prediction may mean prediction that generates predictive samples for the current block based on reference samples in the frame (hereinafter in this document, the current frame) to which the current block belongs.

[139] This is described below with reference to FIG. 8. When intra prediction is applied to the current block 801, adjacent reference samples that should be used for intra prediction of the current block 801 can be retrieved. The adjacent reference samples of the current block may include: a total of 2xnH samples, including samples 811 adjacent to the left border of the current block having size nWxnH and samples 812 bordering the lower left portion; a total of 2xnW samples, including samples 821 adjacent to the top border of the current block, and samples 822 adjacent to the top right; and one sample 831 adjacent to the top left of the current block. Alternatively, the current block's neighboring reference samples may include a plurality of columns of top neighboring samples and a plurality of rows of left neighboring samples.

[140] In addition, adjacent reference samples of the current block may include: a total of nH samples 841 adjacent to the right boundary of the current block having size nWxnH; in total there are 851 nW samples adjacent to the lower boundary of the current block; and one sample 842 adjacent to the bottom right of the current block.

[141] However, some adjacent reference samples of the current block have not yet been decoded or may not be available. In this case, the decoding equipment may construct adjacent reference samples to be used for prediction by substituting available samples for unavailable samples. Alternatively, neighboring reference samples to be used for prediction can be constructed through interpolation of the available samples.

[142] When neighboring reference samples are retrieved, (i) the prediction sample may be retrieved based on the average or interpolation of the current block's adjacent reference samples, or (ii) the prediction sample may be retrieved based on the reference sample present in a particular direction (prediction) relative to the prediction sample from neighboring reference samples of the current block. Case (i) may be referred to as "non-directional mode" or "non-angular mode", and case (ii) may be referred to as "directional mode" or "angular mode". In addition, the prediction sample may be generated through interpolation with a second adjacent sample and a first adjacent sample that are located in the opposite direction relative to the prediction direction of the current block's intra prediction mode based on the prediction sample of the current block from the adjacent reference samples. The above case may be called "linear interpolation intraprediction (LIP)". In addition, predictive chrominance samples can be generated from the luminance samples by using a linear model. This case may be called "LM mode". In addition, the temporary prediction sample of the current block may be extracted based on the filtered neighboring reference samples, and the prediction sample of the current block may be extracted by weighting summation of the temporary prediction sample and at least one reference sample extracted according to an intra prediction mode from the existing neighboring references. samples, namely, unfiltered neighboring reference samples. The above case may be called "position-dependent intraprediction (PDPC)". In addition, a reference sample line having the highest prediction accuracy may be selected from several adjacent reference sample lines of the current block to extract a prediction sample by using a reference sample located in the prediction direction in the corresponding line. At this time, intra prediction coding can be performed by indicating (signaling) the reference sample line to be used to the decoding equipment. The above-described case may be called "multiple reference line (MRL) intra-prediction" or "MRL-based intra-prediction". In addition, the current block can be divided into vertical or horizontal sub-segments to perform intra-prediction based on an identical intra-prediction mode, and adjacent reference samples can be retrieved based on the sub-segment and used. Thus, in this case, the intra-prediction mode for the current block is equally applied to the sub-segments, and a neighboring reference sample is retrieved based on the sub-segment and used, thereby improving the intra-prediction performance in some cases. This forecasting method may be referred to as "internal subsegment-based (ISP) or ISP-based forecasting." These intra-prediction methods may be referred to as "intra-prediction types" in contrast to intra-prediction modes (eg, DC mode, planar mode, and directional mode). Types of intra-prediction may be referred to by various terms, such as "intra-prediction schemes" or "additional intra-prediction modes". For example, the intra prediction types (or additional intra prediction modes) may include at least one selected from the group of LIP, PDPC, MRL and ISP, which are described above. The general intra-prediction method, excluding specific intra-prediction types such as LIP, PDPC, MRL and ISP, may be called "normal intra-prediction type". The normal intra-prediction type may mean a case in which specific types of intra-prediction are not applied, and prediction can be performed based on the above-described intra-prediction modes. Meanwhile, post-filtering can be performed on the extracted predictive sample if necessary.

[143] In particular, the intra prediction procedure may include a step of determining an intra prediction mode/type, a step of retrieving neighboring reference samples, and a step of retrieving prediction samples based on the intra prediction mode/type. In addition, if necessary, a post-filtering step can be performed on the extracted predictive sample.

[144] Meanwhile, in addition to the above-described types of intra prediction, affine linear weighted intra prediction (ALWIP) can be used. ALWIP may be referred to as "linear weighted intra prediction (LWIP)" or "matrix weighted intra prediction" or "matrix intra prediction (MIP)". When MIP is applied to the current block, prediction samples for the current block can be retrieved by i) using adjacent reference samples subjected to an averaging procedure, ii) performing a matrix-vector multiplication procedure, and iii) additionally performing a horizontal/vertical interpolation procedure if necessary. The intra-prediction modes used for MIP may be different from the intra-prediction modes used in intra-LIP, PDPC, MRL, ISP or normal intra-forecasting. Intra-MIP prediction modes may be referred to as “intra-MIP prediction modes,” “MIP prediction modes,” or “MIP modes.” For example, various matrices and offsets used in matrix-vector multiplication may be specified according to the intra-prediction modes for MIP. Herein, the matrix may be referred to as a "weighted (MIP) matrix" and the offset may be referred to as a "(MIP)-bias vector" or a "(MIP)-shift vector". The detailed MIP method is described below.

[145] The block recovery procedure based on the intra prediction and the intra prediction module in the encoding equipment may schematically include, for example, the following described below. Step S910 may be performed by the encoding equipment intra prediction unit 185. Step S920 may be performed by a residual processor that includes at least one selected from the group of subtractor 115, converter 120, quantizer 130, dequantizer 140, and inverse converter 150 encoding equipment. In particular, step S920 may be performed by the subtractor 115 of the encoding equipment. In step S930, prediction information may be extracted by intra prediction unit 185 and may be encoded by entropy encoder 190. In step S930, residual information may be extracted by residual processor and may be encoded by entropy encoder 190. Residual information is information regarding residual samples. The residual information may include information regarding quantized transform coefficients for the residual samples. As described above, the residual samples can be extracted as transform coefficients through the encoding equipment transformer 120, and the transform coefficients can be extracted as quantized transform coefficients through the quantizer 130. Information regarding the quantized transform coefficients can be encoded by the entropy encoder 190 through a residual encoding procedure.

[146] The encoding equipment may perform intra prediction for the current block in step S910. The encoding equipment extracts the intra prediction mode/type for the current block, extracts the adjacent reference samples of the current block, and generates prediction samples in the current block based on the intra prediction mode/type and the adjacent reference samples. Herein, the procedures for determining the intra prediction mode/type, extracting adjacent reference samples, and generating prediction samples may be performed simultaneously, or either procedure may be performed before the other procedures. For example, although not shown, the encoding equipment intra prediction module 185 may include an intra prediction mode/type determining module, a reference sample extracting module, and a prediction sample extracting module. The intra prediction mode/type determining module may determine the intra prediction mode/type for the current block, the reference sample extracting module may retrieve adjacent reference samples of the current block, and the prediction sample extracting module may retrieve prediction samples of the current block. Meanwhile, when the predictive sample filtering procedure, which is described below, is executed, the intra prediction unit 185 may further include a predictive sample filter. The encoding equipment may determine a mode/type applied to the current block from a plurality of intra prediction modes/types. The encoding equipment may compare RD costs for intra prediction modes/types and determine the optimal intra prediction mode/type for the current block.

[147] Meanwhile, the encoding equipment may perform a filtering procedure on the prediction samples. Filtering predictive samples can be called "post-filtering." Through the prediction sample filtering procedure, some or all of the prediction samples may be filtered. In some cases, the procedure for filtering predictive samples may be omitted.

[148] The encoding equipment may generate residual samples for the current block based on the (filtered) prediction samples in step S920. The encoding hardware may compare the predicted samples in the original samples of the current block based on the phase and may extract residual samples.

[149] The encoding equipment may encode image information including information (prediction information) regarding intra prediction and residual information regarding residual samples in step S930. The prediction information may include intra prediction mode information and intra prediction type information. The encoding equipment may output the encoded image information in the form of a bitstream. The output bitstream may be transmitted to decoding equipment via a storage medium or a network.

[150] The residual information may include residual encoding syntax, which is described below. The encoding hardware may extract the quantized transform coefficients by transforming/quantizing the residual samples. The residual information may include information regarding the quantized transform coefficients.

[151] Meanwhile, as described above, the encoding equipment can generate a reconstructed frame (including reconstructed samples and a reconstructed block). To this end, the encoding equipment may dequantize/deconvert the quantized transform coefficients and extract the (modified) residual samples. The reason for performing dequantization/deconversion after transforming/quantizing the residual samples is to extract residual samples that are identical to the residual samples extracted by the decoding equipment as described above. The encoding equipment may generate a reconstructed block including reconstructed samples for the current block based on the predictive samples and the (modified) residual samples. Based on the reconstructed block, a reconstructed frame for the current frame can be generated. As described above, an in-loop filtering procedure may further be applied to the reconstructed frame.

[152] A video/image decoding procedure based on the intra prediction and the intra prediction module in the decoding equipment may schematically include, for example, the following described below. The decoding equipment may perform an operation corresponding to an operation performed by the encoding equipment.

[153] Steps S1010 to S1030 may be performed by the intra prediction unit 265 of the decoding equipment. The prediction information in step S1010 and the residual information in step S1040 may be obtained from the bit stream by the entropy decoder 210 of the decoding equipment. A residual processor, including a dequantizer 220 or a decoding equipment inverter 230, or both, may extract residual samples for the current block based on the residual information. In particular, the residual processor dequantizer 220 may perform dequantization based on the quantized transform coefficients extracted based on the residual information, and may extract the transform coefficients. Residual processor inverter 230 may perform inverse transform on transform coefficients and may extract residual samples for the current block. Step S1050 may be performed by the adder 235 or the decoding equipment recovery unit.

[154] Specifically, the decoding equipment can extract the intra prediction mode/type for the current block based on the received prediction information (intra prediction mode/type information) in step S1010. The decoding equipment may extract adjacent reference samples of the current block in step S1020. The decoding equipment may generate prediction samples in the current block based on the intra prediction mode/type and adjacent reference samples in step S1030. In this case, the image decoding equipment may perform a filtering procedure on the prediction samples. Filtering predictive samples can be called "post-filtering." Through the prediction sample filtering procedure, some or all of the prediction samples may be filtered. In some cases, the procedure for filtering predictive samples may be omitted.

[155] The image decoding equipment may generate residual samples for the current block based on the received residual information. The decoding equipment can generate reconstructed samples for the current block based on the predicted samples and residual samples, and can extract the reconstructed block including the reconstructed samples in step S1040. Based on the reconstructed block, a reconstructed frame for the current frame can be generated. As described above, an in-loop filtering procedure may further be applied to the reconstructed frame.

[156] Herein, although not shown, the decoding equipment intra prediction unit 265 may include an intra prediction mode/type determining unit, a reference sample extracting unit, and a prediction sample extracting unit. The intra prediction mode/type determining module may determine the intra prediction mode/type for the current block based on the intra prediction mode/type information obtained from the entropy decoder 210. The reference sample extracting module may extract adjacent reference samples of the current block. The prediction sample retrieval module may retrieve the prediction samples of the current block. Meanwhile, when the above-described procedure for filtering prediction samples is performed, the intra prediction unit 265 may further include a prediction sample filter.

[157] The intra prediction mode information may include, for example, flag information (eg, intra_luma_mpm_flag) indicating whether the most probable mode (MPM) is applied to the current block or the remaining mode is applied. When the MPM is applied to the current block, the prediction mode information may further include index information (eg, intra_luma_mpm_idx) indicating one of the intra prediction mode options (MPM options). Possible options for internal prediction modes (possible MPM options) can be constructed from a list of possible MPM options or an MPM list. In addition, when the MPM is not applied to the current block, the intra prediction mode information may further include remaining mode information (e.g., intra_luma_mpm_remainder) indicating one of the remaining intra prediction modes excluding the intra prediction mode options (MPM options) . The decoding equipment may determine the intra prediction mode of the current block based on the intra prediction mode information. A separate MPM list can be constructed for the MIP described above.

[158] In addition, intraprediction type information may be implemented in various forms. For example, the intra-prediction type information may include intra-prediction type index information indicating one of the intra-prediction types. As another example, the intra prediction type information may include at least one of reference sample line information (e.g., intra_luma_ref_idx) indicating whether or not the MRL is applied to the current block, and which reference sample line is used, when the MRL is applied to the current block, ISP flag information (for example, intra_subpartitions_mode_flag) indicating whether or not ISP is applied to the current block, ISP type information (for example, intra_subpartitions_split_flag) indicating the subsegment segmentation type when the ISP is applied, flag information, indicating whether or not PDCP is applied, or flag information indicating whether or not LIP is applied. In addition, the intra prediction type information may include a MIP flag indicating whether or not a MIP is applied to the current block.

[159] Intra prediction mode information and/or intra prediction type information may be encoded/decoded through the encoding method described herein. For example, intra prediction mode information and/or intra prediction type information may be encoded/decoded by entropy coding (eg, CABAC, CAVLC) based on a truncated binary code (Rice).

[160] Understanding Adaptive Color Conversion (ACT)

[161] Adaptive Color Conversion (ACT) is a color space conversion technology to remove unnecessary overlap between color components and is used in the HEVC version of the on-screen content extension. It can also be applied to VVC.

[162] In the HEVC extension for screen content (HEVC SCC extension), ACT is used to adaptively convert a prediction remainder from an existing color space to a YCgCo color space. One of the two color spaces may optionally be selected by signaling one ACT flag for each transform basis.

[163] For example, the first value (eg, 1) of the flag may indicate that the remainder of the transform basis is encoded in the original color space. The second value (eg, 1) of the flag may indicate that the remainder of the transform basis is encoded in YCgCo color space.

[164] FIG. 11 is a view showing an embodiment of a decoding process to which ACT is applied. In the embodiment of FIG. 11, motion compensated prediction may correspond to inter prediction in the present disclosure.

[165] As shown in FIG. 11, a reconstructed frame (or reconstructed block, reconstructed sample array, reconstructed sample(s), reconstructed signal) may be generated based on the predicted output value and the residual output value. Herein, the residual output value may be an inverse transform output value. As used herein, the inverse transform may be a normal inverse transform. As used herein, the normal inverse transform may be an MTS-based inverse transform or an inverse low frequency non-separable transform (LFNST).

[166] As used herein, the prediction output value may be a prediction block, an array of prediction samples, prediction sample(s), or a prediction signal. The residual output value may be a residual block, an array of residual samples, residual sample(s), or a residual signal.

[167] For example, from an encoding hardware perspective, the ACT process may be performed on residual samples extracted based on the predictive samples. In addition, the output value of the ACT process can be provided as input to the normal conversion process. As used herein, the normal conversion process may be an MTS- or LFNST-based conversion.

[168] Information (parameter) regarding the (inverse) ACT may be generated and encoded by the encoding equipment and may be transmitted to the decoding equipment in the form of a bitstream.

[169] The decoding equipment may receive, parse, and decode the (reverse) ACT related information (parameter), and may perform the reverse ACT based on the (reverse) ACT related information (parameter).

[170] Based on the inverse ACT, (modified) residual samples (or residual block) can be extracted. For example, the (transform) coefficients can be extracted by applying dequantization to the quantized (transform) coefficients. In addition, residual samples can be extracted by performing an inverse transformation on the coefficients (transformation). In addition, (modified) residual samples can be obtained by applying inverse ACT to the residual samples. The information (parameter) regarding the (reverse) ACT is described in detail below.

[171] In an embodiment, the base transform function used in HEVC can be used as a base transform function (conversion kernel) for color space conversion. For example, matrices for forward transformation and inverse transformation can be used, as shown in the equations below.

[172] equation 1

[173]

[174] equation 2

[175]

[176] Herein, C0, C1, and C2 may correspond to G, B, and R. Herein, G denotes a green color component, B denotes a blue color component, and R denotes a red color component. In addition, C0', C1' and C2' may correspond to Y, Cg and Co. Herein, Y denotes a luma signal, Cg denotes a green chrominance signal, and Co denotes an orange component of a chrominance signal.

[177] In addition, to compensate for the change in dynamic range of the residual before and after the color conversion, QP adjustment can be applied to the transform residual by (-5, -5, -3). The details of QP regulation are described below.

[178] Meanwhile, in the encoding and decoding processes according to the embodiment, when ACT is applicable, the following restrictions may apply.

[179] - In case of split-tree encoding/decoding, ACT is disabled. For example, ACT can only be applied to single tree based encoding/decoding.

[180] - When ISP encoding and decoding is applied, ACT may be disabled.

[181] - For a chrominance block to which BDPCM is applied, ACT may be deactivated. Only for a luminance signal block to which BDPCM is applied can ACT be activated.

[182] - When ACT is possible, CCLM may be deactivated.

[183] FIG. 12 is a view showing an embodiment of a sequence parameter set syntax table in which a syntax element associated with an ACT is signaled.

[184] FIG. 13 to 19 are views sequentially showing an embodiment of an encoding basis syntax table in which a syntax element associated with an ACT is signaled.

[185] As shown in FIG. 12, sps_act_enabled_flag 1210 may be used as an ACT enable flag indicating whether ACT is activated or not during the decoding process.

[186] The first value (eg, 0) of sps_act_enabled_flag may indicate that ACT is not used, and the flag cu_act_enabled_flag 1310, 2910 indicating whether or not ACT is used in the coding basis is not provided in the syntax for the coding basis.

[187] A second value (eg, 1) of sps_act_enabled_flag may indicate that ACT may be used, and cu_act_enabled_flag may be provided in the syntax for the encoding basis.

[188] When sps_act_enabled_flag is not obtained in the bitstream, the value of sps_act_enabled_flag may be retrieved as the first value (eg, 0).

[189] In addition, as shown in FIG. 13, cu_act_enabled_flag 1310, 2710 may be used as ACT flags indicating whether or not the remainder of the current encoding basis is encoded in the YCgCo color space.

[190] The first value (eg, 0) of cu_act_enabled_flag may indicate that the remainder of the current encoding basis is encoded in the original color space. A second value (eg, 1) of cu_act_enabled_flag may indicate that the remainder of the current encoding basis is encoded in the YCgCo color space.

[191] When cu_act_enabled_flag is not provided in the bitstream, the flag may be retrieved as the first value (eg, 0). Herein, the source color space may be an RGB color space.

[192] QP method for extracting transformation basis by using ACT QP offset

[193] In an embodiment, in the scaling process for the transform coefficient, the quantization parameter extraction process and the Qp update process may be performed as follows. For example, the process of extracting quantization parameters can be performed using the following parameters.

[194] - Luminance coordinates (xCb, yCb) indicating the relative coordinates of the upper left luminance sample of the current encoding block relative to the upper left luminance sample of the current frame,

[195] - Variable cbWidth indicating the width of the current encoding block based on luminance samples,

[196] - Variable cbHeight indicating the height of the current encoding block based on the luminance signal sample,

[197] - A treeType variable indicating whether a single tree (SINGLE_TREE) or a double tree is used to segment the current encoding tree node, and indicating when a double tree is used, whether the double tree represents a double tree (DAUL_TREE_LUMA) of signal components brightness or dual tree (DAUL_TREE_CHROMA) of chrominance signal components.

[198] In the current process, the luminance signal quantization parameter Qp' _Y , the chrominance signal quantization parameters Qp' _Cb , Qp' _Cr and Qp' _CbCr can be extracted.

[199] The variable location (xQg, yQg) of the luminance signal may indicate the location of the upper left sample of the luminance signals of the current quantization group corresponding to the upper left sample of the current frame. Herein, the horizontal location xQg and the vertical location yQg may be set to the values of the CuQgTopLeftX variable and the CuQgTopLeftY variable, respectively. The variables CuQgTopLeftX and CuQgTopLeftY can be set as predefined values in the coding tree syntax, as shown in FIG. 20.

[200] Herein, the current quantization group may be a quadrangular area in a coding tree block and may share the same qP value _{Y_PRED} . Its width and height may be equal to the width and height of the coding tree node in which the top left sample location of the luminance signals is assigned to each of CuQgTopLeftX and CuQgTopLeftY.

[201] When treeType is SINGLE_TREE or DUAL_TREE_LUMA, the luminance signal quantization parameter prediction value qP _{Y_PRED} can be extracted as specified in the following steps.

[202] 1. The variable qP _{Y_PRED} can be retrieved as follows.

[203] Condition 1: When any of the following conditions is true, the value of qP _{Y_PRED} may be set to a value identical to the value of SliceQp _Y (herein, SliceQp _Y indicates the initial value of the quantization parameter Qp _Y for all slices in the frame, and it can be obtained from a bit stream). Alternatively, the value qP _{Y_PRED} may be set equal to the value of the luminance signal quantization parameter Qp _Y of the last luminance signal encoding basis of the immediately preceding quantization group according to the decoding order.

[204] - Condition 1-1. When the current quantization group is the first quantization group in the slice

[205] - Condition 1-2. When the current quantization group is the first quantization group in the tile

[206] - Condition 1-3. When the current quantization group is the first quantization group in the CTB row in the tile and predefined synchronization occurs (for example, when entropy_coding_sync_enabled_flag is set to 1)

[207] 2. The value of the variable qP _{Y_A} can be retrieved as follows.

[208] Condition 2: When at least one of the following conditions is true, the value of qP _{Y_A} may be set equal to the value of qP _{Y_PRED} . Alternatively, the value qP _{Y_A} may be set equal to the value of the luminance signal quantization parameter Qp _Y of a coding basis that includes a luminance coding block covering a sample location (xQg-1, yQg) of the luminance signals.

[209] - Condition 2-1. Relative to the block identified by the selective location (xCb, yCb), the block identified by the selective location (xQg-1, yQg) is not an accessible neighboring block.

[210] - Condition 2-2. When the CTB including the luminance coding block covering the sample location (xQg-1, yQg) of the luminance signals is different from the CTB including the current luminance coding block at the sample location (xCb, yCb) of the luminance signals, for example, when all of the following conditions represent "true"

[211] - Condition 2-2-1. The value of (xQg-1)>>CtbLog2SizeY is different from the value of (xCb)>>CtbLog2SizeY.

[212] - Condition 2-2-2. The value of (yQg)>>CtbLog2SizeY is different from the value of (yCb)>>CtbLog2SizeY.

[213] 3. The value of the variable qP _{Y_B} can be retrieved as follows.

[214] Condition 3: When at least one of the following conditions is true, the value of qP _{Y_B} may be set equal to the value of qP _{Y_PRED} . Alternatively, the value qP _{Y_B} may be set equal to the value of the luminance signal quantization parameter Qp _Y of the coding basis, which includes a luminance coding block covering the sample location (xQg, yQg-1) of the luminance signals.

[215] - Condition 3-1. With respect to the block identified by the selective location (xCb, yCb), when the block identified by the selective location (xQg, yQg-1) is not an available neighboring block,

[216] - Condition 3-2. When the CTB including the luminance coding block covering the sample location (xQg, yQg-1) of the luminance signals is different from the CTB including the current luminance coding block at the sample location (xCb, yCb) of the luminance signals, for example, when all of the following conditions represent "true"

[217] - Condition 3-2-1. The value (xQg)>>CtbLog2SizeY is different from the value (xCb)>>CtbLog2SizeY,

[218] - Condition 3-2-2. The value of (yQg-1)>>CtbLog2SizeY is different from the value of (yCb)>>CtbLog2SizeY.

[219] 4. The predicted value qP _{Y_PRED} of the luminance signal quantization parameter can be extracted as follows.

[220] When all of the following conditions are true, qP _{Y_PRED} may be set equal to the luminance signal quantization parameter Qp _Y of the coding basis, which includes a luminance coding block covering the sample location (xQg, yQg-1) of the luminance signals.

[221] - Condition 3-1. Relative to the block identified by the selective location (xCb, yCb), the block identified by the selective location (xQg, yQg-1) is an accessible neighboring block

[222] - When the current quantization group is the first quantization group in the CTB row in the tile

[223] Meanwhile, when all conditions are not true, qP _{Y_PRED} can be extracted as shown in the equation below.

[224] equation 3

[225] qP _{Y_PRED} =(qP _{Y_A} +qP _{Y_B} +1)>>1

[226] The Qp _Y variable can be retrieved as shown in the equation below.

[227] equation 4

[228] Qp _Y =((qP _{Y_PRED} +CuQpDeltaVal+64+2*QpBdOffset)%(64+QpBdOffset))-QpBdOffset

[229] Herein, CuQpDeltaVal indicates the difference between the luminance quantization parameter of a coding basis and its predicted value. Its value can be obtained from a bit stream. QpBdOffset indicates the offset of the quantization parameter range for luma and chrominance signals. QpBdOffset can be preset to a predefined constant or derived from the bitstream. For example, QpBdOffset may be calculated by multiplying a predefined constant by the value of a syntax element that indicates the bit depth of the luma or chrominance signals to be sampled. The luminance signal quantization parameter Qp' _Y can be extracted as shown in the following equation.

[230] equation 5

[231] Qp' _Y =Qp _Y +QpBdOffset

[232] When the value of the ChromaArrayType variable indicating the type of the chroma array is not the first value (eg, 0), and treeType is SINGLE_TREE or DUAL_TREE_CHROMA, the following processing may be performed.

[233] - When the value of treeType is DUAL_TREE_CHROMA, the value of the variable Qp _Y may be set to a value identical to the value of the luminance signal quantization parameter Qp _Y of the luminance coding basis covering the sample location (xCb+cbWidth/2, yCb+cbHeight/2) of the signals brightness.

[234] - The variables qP _Cb , qP _Cr and qP _CbCr can be extracted as shown in the equation below.

[235] equation 6

[236] qP _Chroma =Clip3(-QpBdOffset, 63, Qp _Y )

[237] qP _Cb =ChromaQpTable[0][qP _Chroma ]

[238] qP _Cr =ChromaQpTable[1][qP _Chroma ]

[239] qP _CbCr =ChromaQpTable[2][qP _Chroma ]

[240] The chrominance signal quantization parameters Qp' _Cb and Qp' _Cr for the Cb and Cr components and the chrominance signal quantization parameter Qp' _CbCr for the combined Cb-Cr encoding can be extracted as shown in the equation below.

[241] equation 7

[242] Qp' _Cb =Clip3(-QpBdOffset, 63, qP _Cb +pps_cb_qp_offset+slice_cb_qp_offset+CuQpOffset _Cb )+QpBdOffset

[243] Qp' _Cr =Clip3(-QpBdOffset, 63, qP _Cr +pps_cr_qp_offset+slice_cr_qp_offset+CuQpOffset _Cr )+QpBdOffset

[244] Qp' _CbCr =Clip3(-QpBdOffset, 63, qP _CbCr +pps_joint_cbcr_qp_offset+slice_joint_cbcr_qp_offset+CuQpOffsetCbC _r )+QpBdOffset

[245] In the above equation, pps_cb_qp_offset and pps_cr_qp_offset are offsets used to extract _Qp'Cb and _Qp'Cr , and can be obtained from the bitstream for a set of frame parameters. slice_cb_qp_offset and slice_cr_qp_offset are offsets used to extract Qp' _Cb and Qp' _Cr and can be obtained from the bitstream for the slice header. CuQpOffset _Cb and CuQpOffset _Cr are offsets used to extract Qp' _Cb and Qp' _Cr and can be obtained from the bit stream for the transform basis.

[246] In addition, for example, the dequantization process for the conversion factor can be performed using the following parameters.

[247] - Luminance coordinates (xTbY, yTbY) referencing the relative coordinates of the top-left sample of the current luminance transform block relative to the top-left luminance sample of the current frame

[248] - Variable nTbW indicating the width of the conversion block

[249] - Variable nTbH indicating the height of the transform block

[250] - Variable cIdx indicating the color component of the current block

[251] The output of the current process may be an array of d scaled transform factors. In this document, the size of the array d may be (nTbW)x(nTbH). The individual elements composing it can be identified as d[x][y].

[252] For this purpose, the quantization parameter qP can be extracted as follows. When cIdx has a value of 0, qP can be extracted as shown in the equation below.

[253] equation 8

[254] qP=Qp' _Y

[255] Alternatively, when TuCResMode[xTbY][yTbY] is set to 2, the extraction can be performed as shown in the equation below.

[256] equation 9

[257] qP=Qp' _CbCr

[258] Alternatively, when cIdx has a value of 1, qP can be extracted as shown in the equation below.

[259] equation 10

[260] qP=Qp' _Cb

[261] Alternatively, when cIdx has a value of 2, qP can be extracted as shown in the equation below.

[262] equation 11

[263] qP=Qp' _Cr

[264] Thereafter, the quantization parameter qP may be updated as follows. In addition, the rectNonTsFlag and bdShift variables can be retrieved as follows. For example, when transform_skip_flag[xTbY][yTbY][cIdx] is set to 0 (for example, when the transformation of the current transform block is not skipped), the extraction can be performed as shown in the equation below. Herein, the first value (eg, 0) of transform_skip_flag may indicate that whether or not a transform is skipped is determined by another syntax element. A second value (eg, 1) of transform_skip_flag can indicate skipping (eg, skipping) the transform.

[265] equation 12

[266] qP=qP-(cu_act_enabled_flag[xTbY][yTbY]?5:0)

[267] rectNonTsFlag=0

[268] bdShift=10

[269] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] is set to 1 (eg, the transformation of the current transform block is skipped), the extraction can be performed as shown in the equation below.

[270] equation 13

[271] qP=Max(QpPrimeTsMin, qP)-(cu_act_enabled_flag[xTbY][yTbY]?5:0)

[272] rectNonTsFlag=(((Log2(nTbW)+Log2(nTbH)) and 1)==1

[273] bdShift=BitDepth+(rectNonTsFlag?1:0)+((Log2(nTbW)+Log2(nTbH))/2)-+pic_dep_quant_enabled_flag

[274] Herein, QpPrimeTsMin may indicate the minimum quantization parameter value allowed when transform skip mode is applied. It may be defined as a predefined constant, or it may be extracted from the bitstream syntax element associated with it.

[275] As used herein, the suffixes Y, Cb, and Cr may denote the G, B, and R color components in the RGB color model, or the Y, Cg, and Co color components in the YCgCo color model.

[276] Quantization/Dequantization

[277] As described above, the encoding equipment quantizer can extract quantized transform coefficients by applying quantization to the transform coefficients. An encoding equipment dequantizer or a decoding equipment dequantizer may extract transform coefficients by applying dequantization to the quantized transform coefficients.

[278] When encoding and decoding video/still image, the quantization factor can be changed, and the changed quantization factor can be used to adjust the compression ratio. According to an implementation aspect, in consideration of complexity, a quantization parameter (QP) may be used instead of using the quantization coefficient directly. For example, quantization parameters having integer values of 0-63 may be used, and each quantization parameter value may correspond to an actual quantization coefficient. In addition, the quantization parameter QP _Y for the luminance signal component (luminance signal sample) and the quantization parameter QP _C for the chrominance signal component (chrominance signal sample) can be set so that they are different from each other.

[279] In the quantization process, the transform coefficient C can be input and divided by the quantization coefficient Qstep, and based on this, the quantized transform coefficient C' can be obtained. In this case, in consideration of computational complexity, the quantization factor may be multiplied by the scale to be in the form of an integer, and the shift operation may be performed by a value corresponding to the scale value. Based on the multiplication of the quantization coefficient and the scale value, the quantization scale can be extracted. Thus, the quantization scale can be extracted according to the QP. A quantization scale can be applied to the transform coefficient C, and based on this, the quantized transform coefficient C' can be extracted.

[280] The dequantization process is the inverse process of the quantization process. The quantized transform coefficient C' can be multiplied by the quantization coefficient Qstep, and from this the reconstructed transform coefficient C'' can be obtained. In this case, the level scale can be extracted according to the quantization parameter. The level scale can be applied to the quantized transform coefficient C', and based on this, the reconstructed transform coefficient C'' can be extracted. The reconstructed transform coefficient C'' may differ slightly from the original transform coefficient C due to losses in the conversion and/or quantization process. Therefore, encoding equipment can perform dequantization in the same way as decoding equipment.

[281] Meanwhile, weighted adaptive frequency quantization technology in which the quantization intensity is adjusted according to frequency can be applied. Adaptive Weighted Frequency Quantization technology is a method of applying quantization intensities that vary depending on frequency. In adaptive weighted frequency quantization, quantization intensities varying with frequency can be applied by using a predefined quantization scaling matrix. Thus, the above-described quantization/dequantization process can be further performed based on the quantization scaling matrix. For example, different quantization scaling matrices may be used according to the size of the current block and/or whether the prediction mode applied to the current block to generate the residual signal of the current block is inter-prediction or intra-prediction. The quantization scaling matrix may be referred to as a "quantization matrix" or a "scaling matrix". The quantization scaling matrix can be predefined. In addition, for frequency adaptive scaling, frequency-specific quantization scale information for the quantization scaling matrix may be constructed/encoded by the encoding equipment and may be signaled to the decoding equipment. The frequency-specific quantization scale information may be referred to as “quantization scaling information.” The frequency-specific quantization scale information may include scaling list data (scaling_list_data). The (modified) quantization scaling matrix may be retrieved based on the scaling list data. In addition, the frequency-specific quantization scale information may include presence flag information indicating whether scale list data is present or not. Alternatively, when scaling list data is signaled at a high level (eg, SPS), information indicating whether or not the scaling list data at a lower layer (eg, PPS or tile group header, etc.) is further included.

[282] Conversion/Reverse Conversion

[283] As described above, the encoding equipment can extract a residual block (residual samples) based on a block (predictive samples) predicted through intra/inter/IBC prediction, and can extract quantized transform coefficients by applying transform and quantization to the extracted residual samples . Once included in the residual encoding syntax, information (residual information) regarding the quantized transform coefficients can be encoded and output in the form of a bitstream. The decoding equipment can obtain information (residual information) regarding the quantized transform coefficients from the bit stream, and can extract the quantized transform coefficients by performing decoding. Decoding hardware can extract residual samples through dequantization/deconversion based on the quantized transform coefficients. As described above, quantization/dequantization or transform/deconversion or both may be omitted. When the transform/inverse transform is omitted, the conversion factor may be called the "coefficient" or "residual coefficient" or may still be called the "transform factor" for consistency of expression. Whether or not a transform is skipped/transformed may be signaled based on a transform skip flag (eg, transform_skip_flag).

[284] The conversion/inverse conversion can be performed based on the conversion kernel(s). For example, a multiple transform selection (MTS) scheme may be used to perform the transform/de-transformation. In this case, a portion of several sets of transformation kernels can be selected and applied to the current block. The transformation kernel may be called by various terms such as "transformation matrix", "transformation type", etc. For example, a set of transformation kernels may mean a combination of a transformation kernel in the vertical direction (vertical transformation kernels) and a transformation kernel in the horizontal direction (horizontal transformation kernels).

[285] Conversion/reconversion can be performed on a CU or TU basis. Thus, the transform/inverse transform can be applied to the residual samples in the CU or to the residual samples in the TU. The CU size and TU size may be identical, or multiple TUs may be present in a CU area. Meanwhile, the CU size may generally refer to the CB size of the component (sample) luminance signals. TU size may generally refer to the TB size of the component (sample) luminance signals. The CB or TB size of the chroma component samples may be extracted based on the CB or TB size of the luminance component samples according to the component ratio according to the color format (chrominance signal format, e.g., 4:4:4, 4:2 :2, 4:2:0, etc.). The TU size can be retrieved based on maxTbSize. For example, when the CU size is larger than maxTbSize, a plurality of TUs (TB) having maxTbSize may be retrieved from the CU, and conversion/reconversion may be performed on a per-TU (TB) basis. MaxTbSize may be taken into account when determining whether or not various types of internal forecasting, such as ISP, are applied. Information regarding maxTbSize can be predefined. Alternatively, information regarding maxTbSize may be generated and encoded by the encoding equipment and signaled to the encoding equipment.

[286] Entropy coding

[287] As described above with reference to FIG. 2, some or all of the video/image information may be entropy encoded by the entropy encoder 190. Some or all of the video/image information described with reference to FIG. 3 can be entropy decoded by the entropy decoder 310. In this case, video/image information can be encoded/decoded based on the syntax element. Herein, encoding/decoding information may include encoding/decoding by the method described in this paragraph.

[288] FIG. 21 shows a CABAC block diagram for encoding one syntax element. In the CABAC encoding process, firstly, when the input signal is a syntactic element that is not a binary value, the input signal is converted into a binary value through binarization. When the input signal is already a binary value, the input signal bypasses conversion to binary form. As used herein, each binary number 0 or 1 constituting a binary value may be referred to as a "permission element". For example, a binary string (string of permission elements) after conversion to binary form is 110, each of 1, 1 and 0 is called "one resolution element". The permission element(s) for one syntax element may indicate the value of the syntax element.

[289] The binarized sample items may be input to a regular encoding engine or a bypass encoding engine. The regular encoding engine may assign a context model that applies a probability value to the corresponding resolution element, and may encode the resolution element based on the assigned context model. After performing encoding for each resolution element, the regular encoding engine can update the probabilistic model for the permission element. Permission elements encoded in this manner may be referred to as "context-encoded permission elements." The bypass encoding mechanism may omit a procedure for estimating the probability of the input sample element and a procedure for updating the probability model applied to the resolution element after encoding. In the case of a bypass encoding mechanism, the input sample element is encoded by applying a uniform probability distribution (eg, 50:50) instead of assigning a context, thereby increasing encoding speed. Permission elements encoded in this manner may be referred to as "bypass permission elements." A context model can be assigned and updated for each permission element that is to be context-encoded (regularly encoded), and the context model can be specified based on ctxidx or ctxInc; ctxidx can be retrieved based on ctxInc. In particular, for example, a context index (ctxidx) indicating the context model for each of the regularly encoded resolution elements may be retrieved as the sum of a context index increment (ctxInc) and a context index offset (ctxIdxOffset). Herein, ctxInc, varying depending on the permission element, can be retrieved. ctxIdxOffset may be represented by the smallest value of ctxIdx. The smallest value of ctxIdx may be called the "initValue of ctxIdx". ctxIdxOffset is a value generally used to differentiate from context models for other syntax elements, and the context model for one syntax element may be different/derived based on ctxinc.

[290] In the entropy encoding procedure, it is determined whether encoding should be performed through a regular encoding mechanism or encoding should be performed through a bypass encoding mechanism, and the encoding path may be switched. Entropy decoding can perform the same process as entropy encoding in reverse order.

[291] The above-described entropy encoding, for example, can be performed as indicated in FIG. 22 and 23. Referring to FIGS. 22 and 23, encoding equipment (entropy encoder) may perform an entropy encoding procedure to obtain image/video information. The image/video information may include segmentation-related information, prediction-related information (eg, inter/intra prediction classification information, intra-prediction mode information, and inter-prediction mode information), residual information, and in-loop filtering-related information, or may include itself various syntactic elements associated with it. Entropy encoding can be performed on a syntactic element basis. Steps S2210-S2220 of FIG. 22 may be performed by the entropy encoder 190 of the encoding equipment of FIG. 2 described above.

[292] The encoding equipment may perform binarization for the target syntax element in step S2210. Herein, binarization may be based on various binarization methods such as a truncated Rice code binarization process and a fixed-length code binarization process, and the binarization method for a target syntax element may be preset. The binarization procedure may be performed by a binarization module 191 in the entropy encoder 190.

[293] The encoding equipment may perform entropy encoding for the target syntax element in step S2220. The encoding hardware may perform regular (contextual) or bypass encoding on a string of resolution elements of a target syntactic element based on an entropy encoding scheme such as context-adaptive arithmetic coding (CABAC) or context-adaptive variable-length coding (CAVLC). The output may be included in the bit stream. The entropy encoding procedure may be performed by an entropy encoding processor 192 in the entropy encoder 190. As described above, the bit stream may be transmitted to decoding equipment via a (digital) storage medium or a network.

[294] Referring to FIG. 24 and 25, decoding equipment (entropy decoder) can decode encoded image/video information. The image/video information may include segmentation-related information, prediction-related information (eg, inter/intra prediction classification information, intra-prediction mode information, and inter-prediction mode information), residual information, and in-loop filtering-related information, or may include itself various syntactic elements associated with it. Entropy encoding can be performed on a syntactic element basis. Steps S2410 to S2420 may be performed by the entropy decoder 210 of the decoding equipment of FIG. 3 described above.

[295] The decoding equipment may perform binarization for the target syntax element in step S2410. Herein, binarization may be based on various binarization methods such as a truncated Rice code binarization process and a fixed-length code binarization process, and the binarization method for a target syntax element may be preset. The decoding hardware may extract available resolution element strings (possible resolution element strings) for available values of the target syntax element through a binarization procedure. The binarization procedure may be performed by a binarization unit 211 in the entropy decoder 210.

[296] The decoding equipment may perform entropy decoding for the target syntax element in step S2420. Although the decoding hardware sequentially decodes and parses each permission element for the target syntax element from the input bit(s) in the bitstream, and can compare the extracted permission element string with the available permission element strings for the syntax element. When the retrieved permission element string is identical to one of the available permission element strings, the value corresponding to the permission element string may be retrieved as the value of the syntax element. If not, the next bit in the bitstream is further parsed and the above procedure is performed again. Through this process, the start bit or end bit for specific information (a specific syntax element) in the bitstream is not used, but variable length bits are used to signal the information. Due to this, relatively fewer bits are assigned to the low value, thereby increasing the overall coding efficiency.

[297] Decoding equipment may perform contextual or bypass decoding for each bin in a row of bins from the bitstream based on an entropy coding technology such as CABAC or CAVLC. The entropy decoding procedure may be performed by the entropy decoding processor 212 in the entropy decoder 210. The bitstream may include various types of information for decoding images/videos, as described above. As described above, the bitstream may be transmitted to the decoding equipment via a (digital) storage medium or a network.

[298] Herein, a table (syntax table) including syntax elements may be used to represent signaling information from encoding equipment to decoding equipment. The order of syntax elements in a table including syntax elements used herein may indicate the order in which the syntax elements from the bitstream are parsed. The encoding equipment may construct and encode a syntax table such that syntax elements are parsed in the order of parsing by the decoding equipment. The decoding equipment can parse and decode the syntax elements of the syntax table from the bit stream in a parsing order and can thereby obtain the values of the syntax elements.

[299] General procedure for image/video encoding

[300] When encoding images/videos, frames constituting the image/video may be encoded/decoded according to the decoding order in the sequence. The order of frames corresponding to the output order of decoded frames can be set so that it is different from the decoding order, and based on this, forward prediction as well as backward direction can be performed when inter prediction is performed.

[301] FIG. 26 shows an example of a schematic frame decoding procedure to which embodiment(s) of the present document is applicable. In fig. 26, step S2610 may be performed by the entropy decoder 210 of the decoding equipment described above with reference to FIG. 3. Step S2620 may be performed by a prediction module including an intra prediction module 265 and an inter prediction module 260. Step S2630 may be performed by a residual processor including a dequantizer 220 and an inverse converter 230. Step S2640 may be performed by an adder 235. Step S2650 may be performed by a filter 240. Step S2610 may include an information decoding procedure described herein. Step S2620 may include the inter/intra prediction procedure described herein. Step S2630 may include a post-processing procedure described herein. Step S2640 may include a block/frame recovery procedure described herein. Step S2650 may include an in-loop filtering procedure described herein.

[302] Referring to FIG. 26, the frame decoding procedure may schematically include, as described above with reference to FIG. 3, a procedure for obtaining image/video information from the bit stream (by decoding) in step S2610, a frame restoration procedure in steps S2620-S2640, and an in-loop filtering procedure for the reconstructed frame in step S2650. The frame reconstruction procedure may be performed based on the prediction samples and residual samples that are obtained through the inter/intra prediction in step S2620 and the residual processing in step S2630 (the process of dequantizing and de-converting the quantized transform coefficients) described herein. The modified reconstructed frame may be generated through an in-loop filtering procedure for the reconstructed frame generated through the frame restoration procedure. The modified reconstructed frame may be output as a decoded frame and may be stored in a decoded frame buffer or decoding equipment memory 250 for later use as a reference frame in the inter-prediction procedure when the frame is decoded. In some cases, the in-loop filtration procedure may be omitted. In this case, the reconstructed frame may be output as a decoded frame and may be stored in the decoded frame buffer or memory 250 of the decoding equipment for later use as a reference frame in the inter-prediction procedure when the frame is decoded. As described above, the in-loop filtering procedure in step S2650 may include a deblocking filtering procedure, a sampled adaptive offset (SAO) filtering procedure, an adaptive loop filtering (ALF) procedure, and/or a bilateral filtering procedure. Part or all of it may be omitted. In addition, one or some of the deblocking filtering procedure, sampled adaptive offset (SAO) filtering procedure, adaptive loop filtering (ALF) procedure, and bilateral filtering procedure may be applied sequentially, or all of them may be applied sequentially. For example, a filtering procedure to remove blocking may be applied to the reconstructed frame, and then the SAO procedure may be executed. Alternatively, for example, a deblocking filtering procedure may be applied to the reconstructed frame, and the ALF procedure may then be executed. This can be done in the same way as in encoding hardware.

[303] FIG. 27 shows an example of a schematic frame encoding procedure to which embodiment(s) of the present document is applicable. In fig. 27, step S2710 may be performed by a prediction module including an intra prediction module 185 or an inter prediction module 180 of the encoding equipment described above with reference to FIG. 2. Step S2720 may be performed by a residual processor including a transformer 120 and/or a quantizer 130. Step S2730 may be performed by an entropy encoder 190. Step S2710 may include an inter/intra prediction procedure described herein. Step S2720 may include a post-processing procedure described herein. Step S2730 may include an information encoding procedure described herein.

[304] Referring to FIG. 27, the frame encoding procedure may schematically include, as described above with reference to FIG. 2, a procedure for encoding information (eg, prediction information, residual information, and segmentation information) to reconstruct frames and output the information in the form of a bit stream, a procedure for generating a reconstructed frame for the current frame, and a procedure (optionally) for applying in-loop filtering to the reconstructed frame. The encoding equipment can extract (modified) residual samples from the quantized transform coefficients through the dequantizer 140 and the inverse converter 150, and can generate a reconstructed frame based on the prediction samples that are output in step S2710 and the (modified) residual samples. The generated reconstructed frame may be identical to the reconstructed frame generated by the decoding equipment described above. An in-loop filtering procedure may be performed on the reconstructed frame to generate a modified reconstructed frame. The modified reconstructed frame may be stored in a decoded frame buffer or memory 170. Similar to the case in decoding equipment, the modified reconstructed frame may subsequently be used as a reference frame in an inter-prediction procedure when the frame is encoded. As described above, in some cases, part or all of the in-loop filtration procedure may be omitted. When the in-loop filtering procedure is performed, the (in-loop) filtering-related information (parameter) can be encoded by the entropy encoder 190 and output in the form of a bit stream. The decoding equipment may perform the in-loop filtering procedure based on the filtering-related information in the same manner as the encoding equipment.

[305] Through this in-loop filtering procedure, noise such as blocking artifact and ringing artifact generated during image/video encoding can be reduced, and subjective/objective visual quality can be improved. In addition, the encoding equipment and the decoding equipment perform an in-loop filtering procedure, so that the encoding equipment and the decoding equipment can extract the same prediction result, the reliability of frame encoding can be improved, and the amount of data that must be transmitted for frame encoding can be reduced.

[306] As described above, the frame reconstruction procedure may be performed in decoding equipment as well as in encoding equipment. A reconstructed block may be generated based on intra-prediction/inter-prediction on a per-block basis, and a reconstructed frame including the reconstructed blocks may be generated. When the current frame/slice/tile group is an I-frame/slice/tile group, the blocks included in the current frame/slice/tile group can only be reconstructed based on internal prediction. Meanwhile, when the current frame/slice/tile group is a P- or B-frame/slice/tile group, blocks included in the current frame/slice/tile group can be reconstructed based on intra-prediction or inter-prediction. In this case, intra prediction can be applied to some blocks in the current frame/slice/tile group, and inter prediction can be applied to some remaining blocks. The color component of a frame may include a luma component and a chrominance component. Unless expressly limited herein, the method and embodiments proposed herein can be applied to a luma signal component and a chrominance signal component.

[307] Example of coding hierarchy and structure

[308] The encoded video/image according to the present document can be processed, for example, according to a hierarchy and encoding structure that is described below.

[309] FIG. 28 is a view showing a hierarchical structure for a coded image. The encoded image can be divided into a video coding layer (VCL, video coding hierarchy), which carries out the decoding processing of the image and itself, a subsystem for transmitting and storing the encoded information, and a network abstraction layer (NAL, network abstraction hierarchy), which is present between the VCL and subsystem and is responsible for the network adaptation function.

[310] In VCL, VCL data including compressed image data (slice data) may be generated. Alternatively, a parameter set may be generated including information such as a frame parameter set (PPS), a sequence parameter set (SPS) and a video parameter set (VPS) or an additional enhancement information (SEI) message further required for the image decoding process .

[311] In NAL, header information (NAL unit header) is added to the primary byte data sequence (RBSP) generated in the VCL such that the NAL unit can be generated. In this document, RBSP refers to the slice data, parameter set, and SEI message generated in the VCL. The NAL unit header may include NAL unit type information that is indicated according to the RBSP data included in the NAL unit.

[312] As shown in the drawing, a NAL unit may be divided into a VCL NAL unit and a non-VCL NAL unit depending on the RBSP generated in the VCL. A VCL NAL unit may refer to a NAL unit that includes information (slice data) regarding an image. A non-VCL NAL unit may refer to a NAL unit that includes information (a parameter set or SEI message) required for decoding an image.

[313] With header information attached according to the subsystem data standard, the VCL NAL unit and non-VCL NAL unit can be transmitted over the network. For example, a NAL unit may be converted into a predefined standard data form such as H.266/VVC file format, Real Time Transport Protocol (RTP), Transport Stream (TS), and the resulting data may be transmitted over various networks.

[314] As described above with respect to the NAL unit, the NAL unit type may be indicated according to the RBSP data structure included in the NAL unit, and information regarding the NAL unit type may be stored in the NAL unit header and transmitted in signaling signals.

[315] For example, depending on whether or not a NAL unit includes information (slice data) regarding an image, a rough classification is made into a VCL NAL unit type and a non-VCL NAL unit type. The VCL type of a NAL unit can be classified according to the characteristic and type of frame included in the VCL NAL unit, and the type of a non-VCL NAL unit can be classified according to a parameter set type.

[316] NAL unit types indicated according to parameter set types that include non-VCL NAL unit types are listed below as an example.

[317] - NAL unit APS (Adaptation Parameter Set): type of NAL unit including the APS

[318] - DPS (Decode Parameter Set) NAL unit: type of NAL unit including the DPS

[319] - VPS (Video Parameter Set) NAL unit: type of NAL unit including the VPS

[320] - SPS (Sequence Parameter Set) NAL unit: type of NAL unit including the SPS

[321] - PPS (frame parameter set) NAL unit: type of NAL unit including the PPS

[322] The above-described NAL unit types have syntax information for the NAL unit type, and the syntax information may be stored in a NAL unit header and transmitted in signaling signals. For example, the syntax information may be nal_unit_type, and NAL unit types may be indicated by nal_unit_type values.

[323] A slice header (slice header syntax) may include information/parameter that applies to the slice collectively. APS (APS syntax) or PPS (PPS syntax) may include information/parameter applicable to one or more slices or frames together. SPS (SPS syntax) may include information/parameter applicable to one or more sequences together. VPS (VPS syntax) may include information/parameter applicable to multiple layers together. DPS (DPS syntax) may include information/parameter applicable to the entire video collectively. The DPS may include information/parameter associated with concatenation of coded video sequences (CVS). As used herein, the high-level syntax (HLS) may include at least one selected from the group of APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, and slice header syntax.

[324] Herein, image/video information encoded by the encoding equipment and signaled in the form of a bit stream to the decoding equipment may include segmentation-related information, intra/inter-prediction information, residual information, and in-loop filtering information. in the frame, as well as information included in the slice header, information included in the APS, information included in the PPS, information included in the SPS, and/or information included in the VPS.

[325] Block Differential Pulse Code Modulation (BDPCM) overview

[326] The image encoding equipment and the image decoding equipment according to the embodiment can perform differential encoding of the residual signal. For example, the image encoding equipment may encode the residual signal by subtracting the prediction signal from the residual signal of the current block, and the image decoding equipment may decode the residual signal by adding the prediction signal to the residual signal of the current block. The image encoding equipment and image decoding equipment according to the embodiment can perform differential encoding of the residual signal by applying BDPCM described below.

[327] BDPCM according to the present disclosure may be performed in the quantized residual domain. The quantized residual region may include a quantized residual signal (or a quantized residual coefficient), and when BDPCM is applied, transformation of the quantized residual signal may be skipped. For example, when using BDPCM, the transformation of the residual signal may be skipped and quantization may be performed. Alternatively, the quantized residual region may include quantized transform coefficients.

[328] In the embodiment to which BDPCM is applied, the image encoding equipment may extract a residual block relative to a current block predicted in the intra prediction mode, and quantize the residual block, thereby extracting the residual block. When the residual signal differential encoding mode is performed with respect to the current block, the image encoding equipment can perform differential encoding with respect to the residual block to extract the modified residual block. In addition, the image encoding equipment can encode differential encoding mode information indicating the differential encoding mode of the residual signal and the modified residual block, thereby generating a bit stream.

[329] More specifically, when BDPCM is applied to the current block, a predicted block (prediction block) including predicted samples of the current block can be generated by intra prediction. In this case, the intra prediction mode for performing intra prediction may be signaled via a bitstream and may be extracted based on the BDPCM prediction direction described below. In addition, in this case, the intra prediction mode may be determined as one of the vertical prediction direction mode or the horizontal prediction direction mode. For example, when the prediction direction of the BDPCM is a horizontal direction, an intra prediction mode may be determined as a horizontal direction prediction mode, and a prediction block relative to a current block may be generated by intra prediction of the horizontal direction. Alternatively, when the BDPCM prediction direction is a vertical direction, an intra prediction mode may be determined as a vertical prediction direction mode, and a prediction block relative to a current block may be generated by intra prediction of the vertical direction. When applying internal horizontal direction prediction, the value of a pixel adjacent to the left relative to the current block may be determined as the predicted sample value for samples included in the corresponding row of the current block. When applying internal vertical direction prediction, the value of a pixel adjacent to a vertex relative to the current block may be determined as the predicted sample value for the samples included in the corresponding column of the current block. When applying BDPCM to the current block, the method of generating a prediction block with respect to the current block can be equally performed in the image encoding equipment and in the image decoding equipment.

[330] When applying BDPCM to a current block, the image encoding equipment may generate a residual block including residual samples of the current block by subtracting the prediction sample from the current block. The image encoding equipment may quantize the residual block and then encode the difference (or delta) between the quantized residual sample and the quantized residual sample predictor. The image decoding equipment may generate a quantized residual block relative to the current block by obtaining a quantized residual sample of the current block based on the predictor and the difference recovered from the bit stream. Thereafter, the image decoding equipment can dequantize the quantized residual block and then add it to the prediction block, thereby reconstructing the current block.

[331] FIG. 29 is a view illustrating a BDPCM residual sample encoding method according to the present disclosure. The residual block of FIG. 29 may be generated by subtracting a prediction block from a current block in the image encoding equipment. The quantized residual block of FIG. 29 may be generated by quantizing a residual block by image encoding equipment. In fig. 29, r _{i, j} indicates the value of the residual sample of coordinates (i, j) in the current block. When the current block size is MxN, the value of i can be from 0 to M-1, inclusive. In addition, the value of j can range from 0 to N-1, inclusive. For example, the remainder may mean the difference between the original block and the prediction block. For example, r _{i, j} may be retrieved by subtracting the value of the prediction sample from the value of the original sample of coordinates (i, j) in the current block. For example, r _{i, j} may represent the remainder of the prediction after horizontal intra-prediction or vertical intra-prediction is performed using a sample that is not filtered from the top or left edge sample. In horizontal intraprediction, the value of the left adjacent pixel is copied along a line intersecting the prediction block. In vertical internal prediction, the top adjacent line is copied into a separate line of the prediction block.

[332] In FIG. 29, Q(r _{i, j} ) refers to the value of the quantized residual sample of coordinates (i, j) in the current block. For example, Q(r _{i, j} ) may refer to the quantized value r _{i, j} .

[333] BDPCM prediction is performed on the quantized residual samples of FIG. 29, and a modified quantized residual block Rp' having a size of MxN including the modified quantized residual samples rp' may be generated.

[334] When the BDPCM prediction direction is a horizontal direction, the value rp' _{i, j} of the modified quantized residual sample coordinates (i, j) in the current block can be calculated as shown in the equation below.

[335] equation 14

[336]

[337] As shown in Equation 14, when the BDPCM prediction direction is the horizontal direction, the quantized residual sample value Q(r _0,j ) is assigned as is to the coordinate value rp' _0,j (0,j). The value rp' _{i, j} of the other coordinates (i, j) can be extracted as a difference value between the value Q(r _{i, j} ) of the quantized residual sample of coordinates (i, j) and the value Q(r _i-1,j ) of the quantized residual samples of coordinates (i-1,j). Thus, instead of encoding the value Q(r _{i, j} ) of the quantized residual sample of coordinates (i, j), the difference value calculated by using the value Q(r _i-1,j ) of the quantized residual sample of coordinates (i-1,j) as a prediction value, is extracted as a modified quantized residual sample value rp' _{i, j} , and then the value rp' _{i, j} is encoded.

[338] When the BDPCM prediction direction is a vertical direction, the value (rp' _{i, j} ) of the modified quantized residual sample coordinates (i, j) in the current block can be calculated as shown in the equation below.

[339] equation 15

[340]

[341] As shown in Equation 15, when the BDPCM prediction direction is a vertical direction, the quantized residual sample value Q(r _i,0 ) is assigned as is to the coordinate value rp' _i,0 (i,0). The value rp' _{i, j} of the other coordinates (i, j) can be extracted as a difference value between the value Q(r _{i, j} ) of the quantized residual sample of coordinates (i, j) and the value Q(r _{i, j-1} ) of the quantized residual samples of coordinates (i, j-1). That is, instead of encoding the value Q(r _{i, j} ) of the quantized residual sample of coordinates (i, j), the difference value calculated by using the value Q(r _{i, j-1} ) of the quantized residual sample of coordinates (i, j-1) as a prediction value, is extracted as a modified quantized residual sample value rp' _{i, j} , and then the value rp' _{i, j} is encoded.

[342] As described above, the process of modifying a current quantized residual sample value by using a nearby quantized residual sample value as a prediction value may be referred to as "BDPCM prediction".

[343] Finally, the image encoding equipment may encode the modified quantized residual block including the modified quantized residual samples, and may transmit the resulting block to the image decoding equipment. Herein, as described above, conversion of the modified quantized residual block is not performed.

[344] FIG. 30 is a view showing modified quantized residual blocks generated by performing BDPCM of the present disclosure.

[345] In FIG. 30, the horizontal BDPCM shows the modified quantized residual block generated according to Equation 14 when the BDPCM prediction direction is the horizontal direction. In addition, the vertical BDPCM shows a modified quantized residual block generated according to Equation 15 when the prediction direction of the BDPCM is the vertical direction.

[346] FIG. 31 is a flowchart showing a procedure for encoding a current block by applying BDPCM in image encoding equipment.

[347] First, when the current block, which is the encoding target block, is input in step S3110, prediction may be performed on the current block to generate a prediction block in step S3120. The prediction block in step S3120 may be an intra-prediction block, and the intra-prediction mode may be determined as described above. Based on the prediction block generated in step S3120, a residual block relative to the current block may be generated in step S3130. For example, image encoding equipment may generate a residual block (residual sample values) by subtracting a prediction block (predicted sample values) from a current block (original sample values). For example, by executing step S3130, the residual block of FIG. 29 can be formed. For the residual block generated in step S3130, quantization may be performed in step S3140 so as to generate a quantized residual block, and BDPCM prediction may be performed on the quantized residual block in step S3150. The quantized residual block generated by step S3140 may be the quantized residual block of FIG. 29. As a result of the BDPCM prediction in step S3150, the modified quantized residual block of FIG. 30 can be formed according to the prediction direction. Since the BDPCM prediction in step S3150 is described with reference to FIG. 29-30, its detailed description is omitted. Thereafter, the image encoding equipment may encode the modified quantized residual block in step S3160 to generate a bit stream. Herein, the conversion of the modified quantized residual block may be skipped.

[348] BDPCM operation in image encoding equipment described with reference to FIG. 29-31 may be performed in reverse order by image decoding equipment.

[349] FIG. 32 is a flowchart showing a procedure for restoring a current block by applying BDPCM in image decoding equipment.

[350] The image decoding equipment may obtain information (image information) required for reconstructing the current block from the bit stream in step S3210. The required information for reconstructing the current block may include information (prediction information) regarding the prediction of the current block and information (residual information) regarding the remainder of the current block. The image decoding equipment may perform prediction regarding the current block based on information regarding the current block and may generate a prediction block in step S3220. The prediction regarding the current block may be an intra prediction, and the detailed description thereof is identical to the description given above with reference to FIG. 31. In FIG. 32, it is shown that the prediction block generation step S3220 for the current block is performed before the residual block generation steps S3230 to S3250 with respect to the current block. However, there are no restrictions on this. After a residual block relative to the current block is generated, a prediction block relative to the current block can be generated. Alternatively, a residual block relative to the current block and a prediction block relative to the current block may be generated simultaneously.

[351] The image decoding equipment may generate a residual block relative to the current block in step S3230 by parsing residual information of the current block from the bit stream. The residual block generated in step S3230 may be a modified quantized residual block shown in FIG. thirty.

[352] The image decoding equipment may generate the quantized residual block of FIG. 29 by performing BDPCM prediction in step S3240 for the modified quantized residual block of FIG. 30. The BDPCM prediction in step S3240 is a procedure for generating a quantized residual block in FIG. 29 from the modified quantized residual block of FIG. 30, which corresponds to the reverse process of step S3150 performed by the image encoding equipment. For example, when differential encoding mode information (eg, bdpcm_flag) obtained from the bit stream indicates a differential encoding mode in which differential encoding of the residual coefficient is performed as BDPCM is applied, the image decoding equipment performs differential encoding for the residual block to extract modified residual block. Using the residual coefficient to be modified and the residual prediction coefficient, the image decoding equipment can modify at least one residual coefficient to be modified from the residual coefficients in the residual block. The residual prediction coefficient may be determined based on the prediction direction indicated by differential encoding direction information (eg, bdpcm_dir_flag) obtained from the bitstream. The differential encoding direction information may indicate a vertical direction or a horizontal direction. The image decoding equipment may assign the value obtained by adding the residual coefficient to be modified and the residual prediction coefficient to a location of the residual coefficient to be modified. Herein, the residual prediction coefficient may be a coefficient that is immediately before and adjacent to the residual coefficient to be modified, in terms of order according to the prediction direction.

[353] Hereinafter, this document will describe in more detail the BDPCM prediction in step S3240 performed by the image decoding equipment. The decoding equipment may calculate the quantized residual sample Q(r _{i, j} ) by performing the calculation performed by the encoding equipment in reverse order. For example, when the BDPCM prediction direction is a horizontal direction, the image decoding equipment can generate a quantized residual block from the modified quantized residual block by using Equation 16.

[354] equation 16

[355]

[356] As given in Equation 16, the value Q(r _{i, j} ) of the quantized residual sample of coordinates (i, j) can be calculated by summing the values of the modified quantized residual samples from coordinates (0, j) to coordinates (i, j ).

[357] Alternatively, using Equation 17 instead of Equation 16, the value Q(r _{i, j} ) of the quantized residual sample of coordinates (i, j) can be calculated.

[358] equation 17

[359]

[360] Equation 17 is the inverse process corresponding to Equation 14. According to Equation 17, the value Q(r _0,j ) of the quantized residual coordinate sample (0,j) is extracted as the value rp' _0,j of the modified quantized residual coordinate sample ( 0,j). Q(r _{i, j} ) of the other coordinates (i, j) is extracted as the sum of the value rp' _{i, j} of the modified quantized residual sample of coordinates (i, j) and the value Q(r _i-1,j ) of the quantized residual sample of coordinates ( i-1,j). Thus, the difference value rp' _{i, j} is summed by using the value Q(r _i-1,j ) of the quantized residual sample coordinates (i-1,j) as a predictive value, thereby extracting the quantized residual sample value Q(r _{i, j} ).

[361] When the BDPCM prediction direction is a vertical direction, the image decoding equipment can generate a quantized residual block from the modified quantized residual block by using Equation 18.

[362] equation 18

[363]

[364] As given in Equation 18, the value Q(r _{i, j} ) of the quantized residual sample coordinates (i, j) can be calculated by summing the values of the modified quantized residual samples from coordinates (i,0) to coordinates (i, j ).

[365] Alternatively, using Equation 19 instead of Equation 18, the value Q(r _{i, j} ) of the quantized residual sample of coordinates (i, j) can be calculated.

[366] equation 19

[367]

[368] Equation 19 is the inverse process corresponding to Equation 15. According to Equation 19, the value Q(r _i,0 ) of the quantized residual coordinate sample (i,0) is extracted as the value rp' _i,0 of the modified quantized residual coordinate sample ( i,0). Q(r _{i, j} ) of other coordinates (i, j) is extracted as the sum of the value rp' _{i, j} of the modified quantized residual sample of coordinates (i, j) and the value Q(r _{i, j-1} ) of the quantized residual sample of coordinates ( i, j-1). Thus, the difference value rp' _{i, j} is summed by using the quantized residual sample value Q(r _{i, j-1} ) as a predictive value, thereby extracting the quantized residual sample value Q(r _{i, j} ).

[369] When a quantized residual block consisting of quantized residual samples is generated by executing step S3240 according to the above-described method, the image decoding equipment performs dequantization on the quantized residual block in step S3250 to generate a residual block relative to the current block. When BDPCM is applied, the transformation of the current block is skipped as described above. Therefore, the inverse transform of the dequantized residual block may be skipped.

[370] Thereafter, the image decoding equipment may reconstruct the current block in step S3260 based on the prediction block generated in step S3220 and the residual block generated in step S3250. For example, the image decoding equipment may reconstruct the current block (the values of the reconstructed samples) by summing the prediction block (the values of the predicted samples) and the residual block (the values of the residual samples). For example, the reconstructed sample value may be generated by summing the dequantized sample Q ^-1 (Q(r _{i, j} )) with the inner block prediction value. Differential coding mode information indicating whether or not BDPCM is applied to the current block may be signaled via the bitstream. In addition, when BDPCM is applied to the current block, differential encoding direction information indicating the BDPCM prediction direction may be signaled through the bitstream. When BDPCM is not applied to the current block, differential encoding direction information may not be signaled.

[371] FIG. 33 to 35 are views schematically showing syntax for signaling information regarding BDPCM.

[372] FIG. 33 is a view showing the syntax of a sequence parameter set according to an embodiment for signaling BDPCM information. In an embodiment, all RBSP SPSs included in at least one access unit (AU) having a value of 0 as a TemporalId, or provided through external means, can be set to be used before access in the decoding process. In addition, an SPS NAL unit including an SPS RBSP may be configured to have a nuh_layer_id that is identical to the nuh_layer_id of a PPS NAL unit referencing the SPS NAL unit. In CVS, all SPS NAL units that have a particular sps_seq_parameter_set_id value can be set to have identical content. In the seq_parameter_set_rbsp() syntax of FIG. 33, the sps_transform_skip_enable_flag described above and the sps_bdpcm_enabled_flag described below are expanded.

[373] The sps_bdpcm_enabled_flag syntax element may indicate whether or not the internal coding unit intra_bdpcm_flag is enabled in CU syntax. For example, the first value (eg, 0) of sps_bdpcm_enabled_flag may indicate that, for an internal coding unit, intra_bdpcm_flag is not provided in CU syntax. A second value (eg, 1) sps_bdpcm_enabled_flag may indicate that, for an intracoding unit, intra_bdpcm_flag is provided in CU syntax. Meanwhile, when sps_bdpcm_enabled_flag is not provided, the value of sps_bdpcm_enabled_flag may be set to the first value (eg, 0).

[374] FIG. 34 is a view showing an embodiment of a syntax for signaling whether or not a limitation on BDPCM is applied. In an embodiment, a predefined constraint condition during the encoding/decoding process may be signaled using the general_constraint_info() syntax. Using the syntax of FIG. 34, a no_bdpcm_constraint_flag syntax element indicating whether or not the value of the sps_bdpcm_enabled_flag described above should be set to 0 may be signaled. For example, the first value (eg 0) of no_bdpcm_constraint_flag may indicate that no such constraint is applied. When the value of no_bdpcm_constraint_flag is the second value (for example, 1), the value of sps_bdpcm_enabled_flag can be forced to be the first value (for example, 0).

[375] FIG. 35 is a view showing an embodiment of the coding_unit() syntax for signaling information regarding the BDPCM for a coding unit. As shown in FIG. 35, the intra_bdpcm_flag and intra_bdpcm_dir_flag syntax elements may be signaled using the coding_unit() syntax. The intra_bdpcm_flag syntax element may indicate whether or not BDPCM is applied to the current luma coding block located at (x0, y0).

[376] For example, the first value (eg, 0) of intra_bdpcm_flag may indicate that BDPCM is not applied to the current luma coding block. A second value (eg, 1) intra_bdpcm_flag may indicate that BDPCM is applied to the current luma coding block. By indicating that BDPCM is applied, intra_bdpcm_flag may indicate whether or not conversion is skipped, as well as whether or not the intra-luminance signal prediction mode is performed, via intra_bdpcm_dir_flag, which is described below.

[377] Meanwhile, the value of the above-described variable BdpcmFlag[x][y] can be set equal to the value of intra_bdpcm_flag with respect to x=x0...x0+cbWidth-1 and y=y0...y0+cbHeight-1.

[378] The intra_bdpcm_dir_flag syntax element may indicate the BDPCM prediction direction. For example, the first value (eg, 0) of intra_bdpcm_dir_flag may indicate that the BDPCM prediction direction is a horizontal direction. The second value (eg, 1) intra_bdpcm_dir_flag may indicate that the BDPCM prediction direction is a vertical direction.

[379] Meanwhile, the value of the variable BdpcmDir[x][y] can be set equal to the value of intra_bdpcm_dir_flag with respect to x=x0...x0+cbWidth-1 and y=y0...y0+cbHeight-1.

[380] Intra prediction for chrominance block

[381] When intra prediction is performed on the current block, prediction relative to the luminance signal component block (luminance signal block) of the current block and prediction relative to the chrominance signal component block (chrominance signal block) can be performed. In this case, the intra prediction mode for the chrominance signal block may be set separately from the intra prediction mode for the luma signal block.

[382] For example, an intra prediction mode for a chrominance block may be indicated based on the intra chrominance prediction mode information. Intra-chroma prediction mode information may be signaled in the form of the intra_chroma_pred_mode syntax element. For example, the intra-chrominance prediction mode information may indicate one of the following: planar mode, DC mode, vertical mode, horizontal mode, extracted mode (DM), and cross-component linear model (CCLM) mode. As used herein, planar mode may refer to intra-prediction mode 0, DC mode may refer to intra-prediction mode 1, vertical mode may refer to intra-prediction mode 26, and horizontal mode may refer to intra-prediction mode 10. DM may also be called "direct mode". CCLM can also be called "linear model (LM)". CCLM mode may include any of the following: L_CCLM, T_CCLM and LT_CCLM.

[383] Meanwhile, DM and CCLM are dependent intra prediction modes for predicting a chroma block by using luma block information. DM may mean a mode in which an intra prediction mode identical to the intra prediction mode for the luma signal component is applied as an intra prediction mode for the chrominance signal component. In addition, CCLM may mean an intra prediction mode in which, in the process of generating a prediction block for a chrominance block, the reconstructed samples of the luma block are downsampled, and then the samples extracted by applying the CCLM parameters α and β to the downsampled samples are used as predictive samples of a block of chrominance signals.

[384] General mode representation based on cross-component linear model (CCLM)

[385] As described above, the CCLM mode may be applied to a chrominance block. The CCLM mode is an intra-prediction mode using correlation between a luma block and a chrominance block corresponding to a luma block, and is performed by extracting a linear model based on adjacent samples of the luma block and adjacent samples of the chrominance block. In addition, a predictive chrominance block sample can be extracted based on the extracted linear model and the reconstructed luminance block samples.

[386] In particular, when the CCLM mode is applied to the current chrominance block, parameters for the linear model can be extracted based on the neighboring samples used for intra-prediction of the current chroma block and the neighboring samples used for intra-prediction of the current luminance block. For example, a linear model for CCLM can be represented based on an equation, as explained below.

[387] equation 20

[388]

[389] Herein, pred _c (i, j) may mean a predictive sample of coordinates (i, j) of the current chrominance block in the current CU; rec _L '(i, j) may mean a reconstructed sample of coordinates (i, j) of the current block of luminance signals in the CU. For example, rec _L '(i, j) may mean a downsampled reconstructed sample of the current block of luminance signals. The coefficients α and β of the linear model may be signaled or may be extracted from an adjacent sample.

[390] Unified Residue Coding (Unified CbCr)

[391] In the encoding/decoding process according to an embodiment, the remainders of the chrominance signals may be encoded/decoded together. This may be referred to as "united residue coding" or "united CbCr". Whether or not the CbCr joint encoding mode should be applied (activated) can be signaled by the joint encoding mode signaling flag tu_joint_cbcr_residual_flag, which is signaled at the transform basis level. In addition, the selected encoding mode can be extracted by CBF chrominance signals. The tu_joint_cbcr_residual_flag may be present when the value of at least one chrominance CBF for the transform basis is 1. The chrominance QP offset value indicates the difference between the total chrominance QP offset value signaled for the regular chrominance residual encoding mode , and the value of the QP offset of chrominance signals for the combined CbCr encoding mode. The chroma QP offset value may be signaled via PPS or slice header. This QP offset value can be used to control the QP value of the chrominance signals for blocks using the combined chrominance residual coding mode.

[392] In the case in which mode 2, which is the corresponding combined chrominance encoding mode in the table below, is activated for a transform basis while quantization and decoding of the transform basis is performed, its chrominance QP offset can be added to the target extracted from the luminance signals QP of the chrominance signals (used extracted from the luminance signals QP of the chrominance signals).

[393] With respect to other modes, such as modes 1 and 3 in the table below, the QP of chrominance signals can be extracted such that it is obtained for a common Cb or Cr block. This process of restoring the residual chrominance signals (resCb and resCr) from the conversion block can be selected according to the table below. When the present mode is activated, one separate combined residual chrominance block (resJointC[x][y] in the table below) is signaled, and the residual block resCb for Cb and the residual block resCr for Cr can be extracted taking into account information such as tu_cbf_cb , tu_cbf_cr and CSign, which is the sign value expanded in the slice header.

[394] In encoding equipment, the combined chrominance signal component may be extracted as follows. According to the joint encoding mode, resJointC{1,2} can be configured according to the following order. In case of mode 2 (single residue with reduction Cb=C, Cr=CSign*C), the combined residue can be determined according to the equation below.

[395] equation 21

[396] resJointC[x][y]=(resCb[x][y]+CSign*resCr[x][y])/2.

[397] Alternatively, in the case of Mode 1 (single residue with reduction of Cb=C, Cr=(CSign*C)/2), the combined residue may be determined according to the following equation.

[398] equation 22

[399] resJointC[x][y]=(4*resCb[x][y]+2*CSign*resCr[x][y])/5.

[400] Alternatively, in the case of mode 3 (single residue with reduction of Cr=C, Cb=(CSign*C)/2), the combined residue can be determined according to the following equation.

[401] equation 23

[402] resJointC[x][y]=(4*resCr[x][y]+2*CSign*resCb[x][y])/5.

[403]

Table 2 tu_cbf_cb tu_cbf_cr Reduction of Cb- and Cr-residues Mode 1 0 resCb[x][y]=resJointC[x][y]
resCr[x][y]=(CSign*resJointC[x][y])>>1 1 1 1 resCb[x][y]=resJointC[x][y]
resCr[x][y]=CSign*resJointC[x][y] 2 0 1 resCb[x][y]=(CSign*resJointC[x][y])>>1
resCr[x][y]=resJointC[x][y] 3

[404] The above table shows the recovery of residual chrominance signals. CSign means the +1 or -1 sign value specified in the slice header; resJointC[][] means the remainder to be passed. In the table, modes mean TuCResMode, which is described below. The three combined chrominance coding modes in the table can only be supported for the I-slice. For P- and B-cuts, only mode 2 can be supported. Therefore, for P- and B-cuts, the syntax element tu_joint_cbcr_residual_flag can only be provided when both CBF values of the chrominance signals (for example, tu_cbf_cb and tu_cbf_cr) are equal to 1. Meanwhile ,in context modeling tu_cbf_luma and tu_cbf_cb, the ,transformation depth can be removed.

[405] Embodiment 1. QP update method using ACT Qp_offset

[406] As described above, updating the QP so as to apply the ACT can be performed. The QP update described above has several problems. For example, when the above method is used, it is not possible to specify different ACT Qp offsets for individual color components. Additionally, the extracted qP value may have a negative value. Accordingly, in the following embodiment, a method of applying clipping to a Qp value obtained based on the color component value offset QP value ACT is described.

[407] In an embodiment, the quantization parameter qP may be extracted as follows.

[408] First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[409] equation 24

[410] qP=Qp' _Y

[411] ActQpOffset=5

[412] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[413] equation 25

[414] qP=Qp' _CbCr

[415] ActQpOffset=5

[416] Alternatively, when cIdx has a value of 1, the qP and ACT Qp offset can be extracted as shown in the equation below.

[417] equation 26

[418] qP=Qp' _Cb

[419] ActQpOffset=3

[420] The quantization parameter qP may be updated as follows.

[421] When transform_skip_flag[xTbY][yTbY][cIdx] is set to 0, qP can be extracted as shown in the equation below.

[422] equation 27

[423] qP=Max(0, qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[424] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be extracted as shown in the equation below.

[425] equation 28

[426] qP=Max(0, Max(QpPrimeTsMin, qP)-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[427] In another embodiment, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be clipped using the value of QpPrimeTsMin instead of 0, as shown in the equation below.

[428] equation 29

[429] qP=Max(QpPrimeTsMin, qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[430] Meanwhile, in another embodiment, the quantization parameter qP can be extracted as follows.

[431] First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[432] equation 30

[433] qP=Qp' _Y

[434] ActQpOffset=5

[435] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[436] equation 31

[437] qP=Qp' _CbCr

[438] ActQpOffset=5

[439] Alternatively, when cIdx has a value of 1, the qP and ACT Qp offset can be extracted as shown in the equation below.

[440] equation 32

[441] qP=Qp' _Cb

[442] ActQpOffset=5

[443] Alternatively, when cIdx has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[444] equation 33

[445] qP=Qp' _Cr

[446] ActQpOffset=3

[447] The quantization parameter qP may be updated as follows.

[448] When transform_skip_flag[xTbY][yTbY][cIdx] is set to 0, qP can be extracted as shown in the equation below.

[449] equation 34

[450] qP=Max(0, qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[451] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be retrieved as shown in the equation below.

[452] equation 35

[453] qP=Max(0, Max(QpPrimeTsMin, qP)-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[454] In another embodiment, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be clipped using the value of QpPrimeTsMin instead of 0, as shown in the equation below.

[455] equation 36

[456] qP=Max(QpPrimeTsMin, qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[457] Meanwhile, in yet another embodiment, the quantization parameter qP can be extracted as follows.

[458] First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[459] equation 37

[460] qP=Qp' _Y

[461] ActQpOffset=-5

[462] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[463] equation 38

[464] qP=Qp' _CbCr

[465] ActQpOffset=-5

[466] Alternatively, when cIdx has a value of 1, the qP and ACT Qp offset can be extracted as shown in the equation below.

[467] equation 39

[468] qP=Qp' _Cb

[469] ActQpOffset=-5

[470] Alternatively, when cIdx has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[471] equation 40

[472] qP=Qp' _Cr

[473] ActQpOffset=-3

[474] The quantization parameter qP may be updated as follows.

[475] When transform_skip_flag[xTbY][yTbY][cIdx] is set to 0, qP can be extracted as shown in the equation below.

[476] equation 41

[477] qP=Max(0, qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[478] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] is set to 1, qP may be retrieved as shown in the equation below.

[479] equation 42

[480] qP=Max(0, Max(QpPrimeTsMin, qP)+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[481] Meanwhile, in another embodiment, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be clipped using the value of QpPrimeTsMin instead of 0, as shown in the equation below.

[482] equation 43

[483] qP=Max(QpPrimeTsMin, qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[484] In the above description, Y, Cb and Cr may represent three color components. For example, in the ACT transform, Y may correspond to C0. Cb can correspond to C1 or Cg. In addition, Cr may correspond to C2 or Co.

[485] In addition, the ACTQpOffset-5, -5, and -3 values for the three color components may be replaced by other values or other variables.

[486] Embodiment 2. QP-bias control signaling for ACT

[487] In the above-described embodiment, the ACT QP offset controls are set fixed to -5, -5 and -3 for the Y-, Cg- and Co-components. In the present embodiment, in order to provide greater flexibility for adjusting the ACT QP offset, a method for transmitting the ACT Qp offset signals will be described next. The ACT QP offset may be signaled as a parameter in the PPS.

[488] In an embodiment, qp_offset may be signaled according to the syntax table of FIG. 36. Its syntactic elements are as follows.

[489] The pps_act_qp_offsets_present_flag syntax element may indicate whether or not a syntax element associated with an ACT QP offset is present in the PPS. For example, pps_act_qp_offsets_present_flag may indicate whether or not the syntax elements pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset, which are described below, are signaled as PPS.

[490] For example, the first value (eg, 0) of pps_act_qp_offsets_present_flag may indicate that pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset are not signaled through the PPS syntax table.

[491] A second value (eg, 1) of pps_act_qp_offsets_present_flag may indicate that pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset are signaled through the PPS syntax table.

[492] When pps_act_qp_offsets_present_flag is not provided from the bitstream, pps_act_qp_offsets_present_flag may be retrieved as the first value (eg, 0). For example, when a flag (eg, sps_act_enabled_flag signaled in SPS) indicating whether or not ACT is applied has a first value (eg, 0) indicating that ACT is not applied, pps_act_qp_offsets_present_flag may be forced to have the first value (eg , 0).

[493] When the value of the cu_act_enabled_flag syntax element is a second value (eg, 1) indicating that ACT is applied for the current encoding basis, the syntax elements pps_act_y_qp_offset_plus5, pps_act_cb_qp_offset_plus5s and pps_act_cr_qp_offset_plus3 can be used to specify offsets applied to the qP values of the quanta parameters ania for the luminance signal component, the Cb and Cr components, respectively. When the values pps_act_y_qp_offset_plus5, pps_act_cb_qp_offset_plus5 and pps_act_cr_qp_offset_plus3 are not present in the bitstream, each value may be set to 0.

[494] According to the syntax elements, the value of the variable PpsActQpOffsetY can be defined to be pps_act_y_qp_offset_plus5-5. The value of the PpsActQpOffsetCb variable can be set to pps_act_cb_qp_offset_plus5-5. In addition, the value of the variable PpsActQpOffsetCr can be defined equal to pps_act_cb_qp_offset_plus3-3.

[495] In this document, ACT is not an orthonormal transform, so 5, 5, and 3 can be used as constant offset values to be subtracted. In an embodiment, for bitstream alignment, the values of PpsActQpOffsetY, PpsActQpOffsetCb, and PpsActQpOffsetCr may have values ranging from -12 to 12. Additionally, according to the embodiment, in addition to 5, 5, and 3, the Qp offset values may be replaced by other constants values and be used.

[496] In another embodiment, the QP may be adjusted using a more flexible ACT_QP offset. In the following embodiment, an example is described in which the ACT QP offset is signaled in a bitstream. Accordingly, the ACT QP offset may have a wider offset range. Therefore, the QP updated using the ACT QP offset is more likely to be outside the available range, so it is necessary to perform cutoff for the upper and lower limits for the updated QP (more detailed embodiments are described below in Embodiments 6 and 7).

[497] The variables PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, and PpsActQpOffsetCbCr indicating ACT QP offsets may be values extracted using ACT QP offsets signaled via the bit stream, or preset constants. For bitstream matching, PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, and PpsActQpOffsetCbCr can have values ranging from -12 to +12.

[498] When not using a fixed value, the QP offset value is signaled and its value is between -12 and 12, it is necessary to trim the upper limit value of the extracted QP value, in addition to clipping the lower limit value of the QP value , extracted in order to prevent QP having a negative value.

[499] To prevent the qP value from being negative, the smallest qP value may be forced to be 0. Alternatively, the smallest qP value may be set to a value determined by a signaled syntax element. For example, to signal the smallest qP value when the skip mode is applied, a QpPrimeTsMin syntax element may be used to indicate the value of qP applied when the skip mode is applied. The maximum qP value may be limited to the available maximum qP value (eg, 63) or the maximum available qP value determined according to the syntax element signaled.

[500] In the embodiment according to the above, the quantization parameter qP can be extracted as follows. First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[501] equation 44

[502] qP=Qp' _Y

[503] ActQpOffset=PpsActQpOffsetY

[504] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[505] equation 45

[506] qP=Qp' _CbCr

[507] ActQpOffset=PpsActQpOffsetCbCr

[508] Alternatively, when cIdx has a value of 1, the qP and ACT Qp offset can be extracted as shown in the equation below.

[509] equation 46

[510] qP=Qp' _Cb

[511] ActQpOffset=PpsActQpOffsetCb

[512] Alternatively, when cIdx has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[513] equation 47

[514] qP=Qp' _Cr

[515] ActQpOffset=PpsActQpOffsetCr

[516] In an embodiment, the quantization parameter qP may be updated as follows. When transform_skip_flag[xTbY][yTbY][cIdx] is set to 0, qP can be extracted as shown in the equation below.

[517] equation 48

[518] qP=Clip3(0, 63, qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[519] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be retrieved as shown in the equation below.

[520] equation 49

[521] qP=Clip3(0, 63, Max(QpPrimeTsMin, qP)-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[522] In another embodiment, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, the smallest qP value may be cut off using the value of QpPrimeTsMin instead of 0, as shown in the equation below.

[523] equation 50

[524] The quantization parameter qP may be updated as follows.

[525] When transform_skip_flag[xTbY][yTbY][cIdx] is set to 0, qP can be extracted as shown in the equation below.

[526] qP=Clip3(0, 63, qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[527] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be retrieved as shown in the equation below.

[528] qP=Clip3(QpPrimeTsMin, 63, qP-cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[529] In another embodiment, the quantization parameter qP may be updated as follows.

[530] When transform_skip_flag[xTbY][yTbY][cIdx] is set to 0, qP can be extracted as shown in the equation below.

[531] equation 51

[532] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[533] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be extracted as shown in the equation below.

[534] equation 52

[535] qP=Clip3(0, 63+QpBdOffset, Max(QpPrimeTsMin, qP)+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[536] In another embodiment, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, the smallest qP value may be cut off using the value of QpPrimeTsMin instead of 0, as shown in the equation below.

[537] equation 53

[538] The quantization parameter qP may be updated as follows.

[539] When transform_skip_flag[xTbY][yTbY][cIdx] is set to 0, qP can be extracted as shown in the equation below.

[540] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[541] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be retrieved as shown in the equation below.

[542] qP=Clip3(QpPrimeTsMin, 63+QpBdOffset, qP+cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[543] Embodiment 3. Method for enabling ACT when chrominance BDPCM is performed

[544] In an embodiment, when BDPCM is applied to a block of luminance signal components, ACT may be applied to encode/decode the block. However, when BDPCM is applied to a block of chroma components, ACT is limited in that it is applied to encode/decode the block.

[545] Meanwhile, even when BDPCM is applied to a chroma component block, ACT is applied to the block, thereby improving coding efficiency. Fig. 37 shows an embodiment of a syntax configuration for applying ACT even when BDPCM is applied to a chroma component block. As shown in FIG. 37, by removing the condition for obtaining a BDPCM syntax element for a chroma component according to the value of cu_act_enabled_flag indicating whether or not ACT is applied to the current encoding basis, a BDPCM syntax element for it can be obtained regardless of whether ACT is applied or not to the component block. chrominance signals, and BDCPM encoding can be performed accordingly.

[546] Embodiment 4. How to apply ACT even when encoding/decoding is done using CCLM

[547] CCLM and ACT have the intention of removing unnecessary overlap between components. There are some overlapping parts between CCLM and ACT, but even after applying all of them, it is not possible to completely remove the overlap between components. Therefore, the overlap between components can be removed to a greater extent by applying CCLM and ACT together.

[548] The following embodiment describes an embodiment in which CCLM and ACT are used together. When performing decoding, the decoding equipment may apply CCLM first and apply ACT. When ACT is applied to both BDPCM and CCLM for the chrominance signal component, the syntax table for conveying this in the signaling may be modified as shown in FIG. 38. Accordingly, as shown in the syntax table of FIG. 38, from the restrictions for signaling a syntax element associated with intra_bdpcm_chroma and cclm, if(!cu_act_enabled_flag) for signaling a syntax element depending on whether ACT is applied or not applied may be removed from the syntax table.

[549] Embodiment 5. Method of using flexible ACT Qp including integrated CbCr

[550] When the ACT mode is applied, the prediction residual may be converted from one color space (eg, GBR or YCbCr) to a YCgCo color space. In addition, the remainders of the transformation basis can be encoded in the YCgCo color space. As an embodiment of the basic ACT transform (conversion kernel) used for color space conversion, the following conversion kernel can be used as described above.

[551] equation 54

[552]

[553] equation 55

[554]

[555] As described in the above equations, the C0', C1' and C2' (herein, C0'=Y, C1'=Cg, C2'=Co) transformations are not normalized. For example, the L2 norm does not have a value of 1. For example, the L2 transformation norm for an individual component may have a value of approximately 0.6 for C0' and C1' and may have a value of approximately 0.7 for C2'. In this document, the L2 norm is the value obtained as the square root of the sum of the squares of the corresponding coefficients. For example, C0'=2/4*C0+1/4*C1+1/4*C2 can be calculated. Therefore, the C0' norm can be calculated as the square root of (2/4*2/4+1/4*1/4+1/4*1/4). Therefore, it can be calculated as the square root of 6/16 and can be calculated to have a value of approximately 0.6.

[556] When normalized transform is not applied, the dynamic range of an individual component is irregular. In addition, this causes a decrease in the encoding performance of the overall video compression system.

[557] To compensate for the dynamic range of the residual signal, a QP offset value for dynamic range compensation for the individual transform component is transmitted so that QP adjustment can be performed. For example, this embodiment can be applied to a general control method for QP control for ACT conversion as well as combined CbCr.

[558] The individual color components are not encoded independently, but together, so that the method described above in Embodiment 3 for combined CbCr may cause a change in the dynamic range between the individual color components.

[559] In the encoding and decoding methods according to the embodiment, the QP offset adjustment ACT may be set fixed to -5, which may be equally applied to Y, Cg and Co.

[560] In an embodiment, to provide flexible Qp control for individual components and jointCbCr, different ACT Qp offsets may be allowed to be used for Y, Cb, Cr, and/or joint CbCr. The ACT Qp offset values may be determined based on the component index and/or the combined CbCr and/or the combined CbCr mode.

[561] To indicate ACT Qp offsets, ppsActQpOffsetY, ppsActQpOffsetCb, and ppsActQpOffsetCr can be used. In addition, ppsActQpOffsetCbCr can be used to ACT QP offset a combined CbCr mode 2 having a CBF in which all Cb and Cr components have non-zero values. These values (eg, ppsActQpOffsetY, ppsActQpOffsetCb, ppsActQpOffsetCr, and ppsActQpOffsetCbCr) may be preset as predefined values or signaled via a bitstream. The ACT QP offset of the combined CbCr mode may be set in a different manner or to a different value.

[562] In an embodiment, ACT Qp offsets of -5, -5 and -3 can be used for Y, Cb and Cr, and ACT Qp offset of -4 can be used for combined CbCr.

[563] In another embodiment, ACT Qp offsets of -5, -4 and -3 may be used for Y, Cb and Cr, and ACT Qp offset of -3 may be used for the combined CbCr mode in which the value tu_cbf_cb not equal to 0.

[564] In yet another embodiment, the ACT QP bias of the combined CbCr mode 2 may have its own bias value. For another combined CbCr mode, the ACT QP offset may use the offset of the corresponding component. For example, the quantization parameter qP may be defined as follows. First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[565] equation 56

[566] qP=Qp' _Y

[567] ActQpOffset=ppsActQpOffsetY

[568] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[569] equation 57

[570] qP=Qp' _CbCr

[571] ActQpOffset=ppsActQpOffsetCbCr

[572] Alternatively, when cIdx has a value of 1, the qP and ACT Qp offset can be extracted as shown in the equation below.

[573] equation 58

[574] qP=Qp' _Cb

[575] ActQpOffset=ppsActQpOffsetCb

[576] Alternatively, when cIdx has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[577] equation 59

[578] qP=Qp' _Cr

[579] ActQpOffset=ppsActQpOffsetCr

[580] In an embodiment, the quantization parameter qP may be updated as follows.

[581] When transform_skip_flag[xTbY][yTbY][cIdx] is set to 0, qP can be extracted as shown in the equation below.

[582] equation 60

[583] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[584] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be retrieved as shown in the equation below.

[585] equation 61

[586] qP=Clip3(QpPrimeTsMin, 63+QpBdOffset, qP+cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[587] In another embodiment, relative to a combined CbCr mode (eg, in the case of modes 1 and 2) with tu_cbf_cb!=0, the offset for the combined CbCr may be determined using ppsActQpOffsetCb. Alternatively, relative to the combined CbCr mode (eg in the case of mode 3) with tu_cbf_cb==0, the offset for the combined CbCr can be determined using ppsActQpOffsetCr. For example, the above-described embodiment may be modified and applied as follows.

[588] The quantization parameter qP may be updated as follows. First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[589] equation 62

[590] qP=Qp' _Y

[591] ActQpOffset=ppsActQpOffsetY

[592] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP may be extracted as shown in the equation below.

[593] equation 63

[594] qP=Qp' _CbCr

[595] Alternatively, when cIdx has a value of 1, the qP and ACT Qp offset can be extracted as shown in the equation below.

[596] equation 64

[597] qP=Qp' _Cb

[598] ActQpOffset=ppsActQpOffsetCb

[599] Alternatively, when cIdx has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[600] equation 65

[601] qP=Qp' _Cr

[602] ActQpOffset=ppsActQpOffsetCr

[603] When cIdx is not set to 0 and TuCResMode[xTbY][yTbY] is not set to 0, the ACT Qp offset for the combined CbCr mode can be determined according to the pseudo code below.

[604] equation 66

[605] if(TuCResMode[xTbY][yTbY] is 1 or 2)

[606] ActQpOffset=ppsActQpOffsetCb;

[607]else

[608] ActQpOffset=ppsActQpOffsetCr;

[609] In an embodiment, the quantization parameter qP may be updated as follows. When transform_skip_flag[xTbY][yTbY][cIdx] is set to 0, qP can be extracted as shown in the equation below.

[610] equation 67

[611] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[612] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be retrieved as shown in the equation below.

[613] equation 68

[614] qP=Clip3(QpPrimeTsMin, 63+QpBdOffset, qP+cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[615] In another embodiment, regardless of the combined CbCr mode, ppsActQpOffsetY is used when the component index is Y, ppsActQpOffsetCb is used when the component index is Cb, and ppsActQpOffsetCr is used when the component index is Cr, thereby extract qP. For example, the quantization parameter qP can be extracted as follows.

[616] First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[617] equation 69

[618] qP=Qp' _Y

[619] ActQpOffset=ppsActQpOffsetY

[620] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[621] equation 70

[622] qP=Qp' _CbCr

[623] ActQpOffset=(cIdx==1)?ppsActQpOffsetCb:ppsActQpOffsetCr

[624] Alternatively, when cIdx has a value of 1, the qP and ACT Qp offset can be extracted as shown in the equation below.

[625] equation 71

[626] qP=Qp' _Cb

[627] ActQpOffset=ppsActQpOffsetCb

[628] Alternatively, when cIdx has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[629] equation 72

[630] qP=Qp' _Cr

[631] ActQpOffset=ppsActQpOffsetCr

[632] The quantization parameter qP may be updated as follows.

[633] When transform_skip_flag[xTbY][yTbY][cIdx] is set to 0, qP can be extracted as shown in the equation below.

[634] equation 73

[635] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[636] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be retrieved as shown in the equation below.

[637] equation 74

[638] qP=Clip3(QpPrimeTsMin, 63+QpBdOffset, qP+cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[639] Embodiment 6. Method of transmitting Qp-offset including combined CbCr in ACT signaling

[640] Later in this document, an example is described in which the ACT QP offset is signaled via a bit stream to provide greater flexibility. ACT QP offsets may be signaled via SPS, PPS, frame header, slice header, or other types of header sets. The ACT Qp offset of the combined CbCr may be signaled separately or may be derived from the ACT QP offsets for Y, Cb and Cr.

[641] Without loss of generality, FIG. 39 shows an example of a syntax table in which the ACT Qp offset is signaled to the PPS. Similar to the embodiment of FIG. 39, one ACT Qp offset may be signaled for the combined CbCr. In the following, syntax elements specified in the syntax table of FIG. 39.

[642] The pps_act_qp_offsets_present_flag syntax element may indicate whether a syntax element associated with an ACT QP offset is present or not in the PPS. For example, pps_act_qp_offsets_present_flag may indicate whether or not the syntax elements pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4, which are described below, are transmitted in signaling as PPS.

[643] For example, the first value (e.g., 0) of pps_act_qp_offsets_present_flag may indicate that pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4 are not signaled via the PPS syntax blitz.

[644] The second value (for example, 1) pps_act_qp_offsets_pree_flag may indicate that pps_act_y_qp_offset_plusx1, pps_act_cb_qp_offseet_plusx2, pps_act_cr_qp_offset_plusx3 and PPS_AC_AC_AC T_CBCR_QP_OFFSET_PLUSX4 are transmitted in service signals through a syntactic PPS table.

[645] When pps_act_qp_offsets_present_flag is not provided from the bitstream, pps_act_qp_offsets_present_flag may be retrieved as the first value (eg, 0). For example, when a flag (eg, sps_act_enabled_flag signaled in SPS) indicating whether or not ACT is applied has a first value (eg, 0) indicating that ACT is not applied, pps_act_qp_offsets_present_flag may be forced to have the first value (eg , 0).

[646] When the value of the cu_act_enabled_flag syntax element is a second value (eg, 1) indicating that ACT is applied for the current encoding basis, the syntax elements pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX 4 can be used to define offsets to apply to qP values quantization parameters for the luma component, the Cb component, the Cr component, and the combined CbCr component, respectively. When the values pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3 and pps_act_cbcr_qp_offset_plusX4 are not present in the bitstream, each value may be set to 0.

[647] According to the syntax elements, the values of the variables PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr and PpsActQpOffsetCbCr can be determined as shown in the following equation.

[648] equation 75

[649] PpsActQpOffsetY=pps_act_y_qp_offset_plusX1-X1

[650] PpsActQpOffsetCb=pps_act_cb_qp_offset_plusX2-X2

[651] PpsActQpOffsetCr=pps_act_cr_qp_offset_plusX3-X3

[652] PpsActQpOffsetCbCr=pps_act_cbcr_qp_offset_plusX4-X4

[653] Herein, X1, X2, X3 and X4 may indicate predetermined constant values. They may have identical meanings or different meanings, or only some may have identical meanings.

[654] In an embodiment, for bitstream alignment, the values of PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, and PpsActQpOffsetCbCr may be constrained such that they have values ranging from -12 to 12.

[655] According to the definition of variables, the quantization parameter qP can be determined as follows. First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[656] equation 76

[657] qP=Qp' _Y

[658] ActQpOffset=PpsActQpOffsetY

[659] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[660] equation 77

[661] qP=Qp' _CbCr

[662] ActQpOffset=PpsActQpOffsetCbCr

[663] Alternatively, when cIdx has a value of 1, the qP and ACT Qp offset can be extracted as shown in the equation below.

[664] equation 78

[665] qP=Qp' _Cb

[666] ActQpOffset=PpsActQpOffsetCb

[667] Alternatively, when cIdx has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[668] equation 79

[669] qP=Qp' _Cr

[670] ActQpOffset=PpsActQpOffsetCr

[671] In another embodiment of ACT Qp offset signaling, a plurality of ACT QP offsets may be signaled for different combined CbCr modes identified as mode A and mode B.

[672] Combined CbCr mode A may mean a combined CbCr mode having tu_cbf_cb having a non-zero value, for example, mode 1 and mode 2 of Table 2 described above. Joint CbCr mode B may mean jointCbCrmode having tu_cbf_cb having a value of 0, for example mode 3 of table 2 described above. A syntax table modified accordingly is shown in FIG. 40. In the following, syntax elements specified in the syntax table of FIG. 40.

[673] When the value of the cu_act_enabled_flag syntax element is a second value (for example, 1) indicating that ACT is applied for the current encoding basis, syntax elements pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, pps_act_cbcr_qp_offset_mode A_plusX4 and pps_act_cbcr_qp_offset_modeB_plusX5 can be used to define offsets to apply to the qP values of the quantization parameters for the luminance component, the Cb component, the Cr component, and the combined CbCr component, respectively. When the values pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, pps_act_cbcr_qp_offset_modeA_plusX4 and pps_act_cbcr_qp_offset_modeB_plusX5 are not present in the bit stream, each value can be set to 0.

[674] According to the syntax elements, the values of the variables PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, PpsActQpOffsetCbCrModeA and PpsActQpOffsetCbCrModeB can be determined as shown in the following equation.

[675] equation 80

[676] PpsActQpOffsetY=pps_act_y_qp_offset_plusX1-X1

[677] PpsActQpOffsetCb=pps_act_cb_qp_offset_plusX2-X2

[678] PpsActQpOffsetCr=pps_act_cr_qp_offset_plusX3-X3

[679] PpsActQpOffsetCbCrModeA=pps_act_cbcr_qp_offset_modeA_plusX4-X4

[680] PpsActQpOffsetCbCrModeB=pps_act_cbcr_qp_offset_modeB_plusX5-X5

[681] Herein, X1, X2, X3, X4 and X5 may indicate predefined constant values. They may have identical meanings or different meanings, or only some may have identical meanings. In an embodiment, for bitstream alignment, the values of PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, PpsActQpOffsetCbCrModeA, and PpsActQpOffsetCbCrModeB may be constrained such that they have values ranging from -12 to 12.

[682] According to the definition of variables, the quantization parameter qP can be determined as follows. First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[683] equation 81

[684] qP=Qp' _Y

[685] ActQpOffset=PpsActQpOffsetY

[686] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP may be retrieved as shown in the equation below.

[687] equation 82

[688] qP=Qp' _CbCr

[689] Alternatively, when cIdx has a value of 1, the qP and ACT Qp offset can be extracted as shown in the equation below.

[690] equation 83

[691] qP=Qp' _Cb

[692] ActQpOffset=PpsActQpOffsetCb

[693] Alternatively, when cIdx has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[694] equation 84

[695] qP=Qp' _Cr

[696] ActQpOffset=PpsActQpOffsetCr

[697] In addition, when cIdx does not have a value of 0 and TuCResMode[xTbY][yTbY] does not have a value of 0, the ACT Qp offset can be extracted as shown in the equation below.

[698] equation 85

[699] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])?PpsActQpOffsetCbCrModeA:PpsActQpOffsetCbCrModeB

[700] Meanwhile, in another embodiment, when TuCResMode[xTbY][yTbY] has a value of 2, ActQpOffset can be retrieved as shown in the equation below.

[701] equation 86

[702] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])?(PPsQpOffsetCbCrModeA+slice_act_CbCr_qp_offset_ModeA):(PPsQpOffsetCbCrModeB+slice_act_CbCr_qp_offset_ModeB)

[703] In another embodiment of signaling ACT Qp offsets, only ACT QP offsets for Y, Cb and Cr may be signaled as indicated in the syntax table of FIG. 41. ACT QP offset for the combined CbCr can be derived from PpsActQpOffsetY, PpsActQpOffsetCb and/or PpsActQpOffsetCr.

[704] In an embodiment, the ACT Qp offset for CbCr may be set to the value PpsActQpOffsetCb. In another embodiment, the ACT Qp offset for CbCr may be set to a value identical to the value of PpsActQpOffsetCb in the case of a combined CbCr mode in which tu_cbf_cb has a non-zero value, or may be set to a value identical to the value of PpsActQpOffsetCr in the case of a combined CbCr mode in which in which tu_cbf_cb has a value of 0. Alternatively, it can be specified the other way around.

[705] FIG. 41 is a view showing another embodiment of a syntax table in which the ACT Qp offset is signaled to the PPS. According to the definition of syntactic elements in FIG. 41, the quantization parameter qP may be determined as follows. First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[706] equation 87

[707] qP=Qp' _Y

[708] ActQpOffset=PpsActQpOffsetY

[709] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[710] equation 88

[711] qP=Qp' _CbCr

[712] ActQpOffset=(cIdx==1)?PpsActQpOffsetCb:PpsActQpOffsetCr

[713] Meanwhile, in another embodiment, the ActQpOffset value may be determined as follows.

[714] equation 89

[715] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])?PpsActQpOffsetCb:PpsActQpOffsetCr

[716] Alternatively, when cIdx has a value of 1, the qP and ACT Qp offset can be extracted as shown in the equation below.

[717] equation 90

[718] qP=Qp' _Cb

[719] ActQpOffset=PpsActQpOffsetCb

[720] Alternatively, when cIdx has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[721] equation 91

[722] qP=Qp' _Cr

[723] ActQpOffset=PpsActQpOffsetCr

[724] Embodiment 7. Transmission of Qp offsets in ACT signaling at several levels

[725] In an embodiment, the ACT QP offset may be signaled at multiple layers. In addition to signaling the ACT Qp offset at one layer, such as PPS, similar to the above Embodiment 6, the ACT QP offset may be signaled at a lower layer (for example, a slice header, a frame header, or other types of headers, suitable for Qp control).

[726] Two embodiments are described later in this document. Fig. 42 and 43 show examples in which the ACT QP offset is signaled via a slice header and a frame header. Thus, the ACT QP offset can be signaled at multiple levels.

[727] The syntax elements shown in FIG. 1 are described later in this document. 42 and 43. The syntax element pps_slice_act_qp_offsets_present_flag may indicate whether or not the syntax elements slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset and slice_act_cbcr_qp_offset, which are described below, are present in the slice header.

[728] For example, the first value (eg, 0) of pps_slice_act_qp_offsets_present_flag may indicate that slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset are not present in the slice header.

[729] For example, the second value (eg, 1) of pps_slice_act_qp_offsets_present_flag may indicate that slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset are present in the slice header.

[730] The slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset syntax elements may specify offsets for the quantization parameter values qP for the luma component, the Cb component, the Cr component, and the combined CbCr component, respectively. The values of slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset and slice_act_cbcr_qp_offset can be limited so that they have values ranging from -12 to 12. When the values of slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset and slice_act_cbcr_qp_ offset are not present in the bitstream, each value can be set to 0. PpsActQpOffsetY values +slice_act_y_qp_offset, PpsActQpOffsetCb+slice_act_cb_qp_offset, PpsActQpOffsetCr+slice_act_cr_qp_offset and PpsActQpOffsetCbCr+slice_act_cbcr_qp_offset can be limited so that they also have values between -12 and 12.

[731] Various modified embodiments for transmitting the Qp-offset ACT signaling for the combined CbCr at the PPS layer may be used. For example, a single QP offset may be signaled for a combined CbCr, multiple ACT Qp offsets may be signaled for a combined CbCr of different modes, or no ACT Qp offsets for a combined CbCr are signaled, a way to extract this is by using ACTQpOffsets for Y, Cb and Cr and/or jointCbCr mode may be used when signaling is done via the slice header.

[732] Two modified embodiments are shown in FIG. 44 and 45. Figs. 44 shows an embodiment in which the ACT Qp offset is signaled in a slice header. Fig. 45 shows another embodiment in which the ACT Qp offset is signaled in the slice header. In fig. 45, only the ACT Qp offsets for Y, Cb and Cr may be signaled, and the slice-level ACT QP offset for the combined CbCr may be extracted from slice_act_y_qp_offset, slice_act_cb_qp_offset and/or slice_act_cr_qp_offset. This may be determined based on the jointCbCr mode type. In an embodiment, the slice-level ACT Qp offset for CbCr may be set to a value identical to the slice_act_cb_qp_offset value. In another embodiment, in the case of a combined CbCr mode having tu_cbf_cb having a non-zero value, the slice level ACT Qp offset for the combined CbCr may be set to a value identical to the value of slice_act_cb_qp_offset. In addition, in the case of a combined CbCr mode having tu_cbf_cb having a value of 0, the slice-level ACT Qp offset for the combined CbCr may be set to a value identical to the value of slice_act_cr_qp_offset.

[733] Meanwhile, in another embodiment, the syntax element may be signaled in a slice header or a frame header. To implement this, encoding/decoding can be performed as follows.

[734] - The pps_picture_slice_act_qp_offsets_present_flag, indicating whether or not an ACT Qp offset is present in the frame header or slice header, may be signaled in the PPS.

[735] - When ACT is applicable and the value of pps_picture_slice_act_qp_offsets_present_flag is the second value (eg, 1), the flag pic_act_qp_offsets_present_flag indicating whether or not the ACT Qp offset is present in the frame header is signaled in the frame header. Herein, the second value (eg, 1) of pic_act_qp_offsets_present_flag may indicate that ACT Qp offsets for all frame slices corresponding to the frame header are provided in the frame header.

[736] - The first value (eg 0) of pic_act_qp_offsets_present_flag may indicate that ACT Qp offsets for all frame slices corresponding to the frame header are not provided in the frame header. For example, when ACT is applicable and the value of pps_picture_slice_act_qp_offsets_present_flag is the second value (eg, 1) and the value of pic_act_qp_offsets_present_flag is the first value (eg, 0), the ACT Qp offset for the slice may be provided in the slice header.

[737] FIG. 46 is a view showing a PPS syntax table in which pps_pic_slice_act_qp_offsets_present_flag is signaled. The pps_pic_slice_act_qp_offsets_present_flag syntax element may indicate whether or not an ACT Qp offset is provided in the frame header and/or slice header. For example, the first value (eg, 0) of pps_pic_slice_act_qp_offsets_present_flag may indicate that the ACT Qp offset is not provided in the frame header and the slice header. A second value (eg, 1) pps_pic_slice_act_qp_offsets_present_flag may indicate that the ACT Qp offset is provided in the frame header or slice header. When pps_pic_slice_act_qp_offsets_present_flag is not provided in the bitstream, the value of pps_pic_slice_act_qp_offsets_present_flag may be determined to be a constituent of the first value (eg, 0).

[738] FIG. 47 is a view showing a frame header syntax table for transmission in Qp offset signaling ACT. The pic_act_qp_offsets_present_flag syntax element may indicate whether or not an ACT Qp offset is provided in the frame header. The first value (eg 0) of pic_act_qp_offsets_present_flag may indicate that the ACT Qp offset is not provided in the frame header, but in the slice header. A second value (eg 1) pic_act_qp_offsets_present_flag may indicate that the ACT Qp offset is provided in the frame header. When the value of pic_act_qp_offsets_present_flag is not provided in the bitstream, the value may be defined to be 0.

[739] FIG. 48 is a view showing a slice header syntax table for transmission in Qp offset signaling ACT. In the syntax table of FIG. 48, the syntax elements slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset and slice_act_cbcr_qp_offset may specify offsets for the quantization parameter values qP for the luminance, Cb and Cr component. The values slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset and slice_act_cbcr_qp_offset can have values ranging from -12 to 12. In addition, the values PpsActQpOffsetY+slice_act_y_qp_offset, PpsActQpOffsetCb+slice_act_cb_qp_offset and Pps ActQpOffsetCr+slice_act_cr_qp_offset can be limited so that they also have a value range from -12 to 12.

[740] Meanwhile, in the case in which the values of slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset and slice_act_cbcr_qp_offset are not provided in the bit stream, when the value of pps_pic_slice_act_qp_offsets_present_flag is the first value (for example, 0), the values of slice_act_y_qp_offset, slice_act_c b_qp_offset and slice_act_cr_qp_offset can be defined to be 0. Alternatively, когда значение pps_pic_slice_act_qp_offsets_present_flag составляет второе значение (например, 1), значения slice_act_y_qp_offset, slice_act_cb_qp_offset и slice_act_cr_qp_offset могут определяться равными значениям, идентичным значениям pps_act_y_qp_offset, pps_act_cb_qp_offset и pps_act_cr_qp_offset, соответственно.

[741] Meanwhile, when the ACT Qp offset is present in the slice header and in the frame header, the final offset value used to extract the qP value may be determined as a value that is obtained by summing the offset value transmitted in the overhead signals in PPS, and the offset value signaled in the slice header or frame header.

[742] More specifically, in an embodiment, the quantization parameter qP may be determined as follows. First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[743] equation 92

[744] qP=Qp' _Y

[745] ActQpOffset=PPsQpOffsetY+slice_act_y_qp_offset

[746] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[747] equation 93

[748] qP=Qp' _CbCr

[749] ActQpOffset=PPsQpOffsetCbCr+slice_act_CbCr_qp_offset

[750] Alternatively, when cIdx has a value of 1, the qP and ACT Qp offset can be extracted as shown in the equation below.

[751] equation 94

[752] qP=Qp' _Cb

[753] ActQpOffset=PpsActQpOffsetCb+slice_act_Cb_qp_offset

[754] Alternatively, when cIdx has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[755] equation 95

[756] qP=Qp' _Cr

[757] ActQpOffset=PpsActQpOffsetCr+slice_act_Cr_qp_offset

[758] In another embodiment, when multiple ACT Qp offsets for the combined CbCr are signaled, the ActQpOffset for the combined CbCr may be determined as follows.

[759] First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[760] equation 96

[761] qP=Qp' _Y

[762] ActQpOffset=PPsQpOffsetY+slice_act_y_qp_offset

[763] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP may be retrieved as shown in the equation below.

[764] equation 97

[765] qP=Qp' _CbCr

[766] Alternatively, when cIdx has a value of 1, the qP and ACT Qp offset can be extracted as shown in the equation below.

[767] equation 98

[768] qP=Qp' _Cb

[769] ActQpOffset=PpsActQpOffsetCb+slice_act_Cb_qp_offset

[770] Alternatively, when cIdx has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[771] equation 99

[772] qP=Qp' _Cr

[773] ActQpOffset=PpsActQpOffsetCr+slice_act_Cr_qp_offset

[774] In addition, when cIdx does not have a value of 0 and TuCResMode[xTbY][yTbY] does not have a value of 0, the ACT Qp offset can be extracted as shown in the equation below.

[775] equation 100

[776] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])?(PPsQpOffsetCbCrModeA+slice_act_CbCr_qp_offset_ModeA):(PPsQpOffsetCbCrModeB+slice_act_CbCr_qp_offset_ModeB)

[777] In yet another embodiment, when the ACT Qp offset for the combined CbCr is not provided, qP and ActQpOffset for the Y, Cb and/or Cr components are determined, and the ActQpOffset for the combined CbCr can be determined using the ACT Qp- displacements of Y-, Cb- and/or Cr-components as follows. For example, in the above-described embodiment, when TuCResMode[xTbY][yTbY] associated with equation 97 is set to 2, the qP calculation step may be changed and executed as follows.

[778] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[779] equation 101

[780] qP=Qp' _CbCr

[781] ActQpOffset=(cIdx==1])?(PPsQpOffsetCb+slice_act_Cb_qp_offset):(PPsQpOffsetCr+slice_act_Cr_qp_offset)"

[782] Meanwhile, in another embodiment, the value of ActQpOffset may be determined as indicated in the following equation.

[783] equation 102

[784] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])?(PPsQpOffsetCb+slice_act_Cb_qp_offset):(PPsQpOffsetCr+slice_act_Cr_qp_offset)

[785] Embodiment 8. Method of transmitting several sets of ACT Qp offsets in service signals

[786] In the present embodiment, a method for using the ACT Qp offset list is further described. To this end, the following processing may be performed.

[787] a) Multiple sets of ACT Qp offsets may be signaled in the form of a list in a parameter set (eg SPS or PPS). Each set in the list may include ACT Qp offsets for the Y, Cb, Cr, and combined CbCr components. For simplicity, the list of ACT Qp offsets may be signaled in a set of parameters identical to the set of parameters for signaling the list of chrominance Qp offsets.

[788] b) The number of ACT Qp offset sets in the list may be identical to the number of chrominance Qp offset sets signaled in the PPS.

[789] c) The Qp offset ACT used to extract the qP for each encoding basis may be a Qp offset ACT belonging to a list having an index (eg cu_chroma_qp_offset_idx) for the chroma Qp offset for the encoding basis. .

[790] d) As an alternative to b) and c), the following may be performed.

[791] - The number of ACT Qp offset sets in the list may be signaled. The number of ACT Qp offset sets in the list may differ from the number of chrominance Qp offset sets.

[792] - When ACT is applicable, an index indicating the ACT index of the Qp offset used for the encoding basis may be signaled.

[793] Without departing from the above concept, the syntax for signaling a list of ACT Qp offsets may be used as shown in FIG. 49. For example, when cu_act_enabled_flag has a value of 1, pps_act_y_qp_offset, pps_act_cb_qp_offset, pps_act_cr_qp_offset, and pps_act_cbcr_qp_offset can be used to define offsets that should be applied to the qP values of the quantization parameters for the luminance component, Cb component, Cr component, and united CbCr, respectively.

[794] When the values pps_act_y_qp_offset, pps_act_cb_qp_offset, pps_act_cr_qp_offset and pps_act_cbcr_qp_offset are not present, each value may be retrieved as 0.

[795] When the value of cu_act_enabled_flag is a second value (for example, 1), and the value of cu_chroma_qp_offset_flag is a second value (for example, 1), act_y_qp_offset_list[i], act_cb_qp_offset_list[i], act_cr_qp_offset_list[i], and act_cbcr_qp_offset_list[i] can be used to to determine offsets applied to the quantization parameter value qP for the luma component, the Cb component, the Cr component, and the combined CbCr component, respectively. When the values act_y_qp_offset_list[i], act_cb_qp_offset_list[i], act_cr_qp_offset_list[i] and act_cbcr_qp_offset_list[i] are not present, each value may be retrieved as 0.

[796] In the present embodiment, the quantization parameter qP may be determined as follows. First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[797] equation 103

[798] qP=Qp' _Y

[799] ActQpOffset=pps_act_y_qp_offset+(cu_chroma_qp_offset_flag)?act_y_qp_offset_list[cu_chroma_qp_offset_idx]:0+slice_act_y_qp_offset. Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the qP and ACT Qp offset can be extracted as shown in the equation below.

[800] equation 104

[801] qP=Qp' _CbCr

[802] ActQpOffset=pps_act_cbcr_qp_offset+(cu_chroma_qp_offset_flag)?act_cbcr_qp_offset_list[cu_chroma_qp_offset_idx]:0+slice_act_cbcr_qp_offset

[803] Alternatively, when cIdx has a value of 1, the qP and ACT Qp offset can be extracted as shown in the equation below.

[804] equation 105

[805] qP=Qp' _Cb

[806] ActQpOffset=pps_act_cb_qp_offset+(cu_chroma_qp_offset_flag)?act_cb_qp_offset_list[cu_chroma_qp_offset_idx]:0+slice_act_cb_qp_offset. Alternatively, when cIdx has a value of 2, qP and ACT Qp offset can be extracted as shown in the equation below.

[807] equation 106

[808] qP=Qp' _Cr

[809] ActQpOffset=pps_act_cr_qp_offset+(cu_chroma_qp_offset_flag)?act_cr_qp_offset_list[cu_chroma_qp_offset_idx]:0+slice_act_cr_qp_offset

Option 9 implementation. ACT color space conversion method to be applied to lossless and lossy encoding

[810] The conversion (transformation) between color spaces based on the matrices for forward conversion and inverse conversion described above can be organized as follows.

[811]

Table 3 Direct conversion: GBR to YCgCo Reverse Conversion: YCgCo to GBR Y=((G<<1)+R+B+2)>>2
Cg=((G<<1)-R-B+2)>>2
Co=((RB)<<1)+2)>>2 t=Y-Cg
G=Y+Cg
B=t-Co=Y-Cg-Co
R=t+Co=Y-Cg+Co

[812] The conversion cannot achieve restoration to the original state because losses of some values occur in the Co and Cg processing. For example, when a sample RGB color space value is converted to a sample YCgCo color space value, and the resulting value is converted back to the resulting RGB color space value, the value of the original sample is not completely restored. Therefore, the transformation according to Table 3 cannot be used for lossless coding. It is necessary to improve the color space conversion algorithm such that sample value loss does not occur after color space conversion even when lossless encoding is applied. Embodiments 9 and 10 describe a color space conversion algorithm capable of application to lossless encoding as well as lossy encoding.

[813] In the embodiment described below, a method of performing ACT is described by using a reversible color space transform that is applicable to lossy encoding as well as lossless encoding. This reversible color space transformation can be applied to the encoding and decoding methods described above. ACT Qp offset can also be adjusted for the color space conversion below. The color space conversion according to the embodiment can be performed as shown in the following equation. For example, direct conversion from GBR color space to YCgCo color space can be performed according to the following equation.

[814] equation 107

[815] Co=R-B;

[816] t=B+(Co>>1);

[817] Cg=G-t;

[818] Y=t+(Cg>>1);

[819] In addition, the inverse conversion from YCgCo color space to GBR color space can be performed according to the following equation.

[820] equation 108

[821] t=Y-(Cg>>1)

[822] G=Cg+t

[823] B=t-(Co>>1)

[824] R=Co+B

[825] The conversion between the YCgCo color space and the RGB color space according to the above equations is reversible with respect to the original state. Thus, color space conversion according to the equations supports ideal restoration. For example, a sampled value is maintained even if a reverse conversion is performed after a forward conversion. Accordingly, the color space transformation according to the equations can be called "reversible YCgCo-R color transformation". As used herein, R may stand for reversible, which means that restoration to the original state is achieved. The YCgCo-R transform may be provided by increasing the Cg and Co bit depths by 1 compared to the existing transform. If this condition is satisfied, other types of invertible transform, such as the transform described above, can be used.

[826] Since the conversion as shown in the above equations has a different normal value with respect to the above-described conversion, the ACT Qp offsets for Y, Cg and Co can be adjusted to compensate for the change in dynamic range due to the color space conversion.

[827] It is described that when the above-described transformation is applied, the QCT Qp offset according to the embodiment may have a value of (-5, -5, -5) for Y, Cg and Co. However, when the invertible transform according to the present embodiment is applied, as the QCT of the Qp offset according to the embodiment, a value other than (-5, -5, -5) may be specified. For example, as the QCT Qp offset according to the embodiment, the value (-5, 1, 3) for Y, Cg and Co can be used.

[828] In another embodiment, the ACT QP offset may be signaled via a bit stream, similar to embodiment 6 or 7 described above.

[829] For example, when the YCgCo-R transform described above is used in conjunction with the ACT QP offset (-5, 1, 3), it should be noted that encoding loss is not present in a lossy encoding environment (eg, QP 22 , 27, 32, 37), as shown in the drawing below. In addition, it should be noted that when ACT is applied, an encoding performance of 5% is additionally obtained in the lossless encoding implementation.

[830]

Table 4 Subsequence Y U V RGB, TGM 1080p 0.0% 0.2% 0.1% RGB, TGM 720p 0.2% -0.1% 0.1% RGB, animation -0.1% -0.1% 0.0% RGB, mixed content -0.1% 0.0% -0.1% RGB, camera-captured content -0.3% 0.2% -0.3% All in full (RGB) 0.0% 0.0% 0.0%

[831] The VVC specification for incorporating an integrated ACT matrix may be described as the following table.

[832]

Table 5 Residue modification process for blocks using color space transformation
The inputs to this process are as follows:
- nTbW variable indicating the block width,
- nTbH variable indicating the height of the block,
- array r _Y (nTbW)x(nTbH) of residual samples of brightness signals with elements r _Y [x][y],
- array r _Cb (nTbW)x(nTbH) of residual samples of chrominance signals with elements with elements r _Cb [x][y],
- array r _Cr (nTbW)x(nTbH) of residual samples of chrominance signals with elements r _Cr [x][y].
The findings from this process are as follows:
- modified array r _Y (nTbW)x(nTbH) of residual brightness signal samples,
- modified array r _Cb (nTbW)x(nTbH) of residual chrominance signal samples,
- modified array r _Cr (nTbW)x(nTbH) of residual samples of chrominance signals.
Arrays r _Y , r _Cb and r _Cr (nTbW)x(nTbH) of residual samples are modified as follows:
tmp=r _Y [x][y]-(r _Cb [x][y]>>1))
r _Y [x][y]=tmp+r _Cb [x][y]
r _Cb [x][y]=tmp-(r _Cr [x][y]>>1)
r _Cr [x][y]=r _Cb [x][y]+r _Cr [x][y]

[833] For example, residual sample arrays r _Y , r _Cb and r _Cr of size (nTbW)x(nTbH) may be updated as follows.

[834] equation 109

[835] tmp=r _Y [x][y]-(r _Cb [x][y]>>1)

[836] r _Y [x][y]=tmp+r _Cb [x][y]

[837] r _Cb [x][y]=tmp-(r _Cr [x][y]>>1)

[838] r _Cr [x][y]=r _Cb [x][y]+r _Cr [x][y]

[839] Embodiment 10. A method for performing ACT to perform multiple color conversions based on explicit signaling

[840] In the present embodiment, at least one color conversion may be performed by ACT. Which color conversion is to be performed may be determined by flag(s) signaled in the bit stream. The flag(s) may be signaled at multiple levels, eg SPS, PPS, frame header and slice, or at identifiable granularities.

[841] In an embodiment, a predetermined flag may be signaled to indicate which ACT is being applied. For example, when the flag is set to 1, ACT based on reversible color conversion can be applied. When the flag is set to 0, ACT based on irreversible color conversion can be applied.

[842] In another embodiment, a predefined flag for ACT may be signaled to indicate which color mapping is being used. An example of the syntax signaled in the SPS is shown in FIG. 50. Next, the syntax element of FIG. 50. The syntax element sps_act_reversible_conversion may indicate whether or not a conversion equation that is irreversible with respect to the original state is used. The first value (eg 0) of sps_act_reversible_conversion may indicate that ACT uses a conversion equation that is irreversible with respect to the original state. A second value (eg 1) of sps_act_reversible_conversion may indicate that ACT uses a conversion equation that is reversible with respect to the original state.

[843] Accordingly, a lossyCoding variable indicating whether or not lossy coding is performed may be set as indicated in the following equation.

[844] equation 110

[845] lossyCoding=(!sps_act_reversible_conversion)

[846] By using the lossyCoding flag, the pseudo code for the decoding device to perform the inverse conversion from YCgCo to GBR during the decoding process can be expressed as follows.

[847] equation 111

[848] If(sps_act_reversible_conversion==1)

[849]{

[850] //Reversible YCgCo-R transformation

[851] t=Y-(Cg>>1)

[852] G=Cg+t

[853] B=t-(Co>>1)

[854] R=Co+B

[855] }

[856] else {

[857] t=Y-Cg

[858] G=Y+Cg

[859] B=t-Co

[860] R=t+Co

[861] }

[862] Accordingly, the VVC specification shown in Table 5 of Embodiment 9 may be modified as shown in the table below.

[863]

Table 6 Residue modification process for blocks using color space transformation
The inputs to this process are as follows:
- nTbW variable indicating the block width,
- nTbH variable indicating the height of the block,
- array r _Y (nTbW)x(nTbH) of residual samples of brightness signals with elements r _Y [x][y],
- array r _Cb (nTbW)x(nTbH) of residual samples of chrominance signals with elements r _Cb [x][y],
- array r _Cr (nTbW)x(nTbH) of residual samples of chrominance signals with elements r _Cr [x][y].
The findings from this process are as follows:
- modified array r _Y (nTbW)x(nTbH) of residual brightness signal samples,
- modified array r _Cb (nTbW)x(nTbH) of residual chrominance signal samples,
- modified array r _Cr (nTbW)x(nTbH) of residual samples of chrominance signals.
Arrays r _Y , r _Cb and r _Cr (nTbW)x(nTbH) of residual samples are modified as follows:
- If sps_act_reversible_conversion is 1, the r _Y , r _Cb and r _Cr (nTbW)x(nTbH) arrays of residual samples are modified as follows:
tmp=r _Y [x][y]-(r _Cb [x][y]>>1)
r _Y [x][y]=tmp+r _Cb [x][y]
)
r _Cb [x][y]=tmp-(r _Cr [x][y]>>1)
r _Cr [x][y]=r _Cb [x][y]+r _Cr [x][y]
- Otherwise, the r _Y , r _Cb and r _Cr (nTbW)x(nTbH) arrays of residual samples are modified as follows:
tmp=r _Y [x][y]-r _Cb [x][y]
)
r _Y [x][y]=r _Y [x][y]+r _Cb [x][y]
)
r _Cb [x][y]=tmp-r _Cr [x][y]
)
r _Cr [x][y]=tmp+r _Cr [x][y]

[864] According to the above table, the residual update process using color space conversion can use the following parameters as inputs to the current process.

- nTbW variable indicating the block width

[865] - nTbH variable indicating the block height

[866] - array r _Y of size (nTbW)x(nTbH) for residual sampling of brightness signals, consisting of elements r _Y [x][y],

[867] - array r _Cb of size (nTbW)x(nTbH) for residual sampling of chrominance signals, consisting of elements r _Cb [x][y],

[868] - array r _Cr of size (nTbW)x(nTbH) for residual sampling of chrominance signals, consisting of elements r _Cr [x][y],

[869] The output for the current process is as follows:

[870] - updated r _Y array of size (nTbW)x(nTbH) for residual brightness signal sampling,

[871] - updated r _Cb array of size (nTbW)x(nTbH) for residual chrominance signal sampling,

[872] - updated r _Cr array of size (nTbW)x(nTbH) for residual chrominance signal sampling,

[873] By executing the current process, the residual sample arrays r _Y , r _Cb and r _Cr of size (nTbW)x(nTbH) can be updated as follows.

[874] First, when the value of sps_act_reversible_conversion is the second value (eg, 1), the residual sample arrays r _Y , r _Cb , and r _Cr of size (nTbW)x(nTbH) can be updated as shown in the equation below.

[875] equation 112

[876] tmp=r _Y [x][y]-(r _Cb [x][y]>>1)

[877] r _Y [x][y]=tmp+r _Cb [x][y]

)

[878] r _Cb [x][y]=tmp-(r _Cr [x][y]>>1)

[879] r _Cr [x][y]=r _Cb [x][y]+r _Cr [x][y]

[880] Otherwise (eg, when the value of sps_act_reversible_conversion is the first value (eg, 0)), the residual sample arrays r _Y , r _{Cb ,} and r _Cr of size (nTbW)x(nTbH) may be updated as shown in the equation below.

[881] equation 113

[882] tmp=r _Y [x][y]-r _Cb [x][y]

[883] r _Y [x][y]=r _Y [x][y]+r _Cb [x][y]

[884] r _Cb [x][y]=tmp-r _Cr [x][y]

[885] r _Cr [x][y]=tmp+r _Cr [x][y]

[886] The inverse YCgCo transformation and the inverse YCgCo-R transformation have some similarities. In a transformation that is reversible with respect to the original state, where Cg and Co are replaced by Cgp'=Cg<<1 and Cop'=Co<<1, this can work as a lossy inverse transformation. The equation below shows an embodiment of this.

[887] equation 114

[888] t=Y-(Cgp'>>1)=Y-Cg

[889] G=Cgp'+t=Y+Cg

[890] B=t-(Cop'>>1)=t-Co=Y-Cg-Co

[891] R=Cop'+B=t+Co=Y-Cg+Co

[892] Accordingly, in an alternative embodiment, instead of maintaining two color transformations, only a transformation that is invertible with respect to the original state may be used. In the case of lossy encoding, the Cg and Co components can be scaled by a factor of 1/2 when the encoder is running, and can be scaled by a factor of two when the decoder is running. This ensures that the integrated transform can be used even when both lossy and lossless cases are supported. In addition, it has the additional advantage that the bit depth does not change even when lossy encoding is performed.

[893]

Table 7 Reverse Conversion: YCgCo to GBR If(lossyCoding)
{
Cg=Cg<<1;
Co=Co<<1;
}
t=Y-(Cg>>1)
G=Cg+t
B=t-(Co>>1)
R=Co+B

[894] In an embodiment, a flag (eg, actShiftFlag) indicating which ACT transformation is used may be used according to the syntax of FIG. 51. In the syntax table of FIG. 51, the sps_act_shift_flag syntax element may indicate whether or not color component shift execution is applied at the time the ACT is applied. For example, the first value (eg, 0) of sps_act_shift_flag may indicate that color component shifting is not applied while ACT is applied. A second value (eg, 1) of sps_act_shift_flag may indicate that color component shift execution is applied at the time that ACT is applied. The actShiftFlag variable can be set to the value of sps_act_shift_flag. Pseudo code to implement the inverse conversion from YCgCo to GBR in a decoder can be written using actShiftFlag as follows.

[895]

Table 8 Reverse Conversion: YCgCo to GBR If(actShiftFlag)
{
Cg=Cg<<1;
Co=Co<<1;
}
t=Y-(Cg>>1)
G=Cg+t
B=t-(Co>>1)
R=Co+B

[896] Embodiment 11. How to do ACT to perform multiple color conversions using conversion type extraction

In an embodiment, at least one color transformation may be used when performing ACT. In addition, what type of color conversion is used can be derived based on other bitstream information.

[897] In an embodiment, two types of ACT transformations may be available, including an ACT transformation that is reversible with respect to the original state, and an ACT transformation that is irreversible with respect to the original state. ACT transformation types can be retrieved via transformation types. For example, as identified by the variable tuIsTransformSkip, when the transform type is a skip transform, an ACT transform that is invertible with respect to the original state may be used. Otherwise (for example, when the type of transformation is not a skip transformation), an ACT transformation that is irreversible with respect to the original state may be used. Two types of pseudocode can be used.

[898]

Table 9 Reverse Conversion: YCgCo to GBR If(tuIsTransformSkip)
{
//Reversible YCgCo-R transformation
t=Y-(Cg>>1)
G=Cg+t
B=t-(Co>>1)
R=Co+B
}
else {
t=Y-Cg
G=Y+Cg
B=t-Co
R=t+Co
}

[899]

Table 10 Reverse Conversion: YCgCo to GBR If(tuIsTransformSkip)
{
Cg=Cg<<1;
Co=Co<<1;
}
t=Y-(Cg>>1)
G=Cg+t
B=t-(Co>>1)
R=Co+B

[900] In another embodiment, ACT transform types may be determined based on the QP values. When the Qp value is less than or equal to a predefined threshold value (eg, QpPrimeTsMin), an ACT transformation that is invertible with respect to the original state can be used. Otherwise (for example, when the Qp value exceeds a predetermined threshold), an irreversible ACT transformation may be used.

[901] [902] Embodiment 12. QP extraction method using ACT QP offset

[903] The present embodiment is related to embodiments 1 and 2 described above. In embodiments 1 and 2, it has already been described that the recovered _Qp'Y , _Qp'CbCr , _Qp'Cb and _Qp'Cr are included as QP. The methods described in embodiments 1 and 2 correct the obtained Qp values by using the ACT QP offset and apply significant clipping technology such that the adjusted QP values for scaling the transform coefficients are not outside the effective range.

[904] The present embodiment describes a method for incorporating a QP offset ACT into a QP extraction process to extract _Qp'Y , _Qp'CbCr , _Qp'Cb and _Qp'Cr . The QP extraction process already includes a predefined clipping step such that the extracted QP values are not outside the effective range. Therefore, incorporating the ACT QP offset into the QP extraction process avoids an additional pruning step, simplifies the overall QP extraction step for the transform coefficient scaling process, and ensures that the final QP is not outside the effective range.

[905] As described in the embodiment described above, the ACT QP offset may be pre-specified as a constant or may be signaled via a bitstream. Without departing from consistency, the ACT QP offsets for Y, Cb, Cr and CbCr may be described as ppsActQpOffsetY, ppsActQpOffsetCb, ppsActQpOffsetCr and ppsActQpOffsetCbCr in the description below. ppsActQpOffsetY, ppsActQpOffsetCb, ppsActQpOffsetCr, and ppsActQpOffsetCbCr may be constants or variables having values ranging from -M to N. In this document, each of M and N may be set to 12 in the case of lossy encoding and may be set to 0 in the case of lossless encoding. losses in the embodiment. In addition, at least one ACT QP offset can be derived from another ACT QP offset value. For example, ppsActQpOffsetCbCr may be set to a value identical to ppsActQpOffsetCb or ppsActQpOffsetCr based on the combined CbCr mode.

[906] Decoding processing for Qp extraction using the QP offset ACT may be performed as described below. First, in the case of the quantization parameter extraction process, the following parameters can be used for the current process.

[907] - Luminance coordinates (xCb, yCb) indicating the relative coordinates of the upper left luminance sample of the current encoding block relative to the upper left luminance sample of the current frame,

[908] - Variable cbWidth indicating the width of the current encoding block based on luminance samples,

[909] - Variable cbHeight indicating the height of the current encoding block based on the luma sample,

[910] - A treeType variable indicating whether a single tree (SINGLE_TREE) or a double tree is used to segment the current encoding tree node, and indicating when a double tree is used, whether the double tree represents a double tree (DAUL_TREE_LUMA) of signal components brightness or dual tree (DAUL_TREE_CHROMA) of chrominance signal components.

[911] In the present quantization parameter extraction process, the luma signal quantization parameter _Qp'Y and the chrominance signal quantization parameters _Qp'Cb , _Qp'Cr and _Qp'CbCr can be extracted.

[912] After this, the variable Qp _Y can be retrieved as shown in the equation below.

[913] equation 115

[914] Qp _Y =((qP _{Y_PRED} +CuQpDeltaVal+64+2*QpBdOffset)%(64+QpBdOffset))-QpBdOffset

[915] The luminance signal quantization parameter Qp' _Y can be extracted as shown in the following equation.

[916] equation 116

[917] actQpOffsetY=cu_act_enabled_flag[xCb][yCb]?ppsActQpOffsetY:0

[918] Qp' _Y =Qp _Y +QpBdOffset+actQpOffsetY

[919] Qp' _Y =Clip3(0, 63+QpBdOffset, Qp' _Y )

[920] When the value of the ChromaArrayType variable indicating the type of the chroma array is not the first value (eg, 0), and treeType is SINGLE_TREE or DUAL_TREE_CHROMA, the following processing may be performed.

[921] - When the value of treeType is DUAL_TREE_CHROMA, the value of the variable Qp _Y may be set to a value identical to the value of the luminance signal quantization parameter Qp _Y of the luminance coding basis covering the sample location (xCb+cbWidth/2, yCb+cbHeight/2) signals brightness.

[922] - The variables qP _Cb , qP _Cr and qP _CbCr can be retrieved as shown in the equation below.

[923] equation 117

[924] qP _Chroma =Clip3(-QpBdOffset, 63, Qp _Y ) (1118)

[925] qP _Cb =ChromaQpTable[0][qP _Chroma ]

(1119)

[926] qP _Cr =ChromaQpTable[1][qP _Chroma ]

(1120)

[927] qP _CbCr =ChromaQpTable[2][qP _Chroma ]

(1121)

[928] - The chrominance signal quantization parameters Qp' _Cb and Qp' _Cr for the Cb and Cr components and the chrominance signal quantization parameter Qp' _CbCr for the combined Cb-Cr encoding can be extracted as shown in the equation below.

[929] equation 118

[930] actQpOffsetCb=cu_act_enabled_flag[xCb][yCb]?ppsActQpOffsetCb:0

[931] actQpOffsetCr=cu_act_enabled_flag[xCb][yCb]?ppsActQpOffsetCr:0

[932] actQpOffsetCbCr=cu_act_enabled_flag[xCb][yCb]?ppsActQpOffsetCbCr:0

[933] Qp' _Cb =Clip3(-QpBdOffset, 63, qP _Cb +pps_cb_qp_offset+slice_cb_qp_offset+CuQpOffset _Cb +actQpOffsetCb)+QpBdOffset

[934] Qp' _Cr =Clip3(-QpBdOffset, 63, qP _Cr +pps_cr_qp_offset+slice_cr_qp_offset+CuQpOffset _Cr +actQpOffsetCr)+QpBdOffset

[935] Qp' _CbCr =Clip3(-QpBdOffset, 63, qP _CbCr +pps_joint_cbcr_qp_offset +

[936] slice_joint_cbcr_qp_offset+CuQpOffset _CbCr +actQpOffsetCbCr)+QpBdOffset

[937] [938] Then, the dequantization process for the conversion factor can be performed, and the following information can be used as input to the current process.

[939] - Luminance signal coordinates (xTbY, yTbY), referring to the relative coordinates of the top left sample of the current luminance transform block relative to the top left luminance sample of the current frame,

[940] - Variable nTbW indicating the width of the conversion block,

[941] - Variable nTbH indicating the height of the transform block,

[942] - Variable predMode indicating the coding basis prediction mode,

[943] - Variable cIdx indicating the color component of the current block

[944] The output of the dequantization process for the transform factor may be an array of d scaled transform factors. In this document, the size of the array d may be (nTbW)x(nTbH). The individual elements composing it can be identified as d[x][y].

[945] When executing the current process, the quantization parameter qP may be retrieved as follows. When cIdx has a value of 0, qP can be extracted as shown in the equation below.

[946] equation 119

[947] qP=Qp' _Y

[948] Alternatively, when TuCResMode[xTbY][yTbY] is set to 2, the extraction may be performed as shown in the equation below.

[949] equation 120

_[950] qP=Qp' _CbCr

[951] Alternatively, when cIdx has a value of 1, qP can be extracted as shown in the equation below.

[952] equation 121

[953] qP=Qp' _Cb

[954] Alternatively, when cIdx has a value of 2, qP can be extracted as shown in the equation below.

[955] equation 122

[956] qP=Qp' _Cr

[957] The quantization parameter qP may be updated as follows. In addition, the rectNonTsFlag and bdShift variables can be retrieved as follows. When transform_skip_flag[xTbY][yTbY][cIdx] is set to 0, the extraction can be done as shown in the equation below.

[958] equation 123

[959] rectNonTsFlag=(((Log2(nTbW)+Log2(nTbH)) and 1)==1)?1:0

[960] bdShift=BitDepth+rectNonTsFlag+((Log2(nTbW)+Log2(nTbH))/2)-5+pic_dep_quant_enabled_flag. Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] is set to 1 (eg, the transformation of the current transform block is skipped), the extraction can be performed as shown in the equation below.

[961] equation 124

[962] qP=Max(QpPrimeTsMin, qP)

[963] rectNonTsFlag=0

[964] bdShift=10

[965] Encoding and decoding methods

[966] Hereinafter, this document will describe an image encoding method performed by an image encoding device and an image decoding method performed by an image decoding device with reference to FIGS. 52 and 53.

[967] First, the operation of the decoding apparatus is described in the following. An image decoding apparatus according to an embodiment may include a storage device and a processor. The decoding apparatus can perform decoding according to the operation of the processor. For example, as shown in FIG. 52, the decoding apparatus may determine a prediction mode of the current block in step S5210 based on the prediction information obtained from the bit stream. Next, the decoding apparatus may obtain, based on the prediction mode of the current block, a flag indicating whether or not color space conversion is applied to the residual sample of the current block in step S5220. Next, the decoding apparatus may determine the quantization parameter of the current block based on the flag in step S5230. Next, the decoding apparatus may determine the conversion coefficient of the current block based on the quantization parameter in step S5240.

[968] Herein, a decoding apparatus may determine a quantization parameter by applying clipping to the quantization parameter such that the value of the quantization parameter has a value in a predetermined range. Herein, the lower limit value of the predetermined range may be 0. In addition, the upper limit value of the predetermined range may be determined based on a syntax element indicating a sampling bit depth. For example, the upper limit value of the predetermined range may be determined to be 63+QpBdOffset. QpBdOffset denotes the offset of the range of quantization parameters for luma and chrominance signals and can be preset as a predefined constant or can be derived from the bit stream. For example, QpBdOffset may be calculated by multiplying a predefined constant by the value of a syntax element indicating the bit depth of the luma or chroma samples.

[969] In addition, the decoding apparatus may determine a quantization parameter based on the color component of the current block, may determine a quantization parameter offset based on the color component of the current block, and then may reset the quantization parameter by using the quantization parameter offset, thereby determining the quantization parameter. For example, the decoding device may add the quantization parameter offset to the quantization parameter, thereby resetting the quantization parameter.

[970] The quantization parameter offset may be determined as follows. When color space transformation is applied to the residual sample of the current block, and the color component of the current block is a component of luma signals, the quantization parameter offset value may be determined to be -5. Alternatively, when the color space transformation is applied to the residual sample of the current block, and the color component of the current block is the Cb component of the chrominance signals, the quantization parameter offset value may be determined to be 1. Alternatively, when the color space transformation is applied to the residual sample of the current block, and the color component of the current block is the Cr component of the chrominance signals, the quantization parameter offset value can be determined to be 3.

[971] Meanwhile, the residual sample of the current block is determined based on the conversion coefficient of the current block, and the value of the residual sample can be updated based on whether color space conversion is applied or not. For example, when color space transformation is applied, the residual sample value may be updated based on the sample value that is obtained by applying a right shift operation to the residual sample value of the chrominance signal component. In an embodiment, the residual sample luma component value of the residual sample may be updated by adding, with the residual sample luma value, the value obtained by applying a right shift operation to the residual sample value of the Cb chrominance signal component. In addition, the residual sample value Cb of the chrominance signal component of the residual sample may be updated with a value obtained by subtracting a first value and a second value from the residual sample value of the luma signal component, wherein the first value is obtained by applying a right shift operation to the residual sample value Cb- chrominance component, and the second value is obtained by applying a right shift operation to the residual sample value of the chrominance Cr component.

[972] The operation of the encoding apparatus is described later in this document. An image encoding apparatus according to an embodiment may include a memory and a processor. The encoding apparatus may perform encoding according to operation of the processor in a manner that corresponds to decoding by the decoding apparatus. For example, as shown in FIG. 53, the encoding device may determine a prediction mode of the current block in step S5310. Next, the encoding device may determine a quantization parameter of the current block depending on whether or not color space transformation is applied to the residual sample of the current block in step S5320. Next, the encoding device may determine the transform coefficient of the current block based on the quantization parameter in step S5330. Next, the encoding apparatus may encode, based on the prediction mode of the current block, information indicating whether or not color space transformation is applied to the residual sample of the current block in step S5340.

[973] According to the operation of the decoding apparatus described above, the encoding apparatus may determine a quantization parameter by applying clipping to the quantization parameter such that the value of the quantization parameter has a value in a predetermined range. Herein, the lower limit value of the predetermined range may be set to 0. In addition, the upper limit value of the predetermined range may be set to the predetermined upper limit value. The upper limit value may be determined based on the sampling bit depth. For example, the upper limit value of the predetermined range may be determined to be 63+QpBdOffset. As used herein, QpBdOffset denotes the offset of the quantization parameter range of luma signals and chrominance signals, and may be preset as a predefined constant or may be obtained from a bit stream. For example, QpBdOffset may be calculated by multiplying a predefined constant by the value of a syntax element indicating the bit depth of the luma or chroma samples.

[974] In addition, the encoding device may determine a quantization parameter based on the color component of the current block, may determine a quantization parameter offset based on the color component of the current block, and then may reset the quantization parameter by using the quantization parameter offset, thereby determining the quantization parameter. For example, the encoder may add the quantization parameter offset to the quantization parameter, thereby resetting the quantization parameter.

[975] The quantization parameter offset may be determined as follows. When color space transformation is applied to the residual sample of the current block, and the color component of the current block is a component of luma signals, the quantization parameter offset value may be determined to be -5. Alternatively, when the color space transformation is applied to the residual sample of the current block, and the color component of the current block is the Cb component of the chrominance signals, the quantization parameter offset value may be determined to be 1. Alternatively, when the color space transformation is applied to the residual sample of the current block, and the color The component of the current block is the Cr component of the chrominance signals, the quantization parameter offset value can be determined to be 3.

[976] Meanwhile, the residual sample of the current block is determined based on the conversion coefficient of the current block, and the value of the residual sample can be updated based on whether color space conversion is applied or not. For example, when color space transformation is applied, the residual sample value may be updated based on the sample value that is obtained by applying a right shift operation to the residual sample value of the chrominance signal component.

[977] Application option

[978] Although the exemplary methods of the present disclosure described above are presented as a sequence of operations for clarity of description, it is not intended to limit the order in which the steps are performed, and the steps may be performed simultaneously or in a different order as necessary. To implement the method of the present disclosure, the described steps may further include other steps, may include the remaining steps except some steps, or may include other additional steps except some steps.

[979] In the present disclosure, image encoding equipment or image decoding equipment that performs a predetermined operation (step) may perform an operation (step) of confirming a condition or situation for executing the corresponding operation (step). For example, if it is described that a predetermined operation is performed when a predetermined condition is satisfied, the image encoding equipment or the image decoding equipment may perform the predetermined operation after determining whether or not the predetermined condition is satisfied.

[980] The various embodiments of the present disclosure are not intended to be a list of all possible combinations and are intended to describe characteristic aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more of them.

[981] Various embodiments of the present disclosure may be implemented in hardware, firmware, software, or a combination of the above. If the present disclosure is implemented in hardware, the present disclosure may be implemented in application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays. (FPGA), general processors, controllers, microcontrollers, microprocessors, etc.

[982] In addition, the image decoding equipment and image encoding equipment to which embodiments of the present disclosure apply may be included in a multimedia broadcast transceiver device, a mobile communication terminal, a home theater system video device, a digital cinema system video device, a surveillance camera, video chat device, real-time communication device such as video communication, mobile streaming device, storage medium, video recording camera, video-on-demand (VoD) service device, OTT video device ( video over networks), Internet streaming service device, 3D video device, video telephony system video device, medical video device, etc. and can be used to process video or data signals. For example, OTT video devices may include a gaming console, Blu-ray player, Internet-enabled television, home theater system, smartphone, tablet PC, digital video recorder (DVR), etc.

[983] FIG. 54 is a view showing a content streaming system to which an embodiment of the present disclosure is applicable.

[984] Referring to FIG. 54, the content streaming system to which the embodiment(s) of this document applies may mainly include an encoding server, a streaming server, a web server, a media storage, a user device, and a media input device.

[985] The encoding server compresses content input from media input devices such as a smartphone, camera, video recorder, etc. into digital data to form a bitstream, and transmits the bitstream to the streaming server. As another example, when media input devices such as smartphones, cameras, video recorders, etc. directly generate a bitstream, the encoding server may be omitted.

[986] The bit stream may be generated by an image encoding method or image encoding equipment to which an embodiment of the present disclosure is applied, and the streaming server may temporarily store the bit stream while transmitting or receiving the bit stream.

[987] The streaming server transmits media data to the user device based on the user's request through the web server, and the web server serves as a medium for informing the user regarding the service. When a user requests a desired service from a web server, the web server may deliver it to the streaming server, and the streaming server may stream the media data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server serves to manage the command/response between devices in the content streaming system.

[988] The streaming server may receive content from a media storage and/or encoding server. For example, when content is received from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may maintain the bit stream for a predetermined time.

[989] Examples of a user device may include a mobile phone, a smartphone, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable media player (PMP), a navigation device, a tablet PC, tablet PCs, ultrabooks, wearable devices (such as smart watches, smart glasses, head-mounted displays), digital TVs, desktop computers, digital signage system, etc.

[990] Each server in a content streaming system may operate as a distributed server, in which case data received from each server may be distributed.

[991] The scope of the disclosure includes software or computer executable instructions (e.g., operating system, application, firmware, program, etc.) to enable operations according to the methods of various embodiments in hardware or a computer, non-volatile computer-readable medium having such software or instructions stored and executed in hardware or a computer.

Industrial applicability

[992] Embodiments of the present disclosure may be used to encode or decode an image.

Claims (64)

1. An image decoding device containing: Memory device; And at least one processor coupled to a storage device, wherein at least one processor is configured to: obtain color space transformation information for applying the color space transformation to the residual sample of the current block based on a prediction mode being an intra prediction mode and a tree type being a single tree type; retrieve a first quantization parameter for the current encoding unit; extract the final quantization parameter based on adding the value of the first quantization parameter for the current encoding unit to the offset value of the quantization parameters for the current encoding unit; extract a transform coefficient for the current coding unit based on the final quantization parameter for the current coding unit; And extract a residual sample for the current coding unit based on a transform factor for the current coding unit; wherein the current coding unit contains a block of luminance signal components, a block of first color components and a block of second color components, wherein, based on the color space conversion information indicating that color space conversion is not applied to the residual sample of the current coding unit, the quantization parameter offset values for the luminance component block, the first color component block, and the second color component block of the current coding unit are extracted as 0, respectively. wherein, based on the color space transformation information indicating the application of the color space transformation to the residual sample of the current encoding unit, the quantization parameter offset values for the luminance signal component block, the first color component block, and the second color component block of the current encoding unit are extracted as different from each other, the residual sample values for the luma component block, the first color component block, and the second color component block are respectively updated based on the residual sample value for the luma component block, the residual sample value for the first color component block is updated based on subtracting the first value and the second value from the residual sample value for the luma component block, wherein the first value is a value obtained by applying a right shift operation to the residual sample value for the first color component block, and the second value is a value obtained by applying a right shift operation to the residual sample value for the block of the second color components, the residual sample value for the block of the second color components is updated based on the summation of the updated residual sample value for the block of the first color components and the residual sample value for the block of the second color components. 2. Device according to claim 1, wherein, based on the color space transformation information indicating application of the color space transformation to the residual sample of the current encoding unit, the quantization parameter offset value of the luminance signal component block is extracted as a negative value. 3. Device according to claim 1, wherein, based on the color space transformation information indicating application of the color space transformation to the residual sample of the current encoding unit, the quantization parameter offset value of the luminance signal component block is extracted as -5. 4. Device according to claim 1, wherein, based on the color space transformation information indicating application of the color space transformation to the residual sample of the current encoding unit, the quantization parameter offset value of the block of the first color components is extracted as 1. 5. Device according to claim 1, wherein, based on the color space transformation information indicating application of the color space transformation to the residual sample of the current encoding unit, the quantization parameter offset value of the block of the second color components is extracted as 3. 6. Device according to claim 1, wherein the block of first color components is a Cb block of chrominance signals and the block of second color components is a Cr block of chrominance signals. 7. Device according to claim 1, wherein, based on the color space transformation information indicating application of the color space transformation to the residual sample of the current coding unit, the values of the residual samples for the luminance signal component block, the first color component block and the second color component block are further updated based on the sample value obtained by applying the operation right shift to the residual sample value for the block of first color components. 8. Device according to clause 7, wherein the sample value obtained by applying a right shift operation to the residual sample value for the block of first color components corresponds to half of the residual sample value for the block of first color components. 9. Device according to claim 1, wherein, based on the color space transformation information indicating the application of the color space transformation to the residual sample of the current coding unit, the residual sample value for the luma component block is updated based on the summation of the sample value obtained by applying a right shift operation to the residual sample value for the first color block components, with the residual sample value for the luma component block. 10. Device according to clause 9, wherein the sample value obtained by applying a right shift operation to the residual sample value for the block of first color components corresponds to half of the residual sample value for the block of first color components. 11. An image encoding device comprising: Memory device; And at least one processor coupled to a storage device, wherein at least one processor is configured to: extract color space transformation information applied to the residual sample of the current block based on a prediction mode being an intra prediction mode and a tree type being a single tree type; extract a residual sample for the current encoding unit based on the applied color space transformation; extract a transform coefficient for the residual sample of the current coding unit; perform quantization based on the transform coefficient and the final quantization parameter for the current encoding unit; wherein color space conversion information for applying the color space conversion to the residual sample of the current block is encoded into a bitstream, wherein the final quantization parameter is extracted based on adding the value of the first quantization parameter for the current coding unit to the quantization parameter offset value for the current coding unit, wherein the current coding unit contains a block of luminance signal components, a block of first color components and a block of second color components, wherein, based on the color space conversion information indicating that color space conversion is not applied to the residual sample of the current coding unit, the quantization parameter offset values for the luminance component block, the first color component block, and the second color component block of the current coding unit are extracted as 0, respectively. wherein, based on the color space transformation information indicating the application of the color space transformation to the residual sample of the current encoding unit, the quantization parameter offset values for the luminance signal component block, the first color component block, and the second color component block of the current encoding unit are extracted as different from each other, the residual sample values for the luma component block, the first color component block, and the second color component block are respectively updated based on the residual sample value for the luma component block, the residual sample value for the first color component block is updated based on subtracting the first value and the second value from the residual sample value for the luma component block, wherein the first value is a value obtained by applying a right shift operation to the residual sample value for the first color component block, and the second value is a value obtained by applying a right shift operation to the residual sample value for the block of the second color components, the residual sample value for the block of the second color components is updated based on the summation of the updated residual sample value for the block of the first color components and the residual sample value for the block of the second color components. 12. An image data transmission device, comprising: at least one processor configured to receive a bit stream generated by the image encoding method; And a transmitter configured to transmit a stream of bits, wherein the image encoding method comprises the steps of: extracting color space transformation information applied to the residual sample of the current block based on a prediction mode being an intra prediction mode and a tree type being a single tree type; extracting a residual sample for the current encoding unit based on the applied color space transformation; extracting a transform coefficient for the residual sample of the current coding unit; perform quantization based on the transform coefficient and the final quantization parameter for the current encoding unit; wherein color space conversion information for applying the color space conversion to the residual sample of the current block is encoded into a bitstream, wherein the final quantization parameter is extracted based on adding the value of the first quantization parameter for the current coding unit to the quantization parameter offset value for the current coding unit, wherein the current coding unit contains a block of luminance signal components, a block of first color components and a block of second color components, wherein, based on the color space conversion information indicating that color space conversion is not applied to the residual sample of the current coding unit, the quantization parameter offset values for the luminance component block, the first color component block, and the second color component block of the current coding unit are extracted as 0, respectively. wherein, based on the color space transformation information indicating the application of the color space transformation to the residual sample of the current encoding unit, the quantization parameter offset values for the luminance signal component block, the first color component block, and the second color component block of the current encoding unit are extracted as different from each other, the residual sample values for the luma component block, the first color component block, and the second color component block are respectively updated based on the residual sample value for the luma component block, the residual sample value for the first color component block is updated based on subtracting the first value and the second value from the residual sample value for the luma component block, wherein the first value is a value obtained by applying a right shift operation to the residual sample value for the first color component block, and the second value is a value obtained by applying a right shift operation to the residual sample value for the block of the second color components, the residual sample value for the block of the second color components is updated based on the summation of the updated residual sample value for the block of the first color components and the residual sample value for the block of the second color components.