RU2795812C1 - Method and device for image coding/decoding using adaptive color conversion and method for bitstream transmission - Google Patents

Abstract

FIELD: means for video coding.

SUBSTANCE: prediction mode of the current coding unit is determined. Retrieve the colour space transformation applied to the residual sample of the current coding unit based on the prediction mode. Retrieve the residual sample of the current coding unit based on the applied colour space transformation. The transform coefficient for the residual sample of the current coding unit is retrieved. Quantization is performed based on the transform coefficient and final quantization parameter for the current coding unit. Colour space conversion information for applying a colour space conversion to a residual sample of the current coding unit is encoded in a bitstream. The final quantization parameter is derived based on the first quantization parameter for the current coding unit.

EFFECT: increase in the efficiency of video coding.

15 cl, 54 dwg, 10 tbl

Description

The technical field to which the invention belongs

[1] The present disclosure relates to an image encoding/decoding method and apparatus. More specifically, the present disclosure relates to an image encoding/decoding method and apparatus using adaptive color conversion and a method for transmitting a bitstream generated by the image encoding method/device of the present disclosure.

State of the art

[2] Recently, the demand for high-quality high definition images, such as high definition (HD) images and ultra high definition (UHD) images, has been growing in various fields of technology. As the resolution and quality of image data increases, the amount of information or bits to be transmitted increases relatively compared to existing image data. An increase in the amount of information or bits to be transmitted results in an increase in transmission costs and storage costs.

[3] Accordingly, there is a need for a high-performance image compression technology for efficiently transmitting, storing, and reproducing information regarding high-quality, high-resolution images.

The essence of the invention

Technical task

[4] The present disclosure is directed to providing an image encoding/decoding method and apparatus with improved encoding/decoding efficiency.

[5] In addition, the present disclosure is directed to providing an image encoding/decoding method and apparatus for improving encoding/decoding efficiency by performing adaptive color conversion.

[6] In addition, the present disclosure is directed to providing a method for transmitting a bitstream generated by an image encoding method or apparatus according to the present disclosure.

[7] In addition, the present disclosure is directed to providing a recording medium storing a bit stream generated by an image encoding method or apparatus according to the present disclosure.

[8] In addition, the present disclosure is directed to providing a recording medium storing a bitstream that is received, decoded, and used to reconstruct an image by an image decoding apparatus according to the present disclosure.

[9] Those skilled in the art should appreciate that the technical objectives to be achieved in this disclosure are not limited to the above technical objectives, and other technical objectives that are not described herein should be clearly understood from the description below.

Technical solution

[10] According to an aspect of the present disclosure, there is provided an image decoding method performed by an image decoding apparatus, the method including: determining a prediction mode of a current block based on prediction information obtained from a bit stream; obtaining, based on the prediction mode of the current block, a flag indicating whether or not a color space transformation is applied to the residual sample of the current block; determining a quantization parameter of the current block based on the flag; and determining a transform coefficient (transform coefficient) of the current block based on the quantization parameter. Herein, determination of a quantization parameter may be performed by applying a cut to the quantization parameter such that the value of the quantization parameter has a value in a predetermined range.

[11] In addition, according to an aspect of the present disclosure, an image decoding apparatus is provided, including a memory and at least one processor, wherein the at least one processor is configured to determine a prediction mode of a current block based on of the prediction information obtained from the bitstream, obtain, based on the prediction mode of the current block, a flag indicating whether or not a color space transform is applied to the residual sample of the current block, determine a quantization parameter of the current block based on the flag, and determine a transform coefficient of the current block based on quantization parameter. Herein, the processor may be configured to determine a quantization parameter by applying a cut to the quantization parameter such that the value of the quantization parameter has a value in a predetermined range.

[12] In addition, according to an aspect of the present disclosure, an image encoding method is provided by an image encoding device, the method including: determining a prediction mode of a current block; determining a quantization parameter of the current block depending on whether or not a color space transform is applied to the residual sample of the current block; determining a transform coefficient of the current block based on the quantization parameter; and encoding, based on the prediction mode of the current block, information indicating whether or not a color space transformation is applied to the residual sample of the current block. Herein, determination of a quantization parameter may be performed by applying a cut to the quantization parameter such that the value of the quantization parameter has a value in a predetermined range.

[13] In addition, according to another aspect of the present disclosure, a transmission method is provided for transmitting a bit stream generated by an image encoding apparatus or an image encoding method of the present disclosure.

[14] In addition, according to another aspect of the present disclosure, a computer-readable recording medium is provided that stores a bit stream generated by the image encoding method or image encoding apparatus of the present disclosure.

[15] The features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the following detailed description of the present disclosure and do not limit the scope of the present disclosure.

Benefits of the Invention

[16] According to the present disclosure, it is possible to provide an image encoding/decoding method and apparatus with improved encoding/decoding efficiency.

[17] In addition, according to the present disclosure, it is possible to provide an image encoding/decoding method and apparatus capable of improving encoding/decoding efficiency by performing adaptive color conversion.

[18] In addition, according to the present disclosure, it is possible to provide a method for transmitting a bit stream generated by an image encoding method or apparatus according to the present disclosure.

[19] In addition, according to the present disclosure, it is possible to provide a recording medium storing a bit stream generated by the image encoding method or apparatus according to the present disclosure.

[20] In addition, according to the present disclosure, it is possible to provide a recording medium storing a bit stream that is received, decoded, and used to reconstruct an image by the image decoding apparatus according to the present disclosure.

[21] Those skilled in the art should appreciate that the benefits that can be achieved through the present disclosure are not limited to those specifically described above, and other benefits of the present disclosure should be more clearly understood from the description below.

Brief description of the drawings

[22] FIG. 1 is a view schematically showing a video coding system to which an embodiment of the present disclosure is applicable.

[23] FIG. 2 is a view schematically showing an image encoding apparatus to which an embodiment of the present disclosure is applicable.

[24] FIG. 3 is a view schematically showing an image decoding apparatus to which an embodiment of the present disclosure is applicable.

[25] FIG. 4 is a view showing an image segmentation structure according to the embodiment.

[26] FIG. 5 is a view showing an embodiment of a block segmentation type according to the multi-type tree structure.

[27] FIG. 6 is a view showing a signaling mechanism of blocking information in a nested multi-type tree quadtree structure according to the present disclosure.

[28] FIG. 7 is a view showing an embodiment in which a CTU is segmented into multiple CUs.

[29] FIG. 8 is a view showing neighboring reference samples according to an embodiment.

[30] FIG. 9 and 10 are views showing intra prediction according to the embodiment.

[31] FIG. 11 is a view showing an embodiment of a decoding process to which ACT is applied.

[32] FIG. 12 is a view showing an embodiment of a sequence parameter set syntax table in which an ACT-related syntax element is signaled.

[33] FIG. 13-19 are views sequentially showing an embodiment of an encoding basis syntax table in which an ACT-related syntax element is signaled.

[34] FIG. 20 is a view showing the syntax of a coding tree according to an embodiment.

[35] FIG. 21 is a view showing a block diagram of a CABAC according to an embodiment for encoding one syntax element.

[36] FIG. 22-25 are views showing entropy encoding and decoding according to an embodiment.

[37] FIG. 26 and 27 are views showing examples of frame encoding and decoding procedures according to the embodiment.

[38] FIG. 28 is a view showing a hierarchical structure for an encoded picture according to an embodiment.

[39] FIG. 29 is a view showing a BDPCM residual sample encoding method according to an embodiment.

[40] FIG. 30 is a view showing modified quantized residual blocks generated by performing BDPCM according to an embodiment.

[41] FIG. 31 is a flowchart showing a procedure for encoding a current block by applying BDPCM in an image encoding apparatus according to an embodiment.

[42] FIG. 32 is a flowchart showing a procedure for recovering a current block by applying BDPCM in an image decoding apparatus according to an embodiment.

[43] FIG. 33-35 are views schematically showing the syntax for signaling information regarding BDPCM.

[44] FIG. 36-51 are views showing a syntax table for signaling an ACT syntax element according to each individual embodiment of the present disclosure.

[45] FIG. 52 is a view showing an image decoding method according to an embodiment.

[46] FIG. 53 is a view showing an image encoding method according to an embodiment.

[47] FIG. 54 is a view showing a content streaming system to which an embodiment of the present disclosure is applicable.

Optimal Mode for Carrying Out the Invention

[48] Hereinafter, embodiments of the present disclosure are described in detail with reference to the accompanying drawings so that they can be easily implemented by those skilled in the art. However, the present disclosure may be implemented in any number of forms and is not limited to the embodiments described herein.

[49] In describing the present disclosure, if it is determined that a detailed description of an associated known function or construct would unnecessarily ambiguize the scope of the present disclosure, the detailed description thereof is omitted. In the drawings, parts not related to the description of the present disclosure are omitted and like reference numbers are attached to like parts.

[50] In the present disclosure, when a component is "connected", "coupled", or "linked" to another component, this may include not only a direct connection-based relationship, but also an indirect connection-based relationship in which an intermediate component is present. In addition, when a component "includes" or "has" other components, it means that other components may be included in addition to, rather than excluding, other components, unless otherwise noted.

[51] In the present disclosure, the terms "first", "second", etc. may be used only for the purpose of distinguishing one component from other components and do not limit the order or importance of components unless otherwise noted. Accordingly, within the scope of this disclosure, the first component in one embodiment may be referred to as the "second component" in another embodiment, and similarly, the second component in one embodiment may be referred to as the "first component" in another embodiment.

[52] In the present disclosure, components that are different from each other are intended to clearly describe each feature and do not mean that the components are necessarily separated. Thus, multiple components may be integrated and implemented in a single hardware or software module, or a single component may be distributed and implemented in multiple hardware or software modules. Therefore, even if not stated otherwise, such embodiments in which components are integrated or a component is distributed are also included within the scope of the present disclosure.

[53] In the present disclosure, the components described in various embodiments do not necessarily mean essential components, and some components may be optional components. Accordingly, an embodiment consisting of a subset of the components described in the embodiment is also included within the scope of the present disclosure. In addition, embodiments including other components, in addition to the components described in the various embodiments, are included within the scope of the present disclosure.

[54] The present disclosure relates to image encoding and decoding, and the terms used in the present disclosure may have a general meaning widely used in the technical field to which the present disclosure belongs, unless first defined in the present disclosure.

[55] In the present disclosure, "video" may mean a set of images in sequence according to the passage of time. "Frame" generally means a basis representing one image in a particular time period, and a slice/tile is an encoding basis constituting a part of a frame in encoding. One frame may consist of one or more slices/tiles. In addition, a slice/tile may include one or more coding tree units (CTUs). One frame may consist of one or more slices/tiles. One frame may consist of one or more tile groups. One group of tiles may include one or more tiles. A brick can mean a quadrangular area of CTU-rows in a tile in a frame. One tile may include one or more bricks. Brick can mean a quadrangular area of CTU-rows in a tile. One tile may be segmented into multiple bricks, and each brick may include one or more CTU rows belonging to the tile. A tile that does not segment into multiple bricks can also be treated as a brick.

[56] In the present disclosure, "pixel" or "pel" can mean the smallest unit constituting one frame (or image). In addition, "sample" can be used as a term corresponding to a pixel. A sample can generally represent a pixel or a pixel value, and can only represent a luminance component pixel/pixel value, or only a chrominance component pixel/pixel value.

[57] In the present disclosure, a "unit" may represent a basic unit of image processing. The unit may include at least one of a specific area of the frame and information associated with the area. One unit may include one luminance block and two chrominance blocks (eg, Cb, Cr). The unit can be used interchangeably with terms such as "array of samples", "block" or "zone" in some cases. In general, an MxN block may include samples (or arrays of samples) or a set (or array) of transform coefficients of M columns and N rows.

[58] In the present disclosure, "current block" may mean one of "current coding block", "current coding unit", "target coding block", "target decoding block", or "target processing block". When prediction is performed, "current block" may mean "current prediction block" or "target prediction block". When transformation (inverse transformation)/quantization (dequantization) is performed, "current block" may mean "current transformation block (transformation current block)" or "transformation target block (transformation target block)". When filtering is performed, "current block" may mean "target filtering block".

[59] In addition, in the present disclosure, "current block" may mean "luminance signal block of the current block", if not explicitly specified as a block of chrominance signals. "Current block chroma block" may be expressed by including an explicit description of the chroma block, such as "chroma block" or "current chrominance block".

[60] In the present disclosure, the terms "/" and "," are to be interpreted as indicating "and/or". For example, the expression "A/B" and "A, B" can mean "A and/or B". Additionally, "A/B/C" and "A, B, C" may mean "at least one of A, B and/or C".

[61] In the present disclosure, the term "or" is to be interpreted as indicating "and/or". For example, the expression "A or B" may contain 1) only "A", 2) only "B", and/or 3) "both A and B". In other words, in the present disclosure, the term "or" is to be interpreted as indicating "in addition or alternatively".

[62] Overview of the Video Coding System

[63] FIG. 1 is a view showing a video coding system according to the present disclosure.

[64] A video coding system according to an embodiment may include a source device 10 and a receiver 20. The source device 10 may deliver the encoded information or video and/or image data to the receiver 20 in the form of a file or streaming through a digital storage medium. or network.

[65] The source device 10 according to the embodiment may include a video source generator 11, an encoding equipment 12, and a transmitter 13. The receiving device 20 according to the embodiment may include a receiver 21, a decoding equipment 12, and a rendering module 23. The encoding equipment 12 may be referred to as "video/image coding equipment", and the decoding equipment 12 may be referred to as "video/image decoding equipment". The transmitter 13 may be included in the coding equipment 12. The receiver 21 may be included in the equipment 12 decoding. The rendering module 23 may include a display, and the display may be configured as a separate device or an external component.

[66] The video source generator 11 can acquire a video/image through a process of capturing, synthesizing, or generating a video/image. The video source generator 11 may include a video/image capture device and/or a video/image generator. The video/image capture device may include, for example, one or more cameras, video/image archives including previously captured videos/images, and the like. The video/image generation apparatus may include, for example, computers, tablet computers, and smartphones, and may (electronically) generate video/images. For example, the virtual video/image may be generated via a computer or the like. In this case, the video/image capturing process may be replaced by a linked data generation process.

[67] The encoding equipment 12 may encode the input video/image. The encoding equipment 12 may perform a series of procedures such as prediction, transformation, and quantization for compression and encoding efficiency. The encoding equipment 12 can output encoded data (encoded video/image information) in the form of a bitstream.

[68] The transmitter 13 may transmit the encoded information or video/image data output in the form of a bit stream to the receiver 21 of the decoding receiver 20 via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. The transmitter 13 may include an element for generating a media file via a predetermined file format, and may include an element for transmission via a broadcast/communications network. The receiver 21 may extract/receive the bitstream from the storage medium or network and transmit the bitstream to the decoding equipment 12.

[69] The decoding equipment 12 can decode the video/image by performing a sequence of procedures such as dequantization, inverse transformation, and prediction corresponding to the operation of the encoding equipment 12.

[70] The rendering unit 23 may render the decoded video/image. The rendered video/image can be displayed through the display.

[71] Overview of Image Coding Equipment

[72] FIG. 2 is a view schematically showing image encoding equipment to which an embodiment of the present disclosure is applicable.

[73] As shown in FIG. 2, the image encoding equipment 100 may include an image segmentation unit 110, a subtractor 115, a transform 120, a quantizer 130, a dequantizer 140, an inverse transform 150, an adder 155, a filter 160, a memory 170, an inter prediction unit 180, an intra prediction unit 185. and an entropy encoder 190. Inter prediction module 180 and intra prediction module 185 may be collectively referred to as a "prediction module". Transformer 120, quantizer 130, dequantizer 140, and inverse transform 150 may be included in the residual processor. The residual processor may further include a subtractor 115.

[74] All or at least a portion of the plurality of components configuring the image encoding hardware 100 may be configured by a single hardware component (eg, encoder or processor) in some embodiments. In addition, the storage device 170 may include a decoded frame buffer (DPB) and may be configured by a digital storage medium.

[75] The image segmentation unit 110 may segment an input image (either a frame or a movie frame) input to the image encoding equipment 100 into more than one processing unit. For example, a processing unit may be referred to as a "coding unit (CU)". A coding unit may be obtained by recursively segmenting a coding tree unit (CTU) or a largest coding unit (LCU) according to a quadtree, binary tree, and ternary tree (QT/BT/TT) structure. For example, one coding unit may be segmented into multiple deeper coding units based on a quadtree structure, a binary tree structure, and/or a ternary structure. To segment a coding unit, a quadtree structure may be applied first, and subsequently a binary tree structure and/or a ternary structure may be applied. The encoding procedure according to the present disclosure may be performed based on a final encoding unit that is no longer segmented. The largest coding unit may be used as the final coding unit, or the larger depth coding unit obtained by segmenting the largest coding unit may be used as the final coding unit. Here, the encoding procedure may include a prediction, transformation, and restoration procedure, which is described below. As another example, the encoding procedure processing unit may be a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transformation unit may be split or segmented from the final coding unit. The prediction unit may be a selective prediction unit, and the transform unit may be a unit for extracting a transform coefficient and/or a unit for extracting a residual signal from a transform coefficient.

[76] The prediction unit (inter prediction unit 180 or intra prediction unit 185) can perform prediction on a block to be processed (current block) and generate a prediction block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied based on the current block or CU. The prediction module may generate various information related to the prediction of the current block, and pass the generated information to the entropy encoder 190. The prediction information may be encoded in the entropy encoder 190 and output in the form of a bit stream.

[77] The intra prediction module 185 may predict the current block by referring to the samples in the current frame. The reference samples may be located in the current block's surroundings, or may be spaced apart according to the intra prediction mode and/or intra prediction technology. The intra prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional mode may include, for example, a DC mode and a planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the granularity of the prediction direction. However, this is just an example, more or less directional prediction modes may be used depending on the setting. The intra prediction unit 185 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

[78] The inter-prediction unit 180 may derive a predicted block for the current block based on a reference block (reference sample array) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the adjacent block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0 prediction, L1 prediction, bi-prediction, etc.). In the case of inter prediction, an adjacent block may include a spatial adjacent block present in the current frame and a temporal adjacent block present in the reference frame. The reference frame including the reference block and the reference frame including the temporal neighbor block may be the same or different. The temporary neighbor block may be referred to as a "collocated reference block", "collocated CU (colCU)", or the like. A reference frame including a temporal adjacent block may be referred to as a "collocated frame (colPic)". For example, inter prediction module 180 may configure a list of motion information candidates based on adjacent blocks and generate information indicating which candidate is used to derive the motion vector and/or the reference frame index of the current block. Inter prediction may be performed based on various prediction modes. For example, in the case of the skip mode and the merge mode, the inter prediction unit 180 may use the neighboring block motion information as the current block motion information. In the case of the skip mode, unlike the combine mode, the residual signal may not be transmitted. In the case of a motion vector prediction (MVP) mode, the motion vector of an adjacent block may be used as a motion vector predictor, and the motion vector of the current block may be signaled by encoding a motion vector difference and an indicator for the motion vector predictor. The motion vector difference may mean the difference between the motion vector of the current block and the motion vector predictor.

[79] The prediction module may generate a prediction signal based on various prediction methods and prediction technologies described below. For example, the prediction module may not only apply intra prediction or inter prediction, but also apply both intra prediction and inter prediction at the same time to predict the current block. A prediction method that simultaneously applies both intra prediction and inter prediction to predict the current block may be referred to as "combined inter and intra prediction (CIIP)". In addition, the prediction module may perform intra-block copy (IBC) to predict the current block. In-block copying can be used for image/video encoding of game content and the like, such as on-screen content coding (SCC). IBC is a method for predicting the current frame using a previously reconstructed reference block in the current frame at a location separated from the current block by a predetermined distance. When IBC is applied, the location of the reference block in the current frame may be encoded as a vector (block vector) corresponding to a predetermined distance. IBC essentially performs prediction in the current frame, but may be performed similar to inter-prediction in that the reference block is retrieved within the current frame. Thus, the IBC may use at least one of the inter-prediction technologies described in the present disclosure.

[80] The predictive signal generated by the prediction module may be used to generate a recovered signal or generate a residual signal. The subtractor 115 may generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (predicted block or predictive sample array) output from the predictor from the input image signal (original block or original sample array). The generated residual signal may be transmitted to transducer 120.

[81] Converter 120 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technology may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loeve Transform (KLT), Graph Based Transform (GBT), or Conditionally Nonlinear Transform (CNT). ). Here, GBT means a transformation obtained from a graph when relationship information between pixels is represented by a graph. CNT means the transformation obtained based on the predictive signal generated using all previously reconstructed pixels. In addition, the transformation process may be applied to square pixel blocks having the same size, or may be applied to blocks having a variable size rather than square.

[82] The quantizer 130 may quantize the transform coefficients and pass them to the entropy encoder 190. The entropy encoder 190 may encode the quantized signal (information regarding the quantized transform coefficients) and output a bitstream. The information regarding the quantized transform coefficients may be referred to as "residual information". The quantizer 130 may repackage the quantized transform coefficients in block form into one-dimensional vector form based on the scan order of the coefficients, and generate information about the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form.

[83] The entropy encoder 190 may implement various coding methods such as exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like, for example. Entropy encoder 190 may encode information needed for video/image reconstruction other than quantized transform coefficients (eg, syntax element values, etc.) together or separately. Encoded information (eg, encoded video/image information) may be transmitted or stored in units of network abstraction layers (NALs) in the form of a bitstream. The video/image information may further include information regarding various parameter sets such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The signaling information, the signaling information and/or the syntax elements described in the present disclosure may be encoded via the encoding procedure described above and included in the bit stream.

[84] The bit stream may be transmitted over a network or may be stored on a digital storage medium. The network may include a broadcast network and/or a communications network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. A transmitter (not shown) transmitting a signal output from the entropy encoder 190 and/or a storage module (not shown) storing the signal may be included as an internal/external member of the encoding equipment 100, and alternatively, a transmitter may be included in the entropy encoder 190 .

[85] The quantized transform coefficients output from quantizer 130 may be used to generate a residual signal. For example, the residual signal (residual block or residual samples) may be recovered by applying dequantization and inverse transform to the quantized transform coefficients via dequantizer 140 and inverse transform 150.

[86] The adder 155 adds the recovered residual signal with the predictive signal output from the inter prediction unit 180 or the intra prediction unit 185 so as to generate a recovered signal (frame recovered, block recovered, recovered sample array). If there is no remainder for a block to be processed, such as in a case in which the skip mode is applied, the predicted block may be used as the reconstructed block. The adder 155 may be referred to as a "recovery module" or "recovered block generator". The generated recovered signal may be used to intra-predict the next block to be processed in the current frame, and may be used to inter-predict the next frame through filtering as described below.

[87] The filter 160 can improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 160 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame, and store the modified reconstructed frame in memory 170, namely the DPB of memory 170. Various filtering techniques may include, for example, filtering to deblock , filtering based on the sampled adaptive bias, adaptive loop filter, bilateral filter, and the like. The filter 160 may generate various filtering-related information and pass the generated information to the entropy encoder 190 as described below in the description of each filtering method. Filtering related information may be encoded by entropy encoder 190 and output in the form of a bitstream.

[88] The modified reconstructed frame transmitted to the memory 170 may be used as a reference frame in the inter prediction module 180. When inter prediction is applied via the image encoding equipment 100, prediction mismatch between the image encoding equipment 100 and the image decoding equipment can be eliminated, and encoding efficiency can be improved.

[89] The DPB of the memory 170 may store the modified reconstructed frame for use as a reference frame in the inter prediction module 180. The memory 170 may store block motion information from which motion information in the current frame is extracted (or encoded) and/or motion information of blocks in the frame that have already been recovered. The stored motion information may be transmitted to the inter-prediction unit 180 and used as spatial neighbor block motion information or temporal neighbor block motion information. The memory 170 may store the recovered samples of the recovered blocks in the current frame and may pass the recovered samples to the intra prediction module 185 .

[90] Overview of image decoding equipment

[91] FIG. 3 is a view schematically showing image decoding equipment to which an embodiment of the present disclosure is applicable.

[92] As shown in FIG. 3, the image decoding equipment 200 may include an entropy decoder 210, a dequantizer 220, an inverse transform 230, an adder 235, a filter 240, a memory 250, an inter prediction unit 260, and an intra prediction unit 265. The inter prediction module 260 and the intra prediction module 265 may be collectively referred to as a "prediction module". Dequantizer 220 and inverse converter 230 may be included in the residual processor.

[93] All or at least a portion of the plurality of components configuring the image decoding equipment 200 may be configured by a hardware component (eg, decoder or processor) according to the embodiment. In addition, the storage device 250 may include a decoded frame buffer (DPB) or may be configured by a digital storage medium.

[94] The image decoding equipment 200, which receives a bit stream including video/image information, can restore an image by performing a process corresponding to the process performed by the image encoding equipment 100 of FIG. 2. For example, the image decoding equipment 200 may perform decoding using the processing unit used in the image encoding equipment. Thus, the decoding processing unit, for example, may be an encoding unit. The coding unit may be obtained by segmenting the coding tree unit or the largest coding unit. The reconstructed image signal decoded and output by the decoding equipment 200 can be reproduced by the playback equipment.

[95] The image decoding equipment 200 may receive a signal output from the image encoding equipment of FIG. 2 in the form of a bitstream. The received signal may be decoded via entropy decoder 210. For example, entropy decoder 210 may parse the bit stream to extract information (eg, video/image information) needed for image recovery (or frame recovery). The video/image information may further include information regarding various parameter sets such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The image decoding equipment may further decode the frame based on the parameter set information and/or general constraint information. Signaling/receiving information and/or syntax elements described in the present disclosure may be decoded by a decoding procedure and obtained from a bitstream. For example, entropy decoder 210 decodes information in the bitstream based on an encoding method such as Exponential Golomb, CAVLC, or CABAC encoding, and output syntax element values required for image reconstruction, and quantized transform coefficient values for the remainder. More specifically, the CABAC entropy decoding method may receive a bin corresponding to each syntax element in the bit stream, determine a context model using decoding target syntax element information, neighboring block decoding information, and decoding target block, or symbol/bin information decoded at the previous stage, and perform arithmetic decoding for the permission element by predicting the occurrence probability of the permission element according to the determined context model, and generating a symbol corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method can update the context model by using the decoded symbol/bin information for the context model of the next symbol/bin after the context model is determined. Prediction related information from the information decoded by the entropy decoder 210 may be provided to a prediction unit (inter prediction unit 260 and intra prediction unit 265) and a residual value for which entropy decoding is performed in the entropy decoder 210, i. quantized transform coefficients and related parameter information may be input to the dequantizer 220. In addition, information regarding filtering from the information decoded by the entropy decoder 210 may be provided to the filter 240. Meanwhile, a receiver (not shown) for receiving a signal output from the equipment image encoding equipment may be further configured as an internal/external element of the image decoding equipment 200, or the receiver may be a component of the entropy decoder 210.

[96] Meanwhile, the image decoding equipment according to the present disclosure may be referred to as "video/image/frame decoding equipment". The image decoding equipment can be classified into an information decoder (video/image/frame information decoder) and a sampling decoder (video/image/frame samples decoder). The information decoder may include an entropy decoder 210. The selective decoder may include at least one of a dequantizer 220, an inverse transform 230, an adder 235, a filter 240, a memory 250, an inter prediction module 260, or an intra prediction module 265.

[97] The dequantizer 220 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 may recompose the quantized transform coefficients in the form of a two-dimensional block. In this case, the re-arranging may be performed based on the scan order of the coefficients performed in the image encoding equipment. The dequantizer 220 may perform dequantization on the quantized transform coefficients by using a quantization parameter (eg, quantization step size information) and obtain the transform coefficients.

[98] The inverse transform 230 may inversely transform the transform coefficients to obtain a residual signal (residual block, array of residual samples).

[99] The prediction module may perform prediction for the current block and may generate a predicted block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoder 210, and may determine a specific intra/inter prediction mode.

[100] Identical to that described in the prediction module of the image encoding equipment 100, the prediction module can generate a prediction signal based on various prediction methods (technologies) which are described below.

[101] The intra prediction module 265 may predict the current block by referring to the samples in the current frame. The description of the intra prediction module 185 applies equally to the intra prediction module 265 .

[102] The inter prediction unit 260 may derive a predicted block for the current block based on a reference block (reference sample array) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the adjacent block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0 prediction, L1 prediction, bi-prediction, etc.). In the case of inter prediction, an adjacent block may include a spatial adjacent block present in the current frame and a temporal adjacent block present in the reference frame. For example, inter-prediction unit 260 may configure a motion information candidate list based on neighboring blocks, and derive the current block's motion vector and/or reference frame index based on the received candidate selection information. Inter prediction may be performed based on various prediction modes, and the prediction information may include information indicative of the inter prediction mode for the current block.

[103] The adder 235 may generate a reconstructed signal (recovered frame, reconstructed block, reconstructed sample array) by summing the obtained residual signal with the predictive signal (predicted block, predicted sample array) output from the prediction module (including the inter prediction module 260 and /or internal prediction module 265). If there is no remainder for a block to be processed, such as in a case in which the skip mode is applied, the predicted block may be used as the reconstructed block. The description of adder 155 is equally applicable to adder 235. Adder 235 may be referred to as a "recovery module" or "recovered block generator". The generated recovered signal may be used to intra-predict the next block to be processed in the current frame, and may be used to inter-predict the next frame through filtering as described below.

[104] The filter 240 can improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 240 may generate a modified reconstructed frame by applying various filtering methods to the reconstructed frame and store the modified reconstructed frame in memory 250, namely the DPB of storage 250. Various filtering methods may include, for example, filtering to deblock , filtering based on a sampled adaptive bias, an adaptive loop filter, a bilateral filter, and the like.

[105] The (modified) reconstructed frame stored in the DPB of the memory 250 may be used as a reference frame in the inter prediction unit 260 . The memory 250 may store block motion information from which motion information in the current frame is extracted (or decoded) and/or motion information of blocks in the frame that have already been recovered. The stored motion information may be transmitted to the inter-prediction unit 260 so that it is used as spatial neighbor block motion information or temporal neighbor block motion information. The memory 250 may store the recovered samples of the recovered blocks in the current frame and pass the recovered samples to the intra prediction module 265 .

[106] In the present disclosure, the embodiments described in the filter 160, the inter prediction unit 180, and the intra prediction unit 185 of the image coding equipment 100 can be equally or appropriately applied to the filter 240, the inter prediction unit 260, and the intra prediction unit 265 image decoding equipment 200.

[107] General representation of image segmentation

[108] The video/image coding method according to the present disclosure may be based on an image segmentation structure as follows. In particular, the prediction, residual processing ((inverse) transform, (de)-quantization, etc.), syntax element encoding, and filtering procedures described below may be performed on a CTU, CU (and/or TU, PU) basis. ) extracted based on the image segmentation structure. An image may be segmented in units of blocks, and a block segmentation procedure may be performed in the image segmentation unit 110 of the encoding equipment. The segmentation related information may be encoded by the entropy encoder 190 and transmitted to the image decoding equipment in the form of a bitstream. The entropy decoder 210 of the image decoding equipment can extract the segmentation structure into blocks of the current frame based on the segmentation-related information obtained from the bit stream, and based on this, can perform a series of procedures (e.g., prediction, residual processing, block/frame recovery, in-loop filtering). etc.) for image decoding.

[109] Frames may be segmented into a sequence of coding tree units (CTUs). Fig. 4 shows an example in which a frame is segmented into CTUs. The CTU may correspond to a coding tree block (CTB). Alternatively, the CTU may include a luminance sample coding tree block and two corresponding chrominance samples coding tree blocks. For example, for a frame that contains three arrays of samples, the CTU may include an NxN block of luma samples and two corresponding blocks of chrominance samples.

[110] General representation of CTU segmentation

[111] As described above, a coding unit can be obtained by recursively segmenting a coding tree unit (CTU) or a largest coding unit (LCU) according to a quadtree/binary tree/ternary tree (QT/BT/TT) structure. For example, the CTU may first be segmented into quadtree structures. Thereafter, the leaf nodes of the quadtree structure may further be segmented by a multi-type tree structure.

[112] Quadtree segmentation means that the current CU (or CTU) is segmented equally into four. By segmenting according to the quadtree, the current CU can be segmented into four CUs having the same width and the same height. When the current CU is no longer segmented into a quadtree structure, the current CU corresponds to a leaf node of the quadtree structure. The CU corresponding to the leaf node of the quadtree structure can no longer be segmented and can be used as the final coding unit described above. Alternatively, the CU corresponding to the leaf node of the quadtree structure may further be segmented by a multi-type tree structure.

[113] FIG. 5 is a view showing the block segmentation type according to the multi-type tree structure. The partitioning according to the multi-type tree structure may include two types of partitioning according to the binary tree structure and two types of partitioning according to the ternary tree structure.

[114] The two types of splitting according to the binary tree structure may include vertical binary splitting (SPLIT_BT_VER) and horizontal binary splitting (SPLIT_BT_HOR). Vertical binary splitting (SPLIT_BT_VER) means that the current CU is split equally in half in the vertical direction. As shown in FIG. 4, by vertical binary splitting, two CUs having a height identical to the height of the current CU and having a width that is half the width of the current CU can be formed. Horizontal binary splitting (SPLIT_BT_HOR) means that the current CU is split equally in half in the horizontal direction. As shown in FIG. 5, by horizontal binary splitting, two CUs having a height that is half the height of the current CU and having a width identical to the width of the current CU can be formed.

[115] The two types of splitting according to the ternary tree structure may include vertical ternary splitting (SPLIT_TT_VER) and horizontal ternary splitting (SPLIT_TT_HOR). In vertical ternary splitting (SPLIT_TT_VER), the current CU is split in the vertical direction at a ratio of 1:2:1. As shown in FIG. 5, by vertical ternary partitioning, two CUs having a height identical to the height of the current CU and having a width that is 1/4 of the width of the current CU, and a CU having a height identical to the height of the current CU and having a width that is is half the width of the current CU. In horizontal ternary splitting (SPLIT_TT_HOR), the current CU is split horizontally in a ratio of 1:2:1. As shown in FIG. 4, by horizontal ternary partitioning, two CUs having a height that is 1/4 of the height of the current CU and having a width identical to the width of the current CU, and a CU having a height that is half the height of the current CU and having a width identical to the width of the current CU.

[116] FIG. 6 is a view showing a signaling mechanism of blocking information in a nested multi-type tree quadtree structure according to the present disclosure.

[117] Here, the CTU is treated as the root node of the quadtree and is segmented for the first time into a quadtree structure. Information (eg, qt_split_flag) indicating whether or not quadtree splitting is performed with respect to the current CU (CTU or quadtree node (QT_node)), is signaled. For example, when qt_split_flag has a first value (eg, "1"), the current CU may be segmented into a quadtree. In addition, when qt_split_flag has a second value (eg, "0"), the current CU is not segmented into a quadtree, but becomes a leaf node (QT_leaf_node) of the quadtree. Each leaf node of the quadtree can then be further segmented into multi-type tree structures. Thus, a leaf node of a quadtree can become a node (MTT_node) of a multitype tree. In the multi-type tree structure, the first flag (eg, Mtt_split_cu_flag) is signaled to indicate whether or not the current node is further segmented. If the corresponding node is further segmented (eg, if the first flag is 1), a second flag (eg, Mtt_split_cu_vertical_flag) may be signaled to indicate the split direction. For example, the split direction may be the vertical direction if the second flag is 1, and may be the horizontal direction if the second flag is 0. Then, a third flag (eg, mtt_split_cu_binary_flag) may be signaled to indicate whether the split type is is a binary split type or a ternary split type. For example, the split type may be a binary split type when the third flag is 1, and may be a ternary split type when the third flag is 0. A node of a multitype tree obtained by binary split or ternary split may further be segmented into multitype tree structures. However, a multitype tree node may not be segmented into quadtree structures. If the first flag is 0, the corresponding multitype tree node is no longer broken, but becomes a leaf node (MTT_leaf_node) of the multitype tree. The CU corresponding to the leaf node of the multi-type tree may be used as the final coding unit described above.

[118] Based on mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-type tree split mode (MttSplitMode) of the CU may be derived as shown in Table 1 below. In the description below, the multi-tree split mode may be referred to as "multi-tree split type" or "split type " to be short.

[119] Tab. 1

MttSplitMode mtt_split_cu_vertical_flag mtt_split_cu_binary_flag SPLIT_TT_HOR 0 0 SPLIT_BT_HOR 0 1 SPLIT_TT_VER 1 0 SPLIT_BT_VER 1 1

[120] FIG. 7 is a view showing an example in which a CTU is segmented into multiple CUs by applying a multi-type tree after applying the quadtree. In FIG. 7, the bold block edges 710 represent quadtree based segmentation and the remaining edges 720 represent multitype tree based segmentation. A CU may correspond to a coding block (CB). In an embodiment, the CU may include a luminance sample coding block and two chrominance sample coding blocks corresponding to the luminance samples. The CB or TB size of the chrominance component (samples) can be derived based on the CB or TB size of the luminance component (samples) according to the ratio of the components according to the color format (chrominance signal format, e.g. 4:4:4, 4:2 :2, 4:2:0, etc.) frame/picture. In the case of a 4:4:4 color format, the CB/TB size of the chrominance components may be set to the CB/TB size of the luminance components. In the case of the 4:2:2 color format, the width of the CB/TB chrominance components may be set to half the width of the CB/TB luminance components, and the height of the CB/TB chrominance components may be set to the height of the CB/TB luminance components. In the case of the 4:2:0 color format, the width of the CB/TB chrominance components may be set to half the width of the CB/TB luminance components, and the height of the CB/TB chrominance components may be set to half the height of the CB/TB luminance components.

[121] In an embodiment, when the CTU size is 128 based on the luminance sample unit, the CU size may be 128x128 to 4x4, which is the same size as the CTU. In one embodiment, in the case of a 4:2:0 color format (or chrominance format), the CB-size of the chrominance signals may be 64x64 to 2x2.

[122] Meanwhile, in the embodiment, the CU size and the TU size may be identical. Alternatively, a plurality of TUs may be provided in a CU area. TU-size, in General, represents the size of the transform block (TB) component (samples) of the luminance signals.

[123] The TU size may be derived based on the largest allowable TB size maxTbSize, which is a predetermined value. For example, when a CU size exceeds maxTbSize, a plurality of TUs (TBs) having maxTbSizes can be extracted from the CUs, and mapping/demapping can be performed in units of TUs (TBs). For example, the largest allowable TB size of luminance signals may be 64x64, and the largest allowable TB size of chrominance signals may be 32x32. If the width or height of a CB segmented according to the tree structure is greater than the largest transform width or height, the CB may be automatically (or implicitly) segmented until the TB-size limit in the horizontal and vertical directions is satisfied.

[124] In addition, for example, when intra prediction is applied, the mode/type of intra prediction may be extracted in units of CU (or CB), and the procedure for extracting adjacent reference samples and generating predictive samples may be performed in units of TU (or TB). In this case, one or multiple TUs (or TBs) may be provided in one CU- (or CB-) area, and in this case, multiple TUs or (TBs) may share the same intra prediction mode/type.

[125] Meanwhile, for the nested multi-type tree quadtree-based coding tree scheme, the following parameters may be signaled as SPS syntax elements from the image encoding equipment to the decoding equipment. For example, at least one of CTU-size, which is a parameter representing the size of the quadtree root nodes, MinQTSize, which is a parameter representing the minimum allowed size of quadtree leaf nodes, MaxBtSize, which is a parameter representing the maximum the allowed size of the root nodes of the binary tree, MaxTtSize, which is a parameter representing the maximum allowed size of the root nodes of the ternary tree, MaxMttDepth, which is the parameter representing the maximum allowed depth of the splitting hierarchy based on the multitype tree from the leaf node of the quadtree, MinBtSize, which is a parameter representing the minimum allowed size of leaf nodes of a binary tree, or MinTtSize, which is a parameter representing the minimum allowed size of leaf nodes of a ternary tree, is signaled.

[126] As an embodiment of using the 4:2:0 chroma format, the CTU size may be set to 128x128 luma blocks and two 64x64 chrominance blocks corresponding to the luma blocks. In this case, MinOTSize may be set to 16x16, MaxBtSize may be set to 128x128, MaxTtSzie may be set to 64x64, MinBtSize and MinTtSize may be set to 4x4, and MaxMttDepth may be set to 4. Quadtree based segmentation can be applied to CTUs to form leaf nodes of a quadtree. A leaf node of a quadtree may be referred to as a "QT leaf node". The leaf nodes of the quadtree can range in size from 16×16 (eg, MinOTSize) to 128×128 (eg, CTU-size). If the QT-node-leaf is 128x128, it may additionally not be segmented into a binary tree/ternary tree. This is due to the fact that in this case, even with segmentation, it exceeds MaxBtsize and MaxTtszie (for example, 64×64). In other cases, QT leaf nodes can additionally be segmented into a multi-type tree. Therefore, a QT leaf node is the root node for a multitype tree, and a QT leaf node can have a value of 0 for the depth of the multitype tree (mttDepth). If the depth of the multitype tree reaches MaxMttdepth (eg 4), additional segmentation may no longer be considered. If the width of a multitype tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, then additional horizontal segmentation may not be considered. If the height of a multitype tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, additional vertical segmentation may not be considered. When segmentation is not considered, the image encoding equipment may skip signaling segmentation information. In this case, the image decoding equipment can extract segmentation information with a predetermined value.

[127] Meanwhile, one CTU may include a luminance sample coding unit (hereinafter referred to as a "luminance signal unit") and two chrominance sample coding units corresponding to it (hereinafter referred to as "chroma units"). The above-described coding tree-based scheme may be applied equally or separately to the luma block and the chrominance block of the current CU. In particular, luminance and chrominance blocks in one CTU may be segmented into an identical block tree structure, in which case, the tree structure is represented as SINGLE_TREE. Alternatively, the luma and chrominance blocks in one CTU may be segmented into separate block tree structures, in which case, the tree structure may be represented as DUAL_TREE. Thus, when the CTU is segmented into dual trees, a block tree structure for the luma block and a block tree structure for the chrominance block may be separately present. In this case, the block tree structure for the luma block may be called "DUAL_TREE_LUMA", and the block tree structure for the chrominance signal component may be called "DUAL_TREE_CHROMA". For groups of P- and B-slices/tiles, the luminance and chrominance blocks in one CTU may be restricted to have the same coding tree structure. However, for groups of I-slices/tiles, the luminance and chrominance blocks may have a separate block tree structure relative to each other. If a separate block tree structure is applied, the luminance CTB may be segmented into CUs based on a particular coding tree structure, and the chroma CTB may be segmented into chrominance CUs based on another coding tree structure. Thus, this means that a CU in an I-slice/tile group to which a separate block tree structure is applied may include a luma component coding block or two chroma component coding blocks, and a CU of a P- or B- group. slices/tiles may include blocks of three color components (luminance component and two chrominance components).

[128] Although a quadtree-based coding tree structure with a nested multi-type tree is described, the structure in which the CU is segmented is not limited to this. For example, a BT framework and a TT framework can be interpreted as a concept included in a multi-segmentation tree (MPT) framework, and a CU can be interpreted as being segmented through the QT framework and the MPT framework. In an example in which a CU is segmented through a QT framework and an MPT framework, a syntax element (for example, MPT_split_type) including information on how many blocks the leaf node of the QT framework is segmented into, and a syntax element (for example, MPT_split_mode) , including information as to which of the vertical and horizontal directions the leaf-node of the QT structure is segmented, may be signaled to determine the segmentation structure.

[129] In another example, the CU may be segmented in a manner different from the QT framework, BT framework, or TT framework. Thus, unlike a lesser CU is segmented 1/4 of a greater depth CU according to the QT structure, a lesser CU is segmented 1/2 of a greater depth CU according to the BT structure, or a lesser CU is segmented by 1 /4 or 1/2 of the deeper CU according to the TT structure, the shallower CU may be segmented into 1/5, 1/3, 3/8, 3/5, 2/3 or 5/8 of the deeper CU in some cases, and the CU segmentation method is not limited to this.

[130] A multi-type tree quadtree coding block structure can provide a very flexible block segmentation structure. Due to the segmentation types supported in a multi-type tree, different segmentation patterns can potentially result in an identical coding block structure in some cases. In the encoding equipment and the image decoding equipment, by limiting the occurrence of such redundant segmentation patterns, the amount of segmentation information data can be reduced.

[131] In addition, when encoding and decoding video/image according to the present document, the image processing basis may have a hierarchical structure. One frame may be divided into one or more tiles, bricks, slices and/or groups of tiles. One slice may include one or more bricks. One brick may include one or more CTU rows in a tile. The slice may include frame bricks, where the number of bricks is an integer. One group of tiles may include one or more tiles. One tile may include one or more CTUs. A CTU may be segmented into one or more CUs. A tile may be a quadrilateral area made up of particular rows of tiles and particular columns of tiles made up of multiple CTUs in a frame. The tile group may include tiles according to a raster scan of the tiles in the frame, wherein the number of tiles is an integer. The slice header may carry information/parameter applicable to the corresponding slice(s) in the slice. When the encoding equipment or the decoding equipment has a multi-core processor, the encoding/decoding procedures for a tile, slice, brick, and/or group of tiles may be performed in parallel.

[132] In the present disclosure, the entity, name, or concept of a slice or group of tiles may be used interchangeably. Thus, a tile group header may be referred to as a "slice header". Herein, a slice may have one of slice types including an inner (I) slice, a predictive (P) slice, and a bi-predictive (B) slice. For blocks in an I-slice, inter prediction is not used for prediction, and only intra prediction can be used. Even so, the original sample value may be encoded and signaled without prediction. For blocks in a P-slice, intra prediction or inter prediction may be used. When cross prediction is used, only uni-prediction can be used. Meanwhile, for blocks in a B-slice, intra prediction or inter prediction may be used. When cross prediction is used, bi-prediction, which is the maximum power, may be used.

[133] According to the characteristic (eg, resolution) of the video image, or considering the efficiency of coding or parallel processing, the encoding equipment can determine the dimensions of the tile/group of tiles, brick, slice, and the largest and smallest coding unit. In addition, information regarding the signified or information for retrieving the signified may be included in the bitstream.

[134] The decoding equipment may obtain information indicating whether or not a CTU in a tile/tile group, brick, slice, or tile of the current frame is segmented into multiple coding units. The encoding equipment and the decoding equipment signal this information only under a specific condition, thereby improving encoding efficiency.

[135] The slice header (slice header syntax) may include information/parameter applicable to the slice collectively. APS (APS Syntax) or PPS (PPS Syntax) may include information/parameter applicable to one or more frames in common. SPS (SPS syntax) may include information/parameter applicable to one or more sequences in common. VPS (VPS syntax) may include information/parameter applicable to multiple layers in common. DPS (DPS syntax) may include information/parameter applicable to the entire video together. The DPS may include information/parameter associated with a combination of coded video sequences (CVS).

[136] In addition, for example, information regarding segmentation and configuration of a tile/group of tiles/brick/cut may be constructed in the coding stage via a high-level syntax and transmitted as a bitstream to the decoding equipment.

[137] Intra prediction overview

[138] Hereinafter, this document describes in detail the intra prediction performed by the encoding equipment and the decoding equipment described above. Intra prediction may mean prediction that generates prediction samples for the current block based on reference samples in the frame (hereinafter, current frame) to which the current block belongs.

[139] This is described next with reference to FIG. 8. When intra prediction is applied to the current block 801, adjacent reference samples to be used for intra prediction of the current block 801 may be extracted. Adjacent reference samples of the current block may include: a total of 2xnH samples including samples 811 adjacent to the left edge of the current block having size nWxnH and samples 812 bordering the bottom left; a total of 2xnW samples including samples 821 adjacent to the top of the current block and samples 822 adjacent to the top right; and one sample 831 bordering the top left of the current block. Alternatively, the adjacent reference samples of the current block may include a plurality of columns of top neighbor samples and a plurality of rows of left neighbor samples.

[140] In addition, adjacent reference samples of the current block may include: a sum of nH samples 841 adjacent to the right boundary of the current block having a size of nWxnH; a total of nW samples 851 adjacent to the lower boundary of the current block; and one sample 842 bordering the bottom right of the current block.

[141] However, some adjacent reference samples of the current block have not yet been decoded or may not be available. In this case, the decoding equipment can construct adjacent reference samples to be used for prediction by substituting the available samples for the unavailable samples. Alternatively, adjacent reference samples to be used for prediction may be constructed through interpolation of the available samples.

[142] When adjacent reference samples are extracted, (i) a predictive sample may be derived based on an average or an interpolation of adjacent reference samples of the current block, or (ii) a predictive sample may be derived based on a reference sample present in a particular direction (prediction) relative to the prediction sample from neighboring reference samples of the current block. Case (i) may be referred to as "non-directional mode" or "non-angular mode", and case (ii) may be referred to as "directional mode" or "angular mode". In addition, the prediction sample may be generated through interpolation with the second neighbor sample and the first neighbor sample, which are located in the opposite direction with respect to the prediction direction of the intra prediction mode of the current block, based on the prediction sample of the current block, from the neighbor reference samples. The above case may be referred to as "linear interpolation intra prediction (LIP)". In addition, the predictive chrominance samples may be generated based on the luminance samples by using a linear model. This case may be referred to as "LM mode". In addition, the temporal prediction sample of the current block may be derived based on the filtered neighboring reference samples, and the prediction sample of the current block may be derived by weighting the temporal prediction sample and at least one reference sample derived according to the intra prediction mode from the existing neighboring reference samples. samples, namely the unfiltered neighboring reference samples. The above case may be referred to as "position dependent intra prediction (PDPC)". In addition, a reference sample line having the highest prediction accuracy may be selected from a plurality of adjacent reference sample lines of the current block to extract a prediction sample by using a reference sample located in the prediction direction in the corresponding line. At this time, intra predictive coding may be performed by indicating (signaling) the reference sample line in use to the decoding equipment. The above case may be referred to as "multiple reference line (MRL) intra prediction" or "MRL intra prediction". In addition, the current block may be divided into vertical or horizontal subsegments to perform intra prediction based on the same intra prediction mode, and adjacent reference samples may be extracted based on the subsegment and used. Thus, in this case, the intra prediction mode for the current block is equally applied to sub-segments, and an adjacent reference sample is extracted based on the sub-segment and used, thereby improving intra prediction performance in some cases. This prediction method may be referred to as "intra-sub-segment prediction (ISP) or ISP-based". These intra prediction methods may be referred to as "types of intra prediction" as opposed to intra prediction modes (eg, DC mode, planar mode, and directional mode). Types of intra prediction may be referred to by various terms such as "intra prediction schemes" or "additional intra prediction modes". For example, intra prediction types (or additional intra prediction modes) may include at least one selected from the group of LIP, PDPC, MRL, and ISP as described above. The general intra prediction method, except for specific types of intra prediction such as LIP, PDPC, MRL, and ISP, may be referred to as "normal intra prediction type". The normal intra prediction type may mean a case in which specific types of intra prediction are not applied, and prediction can be performed based on the above-described intra prediction modes. Meanwhile, if necessary, post-filtering may be performed on the extracted predictive sample.

[143] Specifically, the intra prediction procedure may include an intra prediction mode/type determination step, an adjacent reference sample extraction step, and a prediction sample extraction step based on the intra prediction mode/type. In addition, if necessary, a post-filtering step may be performed on the extracted predictive sample.

[144] Meanwhile, in addition to the above-described types of intra prediction, affine linear weighted intra prediction (ALWIP) can be used. ALWIP may be referred to as "linear weighted intra prediction (LWIP)" or "matrix weighted intra prediction" or "matrix intra prediction (MIP)". When MIP is applied to the current block, the prediction samples for the current block may be derived by i) using neighboring reference samples subjected to an average procedure, ii) performing a matrix-vector multiplication procedure, and iii) additionally performing a horizontal/vertical interpolation procedure, if necessary. The intra prediction modes used for MIP may be different from the intra prediction modes used in intra LIP, PDPC, MRL, ISP or normal intra prediction. The intra prediction modes for MIP may be referred to as "intra MIP prediction modes", "MIP prediction modes", or "MIP modes". For example, various matrices and biases used in matrix-vector multiplication may be defined according to intra prediction modes for MIP. Herein, the matrix may be referred to as a "weighted (MIP)-matrix", and the offset may be referred to as a "(MIP)-shift vector" or "(MIP)-shift vector". The detailed MIP method is described below.

[145] The procedure for recovering blocks based on intra prediction and the intra prediction unit in the coding equipment may schematically include, for example, the following described below. Step S910 may be performed by the coding equipment intra prediction unit 185 . Step S920 may be performed by a residual processor that includes at least one selected from a group of subtractor 115, transform 120, quantizer 130, dequantizer 140, and inverse transform 150 of the encoding equipment. In particular, step S920 may be performed by the subtractor 115 of the encoding equipment. In step S930, prediction information may be extracted by intra prediction unit 185 and may be encoded by entropy encoder 190. In step S930, residual information may be extracted by residual processor and may be encoded by entropy encoder 190. Residual information is information regarding residual samples. The residual information may include information regarding quantized transform coefficients for the residual samples. As described above, the residual samples may be extracted as transform coefficients via the coding equipment transform 120, and the transform coefficients may be extracted as quantized transform coefficients via the quantizer 130. Information regarding the quantized transform coefficients may be encoded by the entropy encoder 190 via a residual encoding procedure.

[146] The encoding equipment may perform intra prediction for the current block in step S910. The encoding equipment extracts the intra prediction mode/type for the current block, extracts the adjacent reference samples of the current block, and generates prediction samples in the current block based on the intra prediction mode/type and the adjacent reference samples. In this document, the procedures for determining the mode/type of intra prediction, extracting adjacent reference samples, and generating predictive samples may be performed simultaneously, or any procedure may be performed before other procedures. For example, although not shown, coding equipment intra prediction module 185 may include an intra prediction mode/type determination module, a reference sample extraction module, and a prediction sample extraction module. The intra prediction mode/type determination module may determine the intra prediction mode/type for the current block, the reference sample extractor may extract adjacent reference samples of the current block, and the prediction sample extractor may extract the current block's prediction samples. Meanwhile, when the prediction sample filtering procedure to be described below is executed, the intra prediction unit 185 may further include a prediction sample filter. The encoding equipment may determine the mode/type applied to the current block from a plurality of intra prediction modes/types. The encoding equipment may compare the RD costs for intra prediction modes/types and determine the optimal intra prediction mode/type for the current block.

[147] Meanwhile, the encoding equipment may perform a filtering procedure on the predictive samples. Filtering predictive samples may be referred to as "post-filtering". Through the prediction samples filtering procedure, some or all of the prediction samples may be filtered. In some cases, the filtering procedure for predictive samples may be omitted.

[148] The encoding equipment may generate residual samples for the current block based on the (filtered) predictive samples in step S920. The encoding equipment may compare the predictive samples in the original samples of the current block on a phase basis and may extract the residual samples.

[149] The encoding equipment may encode image information including information (prediction information) regarding intra prediction and residual information regarding residual samples in step S930. The prediction information may include intra prediction mode information and intra prediction type information. The encoding equipment may output the encoded image information in the form of a bitstream. The output bit stream may be transmitted to the decoding equipment via a storage medium or a network.

[150] The residual information may include a residual encoding syntax, which is described below. The encoding equipment may extract the quantized transform coefficients by transforming/quantizing the residual samples. The residual information may include information regarding the quantized transform coefficients.

[151] Meanwhile, as described above, the coding equipment may generate a reconstructed frame (including reconstructed samples and a reconstructed block). To this end, the encoding equipment may perform dequantization/inverse transform on the quantized transform coefficients and extract the (modified) residual samples. The reason for performing the dequantization/inverse transformation after the transformation/quantization of the residual samples is to extract residual samples that are identical to the residual samples extracted by the decoding equipment as described above. The encoding equipment may generate a reconstructed block including reconstructed samples for the current block based on the predictive samples and the (modified) residual samples. Based on the reconstructed block, a reconstructed frame for the current frame may be generated. As described above, the in-loop filtering procedure may additionally be applied to the reconstructed frame.

[152] A procedure for decoding video/images based on intra prediction and an intra prediction unit in the decoding equipment may schematically include, for example, the following described below. The decoding equipment may perform an operation corresponding to the operation performed by the encoding equipment.

[153] Steps S1010-S1030 may be performed by the intra prediction unit 265 of the decoding equipment. The prediction information in step S1010 and the residual information in step S1040 may be obtained from the bitstream by the entropy decoder 210 of the decoding equipment. A residual processor including dequantizer 220 or decoding equipment inverse converter 230, or both, may derive residual samples for the current block based on the residual information. In particular, the residual processor dequantizer 220 may perform dequantization based on the quantized transform coefficients derived based on the residual information, and may extract the transform coefficients. Residual processor detransformer 230 may perform an inverse transform on the transform coefficients and may extract residual samples for the current block. Step S1050 may be performed by the adder 235 or the decoding equipment recovery module.

[154] In particular, the decoding equipment may derive the intra prediction mode/type for the current block based on the received prediction information (intra prediction mode/type information) in step S1010. The decoding equipment may extract adjacent reference samples of the current block in step S1020. The decoding equipment may generate prediction samples in the current block based on the intra prediction mode/type and adjacent reference samples in step S1030. In this case, the image decoding equipment may perform a prediction sample filtering procedure. Filtering predictive samples may be referred to as "post-filtering". Through the prediction samples filtering procedure, some or all of the prediction samples may be filtered. In some cases, the filtering procedure for predictive samples may be omitted.

[155] The image decoding equipment may generate residual samples for the current block based on the received residual information. The decoding equipment may generate reconstructed samples for the current block based on the predictive samples and the residual samples, and may extract a reconstructed block including the reconstructed samples in step S1040. Based on the reconstructed block, a reconstructed frame for the current frame may be generated. As described above, the in-loop filtering procedure may additionally be applied to the reconstructed frame.

[156] Here, although not shown, the intra prediction module 265 of the decoding equipment may include an intra prediction mode/type determination module, a reference sample extraction module, and a prediction sample extraction module. The intra prediction mode/type determination module may determine the intra prediction mode/type for the current block based on the intra prediction mode/type information obtained from the entropy decoder 210. The reference sample extractor may extract adjacent reference samples of the current block. The prediction sample extractor may extract the prediction samples of the current block. Meanwhile, when the above-described prediction sample filtering procedure is performed, the intra prediction unit 265 may further include a prediction sample filter.

[157] The intra prediction mode information may include, for example, flag information (eg, intra_luma_mpm_flag) indicating whether the most likely mode (MPM) is applied to the current block or the remaining mode is applied. When MPM is applied to the current block, the prediction mode information may further include index information (eg, intra_luma_mpm_idx) indicating one of the possible intra prediction modes (MPM candidates). Intra prediction mode candidates (MPM candidates) may be constructed from a list of MPM candidates or an MPM list. In addition, when MPM is not applied to the current block, the intra prediction mode information may further include remaining mode information (e.g., intra_luma_mpm_remainder) indicating one of the remaining intra prediction modes except for intra prediction mode options (MPM options) . The decoding equipment may determine the intra prediction mode of the current block based on the intra prediction mode information. A separate MPM list may be constructed for the above described MIP.

[158] In addition, the intra-prediction type information may be implemented in various forms. For example, the intra prediction type information may include intra prediction type index information indicating one of the intra prediction types. As another example, the intra prediction type information may include at least one of reference sample line information (e.g., intra_luma_ref_idx) indicating whether or not MRL is applied to the current block, and which reference sample line is used, when the MRL is applied to the current block, ISP flag information (eg, intra_subpartitions_mode_flag) indicating whether or not ISP is applied to the current block, ISP type information (eg, intra_subpartitions_split_flag) indicating the type of sub-segment segmentation, when ISP is applied, flag information, indicating whether or not PDCP is applied, or flag information indicating whether or not LIP is applied. In addition, the intra prediction type information may include a MIP flag indicating whether or not MIP is applied to the current block.

[159] The intra prediction mode information and/or the intra prediction type information may be encoded/decoded via the coding method described herein. For example, the intra prediction mode information and/or the intra prediction type information may be coded/decoded by entropy coding (eg, CABAC, CAVLC) based on a truncated binary code (Rice).

[160] Adaptive Color Transformation (ACT) General Representation

[161] Adaptive Color Transformation (ACT) is a color space conversion (conversion) technology for removing unnecessary overlap between color components and is used in the HEVC extension version for screen content. It can also apply to VVC.

[162] In the HEVC extension for screen content (HEVC SCC extension), ACT is used to adaptively convert a prediction residual from an existing color space to a YCgCo color space. One of the two color spaces may optionally be selected by signaling one ACT flag for each transform basis.

[163] For example, the first value (eg, 1) of the flag may indicate that the rest of the transformation basis is encoded in the original color space. The second value (eg, 1) of the flag may indicate that the remainder of the transformation basis is encoded in YCgCo color space.

[164] FIG. 11 is a view showing an embodiment of a decoding process to which ACT is applied. In the embodiment of FIG. 11, motion-compensated prediction may correspond to inter-prediction in the present disclosure.

[165] As shown in FIG. 11, a reconstructed frame (or reconstructed block, reconstructed sample array, reconstructed sample(s), reconstructed signal) may be generated based on the predicted output value and the residual output value. Herein, the residual output value may be the output value of the inverse transform. In this document, the inverse transformation may be a normal inverse transformation. In this document, the normal inverse transform may be an MTS-based inverse transform or an inverse low frequency inseparable transform (LFNST).

[166] As used herein, the predictive output may be a prediction block, an array of prediction samples, predictive sample(s), or a predictive signal. The residual output value may be a residual block, an array of residual samples, a residual sample(s), or a residual signal.

[167] For example, from an encoding hardware point of view, an ACT process may be performed on the residual samples derived from the predictive samples. In addition, the output value of the ACT process may be provided as an input to the normal transformation process. In this document, the normal conversion process may be an MTS or LFNST based conversion.

[168] Information (parameter) regarding the (inverse) ACT may be generated and encoded by the encoding equipment and may be transmitted to the decoding equipment in the form of a bitstream.

[169] The decoding equipment may receive, parse, and decode the (reverse) ACT-related information (parameter) and may perform the reverse ACT based on the (reverse) ACT-related information (parameter).

[170] Based on the reverse ACT, (modified) residual samples (or residual block) can be extracted. For example, coefficients (transforms) may be extracted by applying dequantization to the quantized coefficients (transforms). In addition, residual samples may be extracted by performing an inverse transform on the coefficients (transform). In addition, (modified) residual samples can be obtained by applying an inverse ACT to the residual samples. Next, the information (parameter) regarding the (reverse) ACT is described in detail.

[171] In an embodiment, a base transform function used in HEVC can be used as a base transform function (transform kernel) for color space conversion. For example, matrices for forward transformation and inverse transformation may be used, as shown in the equations below.

[172] Equation 1

[173]

[174] Equation 2

[175]

[176] Here, C0, C1, and C2 may correspond to G, B, and R. Here, G denotes a green color component, B denotes a blue color component, and R denotes a red color component. In addition, C0', C1' and C2' may correspond to Y, Cg and Co. In this document, Y denotes the luminance signal, Cg denotes the green chrominance signal, and Co denotes the orange component of the chrominance signal.

[177] In addition, in order to compensate for the change in dynamic range of the residue before and after the color conversion, QP adjustment may be applied to the conversion residue by (-5, -5, -3). The details of QP regulation are described below.

[178] Meanwhile, in the encoding and decoding processes according to the embodiment, when the ACT is applicable, the following restrictions may apply.

[179] - In case of dual tree encoding/decoding, ACT is deactivated. For example, ACT may only apply to single tree based encoding/decoding.

[180] - When ISP encoding and decoding is applied, ACT may be deactivated.

[181] - For a chrominance block to which BDPCM is applied, ACT may be deactivated. Only for a luma block to which BDPCM is applied, ACT can be activated.

[182] - When the application of ACT is possible, CCLM can be deactivated.

[183] FIG. 12 is a view showing an embodiment of a sequence parameter set syntax table in which an ACT-related syntax element is signaled.

[184] FIG. 13-19 are views sequentially showing an embodiment of an encoding basis syntax table in which an ACT-related syntax element is signaled.

[185] As shown in FIG. 12, sps_act_enabled_flag 1210 may be used as an ACT-enabled flag indicating whether or not ACT is enabled during decoding.

[186] The first value (eg, 0) of sps_act_enabled_flag may indicate that ACT is not used, and the flag cu_act_enabled_flag 1310, 2910 indicating whether or not ACT is applied in the encoding basis is not provided in the syntax for the encoding basis.

[187] The second value (eg, 1) of sps_act_enabled_flag may indicate that ACT may be used, and cu_act_enabled_flag may be provided in the syntax for the encoding basis.

[188] When sps_act_enabled_flag is not obtained in the bitstream, the value of sps_act_enabled_flag may be retrieved as the first value (eg, 0).

[189] In addition, as shown in FIG. 13, cu_act_enabled_flag 1310, 2710 can be used as ACT flags indicating whether or not the remainder of the current coding basis in YCgCo color space is encoded.

[190] The first value (eg, 0) of cu_act_enabled_flag may indicate that the remainder of the current coding basis is encoded in the original color space. The second value (eg, 1) of cu_act_enabled_flag may indicate that the remainder of the current encoding basis is encoded in the YCgCo color space.

[191] When cu_act_enabled_flag is not provided in the bitstream, the flag may be retrieved as the first value (eg, 0). In this document, the source color space may be an RGB color space.

[192] Transform basis QP extraction method by using ACT QP bias

[193] In an embodiment, in a transform coefficient scaling process, a quantization parameter extraction process and a Qp update process may be performed as follows. For example, the quantization parameter extraction process may be performed using the following parameters.

[194] Luma coordinates (xCb, yCb) indicating the relative coordinates of the top left luma sample of the current coding block relative to the top left luminance sample of the current frame,

[195] - Variable cbWidth indicating the width of the current coding block based on a sample of luminance signals,

[196] - Variable cbHeight indicating the height of the current coding block based on a sample of luminance signals,

[197] - The treeType variable indicating whether a single tree (SINGLE_TREE) or a dual tree is used to segment the current node of the coding tree, and indicating when a dual tree is used, then represents the dual tree as a dual tree (DAUL_TREE_LUMA) of signal components luma or dual tree (DAUL_TREE_CHROMA) of chroma components.

[198] In the current process, the luminance quantization parameter Qp' _Y , the chrominance quantization parameters Qp' _Cb , Qp' _Cr and Qp' _CbCr can be extracted.

[199] The variable location (xQg, yQg) of the luminance signal may indicate the location of the top left luminance sample of the current quantization group corresponding to the top left sample of the current frame. Herein, the horizontal location xQg and the vertical location yQg may be set to the values of the CuQgTopLeftX variable and the CuQgTopLeftY variable, respectively. The variables CuQgTopLeftX and CuQgTopLeftY may be set as predefined values in the coding tree syntax as shown in FIG. 20.

[200] Here, the current quantization group may be a quad area in a coding tree block and may share the same qP _{Y_PRED} value. Its width and height may be equal to the width and height of the coding tree node in which the top left luma sample location is assigned to each of CuQgTopLeftX and CuQgTopLeftY.

[201] When treeType is SINGLE_TREE or DUAL_TREE_LUMA, the prediction value qP _{Y_PRED} of the luminance quantization parameter may be extracted as indicated in the following steps.

[202] 1. Variable qP _{Y_PRED} can be retrieved as follows.

[203] Condition 1: When any of the following conditions is "true", the value of qP _{Y_PRED} may be set to a value identical to the value of SliceQp _Y (here, SliceQp _Y indicates the initial value of the quantization parameter Qp _Y for all slices in a frame, and it can be obtained from the bitstream). Alternatively, the value qP _{Y_PRED} may be set equal to the value of the luminance quantization parameter Qp _Y of the last luminance coding basis of the immediately preceding quantization group according to the decoding order.

[204] - Condition 1-1. When the current quantization group is the first quantization group in the slice

[205] - Condition 1-2. When the current quantization group is the first quantization group in the tile

[206] - Condition 1-3. When the current quantization group is the first quantization group in the CTB row in a tile and a predefined sync occurs (for example, when entropy_coding_sync_enabled_flag is set to 1)

[207] 2. The value of the variable qP _{Y_A} can be retrieved as follows.

[208] Condition 2. When at least one of the following conditions is "true", the value of qP _{Y_A} may be set equal to the value of qP _{Y_PRED} . Alternatively, the value of qP _{Y_A} may be set equal to the value of the luminance quantization parameter Qp _Y of a coding basis that includes a luminance coding block covering a sample location (xQg-1, yQg) of the luminance signals.

[209] - Condition 2-1. With respect to the block identified by the sample location (xCb, yCb), the block identified by the sample location (xQg-1, yQg) is not an available neighbor block,

[210] - Condition 2-2. When the CTB including the luminance coding block covering the sample location (xQg-1, yQg) of the luminance signals is different from the CTB including the current luminance coding block at the sample location (xCb, yCb) of the luminance signals, for example, when all of the following conditions are "true",

[211] - Condition 2-2-1. The value of (xQg-1)>>CtbLog2SizeY is different from the value of (xCb)>>CtbLog2SizeY.

[212] - Condition 2-2-2. The value of (yQg)>>CtbLog2SizeY is different from the value of (yCb)>>CtbLog2SizeY.

[213] 3. The value of variable qP _{Y_B} can be retrieved as follows.

[214] Condition 3. When at least one of the following conditions is "true", the value of qP _{Y_B} may be set equal to the value of qP _{Y_PRED} . Alternatively, the qP _{Y_B} value may be set equal to the value of the luma quantization parameter Qp _Y of a coding basis that includes a luminance coding block covering a sample location (xQg, yQg-1) of the luminance signals.

[215] - Condition 3-1. Regarding the block identified by the sample location (xCb, yCb), when the block identified by the sample location (xQg, yQg-1) is not an available neighbor block,

[216] - Condition 3-2. When the CTB including the luminance coding block covering the sample location (xQg, yQg-1) of the luminance signals is different from the CTB including the current luminance coding block at the sample location (xCb, yCb) of the luminance signals, for example, when all of the following conditions are "true",

[217] - Condition 3-2-1. The value of (xQg)>>CtbLog2SizeY is different from the value of (xCb)>>CtbLog2SizeY,

[218] - Condition 3-2-2. The value of (yQg-1)>>CtbLog2SizeY is different from the value of (yCb)>>CtbLog2SizeY.

[219] 4. The prediction value qP _{Y_PRED} of the luminance quantization parameter may be derived as follows.

[220] When all of the following conditions are true, qP _{Y_PRED} may be set equal to the luminance quantization parameter Qp _Y of a coding basis that includes a luminance coding block covering a sample location (xQg, yQg-1) of the luminance signals.

[221] - Condition 3-1. With respect to the block identified by the sample location (xCb, yCb), the block identified by the sample location (xQg, yQg-1) is an available neighbor block

[222] - When the current quantization group is the first quantization group in the CTB row in a tile

[223] Meanwhile, when all conditions are not true, qP _{Y_PRED} may be extracted as shown in the equation below.

[224] Equation 3

[225] qP _{Y_PRED} =(qP _{Y_A} +qP _{Y_B} +1)>>1

[226] The Qp _Y variable may be extracted as shown in the equation below.

[227] Equation 4

[228] Qp _Y =((qP _{Y_PRED} +CuQpDeltaVal+64+2*QpBdOffset)%(64+QpBdOffset))-QpBdOffset

[229] Here, CuQpDeltaVal indicates the difference between the luminance quantization parameter for the coding basis and its predictive value. Its value may be obtained from the bit stream. QpBdOffset indicates the offset of the range of luminance and chrominance quantization parameters. QpBdOffset may be preset equal to a predefined constant or obtained from a bit stream. For example, QpBdOffset may be calculated by multiplying a predetermined constant by the value of a syntax element that indicates the bit depth of the luminance or chroma sample. The luminance quantization parameter Qp' _Y may be extracted as shown in the equation below.

[230] Equation 5

[231] _Qp'Y = _QpY +QpBdOffset

[232] When the value of the ChromaArrayType variable indicating the chrominance array type is not the first value (eg, 0), and treeType is SINGLE_TREE or DUAL_TREE_CHROMA, the following processing may be performed.

[233] - When the treeType value is DUAL_TREE_CHROMA, the value of the Qp _Y variable may be set to a value identical to the value of the luma quantization parameter Qp _Y of the luminance coding basis covering the sample location (xCb+cbWidth/2, yCb+cbHeight/2) of the signals brightness.

[234] - Variables qP _Cb , qP _Cr and qP _CbCr can be extracted as shown in the equation below.

[235] Equation 6

[236] qP _Chroma =Clip3(-QpBdOffset, 63, Qp _Y )

[237] qP _Cb =ChromaQpTable[0][qP _Chroma ]

[238] qP _Cr =ChromaQpTable[1][qP _Chroma ]

[239] qP _CbCr =ChromaQpTable[2][qP _Chroma ]

[240] The chrominance quantization parameters Qp' _Cb and Qp' _Cr for the Cb and Cr components and the chrominance quantization parameter Qp' _CbCr for the combined Cb-Cr coding may be extracted as shown in the equation below.

[241] Equation 7

[242] Qp' _Cb =Clip3(-QpBdOffset, 63, qP _Cb +pps_cb_qp_offset+slice_cb_qp_offset+CuQpOffset _Cb )+QpBdOffset

[243] Qp' _Cr =Clip3(-QpBdOffset, 63, qP _Cr +pps_cr_qp_offset+slice_cr_qp_offset+CuQpOffset _Cr )+QpBdOffset

[244] Qp' _CbCr =Clip3(-QpBdOffset, 63, qP _CbCr +pps_joint_cbcr_qp_offset+slice_joint_cbcr_qp_offset+CuQpOffsetCbC _r )+QpBdOffset

[245] In the above equation, pps_cb_qp_offset and pps_cr_qp_offset are offsets used to derive Qp' _Cb and Qp' _Cr and may be obtained from the bitstream for the frame parameter set. slice_cb_qp_offset and slice_cr_qp_offset are offsets used to extract Qp' _Cb and Qp' _Cr and may be obtained from the slice header bitstream. CuQpOffset _Cb and CuQpOffset _Cr are the offsets used to derive Qp' _Cb and Qp' _Cr and may be obtained from the bitstream for the transform basis.

[246] In addition, for example, the dequantization process for the transform coefficient may be performed using the following parameters.

[247] - Coordinates (xTbY, yTbY) of luminance signals referring to the relative coordinates of the top left sample of the current luminance transform block relative to the top left sample of the luminance signals of the current frame

[248] - Variable nTbW indicating the width of the transform block

[249] - Variable nTbH indicating the height of the transform block

[250] - Variable cIdx indicating the color component of the current block

[251] The output of the current process may be an array d of scaled transform coefficients. In this document, the size of the array d may be (nTbW)x(nTbH). The individual elements that make it up can be identified as d[x][y].

[252] To this end, the quantization parameter qP can be extracted as follows. When cIdx has a value of 0, qP can be extracted as shown in the equation below.

[253] Equation 8

[254] qP= _Qp'Y

[255] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, extraction may be performed as shown in the equation below.

[256] Equation 9

[257] qP= _Qp'CbCr

[258] Alternatively, when cIdx has a value of 1, qP may be extracted as shown in the equation below.

[259] Equation 10

[260] qP=Qp' _Cb

[261] Alternatively, when cIdx has a value of 2, qP may be extracted as shown in the equation below.

[262] Equation 11

[263] qP=Qp' _Cr

[264] Thereafter, the quantization parameter qP may be updated as follows. In addition, the variables rectNonTsFlag and bdShift can be retrieved as follows. For example, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0 (for example, when the transformation of the current transformation block is not skipped), extraction may be performed as shown in the equation below. Herein, the first value (eg, 0) of the transform_skip_flag may indicate that whether or not a transform is omitted is determined by another syntax element. The second value (eg, 1) of transform_skip_flag may indicate omitting (eg, skipping) the transform.

[265] Equation 12

[266] qP=qP-(cu_act_enabled_flag[xTbY][yTbY]?5:0)

[267] rectNonTsFlag=0

[268]bdShift=10

[269] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1 (eg, the current transform block's transform is skipped), extraction may be performed as shown in the equation below.

[270] Equation 13

[271] qP=Max(QpPrimeTsMin, qP)-(cu_act_enabled_flag[xTbY][yTbY]?5:0)

[272] rectNonTsFlag=(((Log2(nTbW)+Log2(nTbH)) and 1)==1

[273] bdShift=BitDepth+(rectNonTsFlag?1:0)+((Log2(nTbW)+Log2(nTbH))/2)-+pic_dep_quant_enabled_flag

[274] Herein, QpPrimeTsMin may indicate the minimum value of the quantization parameter allowed when the transform skipping mode is applied. It may be defined as a predefined constant, or may be derived from a bitstream syntax element associated with it.

[275] As used herein, the suffixes Y, Cb, and Cr may denote the G, B, and R color components in the RGB color model, or the Y, Cg, and Co color components in the YCgCo color model.

[276] Quantization/Dequantization

[277] As described above, the encoding equipment quantizer can extract quantized transform coefficients by applying quantization to the transform coefficients. An encoding equipment dequantizer or a decoding equipment dequantizer may extract transform coefficients by applying dequantization to the quantized transform coefficients.

[278] When encoding and decoding a video/still picture, the quantization factor may be changed, and the changed quantization factor may be used to adjust the compression ratio. According to an implementation aspect, considering the complexity, a quantization parameter (QP) may be used instead of using the quantization factor directly. For example, quantization parameters having integer values of 0-63 may be used, and each quantization parameter value may correspond to an actual quantization factor. In addition, the quantization parameter QP _Y for the luminance signal component (luminance samples) and the quantization parameter QP _C for the chrominance signal component (chroma samples) may be set such that they are different from each other.

[279] In the quantization process, a transform coefficient C may be input and divided by a quantization coefficient Qstep, and based on this, a quantized transform coefficient C' may be obtained. In this case, considering the computational complexity, the quantization factor may be multiplied by the scale to be in the form of an integer, and the shift operation may be performed by a value corresponding to the scale value. Based on the multiplication of the quantization factor and the scale value, the quantization scale can be extracted. Thus, the quantization scale can be extracted according to QP. A quantization scale may be applied to the transform coefficient C, and based on this, a quantized transform coefficient C' may be extracted.

[280] The dequantization process is an inverse process with respect to the quantization process. The quantized transform coefficient C' may be multiplied by the quantization coefficient Qstep, and based on this, the reconstructed transform coefficient C'' may be obtained. In this case, the level scale may be extracted according to the quantization parameter. Level scaling may be applied to the quantized transform coefficient C', and based on this, the reconstructed transform coefficient C'' may be extracted. The recovered transform coefficient C'' may differ slightly from the original transform coefficient C due to losses in the transform and/or quantization process. Therefore, the encoding equipment can perform dequantization in the same manner as the decoding equipment.

[281] Meanwhile, weighted adaptive frequency quantization technology in which the quantization intensity is adjusted according to frequency can be applied. Weighted adaptive frequency quantization technology is a method of applying quantization intensities that vary with frequency. In adaptive weighted frequency quantization, quantization intensities varying with frequency can be applied by using a predetermined quantization scaling matrix. Thus, the above-described quantization/dequantization process can be further performed based on the quantization scaling matrix. For example, different quantization scaling matrices may be used according to the current block size and/or whether the prediction mode applied to the current block to generate the current block residual signal is inter prediction or intra prediction. The quantization scaling matrix may be referred to as a "quantization matrix" or "scaling matrix". The quantization scaling matrix may be predefined. In addition, for frequency adaptive scaling, frequency-specific quantization scale information for the quantization scaling matrix may be constructed/encoded by the encoding equipment and may be signaled to the decoding equipment. The frequency-specific quantization scale information may be referred to as "quantization scaling information". The frequency-specific quantization scale information may include scaling list data (scaling_list_data). The (modified) quantization scaling matrix may be derived based on the scaling list data. In addition, the frequency-specific quantization scaling information may include presence flag information indicating whether scaling list data is present or not. Alternatively, when scaling list data is signaled at a high level (eg, SPS), information indicating whether or not the scaling list data at a lower level (eg, PPS or tile group header, etc.) is modified or not is further included.

[282] Conversion/Reverse Conversion

[283] As described above, the encoding equipment can extract the residual block (residual samples) based on the block (predictive samples) predicted via intra/inter/IBC prediction, and can extract quantized transform coefficients by applying transform and quantization to the extracted residual samples. . After being included in the residual encoding syntax, information (residual information) regarding the quantized transform coefficients may be encoded and output in the form of a bitstream. The decoding equipment can obtain information (residual information) about the quantized transform coefficients from the bit stream, and can extract the quantized transform coefficients by performing decoding. The decoding equipment may extract the residual samples via dequantization/inverse transform based on the quantized transform coefficients. As described above, quantization/dequantization or transform/inverse transform, or both, may be omitted. When the transform/inverse transform is omitted, the transform coefficient may be referred to as "coefficient" or "residual coefficient", or may still be referred to as "transformation coefficient" for consistency of expression. Whether or not a transform/inverse transform is omitted may be signaled based on a transform skip flag (eg, transform_skip_flag).

[284] The transformation/inverse transformation may be performed based on the transformation kernel(ep). For example, a multiple transform selection (MTS) scheme may be used to perform the transform/inverse transform. In this case, a portion of several sets of transformation kernels may be selected and applied to the current block. The transformation kernel may be referred to by various terms such as "transformation matrix", "transformation type", etc. For example, a set of transformation kernels may mean a combination of a transformation kernel in the vertical direction (vertical transformation kernel) and a transformation kernel in the horizontal direction (horizontal transformation kernel).

[285] Mapping/demapping may be performed on a CU or TU basis. Thus, the transform/inverse transform may be applied to the residual samples in the CU, or to the residual samples in the TU. The CU size and TU size may be identical, or multiple TUs may be present in a CU area. Meanwhile, the CU size may generally mean the CB size of the component (samples) luminance signals. TU-size, in General, may mean the TB-size of the component (samples) of the luminance signals. The CB or TB size of the chrominance component (samples) can be derived based on the CB or TB size of the luminance component (samples) according to the ratio of the components according to the color format (chrominance signal format, e.g. 4:4:4, 4:2 :2, 4:2:0, etc.). The TU size may be derived based on maxTbSize. For example, when the CU size is greater than maxTbSize, a plurality of TUs (TBs) having maxTbSizes can be extracted from the CUs, and mapping/demapping can be performed based on the TUs (TB). MaxTbSize may be taken into account in determining whether or not various types of intra-prediction such as ISP are applied. Information regarding maxTbSize may be predefined. Alternatively, the information regarding maxTbSize may be generated and encoded by the encoding equipment and signaled to the encoding equipment.

[286] Entropy Coding

[287] As described above with reference to FIG. 2, some or all of the video/image information may be entropy encoded by entropy encoder 190. Some or all of the video/image information described with reference to FIG. 3 may be entropy decoded by the entropy decoder 310. In this case, the video/image information may be encoded/decoded based on the syntax element. Herein, encoding/decoding information may include encoding/decoding by the method described in this paragraph.

[288] FIG. 21 shows a CABAC block diagram for encoding one syntax element. In the CABAC encoding process, firstly, when the input signal is a syntax element that is not a binary value, the input signal is converted to a binary value through binarization. When the input signal is already a binary value, the input signal bypasses the binarization. In this document, each binary number 0 or 1 constituting a binary value may be referred to as a "permission element". For example, a binary string (letter string) after binarization is 110, each of 1, 1 and 0 is referred to as "one tile". The permission element(s) for one syntax element may mean the value of the syntax element.

[289] The binarized bins may be input to a regular coding engine or a bypass coding engine. The regular encoding mechanism may assign a context model that applies a probability value to the corresponding bin, and may encode the bin based on the assigned context model. After performing encoding for each bin, the regular encoding engine may update the probability model for the bin. Permission elements encoded in this way may be referred to as "context-encoded permission elements". The bypass encoding mechanism may omit a procedure for estimating the probability of an input bin and a procedure for updating the probability model applied to the bin after encoding. In the case of the bypass coding mechanism, the input bin is encoded by applying a uniform probability distribution (eg, 50:50) instead of assigning a context, thereby improving the coding speed. Permission elements encoded in this way may be referred to as "bypass permission elements". A context model may be assigned and updated for each permission element to be context encoded (regularly encoded), and the context model may be specified based on ctxidx or ctxInc; ctxidx can be retrieved based on ctxInc. In particular, for example, a context index (ctxidx) indicating the context model for each of the regularly-encoded bins may be derived as the sum of the context index increment (ctxInc) and the context index offset (ctxIdxOffset). In this document, ctxInc, which varies depending on the permission element, can be retrieved. ctxIdxOffset may be represented by the smallest value of ctxIdx. The smallest value of ctxIdx may be referred to as "initial value (initValue) ctxIdx". ctxIdxOffset is a value generally used to distinguish from context models for other syntax elements, and the context model for one syntax element may be different/derived based on ctxinc.

[290] In the entropy encoding procedure, it is determined whether to perform encoding through a regular encoding mechanism or to perform encoding through a bypass encoding mechanism, and the encoding path may be switched. Entropy decoding may perform the same process as entropy encoding in reverse order.

[291] The above-described entropy encoding, for example, may be performed as indicated in FIG. 22 and 23. Referring to FIGS. 22 and 23, the encoding equipment (entropy encoder) may perform an entropy encoding procedure to obtain image/video information. The image/video information may include segmentation related information, prediction related information (e.g., inter/intra prediction classification information, intra prediction mode information, and inter prediction mode information), residual information, and in-loop filtering related information, or may include itself various syntactic elements associated with it. Entropy encoding may be performed based on the syntax element. Steps S2210-S2220 of FIG. 22 may be performed by the entropy encoder 190 of the encoding equipment of FIG. 2 described above.

[292] The encoding equipment may perform binarization for the target syntax element in step S2210. Here, the binarization may be based on various binarization methods such as a truncated Rice code binarization process and a fixed length code binarization process, and the binarization method for the target syntax element may be preset. The binarization procedure may be performed by the binarization module 191 in the entropy encoder 190.

[293] The encoding equipment may perform entropy encoding for the target syntax element in step S2220. The encoding hardware may perform regular (context) or bypass encoding for the target syntax element's bin string based on an entropy coding scheme such as context adaptive arithmetic coding (CABAC) or context adaptive variable length coding (CAVLC). The output may be included in the bit stream. The entropy encoding procedure may be performed by the entropy encoding processor 192 in the entropy encoder 190. As described above, the bit stream may be transmitted to the decoding equipment via a (digital) storage medium or network.

[294] Referring to FIG. 24 and 25, the decoding equipment (entropy decoder) can decode the encoded image/video information. The image/video information may include segmentation related information, prediction related information (e.g., inter/intra prediction classification information, intra prediction mode information, and inter prediction mode information), residual information, and in-loop filtering related information, or may include itself various syntactic elements associated with it. Entropy encoding may be performed based on the syntax element. Steps S2410-S2420 may be performed by the entropy decoder 210 of the decoding equipment of FIG. 3 described above.

[295] The decoding equipment may perform binarization on the target syntax element in step S2410. Here, the binarization may be based on various binarization methods such as a truncated Rice code binarization process and a fixed length code binarization process, and the binarization method for the target syntax element may be preset. The decoding hardware may extract the available bin strings (possible bin strings) for the available values of the target syntax element via a binarization procedure. The binarization procedure may be performed by the binarization module 211 in the entropy decoder 210.

[296] The decoding equipment may perform entropy decoding for the target syntax element in step S2420. While the decoding hardware sequentially decodes and parses each bin for the target syntax element from the input bit(s) in the bitstream, and can compare the extracted bin string with the available bin strings for the syntax element. When the retrieved permission element string is identical to one of the available permission element strings, the value corresponding to the permission element string may be retrieved as the value of the syntax element. If not, the next bit in the bitstream is further parsed and the above procedure is performed again. Through this process, the start bit or end bit for particular information (a particular syntax element) in the bit stream is not used, but variable length bits are used to signal information. Due to this, relatively fewer bits are assigned to the low value, thereby improving the overall coding efficiency.

[297] The decoding equipment may perform contextual or bypass decoding for each bin in a bin string from a bit stream based on an entropy coding technology such as CABAC or CAVLC. The entropy decoding procedure may be performed by the entropy decoding processor 212 in the entropy decoder 210. The bit stream may include various types of image/video decoding information as described above. As described above, the bitstream may be transmitted to the decoding equipment via a (digital) storage medium or a network.

[298] Herein, a table (syntax table) including syntax elements may be used to represent the signaling of information from encoding equipment to decoding equipment. The order of the syntax elements in a table including the syntax elements used herein may indicate the parsing order of the syntax elements from the bit stream. The encoding equipment may construct and encode the parse table such that the syntax elements are parsed in parsing order by the decoding equipment. The decoding equipment can parse and decode the syntax elements of the syntax table from the bitstream in parsing order, and thereby can obtain the values of the syntax elements.

[299] General Image/Video Coding Procedure

[300] When encoding images/videos, frames constituting an image/video may be encoded/decoded according to a decoding order in a sequence. The frame order corresponding to the output order of the decoded frames may be set such that it is different from the decoding order, and based on this, forward prediction as well as reverse direction may be performed when inter prediction is performed.

[301] FIG. 26 shows an example of a schematic frame decoding procedure to which the embodiment(s) of the present document is applicable. In FIG. 26, step S2610 may be performed by the entropy decoder 210 of the decoding equipment described above with reference to FIG. 3. Step S2620 may be performed by a prediction module including an intra prediction module 265 and an inter prediction module 260 . Step S2630 may be performed by a residual processor including a dequantizer 220 and an inverse converter 230. Step S2640 may be performed by an adder 235. Step S2650 may be performed by a filter 240. Step S2610 may include an information decoding procedure described herein. Step S2620 may include the inter/intra prediction procedure described herein. Step S2630 may include the post-processing procedure described herein. Step S2640 may include a block/frame recovery procedure described herein. Step S2650 may include the in-loop filtering procedure described herein.

[302] Referring to FIG. 26, the frame decoding procedure may schematically include, as described above with reference to FIG. 3, the procedure for obtaining image/video information from the bitstream (by decoding) in step S2610, the frame recovery procedure in steps S2620 to S2640, and the in-loop filtering procedure for the reconstructed frame in step S2650. The frame recovery procedure may be performed based on the predictive samples and residual samples that are obtained through inter/intra prediction in step S2620 and the residual processing in step S2630 (quantized transform coefficient dequantization and inverse transformation process) described herein. The modified reconstructed frame may be generated through the in-loop filtering procedure for the reconstructed frame generated through the frame recovery procedure. The modified reconstructed frame may be output as a decoded frame and may be stored in a decoded frame buffer or memory 250 of the decoding equipment for later use as a reference frame in an inter prediction procedure when the frame is decoded. In some cases, the in-loop filtering procedure may be omitted. In this case, the reconstructed frame may be output as a decoded frame and may be stored in a decoded frame buffer or memory 250 of the decoding equipment for later use as a reference frame in an inter prediction procedure when the frame is decoded. As described above, the in-loop filtering procedure in step S2650 may include a deblocking filtering procedure, a sampled adaptive offset (SAO) filtering procedure, an adaptive loop filtering (ALF) procedure, and/or a bilateral filtering procedure. Some or all of it may be omitted. In addition, one or more of the deblocking filter procedure, the sampled adaptive offset (SAO) filter procedure, the adaptive loop filter (ALF) procedure, and the bilateral filter procedure may be sequentially applied, or all of them may be sequentially applied. For example, a deblocking filtering procedure may be applied to the reconstructed frame, and then the SAO procedure may be performed. Alternatively, for example, the deblocking filtering procedure may be applied to the reconstructed frame, and then the ALF procedure may be performed. This may be done in the same manner as in the encoding equipment.

[303] FIG. 27 shows an example of a schematic frame encoding procedure to which the embodiment(s) of the present document is applicable. In FIG. 27, step S2710 may be performed by a prediction unit including an intra prediction unit 185 or an encoding equipment inter prediction unit 180 described above with reference to FIG. 2. Step S2720 may be performed by a residual processor including a transform 120 and/or a quantizer 130. Step S2730 may be performed by an entropy encoder 190. Step S2710 may include an inter/intra prediction procedure described herein. Step S2720 may include the post-processing procedure described herein. Step S2730 may include the information encoding procedure described herein.

[304] Referring to FIG. 27, the frame encoding procedure may schematically include, as described above with reference to FIG. 2, a procedure for encoding information (e.g., prediction information, residual information, and segmentation information) for frame recovery and outputting information in the form of a bit stream, a procedure for generating a reconstructed frame for the current frame, and a procedure (optionally) for applying in-loop filtering to the reconstructed frame. The encoding equipment may extract the (modified) residual samples from the quantized transform coefficients via the dequantizer 140 and the inverse transform 150, and may generate a reconstructed frame based on the predictive samples that are output in step S2710 and the (modified) residual samples. The generated reconstructed frame may be identical to the reconstructed frame generated by the decoding equipment described above. An in-loop filtering procedure may be performed on the reconstructed frame to generate a modified reconstructed frame. The modified reconstructed frame may be stored in the decoded frame buffer or memory 170. Similar to the case in decoding equipment, the modified reconstructed frame may be used subsequently as a reference frame in an inter prediction procedure when the frame is encoded. As described above, in some cases, part or all of the in-loop filtering procedure may be omitted. When the in-loop filtering procedure is performed, the (in-loop) filtering related information (parameter) may be encoded by the entropy encoder 190 and output in the form of a bit stream. The decoding equipment can perform an in-loop filtering procedure based on the filtering related information in the same manner as the encoding equipment.

[305] Through this in-loop filtering procedure, noise such as blocking artifact and ringing artifact generated during image/video encoding can be reduced, and subjective/objective visual quality can be improved. In addition, the encoding equipment and the decoding equipment perform an in-loop filtering procedure so that the encoding equipment and the decoding equipment can extract the same prediction result, the frame encoding reliability can be improved, and the amount of data to be transmitted for frame encoding can be reduced.

[306] As described above, the frame recovery procedure may be performed in the decoding equipment as well as in the encoding equipment. A reconstructed block may be generated based on intra-prediction/inter-prediction on a per-block basis, and a reconstructed frame including the reconstructed blocks may be generated. When the current frame/slice/tile group is an I-frame/slice/tile group, blocks included in the current frame/slice/tile group can only be reconstructed based on intra prediction. Meanwhile, when the current frame/slice/tile group is a P or B frame/slice/tile group, blocks included in the current frame/slice/tile group can be reconstructed based on intra prediction or inter prediction. In this case, intra prediction may be applied to some blocks in the current frame/slice/tile group, and inter prediction may be applied to some remaining blocks. A frame color component may include a luminance signal component and a chrominance signal component. Unless expressly limited herein, the method and embodiments provided herein may be applied to the luminance signal component and the chrominance signal component.

[307] An example of a hierarchy and coding structure

[308] The encoded video/image according to the present document may be processed, for example, according to the coding hierarchy and structure, which is described below.

[309] FIG. 28 is a view showing a hierarchical structure for an encoded picture. The encoded image may be divided into a video coding layer (VCL, video coding hierarchy) that conducts image and self decoding processing, a subsystem for transmitting and storing encoded information, and a network abstraction layer (NAL, network abstraction hierarchy) that is present between the VCL and subsystem and is responsible for the network adaptation function.

[310] In VCL, VCL data including compressed image data (slice data) may be generated. Alternatively, a parameter set may be generated including information such as a frame parameter set (PPS), a sequence parameter set (SPS), and a video parameter set (VPS), or a Supplemental Enhancement Information (SEI) message further required for the picture decoding process. .

[311] In the NAL, header information (NAL unit header) is added to the Primary Data Byte Sequence (RBSP) generated in the VCL such that the NAL unit can be generated. In this document, RBSP stands for slice data, parameter set, and SEI message generated in the VCL. The NAL unit header may include NAL unit type information that is indicated according to the RBSP data included in the NAL unit.

[312] As shown in the drawing, the NAL unit may be divided into a VCL NAL unit and a non-VCL NAL unit depending on the RBSP generated in the VCL. A VCL NAL unit may mean a NAL unit that includes information (slice data) about an image. A non-VCL NAL unit may mean a NAL unit that includes information (parameter set or SEI message) required to decode a picture.

[313] With header information attached according to the subsystem data standard, a VCL NAL unit and a non-VCL NAL unit can be transmitted over the network. For example, a NAL unit may be converted into data form of a predetermined standard such as H.266/VVC file format, Real Time Transport Protocol (RTP), Transport Stream (TS), and the resulting data may be transmitted over various networks.

[314] As described above, with respect to the NAL unit, the type of the NAL unit may be indicated according to the RBSP data structure included in the NAL unit, and information regarding the type of the NAL unit may be stored in the header of the NAL unit and signaled.

[315] For example, depending on whether or not the NAL unit includes information (slice data) about an image, a rough classification is made into a VCL NAL unit type and a non-VCL NAL unit type. A VCL type of a NAL unit may be classified according to a characteristic and a frame type included in a VCL NAL unit, and a non-VCL NAL unit type may be classified according to a parameter set type.

[316] NAL unit types indicated according to parameter set types, which include non-VCL NAL unit types, are listed below as an example.

[317] - APS (Adaptation Parameter Set) NAL unit: type of NAL unit including APS

[318] - DPS (Decoding Parameter Set) NAL unit: type of NAL unit including the DPS

[319] - VPS (Video Parameter Set) NAL unit: type of NAL unit including the VPS

[320] - SPS (Sequence Parameter Set) NAL Unit: The type of NAL unit including the SPS

[321] - PPS (frame parameter set) NAL unit: type of NAL unit including the PPS

[322] The above NAL unit types have syntax information for the NAL unit type, and the syntax information can be stored in the NAL unit header and signaled. For example, the syntax information may be nal_unit_type, and NAL unit types may be indicated by nal_unit_type values.

[323] The slice header (slice header syntax) may include information/parameter applicable to the slice collectively. APS (APS syntax) or PPS (PPS syntax) may include information/parameter applicable to one or more slices or frames together. SPS (SPS syntax) may include information/parameter applicable to one or more sequences together. VPS (VPS syntax) may include information/parameter applicable to multiple layers in common. DPS (DPS syntax) may include information/parameter applicable to the entire video together. The DPS may include information/parameter related to coded video sequence (CVS) concatenation. Herein, the High Level Syntax (HLS) may include at least one selected from the group of APS Syntax, PPS Syntax, SPS Syntax, VPS Syntax, DPS Syntax, and Slice Header Syntax.

[324] Here, the image/video information encoded by the encoding equipment and signaled in the form of a bit stream to the decoding equipment may include segmentation-related information, intra/inter prediction information, residual information, and in-loop filtering information. in the frame, as well as information included in the slice header, information included in the APS, information included in the PPS, information included in the SPS, and/or information included in the VPS.

[325] General representation of block differential pulse code modulation (BDPCM)

[326] The image encoding equipment and the image decoding equipment according to the embodiment may perform differential encoding of the residual signal. For example, the image encoding equipment may encode the residual signal by subtracting the prediction signal from the current block residual signal, and the image decoding equipment may decode the residual signal by summing the prediction signal with the current block residual signal. The image encoding equipment and the image decoding equipment according to the embodiment can perform differential encoding of the residual signal by applying BDPCM described below.

[327] BDPCM according to the present disclosure may be performed in a quantized residual region. The quantized residual region may include a quantized residual signal (or a quantized residual coefficient), and when applying BDPCM, transforming the quantized residual signal may be skipped. For example, when applying BDPCM, residual signal transformation may be skipped and quantization may be performed. Alternatively, the quantized residual region may include quantized transform coefficients.

[328] In the embodiment to which BDPCM is applied, the image encoding equipment can extract a residual block relative to the current block predicted in the intra prediction mode and quantize the residual block, thereby extracting the residual block. When the residual signal differential encoding mode is performed with respect to the current block, the image encoding equipment may perform differential encoding with respect to the residual block to extract a modified residual block. In addition, the image encoding equipment can encode differential encoding mode information indicating the differential encoding mode of the residual signal and the modified residual block, thereby generating a bit stream.

[329] More specifically, when BDPCM is applied to a current block, a predicted block (prediction block) including the predicted samples of the current block may be generated by intra prediction. In this case, the intra prediction mode for performing intra prediction may be signaled via a bitstream, and may be derived based on the BDPCM prediction direction described below. In addition, in this case, the intra prediction mode may be determined as one of the vertical prediction direction mode or the horizontal prediction direction mode. For example, when the BDPCM prediction direction is the horizontal direction, the intra prediction mode may be determined as the horizontal prediction direction mode, and a prediction block with respect to the current block may be generated by intra prediction of the horizontal direction. Alternatively, when the BDPCM prediction direction is the vertical direction, the intra prediction mode may be determined as the vertical direction prediction mode, and a prediction block with respect to the current block may be generated by intra prediction of the vertical direction. When applying internal prediction of the horizontal direction, the value of a pixel adjacent to the left of the current block may be determined as a predictive sample value for the samples included in the corresponding row of the current block. When applying intra prediction of the vertical direction, the value of a pixel adjacent to a vertex with respect to the current block may be determined as a predictive sample value for the samples included in the corresponding column of the current block. When applying BDPCM to the current block, the method of generating a prediction block with respect to the current block can be equally performed in the image encoding equipment and in the image decoding equipment.

[330] When applying BDPCM to the current block, the image coding equipment may generate a residual block including residual samples of the current block by subtracting a predictive sample from the current block. The image encoding equipment may quantize the residual block and then encode the difference (or delta) between the quantized residual sample and the quantized residual sample predictor. The image decoding equipment can generate a quantized residual block with respect to the current block by obtaining a quantized residual sample of the current block based on the predictor and the difference recovered from the bitstream. Thereafter, the image decoding equipment may dequantize the quantized residual block and then add it to the prediction block, thereby recovering the current block.

[331] FIG. 29 is a view illustrating a BDPCM residual sample encoding method according to the present disclosure. The remaining block of Fig. 29 may be generated by subtracting a prediction block from the current block in the image encoding equipment. The quantized residual block of FIG. 29 may be generated by quantizing the residual block by the image encoding equipment. In FIG. 29, r _{i, j} indicates the value of the residual sample of coordinates (i, j) in the current block. When the current block size is MxN, the value of i can be from 0 to M-1, inclusive. In addition, the value of j can be from 0 to N-1, inclusive. For example, the remainder may mean the difference between the original block and the prediction block. For example, r _{i, j} may be derived by subtracting the predictive sample value from the original coordinate sample value (i, j) in the current block. For example, r _{i, j} may be a prediction residual after horizontal intra prediction or vertical intra prediction is performed using a sample that is not filtered from the top or left border sample. In horizontal intra prediction, the value of the left neighbor pixel is copied along a line crossing the prediction block. In vertical intra prediction, the upper adjacent line is copied to a separate line of the prediction block.

[332] FIG. 29, Q(r _{i, j} ) refers to the value of the quantized residual sample of coordinates (i, j) in the current block. For example, Q(r _{i, j} ) may refer to the quantized value r _{i, j} .

[333] BDPCM prediction is performed for the quantized residual samples of FIG. 29, and a modified quantized residual block Rp' having a size MxN including modified quantized residual samples rp' can be generated.

[334] When the BDPCM prediction direction is the horizontal direction, the value rp' _{i, j} of the modified quantized residual sample of coordinates (i, j) in the current block may be calculated as shown in the equation below.

[335] Equation 14

[336]

[337] As shown in Equation 14, when the BDPCM prediction direction is the horizontal direction, the quantized residual sample value Q(r _0,j ) is assigned as is to the coordinate value rp' _0,j of (0,j). The value rp' _{i, j} of other coordinates (i, j) can be extracted as the difference between the value Q(r _{i, j} ) of the quantized residual sample of coordinates (i, j) and the value Q(r _i-1,j ) of the quantized residual coordinate samples (i-1,j). Thus, instead of encoding the value Q(r _{i, j} ) of the quantized residual sample of coordinates (i, j), the difference value calculated by using the value Q(r _i-1,j ) of the quantized residual sample of coordinates (i-1,j) as a predictive value, is extracted as a modified quantized residual sample value rp' _{i, j} , and then the value of rp' _{i, j} is encoded.

[338] When the BDPCM prediction direction is the vertical direction, the value (rp' _{i, j} ) of the modified quantized residual sample of coordinates (i, j) in the current block may be calculated as shown in the equation below.

[339] Equation 15

[340]

[341] As shown in Equation 15, when the BDPCM prediction direction is the vertical direction, the quantized residual sample value Q(r _i,0 ) is assigned as is to the coordinate value rp' _i,0 (i,0). The value rp' _{i, j} of other coordinates (i, j) can be extracted as the difference between the value Q(r _{i, j} ) of the quantized residual sample of coordinates (i, j) and the value Q(r _{i, j-1} ) of the quantized residual coordinate samples (i, j-1). Thus, instead of encoding the value Q(r _{i, j} ) of the quantized residual sample of coordinates (i, j), the difference value calculated by using the value Q(r _{i, j-1} ) of the quantized residual sample of coordinates (i, j-1) as a predictive value, is extracted as a modified quantized residual sample value rp' _{i, j} , and then the value of rp' _{i, j} is encoded.

[342] As described above, the process of modifying a current quantized residual sample value by using a nearby quantized residual sample value as a predictive value may be referred to as "BDPCM prediction".

[343] Finally, the image coding equipment may encode a modified quantized residual block including the modified quantized residual samples, and may transmit the resulting block to the image decoding equipment. Here, as described above, no transformation of the modified quantized residual block is performed.

[344] FIG. 30 is a view showing modified quantized residual blocks generated by performing BDPCM of the present disclosure.

[345] FIG. 30, the horizontal BDPCM shows the modified quantized residual block generated according to Equation 14 when the BDPCM prediction direction is the horizontal direction. In addition, the vertical BDPCM shows a modified quantized residual block generated according to Equation 15 when the BDPCM prediction direction is the vertical direction.

[346] FIG. 31 is a flowchart showing a procedure for encoding a current block by applying BDPCM in the image encoding equipment.

[347] First, when the current block, which is the target coding block, is input in step S3110, prediction may be performed on the current block to generate a prediction block in step S3120. The prediction block in step S3120 may be an intra prediction block, and the intra prediction mode may be determined as described above. Based on the prediction block generated in step S3120, a residual block with respect to the current block may be generated in step S3130. For example, the image encoding equipment may generate a residual block (residual sample values) by subtracting a prediction block (predicted sample values) from the current block (original sample values). For example, by performing step S3130, the residual block of FIG. 29 may be formed. For the residual block generated in step S3130, quantization may be performed in step S3140 so as to generate a quantized residual block, and BDPCM prediction may be performed for the quantized residual block in step S3150. The quantized residual block generated by step S3140 may be the quantized residual block of FIG. 29. As a result of the BDPCM prediction in step S3150, the modified quantized residual block of FIG. 30 may be generated according to the prediction direction. Since the BDPCM prediction in step S3150 is described with reference to FIG. 29-30, its detailed description is omitted. Thereafter, the image encoding equipment may encode the modified quantized residual block in step S3160 to generate a bitstream. Herein, the transformation of the modified quantized residual block may be omitted.

[348] The BDPCM operation in the image coding equipment described with reference to FIG. 29-31 may be reversed by image decoding equipment.

[349] FIG. 32 is a flowchart showing a procedure for recovering a current block by applying BDPCM in the image decoding equipment.

[350] The image decoding equipment may obtain information (image information) required to reconstruct the current block from the bitstream in step S3210. The required information for restoring the current block may include information (prediction information) regarding the prediction of the current block and information (residual information) regarding the remainder of the current block. The image decoding equipment may perform prediction on the current block based on the information on the current block, and may generate a prediction block in step S3220. The prediction about the current block may be intra prediction, and its detailed description is identical to that described above with reference to FIG. 31. In FIG. 32, it is shown that step S3220 of generating a prediction block for the current block is performed before steps S3230 to S3250 of generating a residual block with respect to the current block. However, there are no restrictions on this. After a residual block with respect to the current block is generated, a prediction block with respect to the current block may be generated. Alternatively, a residual block with respect to the current block and a prediction block with respect to the current block may be generated at the same time.

[351] The image decoding equipment may generate a residual block with respect to the current block in step S3230 by parsing the residual information of the current block from the bit stream. The residual block generated in step S3230 may be a modified quantized residual block shown in FIG. thirty.

[352] The image decoding equipment may generate the quantized residual block of FIG. 29 by performing BDPCM prediction in step S3240 on the modified quantized residual block of FIG. 30. BDPCM prediction in step S3240 is a procedure for generating the quantized residual block of FIG. 29 from the modified quantized residual block of FIG. 30, which corresponds to the reverse process of step S3150 performed by the image encoding equipment. For example, when differential encoding mode information (eg, bdpcm_flag) obtained from the bit stream indicates a differential encoding mode in which residual coefficient differential encoding is performed as BDPCM is applied, the image decoding equipment performs differential encoding on the residual block to extract modified residual block. Using the residual coefficient to be modified and the residual prediction coefficient, the image decoding equipment can modify at least one residual coefficient to be modified from the residual coefficients in the residual block. The residual prediction coefficient may be determined based on a prediction direction indicated by differential encoding direction information (eg, bdpcm_dir_flag) obtained from the bitstream. The differential encoding direction information may indicate a vertical direction or a horizontal direction. The image decoding equipment may assign a value obtained by summing the residual coefficient to be modified and the residual prediction coefficient to the location of the residual coefficient to be modified. Herein, the residual prediction coefficient may be a coefficient that is immediately before and adjacent to the residual coefficient to be modified in terms of order according to the prediction direction.

[353] Hereinafter, this document describes in more detail the BDPCM prediction in step S3240 performed by the image decoding equipment. The decoding equipment may calculate the quantized residual sample Q(r _{i, j} ) by performing the calculation performed by the encoding equipment in reverse order. For example, when the BDPCM prediction direction is the horizontal direction, the image decoding equipment can generate a quantized residual block from the modified quantized residual block by using Equation 16.

[354] Equation 16

[355]

[356] As given in Equation 16, the value Q(r _{i, j} ) of the quantized residual sample of coordinates (i, j) can be calculated by summing the values of the modified quantized residual samples ranging from coordinates (0, j) to coordinates (i, j ).

[357] Alternatively, using Equation 17 instead of Equation 16, the value Q(r _{i, j} ) of the quantized residual sample of coordinates (i, j) may be calculated.

[358] Equation 17

[359]

[360] Equation 17 is the inverse process corresponding to Equation 14. According to Equation 17, the value Q(r _0,j ) of the quantized residual sample of coordinates (0,j) is extracted as the value of rp' _0,j of the modified quantized residual sample of coordinates ( 0,j). Q(r _{i, j} ) of the other coordinates (i, j) is extracted as the sum of the rp' _{i, j} value of the modified quantized residual sample of coordinates (i, j) and the value of Q(r _i-1,j ) of the quantized residual sample of coordinates ( i-1,j). That is, the difference value rp' _{i, j} is summed by using the quantized residual sample value Q(r _i-1,j ) of the quantized residual sample of coordinates (i-1,j) as the predictive value, thereby extracting the quantized residual sample value Q(r _{i, j} ).

[361] When the BDPCM prediction direction is the vertical direction, the image decoding equipment may generate a quantized residual block from the modified quantized residual block by using Equation 18.

[362] Equation 18

[363]

[364] As given in Equation 18, the value Q(r _{i, j} ) of the quantized residual sample of coordinates (i, j) can be calculated by summing the values of the modified quantized residual samples ranging from coordinates (i,0) to coordinates (i, j ).

[365] Alternatively, using Equation 19 instead of Equation 18, the value Q(r _{i, j} ) of the quantized residual sample of coordinates (i, j) may be calculated.

[366] Equation 19

[367]

[368] Equation 19 is the inverse process corresponding to Equation 15. According to Equation 19, the value Q(r _i,0 ) of the quantized residual sample of coordinates (i,0) is extracted as the value of rp' _i,0 of the modified quantized residual sample of coordinates ( i,0). Q(r _{i, j} ) of the other coordinates (i, j) is extracted as the sum of the rp' _{i, j} value of the modified quantized coordinate residual sample (i, j) and the Q(r _{i, j-1} ) value of the quantized coordinate residual sample ( i, j-1). Thus, the difference value rp' _{i, j} is summed by using the value Q(r _{i, j-1} ) of the quantized residual sample of coordinates (i, j-1) as the predictive value, thereby extracting the quantized residual sample value Q(r _{i, j} ).

[369] When a quantized residual block composed of quantized residual samples is generated by performing step S3240 according to the above method, the image decoding equipment performs dequantization on the quantized residual block in step S3250 to generate a residual block with respect to the current block. When BDPCM is applied, the transformation of the current block is skipped as described above. Therefore, the inverse transform of the dequantized residual block may be skipped.

[370] Thereafter, the image decoding equipment may reconstruct the current block in step S3260 based on the prediction block generated in step S3220 and the residual block generated in step S3250. For example, the image decoding equipment can reconstruct the current block (reconstructed sample values) by summing the prediction block (predicted sample values) and the residual block (residual sample values). For example, the reconstructed sample value may be generated by summing the dequantized quantized sample Q ⁻¹ (Q(r _{i, j} )) with the intra block predictive value. Differential coding mode information indicating whether or not BDPCM is applied to the current block may be signaled via a bitstream. In addition, when BDPCM is applied to a current block, differential coding direction information indicating a BDPCM prediction direction may be signaled via a bitstream. When BDPCM is not applied to the current block, differential coding direction information may not be signaled.

[371] FIG. 33-35 are views schematically showing the syntax for signaling information regarding BDPCM.

[372] FIG. 33 is a view showing the syntax of a sequence parameter set according to an embodiment for signaling BDPCM information. In an embodiment, all RBSP SPSs included in at least one access unit (AU) having a value of 0 as a temporary identifier (TemporalId) or provided via an external means may be set to be used before being accessed in the decoding process. In addition, the SPS NAL unit including the SPS RBSP may be set to have a nuh_layer_id that is identical to the nuh_layer_id of the PPS NAL unit referring to the SPS NAL unit. In CVS, all SPS NAL units having a particular sps_seq_parameter_set_id value can be set to have identical content. In the seq_parameter_set_rbsp() syntax of FIG. 33, the sps_transform_skip_enable_flag described above and the sps_bdpcm_enabled_flag described below are expanded.

[373] The sps_bdpcm_enabled_flag syntax element may indicate whether or not an intra coding unit, intra_bdpcm_flag, is provided in the CU syntax. For example, the first value (eg, 0) of sps_bdpcm_enabled_flag may indicate that, for an intra coding unit, intra_bdpcm_flag is not provided in the CU syntax. The second value (eg, 1) of sps_bdpcm_enabled_flag may indicate that, for an intra coding unit, intra_bdpcm_flag is provided in the CU syntax. Meanwhile, when sps_bdpcm_enabled_flag is not provided, the value of sps_bdpcm_enabled_flag may be set to the first value (eg, 0).

[374] FIG. 34 is a view showing an embodiment of a syntax for signaling whether or not a restriction on BDPCM applies. In an embodiment, a predetermined constraint during the encoding/decoding process may be signaled using the general_constraint_info() syntax. Using the syntax of FIG. 34, a no_bdpcm_constraint_flag syntax element indicating whether or not the value of the sps_bdpcm_enabled_flag described above should be set to 0 may be signaled. For example, the first value (eg, 0) of no_bdpcm_constraint_flag may indicate that such a constraint does not apply. When the value of no_bdpcm_constraint_flag is the second value (eg, 1), the value of sps_bdpcm_enabled_flag may be forced to be the first value (eg, 0).

[375] FIG. 35 is a view showing an embodiment of a coding_unit() syntax for signaling information regarding BDPCM for a coding unit. As shown in FIG. 35, the intra_bdpcm_flag and intra_bdpcm_dir_flag syntax elements may be signaled using the coding_unit() syntax. The intra_bdpcm_flag syntax element may indicate whether or not BDPCM is applied to the current luminance coding block located at (x0, y0).

[376] For example, the first value (eg, 0) of intra_bdpcm_flag may indicate that BDPCM is not applied to the current luminance coding block. The second value (eg, 1) of the intra_bdpcm_flag may indicate that BDPCM is applied to the current luminance coding block. By indicating that BDPCM is applied, the intra_bdpcm_flag may indicate whether or not the conversion is skipped, and whether or not the intra luminance prediction mode is performed, by means of the intra_bdpcm_dir_flag, which is described below.

[377] Meanwhile, the value of the above-described variable BdpcmFlag[x][y] may be set to the value of intra_bdpcm_flag with respect to x=x0...x0+cbWidth-1 and y=y0...y0+cbHeight-1.

[378] The syntax element intra_bdpcm_dir_flag may indicate the direction of BDPCM prediction. For example, the first value (eg, 0) of intra_bdpcm_dir_flag may indicate that the BDPCM prediction direction is a horizontal direction. The second value (eg, 1) of intra_bdpcm_dir_flag may indicate that the BDPCM prediction direction is a vertical direction.

[379] Meanwhile, the value of the variable BdpcmDir[x][y] may be set equal to the value of intra_bdpcm_dir_flag with respect to x=x0...x0+cbWidth-1 and y=y0...y0+cbHeight-1.

[380] Chroma Block Intra Prediction

[381] When intra prediction is performed for the current block, prediction with respect to the luminance signal component block (luminance signal block) of the current block and prediction with respect to the chrominance signal component block (chroma signal block) may be performed. In this case, the intra prediction mode for the chroma block may be set separately from the intra prediction mode for the luma block.

[382] For example, the intra prediction mode for the chroma block may be indicated based on the intra chroma prediction mode information. The intra chroma prediction mode information may be signaled in the form of an intra_chroma_pred_mode syntax element. For example, the intra chrominance prediction mode information may indicate one of planar mode, DC mode, vertical mode, horizontal mode, extracted mode (DM), and cross-component linear model (CCLM) mode. Herein, planar mode may mean intra prediction mode 0, DC mode may mean intra prediction mode 1, vertical mode may mean intra prediction mode 26, and horizontal mode may mean intra prediction mode 10. DM may also be referred to as "direct mode". CCLM may also be referred to as "linear model (LM)". The CCLM mode may include any of the following: L_CCLM, T_CCLM, and LT_CCLM.

[383] Meanwhile, DM and CCLM are dependent intra prediction modes for predicting a chrominance block by using luma block information. DM may mean a mode in which an intra prediction mode identical to the intra prediction mode for the luminance signal component is applied as the intra prediction mode for the chrominance signal component. In addition, CCLM may mean an intra prediction mode in which, in the process of generating a prediction block for a chrominance block, the reconstructed samples of the luminance block are downsampled, and then the samples extracted by applying the CCLM parameters α and β to the downsampled samples are used as predictive samples of the block of chrominance signals.

[384] Cross Component Linear Model (CCLM) Generic Mode Representation

[385] As described above, the CCLM mode may be applied to a block of chrominance signals. The CCLM mode is an intra prediction mode using correlation between a luma block and a chrominance block corresponding to the luminance block, and is performed by extracting a linear model based on neighboring luminance block samples and neighboring chrominance block samples. In addition, the chrominance block predictive sample may be derived based on the extracted linear model and the reconstructed luma block samples.

[386] In particular, when the CCLM mode is applied to the current chrominance block, parameters for a linear model may be derived based on adjacent samples used for intra prediction of the current chrominance block and neighboring samples used for intra prediction of the current luma block. For example, a linear model for CCLM may be represented based on an equation, as explained below.

[387] Equation 20

[388]

[389] Here, pred _c (i, j) may mean a predictive sample of coordinates (i, j) of the current chroma block in the current CU; rec _L '(i, j) may mean a reconstructed sample of coordinates (i, j) of the current luminance block in the CU. For example, rec _L '(i, j) may mean a downsampled reconstructed sample of the current luma block. The coefficients α and β of the linear model may be signaled or may be derived from a neighboring sample.

[390] Combined Residue Coding (Combined CbCr)

[391] In the encoding/decoding process according to the embodiment, the residuals of the chrominance signals may be encoded/decoded together. This may be referred to as "combined residue coding" or "combined CbCr". Whether or not to apply (activate) the CbCr joint coding mode may be signaled by the joint coding mode signaling flag tu_joint_cbcr_residual_flag, which is signaled at the transform basis layer. In addition, the selected coding mode may be extracted by the CBF of the chrominance signals. The tu_joint_cbcr_residual_flag may be present when the value of at least one chrominance CBF for the transform basis is 1. The chrominance QP offset value indicates the difference between the total chrominance QP offset value signaled for regular chrominance residual coding mode , and the QP offset value of the chrominance signals for the combined CbCr coding mode. The chrominance QP offset value may be signaled via the PPS or slice header. This QP offset value may be used to control the QP value of the chrominance signals for blocks using the combined chrominance residual coding mode.

[392] In the case where mode 2, which is the corresponding joint chrominance coding mode in the table below, is activated for a transform basis while quantization and decoding of the transform basis are performed, its chrominance QP offset may be added to the target derived from the luminance signals QP of the chrominance signals (used derived from the luminance signals QP of the chrominance signals).

[393] With respect to other modes such as modes 1 and 3 in the table below, the QP of chrominance signals may be extracted such that it is obtained for a common Cb or Cr block. Such a process for recovering chrominance residuals (resCb and resCr) from the transform block may be selected according to the table below. When the present mode is enabled, one single combined chrominance residual block (resJointC[x][y] in the table below) is signaled, and the resCb residual block for Cb and the resCr residual block for Cr can be extracted considering information such as tu_cbf_cb , tu_cbf_cr , and CSign, which is the sign value expanded in the slice header.

[394] In the coding equipment, the combined component of the chrominance signals can be extracted as follows. According to the joint encoding mode, resJointC{1,2} may be formed according to the following order. In the case of mode 2 (single residue with Cb=C, Cr=CSign*C recovery), the combined residue can be determined according to the equation below.

[395] Equation 21

[396] resJointC[x][y]=(resCb[x][y]+CSign*resCr[x][y])/2.

[397] Alternatively, in the case of mode 1 (single residue with Cb=C, Cr=(CSign*C)/2 reduction), the combined residue can be determined according to the equation below.

[398] Equation 22

[399] resJointC[x][y]=(4*resCb[x][y]+2*CSign*resCr[x][y])/5.

[400] Alternatively, in the case of mode 3 (single residue with Cr=C reduction, Cb=(CSign*C)/2), the combined residue can be determined according to the equation below.

[401] Equation 23

[402] resJointC[x][y]=(4*resCr[x][y]+2*CSign*resCb[x][y])/5.

[403] Tab. 2

tu_cbf_cb tu_cbf_cr Recovery of Cb and Cr residues Mode 1 0 resCb[x][y]=resJointC[x][y]
resCr[x][y]=(CSign*resJointC[x][y])>>1 1 1 1 resCb[x][y]=resJointC[x][y]
resCr[x][y]=CSign*resJointC[x][y] 2 0 1 resCb[x][y]=(CSign*resJointC[x][y])>>1
resCr[x][y]=resJointC[x][y] 3

[404] The above table shows the recovery of the residuals of the chrominance signals. CSign means the +1 or -1 sign value specified in the slice header; resJointC[][] means the remainder to be transferred. In the table, modes mean TuCResMode, which is described below. The three combined chrominance coding modes in the table can only be supported for I-slice. For P- and B-slices, only mode 2 can be supported. Therefore, for P- and B-slices, the tu_joint_cbcr_residual_flag syntax element can only be provided when both chrominance CBF values (eg, tu_cbf_cb and tu_cbf_cr) are equal to 1. Meanwhile, , in context modeling tu_cbf_luma and tu_cbf_cb, the transform depth can be removed.

[405] Option 1 implementation. QP refresh method using ACT Qp_offset

[406] As described above, updating the QP so as to apply ACT can be performed. The above QP update has several problems. For example, when the above method is used, it is not possible to set different ACT Qp offsets for individual color components. Additionally, the extracted qP value may have a negative value. Accordingly, in the following embodiment, a method for applying a cutoff to a Qp value obtained based on a color component value QP offset value ACT is described.

[407] In an embodiment, the quantization parameter qP may be extracted as follows.

[408] First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[409] Equation 24

[410] qP= _Qp'Y

[411] ActQpOffset=5

[412] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp-offset may be extracted as shown in the equation below.

[413] Equation 25

[414] qP= _Qp'CbCr

[415] ActQpOffset=5

[416] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset can be extracted as shown in the equation below.

[417] Equation 26

[418] qP=Qp' _Cb

[419] ActQpOffset=3

[420] The quantization parameter qP may be updated as follows.

[421] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be extracted as shown in the equation below.

[422] Equation 27

[423] qP=Max(0, qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[424] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be extracted as shown in the equation below.

[425] Equation 28

[426] qP=Max(0, Max(QpPrimeTsMin, qP)-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[427] In another embodiment, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be clipped using the value of QpPrimeTsMin instead of 0, as shown in the equation below.

[428] Equation 29

[429] qP=Max(QpPrimeTsMin, qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[430] Meanwhile, in another embodiment, the quantization parameter qP may be extracted as follows.

[431] First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[432] Equation 30

[433] qP=Qp' _Y

[434] ActQpOffset=5

[435] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be extracted as shown in the equation below.

[436] Equation 31

[437] qP= _Qp'CbCr

[438] ActQpOffset=5

[439] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset can be extracted as shown in the equation below.

[440] Equation 32

[441] qP=Qp' _Cb

[442] ActQpOffset=5

[443] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset can be extracted as shown in the equation below.

[444] Equation 33

[445] qP=Qp' _Cr

[446] ActQpOffset=3

[447] The quantization parameter qP may be updated as follows.

[448] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be extracted as shown in the equation below.

[449] Equation 34

[450] qP=Max(0, qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[451] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be extracted as shown in the equation below.

[452] Equation 35

[453] qP=Max(0, Max(QpPrimeTsMin, qP)-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[454] In another embodiment, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be clipped using the value of QpPrimeTsMin instead of 0, as shown in the equation below.

[455] Equation 36

[456] qP=Max(QpPrimeTsMin, qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[457] Meanwhile, in yet another embodiment, the quantization parameter qP may be extracted as follows.

[458] First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[459] Equation 37

[460] qP= _Qp'Y

[461] ActQpOffset=-5

[462] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be extracted as shown in the equation below.

[463] Equation 38

[464] qP= _Qp'CbCr

[465] ActQpOffset=-5

[466] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset can be extracted as shown in the equation below.

[467] Equation 39

[468] qP=Qp' _Cb

[469] ActQpOffset=-5

[470] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset can be extracted as shown in the equation below.

[471] Equation 40

[472] qP=Qp' _Cr

[473] ActQpOffset=-3

[474] The quantization parameter qP may be updated as follows.

[475] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be extracted as shown in the equation below.

[476] Equation 41

[477] qP=Max(0, qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[478] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be extracted as shown in the equation below.

[479] Equation 42

[480] qP=Max(0, Max(QpPrimeTsMin, qP)+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[481] Meanwhile, in another embodiment, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be clipped using the value of QpPrimeTsMin instead of 0, as shown in the equation below.

[482] Equation 43

[483] qP=Max(QpPrimeTsMin, qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[484] In the above description, Y, Cb and Cr may represent three color components. For example, in an ACT transform, Y may correspond to C0. Cb may correspond to C1 or Cg. In addition, Cr may correspond to C2 or Co.

[485] In addition, the values ACTQpOffset-5, -5 and -3 for the three color components can be replaced by other values or other variables.

[486] Option 2 implementation. Transmission in QP-offset control signaling for ACT

[487] In the above embodiment, the QP bias adjustments ACT are fixed at -5, -5, and -3 for the Y, Cg, and Co components. In the present embodiment, in order to provide greater flexibility for adjusting the QP offset ACT, a method for signaling the Qp offset ACT is described below. The ACT QP offset may be signaled as a parameter in the PPS.

[488] In an embodiment, qp_offset may be signaled according to the parse table of FIG. 36. Its syntactic elements are as follows.

[489] The pps_act_qp_offsets_present_flag syntax element may indicate whether or not a syntax element associated with the ACT QP offset is present in the PPS. For example, pps_act_qp_offsets_present_flag may indicate whether or not the pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset syntax elements, which are described below, are transmitted as PPS overhead.

[490] For example, the first value (eg, 0) of pps_act_qp_offsets_present_flag may indicate that pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset are not signaled via the PPS syntax table.

[491] The second value (eg, 1) of pps_act_qp_offsets_present_flag may indicate that pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset are signaled via the PPS syntax table.

[492] When pps_act_qp_offsets_present_flag is not provided from the bitstream, pps_act_qp_offsets_present_flag may be extracted as the first value (eg, 0). For example, when a flag (eg, sps_act_enabled_flag signaled to the SPS) indicating whether or not ACT is applied has a first value (eg, 0) indicating that ACT is not applied, pps_act_qp_offsets_present_flag may be forced to have a first value (eg. , 0).

[493] When the value of the cu_act_enabled_flag syntax element is a second value (eg, 1) indicating that ACT is applied to the current encoding basis, the syntax elements pps_act_y_qp_offset_plus5, pps_act_cb_qp_offset_plus5s and pps_act_cr_qp_offset_plus3 can be used to determine the offsets applied to the qP parameter values. quantization for the luminance signal component, the Cb and Cr components, respectively. When the values pps_act_y_qp_offset_plus5, pps_act_cb_qp_offset_plus5 and pps_act_cr_qp_offset_plus3 are not present in the bitstream, each value may be set to 0.

[494] According to the syntax elements, the value of the variable PpsActQpOffsetY may be determined equal to pps_act_y_qp_offset_plus5-5. The value of the PpsActQpOffsetCb variable may be defined to be pps_act_cb_qp_offset_plus5-5. In addition, the value of the variable PpsActQpOffsetCr may be defined to be pps_act_cb_qp_offset_plus3-3.

[495] Here, ACT is not an orthonormal transformation, so 5, 5, and 3 can be used as constant offset values to be subtracted. In an embodiment, to match bit streams, the values of PpsActQpOffsetY, PpsActQpOffsetCb, and PpsActQpOffsetCr may have values ranging from -12 to 12. In addition, according to an embodiment, in addition to 5, 5, and 3, the Qp offset values may be replaced by other constants values and be used.

[496] In another embodiment, QP may be adjusted using a more flexible ACT_QP offset. In the following embodiment, an example is described in which an ACT QP offset is signaled in a bitstream. Accordingly, the ACT QP offset may have a wider range of offset. Therefore, a QP updated using an ACT QP offset is more likely to be outside the available range, so that clipping for upper and lower limits for the updated QP needs to be performed (more detailed embodiments are described in Embodiments 6 and 7 below).

[497] Variables PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, and PpsActQpOffsetCbCr indicating ACT QP offsets may be values extracted using ACT QP offsets signaled via the bitstream or preset constants. To match the bitstream, PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr and PpsActQpOffsetCbCr can have values between -12 and +12.

[498] When not using a fixed value, the QP offset value is signaled and its value has a value between -12 and 12, it is necessary to clip the upper limit value of the extracted QP value, in addition to clipping the lower limit value of the QP value , extracted in order to prevent QP having a negative value.

[499] In order to prevent a negative value for the qP value, the smallest qP value may be forced to be 0. Alternatively, the smallest qP value may be set to a value determined by the signaling syntax element. For example, to signal the smallest value of qP when the skip transform mode is applied, the QpPrimeTsMin syntax element can be used indicating the value of qP to apply when the skip transform mode is applied. The maximum value of qP may be limited to an available maximum value (eg, 63) qP or a maximum available value of qP determined according to a signaling syntax element.

[500] In the embodiment according to the above, the quantization parameter qP can be extracted as follows. First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[501] Equation 44

[502] qP=Qp' _Y

[503] ActQpOffset=PpsActQpOffsetY

[504] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be extracted as shown in the equation below.

[505] Equation 45

[506] qP= _Qp'CbCr

[507] ActQpOffset=PpsActQpOffsetCbCr

[508] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset can be extracted as shown in the equation below.

[509] Equation 46

[510] qP=Qp' _Cb

[511] ActQpOffset=PpsActQpOffsetCb

[512] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset can be extracted as shown in the equation below.

[513] Equation 47

[514] qP=Qp' _Cr

[515] ActQpOffset=PpsActQpOffsetCr

[516] In an embodiment, the quantization parameter qP may be updated as follows. When transform_skip_flag[xTbY][yTbY][cIdx] is set to 0, qP can be extracted as shown in the equation below.

[517] Equation 48

[518] qP=Clip3(0, 63, qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[519] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be extracted as shown in the equation below.

[520] Equation 49

[521] qP=Clip3(0, 63, Max(QpPrimeTsMin, qP)-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[522] In another embodiment, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, the smallest qP value may be clipped using the QpPrimeTsMin value instead of 0, as shown in the equation below.

[523] Equation 50

[524] The quantization parameter qP may be updated as follows.

[525] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be extracted as shown in the equation below.

[526] qP=Clip3(0, 63, qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[527] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be extracted as shown in the equation below.

[528] qP=Clip3(QpPrimeTsMin, 63, qP-cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[529] In another embodiment, the quantization parameter qP may be updated as follows.

[530] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be extracted as shown in the equation below.

[531] Equation 51

[532] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[533] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be extracted as shown in the equation below.

[534] Equation 52

[535] qP=Clip3(0, 63+QpBdOffset, Max(QpPrimeTsMin, qP)+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[536] In another embodiment, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, the smallest qP value may be clipped using the QpPrimeTsMin value instead of 0, as shown in the equation below.

[537] Equation 53

[538] The quantization parameter qP may be updated as follows.

[539] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be extracted as shown in the equation below.

[540] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[541] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be extracted as shown in the equation below.

[542] qP=Clip3(QpPrimeTsMin, 63+QpBdOffset, qP+cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[543] Option 3 implementation. Method for enabling ACT when chrominance BDPCM is performed

[544] In an embodiment, when BDPCM is applied to a block of luma components, ACT may be applied to encode/decode the block. However, when BDPCM is applied to a chrominance component block, ACT is limited in that it is applied to encode/decode the block.

[545] Meanwhile, even when BDPCM is applied to a block of chrominance signal components, ACT is applied to the block, thereby improving coding efficiency. Fig. 37 shows an embodiment of a syntax configuration for applying ACT even when BDPCM is applied to a chrominance signal component block. As shown in FIG. 37, by removing the condition for obtaining a BDPCM syntax element for a chrominance signal component according to a value of cu_act_enabled_flag indicating whether or not ACT is applied to the current coding base, a BDPCM syntax element for it can be obtained regardless of whether or not ACT is applied to the component block. chrominance signals, and BDCPM encoding can be performed accordingly.

[546] Option 4 implementation. How to apply ACT even when encoding/decoding is done using CCLM

[547] CCLM and ACT have the intention of removing unnecessary overlap between components. There are some overlapping parts between CCLM and ACT, but even after applying all of them, it is not possible to completely remove the overlap between components. Therefore, overlap between components can be removed to a greater extent by applying CCLM and ACT together.

[548] The following embodiment describes an embodiment in which CCLM and ACT are applied together. When performing decoding, the decoding equipment may apply CCLM first and apply ACT. When ACT is applied to both BDPCM and CCLM for the chrominance signaling component, the syntax table to convey this in signaling may be modified as shown in FIG. 38. Accordingly, as shown in the parse table of FIG. 38, of the restrictions on syntax element signaling associated with intra_bdpcm_chroma and cclm, if(!cu_act_enabled_flag) for syntax element signaling depending on whether or not ACT is applied may be removed from the syntax table.

[549] Option 5 implementation. Method for using flexible ACT Qp including pooled CbCr

[550] When the ACT mode is applied, the prediction residual may be converted from one color space (eg, GBR- or YCbCr-) to a YCgCo color space. In addition, the transformation basis residues can be encoded in the YCgCo color space. As an embodiment of the basic ACT transform (transformation kernel) used for color space transformation, the following transformation kernel can be used as described above.

[551] Equation 54

[552]

[553] Equation 55

[554]

[555] As described in the above equations, C0', C1' and C2' (here, C0'=Y, C1'=Cg, C2'=Co) transformations are not normalized. For example, the L2-norm does not have a value of 1. For example, the L2-norm of a transformation for an individual component may have a value of approximately 0.6 for C0' and C1', and may have a value of approximately 0.7 for C2'. In this document, the L2-norm is the value obtained as the square root of the sum of the squares of the respective coefficients. For example, C0'=2/4*C0+1/4*C1+1/4*C2 can be calculated. Therefore, the C0' norm can be calculated as the square root of (2/4*2/4+1/4*1/4+1/4*1/4). Therefore, it can be calculated as the square root of 6/16 and can be calculated as having a value of approximately 0.6.

[556] When the normalized transform is not applied, the dynamic range of an individual component is irregular. In addition, it causes a decrease in the encoding performance of the overall video compression system.

[557] In order to compensate for the dynamic range of the residual signal, a QP offset value for compensating for dynamic range change for an individual transform component is transmitted so that QP adjustment can be performed. For example, this embodiment can be applied to a general control method for QP regulation for ACT conversion as well as combined CbCr.

[558] The individual color components are not encoded independently, but together, so that the method described above in Embodiment 3 for combined CbCr can cause the dynamic range to change between the individual color components.

[559] In the encoding and decoding methods according to the embodiment, the QP offset adjustment ACT may be fixed at -5, which may equally apply to Y, Cg and Co.

[560] In an embodiment, to provide flexible Qp control for individual components and jointCbCr, different ACT Qp offsets for Y, Cb, Cr, and/or combined CbCr may be allowed. ACT Qp offset values may be determined based on component index and/or combined CbCr and/or combined CbCr mode.

[561] To indicate ACT Qp offsets, ppsActQpOffsetY, ppsActQpOffsetCb, and ppsActQpOffsetCr can be used. In addition, ppsActQpOffsetCbCr may be used to ACT QP offset the combined CbCr mode 2 having a CBF in which all Cb and Cr components are non-zero. These values (eg, ppsActQpOffsetY, ppsActQpOffsetCb, ppsActQpOffsetCr, and ppsActQpOffsetCbCr) may be predefined as predefined values or signaled via a bit stream. The ACT QP offset of the combined CbCr mode may be set in another manner or to another value.

[562] In an embodiment, ACT Qp offsets of -5, -5, and -3 may be used for Y, Cb, and Cr, and ACT Qp offsets of -4 may be used for pooled CbCr.

[563] In another embodiment, ACT Qp-offsets of -5, -4, and -3 may be used for Y, Cb, and Cr, and ACT Qp-offsets of -3 may be used for the combined CbCr mode, in which the value of tu_cbf_cb not equal to 0.

[564] In yet another embodiment, the ACT QP offset of the combined CbCr mode 2 may have its own offset value. For another combined CbCr mode, the ACT QP offset may use the offset of the corresponding component. For example, the quantization parameter qP may be defined as follows. First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[565] Equation 56

[566] qP= _Qp'Y

[567] ActQpOffset=ppsActQpOffsetY

[568] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be extracted as shown in the equation below.

[569] Equation 57

[570] qP= _Qp'CbCr

[571] ActQpOffset=ppsActQpOffsetCbCr

[572] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset can be extracted as shown in the equation below.

[573] Equation 58

[574] qP=Qp' _Cb

[575] ActQpOffset=ppsActQpOffsetCb

[576] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset can be extracted as shown in the equation below.

[577] Equation 59

[578] qP=Qp' _Cr

[579] ActQpOffset=ppsActQpOffsetCr

[580] In an embodiment, the quantization parameter qP may be updated as follows.

[581] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be extracted as shown in the equation below.

[582] Equation 60

[583] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[584] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be extracted as shown in the equation below.

[585] Equation 61

[586] qP=Clip3(QpPrimeTsMin, 63+QpBdOffset, qP+cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[587] In another embodiment, relative to the combined CbCr mode (eg, in the case of modes 1 and 2) with tu_cbf_cb!=0, the offset for the combined CbCr may be determined using ppsActQpOffsetCb. Alternatively, with respect to the combined CbCr mode (eg, in the case of mode 3) with tu_cbf_cb==0, the offset for the combined CbCr may be determined using ppsActQpOffsetCr. For example, the above-described embodiment may be modified and applied as follows.

[588] The quantization parameter qP may be updated as follows. First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[589] Equation 62

[590] qP= _Qp'Y

[591] ActQpOffset=ppsActQpOffsetY

[592] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP may be derived as shown in the equation below.

[593] Equation 63

[594] qP= _Qp'CbCr

[595] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset can be extracted as shown in the equation below.

[596] Equation 64

[597] qP=Qp' _Cb

[598] ActQpOffset=ppsActQpOffsetCb

[599] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset can be extracted as shown in the equation below.

[600] Equation 65

[601] qP=Qp' _Cr

[602] ActQpOffset=ppsActQpOffsetCr

[603] When cIdx is not set to 0 and TuCResMode[xTbY][yTbY] is not set to 0, the ACT Qp offset for the combined CbCr mode may be determined according to the following pseudocode.

[604] Equation 66

[605] if(TuCResMode[xTbY][yTbY] is 1 or 2)

[606] ActQpOffset=ppsActQpOffsetCb;

[607] else

[608] ActQpOffset=ppsActQpOffsetCr;

[609] In an embodiment, the quantization parameter qP may be updated as follows. When transform_skip_flag[xTbY][yTbY][cIdx] is set to 0, qP can be extracted as shown in the equation below.

[610] Equation 67

[611] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[612] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be extracted as shown in the equation below.

[613] Equation 68

[614] qP=Clip3(QpPrimeTsMin, 63+QpBdOffset, qP+cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[615] In another embodiment, regardless of the combined CbCr mode, ppsActQpOffsetY is used when the component index is Y, ppsActQpOffsetCb is used when the component index is Cb, and ppsActQpOffsetCr is used when the component index is Cr, whereby extract qP. For example, the quantization parameter qP may be extracted as follows.

[616] First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[617] Equation 69

[618] qP= _Qp'Y

[619] ActQpOffset=ppsActQpOffsetY

[620] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp-offset may be extracted as shown in the equation below.

[621] Equation 70

[622] qP= _Qp'CbCr

[623] ActQpOffset=(cIdx==1)?ppsActQpOffsetCb:ppsActQpOffsetCr

[624] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset can be extracted as shown in the equation below.

[625] Equation 71

[626] qP=Qp' _Cb

[627] ActQpOffset=ppsActQpOffsetCb

[628] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset can be extracted as shown in the equation below.

[629] Equation 72

[630] qP=Qp' _Cr

[631] ActQpOffset=ppsActQpOffsetCr

[632] The quantization parameter qP may be updated as follows.

[633] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be extracted as shown in the equation below.

[634] Equation 73

[635] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[636] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be extracted as shown in the equation below.

[637] Equation 74

[638] qP=Clip3(QpPrimeTsMin, 63+QpBdOffset, qP+cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[639] Option 6 implementation. ACT Signaling Method of Qp Offset Including Combined CbCr

[640] Later in this document, an example is described in which the ACT QP offset is signaled via a bitstream to provide greater flexibility. ACT QP offsets may be signaled via SPS, PPS, Frame Header, Slice Header, or sets of other types of headers. The ACT Qp offset of the combined CbCr may be signaled separately or may be derived from the ACT QP offsets for Y, Cb and Cr.

[641] Without loss of generality, FIG. 39 shows an example of a syntax table in which the ACT Qp offset is signaled in the PPS. Similar to the embodiment of FIG. 39, one ACT Qp offset may be signaled for the combined CbCr. The following describes the syntax elements indicated in the syntax table of FIG. 39.

[642] The pps_act_qp_offsets_present_flag syntax element may indicate whether or not a syntax element associated with the ACT QP offset is present in the PPS. For example, pps_act_qp_offsets_present_flag may indicate whether or not the syntax elements pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4, which are described below, are sent in signaling as PPS .

[643] For example, the first value (eg, 0) of pps_act_qp_offsets_present_flag may indicate that pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4 are not signaled over c syntax PPS table.

[644] The second value (eg, 1) pps_act_qp_offsets_present_flag may indicate that pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4 are signaled via the syntax PPS table.

[645] When pps_act_qp_offsets_present_flag is not provided from the bitstream, pps_act_qp_offsets_present_flag may be extracted as the first value (eg, 0). For example, when a flag (eg, sps_act_enabled_flag signaled to the SPS) indicating whether or not ACT is applied has a first value (eg, 0) indicating that ACT is not applied, pps_act_qp_offsets_present_flag may be forced to have a first value (eg. , 0).

[646] When the value of the cu_act_enabled_flag syntax element is a second value (eg, 1) indicating that ACT is applied to the current encoding basis, the syntax elements pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_off set_plusX4 can be used to define offsets applied to qP values quantization parameters for the luma component, the Cb component, the Cr component, and the combined CbCr component, respectively. When the values pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3 and pps_act_cbcr_qp_offset_plusX4 are not present in the bitstream, each value may be set to 0.

[647] According to the syntax elements, the values of the variables PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr and PpsActQpOffsetCbCr may be determined as shown in the equation below.

[648] Equation 75

[649] PpsActQpOffsetY=pps_act_y_qp_offset_plusX1-X1

[650] PpsActQpOffsetCb=pps_act_cb_qp_offset_plusX2-X2

[651] PpsActQpOffsetCr=pps_act_cr_qp_offset_plusX3-X3

[652] PpsActQpOffsetCbCr=pps_act_cbcr_qp_offset_plusX4-X4

[653] Herein, X1, X2, X3, and X4 may indicate predefined constant values. They may be identical values, or different values, or only some may have an identical value.

[654] In an embodiment, for bitstream matching, the values of PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, and PpsActQpOffsetCbCr may be limited such that they have values ranging from -12 to 12.

[655] According to the definition of variables, the quantization parameter qP can be determined as follows. First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[656] Equation 76

[657] qP=Qp' _Y

[658] ActQpOffset=PpsActQpOffsetY

[659] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be extracted as shown in the equation below.

[660] Equation 77

[661] qP= _Qp'CbCr

[662] ActQpOffset=PpsActQpOffsetCbCr

[663] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset can be extracted as shown in the equation below.

[664] Equation 78

[665] qP=Qp' _Cb

[666] ActQpOffset=PpsActQpOffsetCb

[667] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset can be extracted as shown in the equation below.

[668] Equation 79

[669] qP=Qp' _Cr

[670] ActQpOffset=PpsActQpOffsetCr

[671] In another embodiment of Qp-offset ACT signaling, a plurality of Qp-offset ACTs may be signaled for various combined CbCr modes identified as Mode A and Mode B.

[672] The combined CbCr mode A may mean a combined CbCr mode having tu_cbf_cb having a non-zero value, for example, mode 1 and mode 2 of table 2 described above. Joint CbCr mode B may mean jointCbCrmode having tu_cbf_cb having a value of 0, for example, mode 3 of table 2 described above. The syntax table modified accordingly is shown in FIG. 40. The following describes the syntax elements indicated in the syntax table of FIG. 40.

[673] When the value of the cu_act_enabled_flag syntax element is a second value (eg, 1) indicating that ACT is applied to the current encoding basis, the syntax elements pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, pps_act_cbcr_qp_off set_modeA_plusX4 and pps_act_cbcr_qp_offset_modeB_plusX5 can be used to determine the offsets applied to the qP values of the quantization parameters for the luma component, the Cb component, the Cr component, and the combined CbCr component, respectively. When the values pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, pps_act_cbcr_qp_offset_modeA_plusX4, and pps_act_cbcr_qp_offset_modeB_plusX5 are not present in the bitstream, each value may be set to 0.

[674] According to the syntax elements, the values of the variables PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, PpsActQpOffsetCbCrModeA, and PpsActQpOffsetCbCrModeB may be determined as shown in the equation below.

[675] Equation 80

[676] PpsActQpOffsetY=pps_act_y_qp_offset_plusX1-X1

[677] PpsActQpOffsetCb=pps_act_cb_qp_offset_plusX2-X2

[678] PpsActQpOffsetCr=pps_act_cr_qp_offset_plusX3-X3

[679] PpsActQpOffsetCbCrModeA=pps_act_cbcr_qp_offset_modeA_plusX4-X4

[680] PpsActQpOffsetCbCrModeB=pps_act_cbcr_qp_offset_modeB_plusX5-X5

[681] Here, X1, X2, X3, X4, and X5 may indicate predefined constant values. They may be identical values, or different values, or only some may have an identical value. In an embodiment, for bitstream matching, the values of PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, PpsActQpOffsetCbCrModeA, and PpsActQpOffsetCbCrModeB may be limited such that they have values between -12 and 12.

[682] According to the definition of variables, the quantization parameter qP can be determined as follows. First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[683] Equation 81

[684] qP=Qp' _Y

[685] ActQpOffset=PpsActQpOffsetY

[686] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP may be derived as shown in the equation below.

[687] Equation 82

[688] qP=Qp' _CbCr

[689] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset can be extracted as shown in the equation below.

[690] Equation 83

[691] qP=Qp' _Cb

[692] ActQpOffset=PpsActQpOffsetCb

[693] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset can be extracted as shown in the equation below.

[694] Equation 84

[695] qP=Qp' _Cr

[696] ActQpOffset=PpsActQpOffsetCr

[697] In addition, when cIdx is not set to 0 and TuCResMode[xTbY][yTbY] is not set to 0, the ACT Qp offset may be extracted as shown in the equation below.

[698] Equation 85

[699] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])?PpsActQpOffsetCbCrModeA:PpsActQpOffsetCbCrModeB

[700] Meanwhile, in another embodiment, when TuCResMode[xTbY][yTbY] has a value of 2, ActQpOffset may be extracted as shown in the equation below.

[701] Equation 86

[702] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])?(PPsQpOffsetCbCrModeA+slice_act_CbCr_qp_offset_ModeA):(PPsQpOffsetCbCrModeB+slice_act_CbCr_qp_offset_ModeB)

[703] In another embodiment of signaling the Qp offset ACT, only the Qp offset ACTs for Y, Cb, and Cr may be signaled as indicated in the parse table of FIG. 41. The ACT QP offset for the pooled CbCr may be derived from PpsActQpOffsetY, PpsActQpOffsetCb and/or PpsActQpOffsetCr.

[704] In an embodiment, the ACT Qp offset for CbCr may be set to a value of PpsActQpOffsetCb. In another embodiment, the ACT Qp offset for CbCr may be set to the same value as PpsActQpOffsetCb in the case of combined CbCr mode, in which tu_cbf_cb has a non-zero value, or may be set to the same value as PpsActQpOffsetCr in the case of combined CbCr mode, in where tu_cbf_cb has a value of 0. Alternatively, it may be set in reverse.

[705] FIG. 41 is a view showing another embodiment of the syntax table in which the ACT Qp offset is signaled in the PPS. According to the definition of syntax elements in FIG. 41, the quantization parameter qP may be determined as follows. First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[706] Equation 87

[707] qP= _Qp'Y

[708] ActQpOffset=PpsActQpOffsetY

[709] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp-offset may be extracted as shown in the equation below.

[710] Equation 88

[711] qP= _Qp'CbCr

[712] ActQpOffset=(cIdx==1)?PpsActQpOffsetCb:PpsActQpOffsetCr

[713] Meanwhile, in another embodiment, the ActQpOffset value may be determined as follows.

[714] Equation 89

[715] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])?PpsActQpOffsetCb:PpsActQpOffsetCr

[716] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset can be extracted as shown in the equation below.

[717] Equation 90

[718] qP=Qp' _Cb

[719] ActQpOffset=PpsActQpOffsetCb

[720] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset can be extracted as shown in the equation below.

[721] Equation 91

[722] qP=Qp' _Cr

[723] ActQpOffset=PpsActQpOffsetCr

[724] Option 7 implementation. ACT Qp-offset signaling at multiple levels

[725] In an embodiment, the ACT QP offset may be signaled in multiple layers. In addition to signaling the ACT Qp offset at one layer such as PPS, similar to the above embodiment 6, the ACT QP offset may be signaled at the lower layer (e.g., slice header, frame header, or other types of headers, suitable for Qp control).

[726] Hereinafter, two embodiments are described in this document. Fig. 42 and 43 show examples in which the ACT QP offset is signaled via a slice header and a frame header. Thus, the ACT QP offset can be signaled at several levels.

[727] Hereinafter, the syntax elements shown in FIG. 42 and 43. The syntax element pps_slice_act_qp_offsets_present_flag may indicate whether or not the syntax elements slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset, which are described below, are present in the slice header.

[728] For example, the first value (eg, 0) of pps_slice_act_qp_offsets_present_flag may indicate that slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset are not present in the slice header.

[729] For example, the second value (eg, 1) pps_slice_act_qp_offsets_present_flag may indicate that slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset are present in the slice header.

[730] The slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset syntax elements may indicate offsets for the qP values of the quantization parameters for the luma component, Cb component, Cr component, and combined CbCr component, respectively. The values of slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset can be limited such that they have values between -12 and 12. When slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_ are qp_offset are not present in the bitstream, each value can be set to 0. PpsActQpOffsetY values +slice_act_y_qp_offset, PpsActQpOffsetCb+slice_act_cb_qp_offset, PpsActQpOffsetCr+slice_act_cr_qp_offset and PpsActQpOffsetCbCr+slice_act_cbcr_qp_offset can be limited such that they also have values between -12 and 12.

[731] Various modified embodiments of the ACT signaling of the Qp offset for the combined CbCr at the PPS level may be applied. For example, one QP offset can be signaled for pooled CbCr, multiple ACT Qp offsets can be signaled for pooled CbCr of different modes, or no Qp offset for pooled CbCr is signaled in ACT, a way to extract this by using ACTQpOffsets for Y, Cb and Cr and/or jointCbCr mode may be used when signaling is performed via slice header.

[732] Two modified embodiments are shown in FIG. 44 and 45. FIG. 44 shows an embodiment in which the ACT Qp offset is signaled in a slice header. Fig. 45 shows another embodiment in which the ACT Qp offset is signaled in the slice header. In FIG. 45, only ACT Qp offsets for Y, Cb, and Cr may be signaled, and slice level ACT QP offset for pooled CbCr may be derived from slice_act_y_qp_offset, slice_act_cb_qp_offset, and/or slice_act_cr_qp_offset. This may be determined based on the mode type jointCbCr. In an embodiment, the slice level ACT Qp offset for CbCr may be set to the same value as slice_act_cb_qp_offset. In another embodiment, in the case of a pooled CbCr mode having tu_cbf_cb having a non-zero value, the slice level ACT Qp offset for pooled CbCr may be set to a value identical to slice_act_cb_qp_offset. In addition, in the case of the combined CbCr mode having tu_cbf_cb set to 0, the slice level ACT Qp offset for the combined CbCr may be set to the same value as slice_act_cr_qp_offset.

[733] Meanwhile, in another embodiment, the syntax element may be signaled in a slice header or a frame header. To implement this, encoding/decoding may be performed as follows.

[734] - Flag pps_picture_slice_act_qp_offsets_present_flag indicating whether or not an ACT Qp offset is present in the frame header or slice header may be signaled in the PPS.

[735] - When ACT is applicable and the value of pps_picture_slice_act_qp_offsets_present_flag is the second value (eg, 1), a pic_act_qp_offsets_present_flag flag indicating whether or not an ACT Qp offset is present in the frame header is signaled in the frame header. Herein, a second value (eg, 1) of pic_act_qp_offsets_present_flag may indicate that ACT Qp offsets for all slices of a frame corresponding to a frame header are provided in the frame header.

[736] - The first value (eg, 0) of pic_act_qp_offsets_present_flag may indicate that ACT Qp offsets for all frame slices corresponding to the frame header are not provided in the frame header. For example, when ACT is applicable and the value of pps_picture_slice_act_qp_offsets_present_flag is the second value (eg, 1) and the value of pic_act_qp_offsets_present_flag is the first value (eg, 0), the ACT Qp offset for the slice may be provided in the slice header.

[737] FIG. 46 is a view showing a PPS syntax table in which pps_pic_slice_act_qp_offsets_present_flag is signaled. The pps_pic_slice_act_qp_offsets_present_flag syntax element may indicate whether or not an ACT Qp offset is provided in the frame header and/or slice header. For example, the first value (eg, 0) of pps_pic_slice_act_qp_offsets_present_flag may indicate that the ACT Qp offset is not provided in the frame header and slice header. The second value (eg, 1) pps_pic_slice_act_qp_offsets_present_flag may indicate that the ACT Qp offset is provided in a frame header or a slice header. When pps_pic_slice_act_qp_offsets_present_flag is not provided in the bitstream, the value of pps_pic_slice_act_qp_offsets_present_flag may be determined to be the first value (eg, 0).

[738] FIG. 47 is a view showing a syntax table of a frame header to be transmitted in Qp-offset signaling ACT. The pic_act_qp_offsets_present_flag syntax element may indicate whether or not an ACT Qp offset is provided in the frame header. The first value (eg, 0) of pic_act_qp_offsets_present_flag may indicate that the ACT Qp offset is not provided in the frame header, but in the slice header. The second value (eg, 1) pic_act_qp_offsets_present_flag may indicate that the ACT Qp offset is provided in the frame header. When the value of pic_act_qp_offsets_present_flag is not provided in the bitstream, the value may be determined to be 0.

[739] FIG. 48 is a view showing a slice header syntax table for signaling ACT of a Qp offset. In the syntax table of FIG. 48, the slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset syntax elements may indicate offsets for qP values of quantization parameters for the luma component, Cb component, and Cr component. The values slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset can have values between -12 and 12. In addition, the values PpsActQpOffsetY+slice_act_y_qp_offset, PpsActQpOffsetCb+slice_act_cb_qp_offset, and PpsA ctQpOffsetCr+slice_act_cr_qp_offset can be limited such that they also have a value range of -12 to 12.

[740] Meanwhile, in the case where slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset values are not provided in the bitstream, when the value of pps_pic_slice_act_qp_offsets_present_flag is the first value (for example, 0), the values of slice_act_y_qp_offset, slice_act_ cb_qp_offset and slice_act_cr_qp_offset may be set to 0. Alternatively, when the value of pps_pic_slice_act_qp_offsets_present_flag is a second value (eg, 1), the values of slice_act_y_qp_offset, slice_act_cb_qp_offset, and slice_act_cr_qp_offset may be defined to be identical to the values of pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_ offset, respectively.

[741] Meanwhile, when an ACT Qp offset is present in a slice header and a frame header, the final offset value used to extract the qP value may be determined as the value that is obtained by summing the offset value transmitted in signaling in the PPS, and the offset value signaled in the slice header or frame header.

[742] More specifically, in an embodiment, the quantization parameter qP may be determined as follows. First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[743] Equation 92

[744] qP=Qp' _Y

[745] ActQpOffset=PPsQpOffsetY+slice_act_y_qp_offset

[746] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be extracted as shown in the equation below.

[747] Equation 93

[748] qP= _Qp'CbCr

[749] ActQpOffset=PPsQpOffsetCbCr+slice_act_CbCr_qp_offset

[750] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset can be extracted as shown in the equation below.

[751] Equation 94

[752] qP=Qp' _Cb

[753] ActQpOffset=PpsActQpOffsetCb+slice_act_Cb_qp_offset

[754] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset can be extracted as shown in the equation below.

[755] Equation 95

[756] qP=Qp' _Cr

[757] ActQpOffset=PpsActQpOffsetCr+slice_act_Cr_qp_offset

[758] In another embodiment, when multiple ACT Qp offsets for the combined CbCr are signaled, the ActQpOffset for the combined CbCr may be determined as follows.

[759] First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[760] Equation 96

[761] qP= _Qp'Y

[762] ActQpOffset=PPsQpOffsetY+slice_act_y_qp_offset

[763] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP may be derived as shown in the equation below.

[764] Equation 97

[765] qP= _Qp'CbCr

[766] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset can be extracted as shown in the equation below.

[767] Equation 98

[768] qP=Qp' _Cb

[769] ActQpOffset=PpsActQpOffsetCb+slice_act_Cb_qp_offset

[770] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset can be extracted as shown in the equation below.

[771] Equation 99

[772] qP=Qp' _Cr

[773] ActQpOffset=PpsActQpOffsetCr+slice_act_Cr_qp_offset

[774] In addition, when cIdx is not set to 0 and TuCResMode[xTbY][yTbY] is not set to 0, the ACT Qp offset may be extracted as shown in the equation below.

[775] Equation 100

[776] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])?(PPsQpOffsetCbCrModeA+slice_act_CbCr_qp_offset_ModeA):(PPsQpOffsetCbCrModeB+slice_act_CbCr_qp_offset_ModeB)

[777] In yet another embodiment, when the ACT Qp offset for the pooled CbCr is not provided, the qP and ActQpOffset for the Y, Cb, and/or Cr components are determined, and the ActQpOffset for the pooled CbCr may be determined using the ACT Qp- displacements of Y-, Cb- and/or Cr-components as follows. For example, in the above embodiment, when TuCResMode[xTbY][yTbY] associated with Equation 97 has a value of 2, the qP calculation step may be changed and performed as follows.

[778] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be extracted as shown in the equation below.

[779] Equation 101

[780] qP= _Qp'CbCr

[781] ActQpOffset=(cIdx==1])?(PPsQpOffsetCb+slice_act_Cb_qp_offset):(PPsQpOffsetCr+slice_act_Cr_qp_offset)"

[782] Meanwhile, in another embodiment, the ActQpOffset value may be determined as indicated in the equation below.

[783] Equation 102

[784] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])?(PPsQpOffsetCb+slice_act_Cb_qp_offset):(PPsQpOffsetCr+slice_act_Cr_qp_offset)

[785] Option 8 implementation. Method for signaling multiple sets of ACT Qp offsets

[786] In the present embodiment, the method of using the list of ACT Qp offsets is described below. To this end, the following processing may be performed.

[787] a) Multiple sets of ACT Qp offsets may be signaled in the form of a list in a parameter set (eg, SPS or PPS). Each set in the list may include ACT Qp offsets for Y-, Cb-, Cr- and combined CbCr components. For simplicity, the Qp-offset list ACT may be signaled in a parameter set identical to the parameter set for chrominance Qp-offset list signaling.

[788] b) The number of ACT Qp offset sets in the list may be identical to the number of chrominance Qp offset sets signaled in the PPS.

[789] c) As an ACT Qp-offset used to derive qP for each coding basis, an ACT Qp-offset belonging to a list having an index (eg, cu_chroma_qp_offset_idx) for the Qp-offset of the chrominance signals for the coding basis may be used. .

[790] d) As an alternative implementation of b) and c), the following may be performed.

[791] - The number of sets of ACT Qp offsets in the list may be signaled. The number of ACT Qp-offset sets in the list may be different from the number of chrominance Qp-offset sets.

[792] - When the ACT is applicable, an index indicating the ACT index of the Qp offset used for the coding basis may be signaled.

[793] Without departing from the above concept, a syntax for signaling a list of ACT Qp offsets may be used as shown in FIG. 49. For example, when cu_act_act_enabled_flag matters in 1, pps_act_y_qp_offseet, pps_act_cb_qp_offseet, pps_act_cr_qp_offseet, and pps_act_act_act_act_act_act_act_act_act_act_act_act_act_act_act_act_act_act_act_act_cbcr_qp_offse QP quanties for the component of brightness signals, CB component, CR component and united CbCr, respectively.

[794] When the values pps_act_y_qp_offset, pps_act_cb_qp_offset, pps_act_cr_qp_offset, and pps_act_cbcr_qp_offset are not present, each value may be retrieved as 0.

[795] When the value of cu_act_enabled_flag is a second value (eg, 1) and the value of cu_chroma_qp_offset_flag is a second value (eg, 1), act_y_qp_offset_list[i], act_cb_qp_offset_list[i], act_cr_qp_offset_list[i] and act_cbcr_qp_offset_list[i] may be used to to determine the offsets applied to the value of the quantization parameters qP for the luma component, the Cb component, the Cr component, and the combined CbCr component, respectively. When the values act_y_qp_offset_list[i], act_cb_qp_offset_list[i], act_cr_qp_offset_list[i], and act_cbcr_qp_offset_list[i] are not present, each value can be retrieved as 0.

[796] In the present embodiment, the quantization parameter qP may be determined as follows. First, when cIdx has a value of 0, qP and ACT Qp offset can be extracted as shown in the equation below.

[797] Equation 103

[798] qP=Qp' _Y

[799] ActQpOffset=pps_act_y_qp_offset+(cu_chroma_qp_offset_flag)?act_y_qp_offset_list[cu_chroma_qp_offset_idx]:0+slice_act_y_qp_offset. Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be extracted as shown in the equation below.

[800] Equation 104

[801] qP= _Qp'CbCr

[802] ActQpOffset=pps_act_cbcr_qp_offset+(cu_chroma_qp_offset_flag)?act_cbcr_qp_offset_list[cu_chroma_qp_offset_idx]:0+slice_act_cbcr_qp_offset

[803] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset can be extracted as shown in the equation below.

[804] Equation 105

[805] qP=Qp' _Cb

[806] ActQpOffset=pps_act_cb_qp_offset+(cu_chroma_qp_offset_flag)?act_cb_qp_offset_list[cu_chroma_qp_offset_idx]:0+slice_act_cb_qp_offset. Alternatively, when cIdx has a value of 2, qP and ACT Qp-offset may be extracted as shown in the equation below.

[807] Equation 106

[808] qP=Qp' _Cr

[809] ActQpOffset=pps_act_cr_qp_offset+(cu_chroma_qp_offset_flag)?act_cr_qp_offset_list[cu_chroma_qp_offset_idx]:0+slice_act_cr_qp_offset

Option 9 implementation. Color space ACT conversion method to be applied to lossless coding and lossy coding

[810] Transformation (transformation) between color spaces based on the matrixes for direct transformation and inverse transformation described above can be organized as follows.

[811] Tab. 3

Direct conversion: GBR to YCgCo Reverse conversion: YCgCo to GBR Y=((G<<1)+R+B+2)>>2
Cg=((G<<1)-R-B+2)>>2
Co=((RB)<<1)+2)>>2 t=Y-Cg
G=Y+Cg
B=t-Co=Y-Cg-Co
R=t+Co=Y-Cg+Co

[812] The conversion cannot achieve reset because the loss of some values occurs in the processing of Co and Cg. For example, when the sampled RGB color space value is converted to the YCgCo color space sampled value, and the resulting value is converted back to the resulting RGB color space value, the original sampled value is not completely restored. Therefore, the transformation according to Table 3 cannot be used for lossless coding. It is necessary to improve the color space conversion algorithm such that sample value loss does not occur after color space conversion even when lossless encoding is applied. Embodiments 9 and 10 describe a color space conversion algorithm capable of being applied to lossless coding as well as lossy coding.

[813] In the embodiment described below, a method for performing ACT by using a reversible color space transform, which is applicable to lossy coding as well as lossless coding, is described. This reversible color space transformation can be applied to the encoding and decoding methods described above. ACT Qp offset can also be adjusted for the following color space conversion. The color space conversion according to the embodiment may be performed as shown in the equation below. For example, direct conversion from GBR color space to YCgCo color space can be performed according to the equation below.

[814] Equation 107

[815] Co=R-B;

[816] t=B+(Co>>1);

[817] Cg=G-t;

[818] Y=t+(Cg>>1);

[819] In addition, the inverse transformation from the YCgCo color space to the GBR color space can be performed according to the equation below.

[820] Equation 108

[821] t=Y-(Cg>>1)

[822] G=Cg+t

[823] B=t-(Co>>1)

[824] R=Co+B

[825] The conversion between YCgCo color space and RGB color space according to the above equations is reversible with respect to the original state. Thus, the transformation of the color space according to the equations supports the ideal recovery. For example, the sample value is maintained even if the reverse conversion is performed after the forward conversion. Accordingly, the color space transformation according to the equations may be referred to as "reversible YCgCo-R color transformation". In this document, R can mean reversible, which means that restoration to the original state is achieved. The YCgCo-R transform may be provided by increasing the bit depths of Cg and Co by 1 compared to the existing transform. If this condition is met, other types of reversible transformation may be used, such as the transformation described above.

[826] Since the transform as shown in the above equations has a different normal value from the above transform, the ACT Qp offsets for Y, Cg and Co can be adjusted to compensate for the change in dynamic range due to the color space transform.

[827] It is described that when the above transformation is applied, the QCT Qp offset according to the embodiment may have a value of (-5, -5, -5) for Y, Cg and Co. However, when the reversible transformation according to the present embodiment is applied, as the QCT Qp offset according to the embodiment, a value other than (-5, -5, -5) may be specified. For example, as the QCT Qp offset according to the embodiment, the value (-5, 1, 3) for Y, Cg and Co can be used.

[828] In another embodiment, the ACT QP offset may be signaled via a bit stream, similar to embodiment 6 or 7 described above.

[829] For example, when the YCgCo-R transform described above is used in conjunction with an ACT QP offset (-5, 1, 3), it should be noted that encoding loss is not present in the lossy coding environment (e.g., QP 22 , 27, 32, 37) as shown in the drawing below. In addition, it should be noted that when ACT is applied, an encoding performance of 5% is additionally obtained in the implementation of lossless encoding.

[830] Tab. 4

Subsequence Y U V RGB, TGM 1080p 0.0% 0.2% 0.1% RGB, TGM 720p 0.2% -0.1% 0.1% RGB animation -0.1% -0.1% 0.0% RGB, mixed content -0.1% 0.0% -0.1% RGB camera captured content -0.3% 0.2% -0.3% All (RGB) 0.0% 0.0% 0.0%

[831] The VVC specification for including the integrated ACT matrix may be described as the following table.

[832] Tab. 5

Residual Modification Process for Blocks Using Color Space Transformation
The inputs to this process are the following:
- variable nTbW indicating the width of the block,
- variable nTbH indicating the height of the block,
- array r _Y (nTbW)x(nTbH) of residual samples of luminance signals with elements r _Y [x][y],
- array r _Cb (nTbW)x(nTbH) of residual samples of color signals with elements with elements r _Cb [x][y],
- array r _Cr (nTbW)x(nTbH) of residual samples of color signals with elements r _Cr [x][y].
The outputs of this process are as follows:
- modified array r _Y (nTbW)x(nTbH) of residual samples of luminance signals,
- modified array r _Cb (nTbW)x(nTbH) of residual samples of color signals,
- modified array r _Cr (nTbW)x(nTbH) of residual samples of color signals.
Arrays r _Y , r _Cb and r _Cr (nTbW)x(nTbH) of residual samples are modified as follows:
tmp=r _Y [x][y]-(r _Cb [x][y]>>1))
r _Y [x][y]=tmp+r _Cb [x][y]
r _Cb [x][y]=tmp-(r _Cr [x][y]>>1)
_rCr [x][y]= _rCb [x][y]+ _rCr [x][y]

[833] For example, arrays r _Y , r _Cb and r _Cr of size (nTbW)x(nTbH) residual samples may be updated as follows.

[834] Equation 109

[835] tmp=r _Y [x][y]-(r _Cb [x][y]>>1)

[836] _rY [x][y]=tmp+r _Cb [x][y]

[837] r _Cb [x][y]=tmp-(r _Cr [x][y]>>1)

[838] r _Cr [x][y]=r _Cb [x][y]+r _Cr [x][y]

[839] Option 10 implementation. ACT execution method to perform multiple color transformations based on explicit signaling

[840] In the present embodiment, at least one color conversion may be performed by the ACT. Which color conversion is to be performed may be determined by the flag(s) signaled in the bitstream. The flag(s) may be signaled at multiple levels, eg, SPS, PPS, frame header, and slice, or in identifiable granularity.

[841] In an embodiment, a predefined flag may be signaled to indicate which ACT is applied. For example, when the flag has a value of 1, an ACT based reversible color conversion may be applied. When the flag is set to 0, ACT based irreversible color conversion can be applied.

[842] In another embodiment, a predefined flag for ACT may be signaled to indicate which color conversion is being used. An example of the syntax signaled in the SPS is shown in FIG. 50. In the following, the syntax element of FIG. 50. The sps_act_reversible_conversion syntax element may indicate whether or not a conversion equation that is irreversible with respect to the original state is used. The first value (eg 0) of sps_act_reversible_conversion may indicate that the ACT uses a conversion equation that is irreversible with respect to the original state. The second value (eg, 1) sps_act_reversible_conversion may indicate that the ACT uses a conversion equation that is reversible with respect to the original state.

[843] Accordingly, a lossyCoding variable indicating whether or not lossy encoding is performed may be set as indicated in the equation below.

[844] Equation 110

[845] lossyCoding=(!sps_act_reversible_conversion)

[846] By using the lossyCoding flag, the pseudo-code for the decoding device to perform the inverse conversion from YCgCo to GBR during the decoding process can be expressed as follows.

[847] Equation 111

[848] If(sps_act_reversible_conversion==1)

[849]

[850] //Reversible YCgCo-R Transform

[851] t=Y-(Cg>>1)

[852] G=Cg+t

[853] B=t-(Co>>1)

[854] R=Co+B

[855]}

[856] else {

[857] t=Y-Cg

[858] G=Y+Cg

[859] B=t-Co

[860] R=t+Co

[861]

[862] Accordingly, the VVC specification shown in Table 5 of Embodiment 9 may be modified as shown in the table below.

[863] Tab. 6

Residual Modification Process for Blocks Using Color Space Transformation
The inputs to this process are the following:
- variable nTbW indicating the width of the block,
- variable nTbH indicating the height of the block,
- array r _Y (nTbW)x(nTbH) of residual samples of luminance signals with elements r _Y [x][y],
- array r _Cb (nTbW)x(nTbH) of residual samples of color signals with elements r _Cb [x][y],
- array r _Cr (nTbW)x(nTbH) of residual samples of color signals with elements r _Cr [x][y].
The outputs of this process are as follows:
- modified array r _Y (nTbW)x(nTbH) of residual samples of luminance signals,
- modified array r _Cb (nTbW)x(nTbH) of residual samples of color signals,
- modified array r _Cr (nTbW)x(nTbH) of residual samples of color signals.
Arrays r _Y , r _Cb and r _Cr (nTbW)x(nTbH) of residual samples are modified as follows:
- If sps_act_reversible_conversion is equal to 1, the arrays r _Y , r _Cb and r _Cr (nTbW)x(nTbH) of the residual samples are modified as follows:
tmp=r _Y [x][y]-(r _Cb [x][y]>>1)
r _Y [x][y]=tmp+r _Cb [x][y]
)
r _Cb [x][y]=tmp-(r _Cr [x][y]>>1)
_rCr [x][y]= _rCb [x][y]+ _rCr [x][y]
- Otherwise, the arrays r _Y , r _Cb and r _Cr (nTbW)x(nTbH) of the residual samples are modified as follows:
tmp=r _Y [x][y]-r _Cb [x][y]
)
r _Y [x][y]=r _Y [x][y]+r _Cb [x][y]
)
r _Cb [x][y]=tmp-r _Cr [x][y]
)
_rCr [x][y]=tmp+ _rCr [x][y]

[864] According to the above table, the residual update process using color space conversion may use the following parameters as inputs to the current process.

- variable nTbW indicating the width of the block

[865] - variable nTbH indicating block height

[866] - array r _Y of size (nTbW)x(nTbH) for the residual sample of luminance signals, consisting of elements r _Y [x][y],

[867] - array r _Cb of size (nTbW)x(nTbH) for the residual sample of chrominance signals, consisting of elements r _Cb [x][y],

[868] - array r _Cr of size (nTbW)x(nTbH) for the residual sample of chrominance signals, consisting of elements r _Cr [x][y],

[869] The output for the current process is as follows:

[870] - updated array r _Y of size (nTbW)x(nTbH) for the residual sample of luminance signals,

[871] - updated array r _Cb of size (nTbW)x(nTbH) for residual chrominance sampling,

[872] - updated array r _Cr of size (nTbW)x(nTbH) for residual chrominance sampling,

[873] By executing the current process, the arrays r _Y , r _Cb and r _Cr of size (nTbW)x(nTbH) residual samples can be updated as follows.

[874] First, when the value of sps_act_reversible_conversion is the second value (eg, 1), the arrays r _Y , r _Cb and r _Cr of size (nTbW)x(nTbH) residual samples may be updated as shown in the equation below.

[875] Equation 112

[876] tmp=r _Y [x][y]-(r _Cb [x][y]>>1)

[877] r _Y [x][y]=tmp+r _Cb [x][y]

)

[878] r _Cb [x][y]=tmp-(r _Cr [x][y]>>1)

[879] r _Cr [x][y]=r _Cb [x][y]+r _Cr [x][y]

[880] Otherwise (eg, when the value of sps_act_reversible_conversion is the first value (eg, 0)), the arrays r _Y , r _Cb and r _Cr of size (nTbW)x(nTbH) residual samples may be updated as shown in the equation below.

[881] Equation 113

[882] tmp=r _Y [x][y]-r _Cb [x][y]

[883] r _Y [x][y]=r _Y [x][y]+r _Cb [x][y]

[884] r _Cb [x][y]=tmp-r _Cr [x][y]

[885] _rCr [x][y]=tmp+ _rCr [x][y]

[886] The inverse YCgCo-transformation and the inverse YCgCo-R-transformation have some similarities. In a reversible transformation, when Cg and Co are replaced by Cgp'=Cg<<1 and Cop'=Co<<1, this can work as a lossy inverse transform. The following equation shows an embodiment of the above.

[887] Equation 114

[888] t=Y-(Cgp'>>1)=Y-Cg

[889] G=Cgp'+t=Y+Cg

[890] B=t-(Cop'>>1)=t-Co=Y-Cg-Co

[891] R=Cop'+B=t+Co=Y-Cg+Co

[892] Accordingly, in an alternative embodiment, instead of maintaining two color transformations, only a transformation that is reversible with respect to the original state can be used. In the case of lossy coding, the Cg and Co components may be scaled by a factor of 1/2 when the encoder is running, and may be scaled by a factor of two when the decoder is running. This makes it possible to use the integrated transform even when lossy and lossless cases are supported. In addition, there is such an additional advantage that the bit depth does not change even when lossy encoding is performed.

[893] Tab. 7

Reverse conversion: YCgCo to GBR If(lossyCoding)
{
Cg=Cg<<1;
Co=Co<<1;
}
t=Y-(Cg>>1)
G=Cg+t
B=t-(Co>>1)
R=Co+B

[894] In an embodiment, a flag (eg, actShiftFlag) indicating which ACT is being used may be used according to the syntax of FIG. 51. In the syntax table of FIG. 51, the sps_act_shift_flag syntax element may indicate whether or not a color component shift is performed at the time the ACT is applied. For example, the first value (eg, 0) of the sps_act_shift_flag may indicate that the color component shift execution is not applied at the time the ACT is applied. The second value (eg, 1) of the sps_act_shift_flag may indicate that the color component shift execution is being applied at the time the ACT is being applied. The variable actShiftFlag can be set to the value of sps_act_shift_flag. Pseudo-code to implement the inverse transformation from YCgCo to GBR in the decoder can be written using actShiftFlag, as follows.

[895] Tab. 8

Reverse conversion: YCgCo to GBR If(actShiftFlag)
{
Cg=Cg<<1;
Co=Co<<1;
}
t=Y-(Cg>>1)
G=Cg+t
B=t-(Co>>1)
R=Co+B

[896] Option 11 implementation. ACT execution method to perform multiple color transformations using transformation type extraction

In an embodiment, at least one color transform may be used when performing the ACT. In addition, what type of color conversion is used may be derived based on other bitstream information.

[897] In an embodiment, two types of ACT transformations may be available, including an ACT transformation that is reversible with respect to the original state and an ACT that is irreversible with respect to the original state. ACT conversion types can be retrieved via conversion types. For example, as identified by the tuIsTransformSkip variable, when the transform type is a transform skip, an ACT transform that is reversible with respect to the original state can be used. Otherwise (for example, when the conversion type is not a conversion skip), an ACT conversion that is irreversible with respect to the original state can be used. Two types of pseudocode can be used.

[898] Tab. 9

Reverse conversion: YCgCo to GBR If(tuIsTransformSkip)
{
//Reversible YCgCo-R transformation
t=Y-(Cg>>1)
G=Cg+t
B=t-(Co>>1)
R=Co+B
}
else {
t=Y-Cg
G=Y+Cg
B=t-Co
R=t+Co
}

[899] Tab. 10

Reverse conversion: YCgCo to GBR If(tuIsTransformSkip)
{
Cg=Cg<<1;
Co=Co<<1;
}
t=Y-(Cg>>1)
G=Cg+t
B=t-(Co>>1)
R=Co+B

[900] In another embodiment, ACT conversion types may be determined based on QP values. When the value of Qp is less than or equal to a predetermined threshold value (eg, QpPrimeTsMin), an initial state reversible ACT may be used. Otherwise (for example, when the value of Qp exceeds a predetermined threshold value), an irreversible ACT transformation may be used.

[901] [902] Embodiment 12. QP extraction method using ACT QP bias

[903] The present embodiment is related to Embodiments 1 and 2 described above. In embodiments 1 and 2, it has already been described that recovered Qp' _Y , Qp' _CbCr , Qp' _Cb and Qp' _Cr are included as QP. The methods described in embodiments 1 and 2 correct the obtained Qp values by using the ACT QP offset and apply a significant clipping technique such that the corrected QP values for scaling the transform coefficients are not outside the effective range.

[904] The present embodiment describes a method for including a QP bias ACT in a QP extraction process to extract Qp' _Y , Qp' _CbCr , Qp' _Cb and Qp' _Cr . The QP extraction process already includes a predefined clipping step such that the extracted QP values are not outside the effective range. Therefore, including the QP offset ACT in the QP extraction process avoids an additional clipping step, simplifies the overall QP extraction step for the transform coefficient scaling process, and ensures that the final QP is not outside the effective range.

[905] As described in the embodiment described above, the ACT QP offset may be pre-specified as a constant or may be signaled via a bit stream. Without departing from consistency, ACT QP offsets for Y, Cb, Cr, and CbCr may be described as ppsActQpOffsetY, ppsActQpOffsetCb, ppsActQpOffsetCr, and ppsActQpOffsetCbCr in the description below. ppsActQpOffsetY, ppsActQpOffsetCb, ppsActQpOffsetCr, and ppsActQpOffsetCbCr may be constants or variables having values ranging from -M to N. In this document, each of M and N may be set to 12 in the case of lossy encoding, and may be set to 0 in the case of lossless encoding. losses in the embodiment. In addition, at least one ACT QP offset may be derived from another ACT QP offset value. For example, ppsActQpOffsetCbCr may be set to the same value as ppsActQpOffsetCb or ppsActQpOffsetCr based on the combined CbCr mode.

[906] Decoding processing for Qp extraction using the QP offset ACT may be performed as described below. First, in the case of a quantization parameter extraction process, the following parameters may be used for the current process.

[907] Luma coordinates (xCb, yCb) indicating the relative coordinates of the top left luma sample of the current coding block relative to the top left luminance sample of the current frame,

[908] - Variable cbWidth indicating the width of the current coding block based on a sample of luminance signals,

[909] - Variable cbHeight indicating the height of the current coding block based on a sample of luminance signals,

[910] - The treeType variable indicating whether a single tree (SINGLE_TREE) or a dual tree is used to segment the current node of the coding tree, and indicating when a dual tree is used, then represents the dual tree as a dual tree (DAUL_TREE_LUMA) of signal components luma or dual tree (DAUL_TREE_CHROMA) of chroma components.

[911] In the present quantization parameter extraction process, the luminance quantization parameter Qp' _Y and the chrominance quantization parameters Qp' _Cb , Qp' _Cr and Qp' _CbCr can be extracted.

[912] After that, the variable Qp _Y can be extracted as shown in the equation below.

[913] Equation 115

[914] Qp _Y =((qP _{Y_PRED} +CuQpDeltaVal+64+2*QpBdOffset)%(64+QpBdOffset))-QpBdOffset

[915] The luminance quantization parameter Qp' _Y may be extracted as shown in the equation below.

[916] Equation 116

[917] actQpOffsetY=cu_act_enabled_flag[xCb][yCb]?ppsActQpOffsetY:0

[918] _Qp'Y = _QpY +QpBdOffset+actQpOffsetY

[919] Qp' _Y =Clip3(0, 63+QpBdOffset, Qp' _Y )

[920] When the value of the ChromaArrayType variable indicating the chrominance array type is not the first value (eg, 0), and treeType is SINGLE_TREE or DUAL_TREE_CHROMA, the following processing may be performed.

[921] - When the value of treeType is DUAL_TREE_CHROMA, the value of variable Qp _Y may be set to a value identical to the value of the luminance quantization parameter Qp _Y of the luminance coding basis covering the sample location (xCb+cbWidth/2, yCb+cbHeight/2) of the signals brightness.

[922] - Variables qP _Cb , qP _Cr and qP _CbCr can be extracted as shown in the equation below.

[923] Equation 117

[924] qP _Chroma =Clip3(-QpBdOffset, 63, Qp _Y ) (1118)

[925] qP _Cb =ChromaQpTable[0][qP _Chroma ]

(1119)

[926] qP _Cr =ChromaQpTable[1][qP _Chroma ]

(1120)

[927] qP _CbCr =ChromaQpTable[2][qP _Chroma ]

(1121)

[928] The chrominance quantization parameters Qp' _Cb and Qp' _Cr for the Cb and Cr components and the chrominance quantization parameter Qp' _CbCr for the combined Cb-Cr coding may be extracted as shown in the equation below.

[929] Equation 118

[930] actQpOffsetCb=cu_act_enabled_flag[xCb][yCb]?ppsActQpOffsetCb:0

[931] actQpOffsetCr=cu_act_enabled_flag[xCb][yCb]?ppsActQpOffsetCr:0

[932] actQpOffsetCbCr=cu_act_enabled_flag[xCb][yCb]?ppsActQpOffsetCbCr:0

[933] Qp' _Cb =Clip3(-QpBdOffset, 63, qP _Cb +pps_cb_qp_offset+slice_cb_qp_offset+CuQpOffset _Cb +actQpOffsetCb)+QpBdOffset

[934] Qp' _Cr =Clip3(-QpBdOffset, 63, qP _Cr +pps_cr_qp_offset+slice_cr_qp_offset+CuQpOffset _Cr +actQpOffsetCr)+QpBdOffset

[935] Qp' _CbCr =Clip3(-QpBdOffset, 63, qP _CbCr +pps_joint_cbcr_qp_offset +

[936] slice_joint_cbcr_qp_offset+CuQpOffset _CbCr +actQpOffsetCbCr)+QpBdOffset

[937] [938] Then, the dequantization process for the transform coefficient may be performed, and the following information may be used as input for the current process.

[939] - Coordinates (xTbY, yTbY) of luminance signals referring to the relative coordinates of the top left sample of the current luminance transform block relative to the top left sample of the luminance signals of the current frame,

[940] - Variable nTbW indicating the width of the transform block,

[941] - Variable nTbH indicating the height of the transform block,

[942] - Variable predMode indicating the coding basis prediction mode,

[943] - Variable cIdx indicating the color component of the current block

[944] The output of the dequantization process for the transform coefficient may be an array d of scaled transform coefficients. In this document, the size of the array d may be (nTbW)x(nTbH). The individual elements that make it up can be identified as d[x][y].

[945] When executing the current process, the quantization parameter qP may be extracted as follows. When cIdx has a value of 0, qP can be extracted as shown in the equation below.

[946] Equation 119

[947] qP= _Qp'Y

[948] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, extraction may be performed as shown in the equation below.

[949] Equation 120

_[950] qP= _Qp'CbCr

[951] Alternatively, when cIdx has a value of 1, qP may be extracted as shown in the equation below.

[952] Equation 121

[953] qP=Qp' _Cb

[954] Alternatively, when cIdx has a value of 2, qP may be extracted as shown in the equation below.

[955] Equation 122

[956] qP=Qp' _Cr

[957] The quantization parameter qP may be updated as follows. In addition, the variables rectNonTsFlag and bdShift can be retrieved as follows. When transform_skip_flag[xTbY][yTbY][cIdx] is set to 0, extraction can be performed as shown in the equation below.

[958] Equation 123

[959] rectNonTsFlag=(((Log2(nTbW)+Log2(nTbH)) and 1)==1)?1:0

[960] bdShift=BitDepth+rectNonTsFlag+((Log2(nTbW)+Log2(nTbH))/2)-5+pic_dep_quant_enabled_flag. Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1 (for example, the transformation of the current transformation block is skipped), extraction may be performed as shown in the equation below.

[961] Equation 124

[962] qP=Max(QpPrimeTsMin, qP)

[963] rectNonTsFlag=0

[964] bdShift=10

[965] Encoding and decoding methods

[966] Hereinafter, an image encoding method performed by an image encoding apparatus and an image decoding method performed by an image decoding apparatus are described with reference to FIG. 52 and 53.

[967] First, the operation of the decoding apparatus will be described below. An image decoding apparatus according to an embodiment may include a memory and a processor. The decoding device may perform decoding according to the operation of the processor. For example, as shown in FIG. 52, the decoding apparatus may determine the prediction mode of the current block in step S5210 based on the prediction information obtained from the bit stream. Then, the decoding apparatus may obtain, based on the prediction mode of the current block, a flag indicating whether or not color space transformation is applied to the residual sample of the current block, in step S5220. Then, the decoding apparatus may determine the quantization parameter of the current block based on the flag in step S5230. Then, the decoding apparatus may determine the transform coefficient of the current block based on the quantization parameter in step S5240.

[968] Here, the decoding device can determine the quantization parameter by applying a cut to the quantization parameter such that the value of the quantization parameter has a value in a predetermined range. Herein, the lower limit value of the predefined range may be 0. In addition, the upper limit value of the predefined range may be determined based on a syntax element indicating a sampling bit depth. For example, the upper limit value of the predetermined range may be determined to be 63+QpBdOffset. QpBdOffset denotes an offset of the range of luminance and chrominance quantization parameters, and may be preset as a predefined constant or may be obtained from the bit stream. For example, QpBdOffset may be calculated by multiplying a predetermined constant by the value of a syntax element indicating the bit depth of the luminance or chroma sample.

[969] In addition, the decoding device may determine the quantization parameter based on the color component of the current block, may determine the offset of the quantization parameters based on the color component of the current block, and then may reset the quantization parameter by using the offset of the quantization parameters, thereby determining the quantization parameter. For example, the decoder may add the quantization parameter offset to the quantization parameter, thereby resetting the quantization parameter.

[970] The quantization parameter offset may be determined as follows. When a color space transformation is applied to the residual sample of the current block, and the color component of the current block is a luminance component, the quantization parameter offset value may be determined to be -5. Alternatively, when a color space transform is applied to the residual sample of the current block, and the color component of the current block is a Cb component of the chrominance signals, the quantization parameter offset value may be determined to be 1. Alternatively, when the color space transform is applied to the residual sample of the current block, and the color component of the current block is the Cr component of the chrominance signals, the quantization parameter offset value may be determined to be 3.

[971] Meanwhile, the residual sample of the current block is determined based on the transform coefficient of the current block, and the residual sample value may be updated based on whether or not a color space transform is applied. For example, when a color space transform is applied, the residual sample value may be updated based on the sample value that is obtained by applying a right shift operation to the residual sample value of the chrominance signal component. In an embodiment, the luminance component residual sample value of the residual sample may be updated by adding, with the luma component residual sample value, the value obtained by applying a right shift operation to the chrominance Cb component residual sample value. In addition, the residual sample value of the residual sample chrominance component Cb may be updated with a value obtained by subtracting the first value and the second value from the residual sample value of the luminance component, the first value being obtained by applying a right shift operation to the residual sample value Cb- chrominance signal component, and the second value is obtained by applying a right shift operation to the residual sample value of the chrominance Cr component.

[972] Hereinafter, this document describes the operation of the encoding device. An image encoding apparatus according to an embodiment may include a memory and a processor. The encoding device may perform encoding according to the operation of the processor in a manner that corresponds to decoding by the decoding device. For example, as shown in FIG. 53, the encoder may determine the prediction mode of the current block in step S5310. Then, the encoding device may determine the quantization parameter of the current block depending on whether or not a color space transformation is applied to the residual sample of the current block, in step S5320. Then, the encoder may determine the transform coefficient of the current block based on the quantization parameter in step S5330. Then, the coding apparatus may encode, based on the prediction mode of the current block, information indicating whether or not a color space transformation is applied to the residual sample of the current block, in step S5340.

[973] In accordance with the operation of the decoding device described above, the encoding device can determine the quantization parameter by applying a cut to the quantization parameter such that the value of the quantization parameter has a value in a predetermined range. Herein, the lower limit value of the predetermined range may be set to 0. In addition, the upper limit value of the predetermined range may be set to the predetermined upper limit value. The upper limit value may be determined based on the bit depth of the sample. For example, the upper limit value of the predetermined range may be determined to be 63+QpBdOffset. Herein, QpBdOffset denotes an offset of the range of luminance and chrominance quantization parameters, and may be preset as a predefined constant or may be obtained from a bitstream. For example, QpBdOffset may be calculated by multiplying a predetermined constant by the value of a syntax element indicating the bit depth of the luminance or chroma sample.

[974] In addition, the encoding device may determine the quantization parameter based on the color component of the current block, may determine the offset of the quantization parameters based on the color component of the current block, and then may reset the quantization parameter by using the offset of the quantization parameters, thereby determining the quantization parameter. For example, the encoder may add the quantization parameter offset to the quantization parameter, thereby resetting the quantization parameter.

[975] The quantization parameter offset may be determined as follows. When a color space transformation is applied to the residual sample of the current block, and the color component of the current block is a luminance component, the quantization parameter offset value may be determined to be -5. Alternatively, when a color space transform is applied to the residual sample of the current block, and the color component of the current block is a Cb component of the chrominance signals, the quantization parameter offset value may be determined to be 1. Alternatively, when a color space transform is applied to the residual sample of the current block, and the color component of the current block is the Cr component of the chrominance signals, the quantization parameter offset value may be determined to be 3.

[976] Meanwhile, the residual sample of the current block is determined based on the transform coefficient of the current block, and the residual sample value may be updated based on whether or not a color space transform is applied. For example, when a color space transform is applied, the residual sample value may be updated based on the sample value that is obtained by applying a right shift operation to the residual sample value of the chrominance signal component.

[977] Application

[978] Although the exemplary methods of the present disclosure described above are presented as a sequence of operations for clarity of description, it is not intended to limit the order in which the steps are performed, and the steps may be performed simultaneously or in a different order as needed. In order to implement the method according to the present disclosure, the described steps may further include other steps, may include the remaining steps except for some steps, or may include other additional steps except for some steps.

[979] In the present disclosure, an image encoding equipment or an image decoding equipment that performs a predetermined operation (step) may perform an operation (step) of confirming a condition or situation of performing a corresponding operation (step). For example, if it is described that the predetermined operation is performed when the predetermined condition is satisfied, the image encoding equipment or the image decoding equipment may perform the predetermined operation after determining whether or not the predetermined condition is satisfied.

[980] The various embodiments of the present disclosure are not a list of all possible combinations and are intended to describe characteristic aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more of them.

[981] Various embodiments of the present disclosure may be implemented in hardware, firmware, software, or a combination of the above. If the present disclosure is implemented in hardware, the present disclosure may be implemented using Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGA), general processors, controllers, microcontrollers, microprocessors, etc.

[982] In addition, image decoding equipment and image encoding equipment to which embodiments of the present disclosure apply may be included in a multimedia broadcast transceiver, a mobile communication terminal, a home theater system video device, a digital cinema system video device, a surveillance camera, video chatting device, real-time communication device, for example, for video communication, mobile streaming device, storage media, video recording camera, video-on-demand (VoD) service delivery device, OTT video device ( video over networks), an Internet streaming service provider, a 3D video device, a video telephony system video device, a medical video device, and the like. and can be used to process video signals or data signals. For example, OTT video devices may include a game console, a Blu-ray player, an Internet access TV, a home theater system, a smartphone, a tablet PC, a digital video recorder (DVR), and the like.

[983] FIG. 54 is a view showing a content streaming system to which an embodiment of the present disclosure is applicable.

[984] Referring to FIG. 54, the content streaming system to which the embodiment(s) of the present document is applied may mainly include an encoding server, a streaming server, a web server, a media storage, a user device, and a media input device.

[985] The encoding server compresses content input from media input devices such as a smartphone, camera, camcorder, etc. into digital data to form a bit stream, and transmits the bit stream to the stream server. As another example, when media input devices such as smartphones, cameras, camcorder recorders, etc. directly generate a bitstream, the encoding server may be omitted.

[986] The bitstream may be generated by an image encoding method or image encoding equipment to which an embodiment of the present disclosure applies, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[987] The streaming server transmits media data to the user device based on the user's request through the web server, and the web server serves as a medium for informing the user about the service. When a user requests a desired service from a web server, the web server may deliver it to the streaming server, and the streaming server may transmit media data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server serves to manage the command/response between devices in the content streaming system.

[988] The streaming server may receive content from a media store and/or an encoding server. For example, when content is received from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may keep the bitstream for a predetermined time.

[989] Examples of the user device may include a mobile phone, smartphone, laptop computer, digital broadcast terminal, personal digital assistant (PDA), portable media player (PMP), navigation device, stylus tablet PC, tablet PCs, ultrabooks, wearable devices (e.g., smart watches, smart glasses, head-mounted displays), digital TVs, desktop computers, digital signage system, etc.

[990] Each server in the content streaming system may operate as a distributed server, in which case the data received from each server may be distributed.

[991] The scope of the disclosure includes software or computer-executable instructions (for example, an operating system, application, firmware, program, etc.) for enabling operations according to the methods of various embodiments in equipment or a computer, a non-volatile computer-readable medium , which has such software or commands stored and executed in hardware or a computer.

Industrial Applicability

[992] Embodiments of the present disclosure may be used to encode or decode an image.

Claims (47)

1. An image decoding method performed by an image decoding device, the method comprising the steps of: determining a prediction mode of the current coding unit based on the prediction information in the bit stream; obtaining color space conversion information for applying a color space conversion to a residual sample of the current coding unit based on the prediction mode; extracting a first quantization parameter for the current coding unit; extracting a final quantization parameter based on the first quantization parameter for the current coding unit; extracting a transform coefficient for the current coding unit based on a final quantization parameter for the current coding unit; And extracting a residual sample for the current coding unit based on a transform coefficient for the current coding unit; wherein the current coding unit contains a block of luminance signal components, a block of first color components and a block of second color components, wherein, based on the color space transformation information indicating not applying the color space transformation to the residual sample of the current coding unit, the quantization parameter offset values for the luminance signal component block, the first color component block, and the second color component block of the current coding unit are retrieved as 0, respectively, wherein, based on the color space transformation information indicating the application of the color space transformation to the residual sample of the current coding unit, the quantization parameter offset values for the luminance signal component block, the first color component block, and the second color component block of the current coding unit are retrieved as different from each other. 2. The method according to p. 1, wherein, based on the color space transform information indicating the application of the color space transform to the residual sample of the current coding unit, the offset value of the luminance component block quantization parameters is extracted as a negative value. 3. The method according to claim 1, wherein, based on the color space transform information indicating the application of the color space transform to the residual sample of the current coding unit, the luma component block quantization parameter offset value is extracted as -5. 4. The method according to claim 1, wherein, based on the color space transform information indicating the application of the color space transform to the residual sample of the current coding unit, the offset value of the quantization parameters of the block of the first color components is extracted as 1. 5. The method according to claim 1, in which, based on the color space transformation information indicating the application of the color space transformation to the residual sample of the current coding unit, the offset value of the quantization parameters of the second color component block is extracted as 3. 6. The method according to claim 1, wherein the first color component block is a Cb chroma block and the second color component block is a Cr chrominance block. 7. The method according to claim 1, wherein, based on the color space transform information indicating the application of the color space transform to the residual sample of the current coding unit, the residual sample values for the luma component block, the first color component block, and the second color component block, respectively, are updated based on the residual sample value for the block luminance signal components. 8. The method according to p. 7, wherein, based on the color space transform information indicating the application of the color space transform to the residual sample of the current coding unit, the value of the residual samples for the luma component block, the first color component block, and the second color component block is further updated based on the sample value obtained by applying the operation shift to the right to the value of the residual sample for the block of the first color components. 9. The method according to claim 8, wherein the sample value obtained by applying a right shift operation to the residual sample value for the block of first color components corresponds to half of the residual sample value for the block of first color components. 10. The method according to p. 7, wherein, based on the color space transform information indicating the application of the color space transform to the residual sample of the current coding unit, the residual sample value for the luma component block is updated based on the summation of the sample value obtained by applying the right shift operation to the residual sample value for the first color block. components, with a residual sample value for the block of luma components. 11. The method according to p. 10, wherein the sample value obtained by applying a right shift operation to the residual sample value for the block of first color components corresponds to half of the residual sample value for the block of first color components. 12. The method according to p. 7, wherein, based on the color space transform information indicating the application of the color space transform to the residual sample of the current coding unit, the residual sample value for the first color component block is updated based on the subtraction of the first value and the second value from the residual sample value for the luma component block, wherein the first value is a value obtained by applying a right shift operation to the residual sample value for the first color component block, and the second value is a value obtained by applying a right shift operation to the residual sample value for the second color component block. 13. The method according to p. 12, wherein, based on the color space transform information indicating the application of the color space transform to the residual sample of the current coding unit, the residual sample value for the second color component block is updated based on the sum of the updated residual sample value for the first color component block and the residual sample value for the second color component block. components. 14. An image encoding method carried out by an image encoding device, the method comprising the steps of: determining a prediction mode of the current coding unit; extracting a color space transform applied to the residual sample of the current coding unit based on the prediction mode; extracting a residual sample of the current coding unit based on the applied color space transform; extracting a transform coefficient for the residual sample of the current coding unit; And performing quantization based on the transform coefficient and the target quantization parameter for the current coding unit; wherein the color space conversion information for applying the color space conversion to the residual sample of the current coding unit is encoded in the bitstream, wherein the final quantization parameter is derived based on the first quantization parameter for the current coding unit, wherein the current coding unit contains a block of luminance signal components, a block of first color components and a block of second color components, wherein, based on the color space transformation information indicating not applying the color space transformation to the residual sample of the current coding unit, the quantization parameter offset values for the luminance signal component block, the first color component block, and the second color component block of the current coding unit are retrieved as 0, respectively, wherein, based on the color space transformation information indicating the application of the color space transformation to the residual sample of the current coding unit, the quantization parameter offset values for the luminance signal component block, the first color component block, and the second color component block of the current coding unit are retrieved as different from each other. 15. A non-volatile computer-readable medium storing a bit stream generated by the image encoding method of claim 14.

Images