JP7328446B2 - Method and apparatus for signaling of chrominance quantization parameters - Google Patents

Description

[Cross reference to related application]
This patent application claims priority from International Patent Application PCT/RU2019/000664 filed on September 23, 2019. The entire contents of the disclosure of the above patent application are incorporated by reference.

[Technical field]
TECHNICAL FIELD Embodiments of the present application (disclosure) relate generally to the field of picture processing, and more particularly to methods and apparatus for signaling of chrominance quantization parameters.

Video coding (video encoding and/or decoding) is used in a wide range of digital video applications, such as broadcast digital TV, video transmission over the Internet and mobile networks, real-time conversational applications such as video chat, videoconferencing, DVD and Blu-ray discs. , video content acquisition and editing systems, and camcorders for security applications.

The amount of video data required to render even a relatively short video can be substantial when the data is streamed or across communication networks with limited bandwidth capacity. Difficulties may arise when being communicated. Therefore, video data is typically compressed before being communicated across modern telecommunications networks. The size of the video can also be an issue when the video is stored on a storage device, as memory resources may be limited. Video compression devices often use software and/or hardware at the source to code video data prior to transmission or storage, thereby reducing the amount of data required to represent a digital video image. . The compressed data is then received at the destination by a video decompression device that decodes the video data. Due to limited network resources and ever-increasing demand for higher video quality, compression and decompression techniques that improve compression ratios with little or no sacrifice in picture quality are desirable.

Embodiments of the present application provide apparatus and methods for encoding and decoding according to the independent claims.

These and other objects are achieved by the subject matter of the independent claims. Further realizations are evident from the dependent claims, the detailed description and the drawings.

This disclosure provides:
A method for inverse quantization of a current block of a picture, the method being performed by a decoder, the method comprising:
receiving a bitstream;
obtaining a joint chrominance component residual (JCCR) control flag from the bitstream;
obtaining chrominance mapping information from the bitstream based on JCCR control flags;
obtaining at least one chrominance quantization parameter (QP) offset from the bitstream based on JCCR control flags;
obtaining a QP value for the current chrominance block based on the obtained chrominance mapping information and at least one obtained chrominance QP offset;
performing inverse quantization on the current chrominance block by using the determined QP value.

Here, signaling of PPS and slice header QP offsets for chrominance components for JCCR mode and signaling of SPS chrominance mapping information for JCCR coding mode are performed.

Depending on the SPS JCCR control flag, joint chrominance component residual offset signaling/decoding is performed. Because of the conditional signaling of joint chrominance component residual offsets, less information needs to be signaled, thus saving resources.

In the above method, the bitstream may contain SPS level syntax and the JCCR control flags may be obtained from the SPS level syntax.

In the above method, the JCCR control flag may be sps_joint_cbcr_enabled_flag.

Here, sps_joint_cbcr_enabled_flag specifies whether joint coding of chroma residuals is enabled. where sps_joint_cbcr_enabled_flag equal to 1 specifies that joint coding of chroma residuals is enabled for the coded layer video sequence (CLVS), and sps_joint_cbcr_enabled_flag equal to 0 specifies that joint coding of chroma residuals is enabled. specifies that is disabled for the code layer video sequence, and when absent, the value of sps_joint_cbcr_enabled_flag is assumed to be equal to 0.

In the above method, if the value of sps_joint_cbcr_enabled_flag is 1, at least one obtained chrominance QP offset may be specified by slice_joint_cbcr_qp_offset. Here, slice_joint_cbcr_qp_offset can be present in the slice header syntax and specifies the difference to be added to the value of pps_joint_cbcr_qp_offset_value when determining the value of _Qp'CbCr . The value of slice_joint_cbcr_qp_offset shall be in the range of -12 to +12. When slice_joint_cbcr_qp_offset is not present, it is assumed to be equal to 0. The value of pps_joint_cbcr_qp_offset_value+slice_joint_cbcr_qp_offset shall be in the range of -12 to +12.

Here, the flag slice_joint_cbcr_qp_offset may also be denoted as sh_joint_cbcr_qp_offset.

In the above method, the chrominance mapping information may include delta_qp_in_val_minus1[i][j] and delta_qp_out_val[i][j], and the chrominance mapping information may be obtained from the SPS level syntax included in the bitstream. .

where sps_delta_qp_in_val_minus1[i][j] specifies the delta value used to derive the input coordinates of the jth pivot point of the ith chroma QP mapping table, and if not present, sps_delta_qp_in_val_minus1[0] The value of [j] is assumed to be equal to 0 and i, j are integer values. where delta_qp_out_val[i][j] specifies the delta value used to derive the output coordinate of the jth pivot point of the ith chroma QP mapping table. The value of delta_qp_out_val[0][j] is assumed to be equal to 0 when delta_qp_out_val[0][j] is not present in the bitstream. Here, delta_qp_out_val may also be denoted as sps_delta_qp_diff_val.

In the above method, the SPS level syntax may include the following structure.

Therefore, signaling of PPS and slice header QP offsets for chrominance components for JCCR mode, and signaling of SPS chrominance mapping information for JCCR coding mode, is performed dependent on the SPS JCCR control flags according to the table above. may For example, the sps JCCR control flag is sps_joint_cbcr_enabled_flag signaled in SPS level syntax, eg, seq_parameter_set_rbsp syntax. In particular, it can be appreciated that the value of index 'i' is determined based on the value of sps_joint_cbcr_enabled_flag, thus redundant signaling of JCCR syntax elements can be avoided when JCCR tools are disabled. It can be recognized that the signaling/decoding of pps_joint_cbcr_qp_offset depends on sps_joint_cbcr_enabled_flag, ie pps_joint_cbcr_qp_offset can be signaled or decoded only when the value of sps_joint_cbcr_enabled_flag is true (eg, 1). Due to the conditional signaling of pps_joint_cbcr_qp_offset, less information needs to be signaled, thus saving resources.

where seq_parameter_set_rbsp denotes the Raw Byte Sequence Payload (RBSP) syntax of the sequence parameter set, and sps_num_points_in_qp_table_minus1[i] plus 1 describes the i-th chroma QP mapping table The value of sps_num_points_in_qp_table_minus1[i] ranges from 0 to 36, and when absent, the value of sps_num_points_in_qp_table_minus1[0] is assumed to be equal to 0.

In the above method, the step of obtaining at least one chrominance QP offset from the bitstream based on the JCCR control flags is a picture parameter set (PPS) level syntax of the bitstream based on the JCCR control flags. obtaining at least one chrominance QP offset from .

In the above method, the PPS level syntax may include the following structure.

where pic_parameter_set_rbsp denotes the picture parameter set RBSP syntax, pps_cb_qp_offset and pps_cr_qp_offset specify the offset to the luma quantization parameter _Qp'Y used to derive _Qp'Cb and _Qp'Cr , respectively; The values of pps_cb_qp_offset and pps_cr_qp_offset are in the range -12 to +12 inclusive, and when sps_chroma_format_idc is equal to 0, pps_cb_qp_offset and pps_cr_qp_offset shall not be used in the decoding process, decoders shall ignore these values and shall not be present when the values of pps_cb_qp_offset and pps_cr_qp_offset are assumed to be equal to 0, and sps_joint_cbcr_enabled_flag equal to 1 specifies that joint coding of chroma residuals is enabled for the coded layer video sequence (CLVS), sps_joint_cbcr_enabled_flag equal to 0 specifies that joint coding of chroma residuals is disabled for CLVS; when absent, the value of sps_joint_cbcr_enabled_flag is assumed to be equal to 0. pps_joint_cbcr_qp_offset_value specifies the offset to the luma quantization parameter _Qp'Y used to derive _Qp'CbCr , and the value of pps_joint_cbcr_qp_offset_value is in the range from -12 to +12.

The disclosure further provides a method for inverse quantization of a current block of a picture, the method being performed by a decoder, the method comprising:
receiving a bitstream, the bitstream comprising slice header syntax and PPS syntax;
obtaining a syntax element from the PPS syntax, the obtained syntax element including at least one chrominance quantization parameter (QP) offset;
obtaining chrominance QP offset information from a slice header, wherein the QP offset information is obtained independently from any PPS syntax element within the PPS syntax;
determining a QP value for the current chrominance block dependent on at least one chrominance QP offset obtained from the PPS syntax and chrominance QP offset information obtained from the slice header syntax;
and performing inverse quantization on the current chrominance block by using the determined QP value.

Therefore, the signaling of PPS and slice header QP offsets for chrominance components is performed independently of each other according to the table above. Instead, previously the flag pps_slice_chroma_qp_offsets_present_flag was signaled in the PPS level syntax, which controls whether there are any further offsets to signal in the slice header, i.e., on the decoder side, the decoder should check the value of pps_slice_chroma_qp_offsets_present_flag to determine if there are any additional offsets signaled in the slice header. Compared to the previous method, here in the above method the flag pps_slice_chroma_qp_offsets_present_flag is no longer signaled. In other words, there are always offsets signaled in the slice header, so the decoder knows that there are more offsets signaled in the slice header without checking the value of pps_slice_chroma_qp_offsets_present_flag, in other words, Both the PPS level syntax and the slice header syntax always contain an offset. Decoding/signaling of the chrominance QP offsets in the slice header is therefore easier.

In the above method, the at least one chrominance QP offset obtained from the PPS syntax may include pps_cb_qp_offset, pps_cr_qp_offset, pps_joint_cbcr_qp_offset and cu_chroma_qp_offset_enabled_flag.

であり、iは整数である。 In the above method, if the value of cu_chroma_qp_offset_enabled_flag is 1, at least one chrominance QP offset obtained from the PPS syntax is cu_chroma_qp_offset_subdiv, chroma_qp_offset_list_len_minus1, cb_qp_offset_list[i], cr_qp_offset_list[i] and joint_cbcr_qp_offset_list[i ] But it's okay,

and i is an integer.

In the above method, the chrominance QP offset information obtained from the slice header syntax may include slice_cb_qp_offset and slice_cr_qp_offset.

In the above method, if the value of sps_joint_cbcr_enabled_flag included in the bitstream is 1, the chrominance QP offset information obtained from the slice header syntax may further include slice_joint_cbcr_qp_offset.

In the above method, the PPS syntax may include the following structure.

In the above method, the slice header syntax may include the following structure.

In the above method, the flag pps_slice_chroma_qp_offsets_present_flag may be omitted in the PPS syntax, or
Slice headers and PPS syntax may always contain at least one chrominance QP offset related element.

The disclosure further provides a method for inverse quantization of a current block of a picture, the method being performed by an encoder, the method comprising:
encoding a joint chrominance component residual (JCCR) control flag into the bitstream;
encoding chrominance mapping information into a bitstream based on JCCR control flags;
encoding at least one chrominance quantization parameter (QP) offset into the bitstream based on a JCCR control flag;
and providing a bitstream.

In the above method, the bitstream may contain SPS level syntax, and the JCCR control flags are encoded into the SPS level syntax.

In the above method, the JCCR control flag may be sps_joint_cbcr_enabled_flag.

In the above method, if the value of sps_joint_cbcr_enabled_flag is 1, at least one coded chrominance QP offset may be specified by slice_joint_cbcr_qp_offset.

In the above method, the chrominance mapping information may include delta_qp_in_val_minus1[i][j] and delta_qp_out_val[i][j], and the chrominance mapping information may be encoded into SPS level syntax included in the bitstream. good.

In the above method, the SPS level syntax may include the following structure.

where same_qp_table_for_chroma indicates how many chroma QP mapping tables have been signaled. Specifies that only one chroma QP mapping table is signaled when same_qp_table_for_chroma is equal to 1, this table applies to Cb and Cr residuals, and joint Cb-Cr residuals when sps_joint_cbcr_enabled_flag is equal to 1. applied to the difference. same_qp_table_for_chroma equal to 0 specifies that two for Cb and Cr and one more chroma QP mapping table for joint Cb-Cr when sps_joint_cbcr_enabled_flag is equal to one are signaled in the SPS. The value of same_qp_table_for_chroma is assumed to be equal to 1 when same_qp_table_for_chroma is not present in the bitstream.

where num_points_in_qp_table_minus1[i] plus 1 specifies the number of points used to describe the ith chroma QP mapping table. The value of num_points_in_qp_table_minus1[i] shall be in the range of 0 to 63+QpBdOffset. The value of num_points_in_qp_table_minus1[0] is assumed to be equal to 0 when num_points_in_qp_table_minus1[0] is not present in the bitstream.

In the above method, encoding at least one chrominance QP offset into the bitstream based on JCCR control flags comprises:
encoding the at least one chrominance QP offset into a bitstream picture parameter set (PPS) level syntax based on the JCCR control flag;

In the above method, the PPS level syntax may include the following structure.

The present disclosure may further provide a method for inverse quantization of a current block of a picture, the method being performed by an encoder, the method comprising:
encoding syntax elements from the slice header and PPS syntax into a bitstream, the syntax elements including at least one chrominance quantization parameter (QP) offset;
encoding chrominance QP offset information from the slice header into the bitstream, wherein the QP offset information is obtained independently from any PPS syntax element within the PPS syntax;
and providing a bitstream.

In the above method, the at least one chrominance QP offset encoded from the PPS syntax may include pps_cb_qp_offset, pps_cr_qp_offset, pps_joint_cbcr_qp_offset and cu_chroma_qp_offset_enabled_flag.

であり、iは整数である。 In the above method, if the value of cu_chroma_qp_offset_enabled_flag is 1, at least one chrominance QP offset encoded from the PPS syntax is cu_chroma_qp_offset_subdiv, chroma_qp_offset_list_len_minus1, cb_qp_offset_list[i], cr_qp_offset_list[i] and joint_cbcr_qp_offset_list[i] ] more may contain

and i is an integer.

In the above method, the chrominance QP offset information encoded from the slice header syntax may include slice_cb_qp_offset and slice_cr_qp_offset.

In the above method, if the value of sps_joint_cbcr_enabled_flag included in the bitstream is 1, the chrominance QP offset information encoded from the slice header syntax may further include slice_joint_cbcr_qp_offset.

In the above method, the PPS syntax may include the following structure.

In the above method, the slice header syntax may include the following structure.

In the above method, the flag pps_slice_chroma_qp_offsets_present_flag may be omitted in the PPS syntax, or
Slice headers and PPS syntax may always contain at least one chrominance QP offset related element.

The present disclosure further provides a decoder including processing circuitry for performing the method according to the above method.

The present disclosure further provides an encoder including processing circuitry for performing the above method.

The present disclosure also provides a computer program product containing program code for performing the above method when run on a computer or processor.

The present disclosure further provides a decoder that includes one or more processors and a non-transitory computer-readable storage medium coupled to the processor and storing programming for execution by the processor, the programming being executed by the processor. Then configure the decoder to perform the above method.

The present disclosure further provides an encoder including one or more processors and a non-transitory computer readable storage medium coupled to the processor and storing programming for execution by the processor, the programming being executed by the processor. Then configure the encoder to perform the above method.

The present disclosure further provides a non-transitory computer-readable medium that, when executed by a computing device, carries program code that causes the computing device to perform the above method.

The present disclosure includes a receiving unit configured to receive a bitstream and a first obtaining unit configured to obtain a joint chrominance component residual (JCCR) control flag from the bitstream. , a second obtaining unit configured to obtain chrominance mapping information from the bitstream based on the JCCR control flags; and at least one chrominance quantization parameter (QP, quantization a third obtaining unit configured to obtain a parameter) offset and to obtain a QP value for the current chrominance block based on the obtained chrominance mapping information and the at least one obtained chrominance QP offset; Further providing a decoder comprising a fourth acquisition unit configured and an inverse quantization unit configured to perform inverse quantization on the current chrominance block by using the determined QP value. .

In the above decoder, the bitstream may contain SPS level syntax and the JCCR control flags may be obtained from the SPS level syntax.

In the decoder above, the JCCR control flag may be sps_joint_cbcr_enabled_flag.

In the above decoder, if the value of sps_joint_cbcr_enabled_flag is 1, at least one obtained chrominance QP offset may be specified by slice_joint_cbcr_qp_offset.

In the decoder above, the chrominance mapping information may include delta_qp_in_val_minus1[i][j] and delta_qp_out_val[i][j], and the chrominance mapping information may be obtained from the SPS level syntax included in the bitstream. .

In the decoder above, the SPS level syntax may include the following structure.

In the above decoder, obtaining at least one chrominance QP offset from the bitstream based on the JCCR control flags is a bitstream picture parameter set (PPS) level syntax based on the JCCR control flags. obtaining at least one chrominance QP offset from .

In the decoder above, the PPS level syntax may include the following structure.

The present disclosure is a receiving unit configured to receive a bitstream, the bitstream including slice header syntax and PPS syntax, and to obtain syntax elements from the PPS syntax. A configured first acquisition unit, wherein the acquired syntax element comprises a first acquisition unit including a chrominance quantization parameter (QP) offset, and obtaining chrominance QP offset information from a slice header. wherein the QP offset information is independently obtained from any PPS syntax element in the PPS syntax; and a second obtaining unit configured to obtain from the PPS syntax a determining unit configured to determine a QP value for the current chrominance block in dependence on the obtained chrominance QP offset and the chrominance QP offset information obtained from the slice header syntax; and using the determined QP value. and an inverse quantization unit configured to perform inverse quantization on the current chrominance block.

In the above decoder, the at least one chrominance QP offset obtained from the PPS syntax may include pps_cb_qp_offset, pps_cr_qp_offset, pps_joint_cbcr_qp_offset and cu_chroma_qp_offset_enabled_flag.

であり、iは整数である。 In the decoder above, if the value of cu_chroma_qp_offset_enabled_flag is 1, then at least one chrominance QP offset obtained from the PPS syntax is cu_chroma_qp_offset_subdiv, chroma_qp_offset_list_len_minus1, cb_qp_offset_list[i], cr_qp_offset_list[i] and joint_cbcr_qp_offset_list[i ] But it's okay,

and i is an integer.

In the decoder above, the chrominance QP offset information obtained from the slice header syntax may include slice_cb_qp_offset and slice_cr_qp_offset.

In the above decoder, if the value of sps_joint_cbcr_enabled_flag included in the bitstream is 1, the chrominance QP offset information obtained from the slice header syntax may further include slice_joint_cbcr_qp_offset.

In the decoder above, the PPS syntax may include the following structure.

In the decoder above, the slice header syntax may include the following structure.

In the decoder above, the flag pps_slice_chroma_qp_offsets_present_flag may be omitted in the PPS syntax, or
Slice headers and PPS syntax may always contain at least one chrominance QP offset related element.

The present disclosure provides a first encoding unit configured to encode a joint chrominance component residual (JCCR) control flag into a bitstream, and based on the JCCR control flag, chrominance mapping information. a second encoding unit configured to encode into the bitstream and configured to encode at least one chrominance quantization parameter (QP) offset into the bitstream based on a JCCR control flag; and a providing unit configured to provide a bitstream.

In the above encoder, the bitstream may contain SPS level syntax and the JCCR control flags may be obtained from the SPS level syntax.

In the above encoder, the JCCR control flag may be sps_joint_cbcr_enabled_flag.

In the above encoder, if the value of sps_joint_cbcr_enabled_flag is 1, at least one obtained chrominance QP offset may be specified by slice_joint_cbcr_qp_offset.

In the above encoder, the chrominance mapping information may include delta_qp_in_val_minus1[i][j] and delta_qp_out_val[i][j], and the chrominance mapping information may be obtained from the SPS level syntax included in the bitstream. .

In the above encoder, the SPS level syntax may include the following structure.

In the above encoder, obtaining at least one chrominance QP offset from the bitstream based on the JCCR control flags is a bitstream picture parameter set (PPS) level syntax based on the JCCR control flags. obtaining at least one chrominance QP offset from .

In the encoder above, the PPS level syntax may include the following structure.

The present disclosure is a first encoding unit configured to encode syntax elements from a slice header and PPS syntax into a bitstream, the syntax elements being quantization parameter (QP) ) offset, and a second encoding unit configured to encode chrominance QP offset information from the slice header into the bitstream, the QP offset information being in the PPS syntax and a providing unit configured to provide a bitstream independently obtained from any PPS syntax element in the encoder.

In the above encoder, the at least one chrominance QP offset obtained from the PPS syntax may include pps_cb_qp_offset, pps_cr_qp_offset, pps_joint_cbcr_qp_offset and cu_chroma_qp_offset_enabled_flag.

であり、iは整数である。 In the above encoder, if the value of cu_chroma_qp_offset_enabled_flag is 1, at least one chrominance QP offset obtained from the PPS syntax is cu_chroma_qp_offset_subdiv, chroma_qp_offset_list_len_minus1, cb_qp_offset_list[i], cr_qp_offset_list[i] and joint_cbcr_qp_offset_list[i ] But it's okay,

and i is an integer.

In the above encoder, the chrominance QP offset information obtained from the slice header syntax may include slice_cb_qp_offset and slice_cr_qp_offset.

In the above encoder, if the value of sps_joint_cbcr_enabled_flag included in the bitstream is 1, the chrominance QP offset information obtained from the slice header syntax may further include slice_joint_cbcr_qp_offset.

In the encoder above, the PPS syntax may include the following structure.

In the above encoder, the slice header syntax may include the following structure.

In the encoder above, the flag pps_slice_chroma_qp_offsets_present_flag may be omitted in the PPS syntax, or
Slice headers and PPS syntax may always contain at least one chrominance QP offset related element.

The details of one or more embodiments are set forth in the accompanying drawings and the detailed description below. Other features, objects, and advantages will become apparent from the detailed description, drawings, and claims.

Embodiments of the invention are described in more detail below with reference to the accompanying figures and drawings.
1 is a block diagram illustrating an example of a video coding system configured to implement embodiments of the present invention; FIG. FIG. 3 is a block diagram illustrating another example of a video coding system configured to implement embodiments of the present invention; 1 is a block diagram illustrating an example of a video encoder arranged to implement embodiments of the invention; FIG. 1 is a block diagram showing an exemplary structure of a video decoder arranged to implement embodiments of the present invention; FIG. 1 is a block diagram illustrating an example of an encoding device or decoding device; FIG. FIG. 4 is a block diagram showing another example of an encoding device or a decoding device; 31 is a block diagram showing an exemplary structure of a content supply system 3100 that implements a content distribution service; FIG. 1 is a block diagram illustrating the structure of an example of a terminal device; FIG. Fig. 3 shows a flow chart of a method for inverse quantization of a current block of a picture, performed by a decoder, according to an embodiment of the present disclosure; Fig. 3 shows a flow chart of a method, performed by a decoder, for inverse quantization of a current block of a picture, according to a further embodiment of the present disclosure; 1 shows a flow chart of a method, performed by an encoder, for inverse quantization of a current block of a picture, according to an embodiment of the present disclosure; 1 shows a flow chart of a method for inverse quantization of a current block of a picture, performed by an encoder, according to the present disclosure; 2 shows a decoder according to an embodiment of the present disclosure; Fig. 3 shows a decoder according to another embodiment of the present disclosure; 1 illustrates an encoder according to an embodiment of the present disclosure; 4 illustrates an encoder according to another embodiment of the present disclosure;

In the following, identical reference numerals indicate identical or at least functionally equivalent features, unless explicitly specified otherwise.

In the following description, reference is made to the accompanying drawings which form a part of this disclosure and which show, by way of example, certain aspects of embodiments of the invention or in which embodiments of the invention may be employed. It is understood that embodiments of the invention may be used in other ways and may include structural or logical changes not shown in the drawings. Therefore, the following detailed description should not be taken in a limiting sense, and the scope of the invention is defined by the appended claims.

For example, it is understood that disclosure relating to a described method may also apply to a corresponding device or system configured to perform the method, and vice versa. For example, where one or more particular method steps are recited, the corresponding device may be one or more units, e.g., functional units, for performing the one or more recited method steps. , even if one or more such units are not explicitly described or shown in the drawings (e.g., one unit performs one or more steps, or multiple units each perform one or more of multiple steps). On the other hand, for example, if a particular apparatus is described in terms of one or more units, e.g. Such one or more steps may be included even if not explicitly described or shown in the drawings (e.g., one step may perform the function of one or more units, or multiple steps each perform one or more functions of multiple units). Further, it is understood that features of various exemplary embodiments and/or aspects described herein may be combined with each other unless stated otherwise.

Video coding typically refers to the processing of a sequence of pictures to form a video or video sequence. Instead of the term "picture", the terms "frame" or "image" may be used synonymously in the field of video coding. Video coding (or coding in general) includes two parts: video encoding and video decoding. Video encoding is performed on the source side and is typically performed to reduce (for more efficient storage and/or transmission) the amount of data needed to represent a video picture (e.g. It involves processing the original video picture (by compression). Video decoding is performed at the destination side and typically involves the inverse process compared to the encoder to reconstruct the video picture. Embodiments that refer to "coding" of video pictures (or pictures in general) shall be understood to relate to "encoding" or "decoding" of video pictures or respective video sequences. The combination of encoder and decoder is also called codec (Coding and Decoding).

For lossless video coding, the original video picture can be reconstructed, i.e. the reconstructed video picture has the same quality as the original video picture (assuming no transmission loss or other data loss during storage or transmission). do). In the case of lossy video coding, further compression is performed, for example by quantization, to reduce the amount of data representing a video picture, which cannot be fully reconstructed at the decoder, i.e. reconstructed video The picture quality is low or bad compared to the quality of the original video picture.

Several video coding standards belong to the group of "lossy hybrid video codecs" (ie combine spatial and temporal prediction in the sample domain with 2D transform coding to apply quantization in the transform domain). Each picture of a video sequence is typically partitioned into a set of non-overlapping blocks and coding is typically performed at the block level. In other words, at the encoder, for example, spatial (intra-picture) prediction and/or temporal (inter-picture) prediction is used to generate the predictive block, predicting from the current block (currently being/to be processed) By subtracting blocks to obtain residual blocks, transforming the residual blocks, and quantizing the residual blocks in the transform domain to reduce the amount of data to be transmitted (compression), video is typically is processed, ie coded, at block (video block) level. Meanwhile, at the decoder, the inverse process is applied to the encoded or compressed block compared to the encoder to reconstruct the current block for representation. In addition, the encoder replicates the decoder processing loops so that both use the same prediction (e.g., intra and inter prediction) and/or reconstruction to process, i.e., code, subsequent blocks. Generate.

Embodiments of the video coding system 10, the video encoder 20 and the video decoder 30 are described below with reference to FIGS. 1-3.

FIG. 1A is a schematic block diagram illustrating an exemplary coding system 10, such as a video coding system 10 (or coding system 10 for short), that may utilize the techniques of the present application. Video encoder 20 (or encoder 20 for short) and video decoder 30 (or decoder 30 for short) of video coding system 10 represent example devices that may be configured to perform techniques according to various examples described herein. .

As shown in FIG. 1A, coding system 10 includes source device 12 configured to provide encoded picture data 21 to, for example, destination device 14 for decoding encoded picture data 13 .

The source device 12 includes an encoder 20 and may also or optionally include a picture source 16 , a preprocessor (or preprocessing unit) 18 , eg a picture preprocessor 18 , and a communication interface or unit 22 .

Picture source 16 can be any kind of picture capture device, e.g., a camera for capturing real-world pictures, and/or any kind of picture generation device, e.g., a computer for generating computer animated pictures. graphics processor, or real-world pictures, computer-generated pictures (e.g., screen content, virtual reality (VR) pictures) and/or any combination thereof (e.g., augmented reality (AR) It may also be or include any other type of device for obtaining and/or providing reality (pictures). A picture source may be any kind of memory or storage that stores any of the above pictures.

Pictures or picture data 17 may also be referred to as raw pictures or raw picture data 17, in contrast to the processing performed by pre-processor 18 and pre-processing unit 18. FIG.

The pre-processor 18 is configured to receive (raw) picture data 17 and perform pre-processing on the picture data 17 to obtain a pre-processed picture 19 or pre-processed picture data 19 . . Pre-processing performed by pre-processor 18 may include, for example, cropping, color format conversion (eg, RGB to YCbCr), color correction, or noise removal. It will be appreciated that preprocessing unit 18 may be an optional component.

Video encoder 20 is configured to receive preprocessed picture data 19 and to provide encoded picture data 21 (further details are described below, for example with reference to FIG. 2).

Communication interface 22 of source device 12 receives encoded picture data 21 and renders encoded picture data 21 (or any further processed version thereof) over communication channel 13 for storage or direct reconstruction. ) to another device, such as the destination device 14 or any other device.

Destination device 14 includes decoder 30 (e.g., video decoder 30) and also, i.e., optionally, communication interface or communication unit 28, post-processor 32 (or post-processing unit 32), and display device 34. It's okay.

Communication interface 28 of destination device 14 receives encoded picture data 21 (or any ) and provide encoded picture data 21 to decoder 30 .

Communication interface 22 and communication interface 28 may be via a direct communication link, e.g., a direct wired or wireless connection, between source device 12 and destination device 14, or over any type of network, e.g., a wired or wireless network or configured to transmit or receive coded picture data 21 or coded data 13 via any combination thereof, or any kind of private and public network, or any kind of combination thereof; may

Communication interface 22 may, for example, package encoded picture data 21 into a suitable format, such as packets, and/or perform any type of transmission encoding or processing for transmission over a communication link or network. may be configured to process encoded picture data using.

A communication interface 28, forming the counterpart to communication interface 22, for example, receives the transmitted data and processes the transmitted data using any kind of corresponding transmission decoding or processing and/or depackaging to encode the data. It may be configured to acquire the coded picture data 21. FIG.

Both communication interface 22 and communication interface 28 may be configured as a unidirectional communication interface, as indicated by the arrow for communication channel 13 pointing from source device 12 to destination device 14 in FIG. 1A, or as a bidirectional communication interface. often, e.g., establish connections, e.g., to send and receive messages, authorize and exchange communication links and/or any other information related to data transmissions, e.g., coded picture data transmissions It may be configured as

The decoder 30 is arranged to receive encoded picture data 21 and to provide decoded picture data 31 or decoded pictures 31 (further details are given below, for example with reference to FIG. 3 or FIG. 5). ).

Post-processor 32 of destination device 14 post-processes decoded picture data 31 (also called reconstructed picture data), e.g. decoded picture 31, to obtain post-processed picture data 33, e.g. post-processed picture 33. configured to Post-processing performed by post-processing unit 32 may include, for example, color format conversion (eg, YCbCr to RGB), color correction, cropping or resampling, or processing of decoded picture data 31, for example, for display by display device 34, for example. may include any other processing to prepare for.

A display device 34 of the destination device 14 is configured to receive the post-processed picture data 33 and display the picture to a user or viewer, for example. Display device 34 may be or include any type of display for presenting reconstructed pictures, such as an integrated or external display or monitor. Displays include, for example, liquid crystal displays (LCDs), organic light emitting diode (OLED) displays, plasma displays, projectors, micro LED displays, liquid crystal on silicon (LCoS), digital It may be or include a light processor (DLP, digital light processor) or any kind of other display.

Although FIG. 1A shows source device 12 and destination device 14 as separate devices, device embodiments may also include functionality of both or both, source device 12 or corresponding functionality and destination device 14 or corresponding functionality. . In such embodiments, source device 12 or corresponding functionality and destination device 14 or corresponding functionality may be implemented using the same hardware and/or software or separate hardware and/or software or any of these. It may be realized by a combination.

As will be apparent to those skilled in the art based on the description, the presence and (exact) division of different units or functions within source device 12 and/or destination device 14 as shown in FIG. 1A depends on the actual device and application. may be changed by

Encoder 20 (e.g., video encoder 20) or decoder 30 (e.g., video decoder 30) or both encoder 20 and decoder 30 may include one or more microprocessors, digital signal processors (DSPs), application-specific 1B, such as an application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), discrete logic, hardware, dedicated video coding, or any combination thereof. It may be realized via such a processing circuit. Encoder 20 is implemented via processing circuitry 46 to embody various modules as described with respect to encoder 20 of FIG. 2 and/or any other encoder system or subsystem described herein. good too. Decoder 30 is implemented via processing circuitry 46 to embody various modules as described with respect to decoder 30 of FIG. 3 and/or any other decoder system or subsystem described herein. good too. The processing circuitry may be configured to perform various operations, as described below. If the technology is implemented in part in software, as shown in FIG. 5, the device may store instructions for the software in a suitable non-transitory computer-readable storage medium, and the Instructions may be executed in hardware using one or more processors to implement the techniques. Either video encoder 20 or video decoder 30 may be integrated as part of a combined encoder/decoder (CODEC) within a single device, for example, as shown in FIG. 1B.

Source device 12 and destination device 14 can be any kind of handheld or stationary device, such as notebook or laptop computers, mobile phones, smart phones, tablet or tablet computers, cameras, desktop computers, set-top boxes, televisions, displays. devices, digital media players, video game consoles, video streaming devices (such as content service servers or content distribution servers), broadcast receiver devices, broadcast transmitter devices, etc.; Either type of operating system may or may not be used. In some cases, source device 12 and destination device 14 may be equipped for wireless communication. Accordingly, source device 12 and destination device 14 may be wireless communication devices.

In some cases, the video coding system 10 shown in FIG. 1A is merely an example, and the techniques herein may be used in a video coding setup that does not necessarily involve any data communication between encoding and decoding devices. For example, it may be applied to video encoding or video decoding). In other examples, data is retrieved from local memory, streamed over a network, and the like. A video encoding device may encode and store data in memory, and/or a video decoding device may retrieve data from memory and decode it. In some examples, encoding and decoding are performed by devices that do not communicate with each other, but simply encode data into memory and/or retrieve data from memory and decode it.

For convenience of explanation, embodiments of the present invention are herein referred to as reference software, for example High-Efficiency Video Coding (HEVC) or Versatile Video Coding (VVC), ITU-T video coding experts Developed by the VCEG (Video Coding Experts Group) and the Joint Collaboration Team on Video Coding (JCT-VC) of the ISO/IEC Motion Picture Experts Group (MPEG) It is described with reference to next generation video coding standards. Those skilled in the art will appreciate that embodiments of the present invention are not limited to HEVC or VVC.

Encoders and Encoding Methods FIG. 2 shows a schematic block diagram of an exemplary video encoder 20 configured to implement the techniques of the present application. In the example of FIG. 2, video encoder 20 includes input 201 (or input interface 201), residual computation unit 204, transform processing unit 206, quantization unit 208, inverse quantization unit 210, and inverse transform processing. unit 212, reconstruction unit 214, loop filter unit 220, decoded picture buffer (DPB) 230, mode selection unit 260, entropy coding unit 270, output 272 (or output interface 272). including. Mode selection unit 260 may include inter prediction unit 244 , intra prediction processing unit 254 , and partition unit 262 . Inter prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). A video encoder 20 as shown in FIG. 2 may also be referred to as a hybrid video encoder or a video encoder with a hybrid video codec.

Residual calculation unit 204 , transform processing unit 206 , quantization unit 208 , and mode selection unit 260 may be referred to as forming the forward signal path of encoder 20 . On the other hand, inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, buffer 216, loop filter 220, decoded picture buffer (DPB) 230, inter prediction unit 244 and intra prediction unit 254: The reverse signal path of video encoder 20, which may also be referred to as forming the reverse signal path of video encoder 20, corresponds to the signal path of the decoder (see decoder 30 in FIG. 3). Inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, loop filter 220, decoded picture buffer (DPB) 230, inter prediction unit 244, and intra prediction unit 254 also provide Also referred to as forming a "built-in decoder".

Pictures and picture partitions (pictures and blocks)
The encoder 20 may for example be arranged to receive, via input 201, pictures 17 (or picture data 17), for example pictures of a sequence of pictures forming a video or a video sequence. The received picture or picture data may also be a preprocessed picture 19 (preprocessed picture data 19). For brevity, the following description refers to picture 17. Picture 17 is also used (particularly in video coding to distinguish the current picture from other pictures, e.g. pictures encoded and/or decoded before the same video sequence, i.e. a video sequence that also contains the current picture. ) may also be called the current picture or the picture to be coded.

A (digital) picture is or can be thought of as a two-dimensional array or matrix of samples with intensity values. A sample in the array may also be called a pixel (short for picture element) or a pel. The number of samples in the horizontal and vertical directions (or axes) of an array or picture defines the size and/or resolution of the picture. For color representation typically three color components are used, ie a picture may be represented as or contain three sample arrays. In the RBG format or color space, a picture contains corresponding red, green and blue sample arrays. However, in video coding each pixel is typically represented by a luminance and chrominance format or color space, e.g. YCbCr containing two chrominance components. The luminance (or luma for short) component Y represents the brightness or gray level intensity (eg, such as in a grayscale picture). On the other hand, the two chrominance (or chroma for short) components Cb and Cr represent chromaticity or color information components. Thus, a picture in YCbCr format includes a luminance sample array of luminance sample values (Y) and two chrominance sample arrays of chrominance values (Cb and Cr). Pictures in RGB format may be converted or converted to YCbCr format and vice versa, a process also known as color conversion or conversion. If the picture is monochrome, the picture may contain only the luminance sample array. Thus, a picture can be, for example, an array of luma samples in monochrome format, or an array of luma samples and two corresponding arrays of chroma samples in 4:2:0, 4:2:2 and 4:4:4 color formats. good.

Embodiments of video encoder 20 may include a picture partition unit (not shown in FIG. 2) configured to partition picture 17 into multiple (typically non-overlapping) picture blocks 203 . These blocks are also root blocks, macroblocks (H.264/AVC) or coding tree blocks (CTB) or coding tree units (CTU) (H.265/HEVC and VVC). may be called. A picture partition unit may vary block sizes between pictures or subsets or groups of pictures to use the same block size for all pictures of a video sequence and corresponding grids that define block sizes, and each It may be arranged to partition the picture into corresponding blocks.

In a further embodiment, the video encoder may be configured to directly receive block 203 of picture 17, eg, one, some or all of the blocks forming picture 17. FIG. Picture block 203 may also be referred to as a current picture block or a picture block to be coded.

Similar to picture 17, picture block 203 is also or can be thought of as a two-dimensional array or matrix of samples with intensity values (sample values), but of smaller dimensions than picture 17. FIG. In other words, block 203 may, for example, store one sample array (eg, luma array for monochrome picture 17, or luma or chroma array for color picture) or three sample arrays (eg, luma array for color picture 17). luma and two chroma arrays), or any other number and/or type of arrays depending on the color format applied. The number of samples in the horizontal and vertical directions (or axes) of block 203 define the size of block 203 . Thus, a block may be, for example, an M×N (M columns×N rows) array of samples, or an M×N array of transform coefficients.

An embodiment of video encoder 20 as shown in FIG. 2 may be configured to encode picture 17 block by block, eg, encoding and prediction are performed block 203 .

An embodiment of video encoder 20 as shown in FIG. 2 may be further configured to partition and/or encode pictures by using slices (also called video slices), where pictures are (typically may be partitioned into or encoded using one or more slices, each slice being one or more blocks (e.g., CTUs) or one or more blocks of Groups (eg, tiles (H.265/HEVC and VVC) or bricks (VVC)) may be included.

An embodiment of video encoder 20, such as that shown in FIG. 2, may partition and/or encode pictures by using slice/tile groups (also called video tile groups) and/or tiles (also called video tiles). , and a picture may be partitioned into or encoded using one or more (typically non-overlapping) slice/tile groups, each slice/tile A group may, for example, contain one or more blocks (eg, CTU) or one or more tiles, each tile may, for example, be rectangular in shape, and one or more blocks (eg, CTU), such as complete blocks. may contain full or partial blocks.

Residual Computation Residual computation unit 204 computes, e.g. It may be configured to compute residual block 205 by subtracting the sample values of prediction block 265 from the sample values to obtain residual block 205 in the sample domain.

Transform Transform processing unit 206 applies a transform, such as a discrete cosine transform (DCT) or a discrete sine transform (DST), to the sample values of residual block 205 to obtain It may be configured to obtain transform coefficients 207 . Transform coefficients 207 may also be referred to as transform residual coefficients and represent residual block 205 in the transform domain.

Transform processing unit 206 may be configured to apply an integer approximation of DCT/DST, such as the transform specified for H.265/HEVC. Compared to orthogonal DCT transforms, such integer approximations are typically scaled by a certain factor. An additional scaling factor is applied as part of the transform process to maintain the norm of the residual blocks processed by the forward and inverse transforms. The scaling factor is typically based on certain constraints such as that the scaling factor is a power of two for shift operations, the bit depth of the transform coefficients, the trade-off between precision and implementation cost, etc. selected by A particular scaling factor is specified, for example, for an inverse transform by inverse transform processing unit 212 (and a corresponding inverse transform by, for example, inverse transform processing unit 312 in video decoder 30), and for example, for an inverse transform by transform processing unit 206 in encoder 20. A corresponding scaling factor for the transform may be specified accordingly.

An embodiment of video encoder 20 (respectively, transform processing unit 206) outputs transform parameters, eg, a transform or types of transforms, which are encoded or compressed, eg, directly or via entropy encoding unit 270. so that, for example, video decoder 30 may receive and use transform parameters for decoding.

Quantization Quantization unit 208 may be configured to quantize transform coefficients 207 to obtain quantized coefficients 209, for example, by applying scalar quantization or vector quantization. Quantized coefficients 209 may also be referred to as quantized transform coefficients 209 or quantized residual coefficients 209 .

The quantization process may reduce the bit depth associated with some or all of transform coefficients 207 . For example, an n-bit transform coefficient may be truncated to an m-bit transform coefficient during quantization, where n is greater than m. The degree of quantization may be changed by adjusting a quantization parameter (QP). For example, in scalar quantization, different scaling may be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to finer quantization. On the other hand, a larger quantization step size corresponds to coarser quantization. The applicable quantization step may be indicated by a quantization parameter (QP). A quantization parameter may, for example, be an index into a predetermined set of applicable quantization step sizes. For example, a small quantization parameter may correspond to fine quantization (small quantization step size), a large quantization parameter may correspond to coarse quantization (large quantization step size), and vice versa. The same is true for Quantization may include division by a quantization step size, and corresponding and/or inverse dequantization by inverse quantization unit 210 may include multiplication by a quantization step size, for example. Some standards, eg, HEVC implementations, may be configured to use a quantization parameter to determine the quantization step size. In general, the quantization step size may be calculated based on the quantization parameter using a fixed-point approximation of an equation involving division. Additional scaling factors for quantization and dequantization to restore the norm of the residual block, which can be changed due to the scaling used in the fixed-point approximation of the equations for the quantization step size and quantization parameter. may be introduced. In one exemplary implementation, inverse transform and dequantization scaling may be combined. Alternatively, a customized quantization table may be used and signaled from the encoder to the decoder in the bitstream, for example. Quantization is a non-reversible operation, and loss increases with increasing quantization step size.

A basic quantization parameter is signaled in the bitstream jointly for all luma and chroma components. However, the quantization parameters for the chrominance components can be shifted from the basic one in a picture/slice or tile group within a picture/coding unit within a slice or tile group level. For this purpose, the bitstream stores PPS offsets for the two chrominance components (pps_cb_qp_offset and pps_cr_qp_offset syntax elements), slice offsets for the two chrominance components (slice_cb_qp_offset and slice_cr_qp_offset) and two offset lists (cb_qp_offset_list and cr_qp_offset_list). These are typically signaled in the PPS to allow the QP offset to be applied for the CU level by sending the CU level index pointing to the table.

Embodiments of video encoder 20 (each, quantization unit 208) are configured to output a quantization parameter (QP), for example encoded directly or via entropy encoding unit 270. may also be used, whereby, for example, video decoder 30 may receive and apply a quantization parameter for decoding.

Inverse Quantization Inverse quantization unit 210 by applying the inverse of the quantization scheme applied by quantization unit 208, for example based on or using the same quantization step size as quantization unit 208. , is configured to apply the inverse quantization of quantization unit 208 to the quantized coefficients to obtain dequantized coefficients 211 . The dequantized coefficients 211, also called dequantized residual coefficients 211, are typically not identical to the transform coefficients due to quantization losses, but may correspond to the transform coefficients 207. .

Inverse transform Inverse transform processing unit 212 performs an inverse transform of the transform applied by transform processing unit 206, such as the inverse discrete cosine transform (DCT) or the discrete sine transform (DST) or other transform. It is configured to apply an inverse transform to obtain a reconstructed residual block 213 (or corresponding dequantized coefficients 213) in the sample domain. Reconstructed residual block 213 may also be referred to as transform block 213 .

Reconstruction Reconstruction unit 214 (e.g., adder or summer 214) adds transform block 213 (i.e., reconstructed residual block 213) to prediction block 265, e.g., reconstructed residual It is arranged to obtain the reconstructed block 215 in the sample domain by adding the sample values of the block 213 and the sample values of the prediction block 265 sample by sample.

Filtering A loop filter unit 220 (or “loop filter” 220 for short) filters the reconstructed block 215 to obtain a filtered block 221, or in general, the reconstructed samples. configured to filter to obtain filtered sample values; The loop filter unit is configured, for example, to smooth pixel transitions or improve video quality. Loop filter unit 220 may include a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as an adaptive loop filter (ALF), a noise suppression filter (NSF, noise suppression filter) or any combination of these. In one example, loop filter unit 220 may include a deblocking filter, an SAO filter and an ALF filter. The order of the filtering process may be deblocking filter, SAO and ALF. In another example, a process called luma mapping with chroma scaling (LMCS) (ie adaptive in-loop reshaper) is added. This process is performed before deblocking. In another example, the deblocking filter process also removes internal sub-block edges, e.g., affine sub-block edges, ATMVP sub-block edges, sub-block transform (SBT) edges and intra sub-partition (ISP, intra sub -partition) may be applied to edges. Although loop filter unit 220 is shown in FIG. 2 as being an in-loop filter, in other configurations loop filter unit 220 may be implemented as a post-loop filter. Filtered block 221 may also be referred to as filtered reconstruction block 221 .

Embodiments of video encoder 20 (respectively, loop filter unit 220) encode loop filter parameters (such as SAO filter parameters or ALF filter parameters or LMCS parameters), e.g., encoded directly or via entropy encoding unit 220. output, so that, for example, decoder 30 may receive and apply the same loop filter parameters or respective loop filters for decoding.

Decoded Picture Buffer Decoded picture buffer (DPB) 230 may be a memory that stores reference pictures, or reference picture data in general, for encoding video data by video encoder 20 . The DPB230 supports dynamic random access memory (DRAM), including synchronous DRAM (SDRAM, synchronous DRAM), magnetoresistive RAM (MRAM, magnetoresistive RAM), resistive RAM (RRAM, resistive RAM), or other types. may be formed by any of a variety of memory devices, such as memory devices of A decoded picture buffer (DPB) 230 may be configured to store one or more filtered blocks 221 . Decoded picture buffer 230 may store other previously filtered blocks of the same current picture or a different picture, e.g., a previously reconstructed picture, e.g., previously reconstructed and filtered block 221 . It may also be configured, e.g., for inter-prediction, full previously reconstructed, i.e. decoded, pictures (and corresponding reference blocks and samples), and/or partially reconstructed A current picture (and corresponding reference blocks and samples) may be provided. A decoded picture buffer (DPB) 230 also stores one or more unfiltered reconstructed blocks 215, or generally, for example, if the reconstructed blocks 215 are not filtered by the loop filter unit 220. may be configured to store unfiltered reconstructed samples or other further processed versions of either the reconstructed blocks or samples.

Mode selection (partition and prediction)
Mode select unit 260 includes partition unit 262, inter-prediction unit 244, and intra-prediction unit 254, and uses original picture data, e.g., original block 203 (current block 203 of current picture 17) and reconstructed picture. Data, e.g., of the same (current) picture and/or from one or more previously decoded pictures, e.g., filtered data from decoded picture buffer 230 or other buffers (e.g., line buffers, not shown) and/or configured to receive or obtain unfiltered reconstructed samples or blocks. The reconstructed picture data is used as reference picture data for prediction, eg, inter-prediction or intra-prediction, to obtain predicted blocks 265 or predictors 265 .

Mode select unit 260 determines or selects a partition (including no partition) and a prediction mode (e.g., intra or inter prediction mode) for the current block prediction mode, for computation of residual block 205 and for reprocessing. It may be configured to generate a corresponding predicted block 265 that is used for reconstruction of the constituent block 215 .

Embodiments of mode selection unit 260 determine the best fit or, in other words, minimum residual (minimum residual means better compression for transmission or storage) or minimum signaling overhead (minimum signaling overhead for transmission or storage). ), or select a partition and prediction mode (e.g., from those supported or available by mode selection unit 260) that considers or balances both. may be configured to Mode selection unit 260 is configured to determine the partition and prediction mode based on rate distortion optimization (RDO), i.e., select the prediction mode that provides the lowest rate distortion. good too. Terms such as "best", "least", "optimal", etc. in this context do not necessarily denote the overall "best", "least", "optimal", etc. It may indicate the satisfaction of termination or selection criteria such as values that are below or other constraints that potentially lead to "sub-optimal selection" but reduce complexity and processing time.

In other words, partition unit 262 may be configured to partition pictures from a video sequence into a sequence of coding tree units (CTUs), CTU 203, for example, quad tree partitions (QT, quad -tree-partitioning), binary partitioning (BT) or triple-tree-partitioning (TT) or any combination of these repeatedly to create smaller block partitions or sub-blocks (-tree-partitioning) again forming a block), performing prediction for each of the block partitions or sub-blocks, the mode selection comprising selecting a tree structure for the partitioned block 203, the prediction mode being: Applies to each block partition or sub-block.

The partitioning (eg, by partition unit 260) and prediction processing (by inter prediction unit 244 and intra prediction unit 254) performed by exemplary video encoder 20 are described in greater detail below.

Partitions Partition unit 262 may be configured to partition pictures from a video sequence into a sequence of coding tree units (CTUs), where partition unit 262 is divided into coding tree units (CTUs). 203 may be partitioned (or divided) into smaller partitions, eg smaller blocks of square or rectangular size. For pictures with three sample arrays, the CTU is constructed from an N×N block of luma samples together with two corresponding blocks of chroma samples. The maximum allowable size of a luma block within a CTU is specified as 128x128 in the upcoming versatile video coding (VVC), but in the future it will be specified as something other than 128x128, such as 256x256. There is a possibility that CTUs of a picture may be clustered/grouped as slices/tile groups, tiles or bricks. A tile covers a rectangular area of the picture, and a tile can be divided into one or more bricks. A brick consists of multiple CTU rows within a tile. A tile that is not partitioned into multiple bricks can be called a brick. However, bricks are a true subset of tiles and are not called tiles. There are two modes of tile groups supported in VVC: raster scan slice/tile group mode and rectangular slice mode. In raster scan tile group mode, a slice/tile group contains a sequence of tiles in a tile raster scan of a picture. In rectangular slice mode, a slice includes a number of bricks of a picture that together form a rectangular region of the picture. The bricks in the rectangular slice are in the order of the brick raster scan of the slice. These smaller blocks (which may also be called sub-blocks) may be further partitioned into smaller partitions. This is also called tree partition or hierarchical tree partition, e.g. the root block at root tree level 0 (hierarchical level 0, depth 0) is recursively partitioned, e.g. It may be partitioned into one or more blocks, e.g., nodes at tree level 1 (hierarchy level 1, depth 1), and these blocks may e.g. , it may again be partitioned into two or more blocks at the next lower tree level, eg, tree level 2 (hierarchy level 2, depth 2), etc., until the partitioning ends. Blocks that are not further partitioned are also called leaf blocks or leaf nodes of the tree. A tree that uses a partition into two partitions is called a binary-tree (BT), a tree that uses a partition into three partitions is called a ternary-tree (TT), A tree that uses partitions into four partitions is called a quad-tree (QT, quad-tree).

For example, a coding tree unit (CTU) may code a CTB for luma samples, two corresponding CTBs for chroma samples for a picture with three sample arrays, or a monochrome picture or three separate color planes and samples. It may also be or include a CTB of samples of a picture coded using the syntax structure used to do this. Correspondingly, a coding tree block (CTB) may be an N×N block of samples, for some value of N, whereby the division of components into CTBs is a partition. A coding unit (CU, coding unit) is for coding a coding block of luma samples, two corresponding coding blocks of chroma samples in a picture with three sample arrays, or a monochrome picture or three separate color planes and samples. may be or include a coding block of samples of a picture that is coded using the syntax structure used in . Correspondingly, a coding block (CB) may be an M×N block of samples, for some values of M and N, whereby the division of the CTB into coding blocks is a partition.

For example, in an HEVC embodiment, a coding tree unit (CTU) may be split into CUs by using a quadtree structure denoted as coding tree. The decision whether to code a picture region using inter-picture (temporal) prediction or intra-picture (spatial) prediction is made at the leaf CU level. Each leaf-CU can be further split into 1, 2 or 4 PUs according to the PU split type. Within one PU, the same prediction process is applied and relevant information is sent to the decoder for each PU. After obtaining the residual block by applying the prediction process based on the PU partition type, the leaf CU is transferred to the transform unit (TU) according to another quadtree structure similar to the coding tree for the CU. can partition.

For example, in embodiments according to the latest video coding standard currently under development called Versatile Video Coding (VVC), combined quadtree nested multitype trees using bipartite and tripartite segmentation structures is used, for example, to partition a coding tree unit. In the coding tree structure within the coding tree unit, a CU can have either a square or rectangular shape. For example, a coding tree unit (CTU) is first partitioned by a quadtree. The quadtree leaf nodes can then be further partitioned according to the multi-type tree structure. There are four split types in the multi-type tree structure: vertical split (SPLIT_BT_VER), horizontal split (SPLIT_BT_HOR), vertical split (SPLIT_TT_VER) and horizontal split (SPLIT_TT_HOR). A multi-type tree leaf node is called a coding unit (CU), and unless the CU is too large for the maximum transform length, this segmentation is used for prediction and transform processing without further partitioning. This means that in most cases, CU, PU and TU have the same block size in a quadtree with nested multi-type tree coding block structure. An exception occurs when the maximum supported transform length is less than the width or height of the CU's color components. VVC formulates a unique signaling mechanism for partitioning information in quadtrees with a coding tree structure of nested multi-type trees. In this signaling mechanism, the coding tree unit (CTU) is treated as the root of a quadtree and is first partitioned according to the quadtree structure. Each quadtree leaf node (when large enough to allow it) is then further partitioned by a multi-type tree structure. In the multitype tree structure, the first flag (mtt_split_cu_flag) is signaled to indicate whether the node is further partitioned or not, and when the node is further partitioned, the second flag (mtt_split_cu_vertical_flag) is the split direction. and then a third flag (mtt_split_cu_binary_flag) is signaled to indicate whether the split is a 2-split or a 3-split. Based on the values of mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the CU's multi-type tree split mode (MttSplitMode) can be derived by the decoder based on a predefined rule or table. As shown in FIG. 6, in certain designs, such as 64×64 luma block and 32×32 chroma pipeline designs in a VVC hardware decoder, when either the width or height of the luma coding block is greater than 64 , TT division is prohibited. TT splitting is also prohibited when either the width or height of the chroma coding block is greater than 32. A pipeline design divides a picture into virtual pipeline data units (VPDUs), which are defined as non-overlapping units within a picture. In a hardware decoder, successive VPDUs are processed simultaneously by multiple pipeline stages. Since the VPDU size is roughly proportional to the buffer size in most pipeline stages, it is important to keep the VPDU size small. For most hardware decoders, the VPDU size can be set to the maximum transform block (TB) size. However, in VVC, ternary tree (TT) and binary tree (BT) partitions can lead to an increase in VPDU size.

Furthermore, when part of a treenode block crosses the lower or right picture boundary, the treenode block is forced to split until all samples of all coded CUs lie within the picture boundary. should be noted.

As an example, an Intra Sub-Partitions (ISP) tool may split a luma intra prediction block vertically or horizontally into 2 or 4 sub-partitions, depending on the block size.

In one example, mode selection unit 260 of video encoder 20 may be configured to perform any combination of the partitioning techniques described herein.

As noted above, video encoder 20 is configured to determine or select the best or optimal prediction mode from a (eg, predetermined) set of prediction modes. A set of prediction modes may include, for example, intra-prediction modes and/or inter-prediction modes.

Intra Prediction The set of intra prediction modes includes 35 different intra prediction modes, non-directional modes such as DC (or average) mode and planar mode, or directional modes such as those defined in HEVC. or 67 different intra-prediction modes, e.g. non-directional modes such as DC (or average) mode and planar mode, or directional modes such as defined for VVC . As an example, some conventional angular intra-prediction modes are adaptively replaced by wide-angle intra-prediction modes for non-square blocks, eg, as defined in VVC. As another example, only the long sides are used to compute the average for non-square blocks to avoid splitting operations for DC prediction. In addition, the results of planar mode intra prediction may be further modified by a position dependent intra prediction combination (PDPC) method.

Intra-prediction unit 254 is configured to use reconstructed samples of neighboring blocks of the same current picture to generate intra-prediction blocks 265 according to an intra-prediction mode of a set of intra-prediction modes.

Intra prediction unit 254 (or generally mode selection unit 260) selects intra prediction parameters (or generally the intra prediction mode selected for the block) in the form of syntax elements 226 for inclusion in encoded picture data 21. information) to entropy encoding unit 270 so that, for example, video decoder 30 may receive and use the prediction parameters for decoding.

Inter Prediction A set (or possible) of inter prediction modes are available reference pictures (i.e., previous at least partially decoded pictures stored in DBP 230, for example) and other inter prediction parameters, e.g. , whether the entire reference picture is used to search for the best matching reference block, or only part of the reference picture is used, e.g., the search window area around the area of the current block, and/or For example, depending on whether pixel interpolation is applied, such as half/semi-pel, quarter-pel and/or 1/16-pel interpolation.

In addition to the prediction modes mentioned above, skip mode, direct mode and/or other inter-prediction modes may be applied.

For example, in extended merge prediction, the merge candidate list for such a mode consists of the following five types of candidates in order: spatial MVPs from spatial neighbor CUs, temporal MVPs from co-located CUs, history from FIFO table It is constructed by including the base MVP, paired average MVP and zero MV. Furthermore, bilateral matching-based decoder-side motion vector refinement (DMVR) may be applied to improve the accuracy of MVs in merge mode. MMVD, Merge mode with MVD, is derived from Merge mode with motion vector difference (MVD). The MMVD flag is signaled immediately after sending the skip and merge flags to specify whether MMVD mode is used for the CU. In addition, a CU-level adaptive motion vector resolution (AMVR) scheme may be applied. AMVR allows the MVD of a CU to be coded with different precisions. Depending on the prediction mode for the current CU, the current CU's MVD can be adaptively selected. When a CU is coded in merge mode, a combined inter/intra prediction (CIIP) mode may be applied to the current CU. A weighted average of the inter and intra prediction signals is performed to obtain the CIIP prediction. In affine motion compensated prediction, the affine motion field of a block is described by the motion information of two control points (4 parameters) or three control point motion vectors (6 parameters). Sub-block-based temporal motion vector prediction (SbTMVP) is similar to temporal motion vector prediction (TMVP) in HEVC, but sub-CUs within the current CU predict the motion vector of Bi-directional optical flow (BDOF), formerly called BIO, is a simpler version that requires a very low amount of computation, especially in terms of the number of multiplications and the size of the multipliers. In triangular partition mode, in such mode the CU is evenly divided into two triangular shaped partitions using either diagonal or off-diagonal partitioning. Moreover, bi-prediction mode is extended beyond simple averaging to allow weighted averaging of the two prediction signals.

Inter-prediction unit 244 may include a motion estimation (ME) unit and a motion compensation (ME) unit (both not shown in FIG. 2). The motion estimation unit uses picture block 203 (current block 203 of current picture 17) and decoded picture 231, or at least one or more previously reconstructed blocks, e.g. It may be configured to receive or obtain reconstructed blocks of other/different previous decoded pictures 231 . For example, a video sequence may include a current picture and a previous decoded picture 231, or in other words, the current picture and the previous decoded picture 231 may be part of or form a sequence of pictures forming the video sequence. You may

Encoder 20 selects a reference block from, for example, a plurality of reference blocks of the same or different pictures of a plurality of other pictures, the reference picture (or reference picture index) and/or the position (x,y coordinates) of the reference block and It may be arranged to provide the offset (spatial offset) to the position of the current block as an inter-prediction parameter to the motion estimation unit. This offset is also called a motion vector (MV).

The motion compensation unit is configured to obtain, eg, receive, inter-prediction parameters and perform inter-prediction based on or using the inter-prediction parameters to obtain inter-prediction blocks 265 . Motion compensation performed by the motion compensation unit may include fetching or generating a prediction block based on motion/block vectors determined by motion estimation, and possibly performing interpolation to sub-pixel accuracy. . Interpolation filtering may generate additional pixel samples from known pixel samples, thus potentially increasing the number of candidate predictive blocks that can be used to code a picture block. Upon receiving a motion vector for the PU of the current picture block, the motion compensation unit may find the predictive block pointed to by the motion vector in one of the reference picture lists.

Motion compensation unit may also generate syntax elements associated with the blocks and video slices for use by video decoder 30 in decoding the picture blocks of the video slices. Tile groups and/or tiles and respective syntax elements may be generated or used in addition to or alternative to slices and respective syntax elements.

Entropy Coding Entropy encoding unit 270 applies, for example, an entropy encoding algorithm or scheme (eg, Variable length coding (VLC) scheme, context adaptive VLC scheme (CAVLC), arithmetic coding scheme, binarization, context adaptive binary arithmetic coding (CABAC) , syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding or other entropy coding methods or techniques). or bypassed (uncompressed) to obtain encoded picture data 21, which may be output via output 272, for example, in the form of an encoded bitstream 21, whereby, for example, video decoder 30 may , may receive and use the parameters for decoding. Encoded bitstream 21 may be transmitted to video decoder 39 or may be stored in memory for later transmission or retrieval by video decoder 30 .

Other structural variations of video encoder 20 can be used to encode the video stream. For example, non-transform-based encoder 20 can directly quantize the residual signal without transform processing unit 206 for a particular block or frame. In other implementations, encoder 20 may have quantization unit 208 and inverse quantization unit 210 combined into a single unit.

Decoders and Decoding Methods FIG. 3 shows an example of a video decoder 30 configured to implement the techniques of the present application. Video decoder 30 is configured to receive encoded picture data 21 (eg, encoded bitstream 21 ), eg encoded by encoder 20 , to obtain decoded pictures 331 . The coded picture data or bitstream includes information for composing the coded picture data, eg, data representing picture blocks and associated syntax elements of coded video slices (and/or tile groups or tiles).

In the example of FIG. 3, decoder 30 includes entropy decoding unit 304, inverse quantization unit 310, inverse transform processing unit 312, reconstruction unit 314 (eg, summer 314), loop filter 320, decoded picture It includes a buffer (DBP, decoded picture buffer) 330 , a mode application unit 360 , an inter prediction unit 344 and an intra prediction unit 354 . Inter prediction unit 344 may be or include a motion compensation unit. Video decoder 30 may, in some examples, perform a decoding path generally reciprocal to the encoding path described with respect to video encoder 100 from FIG.

Inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, loop filter 220, decoded picture buffer (DPB) 230, inter prediction unit 344 and intra prediction unit 354, as described with respect to encoder 20. may also be referred to as forming the “built-in decoder” of video encoder 20 . Thus, inverse quantization unit 310 may be functionally identical to inverse quantization unit 110, inverse transform processing unit 312 may be functionally identical to inverse transform processing unit 212, and reconstruction unit 314 may be functionally identical to reconstruction unit 214. Functionally identical, loop filter 320 may be functionally identical to loop filter 220 and decoded picture buffer 330 may be functionally identical to decoded picture buffer 230 . Accordingly, the description provided for each unit and function of video 20 encoder applies to each unit and function of video decoder 30 correspondingly.

Entropy Decoding Entropy decoding unit 304 parses the bitstream 21 (or encoded picture data 21 in general) and, for example, performs entropy decoding on the encoded picture data 21 to, for example, quantize Coefficients 309 and/or decoded coding parameters (not shown in FIG. 3), such as inter-prediction parameters (eg, reference picture indices and motion vectors), intra-prediction parameters (eg, intra-prediction modes or indices), transform parameters. , quantization parameters, loop filter parameters and/or other syntax elements. Entropy decoding unit 304 may be configured to apply decoding algorithms or schemes corresponding to encoding schemes such as those described with respect to entropy encoding unit 270 of encoder 20 . Entropy decoding unit 304 may be further configured to provide inter-prediction parameters, intra-prediction parameters and/or other syntax elements to mode application unit 360 and other parameters to other units of decoder 30. good. Video decoder 30 may receive video slice-level and/or video block-level syntax elements. Tile groups and/or tiles and respective syntax elements may be received and/or used in addition to or alternative to slices and respective syntax elements.

Inverse Quantization Inverse quantization unit 310 extracts a quantization parameter (QP) (or inverse quantization in general) from encoded picture data 21 (e.g., by parsing and/or decoding by entropy decoding unit 304). information about quantization) and the quantized coefficients, and based on the quantization parameter, apply inverse quantization to the decoded quantized coefficients 309, also referred to as transform coefficients 311. It may be configured to obtain dequantized coefficients 311 . The inverse quantization process is determined by video encoder 20 for each video block within a video slice (or tile or group of tiles) to determine the degree of quantization, as well as the degree of inverse quantization to be applied. may include the use of a quantization parameter.

Inverse Transform An inverse transform processing unit 312 receives dequantized coefficients 311, also called transform coefficients 311, and transforms them into dequantized coefficients 311 to obtain a reconstructed residual block 213 in the sample domain. may be configured to apply Reconstructed residual block 213 may also be referred to as transform block 313 . The transform may be an inverse transform, such as an inverse DCT, an inverse DST, an inverse integer transform, or a conceptually similar inverse transform process. Inverse transform processing unit 312 receives transform parameters or corresponding information from encoded picture data 21 (e.g., by parsing and/or decoding by entropy decoding unit 304) into dequantized coefficients 311. It may be further configured to determine the transformation to be applied.

Reconstruction Reconstruction unit 314 (e.g., adder or summer 314) adds reconstructed residual block 313 to prediction block 365, e.g., the sample values of reconstructed residual block 313 and the prediction and the sample values of block 365 to obtain reconstructed block 315 in the sample domain.

filtering
A loop filter unit 320 (either in the coding loop or after the coding loop) filters the reconstructed block 315 to obtain a filtered block 321, eg smoothing pixel transitions. or to improve video quality. Loop filter unit 320 may include a deblocking filter, a sample-adaptive offset (SAO) filter, or one or more other filters, such as an adaptive loop filter (ALF), a noise suppression filter (NSF, noise suppression filter) or any combination of these. In one example, loop filter unit 220 may include a deblocking filter, an SAO filter and an ALF filter. The order of the filtering process may be deblocking filter, SAO and ALF. In another example, a process called luma mapping with chroma scaling (LMCS) (ie adaptive in-loop reshaper) is added. This process is performed before deblocking. In another example, the deblocking filter process also removes internal sub-block edges, e.g., affine sub-block edges, ATMVP sub-block edges, sub-block transform (SBT) edges and intra sub-partition (ISP, intra sub -partition) may be applied to edges. Although loop filter unit 320 is shown in FIG. 3 as being an in-loop filter, in other configurations loop filter unit 320 may be implemented as a post-loop filter.

Decoded Picture Buffer The decoded video block 321 of a picture then stores the decoded picture 331 as a reference picture for later motion compensation for other pictures and/or for output for respective displays. stored in

Decoder 30 is configured to output decoded picture 331 via output 312, for example, for presentation or viewing by a user.

Prediction Inter-prediction unit 344 may be identical to inter-prediction unit 244 (particularly motion compensation unit), intra-prediction unit 354 may be functionally identical to inter-prediction unit 254, partition and/or prediction parameters or coding Based on respective information received from picture data 21 (eg, by parsing and/or decoding by entropy decoding unit 304), segmentation or partition decisions and predictions are performed. Mode application unit 360 performs block-by-block prediction (intra or inter prediction) based on reconstructed pictures, blocks or respective (filtered or unfiltered) samples to obtain predicted blocks 365. It may be configured as

When a video slice is coded as an intra-coded (I, intra-coded) slice, intra-prediction unit 354 of mode adaptation unit 360 uses the signaled intra-prediction mode and the predictions from previously decoded blocks of the current picture. Based on the data, it is configured to generate prediction blocks 365 for picture blocks of the current video slice. When a video picture is coded as an inter-coded (i.e., B or P) slice, an inter-prediction unit 344 (e.g., motion compensation unit) of mode application unit 360 receives motion vectors from entropy decoding unit 304 and other is configured to generate a prediction block 365 for the video block of the current video slice based on the syntax elements of . For inter prediction, a predictive block may be generated from one of the reference pictures in one of the reference picture lists. Video decoder 30 may construct the reference frame lists, List 0 and List 1, using default construction techniques based on reference pictures stored in DPB 330 . The same or similar may be applied to or by embodiments that use tile groups (eg, video tile groups) and/or tiles (eg, video tiles) in addition to or instead of slices (eg, video slices). may be applied, for example the video may be coded using I, P or B tile groups and/or tiles.

Mode application unit 360 is configured to determine prediction information for video blocks of the current video slice by parsing motion vectors or related information and other syntax elements, and predictive blocks for the current video block being decoded. Use prediction information to generate For example, mode application unit 360 uses some of the received syntax elements to determine the prediction mode (e.g., intra or inter prediction) used to code the video blocks of the video slice, the inter prediction slice type (e.g., B slice, P slice or GPB slice), configuration information for one or more of the slice's reference picture lists, motion vectors for each inter-coded video block of the slice, for each inter-coded video block of the slice and other information for decoding the video blocks in the current video slice. The same or similar techniques apply to or by embodiments that use tile groups (eg, video tile groups) and/or tiles (eg, video tiles) in addition to or instead of slices (eg, video slices). may be applied, for example the video may be coded using I, P or B tile groups and/or tiles.

An embodiment of video decoder 30 as shown in FIG. 3 may be configured to partition and/or decode pictures by using slices (also called video slices), where pictures are (typically may be partitioned into or decoded using one or more non-overlapping slices, each slice being one or more blocks (e.g., CTU) or groups of one or more blocks (e.g., , tiles (H.265/HEVC and VVC or bricks (VVC)).

An embodiment of video decoder 30, such as that shown in FIG. 3, is configured to partition and/or decode pictures by using slice/tile groups (also called video tile groups) and/or tiles (also called video tiles). A picture may be partitioned into or decoded using one or more (typically non-overlapping) slices/tile groups, each slice/tile group being: For example, it may include one or more blocks (eg, CTU) or one or more tiles, each tile may be, for example, rectangular in shape, and one or more blocks (eg, CTU), such as complete or partial. block.

Other variations of video decoder 30 can be used to decode encoded picture data 21 . For example, decoder 30 can generate an output video stream without loop filter unit 320 . For example, non-transform-based decoder 30 can directly inverse quantize the residual signal without inverse transform processing unit 312 for a particular block or frame. In other implementations, video decoder 30 may have inverse quantization unit 310 and inverse transform processing unit 312 combined into a single unit.

It should be appreciated that in the encoder 20 and decoder 30 the processing result of the current step may be further processed and then output to the next step. For example, after interpolation filtering, motion vector derivation or loop filtering, further operations such as clipping or shifting may be performed on the result of interpolation filtering, motion vector derivation or loop filtering.

Further operations are applied to the derived motion vectors of the current block (including but not limited to control point motion vectors in affine mode, affine, planar, sub-block motion vectors in ATMVP mode, temporal motion vectors, etc.). It should be noted that For example, motion vector values are constrained to a predetermined range according to their representation bits. If the motion vector representation bits are bitDepth, the range is -2^(bitDepth-1) to 2^(bitDepth-1)-1, where "^" means power. For example, if bitDepth is set equal to 16, the range is -32768 to 32767, and if bitDepth is set equal to 18, the range is -131072 to 131071. For example, the derived motion vector values (e.g., the MVs of four 4x4 sub-blocks within one 8x8 block) are such that the maximum difference between the integer parts of the MVs of the four 4x4 sub-blocks is , constrained to be no more than N pixels, such as no more than 1 pixel. Here we provide two methods to constrain motion vectors according to bitDepth.

FIG. 4 is a schematic diagram of a video coding device 400 according to an embodiment of the disclosure. Video coding device 400 is suitable for implementing the disclosed embodiments as described herein. In embodiments, video coding device 400 may be a decoder such as video decoder 30 of FIG. 1A or an encoder such as video encoder 20 of FIG. 1A.

Video coding device 400 includes entry port 410 (or input port 410) and receiver unit (Rx) 420 for receiving data, and a processor, logic unit or central processing unit (CPU) for processing data. unit) 430, transmitter unit (Tx) 440 and exit port 450 (or output port 450) for transmitting data, and memory 460 for storing data. Video coding device 400 also includes optical-to-electrical (OE) coupled to entry port 410, receiver unit 420, transmitter unit 440 and exit port 450 for the egress or ingress of optical or electrical signals. ) components and electrical-to-optical (EO) components.

Processor 430 is implemented in hardware and software. Processor 430 may be implemented as one or more CPU chips, cores (eg, multi-core processors), FPGAs, ASICs, and DSPs. Processor 430 communicates with ingress port 410 , receiver unit 420 , transmitter unit 440 , egress port 450 and memory 460 . Processor 430 includes coding module 470 . Coding module 470 implements the disclosed embodiments described above. For example, coding module 470 implements, processes, prepares, or provides various coding operations. Accordingly, what is included in coding module 470 provides substantial improvements in the functionality of video coding device 400, resulting in conversion of video coding device 400 to different states. Alternatively, coding module 470 is implemented as instructions stored in memory 460 and executed by processor 430 .

Memory 460, which may include one or more disks, tape drives and solid state drives, stores instructions and data for storing such programs when the programs are selected for execution and for reading during execution of the programs. may be used as an overflow data storage device for storing Memory 460 may be, for example, volatile and/or non-volatile and may include read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and so on. memory) and/or static random-access memory (SRAM).

FIG. 5 is a simplified block diagram of apparatus 500 that may be used as one or both of source device 12 and destination device 14 from FIG. 1A, according to an exemplary embodiment.

Processor 502 in device 500 may be a central processing unit. Alternatively, processor 502 may be any other type of device or devices, now existing or later developed, capable of manipulating or processing information. Although the disclosed implementations can be implemented in a single processor, such as processor 502, as shown, advantages in speed and efficiency can be achieved using more than one processor.

The memory 504 in the device 500 can be a read only memory (ROM) device or a random access memory (RAM) device in implementations. Any other suitable type of storage device can be used as memory 504 . Memory 504 may contain code and data 506 that are accessed by processor 502 using bus 512 . Memory 504 may further include an operating system 508 and application programs 510, including at least one program that enables processor 502 to perform the methods described herein. For example, application programs 510 may include applications 1-N, which further include video coding applications that perform the methods described herein.

Apparatus 500 may also include one or more output devices, such as display 518 . Display 518, in one example, may be a touch-sensitive display that combines the display with a touch-sensitive element operable to sense touch input. A display 518 can be coupled to processor 502 via bus 512 .

Although shown here as a single bus, bus 512 of device 500 may comprise multiple buses. Further, secondary storage 514 can be directly coupled to other components of device 500 or can be accessed over a network and can be a single integrated unit such as a memory card or multiple units such as multiple memory cards. can contain. Accordingly, device 500 can be implemented in a wide variety of configurations.

Joint Coding of Chrominance Residuals (JVET-M0305)
Joint coding of chrominance residuals proposes a chrominance residual coding mode in which a single joint residual block is used to describe the residuals of both Cb and Cr blocks within the same transform unit. When joint residual mode is active, the indicated joint residual is added to the Cb prediction block and subtracted from the Cr prediction block. On the encoder side, the algorithm uses the average of the positive Cb and negative Cr residuals as input to the transform and quantization process.

The concept of joint coding of Cb and Cr is based on the fact that Cb and Cr residuals are inversely correlated with each other. In this mode there is a single residual shown for the two chrominance blocks of the transform unit. The indicated residuals are added to the predicted block in the first channel (typically representing Cb) and subtracted from the predicted block in the second channel (typically representing Cr).

The joint residual mode is indicated by a flag in the bitstream if the coded block flag (cbf) for both Cb and Cr is true. If the mode is activated, a single residual block is decoded. The bitstream syntax and decoding process of joint residual block follows that of Cb residual in VTM-3. The Cr block residual is generated by canceling the decoded joint residual. Since a single residual is used to represent the residual of two blocks, the chroma QP offset parameter is reduced by 2 when joint chrominance residual mode is active.

On the encoder side, the average of positive Cb residual and negative Cr residual is used as joint residual.
resJoint=(resCb-resCr)/2

Joint Chroma Residual Coding with Multimodes (JVET-N0282)
Joint chroma residual coding in multiple modes is an extension of joint chroma residual coding proposed in JVET-M0305. In contrast to JVET-M0305, where the addition of one joint chroma residual coding mode (given by Cr=−Cb) is proposed, this contribution is divided into three for joint chroma residual coding with different mixing factors. Modes (given by Cr=±Cb/2, Cr=±Cb, Cb=±Cr/2) are proposed. The code used to derive the second chroma residual is coded in the tile group header. The use of joint chroma-coding mode is indicated by a TU-level flag, and the selected mode is implicitly indicated by the chroma-coded block flag.

Three modes for joint chroma residual coding are supported. In all of these three joint chroma residual coding modes, a single chroma transform block is coded (using VTM4 residual coding) and other blocks of chroma residual samples use simple arithmetic operations. is derived as

Three joint chroma coding modes are supported:
Mode 1: Cb is coded and Cr is derived according to Cr=CSign*Cb/2 Mode 2: Cb is coded and Cr is derived according to Cr=CSign*Cb Mode 3: Cr is coded and Cb is derived Cb=CSign*Cr/2 where CSign represents the code used to derive the second chroma residual block. CSign is indicated using the Tile Group Header syntax element, which is either -1 or 1. Note that if CSign is equal to -1, mode 2 is the same as the joint chroma coding mode proposed in JVET-M0305.

The use of joint chroma residual coding is indicated by the TU level flag tu_joint_chroma_residual_flag. This flag is present if either or both of the two chroma coded block flag (CBF) syntax elements are equal to one. If tu_joint_chroma_residual_flag is equal to 1, then one of the joint chroma residual coding modes is used. The mode used is indicated by Chroma CBF, as specified in the table below.

If the joint chroma coding mode is selected, the QP for coding the joint chroma components is reduced by 1 (for modes 1 and 3) or 2 (for mode 2).

On the encoder side, the joint chroma residual is derived by corresponding downmixing of the Cb and Cr residuals. One of the three supported chroma coding modes is mixed distortion (i.e., downmixing the Cb and Cr residuals first, then reconstructing the Cb and Cr residuals from the joint chroma residuals without quantization). or distortion obtained by upmixing). Only preselected modes are tested as further modes in the mode determination process (ie using transform, quantization and entropy coding). Due to the low-complexity pre-selection of one candidate mode for each TU, the encoding time does not change substantially for JVET-M0305.

A tile group header syntax element indicating the sign (CSign) for deriving the second chroma component analyzes the correlation between the high-pass filtered versions of the original Cb and Cr components for the tile group. determined by

Joint Chroma Residual Coding Test CE7-2.1/2 discusses an extension of the joint chroma residual coding technique originally described in JVET-M0305 by providing a wider range of joint coding parameters and modes. Specifically, the following are discussed.

1) Three instead of one joint chroma coding mode, signaled using the chroma coded block flag (CBF) syntax element, as in CE7-2.1: JVET-N0282 construct 1 Extending VTM 5.0 by enabling These coding modes apply a simplified cross-component rotation transform, as shown in Table 1.

2) CE7-2.2: As for CE7-2.1, but for joint chroma coding mode signaled via CBF _Cb =1, CBF _Cr =1, two joint chroma coding modes (instead of only one) Allow the coding residual signal to be transmitted. The rotation transform associated with this mode represents the Hadamard transform (see Table 2).

Normative Reconstruction of Chroma Residuals Thus, in extension, in contrast to M0305, where the addition of one single-channel joint chroma residual coding mode (given by Cr=−Cb) is proposed, different Three modes are supported for joint chroma residual coding with mixed factors. This mode is further characterized by the code (ie, joint decoding weight code CSign in JVET-N0282) used to derive the second chroma residual that is coded in the slice header. The use (activation) of the joint chroma coding mode is indicated by the TU level flag tu_joint_cbcr_residual_flag and the selected mode is implicitly indicated by the chroma CBF. The flag tu_joint_cbcr_residual_flag is present if either or both of the chroma CBFs for the TU are 1.

Similar to VTM 5.0, a chroma QP offset (coded at slice level) can be signaled for specific use in a particular joint chroma coding mode. When the corresponding joint chroma coding mode (modes 2 and 4 in the description below) is active in a TU, this chroma QP offset is the applied luma-derived chroma QP during quantization and decoding of that TU. added to. For the other modes (modes 1 and 3 in the discussion below), the chroma QPs are derived in the same way as for conventional Cb or Cr blocks.

A canonical reconstruction process of the chroma residuals (resCb and resCr) from the transmitted transform block (resJointC, or resJointC1 and resJointC2 in CE7-2.2) is summarized in the table below.

Non-Normative Determination of Joint Chroma Residuals in Encoders In the encoder implementations investigated, joint chroma components are derived as described below. Depending on the mode (listed in the table above), resJointC{1,2} is generated by the encoder as follows.

• If mode equals 4 (Hadamard transform, 2 residuals are transmitted), the joint residual is determined according to:
resJointC1[x][y]=(resCb[x][y]+CSign*resCr[x][y])/2
resJointC2[x][y]=(resCb[x][y]-CSign*resCr[x][y])/2

• Otherwise, if the mode is equal to 2 (single residual with reconstruction Cb=C, Cr=CSsign*C), the joint residual is determined according to
resJointC[x][y]=(resCb[x][y]+CSign*resCr[x][y])/2

• Otherwise, if the mode is equal to 1 (single residual with reconstruction Cb=C, Cr=(CSign*C)/2), the joint residual is determined according to
resJointC[x][y]=(4*resCb[x][y]+2*CSign*resCr[x][y])/5

• Otherwise (mode equals 3, ie single residual, reconstructed Cr=C, Cb=(CSign*C)/2), the joint residual is determined according to:
resJointC[x][y]=(4*resCr[x][y]+2*CSign*resCb[x][y])/5

Further details on how the encoder selects joint modes for TUs are given in Sec. 2.4 of JVET-N0282.

First Embodiment of the Present Disclosure In the first embodiment of the present disclosure, signaling of PPS and slice header QP offsets for chrominance components is performed independently of each other according to the following table.

where pic_parameter_set_rbsp indicates the picture parameter set RBSP syntax,
pps_cb_qp_offset and pps_cr_qp_offset specify the offset to the luma quantization parameter _Qp'Y used to derive _Qp'Cb and _Qp'Cr , respectively;
The values of pps_cb_qp_offset and pps_cr_qp_offset are in the range of -12 to +12,
When sps_chroma_format_idc is equal to 0, pps_cb_qp_offset and pps_cr_qp_offset are not used in the decoding process and the decoder shall ignore these values,
When absent, the values of pps_cb_qp_offset and pps_cr_qp_offset are assumed to be equal to 0,
sps_joint_cbcr_enabled_flag equal to 1 specifies that joint coding of chroma residuals is enabled for the coded layer video sequence (CLVS);
sps_joint_cbcr_enabled_flag equal to 0 specifies that joint coding of chroma residuals is disabled for CLVS,
When absent, the value of sps_joint_cbcr_enabled_flag is assumed to be equal to 0.
pps_joint_cbcr_qp_offset_value specifies the offset to the luma quantization parameter Qp' _Y used to derive Qp' _CbCr ,
The value of pps_joint_cbcr_qp_offset_value is in the range between -12 and +12.

In the prior art, the flag pps_slice_chroma_qp_offsets_present_flag is signaled in the PPS level syntax, which controls whether there are any further offsets signaling in the slice header, i. The value of pps_slice_chroma_qp_offsets_present_flag should be checked to determine if there are any further offsets signaled in the header. Compared to the prior art, in the first embodiment, the flag pps_slice_chroma_qp_offsets_present_flag is no longer signaled, in other words there is always an offset signaled in the slice header, so the decoder does not check the value of pps_slice_chroma_qp_offsets_present_flag. , knowing that there is an additional offset signaled in the slice header, in other words both the PPS level syntax and the slice header syntax always contain the offset. Decoding/signaling of the chrominance QP offsets in the slice header is therefore easier.

Second Embodiment of the Present Disclosure In a second embodiment of the present disclosure, signaling of PPS and slice header QP offsets for chrominance components for JCCR mode and signaling of SPS chroma mapping information for JCCR coding mode are , is executed depending on the SPS JCCR control flags according to the table below. For example, the sps JCCR control flag is sps_joint_cbcr_enabled_flag signaled in SPS level syntax (eg, seq_parameter_set_rbsp syntax).

The value of 'i' is determined based on the value of sps_joint_cbcr_enabled_flag, thus recognizing that redundant signaling of JCCR syntax elements can be avoided when JCCR tools are disabled.

It can be recognized that the signaling/decoding of pps_joint_cbcr_qp_offset depends on sps_joint_cbcr_enabled_flag, ie pps_joint_cbcr_qp_offset can be signaled or decoded only when the value of sps_joint_cbcr_enabled_flag is true (eg, 1). Due to the conditional signaling of pps_joint_cbcr_qp_offset, less information needs to be signaled, thus saving resources.

where sps_joint_cbcr_enabled_flag equal to 1 specifies that joint coding of chroma residuals is enabled for the coded layer video sequence (CLVS), and sps_joint_cbcr_enabled_flag equal to 0 specifies that joint coding of chroma residuals is enabled. specifies that is disabled for the code layer video sequence, and when absent, the value of sps_joint_cbcr_enabled_flag is assumed to be equal to zero.

seq_parameter_set_rbsp indicates the Raw Byte Sequence Payload (RBSP) syntax for the sequence parameter set, sps_num_points_in_qp_table_minus1[i] plus 1 is used to describe the ith chroma QP mapping table It should also be noted that the value of sps_num_points_in_qp_table_minus1[i] ranges from 0 to 36, and that when absent, the value of sps_num_points_in_qp_table_minus1[0] is assumed to be equal to 0. be.

The following is a description of the application of the encoding and decoding methods shown in the above embodiments and systems using them.

FIG. 6 is a block diagram showing a content supply system 3100 for realizing a content distribution service. This content delivery system 3100 includes a capture device 3102 , a terminal device 3106 and optionally a display 3126 . Capture device 3102 communicates with terminal device 3106 over communication link 3104 . The communication link may include the communication channel 13 described above. Communication link 3104 includes, but is not limited to, WIFI, Ethernet, cable, wireless (3G/4G/5G), USB or any combination of these types.

The capture device 3102 may generate data and encode the data by encoding methods such as those shown in the embodiments above. Alternatively, the capture device 3102 may deliver the data to a streaming server (not shown in the drawing), which encodes the data and transmits the encoded data to the terminal device 3106. Capture devices 3102 include, but are not limited to, cameras, smart phones or pads, computers or laptops, video conferencing systems, PDAs, in-vehicle devices, or any combination thereof. For example, capture device 3102 may include source device 12 as described above. When the data includes video, video encoder 20 included in capture device 3102 may actually perform the video encoding process. When the data includes audio (ie, voice), an audio encoder included in capture device 3102 may actually perform the audio encoding process. In some practical scenarios, capture device 3102 delivers encoded video and audio data by multiplexing them together. In other practical scenarios, for example in video conferencing systems, encoded audio data and encoded video data are not multiplexed. Capture device 3102 separately delivers encoded audio data and encoded video data to terminal device 3106 .

In the content supply system 3100, the terminal device 310 receives and reproduces encoded data. The terminal device 3106 includes a smartphone or pad 3108 capable of decoding the above encoded data, a computer or laptop 3110, a network video recorder (NVR)/digital video recorder (DVR) 3112, a TV 3114, a set data, such as a set top box (STB) 3116, a video conferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, an in-vehicle device 3124, or any combination thereof; It may be a device with receiving and restoring capabilities. For example, terminal device 3106 may include destination device 14 as described above. When the encoded data includes video, video decoder 30 included in the terminal device is prioritized to perform video decoding. When the encoded data includes audio, an audio decoder included in the terminal device is prioritized to perform the audio decoding process.

Terminal devices with their own displays, such as smartphones or pads 3108, computers or laptops 3110, network video recorders (NVRs)/digital video recorders (DVRs) 3112, TVs 3114, personal digital assistants ( PDA, personal digital assistant) 3122 or in-vehicle device 3124, the terminal device can provide the decoded data to its display. For terminal devices without a display, such as STB 3116, videoconferencing system 3118 or video surveillance system 3120, external display 3126 is contacted to receive and display the decoded data.

When each device in this system performs encoding or decoding, it can use a picture encoding device or a picture decoding device, as shown in the above embodiments.

FIG. 7 is a diagram illustrating the structure of an example terminal device 3106 . After the terminal device 3106 receives the stream from the capture device 3102, the protocol processing unit 3202 analyzes the transmission protocol of the stream. Protocols include Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live Streaming Protocol (HLS), MPEG-DASH, Real Time Transport Protocol ( Real-time Transport protocol (RTP), Real Time Messaging Protocol (RTMP) or any combination thereof of any kind.

After the protocol processing unit 3202 processes the stream, a stream file is generated. The files are output to demultiplexing unit 3204 . A demultiplexing unit 3204 can separate the multiplexed data into encoded audio data and encoded video data. As noted above, in some practical scenarios, for example, in video conferencing systems, encoded audio data and encoded video data are not multiplexed. In this situation, the encoded data is sent to video decoder 3206 and audio decoder 3208 without passing through demultiplexing unit 3204 .

Through a demultiplexing process, a video elementary stream (ES), an audio ES and optional subtitles are generated. A video decoder 3206, including the video decoder 30 as described in the above embodiments, decodes the video ES by decoding methods as shown in the above embodiments to generate video frames and passes this data to the synchronization unit 3212. supply. Audio decoder 3208 decodes the audio ES to generate audio frames and provides this data to synchronization unit 3212 . Alternatively, the video frames may be stored in a buffer (not shown in FIG. 7) before being provided to synchronization unit 3212 . Similarly, audio frames may be stored in a buffer (not shown in FIG. 7) before being provided to synchronization unit 3212 .

Synchronization unit 3212 synchronizes video and audio frames and provides video/audio to video/audio display 3214 . For example, synchronization unit 3212 synchronizes the presentation of video and audio information. Information may be coded into the syntax using timestamps for the presentation of the coded audio and visual data and timestamps for the delivery of the data stream itself.

If subtitles are included in the stream, subtitle decoder 3210 decodes the subtitles, synchronizes them with the video and audio frames, and provides video/audio/subtitles to video/audio/subtitle display 3216 .

The present invention is not limited to the systems described above, and either the picture encoding device or the picture decoding device in the above embodiments can be incorporated into other systems, such as vehicle systems.

Further, FIG. 8 shows a flow chart of a method, performed by a decoder, for inverse quantization of a current block of a picture, according to an embodiment of the present disclosure. In FIG. 8, a method for inverse quantization of the current block of a picture, the method performed by the decoder comprises the following steps:
(1601) receiving a bitstream; (1603) obtaining a joint chrominance component residual (JCCR) control flag from the bitstream; (1607) obtaining at least one chrominance quantization parameter (QP) offset from the bitstream based on a JCCR control flag; (1609) obtained chrominance mapping information from (1611) obtaining a QP value for the current chrominance block based on the mapping information and at least one obtained chrominance QP offset; is shown to include the step of performing quantization;

Furthermore, FIG. 9 shows a flow chart of a method, performed by a decoder, for inverse quantization of a current block of a picture, according to a further embodiment of the present disclosure. In FIG. 9, a method for inverse quantization of the current block of a picture, the method performed by the decoder are the following steps: (1651) receiving a bitstream, which is sliced (1653) including a header syntax and a PPS syntax; and obtaining a syntax element from the (1653) PPS syntax, the obtained syntax element including a chrominance quantization parameter (QP) offset. (1655) obtaining chrominance QP offset information from a slice header, wherein the QP offset information is obtained independently from any PPS syntax element within the PPS syntax; (1657) determining a QP value for the current chrominance block depending on the chrominance QP offset obtained from the PPS syntax and the chrominance QP offset information obtained from the slice header syntax; and performing inverse quantization on the current chrominance block by using the QP value.

Further, FIG. 10 shows a flowchart of a method, performed by an encoder, for inverse quantization of a current block of a picture, according to an embodiment of the present disclosure. In FIG. 10, the method for inverse quantization of the current block of a picture, performed by the encoder, comprises the following steps: (2601) joint chrominance component residual (JCCR) control; (2603) encoding chrominance mapping information into the bitstream based on the JCCR control flags; (2605) at least one chrominance quantization based on the JCCR control flags. It is shown to include the steps of encoding a parameter (QP, quantization parameter) offset into a bitstream and providing a (2067) bitstream.

Further, FIG. 11 shows a flow chart of a method, performed by an encoder, for inverse quantization of a current block of a picture according to this disclosure. Referring to FIG. 11, a method for inverse quantization of a current block of a picture, the method performed by an encoder comprises the following steps:
(2651) encoding syntax elements from the slice header and PPS syntax into a bitstream, the syntax elements including a chrominance quantization parameter (QP) offset; and (2653). (2655) encoding chrominance QP offset information from the slice header into the bitstream, wherein the QP offset information is obtained independently from any PPS syntax element within the PPS syntax; providing a bitstream;

Additionally, FIG. 12 illustrates a decoder 30 according to an embodiment of the present disclosure. The decoder 300 of FIG. 12 comprises a receiving unit 3001 configured to receive a bitstream and a first unit 3001 configured to obtain a joint chrominance component residual (JCCR) control flag from the bitstream. a second obtaining unit 3005 configured to obtain chrominance mapping information from the bitstream based on the JCCR control flag; and at least one chrominance quantum from the bitstream based on the JCCR control flag. a third obtaining unit 3007 configured to obtain a quantization parameter (QP) offset, and a quantization parameter (QP) offset for the current chrominance block based on the obtained chrominance mapping information and the at least one obtained chrominance QP offset; a fourth obtaining unit 3009 configured to obtain a QP value and an inverse quantization unit configured to perform inverse quantization on the current chrominance block by using the determined QP value (3011) and

Additionally, FIG. 13 illustrates a decoder 30 according to another embodiment of the present disclosure. The decoder 30 of FIG. 13 is a receiving unit 3051 configured to receive a bitstream, the bitstream including slice header syntax and PPS syntax, and syntax elements from the PPS syntax. and the obtained syntax elements include a chrominance quantization parameter (QP) offset, from the first obtaining unit 3053 and the slice header a second obtaining unit 3055 configured to obtain chrominance QP offset information, wherein the QP offset information is obtained independently from any PPS syntax element in the PPS syntax; A unit 3055 and a determining unit 3057 configured to determine a QP value for the current chrominance block in dependence on the chrominance QP offset obtained from the PPS syntax and the chrominance QP offset information obtained from the slice header syntax. and an inverse quantization unit 3059 configured to perform inverse quantization on the current chrominance block by using the determined QP value.

Additionally, FIG. 14 illustrates an encoder 20 according to an embodiment of the present disclosure. The encoder 20 of FIG. 14 includes a first encoding unit 2001 configured to encode a joint chrominance component residual (JCCR) control flag into the bitstream, and based on the JCCR control flag: A second encoding unit 2003 configured to encode chrominance mapping information into the bitstream and, based on JCCR control flags, encode at least one chrominance quantization parameter (QP) offset into the bitstream. and a providing unit 3007 configured to provide a bitstream.

Additionally, FIG. 15 illustrates an encoder 20 according to another embodiment of the present disclosure. The encoder 20 of FIG. 15 is a first encoding unit 2051 configured to encode syntax elements from the slice header and PPS syntax into the bitstream, the syntax elements being the chrominance quantization parameters ( a first encoding unit 2051, comprising a QP (quantization parameter) offset, and a second encoding unit 2053, configured to encode chrominance QP offset information from the slice header into the bitstream, the QP offset; The information includes a second encoding unit 2053 and a providing unit 2055 configured to provide a bitstream, independently obtained from any PPS syntax element within the PPS syntax.

Mathematical Operators The mathematical operators used in this application are similar to those used in the C programming language. However, the results of integer division and arithmetic shift operations are defined more precisely, and further operators are defined such as exponential calculation and real-valued division. Numbering and counting conventions generally start with 0, eg, "first" is equivalent to 0th, "second" is equivalent to 1st, and so on.

Logical Operators The following logical operators are defined as follows.

Logical Operators The following logical operators are defined as follows.
x&&y boolean logical "and" of x and y
Boolean logical "union" of x||yx and y
! boolean logic "not"
x?y: if zx is true or not equal to 0, evaluates to the value of y, else evaluates to the value of z

Relational Operators The following relational operators are defined as follows.
> greater than
>= greater than
< less than
<= less than
== equal
When the != not equal relational operator is applied to a syntax element or variable that has been assigned the value 'na' (not applicable), the value 'na' is treated as a separate value for the syntax element or variable. The value "na" is considered unequal to any other value.

Bitwise Operators The following bitwise operators are defined as follows.
& Bitwise "product". When operating on integer arguments, it operates on the two's complement representation of the integer value. When operating on binary arguments that contain fewer bits than other arguments, shorter arguments are extended by adding more significant bits equal to zero.
| Bitwise "sum". When operating on integer arguments, it operates on the two's complement representation of the integer value. When operating on binary arguments that contain fewer bits than other arguments, shorter arguments are extended by adding more significant bits equal to zero.
^ Bitwise "exclusive sum". When operating on integer arguments, it operates on the two's complement representation of the integer value. When operating on binary arguments that contain fewer bits than other arguments, shorter arguments are extended by adding more significant bits equal to zero.
Arithmetic right shift of the two's complement integer representation of x by x>>yy binary digits. This function is only defined for non-negative integer values of y. The most significant bit (MSB) shifted bit as a result of the right shift has a value equal to the MSB of x before the shift operation.
Arithmetic left shift of the two's complement integer representation of x by x<<yy binary digits. This function is only defined for non-negative integer values of y. The least significant bit (LSB) shifted bit as a result of the left shift has a value equal to zero.

Assignment Operators The following assignment operators are defined as follows.
= assignment operator
++ Increment. That is, x++ is equal to x=x+1. When used on an array index, evaluates to the value of the variable before the increment operation.
-- Decrement. That is, x-- is equal to x=x-1. When used on an array index, evaluates to the value of the variable before the decrement operation.
+= Increment by the specified amount. That is, x+=3 equals x=x+3 and x+=(-3) equals x=x+(-3).
-= Decrement by the specified amount. That is, x-=3 equals x=x-3 and x-=(-3) equals x=x-(-3).

Range Notation The following notation is used to specify a range of values.
x=y..zx takes an integer value greater than or equal to y and less than or equal to z, where x, y and z are integers and z is greater than y.

Asin(x) 三角法の逆正弦関数。-1.0以上1.0以下の範囲にある引数xに対して演算し、ラジアンの単位の-π÷2以上π÷2以下の範囲の出力値を有する。
Atan(x) 三角法の逆正接関数。引数xに対して演算し、ラジアンの単位の-π÷2以上π÷2以下の範囲の出力値を有する。

Ceil(x) x以上の最小の整数。
Clip1_Y(x)=Clip3(0,(1<<BitDepth_Y)-1,x)
Clip1_C(x)=Clip3(0,(1<<BitDepth_C)-1,x)

Cos(x) ラジアンの単位の引数xに対して演算する三角法の余弦関数。
Floor(x) x以下の最大の整数。

Ln(x) xの自然対数(eを底とする対数であり、eは自然対数の底の定数2.718281828...である)。
Log2(x) xの2を底とする対数。
Log10(x) xの10を底とする対数。

Round(x)=Sign(x)*Floor(Abs(x)+0.5)

Sin(x) ラジアンの単位の引数xに対して演算する三角法の正弦関数。
Sqrt(x)=√x
Swap(x,y)=(y,x)
Tan(x) ラジアンの単位の引数xに対して演算する三角法の正接関数。 Mathematical Functions The following mathematical functions are defined.

Asin(x) Trigonometry inverse sine function. Operates for an argument x in the range -1.0 to 1.0, and has an output value in the range -π/2 to π/2 in units of radians.
Atan(x) Trigonometry arctangent function. Operates on the argument x and has an output value in the range -π/2 to π/2 in units of radians.

Ceil(x) Smallest integer greater than or equal to x.
Clip1 _Y (x)=Clip3(0,(1<<BitDepth _Y )-1,x)
Clip1 _C (x)=Clip3(0,(1<<BitDepth _C )-1,x)

Cos(x) The trigonometric cosine function operating on the argument x in radians.
Floor(x) Largest integer less than or equal to x.

Ln(x) The natural logarithm of x (the base e logarithm, where e is the constant 2.718281828... in the base of the natural logarithm).
Log2(x) The base 2 logarithm of x.
Log10(x) The base 10 logarithm of x.

Round(x)=Sign(x)*Floor(Abs(x)+0.5)

Sin(x) The trigonometric sine function operated on the argument x in radians.
Sqrt(x)=√x
Swap(x,y)=(y,x)
Tan(x) The trigonometric tangent function that operates on the argument x in radians.

Operational Precedence When the precedence of an expression is not explicitly indicated by the use of parentheses, the following rules apply.
- Higher priority operations are evaluated before any lower priority operations.
- Operations of the same priority are evaluated sequentially from left to right.

The table below specifies the priority of operations from highest to lowest, with higher positions in the table indicating higher priority.

For operators that are also used in the C programming language, the precedence used here is the same as that used in the C programming language.

Textual descriptions of logical operations In text, in the form:
if (condition 0)
statement 0
else (condition 1)
statement 1
...
else /*Informative note about remaining conditions*/
statement n
Statements of logical operations as mathematically described in may be written in the following manner.
...as follows/...the following applies:
- statement 0 if condition 0
- otherwise, if condition 1, then statement 1
－...
- otherwise (informative note on remaining conditions), statement n
Each "if..., if..., if..., if..." statement in the text is immediately followed by "if... then" or "if... applies”. The last condition of "if..., if not..., if..., if not..." is always "if not...". Alternating "if..., if not..., if..., if not..." statements end with "if not, then..." or "...following" or "... can be identified by matching "is true".

In text, the following format:
if(condition 0a&& condition 0b)
statement 0
else if(condition1a||condition1b)
statement 1
...
else
statement n
Statements of logical operations as mathematically described in may be written in the following manner.
...as follows/...the following applies:
- statement 0 if all of the following conditions are true:
- Condition 0a
-Condition 0b
- otherwise, if one or more of the following conditions are true, then statement 1:
- Condition 1a
- Condition 1b
－…
- otherwise, statement n

In text, the following format:
if (condition 0)
statement 0
if (condition 1)
statement 1
Statements of logical operations as mathematically described in may be written in the following manner.
statement 0 when condition 0
if condition 1 then statement 1

Although embodiments of the present invention have been described primarily based on video coding, embodiments of coding system 10, encoder 20 and decoder 30 (and correspondingly system 10), and other embodiments described herein, are also described. , still picture processing or coding, i.e., the processing or coding of individual pictures independent of any previous or subsequent pictures, such as in video coding. In general, if picture processing coding is limited to a single picture 17, only inter prediction units 244 (encoder) and 344 (decoder) may not be available. All other functions (also called tools or techniques) of video encoder 20 and video decoder 30 are used for still picture processing, e.g. (Inverse) transforms 212/312, partitions 262/362, intra prediction 254/354, and/or loop filtering 220, 320 and entropy coding 270 and entropy decoding 304 may equally be used.

For example, embodiments of encoder 20 and decoder 30, and for example the functionality described herein with respect to encoder 20 and decoder 30, may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. Computer-readable storage medium corresponds to a tangible medium such as a data storage medium or any medium that facilitates transfer of a computer program from one place to another, for example, according to a communication protocol. may include a communication medium including In this manner, computer-readable media generally may correspond to (1) tangible computer-readable storage media which is non-transitory or (2) a communication medium such as a signal or carrier wave. Data storage media is any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the techniques described in this disclosure. It's okay. A computer program product may include a computer-readable medium.

By way of example, and not limitation, such computer readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or storage of instructions or data structures. It can include any other medium that can be used to store the desired program code in form and that can be accessed by the computer. Also, any connection is properly termed a computer-readable medium. For example, instructions may be sent to a website, server or other remote site using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio and microwave. If transmitted from a source, coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio and microwave are included in the definition of medium. However, it should be understood that computer readable storage media and data storage media do not include connections, carrier waves, signals or other transitory media, but instead cover non-transitory tangible storage media. be. Disk and disc, as used herein, include compact disc (CD), laser disc, optical disc, DVD (digital versatile disc), floppy disc and Blu-ray disc. A disk usually reproduces data magnetically, and a disk reproduces data optically with a laser. Combinations of the above should also be included within the scope of computer-readable media.

The instructions are written in one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other may be executed by one or more processors such as equivalent integrated or discrete logic circuits. Accordingly, the term "processor," as used herein, may refer to any of the structures described above or any other structure suitable for implementing the techniques described herein. Moreover, in some aspects the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated into a combined codec. good too. Moreover, the techniques may be fully implemented in one or more circuits or logic elements.

The techniques of this disclosure may be implemented in a wide variety of devices or apparatus, including wireless handsets, integrated circuits (ICs), or sets of ICs (eg, chipsets). Various components, modules or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. . Rather, as described above, the various units may be combined with appropriate software and/or firmware into a codec hardware unit, or an interoperable hardware unit including one or more processors as described above. It may be provided by a collection of wear units.

The present disclosure provides the following 19 additional aspects.

A first aspect of a method for inverse quantization of a current block of a picture, the method being performed by a decoder, the method comprising:
receiving a bitstream, the bitstream comprising slice header syntax and PPS syntax;
obtaining a syntax element from the PPS syntax, the obtained syntax element including a chrominance quantization parameter (QP) offset;
obtaining chrominance QP offset information from a slice header, wherein the QP offset information is obtained independently from any PPS syntax element within the PPS syntax;
determining a QP value for the current chrominance block dependent on the chrominance QP offset obtained from the PPS syntax and the chrominance QP offset information obtained from the slice header syntax;
performing inverse quantization on the current chrominance block by using the determined QP value.

A second aspect of the method according to the first aspect, wherein the at least one chrominance QP offset obtained from the PPS syntax comprises pps_cb_qp_offset, pps_cr_qp_offset, pps_joint_cbcr_qp_offset and cu_chroma_qp_offset_enabled_flag.

であり、iは整数である、方法の第3の態様。 A third aspect of the method according to the second aspect, wherein if the value of cu_chroma_qp_offset_enabled_flag is true (eg, if the value of cu_chroma_qp_offset_enabled_flag is 1), the chrominance QP offsets obtained from the PPS syntax are: cu_chroma_qp_offset_subdiv, further comprising chroma_qp_offset_list_len_minus1, cb_qp_offset_list[i], cr_qp_offset_list[i] and joint_cbcr_qp_offset_list[i];

and i is an integer.

A fourth aspect of the method according to any one of the first to third aspects, wherein the chrominance QP offset information obtained from the slice header syntax includes slice_cb_qp_offset and slice_cr_qp_offset .

A fifth aspect of the method according to the fourth aspect, wherein the value of sps_joint_cbcr_enabled_flag (eg, an element of SPS level syntax included in the bitstream) is true (eg, the value of sps_joint_cbcr_enabled_flag is 1) , the chrominance QP offset information obtained from the slice header syntax further comprises slice_joint_cbcr_qp_offset.

A sixth aspect of the method according to any one of the first to fifth aspects, wherein the PPS syntax comprises the following structure.

A seventh aspect of the method according to any one of the first to sixth aspects, wherein the slice header syntax comprises the following structure.

An eighth aspect of the method according to any one of the first to seventh aspects, wherein the flag pps_slice_chroma_qp_offsets_present_flag is omitted in the PPS syntax, or
An eighth aspect of the method, wherein the slice header and PPS syntax always contain elements related to chrominance QP offsets.

A ninth aspect of a method for inverse quantization of a current block of a picture, the method being performed by a decoder, the method comprising:
receiving a bitstream;
obtaining a joint chrominance component residual (JCCR) control flag from the bitstream;
obtaining chrominance mapping information from the bitstream based on JCCR control flags;
obtaining a chrominance quantization parameter (QP) offset from the bitstream based on a JCCR control flag;
obtaining a QP value for the current chrominance block based on the obtained chrominance mapping information and the obtained chrominance QP offset;
performing inverse quantization on the current chrominance block by using the determined QP value.

A tenth aspect of the method according to the ninth aspect, wherein the bitstream comprises SPS level syntax and the JCCR control flag is obtained from the SPS level syntax.

An eleventh aspect of the method according to the ninth or tenth aspect, wherein the JCCR control flag is sps_joint_cbcr_enabled_flag.

A twelfth aspect of the method according to any one of the ninth to eleventh aspects, wherein the chrominance mapping information comprises delta_qp_in_val_minus1[i][j] and delta_qp_out_val[i][j]; is obtained from the SPS level syntax included in the bitstream.

A thirteenth aspect of the method according to any one of the ninth to twelfth aspects, wherein the SPS level syntax comprises the following structure.

A fourteenth aspect of the method according to any one of the ninth to thirteenth aspects, wherein obtaining a chrominance QP offset from the bitstream based on a JCCR control flag comprises:
A fourteenth aspect of the method, comprising obtaining a chrominance QP offset from the PPS level syntax of the bitstream based on a JCCR control flag.

A fifteenth aspect of the method according to any one of the ninth to fourteenth aspects, wherein the PPS level syntax comprises the following structure.

A sixteenth aspect of a decoder comprising processing circuitry for performing a method according to any one of the first through fifteenth aspects.

A seventeenth aspect of a computer program product comprising program code for performing a method according to any one of the above aspects when run on a computer or processor.

An eighteenth aspect of the decoder, comprising:
one or more processors;
a non-transitory computer readable storage medium coupled to the processor and storing programming for execution by the processor, the programming, when executed by the processor, performing a method according to any one of the above aspects. An eighteenth aspect of the decoder, wherein the decoder is configured to:

A nineteenth aspect of a non-transitory computer-readable medium carrying program code, comprising:
A nineteenth aspect of the non-transitory computer-readable medium, the program code, when executed by a computing device, causes the computing device to perform a method according to any one of the above aspects.

Claims (19)

A method for inverse quantization of a current block of a picture, the method being performed by a decoder, the method comprising:
receiving a bitstream;
obtaining a joint chrominance component residual (JCCR) control flag from the bitstream;
obtaining chrominance mapping information from the bitstream based on the JCCR control flag;
obtaining at least one chrominance quantization parameter (QP) offset from the bitstream based on the JCCR control flag;
obtaining a QP value for a current chrominance block based on the obtained chrominance mapping information and the at least one obtained chrominance QP offset;
performing inverse quantization on the current chrominance block by using the obtained QP values. 2. The method of claim 1, wherein the bitstream includes SPS level syntax and the JCCR control flags are obtained from the SPS level syntax. 3. A method according to claim 1 or 2, wherein said JCCR control flag is sps_joint_cbcr_enabled_flag . 4. The method of claim 3, wherein if the value of sps_joint_cbcr_enabled_flag is 1, the at least one obtained chrominance QP offset is specified by slice_joint_cbcr_qp_offset. 5. The chrominance mapping information comprises delta_qp_in_val_minus1[i][j] and delta_qp_out_val[i][j], wherein the chrominance mapping information is obtained from SPS level syntax included in the bitstream. The method according to any one of 6. The method of claim 5 , wherein the SPS level syntax includes the following structure.

obtaining the at least one chrominance QP offset from the bitstream based on the JCCR control flag;
7. The method of claim 1, comprising obtaining the at least one chrominance QP offset from Picture Parameter Set (PPS) level syntax of the bitstream based on the JCCR control flag. Method. 8. The method of claim 7, wherein the PPS level syntax includes the following structure.

A method for encoding a current block of a picture, the method being performed by an encoder, the method comprising:
encoding a joint chrominance component residual (JCCR) control flag into the bitstream;
encoding chrominance mapping information into the bitstream based on the JCCR control flags;
encoding at least one chrominance quantization parameter (QP) offset into the bitstream based on the JCCR control flag;
and providing said bitstream. 10. The method of claim 9 , wherein the bitstream includes SPS level syntax and the JCCR control flags are encoded in the SPS level syntax. A method according to claim 9 or 10 , wherein said JCCR control flag is sps_joint_cbcr_enabled_flag . 12. The method of claim 11 , wherein if the value of sps_joint_cbcr_enabled_flag is 1, the at least one coded chrominance QP offset is specified by slice_joint_cbcr_qp_offset. 10. The chrominance mapping information comprises delta_qp_in_val_minus1[i][j] and delta_qp_out_val[i][j], wherein the chrominance mapping information is encoded in SPS level syntax included in the bitstream. 13. The method of any one of 12 . 14. The method of any one of claims 10-13 , wherein the SPS level syntax comprises the following structure:

Encoding the at least one chrominance QP offset into the bitstream based on the JCCR control flag comprises:
15. The method of any one of claims 9 to 14 , comprising encoding the at least one chrominance QP offset into picture parameter set (PPS) level syntax of the bitstream based on the JCCR control flag. the method of. 16. The method of claim 15 , wherein the PPS level syntax includes the following structure.

a decoder,
one or more processors;
and a non- transitory computer readable storage medium coupled to said processor and storing programming for execution by said processor, said programming when executed by said processor. A decoder configured to perform the method of claim 1. an encoder,
one or more processors;
a non-transitory computer readable storage medium coupled to said processor and storing programming for execution by said processor, said programming, when executed by said processor, any of claims 9 to 16 An encoder configured to perform the method of clause 1. A video data decoding device,
a non-transitory memory storage configured to store video data in the form of a bitstream;
a decoder configured to perform the method according to any one of claims 1 to 8;
video data decoding device including;

Images