WO2021263251A1

WO2021263251A1 - State transition for dependent quantization in video coding

Info

Publication number: WO2021263251A1
Application number: PCT/US2021/049474
Authority: WO
Inventors: Yue Yu; Haoping Yu
Original assignee: Innopeak Technology, Inc.
Priority date: 2021-02-19
Filing date: 2021-09-08
Publication date: 2021-12-30
Also published as: CN116982262A

Abstract

In some embodiments, a video encoder or decoder reconstructs a block of a video through dependent quantization. The video encoder or decoder accesses the quantized elements associated with the block and processes the quantized elements according to an order for the block to generate respective de-quantized elements. The processing includes obtaining a current quantized element of the block from the quantized elements, and determining a quantizer for the current quantized element based on at least two quantized elements preceding the current quantized element or based on a comparison of an element immediately preceding the current quantized element with zero. The processing further includes de-quantizing the current quantized element based on the quantizer to generate a de-quantized element. The video encoder or decoder reconstructs the block based on the de-quantized elements.

Description

STATE TRANSITION FOR DEPENDENT QUANTIZATION IN VIDEO CODING

Cross-Reference to Related Applications

[0001] This application claims priority to U.S. Provisional Application No. 63/151,535, entitled “New Dependent Quantization State Transition for Video Coding,” filed on February 19, 2021, which is hereby incorporated in its entirety by this reference.

Technical Field

[0002] This disclosure relates generally to computer-implemented methods and systems for video processing. Specifically, the present disclosure involves dependent quantization for video coding.

Background

[0003] The ubiquitous camera-enabled devices, such as smartphones, tablets, and computers, have made it easier than ever to capture videos or images. However, the amount of data for even a short video can be substantially large. Video coding technology (including video encoding and decoding) allows video data to be compressed into smaller sizes thereby allowing various videos to be stored and transmitted. Video coding has been used in a wide range of applications, such as digital TV broadcast, video transmission over the internet and mobile networks, real time applications (e.g., video chat, video conferencing), DVD and Blu-ray discs, and so on. To reduce the storage space for storing a video and/or the network bandwidth consumption for transmitting a video, it is desired to improve the efficiency of the video coding scheme.

Summary

[0004] Some embodiments involve dependent quantization for video coding. In one example, a method for reconstructing a block for a video includes accessing a plurality of quantized elements that are associated with the block and processing the plurality of quantized elements according to an order for the block to generate respective de-quantized elements. The processing includes obtaining a current quantized element of the block from the plurality of quantized elements; determining a quantizer for the current quantized element based on at least two quantized elements preceding the current quantized element or based on a comparison of an element immediately preceding the current quantized element with zero; and de-quantizing the current quantized element based on the quantizer to generate a de-quantized element; and reconstructing the block based on the de-quantized elements. [0005] In another example, a non-transitory computer-readable medium has program code that is stored thereon and the program code is executable by one or more processing devices for performing operations. The operations include accessing a plurality of quantized elements that are associated with a block of a video and processing the plurality of quantized elements according to an order for the block to generate respective de-quantized elements. The processing includes obtaining a current quantized element of the block from the plurality of quantized elements; determining a current state for quantization based on a previous state for quantization for an element immediately preceding the current quantized element and a state transition table:

wherein v is a value determined based on at least one quantized element preceding the current quantized element. The processing further includes determining a quantizer for the current quantized element according to the current state for quantization and de-quantizing the current quantized element based on the quantizer to generate a de-quantized element. The operations further include reconstructing the block based on the de-quantized elements.

[0006] In yet another example, a system includes a processing device and a non-transitory computer-readable medium communicatively coupled to the processing device. The processing device is configured to execute program code stored in the non-transitory computer-readable medium and thereby perform operations. The operations include accessing a plurality of elements associated with a block of a video and processing the plurality of elements according to an order for the block. The processing includes obtaining a current element of the block from the plurality of elements, determining a quantizer for the current element based on at least two previous elements before the current element, or based on a comparison of an element immediately preceding the current element with zero, and quantizing the current element based on the quantizer to generate a quantized element. The operations further include encoding the quantized elements into a bitstream representing the video.

[0007] These illustrative embodiments are mentioned not to limit or define the disclosure, but to provide examples to aid understanding thereof. Additional embodiments are discussed in the Detailed Description, and further description is provided there. Brief Description of the Drawings

[0008] Features, embodiments, and advantages of the present disclosure are better understood when the following Detailed Description is read with reference to the accompanying drawings.

[0009] FIG. 1 is a block diagram showing an example of a video encoder configured to implement embodiments presented herein.

[0010] FIG. 2 is a block diagram showing an example of a video decoder configured to implement embodiments presented herein.

[0011] FIG. 3 depicts an example of a coding tree unit division of a picture in a video, according to some embodiments of the present disclosure.

[0012] FIG. 4 depicts an example of a coding unit division of a coding tree unit, according to some embodiments of the present disclosure.

[0013] FIG. 5 depicts an example of a coding block with a pre-determmed order for processing the elements of the coding block

[0014] FIG. 6 depicts an example of two quantizers used for dependent quantization in the prior art video coding technology.

[0015] FIG. 7 depicts an example of a state transition diagram for dependent quantization, according to some embodiments of the present disclosure.

[0016] FIG. 8 depicts an example of a state transition diagram for dependent quantization and the associated state transition table, according to some embodiments of the present disclosure.

[0017] FIG. 9 depicts an example of a process for encoding a block for a video via the dependent quantization, according to some embodiments of the present disclosure.

[0018] FIG. 10 depicts an example of a process for reconstructing a block of a video quantized using the dependent quantization, according to some embodiments of the present disclosure.

[0019] FIG. 11 depicts an example of a computing system that can be used to implement some embodiments of the present disclosure.

Detailed Description

[0020] Various embodiments can provide dependent quantization for video coding. As discussed above, more and more video data is being generated, stored, and transmitted. It is beneficial to increase the coding efficiency of the video coding technology thereby using less data to represent a video without compromising the visual quality of the decoded video. One aspect to improve the coding efficiency is to improve the quantization scheme of the video coding. The latest video coding standards, such as versatile video coding (VVC), have employed dependent quantization techniques.

[0021] In dependent quantization, more than one quantizer is available when quantizing an element (e.g., a pixel value or a transform coefficient) of a coding block (or “block”). The quantizer used to quantize the current element depends on the value of the previous element in the coding block. In some examples, the value of the previous quantized element is used to determine the state of the current element which in turn is used to determine the quantizer for the current element. Since the existing dependent quantization methods only use limited information from previous quantized elements of a block when determining the quantizer for a current element of the block, the quantizer for the current element may be selected inaccurately or inappropriately. Further, the state transition table used to determine the quantizer in the existing dependent quantization has particular values. This may reduce the coding efficiency for video signals whose statistics or distributions do not match those designed for this particular state transition table.

[0022] V arious embodiments described herein address these problems by taking into account multiple elements in a coding block when selecting a quantizer for a current element of the coding block. An element of a coding block may be a residual after inter- or intra-prediction of the coding block. The element may be a transform coefficient of the residual in a frequency domain or the value of the residual in the pixel domain. When selecting the quantizer for an element, the quantized values of multiple elements that have been processed before the element are used. In addition, a state transition table different from the existing state transition table is proposed to improve the coding efficiencies of videos with statistics or distributions that do not match the existing state transition table.

[0023] The following non-limiting examples are provided to introduce some embodiments. In one example, the quantizer for a current element of a coding block is determined based on the parity of the sum of quantized values of all previously processed elements in a coding block. In some implementations, the elements in a coding block are processed according to a pre determined order, for example, from the highest frequency to the lowest frequency if the elements are transform coefficients in the frequency domain. In these examples, the video encoder or decoder calculates the parity of the sum of quantization levels of the elements that precede the current element according to the pre-determined order. The calculated parity is then used to determine the state of the current element according to a state transition table and the quantizer corresponding to the determined state is the quantizer for the current element. [0024] Alternatively, or additionally, the video encoder or decoder calculates the parity of the number of non-zero quantization levels (or zero quantization levels) in the quantized elements that precede the current element according to the pre-determined order. The calculated parity is then used to determine the state of the current element according to a state transition table and the quantizer corresponding to the state is the quantizer for the current element. Compared with the existing dependent quantization methods where only one previous quantized element of the current element is used to determine the state of the current element, the proposed method takes into account the information of multiple previous quantized elements, thereby making the selection of the quantizer more informed and more accurate.

[0025] In another example, the state of the current element is determined according to the state transition table if the quantization level of the previous element of the current element is non-zero. Otherwise, the state remains unchanged and the quantizer of the previous element is used to quantize the current element. This embodiment eliminates the step for determining the state of the current element if the quantization level of the previous element is zero, thereby reducing the computational complexity of the encoding and decoding algorithm. In a further example, an alternative state transition table that is different from the existing state transition table is proposed to capture the statistics or distributions of some videos that were not captured by the existing dependent quantization. The calculation of the state for a current element can be performed using any of the examples disclosed herein.

[0026] The video encoder can use the selected quantizer to quantize the current element. The quantized elements for the coding block can then be encoded into a bitstream for the video. At the decoder side or the encoder side when reconstructing the block for prediction purposes, the dequantization process can determine the state of each quantized element in the block using any method or any combination of methods described above and subsequently determine the quantizer. The determined quantizer can be used to de-quantize the element and the de- quantized elements of the block are then used to reconstruct the block of the video for display (at the decoder) or prediction of other blocks or pictures (at the encoder).

[0027] As described herein, some embodiments provide improvements in coding efficiency by utilizing more information than the existing dependent quantization methods and by employing a different state transition table. By using elements beyond the immediately preceding element of a current element, the state of the current element can be more accurately determined, and thus the selected quantizer is better suited for the current element than the existing methods. In addition, by skipping the state determination for some elements, the computational complexity of the quantization stage can be reduced without sacrificing the visual quality of the encoded video. Furthermore, the utilization of a different state transition table allows the video coding scheme to achieve a high coding efficiency for a different set of video signals. The techniques can be an effective coding tool in future video coding standards.

[0028] Referring now to the drawings, FIG. l is a block diagram showing an example of a video encoder 100 configured to implement embodiments presented herein. In the example shown in FIG. 1, the video encoder 100 includes a partition module 112, a transform module 114, a quantization module 115, an inverse quantization module 118, an inverse transform module 119, an in-loop filter module 120, an intra prediction module 126, an inter prediction module 124, a motion estimation module 122, a decoded picture buffer 130, and an entropy coding module 116.

[0029] The input to the video encoder 100 is an input video 102 containing a sequence of pictures (also referred to as frames or images). In a block-based video encoder, for each of the pictures, the video encoder 100 employs a partition module 112 to partition the picture into blocks 104, and each block containing multiple pixels. The blocks may be macroblocks, coding tree units, coding units, prediction units, and/or prediction blocks. One picture may include blocks of different sizes and the block partitions of different pictures of the video may also differ. Each block may be encoded using different predictions, such as intra prediction or inter prediction or intra and inter hybrid prediction.

[0030] Usually, the first picture of a video signal is an intra-predicted picture, which is encoded using only intra prediction. In the intra prediction mode, a block of a picture is predicted using only data from the same picture. A picture that is intra-predicted can be decoded without information from other pictures. To perform the intra-prediction, the video encoder 100 shown in FIG. 1 can employ the intra prediction module 126. The intra prediction module 126 is configured to use reconstructed samples in reconstructed blocks 136 of neighboring blocks of the same picture to generate an intra-prediction block (the prediction block 134). The intra prediction is performed according to an mtra-prediction mode selected for the block. The video encoder 100 then calculates the difference between block 104 and the intra-prediction block 134. This difference is referred to as residual block 106.

[0031] To further remove the redundancy from the block, the residual block 106 is transformed by the transform module 114 into a transform domain by applying a transform on the samples in the block. Examples of the transform may include, but are not limited to, a discrete cosine transform (DCT) or discrete sine transform (DST). The transformed values may be referred to as transform coefficients representing the residual block in the transform domain. In some examples, the residual block may be quantized directly without being transformed by the transform module 114. This is referred to as a transform skip mode.

[0032] The video encoder 100 can further use the quantization module 115 to quantize the transform coefficients to obtain quantized coefficients. Quantization includes dividing a sample by a quantization step size followed by subsequent rounding; whereas inverse quantization involves multiplying the quantized value by the quantization step size. Such a quantization process is referred to as scalar quantization. Quantization is used to reduce the dynamic range of video samples (transformed or non-transformed) so that fewer bits are used to represent the video samples.

[0033] The quantization of coefficients/samples within a block can be done independently and this kind of quantization method is used in some existing video compression standards, such as H.264, and HEVC. For an N-by-M block, a specific scan order may be used to convert the 2D coefficients of a block into a 1-D array for coefficient quantization and coding. Quantization of a coefficient within a block may make use of the scan order information. For example, the quantization of a given coefficient in the block may depend on the status of the previous quantized value along the scan order. In order to further improve the coding efficiency, more than one quantizer may be used. Which quantizer is used for quantizing a current coefficient depends on the information preceding the current coefficient in encoding/decoding scan order. Such a quantization approach is referred to as dependent quantization.

[0034] The degree of quantization may be adjusted using the quantization step sizes. For instance, for scalar quantization, different quantization step sizes may be applied to achieve finer or coarser quantization. Smaller quantization step sizes correspond to finer quantization, whereas larger quantization step sizes correspond to coarser quantization. The quantization step size can be indicated by a quantization parameter (QP). The quantization parameters are provided in the encoded bitstream of the video such that the video decoder can apply the same quantization parameters for decoding.

[0035] The quantized samples are then coded by the entropy coding module 116 to further reduce the size of the video signal. The entropy encoding module 116 is configured to apply an entropy encoding algorithm on the quantized samples. Examples of the entropy encoding algorithm include, but are not limited to, a variable length coding (VFC) scheme, a context adaptive VLC scheme (CAVLC), an arithmetic coding scheme, a binarization, a context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy encoding techniques. The entropy-coded data is added to the bitstream of the output encoded video 132.

[0036] As discussed above, reconstructed blocks 136 from neighboring blocks are used in the intra-prediction of blocks of a picture. Generating the reconstructed block 136 of a block involves calculating the reconstructed residuals of this block. The reconstructed residual can be determined by applying inverse quantization and inverse transform on the quantized residual of the block. The inverse quantization module 118 is configured to apply the inverse quantization on the quantized samples to obtain de-quantized coefficients. The inverse quantization module 118 applies the inverse of the quantization scheme applied by the quantization module 115 by using the same quantization step size as the quantization module 115. The inverse transform module 119 is configured to apply the inverse transform of the transform applied by the transform module 114 on the de-quantized samples, such as inverse DCT or inverse DST. The output of the inverse transform module 119 is the reconstructed residuals for the block in the pixel domain. The reconstructed residuals can be added to the prediction block 134 of the block to obtain a reconstructed block 136 in the pixel domain. For blocks where the transform is skipped, the inverse transform module 119 is not applied to those blocks. The de-quantized samples are the reconstructed residuals for the blocks.

[0037] Blocks in subsequent pictures following the first intra-predicted picture can be coded using either inter prediction or intra prediction. In inter-prediction, the prediction of a block in a picture is from one or more previously encoded video pictures. To perform inter prediction, the video encoder 100 uses an inter prediction module 124. The inter prediction module 124 is configured to perform motion compensation for a block based on the motion estimation provided by the motion estimation module 122.

[0038] The motion estimation module 122 compares a current block 104 of the current picture with decoded reference pictures 108 for motion estimation. The decoded reference pictures 108 are stored in a decoded picture buffer 130. The motion estimation module 122 selects a reference block from the decoded reference pictures 108 that best matches the current block. The motion estimation module 122 further identifies an offset between the position (e.g., x, y coordinates) of the reference block and the position of the current block. This offset is referred to as the motion vector (MV) and is provided to the inter prediction module 124. In some cases, multiple reference blocks are identified for the block in multiple decoded reference pictures 108. Therefore multiple motion vectors are generated and provided to the inter prediction module 124.

[0039] The inter prediction module 124 uses the motion vector(s) along with other mter- prediction parameters to perform motion compensation to generate a prediction of the current block, i.e., the inter prediction block 134. For example, based on the motion vector(s), the inter prediction module 124 can locate the prediction block(s) pointed to by the motion vector(s) in the corresponding reference picture(s). If there are more than one prediction blocks, these prediction blocks are combined with some weights to generate a prediction block 134 for the current block.

[0040] For inter-predicted blocks, the video encoder 100 can subtract the inter-prediction block 134 from the block 104 to generate the residual block 106. The residual block 106 can be transformed, quantized, and entropy coded in the same way as the residuals of an intra-predicted block discussed above. Likewise, the reconstructed block 136 of an inter-predicted block can be obtained through inverse quantizing, inverse transforming the residual, and subsequently combining with the corresponding prediction block 134.

[0041] To obtain the decoded picture 108 used for motion estimation, the reconstructed block 136 is processed by an in-loop filter module 120. The in-loop filter module 120 is configured to smooth out pixel transitions thereby improving the video quality. The in-loop filter module 120 may be configured to implement one or more in-loop filters, such as a de-blocking filter, or a sample-adaptive offset (SAO) filter, or an adaptive loop filter (ALF), etc.

[0042] FIG. 2 depicts an example of a video decoder 200 configured to implement embodiments presented herein. The video decoder 200 processes an encoded video 202 in a bitstream and generates decoded pictures 208. In the example shown in FIG. 2, the video decoder 200 includes an entropy decoding module 216, an inverse quantization module 218, an inverse transform module 219, an in-loop filter module 220, an intra prediction module 226, an inter prediction module 224, and a decoded picture buffer 230.

[0043] The entropy decoding module 216 is configured to perform entropy decoding of the encoded video 202. The entropy decoding module 216 decodes the quantized coefficients, coding parameters including intra prediction parameters and inter prediction parameters, and other information. The entropy-decoded coefficients are then inverse quantized by the inverse quantization module 218 and subsequently inverse transformed by the inverse transform module 219 to the pixel domain. The inverse quantization module 218 and the inverse transform module 219 function similarly as the inverse quantization module 118 and the inverse transform module 119, respectively, as described above with respect to FIG. 1. The inverse-transformed residual block can be added to the corresponding prediction block 234 to generate a reconstructed block 236. For blocks where the transform is skipped, the inverse transform module 219 is not applied to those blocks. The de-quantized samples generated by the inverse quantization module 118 are used to generate the reconstructed block 236.

[0044] The prediction block 234 of a particular block is generated based on the prediction mode of the block. If the coding parameters of the block indicate that the block is mtra predicted, the reconstructed block 236 of a reference block in the same picture can be fed into the intra prediction module 226 to generate the prediction block 234 for the block. If the coding parameters of the block indicate that the block is inter-predicted, the prediction block 234 is generated by the inter prediction module 224. The intra prediction module 226 and the inter prediction module 224 function similarly as the mtra prediction module 126 and the inter prediction module 124 of FIG. 1, respectively

[0045] As discussed above with respect to FIG. 1, the inter prediction involves one or more reference pictures. The video decoder 200 generates the decoded pictures 208 for the reference pictures by applying the in-loop filter module 220 to the reconstructed blocks of the reference pictures. The decoded pictures 208 are stored in the decoded picture buffer 230 for use by the inter prediction module 224 and also for output.

[0046] Referring now to FIG. 3, FIG. 3 depicts an example of a coding tree unit division of a picture in a video, according to some embodiments of the present disclosure. As discussed above with respect to FIGS. 1 and 2, to encode a picture of a video, the picture is divided into blocks, such as the CTUs (Coding Tree Units) 302 in VVC, as shown in FIG. 3. For example, the CTUs 302 can be blocks of 128x128 pixels. The CTUs are processed according to an order, such as the order shown in FIG. 3. In some examples, each CTU 302 in a picture can be partitioned into one or more CUs (Coding Units) 402 as shown in FIG. 4, which can be used for prediction and transformation. Depending on the coding schemes, a CTU 302 may be partitioned into CUs 402 differently. For example, in VVC, the CUs 402 can be rectangular or square, and can be coded without further partitioning into prediction units or transform units. Each CU 402 can be as large as its root CTU 302, or be subdivisions of a root CTU 302 as small as 4x4 blocks. As shown in FIG. 4, a division of a CTU 302 into CUs 402 in VVC can be quadtree splitting or binary tree splitting or ternary tree splitting. In FIG. 4, solid lines indicate quadtree splitting and dashed lines indicate binary tree splitting. [0047] Dependent Quantization

[0048] As discussed above with respect to FIGS. 1 and 2, quantization is used to reduce the dynamic range of elements of blocks in the video signal so that fewer bits are used to represent the video signal. In some examples, before quantization, an element at a specific position of the block is referred to as a coefficient. After quantization, the quantized value of the coefficient is referred to as a quantization level or a level. Quantization typically consists of division by a quantization step size and subsequent rounding while inverse quantization consists of multiplication by the quantization step size. Such a quantization process is also referred to as scalar quantization. The quantization of the coefficients within a block can be performed independently and this kind of independent quantization method is used in some existing video compression standards, such as H.264, HEVC, etc. In other examples, dependent quantization is employed, such as in VVC.

[0049] For an N-by-M block, a specific scanning order may be used to convert 2-D coefficients of a block into a 1-D array for coefficient quantization and coding, and the same scanning order is used for both encoding and decoding. FIG. 5 shows an example of a coding block with a pre-determined scanning order for processing the coefficients of the coding block. In this example, the coding block 500 has a size of 8 x 8 and the processing starts at the lower right corner at position L₀ and ends at the upper left corner L₆₃. If block 500 is a transformed block, the pre-determined order shown in FIG. 5 starts from the highest frequency to the lowest frequency. In some examples, the processing of the block, such as the quantization, starts from the first non-zero element of the block according to the pre-determined scanning order. For instance, if the coefficients at locations L₀-L₁₇ are all zero and the coefficient at L₁₈ is non-zero, then the quantization starts at the coefficient at L_1S and is performed for each coefficient after L₁₈ in the scanning order.

[0050] In dependent quantization, the quantization of a coefficient within a block may make use of the scanning order information. For example, it may depend on the status of the previous quantization level along the scanning order. In addition, in order to further improve the coding efficiency, more than one quantizer (e.g., two quantizers) is used in the dependent quantization. In the existing dependent quantization, the quantizer used for quantizing a current coefficient depends on the coefficient immediately preceding the current coefficient in the scanning order. [0051] FIG. 6 shows an example of the quantizers used in the dependent quantization adopted by VVC. Two quantizers, namely Q0 and Ql, are used in this example. The quantization step size, D, is determined by a quantization factor which is embedded in the bitstream. Ideally, the quantizer used for quantizing the current coefficient can be explicitly specified. However, the overhead to signal the quantizer reduces the coding efficiency. Alternatively, instead of explicit signaling, the quantizer for the current coefficient can be determined and derived based on the quantization level of the coefficient immediately preceding the current coefficient. For example, VVC uses a four-state model and the parity of the quantization level of the previous coefficient was used to decide the state of the current coefficient. This state is then used to decide the quantizer for quantizing the current coefficient.

Table 1. State Transition Table for Dependent Quantization

[0052] Table 1 shows a state transition table adopted by VVC. The state of a coefficient can take four different values: 0, 1, 2, and 3. The state for the current coefficient can be uniquely determined by the parity of quantization level immediately preceding the current coefficient in the encoding/decoding scanning order. At the start of the quantization at the encoding side or inverse quantization at the decoding side for a block, the state is set to a default value, such as 0. The coefficients are quantized or de-quantized in the pre-determined scanning order (i.e., in the same order they are entropy decoded). After a coefficient is quantized or de-quantized, the process moves on to the next coefficient according to the scanning order. The next coefficient becomes the new current coefficient and the coefficient that was just processed becomes the previous coefficient. The state for the new current coefficient statei is determined according to Table 1, where ki- denotes the value of the quantization level of the previous coefficient. The index i denotes the position of coefficients or quantization levels along the scanning order. Note that in this example, the state only depends on the state of the previous coefficient, state*-_!, and the parity (/q_ i & 1) of the level ki_ of the previous coefficient at location i — 1. This state update process can be formulated as statei = stateTransTable[ state^^l k_i-1 & 1 ] (1) where stateTransTable represents the table shown in Table 1 and the operator & specifies the bit-wise “AND” operator in two’s- complement arithmetic. Alternatively, the state transition can also be specified without a look-up table as follows: state i = ( 32040 » ( ( state_i-1 « 2 ) + ( ( k_i-1 & 1 ) « 1 ) ) ) & 3 (2) where the 16-bit value 32040 specifies the state transition table. The state uniquely specifies the scalar quantizer used. In one example, if the state for the current coefficient is equal to 0 or 1, the scalar quantizer Q0 shown in FIG. 6 is used. Otherwise (the state is equal to 2 or 3), the scalar quantizer Q1 shown in FIG. 6 is used.

[0053] As described above, the performance and complexity involved in the existing dependent quantization may not be optimal. In this disclosure, state transition methods are proposed to improve the dependent quantitation.

[0054] In one example, instead of using the parity of quantization level of the immediate previous position in the scanning order, the parity of the sum of quantization levels of all previous encoded/decoded positions in a coding block is used for the transition of state. Denote k_m as the quantization level of the m-th element in the scanning order. For the current l-th element, index i — 1, i — 2, ... ,0 represent the positions of previous quantization levels in the coding block that precede the current element. In the example shown in FIG. 5, if the current element is at position L₂₄, the previous quantization levels in the coding block that precede the current element would include the quantization levels for previous elements positioned at L₀ to L₂₃. In examples where the processing starts from the first non-zero element in the block along the scanning order, these previous elements would include the elements from the first non- zero element to the one immediately preceding the current element. For example, if the first non zero element in the block shown in FIG. 5 is L₁₈, the previous elements would include elements at positions L₁₈ to L_23.

[0055] The sum of all quantization levels preceding position i, sum(i — 1), can be calculated as:

The state used to determine the quantizer for the current element at position i can be calculated as: statei = stateTransTable[ state_i-1][ sum(i — 1) & 1 ] (4)

The stateTransTable can be the state transition table shown in Table 2, the state transition table to be discussed below with respect to FIG. 8, or any other state transition table suitable for dependent quantization. Table 2

[0056] In addition, while Eqns. (3) and (4) calculate and use sum{i — 1) to determine the state for the current element, other ways can be used. For example, the video encoder or decoder can directly calculate the parity of sum(i — 1) (i.e., sum(i — 1) & 1) without calculating the sum value itself. In one example, the state of the current element can be determined as statei = stateTransTable[ statei^l XOR (LSB(k₀), LSB(k₁), LSI? (&*_·,)) ] (5) where XOR is the exclusive OR operator and LSB (x) is a function taking the least significant bit of the argument x. In another example, the state of the current element can be determined as

where q_ is a counter counting the total number of 1 s in the last bit of the previous quantization levels.

[0057] The examples discussed above for determining the state of the current element based on all the previous elements are for illustration and should not be limiting. Various other ways can be utilized. In addition, in some examples, instead of using all the previous quantization levels in the block, a subset of these quantization levels can be utilized to determine the state for the current element, such as previous l quantization levels with l taking a value between 2 and i — 1. For example, the state for the current element can be determined using the parity of the sum of the l immediately preceding quantization levels with any of the methods discussed above (e.g., the method shown in Eqn. (4), (5) or (6)).

[0058] Compared with the existing dependent quantization methods where only one previous quantization level of the current element is used to determine the state of the current element, the proposed method in this example takes into account the information of multiple previous elements, thereby making the selection of the quantizer more informed and more accurate. [0059] In another embodiment, a state determination method based on the comparison of the value of the quantization level with zero is proposed. After quantization, the quantization level of some elements may be 0. In such cases, there is no need to use the zero value to calculate the transition of state. As a result, the state used to determine the quantizer for the current element encoding or decoding is calculated only if the previous quantization level ki_₁ is not zero, as shown below. if (*(-1 !=0) state i = stateTransTable[ state^^l & 1 ] (7) else state i ⁼ where k_i-1 is the quantization level of position i — 1 in the encoding/decoding scanning order. If the quantization level at position i — 1 is zero, the state will remain unchanged and thus the quantizer will remain unchanged for the current element at position i. FIG. 7 shows a state transition diagram for this state determination method when the transition table of Table 1 is used. In FIG. 7, the state transition differs from that of Table 1 in a way that the state of the current element remains the same as the state of the previous element if the quantization level of the previous element is zero. Table 3 shows an equivalent state transition table for the state transition diagram in FIG. 7. This embodiment eliminates the step for determining the state of the current element if the quantization level of the previous element is zero, thereby reducing the computational complexity of the encoding and decoding algorithm.

Table 3

[0060] In a further embodiment, instead of using the parity of previous level or sum of previous levels, the parity of the number of previous non-zero levels is used for state transition determination. Let cnt represent a counter and set cnt 0 before encoding/decoding the current coding block. After the quantization of an element (at encoding) or de-quantization for the element (at encoding or decoding), the value of the quantization level is available. If this level is not zero, the counter will be increased by 1. The encoder or decoder will move on to process the next element of the block along the scanning order, which becomes the new current element. The state used to determine the quantizer for the new current element can be updated as follows: state_s = stateTransTable[ state^^l cnt & 1 ] (8)

[0061] Alternatively, the counter cnt can be used to count the number of previous levels that are equal to zero. The state for determining the quantizer for the current element can be determined using Eqn. (8). In these examples, the stateTransTable can be state transition table in Table 1 (by replacing ki_ with cnt), Table 2 (by replacing sum(i — 1) with cnt), the state transition table to be discussed below with respect to FIG. 8, or any other state transition table suitable for dependent quantization. Because this embodiment takes into account the information of multiple previous elements, the selection of the quantizer according to this embodiment can be more accurate or more appropriate than the existing methods.

[0062] In the above embodiments, the state transition shown in Table 1-Table 3 are used as examples. Alternatively, or additionally, a different state transition table may be used to determine the quantizer for a current element. FIG. 8 shows an example of a state transition diagram for dependent quantization and the associated state transition table, according to some embodiments of the present disclosure. In FIG. 8,

represents the quantity whose parity is used to determine the state of the current element at position i, state ;. The quantity v_{i - 1} may be, for example, the quantization level k^ of the previous element at position i — 1 in Eqns. (1), (2), and (7), the sum(i — 1) in Eqns. (3) and (4), the c_i-1 in Eqn. (6), or the cnt in Eqn. (8). In other words, the state transition table shown in FIG. 8 can be used to implement the stateTransT able in any of the above state determination methods or other state determination methods that use the information of previous elements. In one example, if the determined state for the current element at position i is 0 or 1, quantizer Q0 is selected for the current element; otherwise, quantizer Q1 is selected for the current element. By using a different state transition table, high coding efficiency can be achieved for video signals having different signal distributions or statistics.

[0063] FIG. 9 depicts an example of a process 900 for encoding a block for a video via the dependent quantization, according to some embodiments of the present disclosure. One or more computing devices (e.g., the computing device implementing the video encoder 100) implement operations depicted in FIG. 9 by executing suitable program code (e g., the program code implementing the quantization module 115). For illustrative purposes, the process 900 is described with reference to some examples depicted in the figures. Other implementations, however, are possible. [0064] At block 902, the process 900 involves accessing a coding block (or block) of a video signal. The block can be a portion of a picture of the input video, such as a coding unit 402 discussed in FIG. 4 or any type of block processed by a video encoder as a unit when performing the quantization.

[0065] At block 904, which includes 906-910, the process 900 involves processing each element of the block according to a pre-determined scanning order for the block (e g., the scanning order shown in FIG. 5) to generate quantized elements. An element of a coding block may be a residual after inter- or intra-prediction. The element may be a transform coefficient of the residual in a frequency domain or the value of the residual in the pixel domain.

[0066] At block 906, the process 900 involves retrieving the current element according to the scanning order. If no element has been quantized for the current block, the current element will be the first element in the block according to the scanning order. As discussed above with respect to FIG. 5, in some cases, the video encoder performs the quantization starting from the first non-zero element in the block according to the scanning order. In those cases, the current element will be the first non-zero element in the block. If there are elements in the block that have been quantized, the current element will be the element after the last processed element in the scanning order.

[0067] At block 908, the process 900 involves determining the quantizer for the current element based on the elements that precede the current element. As discussed above, the quantizer can be selected according to a quantization state (or “state”) for the current element. The quantization state for the current element (“current state”) can be determined using any method discussed above. For example, the current state can be determined using a state transition table and a value calculated based on the quantization levels of one or more elements preceding the current element. The state transition table can be any of Table 1- Table 3 or the one shown in FIG. 8. The value may be calculated as the parity of the sum of the quantization levels of multiple elements preceding the current element (e.g., Eqns. (3)-(6)). The value can also be calculated as the parity of the number of previous non-zero levels (or zero levels) as shown in Eqn. (8). The value may further be the parity of the previous quantization level but the state is determined only when the previous quantization level is non-zero as shown in Eqn. (7). If the previous quantization level is zero, the quantizer for the current element remains the same as the quantizer for the previous element. Various other ways for determining the quantizer for the current element can be used. At block 910, the process 900 involves quantizing the current element using the determined quantizer to generate a quantized element. [0068] At block 912, the process 900 involves encoding the quantized elements (quantization levels) of the block for inclusion in a bitstream of the video. The encoding may include entropy encoding as discussed above with respect to FIG. 1.

[0069] FIG. 10 depicts an example of a process 1000 for reconstructing a block for a video via the dependent dequantization, according to some embodiments of the present disclosure. One or more computing devices implement operations depicted in FIG. 10 by executing suitable program code. For example, a computing device implementing the video encoder 100 may implement the operations depicted in FIG. 10 by executing the program code for the inverse quantization module 118. A computing device implementing the video decoder 200 may implement the operations depicted in FIG. 10 by executing the program code for the inverse quantization module 218. For illustrative purposes, the process 1000 is described with reference to some examples depicted in the figures. Other implementations, however, are possible.

[0070] At block 1002, the process 1000 involves accessing quantized elements (quantization levels) of a coding block of a video signal. The block can be a portion of a picture of the input video, such as a coding unit 402 discussed in FIG. 4 or any type of block processed by a video encoder or decoder as a unit when performing the de-quantization. For an encoder, the quantized elements may be obtained by quantizing the elements of the block. For a decoder, the quantized elements may be obtained by performing entropy decoding on binary strings parsed from an encoded bitstream of the video.

[0071] At block 1004, which includes 1006-1010, the process 1000 involves processing each quantized element of the block according to a pre-determined scanning order for the block (e.g., the scanning order shown in FIG. 5) to generate de-quantized elements. At block 1006, the process 1000 involves retrieving the current quantized element according to the scanning order. If no quantized element has been de-quantized for the current block, the current quantized element will be the first quantized element for the block according to the scanning order. As discussed above with respect to FIG. 5, in some cases, the video encoder performs the quantization starting from the first non-zero element in the block according to the scanning order. In those cases, the first quantized element will be the first non-zero quantization level in the block. If there are elements in the block that have been de-quantized, the current quantized element will be the quantization level after the last de-quantized element in the scanning order. [0072] At block 1008, the process 1000 involves determining the quantizer for the current quantized element based on the quantization levels that precede the current quantized element. As discussed above, the quantizer can be selected according to a quantization state for the current quantized element. The quantization state for the current quantized element can be determined using any method discussed above. For example, the current state can be determined using a state transition table and a value calculated based on the quantization levels of one or more elements preceding the current quantized element. The state transition table can be one of Table 1-Table 3 or the one shown in FIG. 8. The value may be calculated as the parity of the sum of the quantization levels of multiple elements preceding the current quantized element (e g., Eqns. (3)-(6)). The value can also be calculated as the parity of the number of previous non-zero levels (or zero levels) as shown in Eqn. (8). The value may further be the parity of the previous quantization level but the state is determined only when the previous quantization level is non zero as shown in Eqn. (7). If the previous quantization level is zero, the quantizer for the current quantized element remains the same as the quantizer for the previous quantized element. Various other ways for determining the quantizer for the current quantized element can be used At block 1010, the process 1000 involves de-quantizing the current quantized element using the determined quantizer to generate a de-quantized element.

[0073] At block 1012, the process 1000 involves reconstructing the block in the pixel domain based on the de-quantized elements of the block. The reconstruction may include inverse transform as discussed above with respect to FIGS. 1 and 2. The reconstructed block may also be used to perform intra- or inter-prediction for other blocks or pictures in the video by the encoder or the decoder as described above with respect to FIGS. 1 and 2. The reconstructed block may also be further processed to generate a decoded block for displaying along with other decoded blocks in the picture at the decoder side.

[0074] It should be noted that the state transition table shown in Table 1 and in FIG. 8 may both be used during the encoding and decoding of a video. For example, for some blocks in a video, one state transition table is used and for other blocks in the video, the other state transition table is used. Likewise, more than two state transition tables may be used in the encoding and decoding of a video signal.

[0075] It should be further understood that while in the above description, the dependent quantization is applied to video coding, the same technology can also be applied to image coding. For example, when compressing an image, the image can be divided into blocks and elements of each block (with or without transform) can be quantized as discussed above. The de-compression of the image can be performed by de-quantizing the elements in each block as discussed above. [0076] Commtine System Example for Inwlementins Dependent Quantization for Video Codins

[0077] Any suitable computing system can be used for performing the operations described herein. For example, FIG. 11 depicts an example of a computing device 1100 that can implement the video encoder 100 of FIG. 1 or the video decoder 200 of FIG. 2. In some embodiments, the computing device 1100 can include a processor 1112 that is communicatively coupled to a memory 1114 and that executes computer-executable program code and/or accesses information stored in the memory 1114. The processor 1112 may comprise a microprocessor, an application-specific integrated circuit (“ASIC”), a state machine, or other processing device. The processor 1112 can include any of a number of processing devices, including one. Such a processor can include or may be in communication with a computer-readable medium storing instructions that, when executed by the processor 1112, cause the processor to perform the operations described herein.

[0078] The memory 1114 can include any suitable non-transitory computer-readable medium. The computer-readable medium can include any electronic, optical, magnetic, or other storage device capable of providing a processor with computer-readable instructions or other program code. Non-limiting examples of a computer-readable medium include a magnetic disk, memory chip, ROM, RAM, an ASIC, a configured processor, optical storage, magnetic tape or other magnetic storage, or any other medium from which a computer processor can read instructions. The instructions may include processor-specific instructions generated by a compiler and/or an interpreter from code written in any suitable computer-programming language, including, for example, C, C++, C#, Visual Basic, Java, Python, Perl, JavaScript, and ActionScript.

[0079] The computing device 1100 can also include a bus 1116. The bus 1116 can communicatively couple one or more components of the computing device 1100. The computing device 1100 can also include a number of external or internal devices such as input or output devices. For example, the computing device 1100 is shown with an input/output (“I/O”) interface 1118 that can receive input from one or more input devices 1120 or provide output to one or more output devices 1122. The one or more input devices 1120 and one or more output devices 1122 can be communicatively coupled to the I/O interface 1118. The communicative coupling can be implemented via any suitable manner (e.g., a connection via a printed circuit board, connection via a cable, communication via wireless transmissions, etc.). Non-limiting examples of input devices 1120 include a touch screen (e g., one or more cameras for imaging a touch area or pressure sensors for detecting pressure changes caused by a touch), a mouse, a keyboard, or any other device that can be used to generate input events in response to physical actions by a user of a computing device. Non-limiting examples of output devices 1122 include an LCD screen, an external monitor, a speaker, or any other device that can be used to display or otherwise present outputs generated by a computing device.

[0080] The computing device 1100 can execute program code that configures the processor 1112 to perform one or more of the operations described above with respect to FIGS. 1-10. The program code can include the video encoder 100 or the video decoder 200. The program code may be resident in the memory 1114 or any suitable computer-readable medium and may be executed by the processor 1112 or any other suitable processor.

[0081] The computing device 1100 can also include at least one network interface device 1124. The network interface device 1124 can include any device or group of devices suitable for establishing a wired or wireless data connection to one or more data networks 1128. Non limiting examples of the network interface device 1124 include an Ethernet network adapter, a modem, and/or the like. The computing device 1100 can transmit messages as electronic or optical signals via the network interface device 1124.

[0082] General Considerations

[0083] Numerous specific details are set forth herein to provide a thorough understanding of the claimed subject matter. However, those skilled in the art will understand that the claimed subject matter may be practiced without these specific details. In other instances, methods, apparatuses, or systems that would be known by one of ordinary skill have not been described in detail so as not to obscure claimed subject matter.

[0084] Unless specifically stated otherwise, it is appreciated that throughout this specification discussions utilizing terms such as “processing,” “computing,” “calculating,” “determining,” and “identifying” or the like refer to actions or processes of a computing device, such as one or more computers or a similar electronic computing device or devices, that manipulate or transform data represented as physical electronic or magnetic quantities within memories, registers, or other information storage devices, transmission devices, or display devices of the computing platform.

[0085] The system or systems discussed herein are not limited to any particular hardware architecture or configuration. A computing device can include any suitable arrangement of components that provide a result conditioned on one or more inputs. Suitable computing devices include multi-purpose microprocessor-based computer systems accessing stored software that programs or configures the computing system from a general purpose computing apparatus to a specialized computing apparatus implementing one or more embodiments of the present subject matter. Any suitable programming, scripting, or other type of language or combinations of languages may be used to implement the teachings contained herein in software to be used in programming or configuring a computing device.

[0086] Embodiments of the methods disclosed herein may be performed in the operation of such computing devices. The order of the blocks presented in the examples above can be varied — for example, blocks can be re-ordered, combined, and/or broken into sub-blocks. Some blocks or processes can be performed in parallel.

[0087] The use of “adapted to” or “configured to” herein is meant as open and inclusive language that does not foreclose devices adapted to or configured to perform additional tasks or steps. Additionally, the use of “based on” is meant to be open and inclusive, in that a process, step, calculation, or other action “based on” one or more recited conditions or values may, in practice, be based on additional conditions or values beyond those recited. Headings, lists, and numbering included herein are for ease of explanation only and are not meant to be limiting. [0088] While the present subject matter has been described in detail with respect to specific embodiments thereof, it will be appreciated that those skilled in the art, upon attaining an understanding of the foregoing, may readily produce alterations to, variations of, and equivalents to such embodiments. Accordingly, it should be understood that the present disclosure has been presented for purposes of example rather than limitation, and does not preclude the inclusion of such modifications, variations, and/or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art.

Claims

1. A method for reconstructing a block for a video, the method comprising: accessing a plurality of quantized elements that are associated with the block; processing the plurality of quantized elements according to an order for the block to generate respective de-quantized elements, the processing comprising: obtaining a current quantized element of the block from the plurality of quantized elements; determining a quantizer for the current quantized element based on at least two quantized elements preceding the current quantized element or based on a comparison of an element immediately preceding the current quantized element with zero; and de-quantizing the current quantized element based on the quantizer to generate a de-quantized element; and reconstructing the block based on the de-quantized elements.

2. The method of claim 1, wherein the block comprises a coding unit

3. The method of claim 1, wherein the quantized elements associated with the block comprise quantized pixels of the block or quantized transform coefficients of the block.

4. The method of claim 1, wherein determining the quantizer for the current quantized element based on at least two quantized elements preceding the current quantized element comprises: determining the quantizer for the current quantized element based on a parity of a sum of at least two quantized elements preceding the current quantized element according to the order.

5. The method of claim 4, wherein the at least two quantized elements preceding the current quantized element comprise all quantized elements of the block that precede the current quantized element according to the order.

6. The method of claim 1, wherein determining the quantizer for the current quantized element based on the comparison of an element immediately preceding the current quantized element with zero comprises: determining the quantizer for the current quantized element to be a quantizer for the element immediately preceding the current quantized element in response to determining that the element immediately preceding the current quantized element is zero; and determining the quantizer for the current quantized element according to a state transition table in response to determining that the element immediately preceding the current quantized element is not zero.

7. The method of claim 1, wherein determining the quantizer for the current quantized element based on at least two quantized elements preceding the current quantized element comprises: determining the quantizer for the current quantized element based on a parity of a number of non-zero quantized elements or zero quantized elements in the block preceding the current quantized element according to the order.

8. The method of claim 1, wherein determining the quantizer for the current quantized element comprises: determining a current state for quantization based on a previous state for quantization for an element immediately preceding the current quantized element and a state transition table:

wherein v is a value determined based on the at least two quantized elements preceding the current quantized element; and determining the quantizer for the current quantized element according to the current state for quantization.

9. A non-transitory computer-readable medium having program code that is stored thereon, the program code executable by one or more processing devices for performing operations comprising: accessing a plurality of quantized elements that are associated with a block of a video; processing the plurality of quantized elements according to an order for the block to generate respective de-quantized elements, the processing comprising: obtaining a current quantized element of the block from the plurality of quantized elements; determining a current state for quantization based on a previous state for quantization for an element immediately preceding the current quantized element and a state transition table:

wherein v is a value determined based on at least one quantized element preceding the current quantized element; determining a quantizer for the current quantized element according to the current state for quantization; and de-quantizing the current quantized element based on the quantizer to generate a de-quantized element; and reconstructing the block based on the de-quantized elements.

10. The non-transitory computer-readable medium of claim 9, wherein the processing further comprises: prior to determining the current state for quantization, determining whether the quantized element immediately preceding the current quantized element is zero; and in response to determining that the quantized element immediately preceding the current quantized element is zero, determining the quantizer for the current quantized element to be a quantizer for the quantized element immediately preceding the current quantized element, wherein determining the current state for quantization is performed in response to determining that the quantized element immediately preceding the current quantized element is not zero.

11. The non-transitory computer-readable medium of claim 9, wherein determining the quantizer for the current quantized element based on at least one quantized element preceding the current quantized element comprises: determining the quantizer for the current quantized element based on a parity of a sum of at least two quantized elements preceding the current quantized element according to the order.

12. The non-transitory computer-readable medium of claim 9, wherein determining the quantizer for the current quantized element based on at least one quantized element preceding the current quantized element comprises: determining the quantizer for the current quantized element based on a parity of a number of non-zero quantized elements or zero quantized elements in the block preceding the current quantized element according to the order.

13. The non-transitory computer- readable medium of claim 9, wherein the block comprises a coding unit.

14. The non-transitory computer-readable medium of claim 9, wherein the quantized elements associated with the block comprise quantized pixels of the block or quantized transformed coefficients of the block.

15. A system comprising: a processing device; and a non-transitory computer-readable medium communicatively coupled to the processing device, wherein the processing device is configured to execute program code stored in the non- transitory computer-readable medium and thereby perform operations comprising: accessing a plurality of elements associated with a block of a video; processing the plurality of elements according to an order for the block, the processing comprising: obtaining a current element of the block from the plurality of elements; determining a quantizer for the current element based on at least two previous elements before the current element, or based on a comparison of an element immediately preceding the current element with zero; and quantizing the current element based on the quantizer to generate a quantized element; and encoding the quantized elements into a bitstream representing the video.

16. The system of claim 15, wherein determining the quantizer for the current element based on at least two elements preceding the current element comprises: determining the quantizer for the current element based on a parity of a sum of at least two elements preceding the current element according to the order.

17. The system of claim 16, wherein the at least two elements preceding the current element comprise all elements of the block that precede the current element according to the order.

18. The system of claim 15, wherein determining the quantizer for the current element based on the comparison of an element immediately preceding the current element with zero comprises: determining the quantizer for the current element to be a quantizer for the element immediately preceding the current element in response to determining that the element immediately preceding the current element is zero; and determining the quantizer for the current element according to a state transition table in response to determining that the element immediately preceding the current element is not zero.

19. The system of claim 15, wherein determining the quantizer for the current element based on at least two elements preceding the current element comprises: determining the quantizer for the current element based on a parity of a number of non zero elements in the block preceding the current element according to the order.

20. The system of claim 15, wherein determining the quantizer for the current element comprises: determining a current state for quantization based on a previous state for quantization for an element immediately preceding the current element and a state transition table

wherein v is a value determined based on the at least two elements preceding the current element; and determining the quantizer for the current element according to the current state for quantization.