RU2645269C2 - Video decoding method - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: quantized coefficient dequantizing method comprises the steps of: receiving the quantized coefficients representing a video block representing a plurality of pixels and a quantization matrix; and dequantizing, using, at least, the quantization matrix, the conversion size and the quantized coefficients to determine the conversion coefficients. Dequantization includes: a process of multiplying the quantized coefficients by one integer depending on the quantization parameter, the index of the quantized coefficient, and the size of the conversion; a process of clipping the result of multiplying the quantized coefficients by one integer depending on the quantization parameter, the index of the quantized coefficient, and the size of the conversion. Clipping is a clipping process to the right; a process for cutting off the result of the said clipping process to the predetermined bit depth, and storing the result of the said clipping process in memory.

EFFECT: reducing the memory required when decoding video.

4 cl, 14 dwg

Description

FIELD OF TECHNOLOGY

The present invention relates to a method for decoding video.

BACKGROUND OF THE INVENTION

Existing video coding standards, such as H.264 / AVC, typically provide relatively high coding efficiency due to increased computational complexity. As computational complexity increases, coding and / or decoding rates tend to decrease. Also, the desire for high-quality playback tends to increase over time, which tends to increasing requirements for increasing memory and increasing requirements for memory bandwidth. Increasing memory requirements and increasing memory bandwidth requirements are leading to increasingly expensive and computationally complex circuits, especially in the case of embedded systems.

Referring to FIG. 1, many decoders (and encoders) receive (and encoders provide) encoded data for image blocks. Typically, the image is divided into blocks, and each of the blocks is encoded in some way, for example using discrete cosine transform (DCT), and is provided to the decoder. The decoder receives the encoded blocks and decodes each of the blocks in some way, for example, using the inverse discrete cosine transform. In many cases, decoding of the image coefficients of the image block is performed using matrix multiplication. Matrix multiplication can be performed for the horizontal direction, and matrix multiplication can be performed for the vertical direction. As an example, for 8-bit values, the first multiplication in some cases can lead to 16-bit values, and the second multiplication can lead to 24-bit values. In addition, the encoding of each image block is usually quantized, which maps the encoding values to a smaller set of quantized coefficients used for transmission. Quantization requires dequantization using a decoder, which maps the set of quantized coefficients used for transmission to approximate encoding values. The number of desired bits for dequantized data is a design parameter. The possibility of large values resulting from matrix multiplication and the dequantization operation is problematic for systems with limited resources, especially for embedded systems.

SUMMARY OF THE INVENTION

One embodiment of the present invention discloses a method for decoding a video. The method comprises (a) receiving quantized coefficients representing a video block representing a plurality of pixels; (b) dequantizing said quantized coefficients; (c) applying a modification to said dequantized coefficients based on a quantization matrix; (d) an inverse transform of said dequantized coefficients to determine a decoded remainder.

The above and other objects, features and advantages of the invention will become apparent upon consideration of the following detailed description of the invention in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an encoder and a decoder.

FIG. 2 illustrates a decoder with a dequantizer and an inverse transform.

FIG. 3A and FIG. 3B illustrate a modified dequantizer.

FIG. 4 illustrates a modified inverse transform.

FIG. 5 illustrates another decoder.

FIG. 6 illustrates another decoder.

FIG. 7 illustrates another modified dequantizer.

FIG. 8 illustrates another modified inverse transform.

FIG. 9 illustrates another modified dequantizer.

FIG. 10 illustrates another modified inverse transform.

FIG. 11 illustrates another modified dequantizer.

FIG. 12 illustrates another modified dequantizer.

FIG. 13 illustrates another modified dequantizer.

DESCRIPTION OF EMBODIMENTS

Referring to FIG. 2, a decoder for dequantizing and inverting received quantized coefficients from an encoder for an image block is illustrated in the relevant part. The decoder receives the quantized coefficients 200 in the dequantizer 210. The coefficients obtained from the dequantifier 210 are stored in the memory 220. The coefficients stored in the memory 220 are then processed by a pair of inverse transforms 230 to determine the decoded remainder 310. The inverse transform maps data from the transform domain to the spatial domain using the matrix multiplication operator.

The decanter 210 includes a process of scaling cancellation 240. The process of canceling scaling cancels the scale of the quantized coefficients 200. The process of canceling scaling corresponds to multiplying the level values (also called quantized coefficients 200) by one integer depending on the quantization parameter (QP), coefficient index, and transform size (N). An example of a de-scaling process 240 may include a Level * Integer value (Remainder, coefficient index) * 16 for the dequantizer used before the inverse 8x8 transform, and Level * An integer value (Remainder, coefficient index) for the dequantifier used before other transform sizes. The de-scaling process 240 is preferably based on a remainder function, a transform size (N), and / or a coefficient index (eg, position) to determine an intermediate set of 250 values. The remainder is the sum of the quantization parameter (QP) and P * BitIncrement modulo P ((QP + P * BitIncrement)% P). Modulo (remainder of division), as defined in the H.264 / AVC standard, is defined as: x% y, as the remainder of x divided by y, specified only for integers x and y, with x> = 0 and y > 0. In one embodiment, P may take the value 6. Adjustment mechanism A 260 may be applied to values 250, which may be a variable depending on the size of the transform (N) and / or the function of the received Period. The period is the sum of the quantization parameter (QP) and P * BitIncrement divided by P ((QP + P * BitIncrement) / P), where "BitIncrement" is an increment of the bit depth. "/", as defined in the H.264 / AVC standard, is defined as: integer division with truncation of the result in the direction of zero. For example, 7/4 and -7 / -4 are truncated to 1, while -7/4 and 7 / -4 are truncated to -1. In one embodiment, P may take the value 6. The resulting values 250, possibly further modified by mechanism A 260, may be further modified by a factor of 2 ^{(Period + B)} 270. B is a variable that depends on the size of the transform. The results of modification 270 are stored in memory 220. Inverse transform 230 may perform 1-dimensional inverse horizontal transform 280, which is stored in memory 290. Inverse transform 230 may also perform 1-dimensional inverse vertical transform 300, which leads to decoded remainder 310. Conversions 280 and 300 can be swapped at will.

The memory bandwidth of the video decoder illustrated in FIG. 2, when implemented as part of Part 10: Advanced Video Coding, publication ISO: ISO / IEC 14496-10: 2005 - “Information Technology - Coding Of Audio-Visual Objects” (incorporated herein by reference) (H. Standard 264 / AVC), may be limited by some restrictive condition. For example, in Section 8.5.10 of the H.264 / AVC standard, the memory sample for 4 × 4 DC conversion factors is limited to including the following provisions: “The binary signal stream must not contain data that leads to any element f _ij for f for i, j = 0, ..., 3, which exceeds the range of integer values from −2 ^{(7 + bitDepth)} to 2 ^{(7 + bitDepth)} −1 inclusive "and" The stream of binary signals should not contain data that leads to which element-or dcY _ij y dcY with i, j = 0, ..., 3, which exceeds the range of integer values from -2 ^{(7 + bitDepth)} about 2 ^{(7 + bitDepth)} -1 inclusive. " The H.264 / AVC standard includes a similar memory limit for other residual blocks. In addition to including a comprehensive memory bandwidth limit, the H.264 / AVC standard does not include any mechanism to ensure that this limit is respected. Similarly, JCT-VC, "Draft Test Model Under Consideration", JCTVC-A205, JCT-VC Meeting in Dresden, April 2010 (JCT-VC), incorporated herein by reference, also does not include a compliance mechanism memory bandwidth. For reliability, the decoder must be prepared to receive binary signal streams that may violate these boundaries, which may be due to transmission errors, damage to the compatible binary signal stream, or an inappropriate encoder. To mitigate such possible limitations, a decoder often includes additional memory bandwidth at additional cost and complexity to accommodate the incompatible binary signal streams that are provided.

To provide a more computationally reliable decoder with limited memory bandwidth and / or storage requirements, the decoder should be suitably modified. However, along with the modification of the decoder to reduce the required memory, the corresponding distortion characteristic should not be significantly degraded depending on the video transmission speed. Otherwise, although it is possible to reduce the required memory, the resulting video quality will not be suitable for viewers to watch. Modification 270 leads to a change in the coefficient value based on changes in steps in the quantization parameter P and, accordingly, can significantly increase the size of the required memory. For example, the coefficients are doubled every 6 steps in the quantization parameter P. An increased value results in one or more zeros that are included as the least significant bits. Preferably, the decoder is modified appropriately to implement a memory bandwidth compliance mechanism.

Referring to FIG. 3A, with this understanding of the operation of the dequantifier 210 (see FIG. 2), the advanced dequantifier 400 (see FIGS. 3A and 3B) receives the quantized coefficients 405 and cancels the scale (410) of the quantized coefficients, preferably based on the remainder function, the transform size ( N) and / or a coefficient index (eg, position) to determine an intermediate set of values 420. The quantization parameter may be in the form of a matrix of frequency-dependent values. Typically, the position of a value in a quantization matrix refers to its frequency (for example, spatial frequency), and accordingly, the quantizer may be unstable for each block or group of pixels. In general, the quantization parameters can be changed in any suitable way, for example, in each frame, each block, each set of blocks, or otherwise as desired. An optional adjustment mechanism 430 may be used using a variable C, which is preferably a variable depending on the size of the transform (N) and / or the function of one or more received quantization parameters (QP) to determine the resulting data 440. The resulting data 440 from the quantized coefficients 405 may include unreliable data or other data that is not compatible with the standard, and accordingly, the dequantifier 400 may introduce a fixed constraint on the resulting data 440. The result The data 440 can be clipped (450) to a predetermined bit depth, and accordingly, the N × N data block is stored in memory in the dequantifier 400. For example, clipping 450 to a preset bit depth of 16 bits causes any values in excess of 32767 to be set. to the maximum value, namely 32767. Also, clipping to a predetermined bit depth of 16 bits leads to the fact that any values less than -32768 are set to the minimum value, namely -32768. Other bit depths and clipping values may also be used. Thus, the required maximum memory bandwidth is limited by the system in a way independent of the input quantized coefficients 405. This reduces the computational complexity of the system and reduces the required memory, which is especially suitable for embedded systems.

After setting clipping 450, data with a maximum predetermined bit depth is modified using a factor of 460 2 ^{(Period + B)} . Modification results 460 are provided in the form of coefficients 470. Execution result 2 ^{(Period + B)} 460 after clipping 450 reduces distortion loss depending on the transmission speed. Preferably, the adjustment mechanism 430 used for 8 × 8 conversion factors is 2 ^(5-Period) , and 2 ^{(Period + B)} 460 is 2 ^(Period-6) . Process 460 may optionally be based on a transform size function (N) or a received quantization parameter (QP) function. Also, the adjustment mechanism 430 used for transform coefficients of a different size (for example, 4 × 4, 16 × 16 and 32 × 32) preferably has a variable B set to zero, and therefore the value 2 ^{(Period + B)} 460 is equal to 2 ^(Period) . The result of expression 2 ^{(Period + B)} can be implemented as a process of shifting the discharge to the right by (Period + B), as shown in modification 460. In addition, expressions 2 ^(5-Period) , 2 ^{(Period + B)} and 2 ^{( Period-6)} can be implemented as shear processes. Also, B may be a function of N, and C may be a function of N. Referring to FIG. 3B, a specific implementation of FIG. 3A.

Referring to FIG. 3B, an 8 × 8 dequantizer can be described as follows.

Int iAdd = (1 << 5) >> Period

where << is a shift of the discharge to the left, >> is a shift of the discharge to the right, Int is an integer operation, and iAdd is a variable.

Without clipping:

dstCoef = ((iLevel * iDeScale * 16 + iAdd) << Period) >> 6

With clipping:

dstCoef = (CLIP_TO_16BITS (iLevel * iDeScale * 16 + iAdd) << Period) >> 6

Referring to FIG. 3B, 4 × 4, 16 × 16, 32 × 32 and N × N can be described as follows.

Without clipping:

dstCoef = (iLevel * iDeScale) << Period

With clipping:

dstCoef = CLIP_TO_16BITS (iLevel * iDeScale) << Period

In any case, scaling 2 ^{Period + B} in FIG. 3A or scaling 2 ^Period-6 in FIG. 3B is performed before the inverse transform is performed, accordingly, leading to the fact that the first stage of the inverse transform depends on the period of the QP transform. The QP dependency of the embodiments of FIG. 3A and FIG. 3B, after being stored in memory, increases the computational complexity of the system, which could decrease if this dependence on QP decreases.

Referring to FIG. 4, coefficients 470 from the dequantifier 400 (see FIGS. 3A and 3B) are provided to an inverse transform 480 designed to provide a decoded remainder 490 that has acceptable distortion loss depending on the transmission rate. Coefficients 470 are preferably converted using a 1-dimensional inverse horizontal (or vertical) conversion 500. Based on the desired number of output bits to maintain acceptable distortion losses depending on the transmission rate, the result of the conversion 500 can be modified by the right shift process 510 for the desired number of bits . Thus, the selected number of the least significant bits is discarded in order to reduce the required system memory. For example, if 19 bits are supposed to be obtained from the inverse transform 500, and it is desirable to have a 16-bit result, then the process 510 of shifting the bit to the right removes the 3 least significant bits. The resulting shifted bits are cut off (520) to a predetermined threshold value. An example of a predetermined threshold value may be 16 bits. Clipping 520 further enforces a memory bandwidth constraint, the results of which are stored in memory 530. Data stored in memory 530 is significantly reduced by a shift 510 that removes the least significant bit (s). The data stored in the memory 530 is then shifted to the left by the left shift process 540, preferably by the same number of bits as the right shift process 510. A shift results in zeros in the lowest order (s). The shifted data is then preferably converted using a 1-dimensional inverse vertical (or horizontal) transform 550, resulting in a decoded remainder 490.

Loss of distortion depending on the transmission speed depends on the number of bits used in processing and the size of the data block. Preferably, the right shift process 510 and the left shift process 540 are dependent on the block size N (number of horizontal pixels × number of vertical pixels for a rectangular block of pixels). For example, for a 4 × 4 block, the shift can be 3, for an 8 × 8 block, a shift can be 2, for a 16 × 16 block, a shift can be 8, and for a 32 × 32 block, a shift can be 9. Alternatively the right shift process 510 and the left shift process 540 can be determined based on a parameter, for example, a quantization parameter (QP) transmitted in a binary signal stream, internal bit depth increment (IBDI) parameters, conversion precision increase (TPE), or other parameters, selectable by decoder.

Referring to FIG. 5, in another embodiment, the decoder receives the quantized coefficients, which are processed by any suitable dequantizer 600 and any suitable inverse transform 610. It is desirable to include an exact memory bandwidth constraint, which is preferably implemented by enabling clipping function 620. After clipping function 620, data can be stored in memory 630, which is then used for inverse transform 610.

Referring to FIG. 6, in another embodiment, the decoder receives the quantized coefficients, which are processed by any suitable dequantifier 700 and any suitable inverse transform 710. For example, the inverse transform may be the transform illustrated in FIG. 4. It is desirable to include an exact limit on the memory bandwidth to reduce the computational complexity, which is preferably implemented by enabling the clipping function 720. After clipping function 720, data can be stored in memory 730, which is then used for the inverse transform 710. It is also desirable to include an explicit memory bandwidth limit, which is preferably implemented by enabling clipping function 740 between a pair of 1-D transforms. 1-dimensional transformations can be performed in any order or in any way. After clipping function 740, data can be stored in memory 750.

Referring to FIG. 7, in order to reduce the dependence on QP, an embodiment may include a modification 460 (see FIGS. 3A and 3B) performed before storing the resulting coefficients in memory using shift operation 705, for example 2 ^{(QP / 6 + B)} . Similarly to FIG. 3A and FIG. 3B, it is possible to cancel the scale (410) of the quantized coefficients 405 and, if desired, modify them using the adjustment mechanism 430. Thus, the quantization parameters need not be stored together with the coefficients in the memory, since further inverse transformations can be performed in a manner independent of the quantization parameters, as illustrated in FIG. 8.

Referring to FIG. 9, an additional method includes cutting off the number of bits while storing in memory to a suitable number, for example 16 bits. Referring to FIG. 10, an additional method includes cutting off the number of bits while storing in memory to a suitable number, for example 16 bits. Although this approach reduces the computational complexity of the system, the resulting video tends to degrade video quality.

Referring to FIG. 11, a modified dequantizer 800 is particularly suited to de-scaling with a frequency-dependent quantization matrix that includes a quantization matrix 820 QP (i, j). The quantized coefficients 805 are called Level (i, j) 815. Cancellation 810 scaling can be based on the remainder (i, j) 825, which is described as Residue (i, j) = (QP (i, j) + P * BitIncrement)% P where P is preferably 6 and "BitIncrement" is an increment of bit depth. An intermediate result of 835 K (i, j) can be described as K (i, j) = A (Remainder (i, j)) * Level (i, j) where the process of scaling off 810 multiplies by the value A (Remainder (i, j )), which depends on the term Residue (i, j). Clipping 850 may be described as Clip K (i, j) up to 16 bits. Intermediate result 855 J (i, j) can be described as J (i, j) = Clip (K (i, j), 16). Shift process 860 2 ^{Period (i, j)} can be described as C (i, j) = J (i, j) << (Period (i, j)), where Period (i, j) = (QP (i, j) + P * BitIncremenet) / P. Preferably, the process is integer division, and P is 6. Accordingly, the QP (i, j) quantization matrix 820 is provided to the scaling cancel process 800 and the shift process 860, which is the process after clipping 850 and storing it in memory. The provision of QP (i, j) quantization matrix 820 to the shift process 860 is typically provided by a “side channel” process 830. Thus, the shift of the coefficients is performed by the corresponding values in the quantization matrix.

In order to further reduce the computational complexity of the system, it is desirable that the entire quantization matrix, which may be significant in some embodiments, should not be provided in the shift process. Referring to FIG. 12, a modified method 900 for de-scaling a frequency-dependent quantization matrix includes a quantization matrix 920 QP (i, j). The quantized coefficients 905 are called Level (i, j) 915. The cancellation of scaling 910 may be based on the remainder (i, j) 925, which is described as the remainder (i, j) = (QP (i, j) + P * BitIncrement)% P , where P is preferably 6. Intermediate result 935 K (i, j) can be described as K (i, j) = A (Residue (i, j)) * Level (i, j) << (Period (i, j ) -Period (QPmin)), where QPmin is the minimum of the set of values of QP (i, j) in the quantization matrix. Clipping 950 can be described as Clip K (i, j) up to 16 bits. Intermediate result 955 J (i, j) can be described as J (i, j) = Clip (K (i, j), 16). The process 960 shift 2 ^{Period (QPmin)} can be described as C (i, j) = J (i, j) << Period (QPmin), where Period (QPmin) = (QPmin + P * BitIncrement) / P, where P is preferred equals 6. Accordingly, the quantization matrix QP (i, j) is provided to the scaling cancel process and the initial shift process, which is the process before clipping and storing in memory. The minimum quantization function 975 determines the minimum quantization value for the matrix. This minimum quantization value 985 is provided to the shift processes 960. Thus, only one value, namely, the minimum quantization value or Period (QPmin), is usually provided by the side channel process 990. Thus, the coefficients are shifted by the corresponding values in the quantization matrix, but only a limited amount of data needs to be provided in addition to those stored in memory. Typically, the minimum quantization value may be any data set smaller than the entire quantization matrix.

Referring to FIG. 13, a modified de-scaling method 1000 with a frequency-dependent quantization matrix includes a quantization matrix 1020 QP (i, j). The quantized coefficients 1005 are called Level (i, j) 1015. Scaling cancellation 1010 can be based on the remainder (i, j) 1025, which is described as the remainder (i, j) = (QP (i, j) + P * BitIncrement)% P where P is preferably 6. Intermediate result 1035 K (i, j) can be described as K (i, j) = A (Residue (i, j)) * Level (i, j) << PERIOD_CALC1 (QP (i, j) -QPmin), where PERIOD_CALC1 (x) = floor [x / 6]. Clipping 1050 can be described as Clip K (i, j) up to 16 bits. Intermediate result 1055 J (i, j) can be described as J (i, j) = Clip (K (i, j), 16). The process 1060 shift 2 ^{PERIOD_CALC2 (QPmin)} can be described as C (i, j) = J (i, j) << PERIOD_CALC2 (QPmin), where PERIOD_CALC2 (x) = ceil [x + 6 * BitIncrement) / P], where P is preferably 6. The functions of the largest integer (antier) and the smallest integer (ceiling) are preferably floating point operations. Accordingly, the quantization matrix QP (i, j) is provided to the scaling cancel process and the initial shift process, which is the process before clipping and storing in memory. The minimum quantization function 1075 determines the minimum quantization value for the matrix, for example, using the largest integer and smallest integer functions. This minimum quantization value 1085 is provided to the shift processes 1060 to make the shift smaller than it would otherwise, and then the corresponding shift in the other direction is less than it would otherwise. Thus, only one value, namely the minimum quantization value or PERIOD_CALC2 (QPmin), is usually provided through the side channel process 1090. Thus, the coefficients are shifted by the corresponding values in the quantization matrix, but only a limited amount of data needs to be provided in addition to those stored in memory. Typically, the values of the smallest integer and / or largest integer can be any data set smaller than the entire quantization matrix.

The terms and expressions that are used in the preceding description of the invention are used there as description terms and not limitation, and when using such terms and expressions there is no intention to exclude equivalents of the shown and described features or parts thereof; it is recognized that the scope of the invention is defined and limited only by the claims, which follows.

Some embodiments of the present invention disclose a method for decoding video, wherein said dequantization is based on a coefficient index, a binary increment, a transform size, and said quantization matrix.

Some embodiments of the present invention disclose a method for video decoding, wherein said modification is based on a coefficient index, a binary increment, a transform size, and said quantization matrix.

Some embodiments of the present invention disclose a method for video decoding, wherein said modification is based on 2 ^{(QP / 6 + B)} , where QP is a quantization matrix and B refers to a transform size.

Some embodiments of the present invention disclose a method for video decoding, wherein said modified dequantized coefficients are cut off before being stored in the memory and said cut off coefficients are read from said memory for said inverse transform.

Some embodiments of the present invention disclose a method for decoding a video, wherein said dequantization and said modification of said dequantized coefficients are together based on said quantization matrix.

Some embodiments of the present invention disclose a method for decoding a video, wherein said modification is a shift operation.

Some embodiments of the present invention disclose a method for video decoding, wherein said dequantized coefficients are cut off before said modification.

Some embodiments of the present invention disclose a method for decoding a video, wherein said modification is based on a single value based on said quantization matrix.

Some embodiments of the present invention disclose a method for decoding a video, wherein said single value is based on a minimum function.

Some embodiments of the present invention disclose a method for decoding video, further comprising another modification acting on said dequantized coefficients based on said quantization matrix, and the resulting data is then cut off by said method.

Some embodiments of the present invention disclose a method for video decoding, wherein said dequantized coefficients are further modified as a result of the operation of the adjustment mechanism.

Some embodiments of the present invention disclose a method for decoding video, where said adjustment mechanism is a variable depending on the size of the transform.

Some embodiments of the present invention disclose a method for decoding a video, wherein said adjustment mechanism is a function of at least one of a received quantization parameter and a transform size.

In other words, although the invention is described in this way, it will become apparent that it can be changed in many ways. Such changes should not be construed as a departure from the essence and scope of the invention, and all such modifications that would be obvious to a person skilled in the art are intended to be included in the scope of the following claims.

Claims (10)

1. A method for dequantizing quantized coefficients representing a video block containing the steps of: (a) accepting (i) quantized coefficients representing a video block representing a plurality of pixels, and (ii) a quantization matrix; and (b) de-quantizing using at least a quantization matrix, transform size, and quantized coefficients to determine transform coefficients; while dequantization includes: - the process of multiplying the quantized coefficients by one integer, depending on the quantization parameter (QP), the index of the quantized coefficient and the size of the transformation (N), - the process of shifting the result of the aforementioned process of multiplying the quantized coefficients by one integer, depending on the quantization parameter, the index of the quantized coefficient and the size of the transformation, and the shift is a shift operation to the right, - a process of cutting off the result of said shift process to a predetermined bit depth and - the process of storing the result of the above clipping process in memory. 2. The method of claim 1, wherein the dequantization uses at least a binary increment, a transform size, and a quantization matrix. 3. The method of claim 1, wherein the multiplication process includes multiplying the quantized coefficients by a quantization matrix modified using quantization information. 4. The method of claim 1, wherein the clipping process includes 16-bit clipping.

Images