WO2012015312A1

WO2012015312A1 - Implementation of run level coding

Info

Publication number: WO2012015312A1
Application number: PCT/NO2011/000212
Authority: WO
Inventors: Lars Petter Endresen; Stian Selnes
Original assignee: Cisco Technology, Inc
Priority date: 2010-07-30
Filing date: 2011-07-22
Publication date: 2012-02-02
Also published as: NO332357B1; NO20101088A1

Abstract

The present invention is related to an implementation of a process for representing transform coefficients in compression/decompression of digital video systems in multi-purpose processors. It provides a method significantly reducing the required processor capacity compared to implementation according to prior art.

Description

IMPLEMENTATION OF RUN LEVEL CODING

Field of the invention

The invention is related to implementation of entropy coding/decoding of transform coefficient data of video compression systems in computer devices.

Background of the invention

Transmission of moving pictures in real-time is employed in several applications like e.g. video conferencing, net meetings, TV broadcasting and video telephony.

However, representing moving pictures requires bulk

information as digital video typically is described by representing each pixel in a picture with 8 bits (1 Byte) . Such uncompressed video data results in large bit volumes, and can not be transferred over conventional communication networks and transmission lines in real time due to limited bandwidth .

Thus, enabling real time video transmission requires a large extent of data compression. Data compression may, however, compromise with picture quality. Therefore, great efforts have been made to develop compression techniques allowing real time transmission of high quality video over bandwidth limited data connections.

In video compression systems, the main goal is to represent the video information with as little capacity as possible. Capacity is defined with bits, either as a constant value or as bits/time unit. In both cases, the main goal is to reduce the number of bits.

The most common video coding method is described in the MPEG* and H.26* standards. The video data undergo four main processes before transmission, namely prediction,

transformation, quantization and entropy coding.

The prediction process significantly reduces the amount of bits required for each picture in a video sequence to be transferred. It takes advantage of the similarity of parts of the sequence with other parts of the sequence. Since the predictor part is known to both encoder and decoder, only the difference has to be transferred. This difference typically requires much less capacity for its

representation. The prediction is mainly based on vectors representing movements. The prediction process is typically performed on square block sizes (e.g. 16x16 pixels). Note that in some cases, predictions of pixels based on the adjacent pixels in the same picture rather than pixels of preceding pictures are used. This is referred to as intra prediction, as opposed to inter prediction.

The residual represented as a block of data (e.g. 4x4 pixels) still contains internal correlation. A well-known method of taking advantage of this is to perform a two dimensional block transform. In H.263 an 8x8 Discrete

Cosine Transform (DCT) is used, whereas H.264 uses a 4x4 integer type transform. This transforms 4x4 pixels into 4x4 transform coefficients and they can usually be represented by fewer bits than the pixel representation. Transform of a 4x4 array of pixels with internal correlation will

probability result in a 4x4 block of transform coefficients with much fewer non-zero values than the original 4x4 pixel block .

Direct representation of the transform coefficients is still too costly for many applications. A quantization process is carried out for a further reduction of the data representation. Hence the transform coefficients undergo quantization. A simple version of quantisation is to divide parameter values by a number - resulting in a smaller number that may be represented by fewer bits. It should be mentioned that this quantization process has as a result that the reconstructed video sequence is somewhat different from the uncompressed sequence. This phenomenon is

referred to as "lossy coding". The outcome from the

quantisation part is referred to as quantized transform coefficients .

Entropy coding is a special form of lossless data

compression. It involves i.a. arranging the image

components in a "zigzag" order employing run-length

encoding (RLE) algorithm that groups similar frequencies together, inserting length coding zeros, and then using Huffman coding on what is left.

In H.264 encoding the DCT coefficients for a block are reordered to group together non-zero coefficients in an array, enabling efficient representation of the remaining zero-valued coefficients. The zigzag reordering path (scan order) is shown in Figure 1. The pattern of the order of the zig-zag scan is configured according to the probability of non-zero coefficients in each positions. Due to the characteristics of the preceding DCT, the probability of non-zero coefficients in a block decreases the downward right diagonal direction of a DCT block. When reordering the coefficients in a zigzag pattern as illustrated in figure 1, the non-zero coefficients will tend to

concentrate in the first positions of the array.

The output of the reordering process is i.e. an one

dimensional array that typically contains one or more clusters of nonzero coefficients near the start, followed by strings of zero coefficients. Due to the large number of zero values, the array is further represented as a series of (run, level) pairs where run indicate the number of zeros preceding a nonzero coefficient, and level indicates the magnitude of the nonzero coefficient. As an example, the input array 16, 0, 0, -3, 5, 6, 0, 0, 0, 0, -7, ... will have the following corresponding run-level values (0,16), (2,-3), (0,5), (0,6), (4,-7)...

When transforming the zigzag array to run-level values, it is computationally expensive to loop over all coefficients and check whether they are non-zero. As the picture resolution increases, this will require a considerable amount of processor capacity an even introduce too much delay, especially if the encoding process is implemented on general purpose shared processors, e.g. on personal computers .

WO-2010/077148 discloses an implementation of a process for representing transform coefficients in

compression/decompression of digital video systems in multi-purpose processors. Quantized transform coefficients representing a block of pixels in a video picture are packed to half the memory size, and then masked so as to generate an array of l's and 0's defining the positions of non-zero coefficients in the corresponding array of packed transform coefficients. This preparation avoids parsing through positions in the array of transform coefficients were zero values occur when generating run-level

representations of the coefficients, thus significantly reducing the required number of loops to execute. However, this method requires that a certain shuffle function reordering data entries in the memory to any order is available as a hard coded process in the processor. This function is p.t. only available in new generation Intel processors. Thus, an alternative method is needed for utilizing the effectiveness of this invention also without access to the shuffle function.

Summary of the invention

The present invention provides a method in a video coding or decoding process in a computer device for calculating run and level representations of respective quantized transform coefficients representing pixel values in a block of a video picture which is inserted row by row in a one dimensional array C, wherein the method includes the steps of packing each quantized transform coefficients in C in a value interval [Max, Min] by setting all quantized

transform coefficients greater than Max equal to Max, and all quantized transform coefficients less than Min equal to Min, masking C by generating an array M containing l's in positions corresponding to positions of C having non-zero values, and O's in positions corresponding to positions of C having zero values, setting a current raster position to zero, where a raster position refers to a raster scan order of the coefficients in the block, for each position

containing a 1 in M, deriving a current zigzag position from current raster position through a table mapping raster positions to zigzag positions, where a zigzag position refers to a zigzag scan order of the coefficients in the block, if current zigzag position is not zero and if current zigzag position minus last zigzag position minus 1 is less than zero, then discarding all stored runs and levels and calculating new runs and levels with an

alternative fall back method, if not, setting run equal to current zigzag position minus last zigzag position minus 1 and level equal to occurring value in the position of C corresponding to current raster position, storing run and level, setting last zigzag position equal to current zigzag position, and incrementing current raster position with the number of positions to the next 1 in M.

Brief description of the drawings

In order to make the invention more readily understandable; the discussion that follows will refer to the accompanying drawings .

Figure 1 illustrates the zigzag pattern that is used in prior art to order the transform coefficients before entropy coding, Figure 2 illustrates the raster scan pattern that is used in the present invention,

Figure 3 is a flow sheet illustrating how the run-level code is calculated according to MPEG4 and H.264,

Figure 4 is a flow chart illustrating a method of the type disclosed in WO-2010/077148 , and

Figure 5 is a flow chart illustrating an example

implementation according to the present invention.

Detailed description of the present invention

In the following description, reference will be made to some standard functions in the library of the general purpose programming language C++ that are directly mapped to compact and efficient low-level CPU instructions. C++ is widely used in the software industry. Some of its

application domains include systems software, device drivers, embedded software, high-performance server and client applications, and entertainment software as well as for implementing coding and decoding of real-time video in general purpose computers.

Figure 3 is a flow sheet illustrating how the run-level code according to MPEG4 and H.264 is calculated in a first implementation according to prior art. After quantizing the transform coefficients (Quant C) in a block, the quantized coefficients are reordered to a one dimensional array according to the earlier mentioned zigzag pattern. The process then enters into a loop for parsing the array for determining the run-level values. First it is checked whether the number of positions in the array is exceeded (I≥16) . If not, it is then checked whether current position in the array contains a zero. If so, both the Run variable and the position index (I) is incremented, and the process is proceeding to the start of the loop again. If not (current position contains a non-zero value) , the current Run variable and the value of the current position is stored as the Run-Level value. The Run variable is then reset, before both the Run variable and the position index

(I) are incremented and the process is proceeding to the start of the loop again. The process ends whenever the position index exceeds the maximum size of the array, which in the example illustrated in figure 3 is 16.

As can be seen from the example of a prior art

implementation illustrated in figure 3, it always has to run through the run-level encoding loop as many times as there are positions in the array (16 times in the

examples) . This becomes very inefficient as most

coefficients in C are zero, and it is computationally expensive to loop over all coefficients and check whether they are non-zero.

Figure 4 is a flow chart illustrating a second

implementation according to the above-mentioned method disclosed in WO-2010/077148. In this implementation, bit- masks and bit-scan instructions which makes it possible to efficiently jump over all the zero valued coefficients is used. It starts as in by quantizing the transform

coefficients in the block. In the example, there are 16 coefficients that are stored in the vector C.

The process then proceeds by packing and re-ordering all the quantized coefficients. This is done by the C++ instruction PACKUSWB, which transforms 16 signed words to unsigned integers and saturates. This means that if a coefficient is larger or smaller than the range of an unsigned byte, the coefficient is set to respectively max or min value of the range, which in this implementation is 255 and 0. In this way, the memory size of each coefficient is reduced from 2 to one byte. The nest step is to mask the quantized, packed and

reordered coefficients. This is done by e.g. applying the C++ functions PCMPGTB and PMOVMSKB. The PCMPGTB function fills a whole byte of l'es in the position of non-zero values, and leave the O'es unchanged in the position of zeros. The PMOVMSKB function creates a 16-bit mask from the most significant bits of 16 bytes. The result of these two functions when applied on the array of quantized, packed and reordered coefficients (C) is a 16-bits array (M) where the l'es indicates the corresponding positions of the nonzero values of C.

If M is nonzero, the C++ function BSF can be used to calculate the index of the first nonzero value of C. BSF returns the bit index of the least significant bit of an integer, i.e. in the case of M, the first position of a 1 starting from the right-hand side.

Hence, this index returned by BSF when applied on M is equal to the run and is used directly as lookup in the C array to determine the level. This is possible since C is already shuffled using PSHUFB instruction.

The Run-value as indicated by the BSF function is then stored, and after looking up the value localized at that position in the C is stored as the level value.

M is finally shifted to the right run+1 times to clear the index bit from M and prepare M for the next iteration in the loop. By doing this, the content of M corresponding to Run-Level values already calculated is removed from M, and the loop can be applied in the same way to calculate the reminding Run-Level values.

Since all the zeroes are being jumped over by effectively by using the BSF instruction, only nonzero coefficients runs are required to calculate all level and run values. The present invention is, i.a., based on the observation that in practice runs of zero coefficients in 4x4 raster scan order as illustrated in figure 2, most often is equivalent to runs in 4x4 zig-zag order as illustrated in figure 1. In the cases where a criterion for this

conversion is fulfilled a method similar to the second implementation according to prior art as described with the shuffle function omitted is used, and in the cases where a criterion for this conversion is not fulfilled a fallback method e.g. according to the first implementation of prior art described above is used. The effect is that in most cases zig-zag scanning is not performed, but only the more effective raster scan and a run-conversion for the non-zero coefficients .

The run-conversion is valid for all cases where the nonzero coefficients appear in the same order for both scans. In theory this is not the most probable case considering all possible permutations, but in practice it is.

Measurements done by the inventors have shown that non-zero coefficients appear in the same order for both scans in 98.5% of the coding time.

According to certain aspects of the present invention, if coefficients with raster scan index 0, 1, 4 and 5 are nonzero, then both scans will return the four coefficients in the same order, but with different runs. The raster scan runs can then be converted to the equivalent zigzag run. As can be seen from figure 1 and 2, the following mapping between the raster scan positions and the zigzag scan position applies: 0-0, 1-1, 2-5, 3-6, 4-2, 5-4, 6-7, 7-12, 8-3, 9-8, 10-11, 11-13, 12-9, 13-10, 14-14, 15-15. In the example above, when the runs for a raster scan indicates non-zeros for scan index 0, 1, 4 and 5, this mapping will imply 0, 1, 2 and 4 as the corresponding zig-zag scan index. Hence, when the order of non-zeros are the same for raster scan and zig-zag scan, the second and the most effective prior art implementation described above can be used, but there is no need to reorder the coefficients, and the shuffle function can be omitted.

Contrary, if for example the raster scan indexes 0, 2 and 4 are non-zero, the order of appearance will be different for the two scans, and since the shuffle function is not available a fallback method for example according to the first implementation according to prior art described above must be used.

The criterion to trigger the fallback is to test whether a calculated run for the raster scan order yields a negative result .

The run calculation is performed by measuring whether one or more run value in raster scan order is negative when using corresponding zigzag scan index in the calculation. I.e. the current raster scan index is derived and mapped to the corresponding zigzag index, using a normal zigzag table as defined in the H.264 recommendation. The calculation is described in detail in the following.

Let Xk denote the zigzag position of the nonzero

coefficients wherein k indexes the occurrence of nonzero coefficients. Further, let R_k denote the run value for the k' th occurrence of a nonzero value in a zigzag scan order, R' k the run value of the k' th occurrence of a nonzero value in raster scan order when using zigzag scan index in the calculation. N is the last k index so that N+l is the total number of nonzeros. A run value for a nonzero coefficient position x_k can be calculated as x_k - Xk-i ^~ 1· For the zigzag scan order, this is always a positive value since,

The total run R_T for N+l runs is, R₀ + Ri + R₂ + - + = x₀ + (xi - XQ-1) + (X2 - i-l) +

(2)

In general, however, the raster scan order of the nonzero coefficients may differ from the zigzag scan order. When using the zigzag positions x_k in the calculation for the corresponding raster positions, this will always lead to at least one negative raster scan run, since any permutation of the positions in (1) will violate (1) . Imagine for example three nonzero coefficients and that the zigzag order returns xo, Xi, x₂, while the raster scan returns XQ, x₂, xi · The total run for the two cases then becomes

Zig-zag: R_T = R₀ + Ri + R₂ = Xo + (xi ^_ Xo^~l) + (x₂ - xi^~l) = X2-2

(3)

Raster: R' _T = R'₀ + R'i + R' 2 = o + ( 2 " Q-1) + (xi - x₂~ 1) = Xi-2

(4)

The last run R' ₂ in (4) is negative since xi < x₂. Hence, the criterion for the fallback method is triggered.

Figure 5 is a flow chart illustrating an example

implementation according to the present invention. As earlier indicated, the procedure may bear some resemblance to the second implementation of prior art as described above, but without the need for reordering data, since the data is processed in the raster scan order which is the same order as the data already is stored in the memory. The differences are explained below. Thus, figure 5 and the following description should be read in conjunction with the remaining specification, in particular figure 4 and the corresponding description above.

In "Quant C" the transformed coefficients of a 4x4 block C is quantized. C is simply a one dimensional data array where the coefficients are inserted consecutively row by row starting from the upper row of the block. The

coefficients are thereby arranged in a raster scan order. "Mask M=C" creates a 16 bit mask M from the packet

coefficients C where a 1-bit indicates a non-zero

coefficient. In the decision box "M=0" the procedure is exited when there is no more non-zero coefficients left to process. "Scan M" scans the mask M in order to find the raster run R'_k. In "Derive Run" run R_k is derived from R' _k via raster index calculation and zigzag table lookup. In decision box "Run<0", it is decided whether to fall back to a conventional run-level coding according to the first implementation of prior art as described above and

illustrated in fig 5, ignoring the previously stored runs and levels, or to proceed by storing current level and run and then loop back to decision box "M=l". The decision to fall back is made if a negative run R_k is detected.

Claims

P a t e n t c l a i m s

1. A method in a video coding or decoding process performed in a computer device for calculating run and level representations of respective quantized

transform coefficients representing pixel values in a block of a video picture which is inserted row by row in a one dimensional array C,

c h a r a c t e r i z e d i n the following steps: packing each quantized transform coefficients in C in a value interval ( [Max, Min] ) by setting all quantized transform coefficients greater than Max equal to Max, and all quantized transform coefficients less than Min equal to Min, masking C by generating an array M containing l's in positions corresponding to positions of C having nonzero values, and 0's in positions corresponding to positions of C having zero values, setting a current raster position to zero, where a raster position refers to a raster scan order of the coefficients in the block, for each position containing a 1 in M, deriving a current zigzag position from current raster position through a table mapping raster positions to zigzag positions, where a zigzag position refers to a zigzag scan order of the coefficients in the block, if current zigzag position is not zero and if current zigzag position minus last zigzag

position minus 1 is less than zero, then discarding all stored runs and levels and calculating new runs and levels with an alternative fall back method, if not, setting run equal to current zigzag position minus last zigzag position minus 1 and level equal to occurring value in the position of C corresponding to current raster position, storing run and level, setting last zigzag position equal to current zigzag position, incrementing current raster position with the number of positions to the next 1 in M.

2. A method according to claim 1,

c h a r a c t e r i z e d i n that the table mapping raster positions to zigzag positions has the following entries: 0-0, 1-1, 2-5, 3-6, 4-2, 5-4, 6-7, 7-12, 8-3, 10-11, 11-13, 12-9, 13-10, 14-14, 15-15.

3. A method according to claim 1 or 2,

c h a r a c t e r i z e d i n that the step of masking further includes the following steps: creating an array C from C where positions

corresponding to positions of non-zero values in C are filled with l's, and positions corresponding to positions of zero values in C are filled with 0's, creating M from C by extracting the most significant bit from values in respective position of C and inserting the bits in corresponding positions in M.

4. A method according to claim 3,

c h a r a c t e r i z e d i n that the step of creating an array C is executed by a C++ function PCMPGTB, and the step of creating M from C is executed by a C++ function PMOVMSKB.

5. A method according to claim 4,

c h a r a c t e r i z e d i n that the step of

determining positions containing non-zero values in C is executed by a C++ function BSF.

6. A method according to one of the claims 1 - 5,

c h a r a c t e r i z e d i n that Max is 255 and Min is 0.

7. A method according to one of the claims 1 - 6,

c h a r a c t e r i z e d i n that the zigzag scan order follows a zigzag path of transform coefficient positions in the block starting in upper left corner heading towards lower right corner, and the raster scan order follows a raster path of transform coefficient positions in the block starting in upper left corner running through coefficient positions from left to right row by row ending in the lower right corner.