WO2003026271A2

WO2003026271A2 - Method for coding and/or decoding video sequences and computer program product

Info

Publication number: WO2003026271A2
Application number: PCT/DE2002/003352
Authority: WO
Inventors: Peter Amon; Jürgen PANDEL; Klaus Illgner-Fehns
Original assignee: Siemens Aktiengesellschaft
Priority date: 2001-09-14
Filing date: 2002-09-10
Publication date: 2003-03-27
Also published as: WO2003026271A3

Abstract

The invention relates to a method for coding and/or decoding video sequences. According to the invention, an encoder (1) is provided with a motion estimator (3) that impinges on three coding branches (5, 6, 7). The predicted error data calculated in said coding branches (5, 6, 7) are encoded differentially, thereby allowing an efficient coding of a plurality of quality levels.

Description

description

Video coding and decoding method and computer program product

The invention relates to a method for coding video sequences and a method for decoding the video sequences. The invention further relates to a computer program product which contains program codes for executing the method.

Methods for coding video sequences are primarily used to effectively compress video sequences. This is because video streams can only be transported over packet-oriented data networks if the data rate required for this is below the bandwidth available in the packet-oriented data networks. Standardized methods, such as MPEG-1, MPEG-2 and H.26, have therefore been developed with which video sequences can be compressed effectively. The standardized methods work with the motion-compensating hybrid coding, one

Combination of lossless redundancy reduction and lossy irrelevance reduction.

The so-called motion-compensating prediction contributes the most to compression. The motion-compensating prediction or prediction takes advantage of the similarity of successive pictures by predicting the picture currently to be coded from pictures already coded. Since usually only certain parts of consecutive pictures move, the picture currently to be coded becomes rectangular when encoded

Macroblocks disassembled. When coding, suitable macroblocks are selected for each of these macroblocks from the images already transmitted and their shift to the macroblocks of the image currently to be coded is calculated. The shifts in the macroblocks are described by motion vectors that can be coded using code tables. Since the image currently to be encoded cannot always be constructed by shifting macroblocks of already encoded images, for example in the case of new objects entering the image, the prediction error or prediction error must also be transmitted. This prediction error results from the difference between the actual picture currently to be coded and the prediction picture constructed by shifting the macroblocks from pictures already coded.

Since the prediction errors of neighboring pixels correlate in areas that are difficult or impossible to predict, a transformation of the prediction errors is carried out to further reduce the redundancy. Depending on the compression process, different transformation processes are used. For example, the discrete avelet transformation (DWT), the discrete cosine transformation (DCT) or the discrete integer transformation are possible. This transformation turns the macroblocks into

Prediction error matrices transformed, which are populated with a variety of spectral coefficients. These prediction error matrices together form the transformed prediction error data.

In order to further reduce the data rate required for the transmission of the video sequence, the transformed prediction error data are quantized before the further coding. After quantization, many spectral coefficients are zero. The transformed and quantized prediction error data can then be effectively compressed by entropy coding.

The finally coded video data stream is finally composed of the entropy-coded prediction error data and the entropy-coded movement information of the macroblocks. The video data stream can also contain information contain various coding parameters. The original video sequence can be reconstructed by decoding this video data stream in a decoder.

At present, the video data streams that can be called up via the packet-oriented data networks are usually offered free of charge. However, since the operators do not want to completely forego commercial use, the video sequences are only encoded in a low quality level. In order to get the Internet users used to fee-based services, efforts are being made to offer video sequences of higher quality levels in addition to free video sequences of low quality for a fee. The entry threshold for the user is to be lowered by staggering the quality of the video sequences. In order to distribute video sequences staggered according to quality levels over the packet-oriented data networks, effective coding and decoding methods are necessary which compress the video data streams as effectively as possible.

Proceeding from this prior art, the object of the invention is to create effective methods for coding and decoding video sequences with which video sequences can be coded according to quality levels,

This object is achieved according to the invention by a method for coding video sequences with the following method steps:

- Estimating current image information from already transmitted image areas and determining prediction error data of different quality levels in different coding branches;

- quantizing and coding the prediction error data of a first quality level; - Calculating difference data by subtracting the prediction error data of the first quality level from prediction error data of a second quality level; and - Quantize and encode the difference data.

This object is further achieved according to the invention by a method for decoding video sequences with the following method steps:

- Receiving video data streams that are assigned to different quality levels;

Decoding the video data streams and performing an inverse quantization for the reconstruction of transformed prediction error data;

- addition of the reconstructed transformed prediction error data;

- Carrying out an inverse transformation for the reconstruction of prediction error data and adding the prediction error data to a transmitted prediction image.

In the method for coding a video sequence, the video sequence is first coded in separate coding branches in different quality levels. However, the transformed prediction error data generated in this way is not completely transmitted. Rather, exclusive prediction error data of a first quality level are completely transmitted. This first prediction error data is subtracted from the prediction error data of the next quality level and the differences calculated in this way are transmitted.

Similarly, the prediction error data of one quality level is subtracted from the prediction error data of the next quality level and these differences are transferred. It has been shown that these differences are often zero, so that the additional quality levels require little additional bandwidth for transmission.

In a preferred embodiment, the image information, which is to be understood as the motion information formed by the motion vectors, is determined on the basis of coding parameters which are relevant for one of the Coding branches are optimized. This movement information can then also be adopted by the other coding branches.

With this design, the computing effort for forming the movement information can be significantly reduced, since the movement information does not have to be calculated repeatedly.

In a preferred embodiment, the transformed prediction error data and the differences of the transformed prediction error data in each coding branch are quantized by multiplying the prediction error data by a multiplier, shifting them in a shift register by a fixed number of digits and rounding them to the parts before the decimal point. The multipliers of the different quality levels are in an integer relationship to each other.

With this procedure, the rasters used in the quantization are brought into register with one another. This means that the grid of a quantization process with a small grid size coincides with a grid of a quantization process with a large grid size.

The coordination of the quantizers prevents the decoder and the encoder from drifting apart.

Further details are the subject of the dependent claims.

An exemplary embodiment of the invention is explained below with reference to the accompanying drawing. Show it:

FIG. 1 shows a block diagram of a coding method with several quality levels; FIG. 2 shows a block diagram of a decoding method for a plurality of video data streams of different quality levels;

Figure 3 is a diagram in which different for the

Quantization usable rasters are illustrated;

FIG. 4 shows a diagram in which the dependence of the image noise on the data rate is shown for different coding methods; and

FIG. 5 shows a diagram in which the dependence of the image noise on the data rate is shown for further coding methods.

In Figure 1, an encoder 1 is shown. The coding process taking place within the encoder is illustrated using the individual components of the encoder 1. It is clear to the person skilled in the art that the components of the encoder described here are synonymous with corresponding method steps of the coding method.

The encoder 1 has an input 2, into which the video data of the video sequence are fed. A movement estimation unit 3 is connected downstream of the input 2. The motion estimation unit 3 segments an image of the video sequence that is currently to be encoded into rectangular macroblocks, which are usually 8 x 8 or 16 x 16 pixels. For each of these macro blocks, the motion estimation unit 3 looks for suitable macro blocks from already transmitted images and calculates their motion vectors. The motion vectors calculated by the motion estimation unit are encoded using an entropy encoder 4 and then transmitted.

In addition to the motion estimation unit 3, the encoder 1 comprises three coding branches, each of which comprises a basic level 5 and an most and second level 6 and 7 are assigned. The basic stage 5 has a motion compensator 8, which calculates a prediction image on the basis of the motion vectors calculated by the motion estimation unit 3 and feeds it to a subtractor 9. In the subtractor 9, the prediction picture is subtracted from the picture currently to be coded. In this way, prediction error data are generated, which are then fed to a transformer 10 with a subsequent quantizer 11. The transformer 10 carries out, for example, a discrete cosine transformation or discrete integer transformation. The transformed and quantized prediction error data are finally encoded by an entropy encoder 12. In addition, the quantized prediction error data are fed to an inverse quantizer 13, which is followed by an inverse transformer 14. In addition to the prediction error data reconstructed by the inverse quantizer and the inverse transformer 14, the prediction image generated by the motion compensator 8 is added. The image stored in an image memory 16 therefore corresponds to the image generated by an assigned decoder.

The first development stage 6 also has a motion compensator 17 which calculates a prediction image from images already transmitted, which is subtracted in a subtractor 18 from the image currently to be coded. A subtractor 18 is followed by a transformer 19 which transforms the prediction error data calculated by the subtractor. The transformed prediction error data is applied to a subtractor 20, which is transformed by it

Subtracts prediction error data of the first construction stage 6, which transforms the prediction error data reconstructed by the inverse quantizer 13 of the basic stage 5. The difference data calculated in the subtractor reach a quantizer 21, which quantizes them and forwards them to an entropy encoder 22. The transformed prediction error data calculated by the transformer 19 also reach a quantizer 23 with a subsequent inverse quantizer 24 and inverse transformer 25. The prediction error data thus reconstructed are added in an adder 26 to the prediction picture generated by the motion compensator 17. The image of the first construction stage 6 thus reconstructed is stored in an image memory 27.

The second development stage 7 also has a motion compensator 28, which calculates a prediction image from already transmitted images, which is subtracted from a picture currently to be coded in a subtractor 29. The prediction error data generated in the process are transformed in a transformer 30 and fed to a subtractor 31. In the subtractor 31, the transformed prediction error data reconstructed by the inverse quantizer 24 of the first development stage 6 are subtracted from the transformed prediction error data of the second construction stage 7. The difference data calculated in this way is fed to a quantizer 32 with subsequent entropy encoder 33. The transformed prediction error data supplied by the transformer 30 of the second construction stage 7 also reach a quantizer 34 with a subsequent inverse quantizer 35 and inverse transformer 36. The prediction error data reconstructed by the inverse quantizer 35 and the inverse transformer 36 are finally added to this in an adder 37 Prediction image generated by the motion compensator 28 is added and the result is stored in an image memory 38.

The block diagram of a decoder 39 is shown in FIG. It is clear to the person skilled in the art that the block diagram shown in FIG. 2 can also be understood as a block diagram of a decoding method.

The decoder 39 comprises an entropy decoder 40 for decoding the motion vectors supplied by the encoder 1. Further the decoder 39 comprises three decoding branches, each of which is assigned to the basic stage 5, the first advanced stage 6 and the second advanced stage 7. The decoding branch, which is assigned to the basic stage 5, in each case comprises an entropy decoder 41 and an inverse quantizer 42. That of the first, too

Decoder branch assigned to configuration stage 6 has an entropy encoder 43 and an inverse quantizer 44. The same applies to the decoding branch assigned to the second configuration stage 7, which is equipped with an entropy decoder 45 with an inverse quantizer 46 connected downstream. The prediction error data reconstructed by the inverse quantizers 42, 44 and 46 are combined by adders 47 and 48 and fed to an inverse transformer 49. The prediction error data generated by the inverse transformer 49 are added in an adder 50 to the prediction image generated by a motion compensator 51 and stored in an image memory 52. In addition, the images stored in the image memory 52 are output at an output 53 to a display device (not shown).

In the case of the encoder 1 presented here, only the transformed and quantized prediction error data of the basic level 5 are completely transmitted. In contrast, in the first advanced level 6 and the second advanced level 7 only the differences to the underlying basic level 5 or first advanced level 6 are transmitted. In order to enable efficient difference formation, the basic level 5 as well as the first and second advanced levels 6 and 7 must be synchronized.

In this context, synchronization is understood to mean the use of the same coding parameters in basic level 5 and in first and second advanced levels 6 and 7. The encoder 1 can also have two or more than three coding branches. In general, the coding branch is first determined, the coding of which is optimized from the point of view of rate distortion optimization. This coding branch is referred to below as the "master". For example, the following coding parameters are defined for this master:

- the decision between intra and inter coding of individual pictures or parts of pictures;

- the mode of intra or inter prediction;

- the block sizes for intra or inter prediction; - the number of the reference image for the inter-prediction; and

- the motion vectors.

The other coding branches, which are also called slaves, adapt to the master during encoding. This means that the slaves always use the coding parameters to code a specific part of the picture that the master also selected. As a result, the coding parameters need only be transmitted to the decoder 39 once. In addition, this type of synchronization only makes differential coding of the transformed prediction error data possible.

With the connection of the first advanced stage 6 and the second advanced stage 7 as well as further possibly existing advanced stages, the data rate required to transmit the entire video data stream supplied by the entropy encoders 12, 22 and 33 also increases.

Since the connection of the first construction stage 6 and the second construction stage 7 as well as further possibly existing construction stages increases the data rate which is necessary for transmitting the entire video data stream supplied by the entropy encoders 12, 22 and 33, the selection of a coding branch becomes the master also specified for which data rate the coding is optimized. If there are more than two coding branches, it is advisable to optimize a middle coding branch to which an average data rate is assigned. As a result, the remaining coding branches use a coding method that deviates only slightly from the optimum. As a result, coding losses due to the synchronization of the coding branches are also minimized.

The encoder 1 shown in FIG. 1 is based on an encoder of the ITU standard H.26L. With encoder 1, the coding branch of the first level forms the master. With encoder 1, the following is determined for each macro block from 16 x 16 pixels on the master and synchronized for all other slaves:

- the decision between intra-coding (prediction from adjacent and already coded blocks of the same picture) and inter-coding (prediction from blocks of previous pictures, maximum five), - the interpolation and deblocking filters for the prediction,

For intra-coding additionally:

- the decision between coding on macro block size (16 x 16 pixels or on a block size of 4 x 4 pixels, and

- the decision between the different intra-coding modes for the local prediction (horizontal, vertical, diagonal),

For inter-coding additionally:

- the reference image (1 to 5) from which the prediction is made;

- the block size (16 x 16, 16 x 8, 8 x 16, 8 x 8, 8 x 4, 4 x 8, 4 x 4) to which the motion vectors refer; and - the motion vectors for the individual blocks.

In order to prevent the encoder 1 and the decoder 39 from running apart, the parameters used in the quantizers 11, 13, 21, 23, 24, 32, 34 and 35 must be coordinated with one another. According to the quantization method used in the H.26 standard, the spectral coefficients to be quantized are the transformed ones Prediction error matrices multiplied by a factor from a quantization table, shifted by a fixed number of digits in a shift register and thus rounded off.

In order to prevent encoder 1 and decoder 39 from drifting apart, the quantization parameters must be adapted to one another. In particular, it is necessary that for the quantizers 11, 21, 23, 32 and 34 and the inverse quantizers 13, 24 and 35 the grids 53, 54 and 55 shown in FIG. 3 are used, which are brought into register with one another. Accordingly, the grid 54 of the first construction stage 6 divides with its quantization stages each quantization stage of the grid 53 of the basic stage 1 into two halves. The grid 54 of the first level therefore has half the step size as the grid 53 of the basic level. Finally, the grid 55 of the second construction stage divides a quantization step of the grid 54 of the first construction stage into two quantization steps. The step size of the grid 55 is therefore only a quarter of the step size of the grid 53.

In the case of an encoder 1, which is based on the standard encoder H.26L, the quantization levels are defined by a quantization parameter in tables A (QP) and B (QP). In order for grids 53, 54 and 55 to be aligned, the following must apply:

A (QPi) / A (QPi_ι) = B (QPi-ι) / B (QPi) = m

where ni is a natural number and i is the index of the quality level. In practice, r = 2 was used for all successive quality levels. The gradation of the quantization steps is used for the prediction error data relating to the luance as well as the chrominance. It should be pointed out that the formation of differences on the basis of the transformed prediction error data must take place in such a way that the encoder 1 and decoder 39 are prevented from drifting apart. Such a possibility is implemented in encoder 1. The reason for this is at

Rounding performed when quantizing according to the H.26L standard.

It can be shown that only under these two conditions does not drift apart.

Another important prerequisite for the feasibility of the difference formation relates to the relationship between the transformed prediction error data, for example after the transformer 10, and the dequantized ones

Prediction error data, for example in front of the inverse transformer 14.

Wherein in Standardcodierer according to H.26L provided algorithm exists between a transformed spectral coefficients K of the transformed production error data and a dequantized spectral coefficients K 'is a ratio of S = K / K' = 2 ^20/676. ² A scaling factor s = 1 is required for differential coding of the prediction error data. Therefore, the relationship between quantization parameters A and B must be:

Taking these conditions into account, the quantization table provided in the H.26L standard was modified. The table begins with a lowest value A (QP _3i ) = 17, as in the prior art. The successive quantization parameters each increase by 12%, the step size doubling after every six quantization parameters. pelt is. A complete quantization table is shown below.

const int JQ [32] [2]

{

{608, 1725},

{544, 1928}

{496, 2114}

{432, 2427}

{384, 2731}

{352, 2979}

10 {304, 3450}

{272, 3856}

{248, 4228}

{216, 4854}

{192, 5462}

15 {176, 5958}

{152,6900}

{136, 7712}

{124, 8456}

{108, 9708}

20 {96, 10924}

{88, 11916}

{76, 13800}

{68, 15424}

{62, 16912}

25 {54, 19416}

{48, 21848}

{44, 23832}

{38, 27600}

{34, 30848},

30 {31, 33824},

{27, 38832},

{24, 43696},

{22, 47664},

{19, 55200},

35 {17, 61696}, ^} ; If, in the present example, the first configuration level 6 selects the quantization parameter 304 in the seventh line of the quantization table, the basic level 5 is operated with the quantization parameter 152 in the thirteenth line and the second configuration level 7 with the parameter 608 in the first line.

Experimental results are shown in FIGS. 4 and 5. The following coding parameters were used for the simulation:

- Sequence foreman;

- Size: QCIF;

- Frame rate: 10 fps;

- length: 10 s; - Number of reference images: 2;

- Inter-coding: 16 x 16, 16 x 8, 8 x 16, 8 x 8, 8 x 4, 4 x 8, 4 x 4;

- Search space for the motion vectors: 32 x 32;

- Resolution of the motion vectors: 1/8 pel; - Rate distortion: Optimization: off;

- Hadamard transformation: used;

- Entropy coding: CABAC

In FIG. 4, a curve 56 represents the image quality as a function of the data rate for a conventional encoder according to the H.26L standard. The image quality is quantified using the signal-to-noise ratio of the Y component (luminance). According to curve 56, the image quality increases with an increasing data rate.

Another curve 57 shown in FIG. 4 shows the relationship between image quality and data rate when using an encoder with a basic level and a first advanced level. A curve 58 represents the relationship between image quality and data rate for an encoder with two independent coding branches without differential coding. It can be seen from FIG. 4 that with the aid of the differential code image quality is already achieved at 195 kilobit / s, which can only be achieved at a data rate of 235 kilobit / s without differential coding. Due to the optimization of the coding on the basic level, the image quality above 195 kilobit / s is somewhat worse than without differential coding. However, it is also entirely possible to optimize the differential coding for the first stage of development.

Figure 5 shows a further diagram in which the curve 56 on

An example of the signal-to-noise ratio of the Y component represents the dependence of the image quality on the bit rate for a conventional coding method.

Curves 59 and 60 each show the dependency of the image quality on the data rate for coding methods in which the video sequence is coded in parallel in three quality levels. Curve 59 relates to a method in which the prediction error data are coded differentially. In contrast, with the coding method according to curve 60, the quality levels are coded independently of one another. It can be seen from FIG. 5 that the second and third quality levels are achieved by the differential coding according to curve 59 even at lower data rates. Since the differential coding according to curve 59 has been optimized with regard to the first configuration level, the image quality of the second configuration level is somewhat poorer than with independent coding in several quality levels.

Due to the differential parallel coding, a higher quality level can be achieved with a lower data rate compared to the parallel non-differential coding. In addition, the parallel differential coding has the advantage that the computational effort for performing the coding method is significantly reduced, since in particular the search for the motion vectors only has to be carried out once.

Claims

claims

1. Method for coding video sequences with the method steps: - Estimating (3) current picture information from picture areas already transmitted and determining transformed prediction error data of different quality levels in different coding branches (5, 6, 7);

- quantizing (11) and coding (12) the prediction error data of a first quality level (5);

- Calculating difference data by subtracting (20) the prediction error data of the first quality level (5) from prediction error data of a second quality level (6); and

- Quantize (21) and encode (22) the difference data.

2. The method as claimed in claim 1, in which the current image information is estimated on the basis of image areas which are stored (27) in one of the coding branches (6), and in which the estimated image information is transmitted to the other coding branches (5, 7).

3. The method according to claim 2, wherein the current image information is determined on the basis of method parameters that are optimized for each coding branch (6).

4. The method according to any one of claims 1 to 3, in which a prediction picture (8, 17, 28) is generated with the aid of the estimated picture information in each coding branch (5, 6, 7), and by subtracting it (9, 18, 29 ) prediction error data are generated from the current image, which are subsequently transformed (10, 19, 30) and quantized (11, 23, 34), the prediction error data being multiplied by multipliers in the course of the quantization (11, 23, 34) that are calculated by Coding branch (5, 6, 7) to coding branch (5, 6, 7) each have an integer ratio.

5. Method for decoding a video sequence with the method steps:

- Receiving video data streams that are assigned to different quality levels (5, 6, 7);

Decoding (41, 43, 45) the video data streams and performing an inverse quantization (42, 44, 46) for the reconstruction of transformed prediction error data;

- addition of the reconstructed transformed prediction error data;

- Performing an inverse transformation for the reconstruction of prediction error data and adding the prediction error data to a transmitted prediction picture.

6. Computer program product which contains codes for executing one of the methods 1 to 5.