DE10219640A1 - Video coding and decoding method and computer program product - Google Patents

Abstract

For the efficient coding of several quality levels, it is proposed to provide a motion estimation unit (3) in an encoder (1) which acts on three coding branches (5, 6, 7). The prediction error data calculated in the coding branches (5, 6, 7) are coded differentially.

Description

The invention relates to a method for coding Video sequences and a method for decoding the Video sequences. The invention further relates to a Computer program product containing program codes for performing the methods.

Methods for coding video sequences are mainly used to effectively compress video sequences. Because video streams can only be packaged Data networks are transported when necessary Data rate below that in the packet-oriented data networks available bandwidth. They are therefore standardized Methods such as MPEG-1, MPEG-2 and H.26L, have been developed with which video sequences can be effectively let it compress. The standardized procedures work with motion-compensating hybrid coding, one Combination of lossless redundancy reduction and lossy irrelevance reduction.

The so-called compression contributes the most to compression motion compensating prediction. The motion compensating Prediction or prediction uses the similarity consecutive pictures by the currently encoded Predicts image from already encoded images. Because mostly only move certain parts of successive images when encoding the image currently to be encoded into rectangular Macroblocks disassembled. When coding for each of these Macro blocks from the images already transferred Macroblocks selected and their shift to the Macroblocks of the currently encoded image are calculated. The Movements of the macro blocks are determined by motion vectors described, which code using code tables to let.

Because the image currently to be encoded is not in every case the shifting of macroblocks of already encoded images can be constructed, for example when new in the picture coming objects, the prediction error or Prediction errors are transmitted. This prediction error results from the difference between the actual currently encoded picture and by moving the Macroblocks constructed from already encoded images Prediction.

Because the prediction errors of neighboring pixels in not or correlate areas that are difficult to predict to further reduce redundancy, transform the Prediction error carried out. Depending on the compression method comb different transformation processes Application. For example, the discrete ones are possible Wavelet transformation (DWT), the discrete cosine transformation (DCT) or the discrete integer transformation. Through this Will transform the macroblocks into Prediction error matrices transformed with a Variety of spectral coefficients are occupied. This Prediction error matrices together form the transformed ones Prediction error.

To the necessary for the transmission of the video sequence To further reduce data rate, the transformed Prediction error data quantized before further coding. To many spectral coefficients are equal to quantization Zero. The transformed and quantized Prediction error data can then be effectively encoded by entropy be compressed.

The encoded video data stream finally settles from the entropy-coded prediction error data and the entropy-encoded motion information of the macroblocks together. The video data stream can also contain information contain various coding parameters. By decoding this video data stream in a decoder can original video sequence can be reconstructed.

Currently, the packet-oriented data networks accessible video data streams are usually offered free of charge. However, since the operators are not entirely commercial The video sequences want to do without use only coded in a low quality level. To the To get users of the Internet accustomed to fee-based services efforts are underway, users alongside free Low quality video sequences against payment To offer video sequences of higher quality levels. Through the The quality of the video sequences should be staggered Entry threshold can be lowered for the user. To video sequences graded according to quality levels via the package-oriented Selling data networks are effective coding and Decoding procedures necessary that the video data streams as possible compress effectively.

The invention is based on this prior art based on the task of effective coding and coding methods Decode video sequences to create with that Have video sequences coded according to quality levels.

• - Schätzen einer aktuellen Bildinformation aus bereits übertragenen Bildbereichen und Bestimmen von Prädiktionsfehlerdaten verschiedener Qualitätsstufen in verschiedenen Codierzweigen;
• - Quantisieren und Codieren der Prädiktionsfehlerdaten einer ersten Qualitätsstufe;
• - Berechnen von Differenzdaten durch Subtraktion der Prädiktionsfehlerdaten der ersten Qualitätsstufe von Prädiktionsfehlerdaten einer zweiten Qualitätsstufe; und
• - Quantisieren und Codieren der Differenzdaten.

This object is achieved according to the invention by a method for coding video sequences with the following method steps:

• - Estimating current image information from already transmitted image areas and determining prediction error data of different quality levels in different coding branches;
• - quantizing and coding the prediction error data of a first quality level;
• Calculating difference data by subtracting the prediction error data of the first quality level from prediction error data of a second quality level; and
• - Quantize and encode the difference data.

• - Empfang von Videodatenströmen, die verschiedenen Qualitätsstufen zugeordnet sind;
• - Decodieren der Videodatenströme und Durchführen einer inversen Quantisierung zur Rekonstruktion von transformierten Prädiktionsfehlerdaten;
• - Addition der rekonstruierten transformierten Prädiktionsfehlerdaten;
• - Durchführen einer inversen Transformation zur Rekonstruktion von Prädiktionsfehlerdaten und Addition der Prädiktionsfehlerdaten auf ein übermitteltes Prädiktionsbild.

This object is further achieved according to the invention by a method for decoding video sequences with the following method steps:

• - Receiving video data streams that are assigned to different quality levels;
• Decoding the video data streams and performing an inverse quantization to reconstruct transformed prediction error data;
• - addition of the reconstructed transformed prediction error data;
• - Carrying out an inverse transformation for the reconstruction of prediction error data and addition of the prediction error data to a transmitted prediction picture.

In the method for coding a video sequence, the Video sequence first in separate coding branches in different quality levels. The generated however, transformed prediction error data is not fully transferred. Rather be exclusive Prediction error data of a first quality level complete transmitted. This first prediction error data is from the prediction error data of the next quality level subtracted and the differences calculated in this way are transferred. The prediction error data of a quality level becomes analog from the prediction error data, the next one in each case Subtracted quality level and transferred these differences. It It has been shown that these differences are often zero are, so the additional quality levels are little need additional bandwidth for transmission.

In a preferred embodiment, the Image information among those from the motion vectors formed movement information are to be understood on the Basis of coding parameters determined for one of the Coding branches are optimized. This movement information can then also be adopted by the other coding branches.

With this design, the computing effort for the formation can the movement information can be significantly reduced because the Movement information cannot be calculated multiple times have to.

In a preferred embodiment, the transformed prediction error data and the differences of the transformed prediction error data in each coding branch quantized by the prediction error data with a Multiplier multiplied in a shift register by one fixed number of digits and moved to the Pre-decimal places are rounded. The multipliers of the different quality levels are in one integer relationship to each other.

With this approach, those in quantization used grid to cover each other. The means that the grid of a quantization process with small grid size itself with a grid of one Quantization process with a large grid size covers.

The coordination of the quantizers prevents the decoder and the encoder drift apart.

Further details are the subject of the dependent Expectations.

An exemplary embodiment of the invention is described below the attached drawing explained. Show it:

Fig. 1 is a block diagram of a coding method with a plurality of quality levels;

Fig. 2 is a block diagram of a decoding method for a plurality of video data streams of different quality levels;

Figure 3 is a diagram illustrating various rasters that can be used for quantization;

Fig. 4 is a diagram showing the dependence of image noise is displayed on the data rate for different coding methods; and

Fig. 5 is a diagram showing the dependence of the image noise is shown on the data rate for further encoding.

An encoder 1 is shown in FIG. 1. The coding process that takes place within the encoder is illustrated using the individual components of the encoder 1 . It is clear to the person skilled in the art that the components of the encoder described here are synonymous with corresponding method steps of the coding method.

The encoder 1 has an input 2 , into which the video data of the video sequence are fed. A movement estimation unit 3 is connected downstream of the input 2 . The motion estimation unit 3 segments an image of the video sequence that is currently to be coded into rectangular macroblocks, which are usually 8 × 8 or 16 × 16 pixels in size. For each of these macroblocks, the motion estimation unit 3 looks for suitable macroblocks from already transmitted images and calculates their motion vectors. The motion vectors calculated by the motion estimation unit are encoded using an entropy encoder 4 and then transmitted.

In addition to the motion estimation unit 3 , the encoder 1 comprises three coding branches, each of which is assigned to a basic level 5 and a first and second advanced level 6 and 7 . The basic stage 5 has a motion compensator 8 , which calculates a prediction image on the basis of the motion vectors calculated by the motion estimation unit 3 and feeds it to a subtractor 9 . In the subtractor 9 , the prediction picture is subtracted from the picture currently to be coded. In this way, prediction error data are generated, which are then fed to a transformer 10 with a subsequent quantizer 11 . The transformer 10 carries out, for example, a discrete cosine transformation or discrete integer transformation. The transformed and quantized prediction error data are finally encoded by an entropy encoder 12 . In addition, the quantized prediction error data are fed to an inverse quantizer 13 , which is followed by an inverse transformer 14 . To the reconstructed by the inverse quantizer and inverse transformer 14 prediction error, the prediction image generated by the motion compensator 8 is added in an adder 15 °. The image stored in an image memory 16 therefore corresponds to the image generated by an assigned decoder.

The first development stage 6 also has a motion compensator 17 which calculates a prediction image from already transmitted images, which is subtracted in a subtractor 18 from the image currently to be coded. A subtractor 18 is followed by a transformer 19 which transforms the prediction error data calculated by the subtractor. With the transformed prediction error subtractor 20 is applied, the 6 which withdraws from these transformed prediction error data of the first construction stage from inverse quantizer 13 of the basic 5 reconstructed transformed prediction error. The difference data calculated in the subtractor arrive at a quantizer 21 , which quantizes them and forwards them to an entropy encoder 22 .

The transformed prediction error data calculated by the transformer 19 also arrive at a quantizer 23 with an inverse quantizer 24 and an inverse transformer 25 connected downstream. The prediction error data thus reconstructed are added in an adder 26 to the prediction image generated by the motion compensator 17 . The image of the first construction stage 6 thus reconstructed is stored in an image memory 27 .

The second development stage 7 also has a motion compensator 28 , which calculates a prediction image from already transmitted images, which is subtracted from a picture to be currently coded in a subtractor 29 . The prediction error data generated in the process are transformed in a transformer 30 and fed to a subtractor 31 . In the subtractor 31 , the transformed prediction error data reconstructed by the inverse quantizer 24 of the first development stage 6 is subtracted from the transformed prediction error data of the second construction stage 7 . The difference data calculated in this way is fed to a quantizer 32 with subsequent entropy encoder 33 . The transformed prediction error data supplied by the transformer 30 of the second construction stage 7 also arrive at a quantizer 34 with an inverse quantizer 35 and an inverse transformer 36 connected downstream. The prediction error data reconstructed by the inverse quantizer 35 and the inverse transformer 36 are finally added in an adder 37 to the prediction image generated by the motion compensator 28 and the result is stored in an image memory 38 .

In Fig. 2 is a block diagram of a decoder 39 is shown. It is clear to the person skilled in the art that the block diagram shown in FIG. 2 can also be understood as a block diagram of a decoding method.

The decoder 39 comprises an entropy decoder 40 for decoding the motion vectors supplied by the encoder 1 . Furthermore, the decoder 39 comprises three decoding branches, each of which is assigned to the basic stage 5 , the first advanced stage 6 and the second advanced stage 7 . The decoding branch, which is assigned to the basic stage 5 , each comprises an entropy decoder 41 and an inverse quantizer 42 . The decoding branch assigned to the first configuration stage 6 also has an entropy encoder 43 and an inverse quantizer 44 . The same applies to the decoding branch assigned to the second configuration stage 7 , which is equipped with an entropy decoder 45 with an inverse quantizer 46 connected downstream. The prediction error data reconstructed by the inverse quantizers 42 , 44 and 46 are combined by adders 47 and 48 and fed to an inverse transformer 49 . The prediction error data generated by the inverse transformer 49 are added in an adder 50 to the prediction image generated by a motion compensator 51 and stored in an image memory 52 . In addition, the images stored in the image memory 52 are output at an output 53 to a display device (not shown).

In the case of the encoder 1 presented here, only the transformed and quantized prediction error data of the basic level 5 are completely transmitted. In contrast, in the first advanced level 6 and the second advanced level 7 only the differences to the underlying basic level 5 or first advanced level 6 are transmitted. In order to enable efficient difference formation, the basic level 5 and the first and second advanced levels 6 and 7 must be synchronized.

In this context, synchronization is understood to mean the use of the same coding parameters in basic level 5 and in first and second advanced levels 6 and 7 . The encoder 1 can also have two or more than three coding branches.

• - die Entscheidung zwischen Intra- und Inter-Codierung einzelner Bilder oder Bildteile;
• - der Modus der Intra- oder Inter-Prädiktion;
• - die Blockgrößen für die Intra- oder Inter-Prädiktion;
• - die Nummer des Referenzbildes für die Inter-Prädiktion; und
• - die Bewegungsvektoren.

In general, the coding branch is first determined, the coding of which is optimized from the point of view of the rate distortion optimization. This coding branch is referred to below as the "master". For example, the following coding parameters are defined for this master:

• - the decision between intra and inter coding of individual pictures or parts of pictures;
• - the mode of intra or inter prediction;
• - the block sizes for intra or inter prediction;
• - the number of the reference image for the inter-prediction; and
• - the motion vectors.

The other coding branches, which are also called slaves, adapt to the master during encoding. This means that the slaves always use the coding parameters to code a specific part of the picture that the master also selected. As a result, the coding parameters need only be transmitted to the decoder 39 once. In addition, this type of synchronization only makes differential coding of the transformed prediction error data possible.

With the connection of the first advanced stage 6 and the second advanced stage 7 as well as further possibly existing advanced stages, the data rate required to transmit the entire video data stream supplied by the entropy encoders 12 , 22 and 33 also increases .

Since the data rate increases, which is required to transmit the entire video data stream supplied by the entropy encoders 12 , 22 and 33 , increases with the connection of the first advanced stage 6 and the second advanced stage 7 as well as any further existing advanced stages, the selection of a coding branch as master also determines the data rate for which the coding is optimized. If there are more than two coding branches, it is advisable to optimize a middle coding branch to which an average data rate is assigned. As a result, the remaining coding branches carry out a coding process that deviates only slightly from the optimum. As a result, coding losses due to the synchronization of the coding branches are also minimized.

• - die Entscheidung zwischen Intra-Codierung (Prädiktion aus benachbarten und bereits codierten Blöcken des gleichen Bildes) und Inter-Codierung (Prädiktion aus Blöcken vorheriger Bilder, maximal fünf),
• - die Interpolations- und Deblocking-Filter für die Prädiktion.

The encoder 1 shown in Fig. 1 is based on an encoder of the H.26L ITU standards. With encoder 1 , the coding branch of the first level forms the master. With encoder 1 , the following is determined for each macro block from 16 × 16 pixels on the master and synchronized for all other slaves:

• - the decision between intra-coding (prediction from neighboring and already coded blocks of the same picture) and inter-coding (prediction from blocks of previous pictures, maximum five),
• - the interpolation and deblocking filters for the prediction.

• - die Entscheidung zwischen Codierung auf Makroblockgröße (16 × 16 Bildpunkte oder auf einer Blockgröße von 4 × 4 Bildpunkten, und
• - die Entscheidung zwischen den verschiedenen Intra-Codierungs-Modi für die örtliche Prädiktion (horizontal, vertikal, diagonal).

For intra-coding additionally:

• - the decision between coding on macro block size (16 × 16 pixels or on a block size of 4 × 4 pixels, and
• - the decision between the different intra-coding modes for the local prediction (horizontal, vertical, diagonal).

• - das Referenzbild (1 bis 5), von dem aus prädiziert wird;
• - die Blockgröße (16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8, 4 × 4), auf die sich die Bewegungsvektoren beziehen; und
• - die Bewegungsvektoren für die einzelnen Blöcke.

For inter-coding additionally:

• - the reference image ( 1 to 5 ) from which the prediction is made;
• - the block size (16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8, 4 × 4) to which the motion vectors relate; and
• - the motion vectors for the individual blocks.

In order to prevent the encoder 1 and the decoder 39 from running apart, the parameters used in the quantizers 11 , 13 , 21 , 23 , 24 , 32 , 34 and 35 must be coordinated with one another. In accordance with the quantization method used in the H.26L standard, the spectral coefficients of the transformed prediction error matrices to be quantized are multiplied by a factor from a quantization table, shifted in a shift register by a fixed number of digits and thereby rounded off.

In order to prevent encoder 1 and decoder 39 from drifting apart, the quantization parameters must be adapted to one another. In particular, it is necessary that the raster 53 , 54 and 55 shown in FIG. 3 are used for the quantizers 11 , 21 , 23 , 32 and 34 and the inverse quantizers 13 , 24 and 35 , which are brought into register with one another. Accordingly, the grid 54 of the first construction stage 6 divides with its quantization levels each quantization stage of the grid 53 of the basic stage 1 into two halves. The grid 54 of the first level therefore has half the step size as the grid 53 of the basic level. Finally, the raster 55 of the second stage of development divides a quantization step of the raster 54 of the first stage of development into two quantization steps. The step size of the grid 55 is therefore only a quarter of the step size of the grid 53 .

In the case of an encoder 1 , which is based on the standard encoder H.26L, the quantization levels are defined by a quantization parameter in tables A (QP) and B (QP). In order for grids 53 , 54 and 55 to be aligned, the following must apply:

A (QP _i ) / A (QP _i-1 ) = B (QP _i-1 ) / B (QP _i ) = n _i

where n _{i is} a natural number and i is the index of the quality level. In practice, n _i = 2 was used for all successive quality levels. The gradation of the quantization steps is used for both the prediction error data relating to the luminance and the chrominance.

It should be pointed out that the formation of differences on the basis of the transformed prediction error data must take place in such a way that the encoder 1 and decoder 39 are prevented from drifting apart. Such a possibility is implemented in encoder 1 . The reason for this is the rounding carried out during the quantization if quantization is carried out in accordance with the H.26L standard.

It can be shown that only among these two Prerequisites, no drifting apart occurs.

Another important prerequisite for the feasibility of difference formation relates to the relationship between the transformed prediction error data, for example after the transformer 10 , and the dequantized prediction error data, for example before the inverse transformer 14 .

Wherein in Standardcodierer according to H.26L provided algorithm exists between a transformed spectral coefficients K of the transformed production error data and a dequantized spectral coefficients K 'is a ratio of S = K / K' = 2 ^{20/676. 2} A scaling factor s = 1 is required for differential coding of the prediction error data. Therefore, the relationship between quantization parameters A and B must be:

A (QP) .B (QP) = 2 ²⁰ .

Taking these conditions into account, the quantization table provided in the H.26L standard was modified. The table begins with a lowest value A (QP ₃₁ ) = 17, as in the prior art. The successive quantization parameters each increase by 12%, the step size being doubled after every six quantization parameters. A complete quantization table is shown below.

If, in the present example, the first configuration level 6 selects the quantization parameter 304 in the seventh line of the quantization table, the basic level 5 is operated with the quantization parameter 152 in the thirteenth line and the second configuration level 7 with the parameter 608 in the first line.

• - Sequence foreman;
• - Size: QCIF;
• - Bildfrequenz: 10 fps;
• - Länge: 10 s;
• - Anzahl der Referenzbilder: 2;
• - Inter-Codierung: 16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8, 4 × 4;
• - Suchraum für die Bewegungsvektoren: 32 × 32;
• - Auflösung der Bewegungsvektoren: 1/8 pel;
• - Rate-distortion: Optimierung: off;
• - Hadamard-Transformation: Verwendet;
• - Entropie-Codierung: CABAC

In Figs. 4 and 5 experimental results are shown. The following coding parameters were used for the simulation:

• - Sequence foreman;
• - Size: QCIF;
• - Frame rate: 10 fps;
• - length: 10 s;
• - Number of reference images: 2;
• - Inter-coding: 16 × 16, 16 × 8, 8 × 16, 8 × 8, 8 × 4, 4 × 8, 4 × 4;
• - Search space for the motion vectors: 32 × 32;
• - Resolution of the motion vectors: 1/8 pel;
• - Rate-distortion: optimization: off;
• - Hadamard transformation: used;
• - Entropy coding: CABAC

In FIG. 4, 56, a curve the image quality as a function of the data rate for a conventional encoder according to the standard H.26L. The image quality is signal-to-noise ratio quantified by means of the Y component (luminance). According to curve 56 , the image quality increases with increasing data rate.

Another curve 57 shown in FIG. 4 shows the relationship between image quality and data rate when using an encoder with a basic stage and a first advanced stage. A curve 58 represents the relationship between image quality and data rate for an encoder with two independent coding branches without differential coding. From FIG. 4 it can be seen that with the aid of differential coding, an image quality is achieved at 195 kilobits / s, which is without differential coding is only achieved at a data rate of 235 kilobits / s. Due to the optimization of the coding on the basic level, the image quality above 195 kilobit / s is somewhat worse than without differential coding. However, it is also entirely possible to optimize the differential coding for the first development stage.

FIG. 5 shows a further diagram, in which the curve 56 uses the example of the signal-to-noise ratio of the Y component to show the dependence of the image quality on the bit rate for a conventional coding method.

Curves 59 and 60 each show the dependency of the image quality on the data rate for coding methods in which the video sequence is coded in parallel in three quality levels. Curve 59 relates to a method in which the prediction error data are coded differentially. In contrast, in the coding method according to curve 60 , the quality levels are coded independently of one another. It can be seen from FIG. 5 that the differential coding according to curve 59 means that the second and third quality levels are achieved even at lower data rates. Since the differential coding according to curve 59 has been optimized with regard to the first configuration level, the image quality of the second configuration level is somewhat poorer than with independent coding in several quality levels.

The differential parallel coding allows Comparison to parallel non-differential coding a higher one even with a lower data rate Reach quality level. It also forms the parallel differential coding the advantage that the computational effort for the implementation of the coding process is significantly reduced is because in particular the search of the motion vectors only needs to be done once.

Claims (6)

1. Method for coding video sequences with the method steps: - Estimating ( 3 ) current image information from already transmitted image areas and determining transformed prediction error data of different quality levels in different coding branches ( 5 , 6 , 7 ); - quantizing ( 11 ) and coding ( 12 ) the prediction error data of a first quality level ( 5 ); - Calculating difference data by subtracting ( 20 ) the prediction error data of the first quality level ( 5 ) from prediction error data of a second quality level ( 6 ); and - Quantize ( 21 ) and encode ( 22 ) the difference data. 2. The method as claimed in claim 1, in which the current image information is estimated on the basis of image areas which are stored ( 27 ) in one of the coding branches ( 6 ), and in which the estimated image information is transmitted to the other coding branches ( 5 , 7 ). 3. The method according to claim 2, wherein the current image information is determined on the basis of method parameters that are optimized for each coding branch ( 6 ). 4. The method according to any one of claims 1 to 3, in which a prediction picture ( 8 , 17 , 28 ) is generated with the aid of the estimated picture information in each coding branch ( 5 , 6 , 7 ), and by its subtraction ( 9 , 18 , 29 ) prediction error data are generated from the current image, which are subsequently transformed ( 10 , 19 , 30 ) and quantized ( 11 , 23 , 34 ), the prediction error data being multiplied by multipliers in the course of the quantization ( 11 , 23 , 34 ) which are calculated by Coding branch ( 5 , 6 , 7 ) to coding branch ( 5 , 6 , 7 ) each have an integer ratio. 5. Method for decoding a video sequence with the method steps: - Receiving video data streams that are assigned to different quality levels ( 5 , 6 , 7 ); - decoding ( 41 , 43 , 45 ) the video data streams and performing an inverse quantization ( 42 , 44 , 46 ) for the reconstruction of transformed prediction error data; - addition of the reconstructed transformed prediction error data; - Carrying out an inverse transformation for the reconstruction of prediction error data and addition of the prediction error data to a transmitted prediction picture. 6. Computer program product that contains codes for performing one of the methods 1 to 5 .