DE4343450A1 - Coder for segment-wise coding of input signal, esp. for video telephone - Google Patents

Abstract

The coder includes a controllable quantiser (1c) and a buffer memory (6) in which quantized and VLC-coded data of the input signal (b) are intermediately stored. Components (2, 3, 4, 7) are provided which evaluate a one-off adjustment of the quantiser for a signal segment to be coded. The evaluation is performed such that the number of bits which are written into the buffer memory on the basis of the coding of the signal segment coincides as closely as possible with a predefined number.

Description

The invention relates to a coder for segmental co dation of an input signal, with a controllable Quantizer and a buffer memory in which quantized and VLC-coded data of the input signal be cached.

Such encoders are used, for example, in videophones applies the according to the CCITT standard H.261 are built (compare for example: Draft Revision of Recommendation H.261: Video Codec for Audiovisual Ser vices at px64 kbit / s. Signal Processing: Image Communi cation 2 (1990) 221-239. Elsevier). The specified lite will be cited below with (I).

Another application is the Coding of still images, as far as the coding corresponds according to the CCITT recommendation T.81. The A still image is then encoded in several Steps with increasing image quality, being in each Step the previously encoded and decoded still image is used as a prediction picture. Recommendation T.81 should be cited in the following with (II) (cf. International Standard DIS 1011918-1. CCITT Recommendation T.81. Digital Compression and Coding of Continuous-Tone Still images. Part I: Requirements and Guidelines (JPEG)).

Finally, encoders with the features listed above are also used in audio coding, as the following document shows, for example: Brandenburg, K. et al .:
Aspec: Adaptive spectral entropy coding of high quality music signals. An Audio Engineering Society preprint. Presented at the 90th Convention 1991 February 19-22 Pa ris. Preprint 3011 (A-4). In the following it should be cited with (III).

However, the explanation of the present invention is intended mainly using a video encoder that works with the H.261 recommendation is compatible. The transfer to the other use cases are then obvious from this particular explanation.

The video data to be encoded is a coder according to (I) fed imagewise. The signal segment indicated at the beginning elements are generally (see below, however) a prediction error belonging to a video image in CIF format or in QCIF format (see (I)).

According to (I), video data in CIF format are again broken down hierarchically into further partial data, namely into twelve block groups (GOB's). Each block group is divided into thirty-three macro blocks and each macro block into six blocks. A block either consists of the 64 luminance values (Y values) of a square image section of 8 × 8 pixels or 64 values of one of the two color difference components (C _B or C _R values). These chrominance values result from a Image section of 16 × 16 pixels, which is horizontally and vertically subsampled in terms of chrominance in a ratio of 1: 2. A macro block contains both luminance and chrominaz information in an image section of 16 × 16 pixels.

A encoder according to (I) works in two main operating modes, namely in INTER mode or in INTRA mode. In INTER mode the differences in the pixel data between one Input picture (current picture) and its decoded and possibly motion-compensated previous picture (Pre  diction picture) coded. In INTRA mode, instead of the pre diction error that input picture without reference to the Coded previous picture.

The coding is done in blocks. The block data or the Differences in the block data become two-dimensional subjected to discrete cosine transformation. The one there certain transformation coefficients become taxable erable quantizers supplied, after their quantization VCL-encoded and written into a buffer memory. Furthermore, the in a feedback branch of the encoder encoded picture (block by block) decoded again in one Image memory stored and as a prediction image for the next input image used.

The quantizer of the encoder is controllable in that the quantization characteristic can be set with it can. Different quantization curves under differ by the length of their quantization inter valle. The is within a quantization curve Length of the quantization intervals (also step size usually called the same. You get a rough quan for characteristic curves with large increments and one fine quantization for characteristic curves with a small step wide. If it is roughly quantized, it becomes a large quan Acceptance errors are generally accepted less VLC data written into the buffer memory. If you quantize finely, the number of Bits in the form of VLC code words in the buffer memory be cached before transmission.

The buffer tank is emptied with a more constant time Bit rate while filling the memory with high time-dependent bit rate can take place. Many or few Bits written into the buffer memory per second  depends on both the setting of the quantizer as well as the size of the prediction error - i.e. of the image data - from.

Which quantization for which data of a video image characteristic is not to be used in (I) ben. The only requirement is that within one Macroblockes to use only one quantization curve is the quantization characteristic of Macro at most block to Macroblock may be changed and that for the DC coefficients in INTRA mode always the same quanti characteristic curve should be used. The possible Quantization step sizes are the integers 2, 4, 6,. . . to 62.

Control of the quantizer is usually dependent buffer level, such as the EP 036 3 682 or EP 028 4 168 shows. Sense of a sol Chen control is an overflow or an idle of the Avoid buffer storage. Since now the capacity storage is hardly a technical problem anymore, so the risk of overflow and the associated information loss of mation no longer exists because it is sufficiently large Memory can be used, it is still wu it is worth it that the fill level of the buffer storage does not exceed the maximum fill level. The reason for that is that - quite pronounced with low transmission bit rates - the time interval between coding a video image on the transmission side and decoding and building the same video image on the emp start side becomes unacceptably large. This time interval For example, in a dialogue via videophones, almost be imperceptible to the participants. Time is acceptable that are in the range of a tenth of a second. Therefore remains with almost unlimited memory capa the problem, the level of the buffer storage  for the reasons given do not have a maximum fill stood to let go.

The control of the quantizer solely by the level of the buffer tank (in the following fill level control usually leads to macroblocks leading to the belong to the same picture or to the same picture difference, with different quantization characteristics be, because the filling level of the buffer storage during the Encoding an image is subject to strong fluctuations can be.

The change from a quantization curve to one another while encoding an image is at the Level control in the sense arbitrary than this Change regardless of the information content of the still to be encoded macro blocks must be made. In this way, important parts of the picture can be roughly quan be tized and others - less important - in turn unnecessarily fine. In other words: with level control the importance of the image content for the viewer disregarded. The level control leads therefore to a subjectively as well as objectively unsatisfied image quality.

The invention has for its object a controller the quantizer so that the Qua subjectivity and objectivity of the decoded images considerably improved.

This task is done with a coder mentioned at the beginning solved by means with which a one in operation times setting the quantizer for one to be encoded Signal segment is estimated such that the number of Bits which, due to the coding of this signal segment in  the buffer memory are written with a before given number matches as closely as possible.

In the case of image coding, this means that for the entire image (signal segment) - i.e. for all macroblocks of an image - just a single shot of the quantisie res is used, provided the estimated setting is under the settings provided in (I) occurs.

This measure uses knowledge makes use of a single quatization kennlienie for all blocks of a picture (in the steady state - see below) subjectively and objectively leads to almost the best possible image quality. Also the so-called "blocking" is reduced. There is one Differentiation between important and unimportant parts of an image is not provided, but is clearly the Case excluded that important parts are rough and unimportant can be finely quantized.

Based on an embodiment, the invention will now are explained in more detail in connection with the figures.

Show it:

Fig. 1 is a block diagram of a encoder with features of the invention and

Fig. 2 is a PSNR diagram in which the difference between a coder with buffer control and a coder according to the invention is shown.

The basic circuit diagram of FIG. 1 shows the most important functional units of a video encoder which operates according to the regulations set out in (I) and which also contains additives according to the invention. Clock lines are not entered or are only hinted at. Likewise, a number of control lines and address lines are omitted or also only hinted at in order not to obscure the figure. However, the person skilled in the art can easily add to them mentally in the figure and add them when the circuit is specified.

With the reference numeral 1 , a source encoder is provided, ie those parts of a video encoder according to (I), the essential components of a transformation unit 1 B, a controllable quantizer 1 C and a feedback loop with a decoder 1 D contains. The decoder 1 D provides prediction images for input images which are fed to the video encoder 1 on a line a2. The input images on line a2 are a selection of the images arriving on line a1 with an image repetition rate of approximately 30 Hz in CIF format. In the game, a switch 8 is controlled so that only every second picture from line a1 to line a2.

A controllable functional unit 1 A gives either the input images from line a2 or the prediction error (also in CIF format) on line b. The prediction error consists of the differences between all data of an input picture and the data of the prediction picture. The signal on line b is the input signal to be coded by the source coder 1 according to the regulations laid down in (I) and mentioned above. After the discrete cosine transformation, it arrives through the unit 1 B via a line c to the controllable quantizer 1 C and from there via a controllable switch 1 E and a section of a line d to a variable length coder 5 , also entropy coder called. The code words of the entro pie coder 5 are written into a buffer memory 6 and read out from this at a bit rate of 64 × p kbit / s and, if appropriate, line-coded transmitted to a receiver. Unless expressly stated otherwise, the case p = 1 is dealt with in the exemplary embodiment.

All controllable functional units of the source encoder 1 are controlled by a control unit 3 , which is indicated by a connection a5 of the control unit 3 to the source encoder 1 . The invention is mainly about the control of the quantizer 1 C such that the level F of the buffer memory 6 moves during operation within prescribed limits, without - contrary to the prior art - the setting of the quantizer 1 C during coding an image is changed. The level F is transmitted via a line a6 to the control unit 3 and evaluated by it in a manner described below.

In addition, there is an exchange of information between the control unit 3 and a number of special modules which essentially serve for the pre-analysis of an image to be coded and which do not have to be part of a coder according to (I). These modules include the switch 8 already mentioned, an image memory 7 and an activity calculator 2 ; they are connected to the control unit 3 via a line system a3. Another special module is a bit counter 4 , which counts the bits written into the buffer memory 6 per image and forwards its counter reading to the control unit 3 via a line a4. The number of bits written into the buffer memory per picture is to be called B in the following picture volume. The image volume B and the output data of the activity computer 2 are required by the control unit 3 to carry out their control functions.

From the activity computer 2 , the control unit 3 receives a quantitative statement about the activity of the signal on line b, called image activity a for short. In the present context, this should be understood to mean any function of the amounts of all image data of an image or difference image. For example, if x _{k is} a quantitative statement (chrominance or luminance value) about the kth pixel of an image, then it would be

a possible quantitative statement about the image stocks vity σ.

In the present example, the expression for σ is

used, ie the square of the "absolute standard" in the Mathematics known expression because it turns out to be special has proven useful. The sum in (2) runs through the Luminance and chrominance values of all pixels in a bil des or difference image in CIF format. Overall, that is 152,064 values per image. A limitation to the Lumi nanz values alone also provides useful values for the image activity.

To understand the following explanations easier, should first a list with the below  symbols used and their meaning are explained. It should be noted in advance that i is the symbol for is the ordinal number with which the Bil which are counted. Sizes - here also coding parameters called - which is required for coding the i-th image will depend or depend on the data of this picture the atomic number i as an index.

In particular:
σ₁ the image activity of the i-th image to be encoded, which is determined by the activity calculator 2 according to formula ( 2 ) before it is encoded.
s _i the quantization step size calculated before the coding of the i-th image, with which this image should be quantized, but usually cannot be quantized because its value does not equal any of the values between 2 and 62 permitted by (I).
sl _i , sh _i two quantization increments permissible according to (I), with each of which an associated part of the i-th image is quantized and which result from the calculated value s _i according to the equations

sl _i = 2 _* INT (s _i / 2);
sh _i = sl _i + 2

result, where INT (x) means the largest integer that is less than x. If values result for these quantities that lie outside the interval provided in (I), then the permissible values 4 or 62 are assigned to them.
B _i image volume of the i-th image, which indicates the number of bits which is written into the buffer memory 6 as a result of the encoding of this image and which corresponds to the state of the bit counter 4 after the encoding of this image.
_i Estimated image volume for the i-th image.
B₀ predefined image volume (6400 bit in the example).
F _i level of the "buffer memory 6 immediately before the coding of the i-th picture
BS _i target image volume, that image volume that the i-th image should have; it is calculated by the controller 3 before coding this picture.
FS _i target fill level, the fill level of the buffer store 6 which it is to have after the coding of the i-th picture and which is calculated by the controller 3 before the coding of the i-th picture.
2 _* S _i slope of the estimation line, which is determined adaptively before the coding of the i-th image.

The image volume B _i cannot be determined from the buffer fill level before and after the i-th image. Because during the coding of a picture, the buffer 6 can run empty because no bits are generated for certain picture sections. In order to practically prevent idling, stuffing bits are added which have nothing to do with the image volume B _i and which have to be removed on the receiver side. Because of these stuff bits, the image volume and the buffer fill level must be determined separately.

The task which the control circuit 3 has to solve can be summarized in the following question: How must the step size s _{i of} the quantizer 1 C be chosen for coding the i-th picture so that the picture volume Bi of the picture still to be coded is as good as possible assumes a predetermined target image volume BS _i ? It is assumed that the picture activity σ _i according to formula (2) of the i-th picture is known before it is encoded. In terms of circuitry, this requirement is met by the interaction of the two controllable switches 8 and 1 E with the image memory 7 . If the switch 8 is in its conductive switching position (connection between the lines a1 and a2 established), the incoming image - be it the i-th image - is read into the memory 7 , while at the same time the difference data between this image and its already encoded and again decoded previous picture - the picture with the number (i-1) - be given on line b. In this phase, the switch 1 E is in its blocking position, so that the coded values are neither supplied to the entropy coder 5 nor to the decoder 1 D and overwrite the data of the (i-1) th picture there. The switch 1 E has more symbolic meaning, since its effect can also be achieved by interrupting the cycle or by interrupting the address supply.

During the blocking phase of the switch 1 E, the activity calculator 2 determines the size σ _i and then makes it available to the control unit 3 . Then the switch 8 is brought by the control unit 3 into its blocking position and the image stored in the memory 7 is read out again. The image of the CIF sequence present on line a1 in the blocking phase of switch 8 is ignored. When the switch 1 E is switched on, the i-th image stored in the memory 7 is coded as usual, but with a setting of the quantizer 1 C, which was previously calculated by the control unit 3 - also with the image activity σ _i . Then, the operations with conductive switch 8 and the off state switch 1 repeat E. exception to this rule to be locked, the switch E 1 as long as the filling level of the buffer memory 6 is higher than a maximum permissible level. The blocking times are chosen so that only whole pictures are excluded from the VLC coding by the entropy coder 5 .

In order to make the answer to the above question understandable, the following observation should be recalled. If the image volume B _{i of} all images of an image sequence is plotted against the quantization step size s _{i in} relation to the signal intensity σ _i , the

B _{i *} (σ _i / s _i ) for i = 1, 2, 3,

for different values of σ _i / s _i and different image sequences, the result can be approximated well by a piece-wise linear curve. It is therefore possible to predict the approximate image volume _i for the i-th image by adaptive, linear extrapolation or extrapolation from the coding parameters B _i-1 , σ _i , σ _i-1 and s _i and s _i-1 . It then applies to the predicted or estimated image volume _i

Before coding the i-th picture, all coding parameters are known except for s _i . If one chooses the target image volume BS _i for B _i , equation (3) is to be understood as an equation for the unknown s _i , which specifies the quantization step size s _i to be used to code the i-th image so that Estimated image volume gives the target image volume BS _i . When coding the first image of a sequence, the step size s _{i is} chosen to be 62.

It still has to be clarified how the slope 2 S _{i of} the estimation line and the target image volume BS _{i are} to be determined and how the quantizer 1 C is to be set if a value s _i results from (3) purely arithmetically for the quantization step size is not provided for in (I), which is therefore not compatible with standards. According to (I), the step size can only assume an even number between two and 62.

For the first two pictures of a picture sequence the Slope of the estimation line to one close by experience set value, namely 40,000 set, d. h., for S₁ and S₂ the value 20,000 is selected in each case. To determine the Slope of the estimation line for the (i + 1) -th picture with i < 2 is first checked whether the condition

is satisfied. If it is not fulfilled, S _{i + 1} = S _{i is} set. Otherwise the quotient

calculated. The value of the auxiliary variable S * _{i + 1} now decides which value is assigned to the (half) slope S _{i + 1} . The calculation of the slope S _{i + 1} is now to be represented by a pseudo program which a person skilled in the art can easily convert into a control program for the control unit 3 . Conditional instructions are skipped in the pseudo program if the condition preceding the instruction is not met.

• 1. Start!  
• 2. If the inequality ( 4 ) is not fulfilled, set S _{i + 1} : = S _i and jump to 7!
• 3. Calculate the size S * _{i + 1} according to equation (5)!
• 4. If S * _{i + 1 is} less than or equal to 7 _* S _i / 8, then set S _{i + 1} : = 7 _* S _i / 8 and jump to 7!
• 5. If S * _{i + 1 is} greater than 9 _* S _i / 8, then set S _{i + 1} : = 9 _* S _i / 8 and jump to 7!
• 6. Set S _{i + 1} : = S _i !
• 7. End!

The target image volume BS _i for the i-th image is determined by the buffer fill level F _i before the coding of the i-th image, specifically according to the equation

BS _i = 5B₀ / 4 for BS _i 5B₀ / 4
BS _i = 3B₀ / 4 for BS _i 3B₀ / 4 (6)
BS _i = -F₁ + 2 _* FS _i otherwise,

where FS _{i means} the target fill level of the buffer memory 6 .

Similar to the calculation of the slope of the estimation straight line, the calculation of the target fill level FS _i is now explained by a pseudo program. First FS₁ = 3200 bit is set. The target level is kept as low as possible in order to keep the delay times mentioned above small. Therefore, the program contains a count variable K, which counts the number of images that would have led to an idle of the buffer memory 6 , in which therefore stuffing bits had to be inserted. This variable K is set to zero before coding begins.

• 1. Start!
• 2. Have stuff bits been inserted in the picture (i-1)? If not, jump to 5!
• 3. Is the level F _i greater than or equal to FS _i-1 - B₀ / 4? If not, jump after 8!
• 4. FS _i : = FS _i-1 + B₀ / 8 K: = 0; jump to 8!
• 5. K: = K + 1;
• 6. K = 5? If not, jump after 8!
• 7. FS _i : = FS _i-1 - B₀ / 8 K: = 0;
• 8. End!

As indicated above, the control unit 3 must make a further decision in the case (which is the rule) that equation (3) gives a value for s _i that is not one of the even numbers from 2 to 62. It should be emphasized here again that with s _{i are} always the values calculated using equation (3). In the present example, the procedure is as follows:

It is

s _iL = 2 _* INT (s _i / 2) and
s _iH = s _iL + 2 (6)

where INT (x) means the largest integer that is less than x. s _iL and s _iH are allowable increments according to (I) if they are not less than 2 or greater than 62. However, quantization with step size 2 is avoided - ie replaced by step size 4 - because the smallest step size almost inevitably leads to a buffer overflow.

The i-th image should now be quantized with the two step sizes that result from equation (6). Before coding the i-th image, it must be determined which portion of this image is to be quantized with s _iL and which portion with s _iH . To answer this question, two (positive) weights a _iH and a _iL with a _iH + a _iL = 1 are determined such that

a _{iH * i} (s _iH ) + a _{iL * i} (s _iL ) = BS _i (7)

becomes. Equation (7) contains the _{image volumes} estimated for s _iH and s _iL according to equation (3) and the target image volume. The resolution of (6) according to the weights gives the image parts sought. Since an image consists of N = 12 block groups (GOB's), N _L = INT (N _* a _iL ) block groups are quantized with the step size s _iL and the rest with the step size s _iH . The selection of the N _L block groups is made with a uniform distribution over the whole picture.

In FIG. 2, the number of images of a known test purposes image sequence ( "Ale xis") is plotted of 108 images on the horizontal axis. The PSNR (peak signal noise ratio) is shown on the vertical axis. The upper curve shows the PSNR for coding according to the invention and the lower one for coding with buffer control. The improvement with a coder according to the invention is obvious.

Claims (6)

1. Coder for segment-wise coding of an input signal (b) with a controllable quantizer ( 1 C) and a buffer memory ( 6 ), in which quantized and VLC-coded data of the input signal (b) are temporarily stored, characterized by means ( 2 , 3 , 4 , 7 ), which are intended to estimate a one-time setting of the quantizer ( 1 C) for coding the signal segment in such a way that the number of bits resulting from the coding of this signal segment in the buffer memory ( 6 ) is registered as closely as possible with a given number. 2. Coder according to claim 1, characterized in that for the case in which the estimated setting of the quantizer ( 1 C) is not standard-compatible, means ( 3 ) are provided with which this setting is converted into two standard-compatible settings, and that then part of the segment with the first setting and the second part with the second setting of the quantizer ( 1 C) is quantized. 3. Coder according to claim 1 or 2, characterized in that means ( 2 ) are provided for the coding of a signal segment to determine its activity, which is a function of the amounts of all data to be coded in a signal segment and which are used as estimation parameters for estimating the Setting the quantizer ( 1 C) is used. 4. Coder according to one of the preceding claims, characterized in that means ( 2 , 3 ) are provided for performing the estimation of the setting quantizer ( 1 C) for the current signal segment using coding parameters of previous signal segments. 5. Coder according to one of the preceding claims, characterized in that means ( 3 , 1 E) are provided for preventing the VLC coding of signal segments as long as the level of the buffer memory ( 6 ) is above a maximum level. 6. Coder according to one of the preceding claims, characterized in that the coder is a video encoder ( 1 ) and that the signal segments are video images or differences in video images (b).