WO1999038332A1

WO1999038332A1 - Method and device for converting image data blocks into image lines

Info

Publication number: WO1999038332A1
Application number: PCT/DE1999/000013
Authority: WO
Inventors: Christoph Heer; Mladen Berekovic
Original assignee: Siemens Aktiengesellschaft
Priority date: 1998-01-27
Filing date: 1999-01-07
Publication date: 1999-07-29
Also published as: JP2002502049A; KR20010034420A; EP1051854A1; WO1999038332A9

Abstract

The invention provides a special read- and write-addressing technique which halves the size of the memory required for the conversion by eliminating the need for the memory to 'double up'. Normally this is made necessary by simultaneous write- and read accesses.

Description

description

Method and device for converting image data blocks into image lines.

The invention relates to a method and a device in which image data blocks which follow one another in time in the line direction and contain, for example, brightness and color information for pixels, are converted into image lines in such a way that between the writing of the

Image data blocks Information for pixels of an image line is output from a memory in the correct sequence in time.

Such a method or such an arrangement is known from US Pat. No. 5,563,623 in connection with a control of an actively addressable display unit.

In the upcoming object-based video standard MPEG-4, additional memories for the objects of a previous picture are required for the prediction, since the picture to be displayed arises from the composition of different picture objects and no longer represents them completely and unchanged. Two separate memories are therefore necessary, namely a so-called frame buffer for holding image data blocks and a memory for image objects, which means a higher hardware expenditure in comparison to earlier video standards.

The object on which the invention is based is now a device or an arrangement for converting

Specify image data blocks in image lines in which the total memory requirement is as small as possible.

This object is achieved with regard to the method by the features of patent claim 1 and with regard to the arrangement by the features of patent claim 2. The further claims relate to advantageous configurations of the device according to the invention.

The invention is explained in more detail below with reference to an exemplary embodiment shown in the drawings. It shows

FIG. 1 shows an illustration to explain the data structures and memory addressing,

FIG. 2 shows a flow chart to explain the method,

FIG. 3 shows a block diagram of a device for carrying out the method,

Figure 4 is a block diagram for explaining the function of an input address generator of Figures 3 and

FIG. 5 shows a block diagram for explaining the output address generator from FIG. 3.

The underlying object is achieved according to the invention in that image block data are written into a memory and image lines are read from the memory in such a way that the memory only has to be dimensioned so large that it can accommodate a line of image blocks. A so-called doubling of the memory due to the simultaneous write and read operations is not necessary.

In the first part of FIG. 1, an image B consisting of image block lines 1 ... 36 is shown, each image block line having 44 image blocks. The picture blocks of picture block row 1 are labeled 1.1, ..., 1.44, the picture blocks of picture block row 2 are labeled 2.1, ..., 2.44 and the last block of picture block row 36 is labeled 36.44. Each image block has, as indicated by way of example in image block 1.1, M = 16 words W] _ ... Wι_g on. In the example described here, the CCIR-601 standard with 704 x 576 pixels per field (frame) and a 4: 2: 2 format for brightness and color information is assumed. In the lower part of FIG. 1 this is exemplified by a word W _n in which the image information for 16 pixels is divided into 8 columns SP1 ... SP8, with column SP1 two brightness information Y0 and Yl and two color difference values U0 and V0 and column SP8 has two brightness values Y14 and Y15 and two color difference values U7 and V7. Each line of a word in an image block thus has the data of 16 pixels, two pixels sharing a common color described by two color difference values.

In addition, triples T21, T22, T31 and T32 are indicated as examples in the upper first part of FIG. 1 in the picture block lines 2 and 3, the triplet T21 consisting of the picture blocks 2.1, 2.2 and 2.3, the triplet T22 consisting of the picture blocks 2.4, 2.5 and 2.6, the triple T31 consist of blocks 3.1, 3.2 and 3.3 and the triple T32 consist of image data blocks 3.4, 3.5 and 3.6.

In a second part of FIG. 1, a memory M is shown with memory blocks M]... M48 and three memory blocks Mη_, M2 and M ₃ up to M45, M47 and M48 to memory triple TM1 to memory block triple TM16 are summarized. A third part of FIG. 1 shows a video image V with image lines L _] _ ... L576, the image line L] _ initially having a pixel P1 and at the end of the line having a pixel 704.

A memory block represents a memory area that can hold 16 x 16 pixels, for example. Memory blocks can advantageously correspond to exactly one image block, but in principle a different size of a memory block is also possible. A flow chart is shown in FIG. 2 to explain the exemplary method. It is clear from this that first, in a first step, all image blocks 1.1 ... 1.44 of the first block line 1 of image B are calculated and stored in the memory M. Thereupon, in a second step, the first image line L_ is output as the first line of the memory M up to the memory block M44. As soon as the first line L_ has been output, the first three lines of the memory blocks can already be written with the first three image blocks 2.1, 2.2 and 2.3 of the next block line 2 of the image B. These are stored nested, that is, in the first memory block triple TM1, as entered in the second part of FIG. 1, the first words W Worte (2.1) ..., Wχ (2.3) of the first trip ice T21 and in the memory block triple TM2 the second words Words W ₂ (2.1) ... W ₂ (2.3) of the first trip ice up to the sixteenth words W g (2.1), ..., W g (2.3) of the first trip ice are stored in the memory block triple TM16. As soon as the second line L _{2 has been} output, the respective second lines of the memory blocks can be written with the next triple T22 of the next block line 2. These are also nested in a corresponding manner. At the same time, the third image line L3 is output. If all m = 16 image lines have been output or all k = m triples of the second image line have been written in, the entire memory, that is to say the memory blocks M ^ ... M48, is already filled again with the data for the next m = 16 lines. However, these picture lines are now stored "block by block". The first line of the second picture block line 2 contains the words W ₁ (2.1) ... W liegt (2.44) in the first memory block triple TM1. Accordingly, the picture line 17 can be read out line by line of the first memory block The memory block triple TM2 is then read out and at the same time the triple T31 of the third block line is written into the memory block triple TM1 read out and described with the new image blocks, the process can start again and convert block lines 3 and 4 into image lines 33 to 64 etc. With z = 576 image lines, the above-mentioned sub-procedure must be carried out z / 2 * m = 18 times from the second step, whereby, of course, no further block line is saved during the last run.

FIG. 3 shows a block circuit for carrying out the method specified above, which has the memory M, an input address generator EAG, a write switch SFW, an output address generator AAG and a read switch LSW in addition to a clock supply CLK. The clock supply CLK is connected to the input address generator EAG, the switching mechanism SSW, the reading switching mechanism LSW and the output address generator AG. The input address generator is controlled by an output signal MODI of the write switch and generates a write address SADR for the memory M. In addition, the write switch SSW generates a write activation signal SEN for the memory M. The write switch reports to the read switch LSW with the aid of a signal BZS that a Block line was written and the read switch LSW reports to the write switch using a signal ZL that a picture line has been read. The output address generator is driven by an output signal MOD2 of the read switching mechanism and generates a read address LADR in the memory M. Data DI is written into the memory M and data DO is read out.

An input memory MI and / or a FIFO memory FIFO are optionally additionally provided, indicated by dashed lines. If there is an input memory MI for storing the data DI, it can advantageously also be controlled by the input address generator. In the event that a FIFO memory is available for receiving the output data DO, this can advantageously be controlled by a signal FIN generated in the read switching mechanism. 14

15

or B-picture). A frame buffer 201 is used to temporarily disturb to re-ordering and processing needs. Each input picture is divi before encoding. Temporal redundancy of each MB may be re 202 and motion compensation 203. 5

After necessary motion compensation, a MB is subjected to discr 204 and DCT coefficient quantization 205 based on quantizatio stepsize. The quantized MB of I- or P-picture is inverse qua cosine transformed 210, and subjected to corresponding motion c quantized MB is reconstructed and stored locally in frame b estimation and compensation needs. The quantized MB is ru (variable length coding) at 206, together with all necessary si encoded bitstream of the input sequence. The encoded bitstream 207 of the encoder for output at 208 at desired data rates.

Methods of motion estimation 202, motion compensation 203, quantization 205, run-length VLC encoding 206, inverse quantiz pp _{Iied ln wh | ch the daemιin} 210 may be those defined and / or allowed in the ISO / IEC MPE ^ r _t ed _{by MB acti} -> «*« * o _Pt , _{ona cu} ; 20 Before encoding, a target quality of encoding (219) and maxi are set. The maximum and minimum bit rates (BR _mκ and RR define the boundary bit rates which the encoder shall operate at, the encoded picture quality that the encoder shall target by co rate within the given bit rate boundaries. The minimum bit ra

25

Initially, target bit rate estimation 223 may be performed based max min bit rates to generate an initial target bit rate (BR _largel ). based on experimentally determined fixed values which m picture (s) to be coded. With the target bit rate, the bit all

30 determine a target number of bits for the picture to be coded (

15

determines a reference quantization step size (QS _r for each MB based on the estimated number of bits for the picture (T _IPB ) and the bit utilization by the run-length VLC encoder 206. The QS _ref of each MB in the picture to be coded can be computed, for example, as follows:

where D _IPB is Virtual buffer fullness of corresponding I-, P-, or B-picture, updated (after coding each MB) by the difference between the bits used by the MB and the bits allocated to the MB based on the corresponding T _{! PB} , a set of initial values for D „D _P , D _B may be assumed at sequence Start, and K, is a constant (eg. 31).

Adaptive quantization 218 may be applied in which the determined QS _ref \ ^' s scaled according to the local activities of the MB as generated by MB activity calculation process 216 and the average MB activity of the previously coded (or optionally current picture) as produced by the frame activity average process 217. Example implementations of the MB activity calculation 216, frame activity average 217, and adaptive quantization process 218 are found in MPEG-2 TM5. The output quantization stepsize (QS) is used to quantized DCT coefficients of the MB.

After encoding a picture, the number of bits (S _{l PB} , corresponding value for I-, P-, or B- picture) generated by encoding the picture is accumulated by Frame Bit-Count 214, and the quantization stepsize (QS) is averaged by Frame Q Step Average 213 process. The encoded picture quality is also determined by Frame Quality Measure 220 process. One method of determining the encoded picture quality is by the average value of the reference quantization stepsize (QS _average ) used for coding the picture since it indicates roughly the amount of quantization noise in the encoded picture. In this method, the target quality set at 219 is defmed as the target reference quantization stepsize (QS _ωrgel ). Three different QS _rule values

Claims

16

may be set at 219 for corresponding I-, P-, and B-pictures; alternatively, the values can be determined by:

QS, _arga for I-pictures = QS _Mτga QS _{mrg «} for P-pictures = K _P * QS _ωrga QS _mrgel for B-pictures = K _B * QS _arget

where K _P and K _B are constants which can be experimentally determined (example K _P = 1.0 d K _B = 1.4).

A fiirther condition may be applied to the final ÖS " _r - _" ., For B-pictures such that it is not lower than the QS _averagf of the last coded I- or P-picture so that bits can be efficiently used to improve the quality of anchor I- or P-pictures first before improving the quality of the B- pictures.

In the process of Frame Quality Measure 220 process, the target quality QS _Mτga is compared to the picture quality QS _average . Since a higher value of QS _merage implies higher quantization noise and therefore lower encoded picture quality, when the value of QS _avemge is found to be higher than QS _arga , the difference of the two values will be used to increase the target bit rate BR _urg at target bitrate estimator 222. On the other hand, when QS _merage is lower than QS _Mrge "BR _mτget is decreased at target bitrate estimator 222. Optionally, the value of QS ^^, may be used at the rate Controller 215 as a lower limit for the final output QS _ref value so that when this target is reached (target quality reached), bits are saved for future encoding immediately. Hence, the output reference quantization stepsize at rate Controller 215 may be set according to:

if (QS _ref <QS _urg , then QS _ref = QS _ωrget .

For target bit rate estimation, a rate-quantization model for example one developed by Wei

Ding and Bede Liu, "Rate Control of MPEG Video Coding and Recording by Rate- Quantization Modeling", IEEE Trans, on Circuit and Systems for Video Technology, Vol. 17

6, No. 1, Feb. 1996, may be adopted. To avoid large estimation error or complex local fitting of the rate-quantization model, an alternative embodiment of the present invention may consist of a method of target bitrate estimation comprising the steps of predicting a bitrate (BR _predictat ) at a Bitrate Predictor 221 based on bits used for encoding the last I-, P-, and B- picture, and esümating a new target bitrate (BR _Mrge ^) at the target bitrate estimator 222 based on the said predicted bit rate and the difference between QS _awrngr and QS _lorχ ". The predicted bitrate (BR _{pre (iicled} ) and the new target bitrate (BR _Mrgel ) may be computed as follows:

(S _j + n _p ^ S _p + n _B ^ S _B )

BR predicted ₍ . _{+ / |}}

Dp. Dp _j . f (OS average - O —S target>) ^ϋK targβ ~ ^ßK predicted ^{+ K} 2 *

ÖS average

if (BR _ωτget > BR, then BR _mrga = 5R „if (BR _mrgel <5R _TO . _π ), then BR _mga = BR _m

where

S „S _P , S _B are number of bits generated by previously encoded I-, P-, B-picture, n _P is the total number of P-pictures in the current GOP, n _B is the total number of B-pictures in the current GOP, and K ₂ may be a constant, or a factor of BR _predicud , BR ^, or BR _arga .

The new target bit rate (5R _Mr. _Rt ) can be computed after encoding each picture or after encoding a certain number of pictures. The bit allocation process 212 and the rate controller 215 are updated with the new BR _urge , value once it is determined. An embodiment of the bit allocation process 212 comprises the steps of.

a) Before encoding the first picture in a group of pictures (GOP), determining R which