DE4419678A1 - Storing picture, video and film information in semiconductor memories - Google Patents

Abstract

The picture data are subjected to a two-dimensional discrete cosine transformation (DCT) and then compressed by means of a quantisation stage (Q) and a Huffman encoding (VLC) before being passed to the memory. When they are read out, they are subjected to the corresponding reverse operations (VLD, Q-1, DCT) for decompressing.

Description

requirements

Generally, here is a special technical aspect for saving memory for Image processing algorithms related to MPEG1 (standard ISO / IEC 11 172) and MPEG2 (ISO / IEC 13818) and related standards described. For that are the current standards for JPEG, MPEG1 and MPEG2 with their algorithms as Basis and regarded as known to the reader.

2. Introduction

An MPEG encoder block diagram is shown in Fig. 1. Two image memories (image memories A, B) are required for processing the image data, the first (A) being used for reordering the incoming images, the second (B) for temporarily storing the necessary images to avoid error propagation, as he did must be installed in every MPEG decoder.

Depending on the image formats and standards to be processed, these memories have sizes between approx. 300 kByte (1/4 TV picture) and more than 4 MByte (HDTV picture). At this Instead, the method for minimizing memory presented here attacks with which it it is possible to drastically reduce this storage capacity.

The reduction in memory size (B) for an MPEG2 decoder is an example here listed:

Without memory minimization: size of two CCIR 601 norm images, at 4: 2: 0 Chroma format: 10 MBlt = 2 images × 720 image columns × 288 image lines × 2 interlaced Images × (8 bit (luminance) + 4 bit (chrominance)) compared to a maximum of approx. 4 MBlt depending on the desired image quality Memory minimization.

In principle, the storage capacity for compressed amounts of data can be as follows downsize:

Sp _k = Sp _n / k (1)

Spk: storage capacity for compressed data volume
Spn: storage capacity for uncompromised amount of data
k: compression factor

The compression factor (k) depends on the compression method and the desired (image) quality of the data to be saved.

3. Process for the compressed storage and intermediate storage of image, video and film information in semiconductor memories 3.1. Special procedures

Instead of the uncompressed storage of the original image data in the memory, it is subjected to a two-dimensional discrete cosine transformation (DCT) and then compressed in the memory using a quantization stage (Q) and a special Huffman coding (VLC) (see Fig. 2). The quantization control (Q-ST) controls the quantization.

When reading out the compressed data, these steps are then reversed by means of the corresponding inverse operations (VLD, Q ^-1 , IDCT).

This JPEG-like process has some special features:

• a) Images are only intra-coded
• b) Images are compressed to a fixed size regardless of the image complexity
• c) The quantization control (Q-St) works predictively, d. H. it takes into account during the image compression also the data measurements of the not yet to be expected edited image remnants.
• d) depending on the compression rates desired in the application, special custom Huffman tables used.
• e) if an uncoded block (8 × 8 pixels) contains less data than an encoded one, it will saved uncoded.

3.1. Components for the process for minimizing memory 3.1.1. Discrete Cosine Transform (DCT)

Any architecture that meets the requirements can be used Accuracy according to MPEG or H. 261 standard met.

3.1.2. Quantization (Q)

The basic principles of quantization are analogous to JPEG and also have the same two-stage structure. The MPEG or JPEG table is used as the quantization table. This together with 3.1.3. and 3.1.4. practically defines the compression factor (k).

3.1.3. Huffman coding (VLC)

The VLC ( Fig. 3) reads the quantized data in the Zigzag scan from RAM 1 and writes the first value (DC component in the frequency domain) to the output memory (RAM 2) uncoded. The following AC values are coded analogously to MPEG runlength (with respect to zero) and the runlength / level pairs thus created are then transformed into a variable length bit code using the Huffman table. If the coded block contains more data than the uncoded (this can be seen from the overflow of RAM 2), the block is output DCT-transformed but not compressed.

A bit stream with coded or partially also appears at the output of the VLC uncoded data that is parallelized according to the word length of the memory and can be saved. In a further memory there must be a flag for each block (uncoded / coded block) and the quantization factor q belonging to the block be filed.

This together with 3.1.2. and 3.1.4. practically defines the compression factor (k).

3.1.4. Quantization control (Q-ST)

The quantization control regulates the quantization factor (q) block by block. This happens depending on the following parameters:
a) Fill level of the image memory
b) expected complexity of the image
Suitable algorithms ensure that the image memory is always fully utilized, so that there is an optimum between image quality and memory requirements.

This together with 3.1.2. and 3.1.3. practically defines the compression factor (k).

3.1.5. Huffman decoding (VLD)

The image memory data is decoded using the inverse method VLC, whereby of course the flag for coded or uncoded blocks is evaluated got to.

3.1.6. Dequantization (Q ^-1 )

Works analog to quantization and generates the 12 bit wide input data for the inverse DCT.

3.1.7. Inverse Discrete Cosine Transform (IDCT)

Again, any hardware that meets the requirements of the above. Standards supported, be used.

3.2. General procedures

The image data to be stored can also be saved using other types of compression save.

3.2.1. Process without transformations

You can generally view the image data without first using DCT / IDCT or others having transformed special transformations in the frequency domain, too compress using the Shannon entropy theorem in the local area.

This can be done as follows. You first calculate an optimal one Huffman table of data in the spatial area (either on-line per image section, or fixed Precalculated) and thus uses the frequency distribution of the brightness or Chrominance values in the local area. The advantage of this method is that it is less Complexity of compression. To guarantee a fixed output data rate, this method can be extended by a suitable quantization level.

Another way of compression is to code the differences the brightness and chrominance values e.g. B. according to the DPCM process. There will be one nonlinear assignment between difference value and output value calculated and saved.

3.3. Possible other areas of application of the method except for MPEG, JPEG or image processing

In principle, audio or other data can also be compressed in semiconductor memories save. For this you can do the above. Procedure with his u. G. Variants too to use.  

The described method can also be used to data z. B. from the measurement or to store audio technology in a semiconductor memory. You can't do that only reduce the storage capacity, but also the memory bandwidth.

If you e.g. B. receives a data stream behind an analog-digital converter (ADC), the memory access time of a semiconductor memory connected behind the ADC exceeds the access time with the help of the compression method reduce to memory.

4. Variants of the procedure 4.1. Particularly high data compression

If an even higher data compression than with the procedure under 3.1. wanted , but the same high demands are placed on the image quality, one must optimal Huffman table can be found for an image section to be determined. A complete Huffman encoder must then be implemented in the VLC and save a table for the selected image section in the memory.

4.2. Prevention of error propagation

When using the method according to 3. on the memory in the prediction loop (image memory B) in the case of encoders or decoders, the errors resulting from this compression are normally added up to the next I-picture. If this is not desired, the error propagation can be limited to the successive B-images using the following procedure (see Fig. 4).

First, an I or P picture is written uncompressed into the memory, then incoming I or P picture becomes the one for the first 64 lines (due to the maximum Length of the motion vector) also stored uncompressed and with the uncompressed I / P image decompensated. Now at the same time it starts Read out the I-picture, compress it analogously to method 3. and move it to another location write back to memory.

The storage effort is therefore 1 image + x lines (x = max. Length of the Motion vector vertically), as well as an image compressed.

4.3. Random memory access

By storing a bit stream with codes of variable length, it is not after 3 possible to access individual (image) blocks, pixels, data elements or similar. For the application of the method to the image memory (B) in the prediction loop of the However, this is essential for encoders or for the buffer memory for the decoder motion decompensation varies depending on the motion vectors Image areas must be read out.

By using a block-based DCT / IDCT (i. A. Transformation and variable length coding) is the smallest freely accessible element of the 8 × 8 pixels large block (at MPEG). To enable this random access, you save the address of the first bit of each block together with the one under 3.1.3. mentioned flag and quantization factor in an address memory.

When reading out any image parts from the image memory by means of indirect addressing via the address memory, u. In addition to the desired user information, parts may also be read that are not actually required (see Fig. 5). This may increase the required memory bandwidth compared to the normally required (see 5.1.).

5. Description of the procedure for an MPEG 2 decoder

The image data of an image t ( Fig. 6), reordered and coded according to the MPEG 2 standard, are read in for decoding by the MPEG decoder ( Fig. 7).

In the decoder, the bit stream is decoded picture-wise via a so-called video buffering verifier (VBV), the variable length decoder (VLD), the dequantizer (Q ^-1 ) and the two-dimensional inverse discrete cosine transformation (IDCT) according to the MPEG 2 standard and the decompensation level fed. In the decompensation stage, the image data - if it is a P or B image - is matched with the image data of an (I or P) image t-2 (or t-1 and t-2 at B) in the memory Images).

The data from the image memory required for decompensation is normally addressed using the vectors coming from the Variable Length Decoder (VLD) from Fig. 7. These vectors indicate, by macroblock, a shift position of a macroblock from image t to the position of a macroblock from image t-2 (t-1 also possible with B images). The blocks from image t in the decompensation stage still require the associated blocks or pixels from the memory for the decompensation (image t-1, t-2). For this purpose, the data stored in compressed form in the image memory is decompressed again (according to 3.) and fed to the decompensation stage via the memory controller. I-pictures are directly decompensated without information from the memory. The image data for image t thus decompensated again correspond to a decoded normal image with luminance and chrominance values according to the CCIR standard.

The decoded picture data is now, if it is an I or P picture the memory controller is compressed and written to the image memory in order for a future P or B picture to be decoded are available. In this way the last two P-pictures (t-1) and (t-2) or an I- (t-2) and a P-picture (t-1) kept compressed.

The B-pictures are passed on from the decompensation stage directly to the output unit and are not buffered. The subsequent unit must be able to accept the image data block by block and not line by line as required for monitors. An exemplary timing is shown in Fig. 8.

5.1. Image and address memory for MPEG 2 decoders 5.1.1 Memory bandwidth

Since the image data according to the method from 3. specially coded bit stream in the Memory are written, it is as under 4.3. mentioned, not possible, optional access individual pixel-addressed image areas.

To read a macro-block-sized area of an image in the CCIR 601 (4: 2: 0) standard (4 blocks of luminance and two blocks of chrominance), 17/6 times the "useful information" must be read out of the image memory (see Fig. 5 ). When reading an image according to CCIR 601 4: 2: 0 PAL norm, this leads to a memory bandwidth of the image memory of:
720 (image columns) × 288 (image lines) × (8 + 4 bits per pixel) × 17/6 × 50 (Hz) = approx. 352 MBit / s, which corresponds to approx. 44 MByte / s.

Taking into account the timing shown in Fig. 8 and the division of the image data memory into two field memories, each with two memory banks, the bandwidth required for a word length of 36 bits is calculated as follows:
720 (image columns) × 288 (image lines) × (8 + 4 bits per pixel) × (17/6 (read out for decompensation) + 6/6 (read out for image output)) × 50 (Hz) = approx. 477 Mbit / s, which corresponds to approx. 60 MByte / s, i.e. a memory access time of 67 ns.

If one chooses a compression factor (k) for the highest picture quality Image memory data of k = 3 are averaged over 1/50 s but only 44/3 MBytes = 14.7 MByte / s transferred.

5.1.2. Data storage size

For the memory size of the image memory for compressed storage of the data (see formula 1) this means for the above case with k = 3:
720 (image columns) × 288 (image lines) × (8 + 4 bits per pixel) × 4 (memory banks, 2 I and 2 P images) / k = approx. 10 / k Mbit = approx. 3.3 Mbit.

5.1.3. Address memory

As in 4.3. the start addresses (bstadr) must be executed for each block with the quantization factor (q) and the flag (f) for coded or uncoded data in be stored in an address memory. Under the conditions made above the word width of the address for the image memory is:

5 bits (q) + 1 bits (f) + 20 bits (bstadr) = 26 bits per block.

The number of required entries in the address memory depends on the number of blocks (size) and the norm of the picture:
720 (image columns) / 8 (horizontal block size) × 288 (image lines) / 8 (vertical block size) × 1.5 (Luma & Chroma) = 4860 blocks per memory bank.

This results in a total of 19,440 26-bit entries, i. H. approx. 505 kbit for the Address memory.

5.1.4. Total memory requirements

Overall, this results in an MPEG 2 decoder for the image memory Memory requirement of 3.3 MBit (data memory) + 0.5 MBit (address memory) = approx.3.8 MBit compared to 10 MBit for the uncompressed storage of the images with CCIR 601, 4: 2: 0.

6. Comparison for hardware expenditure for an MPEG 2 decoder with / without memory minimization

For the special control of the image memory in the MPEG 2 decoder, additional hardware expenditure is required from a two-dimensional DCT, a quantizer (Q) and a Huffman encoder (VLC) including quantization control (Q-ST) to write to the partial memory areas, which is done by a multiplexer one of the two areas can be switched (see Fig. 7).

To read from the image memory, an IDCT, a dequantizer (Q ^-1 ) and a Huffman decoder (VLD) are required for each partial memory.

The estimated hardware expenditure with and without memory minimization is in the following in comparison representative of an MPEG 2 decoder, for  Image processing of CCIR 601 images, in 4: 2: 0 format, as described after 5. listed.

Additional estimated hardware expenditure for "basic functionality"

Possibly. could be in time-division multiplex operation of individual modules (e.g. DCT's, Q's) when using fast technologies, the hardware expenditure can be reduced even further. This can only be done in the individual application (limit frequency of the technology compared to the standard-dependent data volumes to be processed). The basic idea to be patented is independent of this.

Claims (3)

1. A method for storing image, video and film information in semiconductor memories, characterized in that the image data is stored in compressed form in the memory and is decompressed again when read out. 2. The method according to claim 1, characterized in that the image data undergoes a two-dimensional discrete cosine transformation (DCT), then compressed in a memory by means of a quantization stage (Q) and a Huffman coding (VLC) and when read out by means of the corresponding inverse operations (VLD, Q ^-1 , DCT) can be decompressed again. 3. The method according to claim 2, characterized in that the Quantization level (Q) through a predictive quantization control (Q-ST) is controlled.