GB2306829A - Digital video decoding apparatus - Google Patents

Abstract

Digital video signals are often coded such that at least some of the frames of video data are coded in dependence on data contained in other frames eg. MPEG coded. An apparatus for decoding such a digital video signal receives the coded video signal and decodes the frames of data from which other frames in the video signal are derived. These frames are then stored eg. in DRAM 20, and can be output as desired. Furthermore, data compression means can be used prior to storage of the frames in conjunction with a corresponding decompression means at the output of the store.

Description

f DIGITAL VIDEO DECODING APPARATUS This invention relates to digital video decoding apparatus of the type which is used for decoding digital video signals which having a reduced bit rate. These may be of the type which is received by a broadcast transmission or from some digital storage medium.

Various proposals for reducing the bit rate of digital video signals have been made such as the MPEG-1 and the MPEG-2 algorithms from the Motion Pictures Experts Group (MPEG). The present invention will be described by way of example with reference to the MPEG standard.

Both MPEG standard reduces the bit rate of a digital video signal by defining the frames making up a video sequence as being of three types. These are as follows: 1. Intra (I) Pictures.

These are coded without reference to other pictures.

They serve as reference or access points to the coded video sequence and thus form a point where decoding can begin.

2. Predicted Pictures (P) These are coded with reference to the motion compensated prediction derived from a previous I or P picture. Only the data defined in the changes between the I or P frame and the P frame under consideration need to be stored or transmitted to enable the P frame to be reconstructed.

3. Bi-Directional Predicted Pictures (B) These are coded with reference to a forward motion compensated prediction from a previous I or P picture and a backward motion compensated prediction from a future I or P picture. They provide the greatest degree of compression of data. A decoder for the MPEG sequence reconstructs a new frame from this data and provides it for display.

In a transmission system the B frame data is sent at the same rate as the frames which need to be displayed.

Therefore, the decoder does not need to store the whole of this final frame. One field is sufficient, the other is decoded and displayed in real time.

In real systems the display device is likely to be a PAL or NTSC television where a frame of data is formed from two interlaced fields. As the decoder builds the frame a line at a time, successively from the top, it has to display a line and then store a line alternately, passing the stored information to the display device for the second interlaced field only after the first has been completed.

It will be appreciated from this that it is necessary to store data for up to two and a half frames, one I frame, one P frame and half of the B frame being displayed to support the MPEG decoding process. The amount of data represented in a complete frame is a much as five megabits and therefore up to twelve and a half megabits of dynamic random access memory (DRAM) storage is required for video decoding.

The sequence of I, P and B frames and the frames from which they are decoded is illustrated with reference to Figure 1.

In a receiver and decoder system for MPEG video, memory is required not only to achieve the MPEG video decoding but also for other functions such as MPEG audio decoding, holding the program for the control microprocessor, collecting and storing data for electronic program guides, and parsing the received data stream to extract just the data required for the program to be viewed.

This memory is normally provided in the form of DRAM and the cost of it represents a very large proportion of the overall cost of the receiver/decoder. It can often represent as much as a third of the retail price.

One way in which costs are reduced in high volume consumer products is to replace two or more discreet components with a single, more integrated component. The savings in this are typically the packaging costs of one or more of the components. However, because of the high cost of the DRAM this process of integration can only have a limited effect on possible cost reductions.

In accordance with preferred embodiments of the present invention various ways are proposed for reducing the memory requirements for a digital video decoder of the MPEG type.

The invention is defined in its various aspects. In the appended claims to which reference should now be made.

Figure 1 shows a sequence of MPEG frames; Figure 2 shows a standard MPEG decoder; Figure 3 shows a partially integrated MPEG decoder; Figure 4 shows a totally integrated MPEG decoder; Figure 5 shows how the central program is loaded into and accessed from DRAM; Figure 6 shows how the DRAM in Figure 5 can be reduced in accordance with an embodiment of the invention; Figure 7 shows a standard bit map for character display; Figure 8 shows storage of character data in accordance with an embodiment of the invention; Figure 9 shows a flow diagram for implementing the storage system in Figure 8; Figure 10 shows converted DRAM frame storage for use in an MPEG decoder; Figure 11 shows a first improvement in the DRAM storage of Figure 10; and Figure 12 shows a second improvement on the DRAM storage of Figure 10.

The various techniques for reducing the memory requirement in a digital video decoder are now described by way of example with reference to the accompanying drawings in which: 1. Memory Combination This comprises combining the memory requirement for MPEG audio and video decoding with the memory required for the rest of the decoder, for example that required for onscreen displays and the like. Receiver decoder systems have a number of buffers to process the received data stream to verify the MPEG data before it is passed to be audio and video decoders. The purpose of this is to smooth out the data being passed to the MPEG decoders and to synchronise the audio and video outputs. By combining the DRAM in the various sections of the design into a single area these buffers can be integrated together.A typical saving that can be expected by doing this is of the order of 100 kilobits.

Figure 2 shows the memory arrangement of a typical MPEG receiver decoder. This comprises an MPEG transport stream demultiplexer 2 which receives the incoming MPEG encoded data. A two megabit buffer memory 4 is coupled to this for storage of received data which enables data to be output at the correct times to an MPEG audio decoder 6 and an MPEG video decoder and on-screen display generator 8.

Also coupled to the MPEG transport stream demultiplexer is a control microprocessor 10 which controls its operation. Program data for the control microprocessor is held in a permanent store 12 and from this is loaded into program memory 14 from where it can be accessed by the microprocessor 10.

The MPEG audio decoder is coupled to a 64 kilobit buffer memory 16 for holding intermediate data and the MPEG video decoder is coupled to a 16 megabit frame store memory 18, also for holding intermediate data. The buffer memory 4, the program memory 14, the buffer memory 16, and the frame store memory 18 are all DRAM memories and the amount of DRAM memory is therefore over 20 megabits.

Because DRAM is manufactured in standard sizes, and because these sizes are largely controlled by the PC computer market place, there is a trend towards larger and larger capacities, thus making packages smaller than four megabits more expensive and less obtainable. This trend is set to continue and so it is likely that the same statement will apply to packages smaller than 16 megabits in a few years time since these are already replacing the four megabit sizes in the PC market place.

Some of the newer MPEG chips have already allowed the combination of the audio and video decoder memories and the program and transport multiplexer memories. This in turn allows the small DRAM packages to be removed thus easing the purchasing requirements. An illustration of this is seen in Figure 3 in which the MPEG audio and video decoders have been combined into a joint and audio and video decoder and on-screen display generator 7. The program and buffer memories for control and storage relating to the MPEG transport stream demultiplexer are combined in a single DRAM memory 18. Similarly the buffer and frame store memories associated with the audio and video decoders are combined in a single DRAM memory 20.

This, therefore, leads to a reduction in DRAM capacity by 64 kilobits and also to a reduction in integrated circuit packaging by virtue of the combination of the MPEG audio and video decoders.

Taking the integration one step further we get to the situation shown in Figure 4 in which the MPEG transport stream demultiplexer and MPEG audio and video decoders and on-screen display generator have been combined in a single chipset 22 coupled to a single DRAM memory 24 of 20 megabits which performs all the functions of the four separate DRAM memories in Figure 2. This sharing of buffers allows the memory allocation in the DRAM to be dynamically reconfigured to provide more memory for specific functions as they are required. For example, high resolution on-screen displays require memory only when they are used. This memory is free for use by other functions when OSD is not required. Thus, there is a reduction in DRAM capacity and a significant reduction in packaging costs.

2. Program Overlay The control program stored in the permanent storage memory 12 of Figure 2 is normally all transferred to the program memory 14 of Figure 2 from where it is then executed. This allows full speed operation of the program since DRAM can be accessed much more quickly than permanent storage memory such as EPROM. This process is shown in Figure 5. In Figure 5a the program is written frm the permanent storage 12 to the DRAM 14 and in Figure Sb the control program is executed by the control microprocessor 10 from the DRAM 14. The permanent storage 12 has no further use at this stage.

In most application programs there are large sections which are rarely used except for on specific occasional events. There are also large sections for the which speed of operation is relatively unimportant.

Instead of transferring all of the control programs to the DRAM a smaller DRAM may be used if only the essential parts are transferred. The rest may be kept in permanent storage until needed or may even be executed slowly directly from the permanent storage area 12, thus allowing the size of the program memory DRAM to be reduced.

This operation is illustrated with reference to Figure 6. In Figure 6a only the essential portions of control program, eg. the main flow structure of the program for demultiplexing the MPEG data is transferred to a smaller DRAM 15 from the permanent storage 12. As illustrated this DRAM includes a spare area 22. This area is left free for temporary insertion of other program modules. In Figure 6b is illustrated the situation where one of the other program modules is required. The control microprocessor determines that a module from permanent storage is required. This is then downloaded from the permanent storage 12 to the spare area 26 in the DRAM 15 from where it may be executed. Alternatively the module may be executed directly from the permanent storage area via the link 28 (albeit more slowly).

3. Compression of On-Screen Display Data On-screen display (OSD) data can be a significant quantity. With the more advanced digital video receiver decoders which are now proposed colour displays requiring four bits per pixel are proposed which can require as much as 1.65 megabits of data storage per screenfull of data.

As on-screen displays often have large areas of repeated colour it is proposed to use a storage method that can represent these areas of repeated colour by a simpler method in which a short message such as "fill the next 36 pixels with colour 27" is used instead of repeating the message "this pixel is colour 27" 36 times. The savings which could be achieved by this method are dependent on the OSD contents. It is expected that typically 50k (more than 800 kilobits) of the required storage could be saved.

An example of a single character from an on-screen display and the data storage requirement for it is shown in Figure 7. In Figure 7a the actual representation of the character on the pixels of display is shown. In this case it is a representation of the letter A displayed in colour X on a background of colour Y. The data storage required by a conventional system to represent this is shown in Figure 7b. It will be seen for each pixel there is a data storage location containing either colour X or colour Y as is appropriate to the colour to be displayed.

The total memory requirement to store the data shown in Figure 7, assuming there are four bits per pixel, is 1,024 bits, this being equal to a grid of 16 pixels by 16 pixels with four bits per pixel.

It can be seen from Figure 7b that there are many instances where the same data is entered repeatedly, scanning along the lines of the display. By using a method of simply indicating this repetition the storage space required may be reduced. One method of doing this is to use a code to signal "repeat" N times the colour C".

The table in Figure Bb shows this in action. Instead of entering the colour required many times a repeat code of three entries is used to signal a repeated colour. One of these entries is shown as the first entry in the first line of Figure 8b. The first entry is the repeat code signal "RC", the second entry is the number of pixels of the colour that is required, and the third entry is the actual colour required. Hence in the first line RC, 7, Y means repeat the colour Y seven times. This will correspond to the first seven pixels of the first line of Figure 8a.

If there are four bits per entry in Figure 8b then the total amount of storage required to represent the character A will be 648 bits. Clearly this is a siyniicane saving on Figure 7 amounting to 36.7. This situation will be improved further in a real system as the repeat coding for the background would extend across adjacent characters, typically saving a further three entries per line. Extending the above calculation to encompass this gives a new data space requirement of 456 characters amounting to a saving of 55.4k over the encoded data storage.

In the above calculations it has been assumed that the repeat code value is a four bit number. That is to say it is the same as the colour data. In practice this may not be practical as the maximum length of repeated colour sections would be only 16 pixels. It may often be necessary to have a greater length of repeated signals.

For example, for the spaces between lines and characters the entire width of the television display might be a single colour over a number of lines. If we were to select an 8 bit repeat value which allows repeats up to 256 pixels it would be a maximum saving of 98.8W of data storage. However, a saving for short repeated sections in areas of text would be adversely affected as it would not be practical to use repeat coding for less than four repeated pixels instead of the three used for the four bit coding. The saving for the letter A in the example above, for example, would be reduced from 36.7% to 26.2% if 8 bits per pixel were used. Another drawback is that the data entry used to represent the repeat code indication would have to replace one of the codes assigned to the colour choices.In a four bit system this would limit the number of colours available to 15 instead 16. This may not be a serious problem but, if access to all possible colours is required, then the colour that has been replaced by the repeat code can still be displayed by using the repeat code with a colour entry to match the replaced colour. For example, if the value 0 is being used to signal a repeat code requirement, then two pixels of colour 0 can be displayed by entering 0, 2, 0 - repeat code repeat twice colour 0.

Whilst this method makes access to this one colour a little awkward, and lengthens the data required for just one or two pixels of the colour, the potential savings in other areas can easily outweigh this limitation. If a fully programmable colour look up table (CLUT) is available, then the colour codes may be programmed to make the best use of the repeat coding method, taking into account least used colours and colours with the longest repeats.

A flow chart showing the decoding of data stored as shown in Figure 8 is shown in Figure 9. It will be seen that this flow chart identifies whether or not data stored is the repeat code and then fetches the repeat value and repeat colour data from memory for outputting the colour data over the required number of pixels. In the event that the data is not a repeat code data all that happens is that the required output colour data is displayed.

4. Frame Compression The amount of DRAM memory required to support full decoding of an MPEG signal can generally be considered as that shown in the system of Figure 10. This comprises the video decoder block 8 with its associated frame store memory 18.

Incoming data is passed to an I frame decoder 30 which decodes the first intraframe in a sequence. This is stored in a first portion 32 of the frame store memory 18.

Incoming data also passes to a P frame decoder 34 which reads I frame data from the portion of memory 32 where the I frame is stored. From the I frame data and the P frame data a P frame is decoded and stored in a portion 36 of memory 18. The B frame data is fed to a B frame decoder 38 which also receives data from the I frame storage portion 32 and the P frame storage portion 36.

From this a half frame or field of interlaced B frame data is decoded and stored in a storage portion 40 of memory 18. The other field is received and output in real time.

A final portion 42 of memory 18 is used to store onscreen display data ready for output.

All four memory portions, 32, 36, 40, and 42 have outputs to an output combiner which is controlled to select the appropriate portion of data for display. This also receives an input 44 directly from the B frame decoder. In operation firstly an I frame is output, as a pair of interlaced fields. Next a series of B frames are output with the fields being read alternately from the B frame storage area and directly from the B frame decoder 38.

Next a P frame is output from the storage portion 36 followed by further B frames. The process repeats until the next I frame is reached.

The on-screen display data is stored in memory portion 42 as a pixel by pixel bit map.

One way in which the DRAM requirement for the Figure 10 can be reduced is illustrated in Figure 11. If the MPEG I frame data is stored in the form it is received which will be compressed rather than in its decoded form less DRAM is required. Thus, the I frame decoder 30 is now connected to the output of the I frame storage area 32 and the output combiner 45. A further I frame decoder 31 is required in this arrangement to supply I frame data to the P frame decoder and the B frame decoder such that they can decode and make available for storage the respective P and B frame data.

The I frame MPEG data is only decoded as required.

Using this method can save up to four megabits of DRAM memory.

As an alternative the arrangement of Figure 12 can be used. This is a variation on the circuit of Figure 10.

The variation is that data being read into and read out of the various DRAM portions 32, 36, and 40 is compressed before being stored and then decompressed when it is read out.

The compression and decompression is presently different to MPEG compression and decompression in that it does not require data from other frames and can be implemented either in hardware or software using any appropriate compression and decompression algorithm. If it is implemented in software a suitable microprocessor will be required. Less DRAM could be saved if not all the frames were compressed.

Us inc the ar=gement of Figure 12 the P and B frame decoders 34 and 38 will require random access into the I and P frame stores. This is because the MPEG data can effectively contain instructions such as move a particular block which was in the bottom right hand corner to the middle of the display. To achieve this with the compressed frame storage methods pointers may be included in the compressed data or, alternatively, in a separate look up table thereby allowing the decompression process to skip around the stored data when access to a particular portion is required at random.

It will be appreciated that using one or more of the ideas described above along with the memory sharing permitted by integration of the transport demultiplexing functions with the MPEG audio and video decoding functions it should be possible to reduce the memory requirements of a whole receiver decoder to 16 megabits or less. This represents a substantial cost saving and also allows a single large DRAM device to be used rather than a combination of the smaller devices.

Where compression and decompression are used in a software driven system a fast microprocessor is required encompassing dedicated hardware to assist it with specific functions and coupled to one of the fast access types of DRAM.

If a sufficiently powerful microprocessor is selected then all of the receiver control functions can also be handled by it thereby again further reducing the cost of the receiver decoder.

Claims (15)

1. .A method for decoding a digital video signal coded such that at least some of the frames of video data are coded in dependence on data contained in other frames comprising the steps of receiving the coded video signal, decoding and storing the frames from which other frames are derived, decoding and storing the frames coded in dependence on the previously decoded frames, outputting the thus decoded frames and compressing at least one frame prior to storing it and decompressing it prior to output.

2. A method according to claim 1 comprising compressing all the frames prior to storage and decompressing them prior to output.

3. A method according to claim 1 or 2 in which the video signal is interlaced and for at least some of the frames only alternate interlaced fields are stored after decoding.

4. Apparatus for decoding a digital video signal coded such that at least some of the frames of video data are coded in dependence on data contained in other frames comprising means for receiving the coded video signal, means for decoding the frames of data from which other frames are derived, means for storing the frames of data from which other frames are derived, means for outputting the thus decoded frames, and means for compressing at least one frame of the video signal for storage in the storing means and means for decompressing the thus compressed frame prior to the output means.

5. Apparatus according to the claim 4 including means for compressing all the frames of the video signal prior to sending them to the storing means and means for decompressing the stored frames before sending them to the output means.

6. Apparatus according to claim 4 or 5 in which the storing means also stores data relating to on-screen displays for use in the decoding apparatus.

7. Apparatus according to any of claims 4, 5 or 6 in which the storing means also stores a control program for use with a control microprocessor for the decoding apparatus.

8. Apparatus according to any of claims 4 to 8 in which the storing means also stores decoded audio data received with the coded video signal.

9. Apparatus according to any of claims 4 to 8 in which the storing means also comprises a buffer storage for demultiplexing the received coded digital signal.

10 Apparatus according to any of claims 4 to 9 in which the storing means comprises a single integrated circuit memory.

11. Apparatus according to any of claims 4 to 10 in which the digital video signal is coded and decoded in accordance with the Motion Pictures Expert Group (MPEG) standard.

12. Apparatus according to any of claims 7 to 11 including a permanent memory storage for the microprocessor control program and means for reading portions of this program to the storing means as they are required.

13. Apparatus according to claim 12 in which some portions of the control program are read directly from the permanent memory storage Lor execution by the micLoprocessor.

14. A method for storing a multi-coloured on-screen display for use with a video decoder comprising the steps of storing data defining a colour to be applied along a raster scanned line and storing data defining the number of pixels to which that colour is to be applied.

15. A method according to claim 5 wherein if the number of pixels to which the colour is to be applied is applied is less than a predetermined number, colour data is stored in bit map form.