GB2344031A

GB2344031A - Processing Digital Signals

Info

Publication number: GB2344031A
Application number: GB9825264A
Authority: GB
Inventors: Alois Martin Bock
Original assignee: NDS Ltd; Tandberg Television AS
Current assignee: Synamedia Ltd; Ericsson Television AS
Priority date: 1998-11-19
Filing date: 1998-11-19
Publication date: 2000-05-24
Anticipated expiration: 2018-11-19
Also published as: GB9825264D0; GB2344031B

Abstract

In an arrangement in which a plurality of bitstreams intended to be received by respective buffers are combined by a multiplexer, the order in which said bitstreams (eg video packet streams) are applied to the multiplexer is based on the computed extent to which contributions from each bit stream will occupy the intended receive buffers. As described the object is to prevent the receive buffers underflowing and overflowing. As shown counters 302 count data bytes and send size of picture data to corresponding decoder buffer models 303 to emulate occupancy of the corresponding actual decoder buffers.

Description

Improvements in or Relating to Processing Digital Signals The present invention relates to the multiplexing and remultiplexing of digital signals, and more advantageously methods and apparatus for improving the multiplexing and remultiplexing of digital video signals.

In a digital video broadcast system, digital video signals are compressed prior to transmission in order to make efficient use of the available channel bandwidth. Several standards for digital video compression have emerged including two Moving Picture Experts Group (MPEG) standards, MPEG-1 and MPEG-2, These standards describe the requirements for decoders and specify the format of a digital bitstream that decoders must be able decode. A digital bitstream may comprise video, audio, control or other data. Decoders must be designed to comply with these standards, although encoders and other transmission equipment may be designed in a variety of ways so long as a decoder is able to decode the transmitted signals.

In a multiplexer, a plurality of digital video broadcast signals in the form of bitstreams are input, often via an input buffer, and are multiplexed together to form a single multiplexed bitstream. A multiplexed bitstream comprises interspersed packets (or contributions) from each of the input bitstreams.

Many methods exist for selecting the way in which packets from each of the input streams are selected for inclusion in the multiplexed bitstream, some of which are discussed further below. A packet identification (PID) number is allocated to each packet in the multiplexed bitstream to identify from which particular individual bitstream it originates. At a decoder, the multiplexed bitstream can be filtered according to a specified PID, to enable an individual bitstream to be extracted.

In a typical multiplexer there is usually one input buffer per input bitstream.

Each input buffer in the multiplexer is used to temporarily store the incoming bitstream. The bitrate of a bitstream transmitted to an input buffer is usually constant. The bitrate out of an input buffer into the multiplexer, however, is often at a variable rate, since packets may be removed from each of the input buffers and inserted into the multiplexed bitstream in a non-uniform manner.

Input buffers are also present in decoders. These decoder buffers are generally smaller than the input buffers found in multiplexers, although they may equally be of the same size. In a decoder, encoded video frames are removed from a decoder buffer at a constant rate (the frame rate), as dictated by the various digital video compression standards. However, the bitrate, for any particular individual bitrate, into a decoder buffer is not generally constant due to the way in which multiplexed bitstreams are constructed. Due to the difference in input and output bitrates, serious consequences may arise if a decoder buffer is allowed either to overflow or underflow. Effective management of the decoder buffers must be maintained, and the occupancy of decoder buffers must be managed so that they are always at least part full.

Management of the decoder buffers must be performed at the transmission end of the system, since the decoders have no control over the data they receive.

Further problems may arise when using statistical multiplexing methods. Statistical multiplexing of compressed video bitstreams poses a number of problems which can be addressed in different ways. The aim is to achieve bitstreams which are of consistently high quality, even during critical sequences, yet remain decodable in normal decoder buffer sizes as defined in the relevant video compression standards. Amongst many other requirements, dynamic range and speed of response to bitrate changes are crucial factors for determining the performance of statistical multiplexing systems. One such implementation of real-time statistical multiplexing is described in our published patent EP0759667A.

One of the problems that arises in a multiplexer is deciding how to form the output multiplexed bitstream from the multiple input bitstreams. At the same time, it is important that the multiplexer selects data for transmission in such a way that a decoder buffer neither underflows nor overflows, which could result in a breakdown of the decoding process.

Referring now to Figure 1, there is shown an example of a typical prior art system for selecting input bitstreams for inclusion in a multiplexed digital bitstream.

A number of input bitstreams 204 are input to a number of input buffers 201.

Each input bitstream may represent video, audio, data or other signals.

The input buffers 201 are used to temporarily store the input bitstreams 204 before passing the stored input bitstreams to a multiplexer 203. A selection controller 202 is connected to the input buffers 204 and the multiplexer 203 and is used to select which of the stored input bitstreams is to be input to the multiplexer 203. The multiplexer 203 produces a multiplexed bitstream 205 containing contributions from each of the input bitstreams. The multiplexed bitstream is then transmitted, via a transmission channel 102, to a decoder one of which is shown at 103.

In conventional systems each of the input bitstreams 204 is allocated a fixed priority level. When a predetermined unit of data (or packet) is available in an input buffer this is signalled to the selection controller 202. The selection controller 202 decides from which of the input buffers 201 the next available unit of data is to be taken for inclusion in the multiplexed bitstream 205. If only one input buffer has a unit of data available then it is this input buffer that will be selected. If more than one input buffer has a unit of data available then a decision as to which one to choose must be made.

In the case where a fixed priority level is allocated to each input bitstream the decision will be based on the highest priority level input buffer having an available unit of data. Problems may arise, Yhowever, with the abovedescribed conventional system, especially in the situation where a high priority input bitstream contains more units of data than a lower priority input bitstream. This could result in the high priority input bitstream'hogging'the multiplexer. This can result in the lower priority input bitstreams causing the decoder buffer to underflow in a decoder, and the higher priority input bitstreams causing decoder buffer overflow. Both decoder buffer overflow and underflow can result in the breakdown of the decoding process and picture quality may be adversely affected.

Another common variation on the above-described method involves randomly allocating priority levels for each input bitstream 204. This method, however, shares many of the above-mentioned problems.

In a decoder, a decoder buffer is used to temporarily store received data until sufficient data has been received to allow decoding of an encoded frame to take place. Since the size of the input buffer is dictated by compression standards, only a limited amount of data may be stored. Frames are removed from the decoder buffer at a constant rate (the frame rate). If a frame is due to be removed from the decoder buffer, but the frame is not entirely present in the decoder buffer, the decoding process may breakdown.

Accordingly, one aspect of the present invention is to provide an improved way of selecting an input bitstream for inclusion in a multiplexed bitstream, by reducing the occurrence of underflow or overflow.

According to one aspect of the present invention there is provided a method of transmitting a plurality of individual bitstreams intended to be received by respective receiver buffers, the method comprising the steps of presenting the bitstreams to a multiplexer ; combining the presented bitstreams by means of the multiplexer to form a multiplex signal ; and transmitting the multiplex signal ; wherein the step of combining the bitstreams by means of the multiplexer includes releasing contributions from each bitstream into the multiplexer; computing the extent to which the released contributions will occupy the intended receiver buffers; and employing the computations so made to select the order in which the bitstreams make their contributions to the multiplex signal.

According to a second aspect of the present invention there is provided apparatus for transmitting a plurality of individual bitstreams intended to be received by respective receiver buffers, the apparatus comprising: a multiplexer to receive and combine the bitstreams to form a multiplex signal by taking contributions from each bitstream ; a computer to compute the extent to which the contributions from the bitstreams will occupy the intended receiver buffers ; a controller to select, by reference to the contributions made by the computer, the order in which the bitstreams contribute to the multiplexer ; and a transmission channel to transmit the multiplex signal.

The invention will now be described by way of example, with reference to the accompanying drawings, in which: Figure 1 is a diagram showing a typical prior art system; Figure 2 is a diagram showing an overview of a typical broadcasting system; Figure 3 is a diagram showing one embodiment of the present invention; Figure 4 is a diagram showing a representation of a section of a digital video broadcast bitstream; Figure 5 is a diagram showing how a de-multiplexer may be used to provide the inputs to the system of Figure 3; Figure 6 is a flow diagram showing the steps involved in the selection procedure of the system of Figure 3; and Figure 7 is a diagram showing an example of a second embodiment of the present invention.

Figure 2 is a diagram showing an overview of a typical broadcasting system. One or more encoders 100 each encode source material to produce a compressed digital input bitstream 104. The input bitstreams 104 are fed to a multiplexer 101 which intersperses data from each bitstream to form a multiplexed bitstream 105. The multiplexed bitstream 105, which may include video, audio, data and control signals, is then transmitted through a transmission channel 102 to one or more receivers, one of which is shown at 103. It will be apparent to anyone skilled in the art that the transmission channel could include any type of transmission channel, including satellite, cable, microwave, terrestrial and communications networks.

Referring now to Figure 3, there is shown a diagram showing one embodiment of the present invention.

An input bitstream 204 is input to a picture header decoder 301 and a counter 302. The picture header decoder 301 is connected to an input buffer 304 and the counter 302. The input buffer 304 is also connected to a multiplexer 306.

The counter 302 is connected to an input on a decoder buffer model 303. A selection controller 305 accepts an input from each of the decoder buffer models, one of which is shown at 303, and provides a control signal 307 to the multiplexer 306.

Figure 4 shows a section of a video bitstream, which comprises both picture headers 400 and picture data 401. The picture header decoder 301 functions to detect the presence of picture headers in the video bitstream. When a picture header 400 is detected the counter 302 is reset to coincide with the start of the picture data 401. The counter then starts counting the number of bytes contained in the picture data. When the start of the next picture header is detected, the picture header decoder 301 causes the counter to send the size of the picture data 401 to the decoder buffer model 303 and also causes the counter to reset to begin counting the size of the next picture data.

As previously described, pictures are removed from the decoder buffer at a constant rate, as dictated by the various digital video compression standards.

Each picture has an associated time-stamp, which indicates to a decoder a time by which the picture should be removed from the decoder buffer to be decoded. This helps ensure that the picture is decoded in good time to be displayed at the required time.

The decoder buffer model 303 models the behaviour of a decoder buffer in a decoder. In order to maintain the correct flow of data from the multiplexer to a receiver the decoder buffers must ideally never be allowed to underflow or overflow. The decoder buffer model allows the performance of a decoder buffer to be simulated to assist in maintaining the correct flow of data to the decoder.

As data is removed from the input buffer 304 to be included in the multiplexed bitstream the size of that data is added to the decoder buffer model 303. The time-stamps are also monitored which allows the decoder buffer models to simulate the removal of pictures from real decoder buffers. This process enables the decoder buffer model 303 to accurately model the occupancy of an actual decoder buffer.

Each input bitstream 204 has an associated decoder buffer model and the selection controller 305 monitors the occupancy of each them. The selection controller 305 is responsible for selecting from which one of the input bitstreams data is to be next included in the output multiplexed bitstream 310 by the multiplexer 306. The selection controller decides based on whether a packet is available in an input buffer and the occupancy of each decoder buffer model. This ensures that decoder buffer overflow or underflow is minimised.

Figure 6 is a flow diagram showing the steps involved in the selection procedure. The first step 500 is to initialise each of the decoder buffer models.

The decoder buffer models then start to monitor the occupancy of a decoder buffer as data is input to the system. The decoder buffer model with the lowest occupancy is then selected by step 501. A packet from the input buffer corresponding to that decoder buffer model is then removed from the input buffer and is included in the output multiplexed bitstream. The size of the packet is then added to the respective decoder buffer model. The timestamps are checked by step 503 and any pictures which would have been removed from the decoder buffer are also removed from the decoder buffer model in step 504.

The method of the present invention therefore provides a simple and effective way of improving the selection of bitstreams for inclusion in a multiplexed bitstream; thereby preventing or significantly reducing overflow or underflow in a decoder buffer. The present invention is best suited to situations where there are a high number of input bitstreams with low data rates. At high data rates the entire decoder buffer is needed to store intra-coded frames and statistical multiplexing can only be achieved by appropriate quantisation of the encoded video streams (e. g. in a closed loop system). As the data rate drops, the size of intra-coded frames is reduced so that more and more of the rate buffer can be used for the purpose of multiplexing.

The present invention is of particular advantage in the field of re-multiplexing.

Figure 5 is a diagram showing how a demultiplexer can be used to provide input bitstreams to the system of Figure 3. A multiplexed bitstream 700 containing many individual bitstreams is filtered by a number of packet identifier (PID) filters 702. Each PID filter extracts a single bitstream 204 which may then be used as an input, for example, to the system of Figure 3.

Each of the extracted bitstreams may then be re-multiplexed using the method and apparatus of the present invention.

The present invention can also be used to particular advantage in remultiplexing applications such as that shown in the example depicted in Figure 7.

Figure 7 shows two input multiplexed bitstreams 600 and 601, each at a different data rate and both containing a number of individual bitstreams. A bitstream splitter 602 is used to extract a number of those bitstreams into a single multiplexed bitstream as illustrated at 603 and 604. These two bitstreams are then re-multiplexed together at a multiplexer 605, to give an output multiplexed bitstream 606.

Re-multiplexing using the present invention is particularly advantageous, especially where a broadcaster or cable television head-end is receiving a multiplexed bitstream containing many channels, and wishes to re-transmit only a portion of the received channels with the possible addition of other local channels. Conventionally the broadcaster would have needed to receive and decode each of the received channels and re-code and re-multiplex the channels required for transmission. Using the present invention, decoding and re-coding is not required. This reduces the cost of such a system, reduces the latency through such a system and does not result in unnecessary degradation of picture quality.

This is an important application of the present invention and it provides a simple but effective solution to the problem of re-multiplexing bitstreams, especially where the individual bitstreams are at different bitrates, without having to decode and subsequently re-encode each bitstream so that all bitstreams are at the same bitrate.

This arrangement would not be possible under normal circumstances (in an open loop system) since the streams being input to the multiplexer are at different bitrates. Re-multiplexing in this situation would be best suited to a closed loop system, which would involve decoding and subsequent re-coding of the broadcast bitstream.

The present invention may, however, be used in this situation to allow the remultiplexing of bitstreams at different bitrates. As described above, by monitoring the occupancy of a decoder buffer model, the input bitstream from data is to be taken to be next included in the multiplexed output bitstream is selected so that the decoder buffer in a decoder will neither overflow or underflow.

The present invention is of particular use in open-loops systems. Open-loop systems are those which have no feedback path to control the video encoding rate. In such systems, video packets can only be shuffled around and rearranged in order to cope with the video bitrate, but no mechanisms exist for changing the video bitrate.

The present invention can also be of use in closed-loop statistical multiplexing systems, where the performance of statistical multiplexing can be improved through better management of decoder buffers, using the techniques outlined above with reference to the accompanying drawings.

The above described embodiments of the present invention, illustrate the method and apparatus with reference to use in a digital broadcasting environment. It will be appreciated by those skilled in the art that the above described techniques are not limited to digital broadcasting, and may be used in many other applications, including the transmission of audio, data and other bitstreams in a multiplexed bitstream. References to pictures or frames can also refer to other convenient units of data.

Claims

CLAIMS 1. A method of transmitting a plurality of individual bitstreams intended to be received by respective receiver buffers, the method comprising the steps of: presenting the bitstreams to a multiplexer ; combining the presented bitstreams by means of the multiplexer to form a multiplex signal ; and transmitting the multiplex signal; wherein the step of combining the bitstreams by means of the multiplexer inclues releasing contributions from each bitstream into the multiplexer; computing the extent to which the released contributions will occupy the intended receiver buffers; and employing the computations so made to select the order in which the bitstreams make their contributions to the multiplex signal.
2. The method of claims 1 wherein the step of employing the computations further comprises employing the contributions to substantially prevent the receiver buffers from underflowing.
3. The method of claims 1 or 2, wherein the step of employing the computations further comprises employing the contributions to substantially prevent the receiver buffers from overflowing.
4. The method of claims 1 to 3, wherein the step of presenting the bitstreams to the multiplexer further comprises presenting the bitstreams at substantially different data rates.
5. The method of claims 1 to 4, wherein the step of presenting the bitstreams to the multiplexer further comprises presenting the bitstreams at substantially the same data rates.
6. The method of claims 1 to 5 further comprising extracting the individual bitstreams by from at least one previously multiplexed signal.
7. Apparatus for transmitting a plurality of individual bitstreams intended to be received by respective receiver buffers, the apparatus comprising: a multiplexer to receive and combine the bitstreams to form a multiplex signal by taking contributions from each bitstream; a computer to compute the extent to which the contributions from the bitstreams will occupy the intended receiver buffers; a controller to select, by reference to the contributions made by the computer, the order in which the bitstreams contribute to the multiplexer ; and a transmission channel to transmit the multiplex signal.
8. The apparatus of claim 7, wherein the controller is further adapted to select the order of bitstreams to substantially prevent the receiver buffers from underflowing.
9. The apparatus of claims 7 or 8, wherein the controller is further adapted to select the order of bitstreams to substantially prevent the receiver buffers from overflowing.
10. The apparatus of claims 7,8 or 9, wherein the multiplexer is adapted to receive the bitstreams at substantially different data rates.
11. The apparatus of claims 7 to 10, wherein the multiplexer is adapted to receive the bitstreams at substantially the same data rates.
12. The apparatus of claims 7 to 11, further comprising a de-multiplexer to extract the individual bitstreams from at least one previously multiplexed signal.