GB2357674A

GB2357674A - Compressed bitstreams

Info

Publication number: GB2357674A
Application number: GB9925331A
Authority: GB
Inventors: Fairbairn Bruce Devlin; Violet Snell
Original assignee: Snell and Wilcox Ltd
Current assignee: Snell Advanced Media Ltd
Priority date: 1999-10-26
Filing date: 1999-10-26
Publication date: 2001-06-27
Anticipated expiration: 2019-10-26
Also published as: AU1044001A; GB2357674B; GB9925331D0; WO2001031928A1

Abstract

A multiplexed MPEG-2 stream having video and audio elementary streams is prepared for continuous looping by: estimating the differential buffer delay between the two streams; removing from the end of the audio stream a stream section approximately equal to said differential delay; and inserting that stream section at the beginning of the audio stream.

Description

2357674 COMPRESSED BITSTREAMS This invention relates to compressed

bit-streams and in the most important example to MPEG-2 bit-streams.

It is an important requirement for test and other purposes to produce MPEG-2 bit-streams which can be played in a loop, without discontinuities that would create difficulties for a downstream decoder. This problem is addressed by the invention of EP-A-0 893 030, to which reference is directed.

In the production of a looped transport stream, a further problem arises from the need to ensure synchronism between decoded video and audio, or indeed between other elementary streams having a time critical relationship.

It is important to realise that the video and audio signals are delayed as they pass through their respective encoders. There is also a corresponding delay in their decoders. The delay is caused by the compressed data bytes waiting in the encode 1 decode buffers so that the overall end-to-end delay is constant. This will be true regardless of the nature of the encoding.

Unfortunately, in a typical system, the delays in the video chain are different from the delays in the audio chain, because the buffers are of very different sizes as is shown in the diagram below.

Under normal circumstances, extra delay needs to be added to the audio so that the final decoded audio-video synchronisation is maintained. If the audio decoder buffer were big enough, it would be a simple task to instruct the decoder to "hold on" to the compressed data bytes for a longer time than usual and hence obtain synchronisation. Unfortunately, the audio buffer is not big enough to allow this to happen.

This situation becomes especially difficult when trying to create a seamlessly looped MPEG transport stream with perfectly synchronised audio and video components.

It is therefore an object of one form of the present invention to provide method and apparatus for looping a transport stream, which ensures synchronism between decoded elementary streams.

Accordingly, the present invention consists, in one aspect, in a method of producing a multiplexed stream comprising at least two elementary streams which have been compression coded in respective encoders which each have an encoder buffer, the elementary streams requiring to be synchronised when decoded, comprising the steps of determining the differential buffer delay between the two streams; cutting from one end of one elementary stream a stream section approximately equal to said differential delay; pasting said 5 stream section at the other end of said one elementary stream.

The invention will now be described by way of example with reference to the accompanying drawings, in which:

Figure 1 is a block diagram illustrating the different buffer sizes of typical video and audio encoderldecoder pairs (codecs); Figure 2 is a block diagram illustrating the use of a compensating delay at an audio encoder; Figure 3 is a block diagram illustrating the use of a compensating delay at an audio decoder; Figure 4 is a diagram illustrating the desired timing relationship between video and audio after decoding of the continuous stream which results from looping of a transport stream segment; Figure 5 is a diagram similar to Figure 4, illustrating the desired timing relationship between video and audio before decoding; Figure 6 is a diagram illustrating the consequential timing requirement on the video and audio streams of the transport stream segment which is to be looped; and Figure 7 is a diagram similar to Figure 6, illustrating one process according to the present invention.

The example will be taken of a single programme MPEG-2 transport stream, with a requirement for precise synchronism between video and audio.

As mentioned above, video and audio signals are typically delayed by different amounts as they pass through their respective encoders. This is illustrated in Figure 1, which also shows the corresponding delay in the respective decoders.

Under normal circumstances, extra delay needs to be added to the audio at the encoder, so that the final decoded audio-video synchronisation is maintained. This is shown in Figure 2. If the audio decoder buffer were big enough, it would be a simple task to instruct the decoder to "hold on" to the compressed data bytes for a longer time than usual and hence obtain synchronisation. Figure 3 illustrates this approach. Unfortunately, the audio buffer is not big enough to allow this to happen.

Figure 4 shows the desired relationship of the audio and video after decoding.:rhe loops start co-timed; end co-timed and remain co-timed every time the loops are decoded. If we now look at the timing relationships in the MPEG stream just prior to decoding, we need to shift the streams by the same amount as the delay in the respective decoder.

Figure 5 now shows the desired relationship of the audio and video prior to decoding. When the decoder delays are applied to the loops above, the desired synchronisation will be achieved. Notice that there is a differential delay 8 between the components, which under normal circumstances is greater than the delay through the audio decoder buffer and cannot be corrected by simple numerical manipulation of the compressed audio parameters.

In essence we cannot make a simple AV loop where each component starts at the same time T. When the components are multiplexed together for synchronisation at the output of the decoder, there will always be too great a delay in the audio (or other time critical component).

The essential feature of the invention is described in Figure 7 in which the problem is explained. If we can find a method for defining a value of the delay 3 then we can create the following simple multiplex:

the video starts at time T and has duration D the audio (or other co-timed stream) starts at time T + 8 and has duration D In effect we are pre-compensating for the differential delays in the decoders by cutting a portion of the audio from the end of the stream and pasting it onto the front of the stream. Provided that the value that is chosen for 8 is sufficiently close to the differential audio and video coder delays, the decoder buffer should be able to accommodate the residual offset between video and audio streams.

The problem is exacerbated by the properties of the associated component to be synchronised:

Audio is delivered in fixed size frames. This means that the value of 8 must be rounded so that the number of bytes chopped is an integer number of audio frames. Other content may have qualities which mean that large segments may have to be chopped. Subtitles for example may need to be split at whole page description boundaries, depending on the presentation model being used.

One starting point for 8 is the vbv-delay of the first 1 frame of the video image. In the case of video and audio streams, there will then be determined that integral number of audio frames which approximates most closely to this vbv-delay. Other, similar values could be used. Indeed, if the intent is to produce a stream which is looped continuously to exercise and test decoders, there might deliberately be chosen a number of audio frames which is not the closest approximation to the vbv-delay. This would exercise more strenuously the buffer management of the decoder. Thus, calculations could be performed to define a range of 3 values ranging from decoder buffer management terms from: "difficult to avoid underflow" through "easy" to "difficult to avoid overflow".

An alternative to the preparation of a multiplexed transport stream that can be looped, is the processing of an existing transport stream (if it conforms with certain constraints) to make it capable of being looped. The processes required to make each elementary stream "loopable" are discussed in the earlier mentioned reference. In this case, the section of - say the audio stream which is to be cut from the end and pasted at the beginning of the transport stream will effectively overwrite the beginning of the audio stream.

Similar considerations will apply to the concatenating, rather than looping of streams. If, for example, three existing transport streams are to be concatenated and then looped, the end section of the audio stream of the third transport stream will be pasted at the start of the audio stream of the first transport stream. The "breaks" in the video and the audio between the first and second transport streams (and between the second and third) will be offset from each other by a corresponding amount.

Claims

1. A method of producing a multiplexed stream comprising at least two elementary streams which have been compression coded in respective encoders which each have an encoder buffer, the elementary streams requiring to be synchronised when decoded, comprising the steps of determining the differential buffer delay between the two streams; cutting from one end of one elementary stream a stream section approximately equal to said differential delay; pasting said stream section at the other end of said one elementary stream.

2. A method according to Claim 1, wherein the two elementary streams are respectively video and audio.