CA2296375A1

CA2296375A1 - Method and apparatus for video compression and transmission with reduced delay

Info

Publication number: CA2296375A1
Application number: CA002296375A
Authority: CA
Inventors: Yendo Hu
Original assignee: Individual
Current assignee: Tiernan Communications Inc
Priority date: 1997-07-18
Filing date: 1998-07-16
Publication date: 1999-01-28
Also published as: AU8489798A; WO1999004575A1; EP0995318A1

Abstract

An apparatus for video compression includes a video compressor and a buffer.
The video compressor compresses an input stream of video data frames at a compression rate to provide compressed video data at an average compressor output rate. The buffer receives the compressed video data from the video compressor to provide a compressed video data stream. The compressed video data stream is transmitted from the buffer onto a transmission channel at a channel rate greater than the average compressor output rate such that utilization of the transmission channel is less than full. Reducing the channel utilization provides a corresponding reduction in buffer delay.

Description

METHOD AND APPARATUS FOR VIDEO COMPRESSION AND TRANSMISSION WTTH REDUCED DELAY
BACKGROUND OF THE INVENTION
Video teleconferencing systems significantly increase the effectiveness of personal communication over existing voice-based systems. Often, video sequences carry information that would otherwise be impossible to describe in words. For example, the visual impression a person obtains from an abstract modern art painting ill a gallery cannot be achieved by simply describing it in words. Two key enabling technologies that make video teleconferencing systems possible are video compression and real time transmission.
Video compression technology reduces the channel bandwidth needed to transmit sequences of video information. For example, the emerging MPEG2 standard reduces the channel bandwidth requirement by a hundred fold without a significant degradation in video quality. The video compression technology reduces the channel bandwidth requirements to a level that is also realistic for transport by the transmission networks that exist today.

Real-time transmission allows for real time, interactive communication. In order for people to comfortably interact, it is generally understood that the delay between source capture and destination display must not exceed the typical human response time, which is about 300 ms.
A conventional video teleconferencing system as shown in FIG. 1 includes four major elements: a video compression unit 10 and a video buffer 12 at a source site 20, a transmission channel 14 connecting the source site 20 to a destination site 22, and a video decompression unit 16 at the destination site 22. The video compression unit 10 captures and compresses input video information 8 into a stream 24 of video bits following some type of compression standard such as MPEG2 or I-L261. The video buffer 12 receives non-constant or bursty data stream 24 from the video compression unit 10. The data stream is bursty due to the amounts of temporal and spatial encoding used in compressing the video information. The video buffer 12 smooths out the data stream 24 so that a constant rate data stream 26 can be fed into the transmission channel 14. The channel 14 delivers the data stream from the source site 20 to the destination site 22. The decompression unit 16 decompresses the received data stream 28 to recover the original video information for display. All four elements contribute to an overall system delay. If the system delay increases beyond the human response time, it can significantly reduce the effectiveness of the video teleconferencing system. It is thus important to minimize the delay of each element. The present invention is directed to reducing the delay contributed by the video buffer.
SUMMARY OF THE INVENTION
According to the present invention, an apparatus for video compression includes a video compressor and a buffer. The video compressor compresses an input stream of video data frames at a compression rate to provide compressed video data at an average compressor output rate. The buffer receives the compressed video data from the video compressor to provide a compressed video data stream. The compressed video data stream is transmitted from the buffer onto a transmission channel at a channel rate greater than the average compressor output rate such that utilization of the transmission channel is less than full. Reducing the channel utilization provides a corresponding reduction in buffer delay which improves end-to-end delay to benefit real-time transmission applications.
According to a preferred embodiment, a compression unit includes a video compressor, a buffer and a multiplexer for multiplexing the compressed video data stream with other data streams onto the transmission channel. The compression unit further includes a data gate for gating the compressed video data stream into the multiplexer at a multiplexer input rate when the buffer contains compressed video data, wherein the multiplexer input rate is greater than the average compressor output rate.
According to another aspect of the invention, the compression unit further comprises a time stamp generator coupled to the multiplexer for generating a time stamp for each compressed video data frame. The time stamp is adjusted by a delay offset value to account for delay due to the buffer.
In an embodiment of the compression unit, the video compressor comprises an MPEG2 compliant video compressor and the time stamp generator comprises a presentation time stamp (PTS) generator for generating a PTS value for each compressed video data frame. The PTS value is reduced by a delay offset value to account for delay due to the buffer.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram of a conventional system for real-time transmission of compressed video information.
FIG. 2 is a graphical representation of a mathematical model of a video buffer.
FIG. 3 is a graphical representation of a relationship between maximum buffer delay and channel bandwidth.
FIG. 4 is a block diagram of a real-time transmission system in accordance with the present invention.
FIG. 5 is a schematic block diagram of the MPEG2 compressor in FIG. 4.

FIG. 6 is a schematic block diagram of the PTS generator in FIG. 5.
The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular description of preferred embodiments of the invention, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views.
DETAILED DESCRIPTION OF THE INVENTION
A model is first presented to understand how data buffering in the conventional system of FIG. I contributes to the overall system delay. As noted in the background, the video compression unit 10 encodes input data into a format (e.g., MPEG) that is suitable for channel transmission. The buffer 12 serves as a temporary storage house for the data coming out of the compression unit 10. This buffer 12 is necessary only because the output of the compression unit 10 is bursty. The channel 14 then takes data out of the buffer 12 at a constant rate and delivers it to the video decompression unit 16.
The following presents output equations for each system element with reference to FIG.
2.
The video compression unit 10 (FIG. 1 ) outputs data periodically with a frame period P. The frame period P is equal to 33ms for NTSC formatted video and SOms for PAL formatted video. The compressed data A of each frame is sent out within the period P. The total data size of each compressed frame is defined by the function f(i), where i is the frame count (i is an integer). Assuming that the video compression unit 10 can only output compressed data at a fixed rate Rcp, or zero, the compression unit output data rate, esor(t), which corresponds to data stream 24 (FIG. 1 ), is then:
esor(t) = Rcp * POS( f{INT(t/P)) - (t - P*INT(t/P))*Rcp) where:
INT(#) gives the integer of #
POS(#) returns a 1 if # is positive and 0 otherwise t is time in seconds The buffer 12 (FIG. 1 ) receives data stream 24 at a rate defined by the function esor(t). The buffer 12 outputs the data stream 26 defined by the function buffout(t).
The buffer 12 outputs data stream 26 at a constant rate of Rch only when there is data within the buffer. The fullness of the buffer is given by the function bufflevel(t). FIG. 2 graphically indicates characteristics of the buffout(t) and bufflevel(t) functions.
As shown in FIG. 2, the buffer fullness bufflevel(t) is the difference between the input rate esor(t) (corresponding to stream 24 in FIG. 1 ) and the output rate buffout(t) (corresponding to stream 26 in FIG. 1):
bufflevel(t) = f Y-0,y~' {esor(y) - buffout(y)}
The function buffout(t) is not an easy function to model. For the purpose of this analysis, a new variable, Tpz(i), is defined as the latest time instant in the past when buffout(t) was zero starting from the current frame i. It can be seen in FIG.
2 that the dependency of bufflevel(t) on esor(t) stops at Tpz. Thus bufflevel(t) is defined as:
bufflevel(t) _ ~ Y=TPz(iNT(t!P)),y<' {esor(y) - Rch}
The channel 14 (FIG. 1) can deliver data at any rate up to a maximum level.
This maximum level is defined as Rch. The function buffout(t) defined above satisfies this constraint. In this analysis, the transmission delay across the channel is not considered to be significant and can be ignored.
The video decompression unit 16 (FIG. 1) must receive a full video frame i before it can output a decoded image for display. For this analysis, the time required by the receiver to decode the image is assumed to be zero.
Referring again to FIG. 2, a delay D(i) is the time it takes to present the 1~" frame when the i'" frame compressed data is available to the buffer 12 (FIG. 1). In specific terms, the delay is the time it takes to deliver the last bit of each frame to the destination site 22 (FIG. 1 ) starting from the first bit going into the buffer 12. For example, this delay can be seen graphically in FIG. 2 for value D(4). At time is the first bit of frame i=4 enters the buffer 12 and at time tE the last bit of frame i=4 enters the buffer 12. This last bit from frame i=4 does not leave the buffer 12 until time tF, yielding the delay indicated as D(4) in FIG. 2.
Given tf(i) as the time when the first bit of the i'" frame is introduced to the buffer, then the delay is given below:
D(i) _ (bufflevel(tf(i)) + f(i)) / Rch - ~ y=Tpz(i), y<'f(i) {esor(y)} /Rch - (tf(i)- Tpz(i)) + f(i)/Rch The maximum delay is then:
Dmax = max; {D(i)}
The maximum delay Dmax is inversely proportional to the channel bandwidth Rch, as shown graphically in FIG. 3. In this graph, Imax is the argument of max{D(i)}.
The functions esor, tf, and Tpz are assumed to stay unchanged over the Rch range. The plot shows that by increasing the channel bandwidth, Rch, the overall delay decreases.
In most video transmission applications, the maximum channel bandwidth allowable (Rchmax) is constrained by the physical structure of the channel 14 (FIG. 1 ).
For systems that require high quality video, all bits transmitted through the channel hold relevant video information and the system is operated such that the channel is 100%
utilized. Utilization is defined here as the ratio of the bandwidth required to the available channel bandwidth. To achieve 100% utilization of the channel bandwidth, the conventional compression unit 10 (FIG. 1) is configured to generate an overall average bit rate Avbr that is equal to Rchmax. However, the 100% utilization of this compression configuration does not take into consideration the overall delay introduced by the buffer unit 12 (FIG. 1). Referring to FIG. 3, the curve D,(Imax) illustrates a _'7_ system wherein Rchmax crosses the curve where the slope is steep, which translates into a significant delay DA introduced by the buffer unit.
The present invention provides a method and apparatus wherein the bandwidth utilization is reduced to provide a corresponding reduction in the buffer delay. As described further herein, the bandwidth utilization is reduced by effectively reducing the average bit rate Avbr of the compression unit relative to the channel rate Rchmax.
Referring again to FIG. 3, the curve DZ(Imax) illustrates a system for which the utilization value has been reduced. It can be seen that for the same value of Rchmax, the corresponding delay Do is reduced. The relationship between the channel utilization and the reduction in buffer delay depends on the function esor(t), which is compression algorithm dependent.
The present invention provides flexibility to balance tradeoffs between video quality and video delay. In video teleconferencing applications where both the video quality and the video delay are important, this flexibility becomes very valuable.
Referring now to FIG. 4, an exemplary embodiment of a real-time video transmission system according to the present invention is shown. The system includes the following elements: a composite camera 100, an MPEG2 compressor 104, a modulator 108, a satellite repeater 110, a demodulator 112, an MPEG2 video decompressor 116 and a composite video display 120.
The composite camera 100 captures motion information into an analog composite format following a conventional video standard such as NTSC or PAL.
The MPEG2 compressor 104 receives this analog signal and compresses it following the MPEG2 standard. One of the key parameters that can be defined for the MPEG2 compressor is the average compressed output bit rate. For broadcast quality video, a value of 8 Mbps for this parameter is typical. The compressor 104 generates a smooth output stream into the modulator 108. The modulator 108 modulates the digital bit stream from the compressor 104 into a suitable radio frequency signal for transmission via the satellite 110.

_g_ The demodulator 112 demodulates an RF signal received from the satellite 110 to recover the compressed digital bit stream. The MPEG2 decompressor 116 decodes the compressed digital bit stream following the MPEG2 standard and outputs an analog composite signal following the NTSC standard. The composite video display 120 then S displays the images represented by the composite signal.
Referring now to FIG. 5, a schematic block diagram of the MPEG2 compressor 104 is shown. A video digitizer 200 converts composite analog video signals (e.g., NTSC or PAL standard) into CCIR-601 digital format. A video preprocessor 204 performs real time filtering on the CCIR-601 digital video signal to increase compression efficiency. A video compressor 214 compresses the preprocessed digital video signal to provide an elementary video stream 232 following the MPEG2 video standard (ISO/IEC 13818-2).
An audio digitizer 202 converts analog audio signals into digital audio signals following the AES/EBU standard (ANSI 4.40 - 1992). An audio compressor 208 compresses the digital audio signals to provide an elementary audio stream 235 following the MPEG2 audio standard (ISO/IEC 13818-3). A data compressor 210 compresses data following a custom-defined compression technique to provide an elementary data stream 237. A table generator 212 generates transport tables following the MPEG2 system section standard (ISOIIEC 13818-1). A multiplexer multiplexes the video, audio and data elementary streams along with the transport tables into a single transport stream 241 following the MPEG2 system section standard (ISO/IEC 13818-1).
A physical layer interface 220 converts the transport stream 241 into an analog signal suitable for transmission over various different physical media (e.g.
RS422, DS3, 6.703, ASI). A data timing manager 226 generates time stamps and system clocks to the various elements to ensure that the transport stream complies with the system section standard (ISO/IEC 13818-1). A system controller 222 initializes, monitors, and periodically updates each of the elements within the compressor 104. A

front panel 224 and a network management interface 225 provide user access to customize the compressor configuration.
The low delay aspect of the present invention is implemented in the exemplary embodiment using a video buffer 216, a compress data gate 228 and a PTS
generator 230. The buffer 216 receives a video data stream from the video compressor on line 232 at a compressed video output clock rate on line 234. The output clock rate is typically set not to exceed 54MHz. This output clock rate corresponds to the instantaneous or peak rate for the function esor(t) defined in the model above (FIG. 2).
The buffer has a buffer depth that is deep enough to handle the maximum variable buffer verifier (VBV) buffer size as defined by MPEG2 and set within the video compressor 214. For MPEG2 4:2:2 profile @ main level, for example, the maximum VBV buffer size is set at 9.1 Mbits.
The data gate 228 gates the data output from the buffer 216 and into the multiplexer 218. The gate 228 provides a multiplexer clock input signal 238 that clocks 1 S data to the multiplexer 218 at a rate of Rg bits per second when there is data in the buffer, and 0 bits per second when there is no data. The gate 228 is implemented by generating a clock pulse at a gate rate of Rg when the buffer is not empty as indicated by the status of a buffer empty flag 237. The gate rate Rg corresponds to the channel rate Rchmax (FIG. 3).
As noted above, it has been found according to the present invention that a reduction in the channel bandwidth utilization provides a corresponding reduction in buffer delay. In an exemplary embodiment that uses a video compressor model available from IBM, it has been empirically determined that an average compressor output rate Avbr for the video compressor 214 that is set at 0.833 * Rg (i.e., Rg/Avbr=1.2) provides a reduction in buffer delay of about I 00ms. Thus, the gate rate Rg is 20% larger than the average compressor output rate Avbr. For other video compressors, corresponding ratios for Rg/Avbr can be empirically determined to provide a desired reduction in buffer delay. There is, of course, a tradeoff between video quality and video delay. That is, in order to reduce the delay, the video quality is reduced since the average compressor output rate Avbr is below the value that corresponds to 100% utilization. In general, setting the ratio Rg/Avbr too high can lead to unacceptable video quality.
In an MPEG2 compliant system, the total video system delay is determined by time stamps that are included at the beginning of each compressed video frame.
More specifically, the MPEG2 system section standard (ISO/IEC 13818-1) specifies a PTS
field within a packetized elementary stream (PES) header at the beginning of the compressed video frame, which determines the time elapsed from the time the uncompressed frame is captured (e.g., in MPEG2 compressor 104, FIG. 4) to the time the uncompressed frame is displayed at the video decompressor output (e.g., decompressor 116, FIG. 4). Referring now to FIG. 6, a PTS generator 230 intercepts the PTS time stamp generated by data timing manager 226 (FIG. 5) and applies an offset value from PTS offset register 2302 to reduce the value of the PTS time stamp.
The adjusted PTS time stamp is entered into the multiplexer 218 (FIG. 5) to be inserted into the PES header. In the exemplary embodiment, an offset value of 100ms has been determined empirically for the IBM model ME31 video compressor that effectively reduces the overall system delay by that amount. It should be noted, however, that for other video compressors, corresponding PTS offset values can be empirically determined to account for the reduction of the delay required by the buffer 216 {FIG. 5).
While this invention has been particularly shown and described with references to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

-11- What is claimed is:

1. Apparatus for video compression comprising:
a video compressor for compressing an input stream of video data frames at a compression rate to provide compressed video data at an average compressor output rate;
a buffer for receiving the compressed video data from the video compressor to provide a compressed video data stream and for transmitting the compressed video data stream onto a transmission channel at a channel rate greater than the average compressor output rate such that utilization of the transmission channel is less than full.

2. The apparatus of Claim 1 further comprising a data gate for gating the compressed video data stream onto the transmission channel at the channel rate when the buffer contains compressed video data.

3. The apparatus of Claim 1 wherein a ratio of the channel rate to the average compressor output rate is selected to provide a desired buffer delay.

4. A compression unit comprising:
a video compressor for compressing an input stream of video data frames at a compression rate to provide compressed video data at an average compressor output rate;
a buffer for receiving the compressed video data from the video compressor to provide a compressed video data stream;
a multiplexer for multiplexing the compressed video data stream with other data streams onto a transmission channel;

a data gate for gating the compressed video data stream into the multiplexer at a multiplexer input rate when the buffer contains compressed video data, wherein the multiplexer input rate is greater than the average compressor output rate and a ratio of multiplexer input rate to the average compressor output rate is selected to provide a desired buffer delay.

5. The compression unit of Claim 4 wherein the ratio is about 1.2.

6. The compression unit of Claim 4 further comprising a time stamp generator coupled to the multiplexer for generating a time stamp for each compressed video data frame, the time stamp adjusted to account for delay due to the buffer.

7. The compression unit of Claim 6 wherein the video compressor comprises an MPEG2 compliant video compressor and wherein the time stamp generator comprises a presentation time stamp (PTS) generator for generating a PTS value for each compressed video data frame, the PTS value reduced by a delay offset value to account for reduction of the delay required by the buffer.

8. A method of video compression comprising the steps of:
compressing an input stream of video data frames at a compression rate to provide compressed video data at an average compressor output rate;
receiving the compressed video data in a buffer to provide a compressed video data stream; and transmitting the compressed video data stream from the buffer onto a transmission channel at a channel rate greater than the average compressor output rate such that utilization of the transmission channel is less than full.

9. The method of Claim 8 further comprising providing a data gate for gating the compressed video data stream onto the transmission channel at the channel rate when the buffer contains compressed video data.

10. The method of Claim 8 further comprising selecting a ratio of the average compressor output rate to the channel rate to provide a desired buffer delay.

11. A method of compression comprising the steps of:
compressing an input stream of video data frames at a compression rate to provide compressed video data at an average compressor output rate;
receiving the compressed video data in a buffer to provide a compressed video data stream;
gating the compressed video data stream into a multiplexer at a multiplexer input rate when the buffer contains compressed video data, wherein the multiplexer input rate is greater than the average compressor output rate;
and multiplexing the compressed video data stream with other data streams onto a transmission channel.

12. The method of Claim 11 further comprising generating a time stamp for each compressed video data frame and adjusting the time stamp by a delay offset value to account for delay due to the buffer.

13. The method of Claim 12 wherein the video compression is provided by an MPEG2 compliant video compressor and wherein the time stamp comprises a PTS value for each compressed video data frame and including reducing the PTS value by a delay offset value to account for reduction of delay required by the buffer.