WO2014102337A1

WO2014102337A1 - A method and system for managing a queue of video frames

Info

Publication number: WO2014102337A1
Application number: PCT/EP2013/078066
Authority: WO
Inventors: Mirko GÜNTHER; Mirek CHALINSKI
Original assignee: Barco Nv; Barco Control Rooms Gmbh
Priority date: 2012-12-27
Filing date: 2013-12-27
Publication date: 2014-07-03
Also published as: BE1022040B1; GB201223407D0; GB2509313A

Abstract

The invention relates to a method for managing the queue of video frames between a decoder and the renderer, based on determining the current fill level of the queue and adjusting the delivery to the renderer based on at least the current fill level of the queue; and combination of said method with frame estimation and/or combination on renderer control.

Description

A METHOD AND SYSTEM FOR MANAGING A QUEUE OF VIDEO FRAMES Field of the invention

The invention relates to methods, systems and software for managing one or more queues of video frames.

Background

In control room and monitoring applications compressed video streams, from encoders of different vendors, are delivered as network packets across non-trivial network architectures. Expectation is that these video streams will be displayed in an evenly distributed manner over time so that the viewer perceives fluid constant movements. Several network video standards provide so called presentation timestamps (PTS) for each frame. Also frame rates may be encoded in the video streams. With the help of frame rate and the PTS information the video frame distribution over time can be reconstructed. Unfortunately in reality this is not always feasible because:

(1) Not all encoder vendors support correct PTS. This includes missing PTSs or even wrong calculated PTSs.

(2) Not all encoders support valid frame rate information in their streams.

(3) Delays may happen on network switches, transmitters or receivers which introduce network jitter to the network packets of the video frames.

(4) Due to changing bandwidth requirements on CPUs during the encoding process it may also happen that jitter will be introduced to network stream.

Usually a queue is used to eliminate the jitter of the incoming stream. The frames are stored in the queue as they are received from the decoder. The incoming frames are stored in this queue during, for example, a predetermined amount of time. The frames can then be de-queued by the renderer at a constant output rate. When the average reception rate of the frames in the queue is equal to the constant output rate, then the queue allows de-queuing the frames at a constant output rate although the frames were received at an irregular rate. This eliminates the irregular arrival times, but introduces an additional delay and therefore increases the end-to-end latency. If the queue delay is set to a value large enough to cover the worst case jitter, then a regular delivery of frames to the renderer would be guaranteed. However in practical applications this would result in a long queue delay and in an unacceptable end-to-end latency.

In contrast to the above described fixed jitter queue management, several adaptive jitter queue management schemes were proposed. In 6,684,273 B2 a method is disclosed to adapt the queue size at the measured delay of the received packets. In 7,983,309 B2 a buffering time for one or more frames received by a frame buffer is determined based on a specific buffering time associated with a specific frame and on information representative of specific amount of data stored in the frame buffer.

US8015306 describes synchronizing streaming media with multiple output devices (plural), such streaming media being served by one or more media servers. A master device is disclosed, taking action to realize such synchronization. This action is requesting from the media server more data to place in the buffer. US 8015306 in addition describes how each (slave) output device in itself performs actions to maintain buffer fill levels but limits this action to play back corrections (speeding up or slowing down) and specifies that this is to keep the buffer filling between a low and high boundary. US8015306 solves a problem of synchronizing and the master device therein might try to maintain a nominal buffer level, this is achieved by input control to the buffer. The slave device actions do not aim at nominal level control but on boundary control. US8015306 provide also a notion of conditional time tagging.

US5812699 mentions maintaining a buffer level, however in the context of compression, so delivering to a coder. US2012/0106651 describes use of a notion of time, more in particular transfer time, and uses the control to mode of working of a decoder. While Figure 6 thereof does mention control actions of the renderer buffer between the decoder and the renderer (in addition to control actions of the decoder buffer) but does this by changing the operating mode of the decoder (stopping it and only resuming if a lower level is achieved).

US 5677969 discusses actions to consider decoder buffer level (hence before the decoder) but describes only actions at rate controller and encoder.

US 2008107031 discussed control of transmission rates and filtering out video packet in case a packet queue is full in a context of dynamically changing links.

Aim of the invention

The aim of the invention is to presents methods, systems and software which requires neither presentation time stamps nor the frame rate information encoded in the video frames while enabling fluid and constant presentation of arbitrary video streams

Summary of the invention

The various aspects of the invention are described in the claims 1 to 49.

In a first aspect of the invention a method for managing a queue of video frames, typically between a decoder and a renderer (display unit for such decoded video frames) is disclosed wherein the method comprising determining the current fill level of the queue (e.g. counting the amount of video frame in the queue); and adjusting or adapting the delivery to the renderer by comparing the current fill level of the queue with a first predetermined fill level to hold the current fill level P as close as possible to this first predetermined fill level.

In an embodiment thereof the step of adjusting the delivery is operable only when the current fill level is larger than a second predetermined fill level, more in particular in such case the delivery of the video frames is delayed, more preferably with a delay depending on the difference (e.g. proportional) between the current fill level and the second predetermined fill level.

In an embodiment thereof in case a reliable estimate of the video frame rate can be computed, time information is tagged to a video frame before the video frame is delivered to the renderer, more in particular such reliable estimate is deemed available when a predefined minimum amount of video frames has passed through the queue and/or when the video frame rate (as estimated) exceeds a predefined minimum frame rate, more preferably the estimate of the video frame rate is determined from the accumulated time spend in the queue of a predetermined amount of video frames (preferably equal to the predefined minimum amount).

The method of the first aspect of the invention and its embodiments which can be combined is part of handling video frames by a renderer, whereby the renderer drops the video frame if delivered when the renderer's video frame memory is full and/or those handling includes tagging a delivered video frame with a time indication; and if the renderer with its display rate cannot display the video frame such that the time indication tagged to the video frame falls within a predetermined time window, the renderer drops the video frame; and/or the renderer displays the video frame in accordance with the tagged time indication.

In a second aspect of the invention a controller suited for managing the queue of a system, comprising a decoder and a queue for temporarily storing a plurality of video frames and a renderer, wherein the controller comprises means (hardware or a combination of hardware and software) for executing any of the methods presented.

In a third aspect of the invention a system is provided, the system comprising a decoder, a queue for temporarily storing a plurality of video frames and a renderer, further comprising a controller for managing the queue, the controller comprising means for executing any of the methods presented.

In a fourth aspect of the invention a computer program product and related machine readable signal storage medium, comprising code segments that when executed on a suitable processing engine implement those steps in any of the methods presented.

In an aspect the present invention provides methods and systems for managing a queue of video frames between a specific device (e.g. a decoder) of specific media (e.g. video) and a specific single output device (e.g. a renderer). A step can be included of adjusting the delivery to the specific single output device, e.g. to the renderer, e.g. by comparing at least the current fill level of the queue with a first predetermined fill level and not to synchronize but to hold the current fill level P as close as possible to this first predetermined fill level. A step can be provided of holding the current fill level P as close as possible to this first predetermined fill level. Optionally the specific single output device, e.g. the renderer is not allowed to take further action. Video frames may be dropped when the specific single output device, e.g. Tenderer's video frame memory is full or carrying out more elaborate time based video frame dropping. A nominal buffer level can be obtained by control of the delivery from the queue. Time tagging and/or an indication of the reliable video frame rate estimate, e.g. using the time tagging for conditional using or for determining the time indication or how to compute such estimate or tag based dropping or displaying is described. Accordingly queue buffer fill level management of video data obtained from a decoder is described. Steering preferably acts on the video compression processing. A comparison with a threshold may be made, where this threshold is made dynamically, to achieve an upper and lower level control (as shown in Figure 5). Hence delivery to the renderer is adjusted and a buffer level maintained. Hence control of a queue is described after the decoder by acting on the delivery towards the renderer. A fixed link can be provided between a decoder and a renderer, managing the queue therein between but acting on the delivery from the queue to the renderer, while the buffer level is maintained rather than invoking a mechanism when the queue is full.

Brief description of the drawings

Figure 1: Example queue for decoded frames with Q_max = 5 , Q_min = 3 and P = 4

Figure 2 shows a flow chart of the method in accordance with an embodiment of the invention.

Figure 3 shows a schematic of the invented systems and subsystems. Figure 4 shows a generic flowchart of the invented methods. Detailed description of the invention

The invention provides for methods, systems and software for queue management, enabling amongst other evenly display in time of the video frames. A method is disclosed where the jitter of the incoming stream is eliminated for a minimum increase of the latency, with a constant queue size. The queue or frame buffer has a closed loop control system to keep the fill level of the queue constant. The average frame rate of the decoder output stream can be different from the display frame rate. Note that the invented methods can be implemented in a dedicated ASIC or be executed as software on a general purpose unit or a graphic processing unit or a combination of those.

Contrary to US8015306 the present invention provides managing the queue of video frames between a specific device (being a decoder) of specific media (video) and specific single output device (being a renderer). The present invention provides adjusting the delivery to the renderer instead and does that by comparing at least the current fill level of the queue with a first predetermined fill level and not to synchronize but to hold the current fill level P as close as possible to this first predetermined fill level. The present invention provides holding the current fill level P as close as possible to this first predetermined fill level. The invention does also allow the renderer to take further action and describes dropping video frames when the Tenderer's video frame memory is full or carrying out more elaborated time based video frame dropping. The invention provides to maintain a nominal buffer level by control of the delivery from the queue. Besides discussing time tagging the invention indicates the reliable video frame rate estimate based approaches using the time tagging for conditional using or for determining the time indication or how to compute such estimate or tag based dropping or displaying.

Contrary to US5812699 the invention provides queue buffer fill level management of video data obtained from a decoder. The steering is as a consequence entirely different as it acts on the video compression processing instead and also the indicated context or goals are entirely different. Moreover while a comparison with a threshold is made, this threshold is done dynamically, to achieve again an upper and lower level control actually, as shown in Figure 5.

Contrary to US2012/0106651 in the invention it is the delivery to the renderer which is adjusted and where the goal is to maintain a buffer level. The notion of time and use thereof is also entirely different. Contrary to US 5677969 the invention discusses queue control of a queue after the decoder by acting on the delivery towards the renderer.

Contrary to US 2008107031 the invention focusses on a fixed link between a decoder and a renderer, managing the queue therein between but acting on the delivery from the queue to the renderer, while the buffer level is maintained rather than invoking a mechanism when the queue is full.

Embodiment

In order to overcome missing or incorrect PTS (presentation time stamp) and frame rate information and obtain a fluid and even presentation of a video stream, we hereby apply a frame buffer or queue in a sort of closed loop control system as a solution to the above described problem. In the first part of the description it is assumed that the average frame rate of the decoder output is equal to the display frame rate although the invention is not limited thereto.

Frame rate calculation

The average frame rate of the decoder's output stream is denoted f_s. Since no in-band frame rate can be used, we calculate the average frame rate f_s by deriving it from the accumulated period of time T_{acc n} of n previously decoded frames. Preferably this will be calculated as a moving average. The average frame rate will be updated at the end of

— i

the decoding of each frame. The average frame duration T_s =— will be updated

fs

according to f_s. The average frame rate is calculated over time intervals of for instance the least 60 frames which were received by the queue.

n

for n≥ 60

acc.n As long as n < 60 frames, the frame rate will not be used to generate new frame rate and time information. If a frame rate below 10 frames per second is calculated, the frame rate is not used to generate new frame rate and time information too, because such a small frame rate indicates in most cases either an instable stream or a signal lost/found situation. If the stream has a frame rate below a certain value, for instance 10 fps, because it's a thumbnail stream or similar, the practical experience showed, it's better to let the renderer to do its best to render such a stream.

Queue for decoded frames

Decoded frames will be queued in the frame buffer. The maximum queue size is denoted Q_max and the queue's minimum fill level is denoted Q_mi_n. The actual fill level is denoted P. In general:

If the queue is full (P = Q_max), new frames to be queued shall drop the oldest frames in the queue.

The choice of Q_max and Q_mi_n is important. The memory size that is needed to store the frames in the queue is proportional with {Qmax + 1)· The queue delay is proportional with (Qmin + l)because {Qmin + 1) ^iS the number of frames that must be kept in the queue. It is therefore important to find a minimum value of Q_max and Q_mi_n that in practical situations will not result in queue overrun or under-run.

Closed loop to keep the queue fill level equal to Q,,,,,,

The current fill level of the queue is used as the closed loop control value. The task of the closed loop is to hold the current fill level P as close as possible to Q_mi_n. After starting the stream, the queue does not start to deliver the frames to the renderer as long as the fill level P < Q_min - Once the minimum fill level is reached, the queue will try to hold P = Qmin by adjusting the delivery rate to the renderer.

There are two transient situations:

(1) Initially the closed loop shall wait with delivery of frames to the renderer until P = Qmin is reached.

(2) If P = Qmin is reached or P > Q_mi_n, but no f_s can yet be calculated, the next frames shall be delivered immediately. Neither time information nor any duration will be added to the frame's information; the renderer tries to work on the frame immediately. If the Tenderer's internal structures are full, the frame will be dropped.

There are two regime situations:

(1) If P≥ Qmin and f_s can be derived, the next frame shall be delivered immediately. It will be tagged with the current time t_now and its duration T_s, derived from f_s . Usually the renderer is driven by the display frame rate V-Sync of the graphics device. If the renderer can't handle the frame before t_now + T_s expired, then it will drop the frame.

(2) If the P < Qmin ^tne delivery of frames out of the queue will be delayed during a time period t_steep . The delay time depends on the difference between the Q_min and P : tsieep = .Qmin - P) 7 (equation 1)

The default value of Q_max = 4 and (condition 1) the default value of Q_mi_n = 3 (condition 2)

Equation 1 and conditions 1 and 2 were both experimentally defined for jitter- free operation lowest latency and minimum memory size using a large number of encoders from different vendors and different network conditions.

This means that the queue should have 4 frames for optimal operation (P = Q_mi_n and tsiee ^e T_s, T_s, r_s} for P < 3

The average delay T_delay caused by the queue is: T_delay = ^{Qm +1}

is

If the delivery of frame k to the renderer is delayed for an amount of time t_steep, then the renderer will repeat frame (k— 1).

If the renderer is not able to process frames, due to the fact of expired t_now + T_s or because of overruns in the queue, then some frames will be dropped.

Queue under-run

If the queue runs out of frames because of a signal loss of the stream or a too huge jitter in the stream, then the renderer has to double the last frame.

Queue over-run

If the queue runs over because of a burst in the stream or because it's not possible to deliver the frames as fast as needed, the queue drops the oldest frames in the queue to make room for the new one. De-coupling of the queue from the renderer

Accepting frames into the queue and delivering frames further to the renderer must be decoupled. The queue needs to run in a thread separate from the rendering task's thread(s). This avoids that the queue capability to accept further frames gets blocked by rendering delays.

Average frame rate of the incoming stream f_s is different from the display frame rate

The algorithm is also working if the average frame rate f_s of the incoming stream is different from the display frame rate f_d . The goal of the queue is to de-jitter the stream. The queue is filled with frames from the decoder at an average rate equal to f_s . This means that the queue also has to deliver frames to the renderer at an average rate f_s.

If the average frame rate of the stream is lower as the frame rate of the display, the renderer has to repeat frames by its own.

If the frame rate of the stream is larger than the frame rate of the display, then the queue runs over and drops frames, because the renderer doesn't accept more data as it can process.

Other embodiments of the invention

The invention is useful for all cases where video frames must be presented, renderer or displayed in accordance with whatever timing requirement either in absolute time or relative with respect to each other or both, for instance timing requirements related to realizing a fluid presentation of a video stream, for instance by performing an evenly presentation in time of the video frames.

Further it is clear that the invention is equally applicable to situations where multiple queues are used (e.g. related to different type of video frames) and that the parameters such as minimum buffer fill level, maximum buffer fill level, delay computation parameters can be different per queue.

The invention manages one or more queues (denoted frame buffer(s)) for decoded frames, wherein decoded frames will be queued, more in particular the invented methods handles one or more of different situations such as (1) if the queue is full (P ⁼ Qmax) or nearly full, a replacement strategy (e.g. new frames to be queued shall drop the oldest frames in the queue or alternative strategies ) or (2) other situations.

As presented above a closed loop to keep the queue fill level equal to Q_mi_n can be used but actually any target number between Q_mi_n and Q_max can be selected. The current fill level of the queue is used as the control value and any control strategy (for instance PID) can be used thereon (open loop, closed loop) based on the present current fill level P, but also earlier values or even predicted values., preferably with the task to hold the current fill level P as close as possible a target value (either fixed or also time varying)

Naturally the invented methods are capable to recognize initialization or transient aspects to fill the queue.

In an embodiment of the invented methods computed or estimated frame rate data is used and the invented methods (either for queue management perspective or rendering management or both) are capable to operate even when such data is not (judged to be) reliable. Further the invented methods provide a solution to avoid blocking due to rendering delays by decoupling the accepting of frames into the queue and delivering frames further to the renderer, by using multi-threading, hence running the various sub methods in separate threads.

As indicate above the invented methods may rely on frame rate calculations, based on historic averaged data, preferably these methods are invoked only when they can produce reliable data.

As conclusion it can be stated that the invention relates to a method for managing the queue of video frames between a decoder and the renderer, based on determining the current fill level of the queue and adjusting the delivery to the renderer based on at least the current fill level of the queue; and combination of said method with frame estimation and/or combination on renderer control.

Figure 3 shows a system with a first subsystem (e.g. a decoder) (10), a queue (20) and a second subsystem (e.g. a renderer)(30) , a first means (CPU, GPU, ASIC) (40), measuring queue information (current and/or previous) and creating based thereon signals for accepting frames in the queue and/or transmitting frame outside the queue, and further second means (CPU, GPU, ASIC) (50), generating steering signals for the second subsystem. Said means may communicate with each other, and may further receive first subsystem information. Note that (30)(40) may be one system but preferably the methods running for queue and second subsystem management (or even management of the first subsystem) run an own thread of control.

Figure 4 shows a general flow chart with a method (100) of determining a delivery rate (as part of managing the queue fill level), a method (120) of managing the rendering (as part of establishing quality display) and/or a method (130) determining estimated frame rate information, more in particular the fill level (105) is inputted and leads to a delivery rate (115) while the outcome (125) is either displaying based on added time information (as based on an estimate frame rate) or displaying based on just repeating a frame or just dropping a frame. The input (135) relates to the time spend in a queue and based on the reliability of the estimate, to be tagged time information (145) is provided.

Claims

1. A method for managing the queue of video frames between a decoder and the renderer, comprising the steps of: determining the current fill level of the queue; and adjusting the delivery to the renderer by comparing at least the current fill level of the queue with a first predetermined fill level to hold the current fill level \? as close as possible to this first predetermined fill level.

2. The method of claim 1, wherein said step of adjusting the delivery is operable only when the current fill level is larger than a second predetermined fill level.

3. The method of claim 2, wherein when said step of adjusting the delivery is not operable, delivery of the video frames is delayed.

4. The method of claim 3, wherein the delay depends on the difference between the current fill level and the second predetermined fill level.

5. The method of claim 2, 3 or 4 wherein the first and second predetermined fill levels are equal.

6. The method of any of the preceding claims wherein if the current fill level exceeds a third predetermined fill level, new frames to be queued shall drop the oldest frames in the queue.

7. The method of any of the preceding claims, wherein in case a reliable estimate of the video frame rate can be computed, time information is tagged to a video frame before the video frame is delivered to the renderer.

8. The method of claim 7, wherein such reliable estimate is deemed available when a predefined minimum amount of video frames has passed through the queue.

9. The method of claim 7 or 8, wherein such reliable estimate is deemed available when the video frame rate (as estimated) exceeds a predefined minimum frame rate.

10. The method of claim 7, 8 or 9, wherein the estimate of the video frame rate is determined from the accumulated time spend in the queue of a predetermined amount of video frames (equal to the predefined minimum amount).

11. The method of claim 9 or 10, wherein this estimate is updated upon decoding of a new frame

12. The method of claim 9, 10, or 11, wherein said estimate is computed as a moving average.

13. A method for handling video frames by a renderer, based on the method of any of the preceding claims for delivery of video frames from the queue between a decoder and the renderer; whereby the renderer drops the video frame if delivered when the Tenderer's video frame memory is full.

14. A method for handling video frames by a renderer, based on the method of any of the preceding claims for delivery of video frames from the queue between a decoder and the renderer; further comprising the steps of: tagging a delivered video frame with a time indication; and if the renderer with its display rate cannot display the video frame such that the time indication tagged tot the video frame falls within a predetermined time window, the renderer drops the video frame.

15. The method of any of the preceding claims for handling video frames by a renderer, wherein the renderer displays the video frame in accordance with the tagged time indication.

16. The method of any of the preceding claims wherein the time indication is based on the estimated video frame rate.

17. The method of any of the preceding claims for handling video frames by a renderer, wherein the renderer repeats displaying a previous video frame if other video frames are delayed.

18. A controller suited for managing the queue of a system, comprising a decoder, a queue for temporarily storing a plurality of video frames and a renderer, wherein the controller comprises:

means for determining the current fill level of the queue; and

Means for adjusting the delivery to the renderer by comparing at least the current fill

p level of the queue with a first predetermined fill level to hold the current fill level ¹ as close as possible to this first predetermined fill level.

19. The controller of claim 18, wherein the means for adjusting is adapted so that the delivery is operable only when the current fill level is larger than a second predetermined fill level.

20. The controller of claim 19, wherein the means for adjusting is adapted so that if the delivery is not operable, delivery of the video frames is delayed.

21. The controller of claim20, adapted so that the delay depends on the difference between the current fill level and the second predetermined fill level.

22. The controller of claim 19, 20 or 21 adapted so that the first and second predetermined fill levels are equal.

23. The controller of any of the claims 18 to 22 adapted so that if the current fill level exceeds a third predetermined fill level, new frames to be queued shall drop the oldest frames in the queue.

24. The controller of any of the claims 18 to 23, adapted so that in case a reliable estimate of the video frame rate can be computed, time information is tagged to a video frame before the video frame is delivered to the renderer.

25. The controller of claim 24, adapted so that such reliable estimate is deemed available when a predefined minimum amount of video frames has passed through the queue.

26. The controller of claim 24 or 25, adapted so that such reliable estimate is deemed available when the video frame rate (as estimated) exceeds a predefined minimum frame rate.

27. The controller of claim 24, 25 or 26, adapted so that the estimate of the video frame rate is determined from the accumulated time spend in the queue of a predetermined amount of video frames (equal to the predefined minimum amount).

28. The controller of claim 26 or 27, adapted so that this estimate is updated upon decoding of a new frame

29. The controller of claim 26, 27, 28 adapted so that said estimate is computed as a moving average.

30. A system, comprising a decoder, a queue for temporarily storing a plurality of video frames and a renderer, further comprising a controller for managing the queue, the controller comprising

Means for determining the current fill level of the queue; and

Means for adjusting the delivery to the renderer by comparing at least the current fill level of the queue with a first predetermined fill level to hold the current fill level r as close as possible to this first predetermined fill level.

31. The system of claim 30, wherein the means for adjusting is adapted so that the delivery is operable only when the current fill level is larger than a second predetermined fill level.

32. The system of claim 31, wherein the means for adjusting is adapted so that the delivery is not operable, delivery of the video frames is delayed.

33. The system of claim 32, adapted so that the delay depends on the difference between the current fill level and the second predetermined fill level.

34. The system of claim 31, 32 or 33 adapted so that the first and second predetermined fill levels are equal.

35. The system of any of the claims 30 to 34 adapted so that if the current fill level exceeds a third predetermined fill level, new frames to be queued shall drop the oldest frames in the queue.

36. The system of any of the claims 30 to 35, adapted so that in case a reliable estimate of the video frame rate can be computed, time information is tagged to a video frame before the video frame is delivered to the renderer.

37. The system of claim 36, adapted so that such reliable estimate is deemed available when a predefined minimum amount of video frames has passed through the queue.

38. The system of claim 36 or 37, adapted so that such reliable estimate is deemed available when the video frame rate (as estimated) exceeds a predefined minimum frame rate.

39. The system of claim 36, 37 or 38, adapted so that the estimate of the video frame rate is determined from the accumulated time spend in the queue of a predetermined amount of video frames (equal to the predefined minimum amount).

40. The system of claim 38 or 39, adapted so that this estimate is updated upon decoding of a new frame

41. The system of claim 38, 39, or 40 wherein said estimate is computed as a moving average.

42. The system of any of the claims 30 to 41 wherein the renderer drops the video frame if delivered when the Tenderer's video frame memory is full.

43. A system of any of the claims 30 to 42 further comprising: means for tagging a delivered video frame with a time indication; and the renderer is adapted so that if the renderer with its display rate cannot display the video frame such that the time indication tagged tot the video frame falls within a predetermined time window, the renderer drops the video frame.

44. The system of claim 43 wherein the renderer is adapted to display the video frame in accordance with the tagged time indication.

45. The system of any of the claims 30 to 44 adapted so that the time indication is based on the estimated video frame rate.

46. The system of any of the claims 20 to 45 wherein the renderer is adapted to repeat displaying a previous video frame if other video frames are delayed.

47. A computer program product comprising code segments that when executed on a suitable processing engine implement those steps in any of the methods of claims 1 to 17.

48. The computer program product of claim 47, wherein the steps of the methods of claim 1 to 17 related to queue management and the steps of the methods of claim 1 to 17 related to renderer are separate threads.

49. A machine readable signal storage medium, storing the computer program product of claim 47 or 48.