WO2008073881A2 - Système et procédé de communication interactive à faible retard utilisant de multiples connexions tcp et un codage dimensionnable - Google Patents

Système et procédé de communication interactive à faible retard utilisant de multiples connexions tcp et un codage dimensionnable Download PDF

Info

Publication number
WO2008073881A2
WO2008073881A2 PCT/US2007/086958 US2007086958W WO2008073881A2 WO 2008073881 A2 WO2008073881 A2 WO 2008073881A2 US 2007086958 W US2007086958 W US 2007086958W WO 2008073881 A2 WO2008073881 A2 WO 2008073881A2
Authority
WO
WIPO (PCT)
Prior art keywords
scalable
key frame
tcp
receiver
over
Prior art date
Application number
PCT/US2007/086958
Other languages
English (en)
Other versions
WO2008073881A3 (fr
Inventor
Jacob Chakareski
Roi Sasson
Alexandros Eleftheriadis
Ofer Shapiro
Original Assignee
Vidyo, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from PCT/US2006/061815 external-priority patent/WO2007067990A2/fr
Priority claimed from PCT/US2006/062569 external-priority patent/WO2007076486A2/fr
Priority claimed from PCT/US2007/063335 external-priority patent/WO2007103889A2/fr
Application filed by Vidyo, Inc. filed Critical Vidyo, Inc.
Publication of WO2008073881A2 publication Critical patent/WO2008073881A2/fr
Publication of WO2008073881A3 publication Critical patent/WO2008073881A3/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/643—Communication protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/22—Traffic shaping
    • H04L47/225—Determination of shaping rate, e.g. using a moving window
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
    • H04L47/2416—Real-time traffic
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/26—Flow control; Congestion control using explicit feedback to the source, e.g. choke packets
    • H04L47/263—Rate modification at the source after receiving feedback
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/27—Evaluation or update of window size, e.g. using information derived from acknowledged [ACK] packets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/28—Flow control; Congestion control in relation to timing considerations
    • H04L47/283—Flow control; Congestion control in relation to timing considerations in response to processing delays, e.g. caused by jitter or round trip time [RTT]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/29—Flow control; Congestion control using a combination of thresholds
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate

Definitions

  • the present invention relates to low-delay, interactive communication systems.
  • the invention relates to achieving low latency in packet-based communication systems in which multiple Transmission Control Protocol (TCP) connections are used for transmitting scalable coded data.
  • TCP Transmission Control Protocol
  • TCP Transmission Control Protocol
  • IP Internet
  • TCP uses an Additive Increase Multiplicative Decrease (AIMD) rate control mechanism to ensure fair use of shared network resources (e.g., the available bit rate).
  • AIMD Additive Increase Multiplicative Decrease
  • TCP/ AIMD operation whenever all outstanding packets sent within the last round-trip time (RTT) cycle are acknowledged by the receiver, TCP increases the transmission rate of the sender by a constant amount additively.
  • RTT round-trip time
  • TCP detects congestion (or packet loss) by not having all outstanding packets acknowledged by the onset of the next RTT period, it halves the transmission rate of the sender, i.e., it multiplicatively reduces the rate by a factor of 1/2.
  • Such TCP/ AIMD rate control operation can create significant variations in the transmission bit rates, leading to exceedingly high latencies in packet delivery. This drawback makes TCP unsuitable for transport of interactive media packets, which are typically characterized by stringent delivery deadlines.
  • TCP transport In some situations involving interactive multimedia communications, however, it is necessary to employ TCP transport in spite of its drawbacks. For example, corporate firewalls are sometimes set to block all traffic to, and from, the corporate Local Area Network (LAN) except over TCP connections. Therefore, media packets from the outside world destined for a receiver on the corporate LAN must be delivered via LAN.
  • LAN Local Area Network
  • TCP Transmission Control Protocol
  • the first of these studies describes a TCP-friendly scheme, which provides an equation-based rate control technique as an alternative to the TCP/ AIMD rate control mechanism while preserving the feature of sharing in a fair manner the available network bit rate with existing TCP flows.
  • the equation-based rate control technique yields smoother send rate fluctuations (than TCP/ AIMD) in response to network congestion, and therefore makes it more suitable for streaming applications.
  • the second of the cited studies i.e., Multipath Live Streaming via TCP: Performance and Benefits
  • the third and fourth of the cited studies explore transmission over multiple TCP connections on the same network path as a way to increase TCP throughput in media streaming.
  • These two studies deal only with stored (pre-encoded) media content in the context of multimedia content management systems and streaming applications, respectively; furthermore, they treat the individual media packets uniformly, and do not take advantage of a possible scalable structure in the transmitted media.
  • scalable coding is used in the transmitted media, different packets have different importance in terms of how they affect the reconstruction quality of the media in the receiver.
  • Scalable coding is a well-known technique in multimedia data encoding, in which the encoder generates two or more "scaled" bitstreams that collectively represent a given medium in a bandwidth-efficient manner.
  • Scalability can be provided in a number of different dimensions, namely temporal, spatial, and quality (also referred to as SNR (Signal-to-Noise Ratio) scalability) dimensions.
  • a video signal may be scalable-coded in different layers at CIF and QCIF resolutions, and at frame rates of 7.5, 15, and 30 frames per second (fps).
  • any combination of spatial resolutions and frame rates may be obtainable from the codec bitstream.
  • the bits corresponding to the different layers can be transmitted as separate bitstreams (i.e., one stream per layer), or they can be multiplexed together in one or more bitstreams.
  • the coded bits corresponding to a given layer may be referred to as that layer's bitstream, even if the various layers are multiplexed and transmitted in a single bitstream.
  • Codecs specifically designed to offer scalability features include, for example, MPEG-2 (ISO/IEC 13818-2, also known as ITU-T H.262) and the currently developed SVC (known as ITU-T H.264 Annex G or MPEG-4 Part 10 SVC).
  • Scalable coding techniques specifically designed for video communication are described, for example, in commonly assigned International Patent Application No.
  • PCT/US06/028365 "SYSTEM AND METHOD FOR SCALABLE AND LOW-DELAY VIDEOCONFERENCING USING SCALABLE VIDEO CODING.” It is noted that even codecs that are not specifically designed to offer scalability features can exhibit scalability characteristics in the temporal dimension. For example, consider an MPEG-2 Main Profile codec, a non-scalable codec, which is used in DVDs and digital TV environments. Further, assume that the codec is operated at 30 fps and that a group of pictures (GOP) structure of
  • P pictures By sequential elimination of the B pictures, followed by elimination of the P pictures, it is possible to derive a total of three temporal resolutions: 30 fps (all picture types included), 10 fps (I and P only), and 2 fps (I only).
  • the sequential elimination process results in a decodable bitstream because the MPEG-2 Main Profile codec is so designed that coding of the P pictures does not rely on the B pictures, and, similarly, coding of the I pictures does not rely on other P or B pictures.
  • single-layer codecs with temporal scalability features are considered to be a special case of scalable video codecs, and understood to be included in the term "scalable video coding” unless explicitly indicated otherwise.
  • Scalable codecs typically have a pyramidal bitstream structure in which one of the constituent bitstreams (called the “base layer”) is essential in recovering the original medium at some basic quality.
  • base layer one of the constituent bitstreams
  • enhancement layer(s) Use of one or more of the remaining bitstream(s) (called the “enhancement layer(s)" together with the base layer increases the quality of the recovered medium.
  • Data losses in the enhancement layers may be tolerable, but data losses in the base layer can cause significant distortions or complete loss of the recovered medium.
  • Simulcasting is a coding solution that is less complex than scalable coding but has some of the advantages of the latter.
  • two different versions of the source are encoded (e.g., at two different spatial resolutions) and transmitted. Each version is independent, in that its decoding does not depend on reception of the other version.
  • simulcasting is considered to be a special case of scalable coding (where no inter layer prediction is performed), and referred to simply as scalable coding unless explicitly indicated otherwise.
  • Systems and methods for packet-based communication of scalable coded media include mechanisms for TCP-based transport of media packets for low-delay, interactive communication applications such as videoconferencing.
  • Multiple TCP connections are established between sender and receiver for communication of the media packets.
  • the sender makes scheduling decisions based on the media packets' importance in the scalable coding structure and on feedback from the receiver (e.g., on the status of individual TCP connections).
  • the systems and methods take into account the varying importance of the scalable coded packets to the quality of the reconstructed media when making scheduling decisions. Such decisions are made to maintain low latency packet delivery and to provide an acceptable audio-visual presentation experience of the received media despite the TCP rate control mechanism.
  • the systems and methods overcome the limitations TCP and its AIMD rate control mechanism that cause detrimental delay in interactive media applications.
  • FIG. 1 is a schematic diagram illustrating the architecture of a typical TCP-based communication system (prior art);
  • FIG. 2 is a schematic diagram illustrating the architecture of an exemplary communication system having multiple TCP connections, a Scheduling Inverse Multiplexer (S-IMUX), and a Feedback Multiplexer (F-MUX), in accordance with the principles of the present invention
  • S-IMUX Scheduling Inverse Multiplexer
  • F-MUX Feedback Multiplexer
  • FIG. 3 is a schematic diagram illustrating the operation of the Scheduling Inverse Multiplexer (S-IMUX), in accordance with the principles of the present invention
  • FIG. 4 is a schematic diagram illustrating the operation of the Feedback Multiplexer (F-MUX), in accordance with the principles of the present invention.
  • FIG. 5 is a schematic diagram illustrating an exemplary scalable video picture coding structure (prior art).
  • FIG. 1 shows the architecture of a conventional system 100 for TCP-based transmission of media data between a sender 110 and a receiver 120 over a network 130.
  • an encoder 112 produces audio or video data that is directly provided to a TCP stack (e.g., TCP/IP stack 114), which then transmits packet-data through a Network Interface Controller (NIC) 116 over network 130 to receiver 120.
  • NIC Network Interface Controller
  • NIC 126 in receiver 120 receives the packet-data and provides the data via TCP/IP stack 124 to the receiver's decoder 122 for decoding and display.
  • the TCP/IP components of sender 110 and receiver 120 ensure reliable delivery of the transmitted data by performing retransmission, and also apply the TCP flow control and congestion avoidance algorithms. It is noted that the connections at sender 110, from encoder 112 to TCP/IP stack 114, and at receiver 120, from TCP/IP stack 124 to decoder 122, are both unidirectional connections.
  • the connections between the TCP/IP components and the NICs are all bi-directional, as TCP is an inherently bi-directional transport protocol (i. e., since TCP acknowledgment packets are transmitted from the Receiver to the Sender, as per standard TCP operation).
  • FIG. 1 shows the architecture of system 100 in the context of live audio or video transmission, it is to be understood that the same architecture can be used for low-delay transmission of pre-coded data.
  • encoder 112 is replaced by a component that obtains pre-coded data from mass storage, random access memory, or another suitable digital memory device.
  • FIG. 1 shows system 100 having a one-way sender-receiver connection, it is to be understood that the system architecture shown is readily extended to two-way (interactive) communication. In such case, encoders 112 and decoders 116 are placed in symmetric positions in both the receiver and sender.
  • FIG. 1 shows the architecture of system 100 in the context of live audio or video transmission, it is to be understood that the same architecture can be used for low-delay transmission of pre-coded data.
  • encoder 112 is replaced by a component that obtains pre-coded data from mass storage, random access memory, or another suitable digital memory device.
  • FIG. 1 shows system 100 having a one-way sender-re
  • sender 210 makes scheduling decisions based on the transmitted media packets' importance and feedback from receiver 222.
  • sender 210 has an encoder 212, which is a scalable encoder that is connected to a Scheduling Inverse Multiplexer (S-IMUX) 218.
  • S-IMUX 218 demultiplexes scalable coded?? packets received from encoder 212 over a plurality of TCP connections to TCP/IP stack 114.
  • FIG. 2 shows only two such connections in system 200 (i.e., Conn. 0 and Conn. 1). It will be understood, however, that any suitable number of TCP connections may be used.
  • S- IMUX 218 is responsible for managing the transmission of the different encoded media packets over the plurality of TCP connections, taking into account the system's state as well as each packet's priority with respect to its role or hierarchical position in the scalability structure of encoder 212.
  • the encoded media packets are transported via TCP/IP stack 114 through NIC 112 over communication network 130 to receiver 220.
  • packets received through NIC 122 and TCP/IP stack 116 over the multiple TCP connections Conn. 0 and Conn. 1 are collected by a Feedback Multiplexer (F-MUX) 228.
  • F-MUX 228 forwards the packets to decoder 222 for decoding and display or playback, and also produces the receiver feedback packets that are required in the operation of S-IMUX 218 at sender 210.
  • connections between S-DMUX 218 and the TCP/IP component 114 in sender 210, and TCP/IP component 116 and F-MUX 228 in receiver 220 are both bi-directional. This is because application-level feedback packets are transmitted from receiver 220 to sender 210, in addition to, and separately from, the TCP acknowledgement packets.
  • system 200 is shown in FIG. 2 in the context of live audio or video transmission. However, as in the case of system 100, the shown system 200 architecture can be used for low-delay transmission of pre-coded data.
  • the shown system 200 architecture is readily extended to two-way (interactive) communication by duplicating the sender modules in the receiver and vice versa.
  • the inventive system 200 differs fundamentally from conventional systems (e.g., system 100) in at least two ways.
  • FIGS. 3 and 4 show the operations of S-IMUX 218 and F-MUX 228, respectively.
  • sender 210 establishes a plurality of TCP connections with receiver 220 (e.g., 'N' connections, where N is a positive integer and is a design parameter).
  • FIG. 3 shows the operation of S-IMUX 218 with the N connections already in place and available for use.
  • media packets are passed by scalable encoder 212 to S-IMUX 218 they are placed in an input buffer 310.
  • S-IMUX 218 then decides if and when to transmit the buffered packets (e.g., Pj - Pk), and over which of the N connections to transmit each of the packets. The decisions are made by S-IMUX 218's Scheduling and Routing Unit (SRU) 320, which makes the decisions based on the importance of the individual packets (e.g., relative to the scalability structure) and also on feedback received from receiver 220's F-MUX 228 in the form of acknowledgement packets (e.g., ACKj - ACKk). SRU 320 includes suitable scheduling algorithms for this purpose. When a packet (e.g., Pj) is to be transmitted over a given connection (e.g., connection 1), it is placed in that connection's Output Buffer 330 until removed or replaced by SRU 320.
  • a packet e.g., Pj
  • packets P, ..., Pk, ..., for k > j are provided by scalable encoder 212 to S-IMUX 218 and transmitted on Connection 1, ..., Connection 0, ... at times t,, ..., t k , ..., respectively, as decided or determined by SRU 320.
  • Receiver 210's F-MUX 228 acknowledges the receipt of these packets via acknowledgement packets (ACK j - ACK k ) sent back to SRU 320.
  • the arrival times of the respective acknowledgements at the sender are denoted by t' j5 ..., t' k ,... .
  • Error control in SRU 320's scheduling algorithm may be incorporated in the following manner. Let the current packet operated on by SRU 320 be P j . SRU 320 transmits packet P j on Connection 1 at the time instance t j . SRU 320 then waits up to 'T' units of time to receive the corresponding acknowledgement on Connection 1, where T is a design parameter. If an acknowledgement arrives by time t, + T, SRU 320 proceeds to the next packet in the input buffer. If, however, no such acknowledgement packet has arrived by time t, + T, SRU 320 flags Connection 1 as being unavailable at the moment (due to packet loss or congestion experienced thereon) and prepares for other packet scheduling steps. It is noted that TCP will continue trying to deliver this packet P j on Connection 1 due to its property of reliable delivery.
  • a "key video picture” or “key audio frame” (or parts thereof) is a picture or audio frame for which delivery is necessary in order to ensure an uninterrupted visual experience of the media presentation at the receiver.
  • scalable coding a key picture or key audio frame corresponds to the lowest temporal layer across all scalability dimensions provided by the encoder. In the following description, all such packets are referred to as key packets, without differentiating whether the encoded media is audio or video.
  • S- IMUX 218 discards P j and all subsequent packets received from scalable encoder 212 until a new key video picture or audio frame packet P ⁇ , for k > j, is received for transmission. S-IMUX 218 then proceeds to transmit this new packet using the procedure described above for packet P j .
  • SRU 320 checks in a round-robin fashion if another connection (e.g., Connection 2) can be used to retransmit packet P j .
  • SRU 320 may do this, for example, by verifying that the last packet sent on a particular connection (e.g., Connection 2) has been eventually acknowledged, i.e., it is no longer marked or flagged as unavailable. If that is the case, SRU 320 then transmits packet P j on Connection 2.
  • SRU 320 will repeat the process of retransmitting packet P j over other connections scanned in a round-robin fashion, until eventually the packet is acknowledged on one of the connections. When one such acknowledgement arrives, SRU 320 is done with packet P j and can move on to transmitting another packet from the input buffer 310. This other packet is not necessarily the packet immediately following P j in input buffer 310.
  • SRU 320 When the receipt of key packet P j is acknowledged after an initial failed transmission attempt, SRU 320 is in a congestion recovery mode. In order to minimize the amount of data to be transmitted, SRU 320 selects the next packet for transmission to be either the earliest key packet present in input buffer 310 or, if no such packet is yet available, it selects the latest packet P k , where k > j. In this process, SRU 320 will skip over to the selected key packet in input buffer 310, and discard (i.e., not transmit) all other in-between packets received from scalable encoder 212. Transmission of the selected packet proceeds in the same manner as described herein.
  • SRU 320' s scheduling algorithm is designed to allow the communication network to recover from the temporary congestion as detected by the missing acknowledgement ACK j on Connection 1.
  • SRU 320 sends no data until the next key picture (e.g., Pk) is due to be transmitted, SRU 320 in fact provides for faster congestion recovery of the communication network.
  • the intervening packets discarded by SRU 320 are not crucial for the continuous reconstruction of the media presentation at the receiver. It is expected that the temporary reduction in visual or audio quality of the presentation at the receiver due to non-receipt of the intervening packets is not dramatic, due to the scalable nature of the media encoding.
  • the scheduling algorithm of SRU 320 may continue to use a particular connection for subsequent transmissions of new packets, as long as the previous transmissions (on this same connection) are acknowledged in a timely manner (e.g., within the timer expiration limit T). While a connection is healthy (i.e., it has not timed out on a transmission), there is no reason to switch to any of the other N-I TCP connections. Continued use of a healthy connection allows the other connections to remain open to potentially receive any pending acknowledgements for recent transmissions thereon, and thereby indicate recovery from congestion and/or packet loss that might have affected some of them recently.
  • SRU 320 The detailed processing steps of SRU 320 are listed in TABLE I using pseudo-code.
  • n e ⁇ 0, 1, ..., N-I ⁇ represents the connection number
  • P is the current packet
  • t denotes the current system time
  • t 0 is a helper variable that stores time values.
  • the flag 's' is used to signal if packet skipping in input buffer 310 has to occur after an initial failed transmission attempt of a key packet (i.e., the first transmission of a packet timed-out). The flag is not necessary for non-key packets, as they are not retransmitted and the skipping can occur immediately.
  • the function Free(n) is defined to return a 0 if connection 'n' is currently waiting for an acknowledgement packet and is thus unavailable for transmission, and 1 otherwise.
  • Free(n) can be trivially implemented by associating a parameter 'ack_state' with each connection, which is set to 1 when a packet is transmitted, and reset when the corresponding acknowledgement is received. In such implementation, Free(n) simply returns the value of that flag for connection n. It is assumed that ACKs received at S- IMUX 218 are processed asynchronously to the processing steps shown below.
  • the value for the time-out parameter T is preferably selected in consideration of the round-trip time (RTT) observed on the network path between sender 210 and receiver 220.
  • RTT round-trip time
  • a judiciously selected T would not incur unnecessary retransmissions of media packets due to the late arrival of acknowledgements for the previous transmissions.
  • T should not unnecessarily delay retransmissions waiting for acknowledgements that will never materialize at the sender.
  • the value selected for T must also account for the dynamics of the RTT over time and the related dispersion of its values.
  • the processing steps listed in TABLE 1 may further include an upper limit on the number of retransmission attempts for a key frame, after which the connection is considered lost or not in service. This upper limit may be expressed by a second time-out parameter, T2, which may be set at a value several times that of parameter T.
  • T is computed as mean(RTT) + ⁇ *std(RTT), where the multiplier ⁇ has the value 3 or 4.
  • This quantity is dynamically updated as the values of the mean RTT and its standard deviation are (re)computed over time (i.e., online).
  • the statistics of the RTT can be computed online by sender 210 based on the ACK packets or, if RTCP reports are available in system 200, they can be obtained through their periodic exchange between senders and the receivers.
  • FIG. 4 shows the operation of F-MUX 228 at receiver 220 corresponding to the operation of S-IMUX 218 with the N connections already in place and available for use (FIG. 3).
  • the N TCP connections e.g., Connection 0, ..., Connection N-I
  • FCU F-MUX Feedback and Combiner Unit
  • FCU 410 is also responsible for reassembling the different packets arriving on the multiple connections into a single packet stream created in F-MUX output buffer 430, as well as for discarding duplicate packets that may be received due to the retransmissions performed by S-IMUX 218's SRU 320.
  • FCU 410 is also connected to a set of N connection buffers 420 ('Connection 0 Buffer' through 'Connection N-I Buffer'), which are used by the FCU to temporarily store incoming packets for reordering and to also know which packets have already been received so that duplicates are eliminated.
  • scalable encoder 212 (FIG. 2) is a scalable video encoder that operates with two spatial layers and three temporal layers, as described in International Patent Application PCT/US06/028365.
  • FIG. 5 shows an exemplary picture coding structure 500 for such an encoder.
  • the prediction paths in structure 500 that may be used by encoder 212 are designated by arrows.
  • the key packets for such an encoder are those that carry LO data, i.e., the lowest spatial and temporal layer.
  • the systems and methods of the present invention are designed to ensure the timely delivery of this data, as it is crucial for decoding of the data of all other layers (Ll -L2, S0-S2).
  • FCU 410 will have to create an output packet stream in the output buffer 430 so that lower layers precede higher layers for the same temporal instance, while maintaining proper temporal ordering of pictures (in coding order).
  • the four pictures e.g., (LO, SO) . . .(L2, S2)
  • FIG. 5 the four pictures (e.g., (LO, SO) . . .(L2, S2)) shown in FIG. 5 are received in different packets across a set of N connections. Due to the design of S-DMUX 218 (application-level retransmissions over TCP connections that ultimately ensure reliable delivery), it is theoretically possible to have multiple copies of a packet arriving at FCU 410 as well as out of order arrivals.
  • FCU 410 then places arriving packets in the corresponding connection buffers 420, and at the same time it continuously tries to assemble the output buffer 430 stream by including, in sequence, packets for LO, SO, the first L2 and S2, Ll and S 1 , and finally the second L2 and S2 (assuming all layers are to be received at the FCU). In doing so, FCU 410 examines all connection buffers 420 to find the appropriate packet at each step in sequence. When such a packet is found, it is removed from the buffer where it is located. While searching for the correct packet, FCU 410 can also remove duplicate packets that are no longer needed using the timing or picture ordering information that all standard codecs embed in their packetized data.
  • TCP/IP components of the sender and receiver are assumed to be "black boxes,” and accessible only through their standard interfaces (e.g., sockets).
  • TCP source code When access to TCP source code is available to the designer, it may be possible to utilize TCP's acknowledgement status information and to thereby avoid transmitting an application- level acknowledgment packet from the receiver to the sender, in accordance with the present invention.
  • the bit rate savings may not be very significant, especially in a two-way communication system where large amounts of media data flow in both directions. It will be understood that in accordance with the present invention, the transmission techniques described herein may be implemented using any suitable combination of hardware and software.
  • the software for implementing and operating the aforementioned rate estimation and control techniques can be provided on computer-readable media, which can include, without limitation, firmware, memory, storage devices, microcontrollers, microprocessors, integrated circuits, ASICs, on-line downloadable media, and other available media.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Multimedia (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

L'invention concerne des systèmes et des procédés de communication de signaux audiovisuels codés dimentionnables sur de multiples connexions TCP/IP. L'expéditeur planifie et donne priorité à la transmission de paquets de données codés dimentionnables individuels sur la pluralité de connexions TCP selon leur importance relative dans la structure de codage dimentionnables pour une qualité de reconstruction de signaux et de selon la réponse du récepteur. La distribution de paquets de faible latence sur les multiples connexions TCP/IP est maintenue en évitant la transmission ou la retransmission de paquets qui sont moins importants pour une qualité de support reconstruite.
PCT/US2007/086958 2006-12-08 2007-12-10 Système et procédé de communication interactive à faible retard utilisant de multiples connexions tcp et un codage dimensionnable WO2008073881A2 (fr)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
USPCT/US2006/061815 2006-12-08
PCT/US2006/061815 WO2007067990A2 (fr) 2005-12-08 2006-12-08 Systemes et procedes relatifs a l'elasticite d'erreur et a l'acces aleatoire dans des systemes de communication video
USPCT/US2006/062569 2006-12-22
PCT/US2006/062569 WO2007076486A2 (fr) 2005-12-22 2006-12-22 Systeme et procede pour la videoconference utilisant le decodage video echelonnable et serveurs de videoconference de composition d'images
USPCT/US2007/063335 2007-03-05
PCT/US2007/063335 WO2007103889A2 (fr) 2006-03-03 2007-03-05 Système et procédé permettant de fournir la robustesse aux erreurs, l'accès direct et la commande de débit dans des communications vidéo échelonnables

Publications (2)

Publication Number Publication Date
WO2008073881A2 true WO2008073881A2 (fr) 2008-06-19
WO2008073881A3 WO2008073881A3 (fr) 2008-08-28

Family

ID=39512075

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/US2007/083351 WO2008073610A1 (fr) 2005-07-20 2007-11-01 Système et procédé de commande de débit de transmission dans des communications numériques par commutation de paquets
PCT/US2007/086958 WO2008073881A2 (fr) 2006-12-08 2007-12-10 Système et procédé de communication interactive à faible retard utilisant de multiples connexions tcp et un codage dimensionnable

Family Applications Before (1)

Application Number Title Priority Date Filing Date
PCT/US2007/083351 WO2008073610A1 (fr) 2005-07-20 2007-11-01 Système et procédé de commande de débit de transmission dans des communications numériques par commutation de paquets

Country Status (1)

Country Link
WO (2) WO2008073610A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7948887B2 (en) * 2008-06-24 2011-05-24 Microsoft Corporation Network bandwidth measurement
FR2992126A1 (fr) * 2012-06-15 2013-12-20 France Telecom Procede et dispositif d'estimation rapide et peu intrusive de la bande passante disponible entre deux noeuds ip
EP2996285B1 (fr) 2013-05-30 2017-09-06 Huawei Technologies Co., Ltd. Procédé, appareil, et système de programmation

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020163918A1 (en) * 2001-05-04 2002-11-07 Globespan Virata, Incorporated System and method for distributed processing of packet data containing audio information
US20040001479A1 (en) * 2002-07-01 2004-01-01 Pounds Gregory E. Systems and methods for voice and data communications including a network drop and insert interface for an external data routing resource

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5555244A (en) * 1994-05-19 1996-09-10 Integrated Network Corporation Scalable multimedia network
US6973622B1 (en) * 2000-09-25 2005-12-06 Wireless Valley Communications, Inc. System and method for design, tracking, measurement, prediction and optimization of data communication networks
US7225459B2 (en) * 2001-10-17 2007-05-29 Numerex Investment Corproation Method and system for dynamically adjusting video bit rates

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020163918A1 (en) * 2001-05-04 2002-11-07 Globespan Virata, Incorporated System and method for distributed processing of packet data containing audio information
US20040001479A1 (en) * 2002-07-01 2004-01-01 Pounds Gregory E. Systems and methods for voice and data communications including a network drop and insert interface for an external data routing resource

Also Published As

Publication number Publication date
WO2008073881A3 (fr) 2008-08-28
WO2008073610A1 (fr) 2008-06-19

Similar Documents

Publication Publication Date Title
US7933294B2 (en) System and method for low-delay, interactive communication using multiple TCP connections and scalable coding
US9179160B2 (en) Systems and methods for error resilience and random access in video communication systems
Wu et al. Transporting real-time video over the Internet: Challenges and approaches
JP5562861B2 (ja) スケーラブルマルチメディアデータストリームをストリーミングする方法及び装置
AU2008204833A1 (en) Improved systems and methods for error resilience in video communication systems
Singh et al. Comparison of multiple-description coding and layered coding based on network simulations
Gürses et al. A simple and effective mechanism for stored video streaming with TCP transport and server-side adaptive frame discard
WO2008073881A2 (fr) Système et procédé de communication interactive à faible retard utilisant de multiples connexions tcp et un codage dimensionnable
Al-Madani et al. Scalable wireless video streaming over real-time publish subscribe protocol (RTPS)
AU2012216587B2 (en) Systems and methods for error resilience and random access in video communication systems
Sanhueza et al. Efficient video streaming rate control based on a deadline-sensitive selection of SVC layers
Zhai Optimal cross-layer resource allocation for real-time video transmission over packet lossy networks
Kropfberger et al. Evaluation of RTP immediate feedback and retransmission extensions [video streaming applications]
Yang et al. An adaptive multi-layer low-latency transmission scheme for H. 264 based screen sharing system
Hou et al. A novel differentiated retransmission scheme for MPEG video streaming over wireless links
Fiandrotti et al. High-performance H. 264/SVC video communications in 802.11 e ad hoc networks
Brennan et al. Split-layer video multicast protocol: A new receiver-based rate-adaptation protocol
Tafleen Fault Tolerance Strategies for Low-Latency Live Video Streaming
Bajić Error control for broadcasting and multicasting: An overview
Bortoleto et al. Large-scale media delivery using a semi-reliable multicast protocol
Begg High quality video streaming with SCTP over CDMA2000
Gürses Optimal streaming of rate adaptable video
Kang et al. Robust Video Streaming over Wireless In-Home Networks
Kim et al. Feedback-based adaptive video streaming over lossy channels
Yücesan Combined Use of Congestion Control and Frame Discarding for Internet Video Streaming

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07865458

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 07865458

Country of ref document: EP

Kind code of ref document: A2