US20030105874A1 - Method and apparatus for real time communication over packet networks - Google Patents
Method and apparatus for real time communication over packet networks Download PDFInfo
- Publication number
- US20030105874A1 US20030105874A1 US10/282,403 US28240302A US2003105874A1 US 20030105874 A1 US20030105874 A1 US 20030105874A1 US 28240302 A US28240302 A US 28240302A US 2003105874 A1 US2003105874 A1 US 2003105874A1
- Authority
- US
- United States
- Prior art keywords
- frames
- transporting
- variable
- buffer
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/64—Hybrid switching systems
- H04L12/6418—Hybrid transport
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/764—Media network packet handling at the destination
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/80—Responding to QoS
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q11/00—Selecting arrangements for multiplex systems
- H04Q11/04—Selecting arrangements for multiplex systems for time-division multiplexing
- H04Q11/0428—Integrated services digital network, i.e. systems for transmission of different types of digitised signals, e.g. speech, data, telecentral, television signals
- H04Q11/0478—Provisions for broadband connections
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04J—MULTIPLEX COMMUNICATION
- H04J3/00—Time-division multiplex systems
- H04J3/02—Details
- H04J3/06—Synchronising arrangements
- H04J3/0635—Clock or time synchronisation in a network
- H04J3/0682—Clock or time synchronisation in a network by delay compensation, e.g. by compensation of propagation delay or variations thereof, by ranging
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/54—Store-and-forward switching systems
- H04L12/56—Packet switching systems
- H04L12/5601—Transfer mode dependent, e.g. ATM
- H04L2012/5614—User Network Interface
- H04L2012/5616—Terminal equipment, e.g. codecs, synch.
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/54—Store-and-forward switching systems
- H04L12/56—Packet switching systems
- H04L12/5601—Transfer mode dependent, e.g. ATM
- H04L2012/5638—Services, e.g. multimedia, GOS, QOS
- H04L2012/5646—Cell characteristics, e.g. loss, delay, jitter, sequence integrity
- H04L2012/5647—Cell loss
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/54—Store-and-forward switching systems
- H04L12/56—Packet switching systems
- H04L12/5601—Transfer mode dependent, e.g. ATM
- H04L2012/5638—Services, e.g. multimedia, GOS, QOS
- H04L2012/5646—Cell characteristics, e.g. loss, delay, jitter, sequence integrity
- H04L2012/5649—Cell delay or jitter
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/54—Store-and-forward switching systems
- H04L12/56—Packet switching systems
- H04L12/5601—Transfer mode dependent, e.g. ATM
- H04L2012/5638—Services, e.g. multimedia, GOS, QOS
- H04L2012/5671—Support of voice
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/54—Store-and-forward switching systems
- H04L12/56—Packet switching systems
- H04L12/5601—Transfer mode dependent, e.g. ATM
- H04L2012/5678—Traffic aspects, e.g. arbitration, load balancing, smoothing, buffer management
- H04L2012/5681—Buffer or queue management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/64—Hybrid switching systems
- H04L12/6418—Hybrid transport
- H04L2012/6429—Terminal adapters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/64—Hybrid switching systems
- H04L12/6418—Hybrid transport
- H04L2012/6472—Internet
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/64—Hybrid switching systems
- H04L12/6418—Hybrid transport
- H04L2012/6475—N-ISDN, Public Switched Telephone Network [PSTN]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/64—Hybrid switching systems
- H04L12/6418—Hybrid transport
- H04L2012/6481—Speech, voice
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/64—Hybrid switching systems
- H04L12/6418—Hybrid transport
- H04L2012/6483—Video, e.g. MPEG
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/64—Hybrid switching systems
- H04L12/6418—Hybrid transport
- H04L2012/6489—Buffer Management, Threshold setting, Scheduling, Shaping
Definitions
- This invention relates to the field of telecommunications and more specifically to a method and apparatus for real time communication over packet networks.
- Real time communications such as audio or video can be encoded using various compression techniques.
- the encoded information can then be placed in data packets with time and sequence information and transported via non-guaranteed Quality of Service (QoS) packet networks.
- QoS Quality of Service
- Non-guaranteed packet switched networks include a Local Area Network (LAN), Internet Protocol Network, frame relay network, or an interconnected mixture of such networks such as an Internet or Intranet.
- LAN Local Area Network
- Internet Protocol Network Internet Protocol Network
- frame relay network or an interconnected mixture of such networks such as an Internet or Intranet.
- One underlying problem with non-guaranteed packet networks is that transported packets are subject to varying loss and delays. Therefore, for real-time communications, a tradeoff exists among the quality of the service, the interactive delay, and the utilized bandwidth. This tradeoff is a function of the selected coding scheme, the packetization scheme, the redundancy of information packeted within the packets, the receiver buffer size, the bandwidth restrictions, and the transporting characteristics of the transport
- the transporting network could be an Intranet, an Internet, wide area network (WAN), local area network (LAN) or other type of network utilizing technologies such as Asynchronous Transfer Mode (ATM), Frame Relay, Carrier Sense Multiple Access, Token Ring, or the like.
- ATM Asynchronous Transfer Mode
- PCs personal computers
- both parties to the communication may be connected to the network via telephone lines. These telephone lines are in communication with a local hub associated with a central office switch and Network Service provider.
- the term “hub” refers to an access point of a communication infrastructure.
- This communication technique however, has a number of disadvantages.
- the maximum bandwidth available depends on the condition of the line. Typically, this bandwidth will be no greater than approximately 3400 Hz.
- a known method for transmitting and receiving data at rates of up to 33.6 kbits/second over such a connection is described in Recommendation V0.34, published by the International Telecommunication Union, Geneva, Switzerland.
- PC-based communication particularly with respect to PC-based telephone communications
- the communicating PC receiving the call generally needs to be running at the time the call is received. This may be feasible for a corporate PC connected to an Intranet. However, such a connection may be burdensome for a home based PC since the home PC may have to tie up a phone line.
- Another disadvantage is that a PC-based conversation is similar to conversing over a speakerphone. Hence, privacy of conversation may be lost. Communicating over a speakerphone may also present problems in a typical office environment having high ambient noise or having close working arrangements.
- PC-based telephone systems often require powerful and complex voice encoders and therefore require a large amount of processing capability. Even if these powerful voice encoders run on a particularly powerful PC, the encoders may slow down the PC to a point where the advantage of document sharing decreases since the remaining processing power may be insufficient for a reasonable interactive conversation. Consequently, a caller may have to use less sophisticated encoders, thereby degrading the quality of the call.
- a general problem encountered in packet switched networks is that the network may drop or lose data packets. Packets may also be delayed during transportation from the sender to the receiver. Therefore, some of the packets at a receiving destination will be missing and others will arrive out of order.
- the immediate past transporting characteristics can be used to infer information about the immediate future transporting characteristics.
- the dynamic network transporting characteristics may be measured using such variables as packet loss, packet delay, packet burst loss, loss auto-correlation and delay variation.
- the present invention relates to a method and apparatus for communicating a real time media input.
- the apparatus includes an encoding device that encodes the media into a plurality of data packets. Each data packet includes a plurality of frames that are created according to a first variable.
- a receiving device unpacks the data packets and buffers the unpacked data packets for a playout according to a second variable.
- the receiving device generates a plurality of utility parameters for evaluating a dynamic characteristic of a transporting network.
- the transporting network transports the data packets from the encoding device to the receiving device.
- a preferred utility parameter is selected. The preferred utility parameter is used to adjust the first and the second variable.
- an apparatus for communicating a real time media input includes a sender having an encoder and a packetizer.
- the encoder partitions and compresses the real time media input into a plurality of frames.
- the packetizer packets the frames into a plurality of data packets according to a redundancy value.
- a transporting network transports the data packets from the sender to a receiver.
- the receiver includes a real decoder and a plurality of computation decoders.
- the real decoder includes a real decoder depacketizer and a real decoder buffer.
- the real decoder depacketizer unpacks the plurality of frames.
- the frames are placed within the real decoder buffer according to a buffer length variable.
- Each computation decoder has a utility parameter for evaluating a dynamic characteristic of the transporting network.
- the computation decoder includes a computation decoder depacketizer and a computation decoder buffer.
- the computation decoder depacketizer unpacks the plurality of frames and communicates the frames to the computation decoder buffer.
- the receiver selects a preferred utility parameter from the utility parameters and communicates a feedback variable to the real decoder buffer.
- the buffer length variable is adjusted in accordance with a change in the dynamic characteristic.
- a system for transmitting real time media includes a first calling device for placing a call to a first processing hub.
- the first processing hub includes an encoder that partitions the call into a plurality of compressed frames.
- a packetizer packetizes the frames into a data packet according to a redundancy variable.
- a transporting network transports the data packet between the first hub and a second hub.
- the second hub includes a decoder for decoding the data packet into the plurality of frames and ordering these frames within a buffer with depth according to a buffer length variable.
- the decoder generates a plurality of utility parameters based on the redundancy value and the buffer length variable.
- the decoder selects a preferred utility parameter from the utility parameters such that the redundancy value and the buffer length are adjusted in accordance with a change in the dynamic characteristic.
- a second calling device receives the call from the second processing hub.
- a method for communicating a real time media input includes the step of communicating the real time media input to a sending device and encoding the input into a plurality of data packets.
- Each data packet comprises a plurality of frames ordered according to a first variable.
- the data packets are transported to a receiving device where they are unpacked and buffered for a playout of the analog input according to a second variable.
- At least two utility parameters are generated for evaluating a dynamic characteristic of a transporting network that transports the data packets from the sending device to the receiving device.
- a preferred utility parameter is selected from the utility parameters.
- the first and the second variable are adjusted according to the preferred utility parameter.
- a method for communicating a real time media input includes the steps of partitioning and compressing the real time media input into a plurality of frames.
- the frames are packetized into a plurality of data packets according to an actual redundancy variable.
- the data packets are transported by a transporting network.
- the plurality of frames are unpacked and arranged within a buffer according to a buffer length variable.
- a plurality of utility parameters are generated for evaluating a dynamic characteristic of the transporting network.
- a preferred utility parameter is selected from the utility parameters wherein the buffer length variable is adjusted in accordance with a change in the dynamic characteristic.
- a method for transmitting real time audio includes the steps of placing a call to a first processing hub.
- the call is partitioned and compressed at the first processing hub into a plurality of frames.
- the frames are packed into a data packet according to a redundancy value.
- the data packet is transported between the first processing hub and a second processing hub by a transporting network.
- the data packet is decoded at the second processing hub into the plurality of frames.
- These frames are ordered within a second hub buffer according to a buffer length.
- a plurality of utility parameters are generated based on the redundancy value and the buffer length.
- a preferred utility parameter is selected from the utility parameters.
- the redundancy value and the buffer length are adjusted in accordance with a dynamic characteristic of the transporting network.
- the call is then received from the second processing hub.
- FIG. 1 shows a general overview of a system for transporting a real time media input over a packet switched network and incorporating a preferred embodiment of the present invention.
- FIG. 2 illustrates a communication channel, including a sender and a receiver, in accordance with the system shown in FIG. 1.
- FIG. 3 is block diagram of a data packet transported between the sender and the receiver shown in FIG. 2.
- FIG. 4 shows an order of the redundant frames in five levels of data packets.
- FIG. 5 is an illustration of a linked list structure of a real decoder buffer shown in FIG. 2.
- FIG. 6 is a flowchart of a GetNode function for accessing the linked list shown in FIG. 5.
- FIG. 7 is a flowchart of a PutNode function for the real decoder shown in FIG. 2 and which accesses the linked list structure shown in FIG. 5.
- FIG. 8 illustrates a state transition diagram of the real decoder buffer illustrated in FIG. 2.
- FIG. 9 illustrates a flowchart of a Time Out function for the real decoder buffer shown in FIG. 2.
- FIG. 10 illustrates a flowchart of a PlayNode function for the real decoder shown in FIG. 2.
- FIG. 11 is a flowchart of a PacketArrival function for the real decoder shown in FIG. 2.
- FIG. 12 illustrates a flowchart of a PlayNode function for one of the computation decoders shown in FIG. 2.
- FIG. 13 is a flowchart of a PacketArrival function for one of the computation decoders shown in FIG. 2.
- FIG. 14 is a graph of a loss utility function U L having a loss rate less than or equal to ten (10).
- FIG. 15 is a graph of a Redundancy utility function U R having a Redundancy less than or equal to three (3).
- FIG. 16 is a graph of delay utility function U D having a delay less than or equal to one (1) second.
- FIG. 17 is a graph of modified loss utility function U L * of the utility function U L shown in FIG. 14.
- FIG. 18 is a graph of a modified redundancy utility function U R * of the redundancy utility function U R shown in FIG. 15.
- FIG. 1 shows an overview of a system 10 for communicating a real time media input 25 and incorporating a preferred embodiment of the present invention.
- the system 10 includes a sending device 20 , a first processing hub 30 , a transporting network 35 , a mapping service 31 , a second processing hub 40 and a receiving device 45 .
- the sending device 20 is a calling device that generates the real time media input 25 .
- the real time media input 25 is a telephone call.
- the sending device 20 generates other types of real-time media inputs such as video, multimedia, streaming applications, or a combination thereof.
- the input 25 is communicated over a telephone line 26 to the first processing hub 30 .
- the first hub 30 is a local hub and is commercially available from U.S. Robotics of Skokie, Illinois as U.S. Robotics EdgeserverTM bearing part number 1098-0.
- the first hub 30 processes the input 25 and converts the input 25 into a form that can be transported by the transporting network 35 .
- the first hub 30 may include an encoding device for encoding the input 25 into a digital format.
- the hub 30 may then compress the digital format into a plurality of frames. These frames could be packetized into a sequence of data packets 36 comprising a plurality of data packets 33 .
- the data packets 33 are then transported by the transporting network 35 to the second processing hub 40 .
- the mapping service 31 maps the phone number being called to an Internet Provider (IP) address of a receiving hub.
- IP Internet Provider
- the receiving hub is a hub closest to the party receiving the call.
- the receiving hub is the second processing hub 40 .
- the transporting network 35 transports the data packet sequence 36 to the selected receiving hub 40 . Because various packets of the sequence 36 may be dropped or lost during transportation, the first packet sequence 36 may differ from the second sequence of data packets 37 .
- the data packets 34 comprising sequence 37 are communicated to the second calling device 45 over a telephone line 41 .
- the first device 20 can place a telephone call to the second calling device 45 in the following maimer.
- Calling device 20 activates an Internet account by calling a toll free number.
- the Internet account then prompts the calling device 20 for identification.
- An identification number such as a phone card number or a credit card number, is entered.
- the calling device 20 is then provided a number of a local processing hub (i.e., the first processing hub 30 ) based on the caller's identification number.
- the first hub 30 is consequently made aware that there is a new user in its area. Once the caller has been identified, the caller 20 calls its assigned local processing hub. The hub will then recognize the caller based on the caller's identification number.
- One advantage of this proposed identification scheme is that it facilitates billing the caller for usage and other types of service charges.
- the caller After identifying itself to the first hub 30 , the caller is asked to enter the phone number that the caller wishes to call.
- the mapping service 31 maps the phone number to an IP address of a sending hub closest to the caller. This phone number facilitates selecting a receiving hub as close as possible to the location of the other party to the call. The selected receiving hub then places a call to the receiving party so that the call can proceed. The caller's voice is then transported as data packets between the sending and the receiving hub.
- One advantage of the system shown in FIG. 1 is that the system samples and compresses the communicated information in close proximity to the transporting network. Preferably, sampling and compressing are performed in the processing hubs 30 , 40 . By performing these tasks inside the processing hub as opposed to, for example, inside a PC, more computation power is available at the sending or receiving end of the call. Therefore, more complex encoders and transporting schemes can be utilized. More sophisticated billing schemes can also be implemented. For example, the price of a telephone conversation can be correlated with the quality and the delay of that particular telephone call. System 10 can also accurately measure one-way delay and can therefore compensate the transportation of data packets based on the varying transporting characteristics of the transporting network 35 .
- the transporting network 35 is a packet switched network and preferably the Internet.
- An Internet is one type of packet switched network: it is a network of networks.
- the Internet is divided into thousands of autonomous systems (“AS”) that are individual networks controlled by an administrative agency.
- AS autonomous systems
- the range of AS sizes can vary greatly.
- LAN local area network
- AS a large AS, such as a telephone company ATM backbone spanning the breadth of the United States is also an AS. Therefore, the term Internet, as that term is used herein, is a meta-network in that it is a scheme for inter-connecting different AS's such that data can be transported between AS's.
- the term “host,” as used herein, is a computer or access point having a unique Internet Protocol (IP) address.
- IP Internet Protocol
- AS's that can be used to transport the stream of data packets between the first and second hub 30 , 40 include nationwide backbones, regional backbones, local Internet providers, online services, Wide Area Networks (WANs), LANs, Intranets, university networks and corporate networks.
- the transporting network 35 transports the sequence of data packets from the first processing hub 30 to the second processing hub 40 .
- the second processing hub 40 receives the sequence of data packets 37 .
- the sequence received 37 differs from the sequence transported 36 because of packet loss and packet delays that frequently occur in packet switched networks.
- the received data packets 33 are decoded by the second hub 40 .
- the second hub first unpacks the packets and then decompresses this information. This decompressed information is then ordered within a buffer.
- the buffer of information is then played out and converted to an analog signal 41 .
- the analog signal 41 is then sent over telephone line 42 .
- the second hub 40 may call the second calling device 45 .
- the second calling device 45 then plays out the analog input 26 .
- the second calling device 45 can generate information and transport this information to the first calling device 20 in a similar fashion.
- the first and the second calling devices 20 , 45 of system 10 shown in FIG. 1 are each associated with telephone call participants. Participants can therefore place telephone calls over a regular telephone rather than have to use a PC speakerphone system. Because telephones are generally more common than PCs, the proposed system 10 will be more available to the public. Telephones also provide a more natural user interface to those individuals who do not use or who are uncomfortable using computers.
- the sending and receiving devices 20 and 45 are electronic communicating devices such as modems, facsimile machines, network computers, PCs, pagers, hand-held communicating devices, personal distal assistants or like devices that communicate audio, video, multimedia or similar applications.
- an interactive transporting environment requires bi-directional transportation of information.
- FIG. 1 Such an interactive environment is shown in FIG. 1 where the first calling device 20 has been described as both the sender and the receiver of telephone calls.
- packet transportation from the first hub 30 acting as a sender to the second hub 40 acting as a receiver will be discussed.
- FIG. 2 illustrates a communication channel 60 in accordance with the system shown in FIG. 1.
- the communication channel includes a sender 65 and a receiver 75 .
- the sender 65 may, for example, be included within the first hub 30 shown in FIG. 1.
- the receiver 75 may be included within the second hub 40 shown in FIG. 1. It should be realized, however, that in an interactive environment where information is transported bi-directionally, a processing hub will normally include both sender 65 and receiver 75 thereby enabling the hub to receive and transmit information simultaneously.
- the sender 65 includes an encoder 80 coupled to a packetizer 90 .
- a first stream of data packets 95 generated by the packetizer 90 is transported by a transporting network 35 .
- the receiver 75 receives a stream of data packets 96 .
- the stream of data packets 96 is supplied to a real decoder 130 and a number of computation decoders 150 .
- the real decoder 130 includes a depacketizer 135 coupled to a buffer 140 .
- the depacketizer 135 operates in accordance with a first variable.
- the first variable is an actual Redundancy variable 115 .
- the size of the real decoder buffer 140 varies in accordance with a BufferLength variable 174 .
- the buffer 140 is coupled to a decoder 162 .
- the decoder 162 provides a digital input 163 to a digital-to-analog converter 164 (i.e., D/A converter 164 ).
- the D/A converter 164 provides signal 165 to the second calling device 166 for playout.
- the first variable is a vector of values. These vectors may represent a plurality of variables providing further control of the communication channel. For example, such variables could be used for identifying the type of codings above being used by the sender, a redundancy parameter, and other types of control identifiers.
- the computation decoders 150 are arranged in parallel to the real decoder 130 . In this configuration, the computation decoders 150 and the real decoder 130 receive the stream of data packets 96 .
- the stream 96 comprises transported data packets 97 .
- Each computation decoder 150 includes a computation decoder depacketizer 152 and a computation decoder buffer 154 .
- a first calling device 70 generates a real time media signal 72 , preferably a telephone call.
- the signal 72 is video, multimedia, a streaming application or a combination thereof.
- the signal 72 is communicated to an analog-to-digital converter 82 (i.e., A/D converter 82 ).
- the A/D converter 82 converts the signal 72 to a digital signal 83 .
- the digital signal 83 is a digital speech wave form.
- the digital signal 83 is communicated to an encoder 80 of the sender 65 .
- the digital signal 83 is communicated to the encoder 80 over a telephone line.
- the digital input 83 (preferably in Pulse Code Modulated (PCM) form) is compressed and partitioned by encoder 80 into a sequence of frames 85 .
- the encoder 80 encodes the digital signal 83 .
- the encoder 80 is an ITU voice encoder complying with Recommendation G.723.1.
- Recommendation G.723.1 describes a code excited linear predictive encoder (CELP).
- CELP code excited linear predictive encoder
- This recommendation G.723.1 specifies a coded representation used for compressing speech or another audio signal component of multimedia services at a low bit rate as part of the overall H.324 family of standards.
- Recommendation G.723.1 is entitled “DUAL RATE SPEECH ENCODER FOR MULTIMEDIA COMMUNICATIONS TRANSMITTING AT 5.3 & 6.3 KBITS/S” and is published by the Telecommunication Standardization Sector of the ITU.
- Recommendation G.723.1 is herein entirely incorporated by reference.
- voice encoders complying with other standards or specifications can be used.
- the digital input 83 to the encoder 80 is a digital speech waveform sampled at 8000 Hz.
- Each sample of the input 83 is represented by a signed 16 bit integer.
- the encoder 80 preferably the G.723.1 encoder, segments the input 83 into frames 85 .
- each frame is 30 milli-seconds (ms) in length. At the preferred sampling rate of 8000 Hz, 30 ms represents 240 samples.
- the preferred G.723.1 encoder can operate at two different bit rates, a low rate of 5.3 kbits/seconds or a high rate of 6.3 kbits/seconds.
- a low rate of 5.3 kbits/seconds or a high rate of 6.3 kbits/seconds.
- 480 bytes i.e., 240 samples times 2 bytes/sample
- the encoding results in a quality that is close to toll quality.
- the low rate setting of 5.3 kbits/s 480 bytes are compressed to 20 bytes. Therefore, between the low and high rate setting, the compression ratio varies from 20 to 24.
- the encoder 80 utilizes silence detection.
- the preferred G723.1 silence detection uses a special frame entitled Silence Insertion Descriptor (SID) frame.
- SID frame generation is described in Recommendation 6,723.1 which has been herein entirely incorporated by reference.
- SID Silence Insertion Descriptor
- An SID frame defines when a silence begins.
- After the encoder 80 transmits an SID frame no further voice data frames are transmitted until the current silence ends. Updated SID frames may, however, be sent. This silencing technique reduces the required overall transfer rate.
- silence detection allows for a dynamic adjustment of the depth of the real decoder buffer 140 .
- the communication channel 60 can thereby compensate for varying transportation characteristics of the transport network 35 .
- the packetizer 90 packets the frames 85 into a plurality of data packets 92 .
- the packetizer 90 places a time stamp and a sequence number into each data packet 92 .
- the time stamp identifies the time a specific data packet 92 was created.
- the sequence number identifies data packet ordering.
- Each data packet 92 includes both a current frame as well as redundant information such that a number of previously packeted frames might be reconstructed if some frames are lost during transportation.
- the number of previous frames or redundant frames is channel coded according to the actual Redundancy variable 115 of the communication channel 60 .
- the actual Redundancy 115 is the variable that determines the number of previous frames packet into each data packet 92 .
- the data packets 92 are ordered in a data packet sequence 95 and transported by the transporting network 35 to the receiver 75 .
- Each data packet time stamp enables the receiver 75 to evaluate certain dynamic transporting characteristics of the transporting network 35 . These transporting characteristics determine how the packetizer 90 packetizes the frames 85 and how the receiver 75 unpacks these frames. These varying transporting characteristics can include such characteristics as the standard deviation of one-way delay or the round trip time for each transported data packet 97 .
- the round trip time is calculated by transporting a copy of the time stamp back to the sender 65 and comparing the received time with the timestamp value.
- the standard deviation of one-way delay is typically approximated by averaging the absolute value of differences between time stamp values and received times for each packet 97 .
- RTP real time protocol
- data packet sequence numbers and time stamps are placed within the RTP header. The sequence numbers and timestamps do not, therefore, need to be reproduced in the data packet payload.
- Other transport protocols that contain timestamps and sequence number information can also be used in place of the RTP protocol.
- the receiver 75 receives a sequence of data packets 96 .
- This sequence of data packets 96 may vary from the sequence of data packets 95 originally communicated to the transporting network 35 .
- the variance between the two data packet sequences 95 , 96 is a function of varying transporting characteristics such as packet loss and packet transport times.
- the receiver 75 receives packets out of order vis-a-vis other data packets comprising the originally transported packet sequence 97 .
- the packetizer 90 adds sequence numbers to the frames 85 before the frames are packetized.
- the receiver 75 has a real decoder buffer 140 that stores the data from the unpacked frames. As long as the sequence number of an arriving packet 97 is greater than the sequence number of the frame being played out by the buffer 140 , the sequence number is used to put the unpacked frame at its correct sequential position in the real decoder buffer 140 .
- the larger the size of the buffer 140 the later a frame can arrive at the receiver 75 and still be placed in a to-be-played-out frame sequence.
- the larger the overall delay can be in transporting the input 83 from the sender 65 to the receiver 75 .
- the receiver 75 includes a real decoder 130 , a decoder 162 and a plurality of computation decoders 150 .
- the real decoder depacketizer 135 receives the data packet sequence 96 . Initially, the depacketizer 135 reads the actual Redundancy variable 115 contained in each data packet 97 . Using the actual Redundancy variable 115 , the depacketizer 135 unpacks the data packets 97 and recovers the frames 85 .
- the frames 85 include both current and redundant frames.
- the real decoder 130 reads the sequence number and the time stamp of a current frame. Redundant frames associated with the current frame have the same time stamp as the current frame since, within a given packet, redundant and current frames were both originally communicated from the packetizer 90 at approximately the same point in time. Since the order or sequence of the redundant frames is known, the redundant frame sequence numbers can be inferred from the current frame sequence number.
- each frame together with its corresponding time stamp and sequence number, defines a node 137 .
- the nodes 137 are forwarded to a real decoder buffer 140 for buffering. Redundant frames are not buffered if an original frame has been previously buffered.
- the buffered frames are then passed on to a decoder 162 .
- the decoder 162 decompresses the frames 142 .
- the decompressed frames 163 are then forwarded to a digital-to-analog converter 164 (i.e., D/A converter 164 ).
- the D/A converter 164 converts the digital data 163 to an analog output 165 .
- This analog output 165 represents the original analog input 72 generated by the first calling device 70 .
- the analog output 165 is forwarded to the second calling device 166 where the output 165 is then played out.
- the present communication channel 60 offers a number of advantages.
- the present communication channel can adapt to varying transporting dynamics and conditions of the transporting network 35 .
- the network transporting dynamics can be assessed by a packet delay distribution and a packet loss percentage, both of which generally vary over time.
- the real decoder 130 have a buffer 140 that has a variable buffer length.
- the buffer length will vary in accordance with the dynamic transporting characteristics of the network 35 .
- the buffer length is proportional to the variance in delay experienced by the transported data packets 97 .
- a non-guaranteed packet switched transporting network such as transporting network 35 , having a highly varying data packet delay results in an increased buffer length. Conversely, where a transporting network experiences a more constant data packet delay, the buffer length will be decreased.
- a buffer length of X milliseconds is employed where X is a dynamic parameter. Utilizing a buffer having a dynamic buffer length of X milliseconds, after the arrival of an unpacked node 137 from the real decoder depacketizer 135 , X milliseconds must on average time out before the buffer 140 can start playing out at a constant rate of 1 frame per 30 milliseconds. Alternatively, the buffer 140 plays out at the frame rate used by the encoder 80 .
- the buffer 140 is implemented as having a doubly linked list (LL) structure.
- the nodes 137 are ordered according to their respective sequence number.
- Each node 137 contains a pointer that points to the preceding and succeeding nodes in the structure.
- Each node 137 is inserted into the buffer 140 at the appropriate linked list position. If a node already exists in the buffer 140 , the redundant node is discarded.
- the buffer 140 plays out frames 142 at periodic instances of time.
- the buffer 140 plays out one frame every 30 ms.
- the receiver 75 contains N computation decoders 150 . These N computation decoders 150 are arranged in parallel with the real decoder 130 .
- the number of computation decoders N is a product of the cardinality of the domain of two variables: the Redundancy and the BufferLength.
- the Redundancy defines the number of previous frames packeted into each data packet 92 .
- the BufferLength variable defines the number of nodes 137 buffered by the real decoder buffer 140 before play-out occurs. As the BufferLength increases, fewer nodes 137 will arrive too late to be played-out by the buffer 140 .
- the computation decoders 150 receive and observe the data packets 97 of the incoming data packet sequence 96 .
- Each computation decoder 150 includes a computation decoder depacketizer 152 coupled to a computation decoder buffer 154 .
- the computation decoders 150 operate differently than the real decoder buffer 140 .
- One difference is that the computation decoders 150 do not read the actual Redundancy variable 115 from an arriving data packet 97 . Rather, each individual computation decoder uses an assigned fixed Redundancy [i] variable 153 .
- This fixed Redundancy 153 is used to extract the frames 85 from the transported data packet 97 .
- the fixed Redundancy [i] variable is a hypothetical Redundancy value and is used by computational decoder [i], and is an index to the computation decoders [i . . . N].
- Each computation decoder 150 computes various characteristics of the transporting network 35 . Preferably, each computation decoder 150 computes an AveDelay [i] and an AveLoss [i]. Each computation decoder 150 also has an assigned Rate [i] 151 .
- the AveLoss [i] parameter 160 is a measure of the average number of the originally transported data packets 95 lost during transportation.
- the AveLoss [i] parameter takes into account the data packets 92 originally transported but accounted for as being lost during transportation since these packets were received too late to be played out by the buffer 140 .
- AveLoss [i] 160 provides one method to quantify a difference between the data packets 95 originally sent by the sender 65 and the data packets 97 actually received by the receiver 75 .
- the AveDelay [i] parameter 158 is a measure of the average time it takes for the data packets 92 to be transported from the sender 65 to the receiver 75 .
- the AveDelay [i] parameter 158 preferably also includes the time required for the buffer 140 to playout the frames 142 . These measures are computed from the time stamp and sequence number associated with the transported data packets 92 .
- AveDelay [i] 158 is equal to the sum of the one way delay plus the receiver buffer time.
- the receiver buffer time can be estimated by multiplying the receiver Buffer Length by the period of the frame rate.
- the one way delay is estimated by adding an estimate of the network delay to the Receive Buffer Delay.
- AveLoss [i] 160 is determined by the flow chart algorithm of the computation decoder as will be discussed with reference to FIG. 12.
- the fixed Redundancy variable 153 associated with each computation decoder can be greater than, less than, or equal to the actual Redundancy variable 115 .
- a particular fixed Redundancy [i] variable 153 of a corresponding computation decoder [i] 150 is greater than the actual Redundancy variable 115 , some of the frames 85 of the data packet 97 are unavailable to the computation decoder 150 .
- the computation decoder 150 only requires the time stamp and the sequence number of a received data packet 97 .
- the time stamp and the sequence number for all the redundant frames can be inferred. These values can be inferred since time stamps remain unchanged and sequence numbers are in sequential order for the hypothetical case of the computation decoder. This is true even when the actual Redundancy parameter 115 and a fixed Redundancy [i] parameter 153 differ.
- Each computation decoder has three unique values associated with it.
- the three values of each computation decoder AveDelay, AveLoss and Rate defines the utility of the computation decoder for a given data packet transportation.
- the three values AveDelay [i] 158 , AveLoss [i] 160 and Rate [i] 151 of each computation decoder 150 are analyzed by a utility function 170 .
- the utility function 170 selects the optimal computation decoder that would have resulted in the highest utility for a transported data packet.
- the utility of a particular computation decoder 150 and therefore the utility of the overall receiver 75 , is application specific.
- the utility is a function of the average delay AveDelay 158 , the average loss rate AveLoss 160 and the Rate 151 .
- Rate is a measure of the bandwidth required to transport the media stream which is increasing with redundancy.
- the AveLoss rate is a function of the actual Redundancy parameter 115
- the utility function 170 is preferably a function of three network transmission characteristics represented by these three variables.
- the utility function 170 preferably has the following form U(AveDelay, AveLoss, Redundancy) and if seperability is desired, it may be expressed as follows:
- U L (AVeLOSS) is the loss utility function
- U D (AveDelay) is the delay utility function
- U R Redundancy
- the utility function can be expressed in other forms, such as a non-seperable, non-linear function in the form of a table.
- the general purpose of the utility function 170 is to rate the different type computation decoders 150 . In this manner, the computation decoding values of Redundancy [i] and BufferLength [i] that would have optimized data packet transportation at a given time is selected. These optimal values determine the new values for the actual Redundancy 115 and the BufferLength 174 .
- the utility function 170 is application specific and can be modified to best fit the type of analog input 72 being transported.
- the application's specific nature of the utility function can be explained by way of the following example. If a specific type of application calls for a maximum loss rate of 10%, a loss utility function U L can be represented by the graph shown in FIG. 14. As shown in this graph, as long as the loss rate is less than or equal to 10%, the loss utility function U L will be equal to 1. Any loss rate greater than 10% will result in the loss utility function U L to be equal to zero (0).
- redundancy utility function U R can be expressed graphically as shown in FIG. 15. According to FIG. 15, as long as the redundancy utility function U R is equal to or less than three (3), the utility function U R will equal one (1). Any Redundancy greater than three (3) will result in a redundancy utility function U R equal to zero (0).
- the third concern in this example is the data packet transportation delay.
- the example will require a delay of less than or equal to one (1) second. Any greater delay will result in the delay utility U D equal to zero (0). This requirement can be graphically represented as shown in FIG. 16.
- the decoder 150 using the lesser amount of redundancy U R or that results in the smaller loss rate U L is selected.
- this selection process is accomplished by slightly altering the loss and redundancy utility function U L * and U R *, respectively.
- a modified loss rate U L * and a modified Redundancy rate U R * is shown in FIGS. 17 and 18, respectively.
- the Redundancy value 172 and BufferLength value 174 of the optimal computation decoder are utilized as follows. First, the Redundancy value 172 is packetized into a feedback data packet 118 that is transported to the packetizer 90 of the sender 65 . The sender 65 adjusts the actual Redundancy variable 115 based on the new Redundancy value 172 .
- the optimal BufferLength value is communicated to the real decoder buffer 140 .
- the real decoder buffer 140 uses the preferred BufferLength value 174 to buffer the nodes 137 .
- the Redundancy and BufferLength are chosen periodically with intervals of one (1) to ten (10) seconds in a typical implementation.
- the fixed Redundancy values 153 and the fixed BufferLength values 156 associated with the computation decode 150 are constant. These values are therefore not adjusted according to the transmission characteristics of the transporting network 35 . Rather, it is the function of all the computation decoders 150 , by using all possible Redundancy value 153 and BufferLength 156 combinations, to determine an optimal value of these two variables based on the transport characteristics of the network 35 at a given time.
- the preferred computation decoder 150 has highest utility for a given data packet transportation and therefore provides the best choice of system variables given the network conditions at a given time. This allows for the flexibility of using a variety of utility functions for different types of applications. For example, for a streaming application, or a one way communication, the AveDelay U D can be quite a bit larger than for an interactive application. On the other hand, a streaming application may require a higher quality than the interactive call or video conference application.
- the real decoder decompression scheme matches the encoder scheme used to compress the input 83 .
- the decoder 162 is a G.723.1 decoder where the input to the decoder is a frame stream.
- the output of the decoder 160 is a waveform in the same format as the analog input 83 for the G.723.1 encoder.
- FIG. 3 illustrates the structure of a data packet 92 transported by the communication channel shown in FIG. 2.
- each data packet has a data packet header that is a thirty-two (32) bit word containing a Redundancy parameter 115 , a current frame and a plurality of redundant frames.
- the RT Header 110 is a Real Time Protocol header containing a sequence number and time stamp.
- the Real Time Protocol contains a field which identifies how the remainder of the data packet is interpreted. Packets of 95 or 96 may be Real Time Protocol packets which contain packets from the protocol described here.
- the message data 108 reads 0000 for data packets 92 transmitted from the sender 65 to the receiver 75 .
- the message data 108 of the packet reads 0001.
- This message data field allows the sender 65 and the receiver 75 to differentiate between data and feedback packets.
- the feedback packet 175 preferably does not contain a frame length or frame data. Rather, the feedback packet 175 contains information relating to the desired value to be used for the actual Redundancy variable 115 .
- the header spare 109 is reserved for later use for data packets from sender to receiver.
- the Redundancy 115 variable represents the number of additional previous frames that each data packet 95 contains.
- the Frame Length 120 represents the length of the following frame data field in bytes.
- FIG. 4 illustrates an example of a preferred order of the redundant frames in five levels of data packets wherein the actual Redundancy variable 115 is set equal to two (2).
- the packetizer 90 packs two previously transmittal frames into each data packet 95 .
- Packet n 192 contains Frame n 186 and its two previous frames: Frame n ⁇ 1 184 and Frame n ⁇ 2 182 .
- Packet n+1 194 contains Frame n+1 188 , along with its two previous frames: Frame n 186 and Frame n ⁇ 1 188 .
- Packet n+2 196 contains Frame n+2 190 along with its two previous frames: Fame n+1 188 and Frame n 186 .
- each packet 95 includes a current frame along with the two previous frames.
- FIG. 5 illustrates an example of a preferred double linked list (LL) 200 .
- This example has a Start node 210 having a sequence number equal to 10 and a Stop node 220 having a sequence number equal to 13 .
- the preferred real decoder buffer implementing the LL 200 keeps track of the first or Start node and the last or Stop node.
- the LL 200 contains all the nodes having sequence numbers that fall between a Start node and a Stop node.
- the LL creates all the nodes falling between and including the Start and Stop nodes, i.e., 10 , 11 , 12 , 13 (i.e., element numbers 210 , 240 , 230 and 220 , respectively). All four nodes 10 , 11 , 12 , and 13 will be created even though frame sequence number 11 and frame sequence number 12 have not yet been received. The created nodes 230 and 240 are marked as missing.
- FIG. 7 illustrates a flowchart of a PutNode function 300 for the real decoder 130 shown in FIG. 2.
- the PutNode function 300 After receiving a node 137 from the depacketizer 135 , the PutNode function 300 must determine where to put this node.
- the PutNode function 300 first determines whether the real decoder buffer 140 is empty 301 . If the real decoder buffer is empty, the function 300 creates the buffer using the new node at step 303 . A Start and a Stop pointer associated with this new node now point to this new node at step 303 .
- the function 300 then updates the real decoder buffer depth and the real decoder buffer state at step 304 .
- the PutNode function 300 determines whether the new node should be placed to the left of the existing Start node 305 . If the new node should be placed to the left of the existing Start node, then at step 307 the missing buffer nodes are created between the existing Start node and the new node. The new start will now point to the new node. The PutNode function 300 then updates the real decoder buffer depth and the real decoder buffer state at step 304 .
- the PutNode function 300 determines whether the new node should be placed to the right of the existing Stop node. If the new node should be placed to the right of the existing Stop node, then the missing buffer nodes between the Stop node and the new node are created at step 310 . The Stop pointer will now point to the new node 310 . The PutNode function 300 then updates the real decoder buffer depth and the real decoder buffer state at step 304 .
- the function 300 finds the existing buffer node with the same sequence number at step 312 .
- the PutNode function 300 determines whether the buffer node is marked as missing at step 314 . If the buffer node has been marked as missing, the buffer node is replaced by the new node at step 315 .
- the function 300 then updates the real decoder buffer depth and the real decoder buffer state at step 304 . If the buffer node has not been marked as missing, then the function 300 updates the real decoder buffer depth and real decoder buffer state at step 304 without replacing the buffer node.
- the second function for accessing the LL 200 is the GetNode function.
- the buffer 140 utilizes the GetNode function for retrieving a node 137 having the smallest sequence number.
- a flowchart of the GetNode function 325 is illustrated in FIG. 6.
- the GetNode function 325 first determines whether the real decoder buffer 140 is empty. If the buffer 140 is empty, then at step 326 a node is created. The newly created node is marked as missing and the node is returned to the buffer 140 . If the buffer 140 is not empty, then at step 329 the node having the smallest sequence number is returned. The GetNode function 325 then adjusts the real decoder buffer depth and buffer state at step 331 .
- the buffer 140 can be in one of three different states: Fill, Normal and Drain. These transitions are controlled by the SetNode and PutNode functions. These three states Fill, Normal and Drain can be represented by the state transition diagram 350 illustrated in FIG. 8. This state transition diagram 350 shows how the real decoder buffer 140 changes state, depending on the buffer depth and the current state of the buffer.
- the objective of the state diagram 350 is to maintain the buffer in its Normal state 354 .
- the objective of the State diagram 350 is to maintain the buffer in its Normal state 354 .
- the objective of the Fill state is to bring the buffer depth back to the Normal state 354 .
- This objective may be achieved by artificially lengthening the next silence period until enough data packets arrive to return the buffer depth back to the Normal state 354 .
- the buffer state remains in the Normal state 354 .
- the buffer state switches back to the Fill state 352 . If the buffer depth increases above the High watermark as shown by transition 360 , the buffer state changes to the Drain state 356 .
- the objective of the Drain state 356 is to shorten the silence periods and therefore reduce the buffer depth until it is returned to the Normal state 354 . As long as the buffer depth is greater than N, the buffer will remain in the Drain state 356 . Once the buffer depth is less than or equal to N as shown in transition 357 , the buffer will become Normal again.
- the buffer control attempts to keep the buffer depth around the set BufferLength. This can be accomplished by setting the Normal water mark (N) equal to the BufferLength, the Low watermark (L) equal to half of the BufferLength and the High watermark (H) equal to 1.5 times the BufferLength.
- N Normal water mark
- L Low watermark
- H High watermark
- the first event is the arrival of the data packet 96 .
- the second event is defined as a TimeOut.
- the arrival of the data packet is defined as a PacketArrival.
- the arrival of transported data packets 96 is an asynchronous event and occurs whenever the real decoder 130 and computation decoders 150 receive a data packet.
- FIG. 11 provides a flowchart for the PacketArrival function 420 of the real decoder 130 .
- the real decoder 130 reads the actual Redundancy variable 115 at step 421 and unpacks the next frame at step 423 .
- the PacketArrival function 420 determines whether the frame has arrived in time for buffer play out or if the buffer is already empty at step 425 .
- a node is created at step 427 .
- the PutNode function 300 as described with reference to FIG. 7 is then implemented at step 479 . If the frame arrives late or if the buffer is not empty, it is determined whether any redundant frames are left 431 . If redundant frames remain at step at step 431 , then the frame unpacking and node generation process returns to step 423 .
- FIG. 13 provides a flowchart for the PacketArrival function 440 of the computation decoders 150 .
- the computation decoder 150 unpacks the frames at step 441 .
- the PacketArrival function 440 determines whether the unpacked frame was received in time for play out and if the buffer is empty. If both are true, then a node is created at step 445 and the PutNode function 300 (shown in FIG. 7) is implemented at step 447 .
- the PacketArrival function 440 then proceeds to determine whether any other redundant frames remains at step 449 . If more frames remain, the PacketArrival function 440 returns to step 441 .
- the PacketArrival function 440 determines whether any redundant frames are remaining at step 449 . If any frames are left at step 449 , then the process returns to step 441 and the next frame is unpacked. In the computational decoder, the actual data frames do not need to be stored in the buffer. Instead, the buffer may be marked with an indication of a data frame being unpacked and stored.
- the second event associated with the real decoder 130 is defined as a TimeOut.
- the TimeOut event is periodic and fixed to the frame size of the encoder 80 . As previously discussed, the frame size and resulting TimeOut for the preferred G.723.1 system occurs every 30 milliseconds.
- FIG. 9 illustrates a flow chart for the TimeOut event 380 .
- the TimeOut function 380 first determines the state of the buffer. If the buffer 140 is in the Fill state 352 , the TimeOut function 380 proceeds to determine whether a silence period is detected at step 384 . If a silence period is detected, the silence period is extended at step 386 until the buffer state switches to Normal 354 . The function then returns to step 400 and executes the PlayNode function as previously described and shown in FIG. 12. If a silence period is not detected, the TimeOut function 380 implements the GetNode function 325 as previously described with reference to FIG. 6. After the GetNode function is implemented at step 325 , the TimeOut function 380 returns to step 400 , the PlayNode function, and the frame with the lowest sequence number is taken out of the buffer and played out.
- the TimeOut function 380 determines that the buffer is in the Drain state 356 , the GetNode function as previously described with reference to FIG. 6 is implemented at step 325 . After the GetNode function at step 325 is implemented, the TimeOut function 380 proceeds to step 396 to determine whether a silence period is detected and whether the buffer 140 is in the Drain state. If both are detected, the function 380 returns to the GetNode function 325 . If both are not detected, the TimeOut function 380 returns to the PlayNode function at step 400 .
- TimeOut function 380 determines that the buffer is in the Normal state 354 , the function 380 proceeds to the GetNode function 325 . The TimeOut function 380 then returns to the PlayNode function at step 400 .
- PlayNode functions There are two different types. The first is the real decoder 130 PlayNode function 390 is illustrated in FIG. 10. The second is the computation decoder PlayNode function 400 , illustrated in FIG. 12. The purpose of the first PlayNode function 390 is to send the frame data to the playout decoder 160 which is called for whenever a TimeOut occurs. It is therefore invoked periodically with the period being equal to the encoder 80 frame length.
- the PlayNode function 390 first determines whether a frame is missing at step 391 . If no frame is missing, the function 390 proceeds to step 393 where a loss bit is set to zero (0). Next, the frame is played at step 395 . The AveLoss and AveDepth statistics are then updated at step 397 .
- step 391 determines at step 391 that a frame is missing
- the function 390 proceeds to step 392 where the loss bit is set to one (1).
- the function 390 determines whether a silence period is detected at step 394 . If a silence period is detected, then the silence is extended at step 398 . If a silence period is not detected at step 394 , then a frame is interpolated at step 396 . This effectively plays a frame that is an estimation of the missing frame.
- the AveLoss and AveDepth statistics are then updated at step 397 .
- the second type of PlayNode function 400 is that of the computation decoders 150 and is illustrated in FIG. 12.
- the second Playnode function 400 first determines at step 402 whether a frame is missing.
- the second Playnode function 400 sets a loss bit equal to one (1) if the frame is missing at step 404 .
- the loss is set to zero (0) if the frame is not missing.
- the second PlayNode function 400 then updates the AveLoss and the AveDepth statistics of the transporting network with these new values at step 408 .
- the preferred utility function 170 evaluates or maps the new value of the variable Bufferlength 174 and a new value of the variable Redundancy 115 .
- the variable BufferLength 174 is altered by first changing the three watermarks as described with reference to the buffer state diagram 350 and then changing the buffer states 352 , 354 and 356 .
- the Normal watermark value in the real decoder will change to the new BufferLength variable.
- Other watermark values (High and Low) may be determined either by alogrithm or by copying some values from computational decoder which yielded the largest utility parameter. If a larger buffer state is changed to a smaller one, then the adjustment of the buffer state may result in a Drain state 356 .
- the buffer 140 starts shortening the silence periods. If the buffer is increased, then the adjustment of the buffer state may result in a Fill state 352 . Consequently, a subsequent silence period will be extended until the buffer fills up to the Normal watermark 354 . The new Redundancy variable will be communicated back to the sender.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
A method and apparatus for communicating a real time media input over a network. The apparatus encodes the input into data packets having a number of frames ordered according to a first variable. A receiving device unpacks and buffers the unpacked data packets for playout according to a second variable. The receiving device generates utility parameters for evaluating a dynamic characteristic of the network that transports the data packets. The receiving device selects a preferred utility parameter and adjusts the first and second variable according to the selected utility parameter. The method includes encoding an analog input into data packets that are transported to a receiving device. The method also includes unpacking the data packets, buffering the unpacked data packets according to a second variable, and generating at least two utility parameters that represent a dynamic characteristic of a network. The method also includes selecting a preferred utility parameter and adjusting the first and the second variables according to the selected preferred utility parameter.
Description
- A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
- 1. Field of the Invention
- This invention relates to the field of telecommunications and more specifically to a method and apparatus for real time communication over packet networks.
- 2. Description of Related Art and Advantages of the Invention
- Real time communications such as audio or video can be encoded using various compression techniques. The encoded information can then be placed in data packets with time and sequence information and transported via non-guaranteed Quality of Service (QoS) packet networks. Non-guaranteed packet switched networks include a Local Area Network (LAN), Internet Protocol Network, frame relay network, or an interconnected mixture of such networks such as an Internet or Intranet. One underlying problem with non-guaranteed packet networks is that transported packets are subject to varying loss and delays. Therefore, for real-time communications, a tradeoff exists among the quality of the service, the interactive delay, and the utilized bandwidth. This tradeoff is a function of the selected coding scheme, the packetization scheme, the redundancy of information packeted within the packets, the receiver buffer size, the bandwidth restrictions, and the transporting characteristics of the transporting network.
- One technique for transporting real time communication between two parties over a packet switched network requires that both parties have access to multimedia computers. These computers must be coupled to the transporting network. The transporting network could be an Intranet, an Internet, wide area network (WAN), local area network (LAN) or other type of network utilizing technologies such as Asynchronous Transfer Mode (ATM), Frame Relay, Carrier Sense Multiple Access, Token Ring, or the like. As in the case for home personal computers (PCs), both parties to the communication may be connected to the network via telephone lines. These telephone lines are in communication with a local hub associated with a central office switch and Network Service provider. As used herein, the term “hub” refers to an access point of a communication infrastructure.
- This communication technique however, has a number of disadvantages. For example, for a home-based PC connected to a network using an analog telephone line, the maximum bandwidth available depends on the condition of the line. Typically, this bandwidth will be no greater than approximately 3400 Hz. A known method for transmitting and receiving data at rates of up to 33.6 kbits/second over such a connection is described in Recommendation V0.34, published by the International Telecommunication Union, Geneva, Switzerland.
- Aside from a limited bandwidth, various delays inherent in the PC solution, such as sound card delays, modem delays and other related delays are relatively high. Consequently, the PC-based communication technique is generally unattractive for real-time communication. As used herein, “real-time communication” refers to real-time audio, video or a combination of the two.
- Another typical disadvantage of PC-based communication, particularly with respect to PC-based telephone communications, is that the communicating PC receiving the call generally needs to be running at the time the call is received. This may be feasible for a corporate PC connected to an Intranet. However, such a connection may be burdensome for a home based PC since the home PC may have to tie up a phone line.
- Another disadvantage is that a PC-based conversation is similar to conversing over a speakerphone. Hence, privacy of conversation may be lost. Communicating over a speakerphone may also present problems in a typical office environment having high ambient noise or having close working arrangements.
- In addition, PC-based telephone systems often require powerful and complex voice encoders and therefore require a large amount of processing capability. Even if these powerful voice encoders run on a particularly powerful PC, the encoders may slow down the PC to a point where the advantage of document sharing decreases since the remaining processing power may be insufficient for a reasonable interactive conversation. Consequently, a caller may have to use less sophisticated encoders, thereby degrading the quality of the call.
- A general problem encountered in packet switched networks, however, is that the network may drop or lose data packets. Packets may also be delayed during transportation from the sender to the receiver. Therefore, some of the packets at a receiving destination will be missing and others will arrive out of order.
- In a packet switched network whose transporting characteristics vary relatively slowly, the immediate past transporting characteristics can be used to infer information about the immediate future transporting characteristics. The dynamic network transporting characteristics may be measured using such variables as packet loss, packet delay, packet burst loss, loss auto-correlation and delay variation.
- The present invention relates to a method and apparatus for communicating a real time media input. The apparatus includes an encoding device that encodes the media into a plurality of data packets. Each data packet includes a plurality of frames that are created according to a first variable. A receiving device unpacks the data packets and buffers the unpacked data packets for a playout according to a second variable. The receiving device generates a plurality of utility parameters for evaluating a dynamic characteristic of a transporting network. The transporting network transports the data packets from the encoding device to the receiving device. A preferred utility parameter is selected. The preferred utility parameter is used to adjust the first and the second variable.
- In another aspect of the invention, an apparatus for communicating a real time media input includes a sender having an encoder and a packetizer. The encoder partitions and compresses the real time media input into a plurality of frames. The packetizer packets the frames into a plurality of data packets according to a redundancy value. A transporting network transports the data packets from the sender to a receiver. The receiver includes a real decoder and a plurality of computation decoders. The real decoder includes a real decoder depacketizer and a real decoder buffer. The real decoder depacketizer unpacks the plurality of frames. The frames are placed within the real decoder buffer according to a buffer length variable. Each computation decoder has a utility parameter for evaluating a dynamic characteristic of the transporting network. The computation decoder includes a computation decoder depacketizer and a computation decoder buffer. The computation decoder depacketizer unpacks the plurality of frames and communicates the frames to the computation decoder buffer. The receiver selects a preferred utility parameter from the utility parameters and communicates a feedback variable to the real decoder buffer. The buffer length variable is adjusted in accordance with a change in the dynamic characteristic.
- In another aspect of the invention, a system for transmitting real time media includes a first calling device for placing a call to a first processing hub. The first processing hub includes an encoder that partitions the call into a plurality of compressed frames. A packetizer packetizes the frames into a data packet according to a redundancy variable. A transporting network transports the data packet between the first hub and a second hub. The second hub includes a decoder for decoding the data packet into the plurality of frames and ordering these frames within a buffer with depth according to a buffer length variable. The decoder generates a plurality of utility parameters based on the redundancy value and the buffer length variable. The decoder selects a preferred utility parameter from the utility parameters such that the redundancy value and the buffer length are adjusted in accordance with a change in the dynamic characteristic. A second calling device receives the call from the second processing hub.
- In still another aspect of the invention, a method for communicating a real time media input includes the step of communicating the real time media input to a sending device and encoding the input into a plurality of data packets. Each data packet comprises a plurality of frames ordered according to a first variable. The data packets are transported to a receiving device where they are unpacked and buffered for a playout of the analog input according to a second variable. At least two utility parameters are generated for evaluating a dynamic characteristic of a transporting network that transports the data packets from the sending device to the receiving device. A preferred utility parameter is selected from the utility parameters. The first and the second variable are adjusted according to the preferred utility parameter.
- In still another aspect of the invention, a method for communicating a real time media input includes the steps of partitioning and compressing the real time media input into a plurality of frames. The frames are packetized into a plurality of data packets according to an actual redundancy variable. The data packets are transported by a transporting network. The plurality of frames are unpacked and arranged within a buffer according to a buffer length variable. A plurality of utility parameters are generated for evaluating a dynamic characteristic of the transporting network. A preferred utility parameter is selected from the utility parameters wherein the buffer length variable is adjusted in accordance with a change in the dynamic characteristic.
- In another aspect of the invention, a method for transmitting real time audio includes the steps of placing a call to a first processing hub. The call is partitioned and compressed at the first processing hub into a plurality of frames. The frames are packed into a data packet according to a redundancy value. The data packet is transported between the first processing hub and a second processing hub by a transporting network. The data packet is decoded at the second processing hub into the plurality of frames. These frames are ordered within a second hub buffer according to a buffer length. A plurality of utility parameters are generated based on the redundancy value and the buffer length. A preferred utility parameter is selected from the utility parameters. The redundancy value and the buffer length are adjusted in accordance with a dynamic characteristic of the transporting network. The call is then received from the second processing hub.
- These and many other features and advantages of the invention will become more apparent from the following detailed description of the preferred embodiments of the invention.
- FIG. 1 shows a general overview of a system for transporting a real time media input over a packet switched network and incorporating a preferred embodiment of the present invention.
- FIG. 2 illustrates a communication channel, including a sender and a receiver, in accordance with the system shown in FIG. 1.
- FIG. 3 is block diagram of a data packet transported between the sender and the receiver shown in FIG. 2.
- FIG. 4 shows an order of the redundant frames in five levels of data packets.
- FIG. 5 is an illustration of a linked list structure of a real decoder buffer shown in FIG. 2.
- FIG. 6 is a flowchart of a GetNode function for accessing the linked list shown in FIG. 5.
- FIG. 7 is a flowchart of a PutNode function for the real decoder shown in FIG. 2 and which accesses the linked list structure shown in FIG. 5.
- FIG. 8 illustrates a state transition diagram of the real decoder buffer illustrated in FIG. 2.
- FIG. 9 illustrates a flowchart of a Time Out function for the real decoder buffer shown in FIG. 2.
- FIG. 10 illustrates a flowchart of a PlayNode function for the real decoder shown in FIG. 2.
- FIG. 11 is a flowchart of a PacketArrival function for the real decoder shown in FIG. 2.
- FIG. 12 illustrates a flowchart of a PlayNode function for one of the computation decoders shown in FIG. 2.
- FIG. 13 is a flowchart of a PacketArrival function for one of the computation decoders shown in FIG. 2.
- FIG. 14 is a graph of a loss utility function UL having a loss rate less than or equal to ten (10).
- FIG. 15 is a graph of a Redundancy utility function UR having a Redundancy less than or equal to three (3).
- FIG. 16 is a graph of delay utility function UD having a delay less than or equal to one (1) second.
- FIG. 17 is a graph of modified loss utility function UL* of the utility function UL shown in FIG. 14.
- FIG. 18 is a graph of a modified redundancy utility function UR* of the redundancy utility function UR shown in FIG. 15.
- FIG. 1 shows an overview of a
system 10 for communicating a realtime media input 25 and incorporating a preferred embodiment of the present invention. Thesystem 10 includes a sendingdevice 20, afirst processing hub 30, a transportingnetwork 35, a mapping service 31, asecond processing hub 40 and a receivingdevice 45. - The sending
device 20 is a calling device that generates the realtime media input 25. Preferably, the realtime media input 25 is a telephone call. Alternatively, the sendingdevice 20 generates other types of real-time media inputs such as video, multimedia, streaming applications, or a combination thereof. - The
input 25 is communicated over atelephone line 26 to thefirst processing hub 30. Preferably, thefirst hub 30 is a local hub and is commercially available from U.S. Robotics of Skokie, Illinois as U.S. Robotics Edgeserver™ bearing part number 1098-0. Thefirst hub 30 processes theinput 25 and converts theinput 25 into a form that can be transported by the transportingnetwork 35. Thefirst hub 30 may include an encoding device for encoding theinput 25 into a digital format. Thehub 30 may then compress the digital format into a plurality of frames. These frames could be packetized into a sequence ofdata packets 36 comprising a plurality ofdata packets 33. Thedata packets 33 are then transported by the transportingnetwork 35 to thesecond processing hub 40. - The mapping service31 maps the phone number being called to an Internet Provider (IP) address of a receiving hub. Preferably, the receiving hub is a hub closest to the party receiving the call. In the system shown in FIG. 1, the receiving hub is the
second processing hub 40. - The transporting
network 35 transports thedata packet sequence 36 to the selected receivinghub 40. Because various packets of thesequence 36 may be dropped or lost during transportation, thefirst packet sequence 36 may differ from the second sequence of data packets 37. Thedata packets 34 comprising sequence 37 are communicated to thesecond calling device 45 over atelephone line 41. - In this proposed scheme, the
first device 20 can place a telephone call to thesecond calling device 45 in the following maimer. Callingdevice 20 activates an Internet account by calling a toll free number. The Internet account then prompts the callingdevice 20 for identification. An identification number, such as a phone card number or a credit card number, is entered. The callingdevice 20 is then provided a number of a local processing hub (i.e., the first processing hub 30) based on the caller's identification number. Thefirst hub 30 is consequently made aware that there is a new user in its area. Once the caller has been identified, thecaller 20 calls its assigned local processing hub. The hub will then recognize the caller based on the caller's identification number. One advantage of this proposed identification scheme is that it facilitates billing the caller for usage and other types of service charges. - After identifying itself to the
first hub 30, the caller is asked to enter the phone number that the caller wishes to call. The mapping service 31 maps the phone number to an IP address of a sending hub closest to the caller. This phone number facilitates selecting a receiving hub as close as possible to the location of the other party to the call. The selected receiving hub then places a call to the receiving party so that the call can proceed. The caller's voice is then transported as data packets between the sending and the receiving hub. - One advantage of the system shown in FIG. 1 is that the system samples and compresses the communicated information in close proximity to the transporting network. Preferably, sampling and compressing are performed in the
processing hubs System 10 can also accurately measure one-way delay and can therefore compensate the transportation of data packets based on the varying transporting characteristics of the transportingnetwork 35. - The transporting
network 35 is a packet switched network and preferably the Internet. An Internet is one type of packet switched network: it is a network of networks. The Internet is divided into thousands of autonomous systems (“AS”) that are individual networks controlled by an administrative agency. The range of AS sizes can vary greatly. For example, a single company with a single Ethernet local area network (“LAN”) is an AS. A large AS, such as a telephone company ATM backbone spanning the breadth of the United States is also an AS. Therefore, the term Internet, as that term is used herein, is a meta-network in that it is a scheme for inter-connecting different AS's such that data can be transported between AS's. Currently, the Internet spans over 140 countries and includes approximately 13 million individual hosts. The term “host,” as used herein, is a computer or access point having a unique Internet Protocol (IP) address. - Alternatively, aside from the Internet, other types of AS's that can be used to transport the stream of data packets between the first and
second hub network 35 transports the sequence of data packets from thefirst processing hub 30 to thesecond processing hub 40. - The
second processing hub 40 receives the sequence of data packets 37. The sequence received 37 differs from the sequence transported 36 because of packet loss and packet delays that frequently occur in packet switched networks. The receiveddata packets 33 are decoded by thesecond hub 40. The second hub first unpacks the packets and then decompresses this information. This decompressed information is then ordered within a buffer. The buffer of information is then played out and converted to ananalog signal 41. Theanalog signal 41 is then sent overtelephone line 42. - Prior to sending the
analog signal 41 over thetelephone line 42, thesecond hub 40 may call thesecond calling device 45. Thesecond calling device 45 then plays out theanalog input 26. Thesecond calling device 45 can generate information and transport this information to thefirst calling device 20 in a similar fashion. - Preferably, the first and the
second calling devices system 10 shown in FIG. 1 are each associated with telephone call participants. Participants can therefore place telephone calls over a regular telephone rather than have to use a PC speakerphone system. Because telephones are generally more common than PCs, the proposedsystem 10 will be more available to the public. Telephones also provide a more natural user interface to those individuals who do not use or who are uncomfortable using computers. - Alternatively, the sending and receiving
devices - Since the first and
second calling devices first calling device 20 has been described as both the sender and the receiver of telephone calls. To provide a more detailed discussion as to how thesystem 10 performs interactive bi-directional communication between the first and thesecond processing hubs first hub 30 acting as a sender to thesecond hub 40 acting as a receiver will be discussed. - FIG. 2 illustrates a communication channel60 in accordance with the system shown in FIG. 1. The communication channel includes a
sender 65 and areceiver 75. Thesender 65 may, for example, be included within thefirst hub 30 shown in FIG. 1. Thereceiver 75 may be included within thesecond hub 40 shown in FIG. 1. It should be realized, however, that in an interactive environment where information is transported bi-directionally, a processing hub will normally include bothsender 65 andreceiver 75 thereby enabling the hub to receive and transmit information simultaneously. - Returning to FIG. 2, the
sender 65 includes anencoder 80 coupled to apacketizer 90. A first stream ofdata packets 95 generated by thepacketizer 90 is transported by a transportingnetwork 35. Thereceiver 75 receives a stream of data packets 96. The stream of data packets 96 is supplied to a real decoder 130 and a number ofcomputation decoders 150. The real decoder 130 includes a depacketizer 135 coupled to abuffer 140. Preferably, the depacketizer 135 operates in accordance with a first variable. Preferably, the first variable is anactual Redundancy variable 115. The size of thereal decoder buffer 140 varies in accordance with a BufferLength variable 174. Thebuffer 140 is coupled to adecoder 162. Thedecoder 162 provides adigital input 163 to a digital-to-analog converter 164 (i.e., D/A converter 164). The D/A converter 164 providessignal 165 to thesecond calling device 166 for playout. - In an alternative embodiment, the first variable is a vector of values. These vectors may represent a plurality of variables providing further control of the communication channel. For example, such variables could be used for identifying the type of codings above being used by the sender, a redundancy parameter, and other types of control identifiers.
- The
computation decoders 150 are arranged in parallel to the real decoder 130. In this configuration, thecomputation decoders 150 and the real decoder 130 receive the stream of data packets 96. The stream 96 comprises transporteddata packets 97. Eachcomputation decoder 150 includes acomputation decoder depacketizer 152 and acomputation decoder buffer 154. - The operation of the communication channel60 will now be described with reference to FIG. 2. A
first calling device 70 generates a realtime media signal 72, preferably a telephone call. Alternatively, thesignal 72 is video, multimedia, a streaming application or a combination thereof. Thesignal 72 is communicated to an analog-to-digital converter 82 (i.e., A/D converter 82). The A/D converter 82 converts thesignal 72 to adigital signal 83. Preferably, where thesignal 72 is a phone call, thedigital signal 83 is a digital speech wave form. - The
digital signal 83 is communicated to anencoder 80 of thesender 65. In the case of a phone call, thedigital signal 83 is communicated to theencoder 80 over a telephone line. The digital input 83 (preferably in Pulse Code Modulated (PCM) form) is compressed and partitioned byencoder 80 into a sequence offrames 85. Theencoder 80 encodes thedigital signal 83. - Preferably, in the case where the communication channel60 is used to communicate voice, the
encoder 80 is an ITU voice encoder complying with Recommendation G.723.1. Recommendation G.723.1 describes a code excited linear predictive encoder (CELP). This recommendation G.723.1 specifies a coded representation used for compressing speech or another audio signal component of multimedia services at a low bit rate as part of the overall H.324 family of standards. Recommendation G.723.1 is entitled “DUAL RATE SPEECH ENCODER FOR MULTIMEDIA COMMUNICATIONS TRANSMITTING AT 5.3 & 6.3 KBITS/S” and is published by the Telecommunication Standardization Sector of the ITU. Recommendation G.723.1 is herein entirely incorporated by reference. Alternatively, voice encoders complying with other standards or specifications can be used. - Preferably, the
digital input 83 to theencoder 80 is a digital speech waveform sampled at 8000 Hz. Each sample of theinput 83 is represented by a signed 16 bit integer. Theencoder 80, preferably the G.723.1 encoder, segments theinput 83 intoframes 85. Preferably, each frame is 30 milli-seconds (ms) in length. At the preferred sampling rate of 8000 Hz, 30 ms represents 240 samples. - The preferred G.723.1 encoder can operate at two different bit rates, a low rate of 5.3 kbits/seconds or a high rate of 6.3 kbits/seconds. In the high rate setting of 6.3 kbit/s, 480 bytes (i.e., 240
samples times 2 bytes/sample) are compressed to 24 bytes. In this high rate setting, where theinput 72 is voice, the encoding results in a quality that is close to toll quality. In the low rate setting of 5.3 kbits/s, 480 bytes are compressed to 20 bytes. Therefore, between the low and high rate setting, the compression ratio varies from 20 to 24. - Preferably, the
encoder 80 utilizes silence detection. The preferred G723.1 silence detection uses a special frame entitled Silence Insertion Descriptor (SID) frame. SID frame generation is described in Recommendation 6,723.1 which has been herein entirely incorporated by reference. During a “silence”, as that term is used herein, no voice data frames are generated by theencoder 80. An SID frame defines when a silence begins. After theencoder 80 transmits an SID frame, no further voice data frames are transmitted until the current silence ends. Updated SID frames may, however, be sent. This silencing technique reduces the required overall transfer rate. Moreover, as will be discussed, silence detection allows for a dynamic adjustment of the depth of thereal decoder buffer 140. The communication channel 60 can thereby compensate for varying transportation characteristics of thetransport network 35. - The
packetizer 90 packets theframes 85 into a plurality ofdata packets 92. Preferably, thepacketizer 90 places a time stamp and a sequence number into eachdata packet 92. The time stamp identifies the time aspecific data packet 92 was created. The sequence number identifies data packet ordering. Eachdata packet 92 includes both a current frame as well as redundant information such that a number of previously packeted frames might be reconstructed if some frames are lost during transportation. In one implementation, the number of previous frames or redundant frames is channel coded according to theactual Redundancy variable 115 of the communication channel 60. Theactual Redundancy 115 is the variable that determines the number of previous frames packet into eachdata packet 92. Thedata packets 92 are ordered in adata packet sequence 95 and transported by the transportingnetwork 35 to thereceiver 75. - Each data packet time stamp enables the
receiver 75 to evaluate certain dynamic transporting characteristics of the transportingnetwork 35. These transporting characteristics determine how thepacketizer 90 packetizes theframes 85 and how thereceiver 75 unpacks these frames. These varying transporting characteristics can include such characteristics as the standard deviation of one-way delay or the round trip time for each transporteddata packet 97. The round trip time is calculated by transporting a copy of the time stamp back to thesender 65 and comparing the received time with the timestamp value. The standard deviation of one-way delay is typically approximated by averaging the absolute value of differences between time stamp values and received times for eachpacket 97. Alternatively, if real time protocol (RTP) is used, data packet sequence numbers and time stamps are placed within the RTP header. The sequence numbers and timestamps do not, therefore, need to be reproduced in the data packet payload. Other transport protocols that contain timestamps and sequence number information can also be used in place of the RTP protocol. - The
receiver 75 receives a sequence of data packets 96. This sequence of data packets 96 may vary from the sequence ofdata packets 95 originally communicated to the transportingnetwork 35. The variance between the twodata packet sequences 95, 96 is a function of varying transporting characteristics such as packet loss and packet transport times. - Because the preferred transporting
network 35 is a non-guaranteed packet switched network, thereceiver 75 receives packets out of order vis-a-vis other data packets comprising the originally transportedpacket sequence 97. To combat this occurrence, as previously mentioned, thepacketizer 90 adds sequence numbers to theframes 85 before the frames are packetized. As will be discussed with reference to the real decoders 130, thereceiver 75 has areal decoder buffer 140 that stores the data from the unpacked frames. As long as the sequence number of an arrivingpacket 97 is greater than the sequence number of the frame being played out by thebuffer 140, the sequence number is used to put the unpacked frame at its correct sequential position in thereal decoder buffer 140. Therefore, the larger the size of thebuffer 140, the later a frame can arrive at thereceiver 75 and still be placed in a to-be-played-out frame sequence. On the other hand, as the size of thebuffer 140 increases, the larger the overall delay can be in transporting theinput 83 from thesender 65 to thereceiver 75. - The
receiver 75 includes a real decoder 130, adecoder 162 and a plurality ofcomputation decoders 150. The real decoder depacketizer 135 receives the data packet sequence 96. Initially, the depacketizer 135 reads theactual Redundancy variable 115 contained in eachdata packet 97. Using theactual Redundancy variable 115, the depacketizer 135 unpacks thedata packets 97 and recovers theframes 85. Theframes 85 include both current and redundant frames. - The real decoder130 reads the sequence number and the time stamp of a current frame. Redundant frames associated with the current frame have the same time stamp as the current frame since, within a given packet, redundant and current frames were both originally communicated from the
packetizer 90 at approximately the same point in time. Since the order or sequence of the redundant frames is known, the redundant frame sequence numbers can be inferred from the current frame sequence number. - Preferably, each frame, together with its corresponding time stamp and sequence number, defines a
node 137. Thenodes 137 are forwarded to areal decoder buffer 140 for buffering. Redundant frames are not buffered if an original frame has been previously buffered. The buffered frames are then passed on to adecoder 162. Thedecoder 162 decompresses theframes 142. The decompressed frames 163 are then forwarded to a digital-to-analog converter 164 (i.e., D/A converter 164). The D/A converter 164 converts thedigital data 163 to ananalog output 165. Thisanalog output 165 represents theoriginal analog input 72 generated by thefirst calling device 70. Theanalog output 165 is forwarded to thesecond calling device 166 where theoutput 165 is then played out. - By monitoring various transporting characteristics of the transporting
network 35, the present communication channel 60 offers a number of advantages. For example, the present communication channel can adapt to varying transporting dynamics and conditions of the transportingnetwork 35. For a non-guaranteed packet switched network, the network transporting dynamics can be assessed by a packet delay distribution and a packet loss percentage, both of which generally vary over time. - In general, as the length of the
real decoder buffer 140 increases, the quality of the played out analog output 169 also increases. Unfortunately, as in the case of transporting a telephone call over the transportingnetwork 35, if the network packet delay is large, maintaining an interactive conversation may be difficult. On the other hand, if the real decoder buffer length is quite small (i.e., small in comparison to the standard deviation of network delay), frames with larger delays will arrive too late to be played out and will consequently be considered lost during transportation over thenetwork 35. Therefore, it is preferred that the real decoder 130 have abuffer 140 that has a variable buffer length. Preferably, the buffer length will vary in accordance with the dynamic transporting characteristics of thenetwork 35. - More preferably, the buffer length is proportional to the variance in delay experienced by the transported
data packets 97. A non-guaranteed packet switched transporting network, such as transportingnetwork 35, having a highly varying data packet delay results in an increased buffer length. Conversely, where a transporting network experiences a more constant data packet delay, the buffer length will be decreased. - A buffer length of X milliseconds is employed where X is a dynamic parameter. Utilizing a buffer having a dynamic buffer length of X milliseconds, after the arrival of an unpacked
node 137 from the real decoder depacketizer 135, X milliseconds must on average time out before thebuffer 140 can start playing out at a constant rate of 1 frame per 30 milliseconds. Alternatively, thebuffer 140 plays out at the frame rate used by theencoder 80. - Preferably, the
buffer 140 is implemented as having a doubly linked list (LL) structure. In such a preferred structure, thenodes 137 are ordered according to their respective sequence number. Eachnode 137 contains a pointer that points to the preceding and succeeding nodes in the structure. Eachnode 137 is inserted into thebuffer 140 at the appropriate linked list position. If a node already exists in thebuffer 140, the redundant node is discarded. Moreover, if the sequence number of the frame being played out 163 by thedecoder 162 is greater than the sequence number of an arrivingnode 137, then the arrivingnode 137 arrived too late and is discarded. Based on the frame length of theencoder 80, thebuffer 140 plays outframes 142 at periodic instances of time. Preferably, as in the case for a G.723.1 encoder, thebuffer 140 plays out one frame every 30 ms. - As shown in FIG. 2, the
receiver 75 containsN computation decoders 150. TheseN computation decoders 150 are arranged in parallel with the real decoder 130. Preferably, the number of computation decoders N is a product of the cardinality of the domain of two variables: the Redundancy and the BufferLength. As noted previously, the Redundancy defines the number of previous frames packeted into eachdata packet 92. The BufferLength variable defines the number ofnodes 137 buffered by thereal decoder buffer 140 before play-out occurs. As the BufferLength increases,fewer nodes 137 will arrive too late to be played-out by thebuffer 140. - Like the real decoder130, the
computation decoders 150 receive and observe thedata packets 97 of the incoming data packet sequence 96. Eachcomputation decoder 150 includes acomputation decoder depacketizer 152 coupled to acomputation decoder buffer 154. - The
computation decoders 150 operate differently than thereal decoder buffer 140. One difference is that thecomputation decoders 150 do not read the actual Redundancy variable 115 from an arrivingdata packet 97. Rather, each individual computation decoder uses an assigned fixed Redundancy [i] variable 153. This fixedRedundancy 153 is used to extract theframes 85 from the transporteddata packet 97. The fixed Redundancy [i] variable is a hypothetical Redundancy value and is used by computational decoder [i], and is an index to the computation decoders [i . . . N]. - Each
computation decoder 150 computes various characteristics of the transportingnetwork 35. Preferably, eachcomputation decoder 150 computes an AveDelay [i] and an AveLoss [i]. Eachcomputation decoder 150 also has an assigned Rate [i] 151. - Even when the
actual Redundancy variable 115 of adata packet 97 is less than the fixed Redundancy [i]parameter 153 of a corresponding computation decoder 150 [i], thecomputation decoder 150 computes two utility parameters: AveLoss [i] 160 and AveDelay [i] 158. - The AveLoss [i]
parameter 160 is a measure of the average number of the originally transporteddata packets 95 lost during transportation. In addition, the AveLoss [i] parameter takes into account thedata packets 92 originally transported but accounted for as being lost during transportation since these packets were received too late to be played out by thebuffer 140. AveLoss [i] 160 provides one method to quantify a difference between thedata packets 95 originally sent by thesender 65 and thedata packets 97 actually received by thereceiver 75. - The AveDelay [i]
parameter 158 is a measure of the average time it takes for thedata packets 92 to be transported from thesender 65 to thereceiver 75. The AveDelay [i]parameter 158 preferably also includes the time required for thebuffer 140 to playout theframes 142. These measures are computed from the time stamp and sequence number associated with the transporteddata packets 92. In this case, AveDelay [i] 158 is equal to the sum of the one way delay plus the receiver buffer time. The receiver buffer time can be estimated by multiplying the receiver Buffer Length by the period of the frame rate. The one way delay is estimated by adding an estimate of the network delay to the Receive Buffer Delay. - AveLoss [i]160 is determined by the flow chart algorithm of the computation decoder as will be discussed with reference to FIG. 12.
- The fixed
Redundancy variable 153 associated with each computation decoder can be greater than, less than, or equal to theactual Redundancy variable 115. When a particular fixed Redundancy [i] variable 153 of a corresponding computation decoder [i] 150 is greater than theactual Redundancy variable 115, some of theframes 85 of thedata packet 97 are unavailable to thecomputation decoder 150. This does not matter, however, since thecomputation decoder 150 only requires the time stamp and the sequence number of a receiveddata packet 97. Moreover, the time stamp and the sequence number for all the redundant frames can be inferred. These values can be inferred since time stamps remain unchanged and sequence numbers are in sequential order for the hypothetical case of the computation decoder. This is true even when theactual Redundancy parameter 115 and a fixed Redundancy [i]parameter 153 differ. - Each computation decoder has three unique values associated with it. The three values of each computation decoder AveDelay, AveLoss and Rate defines the utility of the computation decoder for a given data packet transportation. As shown in FIG. 2, the three values AveDelay [i]158, AveLoss [i] 160 and Rate [i] 151 of each
computation decoder 150 are analyzed by a utility function 170. The utility function 170 selects the optimal computation decoder that would have resulted in the highest utility for a transported data packet. - The utility of a
particular computation decoder 150, and therefore the utility of theoverall receiver 75, is application specific. Preferably, the utility is a function of theaverage delay AveDelay 158, the averageloss rate AveLoss 160 and theRate 151. Rate is a measure of the bandwidth required to transport the media stream which is increasing with redundancy. Since the AveLoss rate is a function of theactual Redundancy parameter 115, the utility function 170 is preferably a function of three network transmission characteristics represented by these three variables. The utility function 170 preferably has the following form U(AveDelay, AveLoss, Redundancy) and if seperability is desired, it may be expressed as follows: - U(AveLoss, AveDelay, Redundancy)=U L(AveLoss)*U D(AveDelay)*U R(Redundancy)
- where UL(AVeLOSS) is the loss utility function, UD(AveDelay) is the delay utility function, and UR(Redundancy) is the Redundancy utility function. Alternatively, the utility function can be expressed in other forms, such as a non-seperable, non-linear function in the form of a table.
- The general purpose of the utility function170 is to rate the different
type computation decoders 150. In this manner, the computation decoding values of Redundancy [i] and BufferLength [i] that would have optimized data packet transportation at a given time is selected. These optimal values determine the new values for theactual Redundancy 115 and theBufferLength 174. - The utility function170 is application specific and can be modified to best fit the type of
analog input 72 being transported. The application's specific nature of the utility function can be explained by way of the following example. If a specific type of application calls for a maximum loss rate of 10%, a loss utility function UL can be represented by the graph shown in FIG. 14. As shown in this graph, as long as the loss rate is less than or equal to 10%, the loss utility function UL will be equal to 1. Any loss rate greater than 10% will result in the loss utility function UL to be equal to zero (0). - In this example, it is further assumed that the specific application is not overly concerned with redundancy as long as no more than three (3) redundant frames are used. The resulting redundancy utility function UR can be expressed graphically as shown in FIG. 15. According to FIG. 15, as long as the redundancy utility function UR is equal to or less than three (3), the utility function UR will equal one (1). Any Redundancy greater than three (3) will result in a redundancy utility function UR equal to zero (0).
- The third concern in this example is the data packet transportation delay. Returning to the example discussed with respect to FIGS. 14 and 15 and given the above UL and UR, it will be assumed that the example will require a delay of less than or equal to one (1) second. Any greater delay will result in the delay utility UD equal to zero (0). This requirement can be graphically represented as shown in FIG. 16.
- Taking the utility functions UL, UR, and UD shown in FIGS. 14, 15, and 16, respectively, one can define an overall utility function U(AveLoss, AveDelay, Redundancy) to be the product of these three individual utility functions. A computation decoder that maximizes this function will specify in an average loss less than or equal to 10%, specify three or fewer redundant frames and specify the smallest possible average delay, given the first two constraints.
- Preferably, where two
computation decoders 150 result in exactly the same delay UD, thedecoder 150 using the lesser amount of redundancy UR or that results in the smaller loss rate UL is selected. Preferably, this selection process is accomplished by slightly altering the loss and redundancy utility function UL* and UR*, respectively. For example, a modified loss rate UL* and a modified Redundancy rate UR* is shown in FIGS. 17 and 18, respectively. - The
Redundancy value 172 andBufferLength value 174 of the optimal computation decoder are utilized as follows. First, theRedundancy value 172 is packetized into a feedback data packet 118 that is transported to thepacketizer 90 of thesender 65. Thesender 65 adjusts theactual Redundancy variable 115 based on thenew Redundancy value 172. - Secondly, the optimal BufferLength value is communicated to the
real decoder buffer 140. Thereal decoder buffer 140 uses the preferredBufferLength value 174 to buffer thenodes 137. Preferably, the Redundancy and BufferLength are chosen periodically with intervals of one (1) to ten (10) seconds in a typical implementation. - It is important to note that the fixed
Redundancy values 153 and the fixed BufferLength values 156 associated with thecomputation decode 150 are constant. These values are therefore not adjusted according to the transmission characteristics of the transportingnetwork 35. Rather, it is the function of all thecomputation decoders 150, by using allpossible Redundancy value 153 andBufferLength 156 combinations, to determine an optimal value of these two variables based on the transport characteristics of thenetwork 35 at a given time. - The preferred
computation decoder 150 has highest utility for a given data packet transportation and therefore provides the best choice of system variables given the network conditions at a given time. This allows for the flexibility of using a variety of utility functions for different types of applications. For example, for a streaming application, or a one way communication, the AveDelay UD can be quite a bit larger than for an interactive application. On the other hand, a streaming application may require a higher quality than the interactive call or video conference application. - The real decoder decompression scheme matches the encoder scheme used to compress the
input 83. Preferably, thedecoder 162 is a G.723.1 decoder where the input to the decoder is a frame stream. The output of thedecoder 160 is a waveform in the same format as theanalog input 83 for the G.723.1 encoder. - FIG. 3 illustrates the structure of a
data packet 92 transported by the communication channel shown in FIG. 2. Preferably, each data packet has a data packet header that is a thirty-two (32) bit word containing aRedundancy parameter 115, a current frame and a plurality of redundant frames. TheRT Header 110 is a Real Time Protocol header containing a sequence number and time stamp. The Real Time Protocol contains a field which identifies how the remainder of the data packet is interpreted. Packets of 95 or 96 may be Real Time Protocol packets which contain packets from the protocol described here. - In a preferred embodiment, the
message data 108 reads 0000 fordata packets 92 transmitted from thesender 65 to thereceiver 75. For thefeedback packet 178 sent from thereceiver 75 to thesender 65, themessage data 108 of the packet reads 0001. This message data field allows thesender 65 and thereceiver 75 to differentiate between data and feedback packets. The feedback packet 175 preferably does not contain a frame length or frame data. Rather, the feedback packet 175 contains information relating to the desired value to be used for theactual Redundancy variable 115. The header spare 109 is reserved for later use for data packets from sender to receiver. For data packets sent from the sender to the receiver, theRedundancy 115 variable represents the number of additional previous frames that eachdata packet 95 contains. TheFrame Length 120 represents the length of the following frame data field in bytes. - FIG. 4 illustrates an example of a preferred order of the redundant frames in five levels of data packets wherein the
actual Redundancy variable 115 is set equal to two (2). With a Redundancy variable set equal to 2, thepacketizer 90 packs two previously transmittal frames into eachdata packet 95. For example, as shown in FIG. 4, with respect toFrame n 186,Packet n 192 containsFrame n 186 and its two previous frames: Frame n−1 184 and Frame n−2 182. Similarly, Packet n+1 194 contains Frame n+1 188, along with its two previous frames:Frame n 186 and Frame n−1 188. Packet n+2 196 contains Frame n+2 190 along with its two previous frames: Fame n+1 188 andFrame n 186. In a scheme having an actual Redundancy variable 115 equal to two (2) therefore, eachpacket 95 includes a current frame along with the two previous frames. - FIG. 5 illustrates an example of a preferred double linked list (LL)200. This example has a
Start node 210 having a sequence number equal to 10 and aStop node 220 having a sequence number equal to 13. The preferred real decoder buffer implementing theLL 200 keeps track of the first or Start node and the last or Stop node. In a preferred embodiment, theLL 200 contains all the nodes having sequence numbers that fall between a Start node and a Stop node. - If the
real decoder buffer 140 receives a frame withinnode 137 having asequence number 10 and another frame withsequence number 13, then the LL creates all the nodes falling between and including the Start and Stop nodes, i.e., 10, 11, 12, 13 (i.e.,element numbers nodes frame sequence number 12 have not yet been received. The creatednodes 230 and 240 are marked as missing. - Two functions are provided for accessing the
LL 200. The first function is the PutNode function. TheLL 200 utilizes the PutNode function to insert a node in the correct LL position. FIG. 7 illustrates a flowchart of aPutNode function 300 for the real decoder 130 shown in FIG. 2. After receiving anode 137 from the depacketizer 135, thePutNode function 300 must determine where to put this node. ThePutNode function 300 first determines whether thereal decoder buffer 140 is empty 301. If the real decoder buffer is empty, thefunction 300 creates the buffer using the new node atstep 303. A Start and a Stop pointer associated with this new node now point to this new node atstep 303. Thefunction 300 then updates the real decoder buffer depth and the real decoder buffer state atstep 304. - Alternatively, if the
buffer 140 is not empty, then atstep 305 thePutNode function 300 determines whether the new node should be placed to the left of the existingStart node 305. If the new node should be placed to the left of the existing Start node, then atstep 307 the missing buffer nodes are created between the existing Start node and the new node. The new start will now point to the new node. ThePutNode function 300 then updates the real decoder buffer depth and the real decoder buffer state atstep 304. - If the
PutNode function 300 determines that the new node should not be placed to the left of the existing Start node, then atstep 309 thePutNode function 300 determines whether the new node should be placed to the right of the existing Stop node. If the new node should be placed to the right of the existing Stop node, then the missing buffer nodes between the Stop node and the new node are created atstep 310. The Stop pointer will now point to thenew node 310. ThePutNode function 300 then updates the real decoder buffer depth and the real decoder buffer state atstep 304. - If the
PutNode function 300 determines that new mode is not to be placed to the left of the existing Start node or to the right of the existing Stop node, then thefunction 300 finds the existing buffer node with the same sequence number atstep 312. ThePutNode function 300 then determines whether the buffer node is marked as missing atstep 314. If the buffer node has been marked as missing, the buffer node is replaced by the new node atstep 315. Thefunction 300 then updates the real decoder buffer depth and the real decoder buffer state atstep 304. If the buffer node has not been marked as missing, then thefunction 300 updates the real decoder buffer depth and real decoder buffer state atstep 304 without replacing the buffer node. - The second function for accessing the
LL 200 is the GetNode function. Thebuffer 140 utilizes the GetNode function for retrieving anode 137 having the smallest sequence number. A flowchart of theGetNode function 325 is illustrated in FIG. 6. Atstep 327, theGetNode function 325 first determines whether thereal decoder buffer 140 is empty. If thebuffer 140 is empty, then at step 326 a node is created. The newly created node is marked as missing and the node is returned to thebuffer 140. If thebuffer 140 is not empty, then atstep 329 the node having the smallest sequence number is returned. TheGetNode function 325 then adjusts the real decoder buffer depth and buffer state atstep 331. - Depending on the buffer depth, the
buffer 140 can be in one of three different states: Fill, Normal and Drain. These transitions are controlled by the SetNode and PutNode functions. These three states Fill, Normal and Drain can be represented by the state transition diagram 350 illustrated in FIG. 8. This state transition diagram 350 shows how thereal decoder buffer 140 changes state, depending on the buffer depth and the current state of the buffer. - There are three critical buffer watermarks shown FIG. 8: Low (L), Normal (N), and High (H). The objective of the state diagram350 is to maintain the buffer in its
Normal state 354. For example, if while in the Normal state, the buffer depth falls below Low, the buffer changes state from theNormal state 354 to theFill state 352 as shown intransition 358. The objective of the Fill state is to bring the buffer depth back to theNormal state 354. This objective may be achieved by artificially lengthening the next silence period until enough data packets arrive to return the buffer depth back to theNormal state 354. As long as the buffer depth stays between Low and High watermarks, the buffer state remains in theNormal state 354. If the buffer depth goes below theLow watermark 358, the buffer state switches back to theFill state 352. If the buffer depth increases above the High watermark as shown bytransition 360, the buffer state changes to theDrain state 356. The objective of theDrain state 356 is to shorten the silence periods and therefore reduce the buffer depth until it is returned to theNormal state 354. As long as the buffer depth is greater than N, the buffer will remain in theDrain state 356. Once the buffer depth is less than or equal to N as shown in transition 357, the buffer will become Normal again. - Preferably, the buffer control attempts to keep the buffer depth around the set BufferLength. This can be accomplished by setting the Normal water mark (N) equal to the BufferLength, the Low watermark (L) equal to half of the BufferLength and the High watermark (H) equal to 1.5 times the BufferLength.
- There are basically two events associated with the
real decoder buffer 140. The first event is the arrival of the data packet 96. The second event is defined as a TimeOut. - The arrival of the data packet is defined as a PacketArrival. The arrival of transported data packets96 is an asynchronous event and occurs whenever the real decoder 130 and
computation decoders 150 receive a data packet. FIG. 11 provides a flowchart for thePacketArrival function 420 of the real decoder 130. After the data packet 96 is received, the real decoder 130 reads theactual Redundancy variable 115 atstep 421 and unpacks the next frame atstep 423. ThePacketArrival function 420 then determines whether the frame has arrived in time for buffer play out or if the buffer is already empty atstep 425. If the frame has arrived in time for play out and the buffer is empty, then a node is created atstep 427. ThePutNode function 300 as described with reference to FIG. 7 is then implemented at step 479. If the frame arrives late or if the buffer is not empty, it is determined whether any redundant frames are left 431. If redundant frames remain at step atstep 431, then the frame unpacking and node generation process returns to step 423. - FIG. 13 provides a flowchart for the
PacketArrival function 440 of thecomputation decoders 150. Once a data packet is received by acomputation decoder 150, thecomputation decoder 150 unpacks the frames atstep 441. Atstep 443, ThePacketArrival function 440 determines whether the unpacked frame was received in time for play out and if the buffer is empty. If both are true, then a node is created atstep 445 and the PutNode function 300 (shown in FIG. 7) is implemented atstep 447. ThePacketArrival function 440 then proceeds to determine whether any other redundant frames remains atstep 449. If more frames remain, thePacketArrival function 440 returns to step 441. - If the frame was received late or it the buffer is not empty, the
PacketArrival function 440 determines whether any redundant frames are remaining atstep 449. If any frames are left atstep 449, then the process returns to step 441 and the next frame is unpacked. In the computational decoder, the actual data frames do not need to be stored in the buffer. Instead, the buffer may be marked with an indication of a data frame being unpacked and stored. - The second event associated with the real decoder130 is defined as a TimeOut. The TimeOut event is periodic and fixed to the frame size of the
encoder 80. As previously discussed, the frame size and resulting TimeOut for the preferred G.723.1 system occurs every 30 milliseconds. FIG. 9 illustrates a flow chart for theTimeOut event 380. - At
step 382, theTimeOut function 380 first determines the state of the buffer. If thebuffer 140 is in theFill state 352, theTimeOut function 380 proceeds to determine whether a silence period is detected atstep 384. If a silence period is detected, the silence period is extended atstep 386 until the buffer state switches toNormal 354. The function then returns to step 400 and executes the PlayNode function as previously described and shown in FIG. 12. If a silence period is not detected, theTimeOut function 380 implements theGetNode function 325 as previously described with reference to FIG. 6. After the GetNode function is implemented atstep 325, theTimeOut function 380 returns to step 400, the PlayNode function, and the frame with the lowest sequence number is taken out of the buffer and played out. - If the
TimeOut function 380 determines that the buffer is in theDrain state 356, the GetNode function as previously described with reference to FIG. 6 is implemented atstep 325. After the GetNode function atstep 325 is implemented, theTimeOut function 380 proceeds to step 396 to determine whether a silence period is detected and whether thebuffer 140 is in the Drain state. If both are detected, thefunction 380 returns to theGetNode function 325. If both are not detected, theTimeOut function 380 returns to the PlayNode function atstep 400. - If the
TimeOut function 380 determines that the buffer is in theNormal state 354, thefunction 380 proceeds to theGetNode function 325. TheTimeOut function 380 then returns to the PlayNode function atstep 400. - There are two different types of PlayNode functions. The first is the real decoder130 PlayNode function 390 is illustrated in FIG. 10. The second is the computation
decoder PlayNode function 400, illustrated in FIG. 12. The purpose of the first PlayNode function 390 is to send the frame data to theplayout decoder 160 which is called for whenever a TimeOut occurs. It is therefore invoked periodically with the period being equal to theencoder 80 frame length. The PlayNode function 390 first determines whether a frame is missing atstep 391. If no frame is missing, the function 390 proceeds to step 393 where a loss bit is set to zero (0). Next, the frame is played atstep 395. The AveLoss and AveDepth statistics are then updated atstep 397. - If the first PlayNode function390 determines at
step 391 that a frame is missing, then the function 390 proceeds to step 392 where the loss bit is set to one (1). Next, the function 390 determines whether a silence period is detected atstep 394. If a silence period is detected, then the silence is extended atstep 398. If a silence period is not detected atstep 394, then a frame is interpolated atstep 396. This effectively plays a frame that is an estimation of the missing frame. The AveLoss and AveDepth statistics are then updated atstep 397. - The second type of
PlayNode function 400 is that of thecomputation decoders 150 and is illustrated in FIG. 12. Thesecond Playnode function 400 first determines atstep 402 whether a frame is missing. Thesecond Playnode function 400 sets a loss bit equal to one (1) if the frame is missing at step 404. The loss is set to zero (0) if the frame is not missing. Thesecond PlayNode function 400 then updates the AveLoss and the AveDepth statistics of the transporting network with these new values atstep 408. - The preferred utility function170 evaluates or maps the new value of the variable Bufferlength 174 and a new value of the
variable Redundancy 115. Thevariable BufferLength 174 is altered by first changing the three watermarks as described with reference to the buffer state diagram 350 and then changing the buffer states 352, 354 and 356. The Normal watermark value in the real decoder will change to the new BufferLength variable. Other watermark values (High and Low) may be determined either by alogrithm or by copying some values from computational decoder which yielded the largest utility parameter. If a larger buffer state is changed to a smaller one, then the adjustment of the buffer state may result in aDrain state 356. Consequently, thebuffer 140 starts shortening the silence periods. If the buffer is increased, then the adjustment of the buffer state may result in aFill state 352. Consequently, a subsequent silence period will be extended until the buffer fills up to theNormal watermark 354. The new Redundancy variable will be communicated back to the sender. - Although the forgoing description of the preferred embodiment will enable a person of ordinary skill in the art to make and use the invention, a detailed C++ language program listing is included below. The program is entitled buffer.cpp and contains routines associated therewith. The program is an implementation of a buffer class for a receiver of the Internet telephony scheme. Additional detailed features of the system will become apparent to those skilled in the art from reviewing these programs.
- While the invention has been described in conjunction with presently preferred embodiments of the invention, persons of skill in the art will appreciate that variations may be made without departure from the scope and spirit of the invention. This true scope and spirit is defined by the appended claims, as interpreted in light of the foregoing.
Claims (64)
1. An apparatus for communicating a real time media input comprising:
an encoding device for encoding the media input into a plurality of data packets, each data packet comprising a plurality of frames created according to a first variable;
a receiving device for unpacking the data packets, and buffering the unpacked data frames packets for a playout of the media input according to a second variable, the receiving device generating a plurality of utility parameters for evaluating a dynamic characteristic of a transporting network that transports the data packets from the encoding device to the receiving device, wherein a preferred utility parameter is selected from the plurality of utility parameters, the preferred utility parameter is used to adjust the first and the second variable so that the encoding device and the receiving device adapt to the dynamic characteristic.
2. The apparatus of claim 1 , wherein the first variable is a vector of values that define the compression and packetizing scheme.
3. The apparatus of claim 1 , wherein the second variable is a vector of values that define the decompression and depacketizing and playout scheme of the receiving device.
4. The apparatus of claim 1 , wherein the real time media input is an audio waveform.
5. The apparatus of claim 1 , wherein the real time media input is a video waveform.
6. The apparatus of claim 1 , wherein the dynamic transporting characteristic is selected from a group consisting of data packet loss, data packet delay, packet burst loss, loss auto-correlation and delay variation.
7. The apparatus of claim 1 , wherein the transporting network comprises a packet switched network.
8. The apparatus of claim 5 , wherein the packet switched network is selected from a group consisting of Local Area Networks, Internet Protocol, frame relay networks, ATM networks, and Wide Area Networks.
9. The apparatus of claim 1 , wherein the transporting network comprises an interconnected switched network of Local Area Networks, Internet Protocol Networks, and frame relays net works, and Wide Area Networks.
10. The apparatus of claim 5 , wherein the packet switched network is an Internet.
11. The apparatus of claim 5 , wherein the packet switched network is an Intranet.
12. The apparatus of claim 1 , wherein the first variable is an actual redundancy parameter.
13. The apparatus of claim 1 , wherein the second variable is an actual buffer length parameter.
14. The apparatus of claim 1 , wherein a function of the preferred utility parameter is packeted into a feedback packet which is transported to the encoding device by the transporting network.
15. The apparatus of claim 12 , wherein the receiving device packets the function of the preferred utility parameter into the feedback packet.
16. An apparatus for communicating a real time media input, the apparatus comprising:
a sender comprising an encoder and a packetizer in which the encoder partitions the real time media input into a plurality of frames and compresses the frames, the packetizer packets the compressed frames into a plurality of data packets according to a redundancy value;
a transporting network that transports the data packets from the sender to a receiver;
the receiver comprising a real decoder and a plurality of computation decoders, the real decoder comprising a real decoder depacketizer and a real decoder buffer wherein the real decoder depacketizer unpacks the plurality of frames, the frames are placed into the real decoder buffer, a buffer depth is controlled by a buffer length variable,
each computation decoder having a utility parameter for evaluating a dynamic characteristic of the transporting network, the computation decoder comprising a computation decoder depacketizer and a computation decoder buffer wherein the computation decoder depacketizer unpacks the plurality of frames, and communicates the frames to the computation decoder buffer,
the receiver selects a preferred utility parameter from the generated utility parameters and communicates a feedback variable to the real decoder buffer, such that the buffer length is adjusted in accordance with a change in the dynamic characteristic.
17. The apparatus of claim 14 , wherein the utility parameter is a function of the redundancy variable and the buffer length variable.
18. The apparatus of claim 14 , wherein the input is an audio waveform.
19. The apparatus of claim 14 , wherein the input is a video waveform.
20. The apparatus of claim 14 , wherein a feedback packet is transported by the transporting means to the digital waveform encoder such that the redundancy value is adjusted in accordance with the change in the dynamic characteristic.
21. The apparatus of claim 14 , wherein the feedback packet is transported by the transporting means to the packetizer.
22. The apparatus of claim 14 , wherein the data packets include a time stamp, a sequence number, a redundancy parameter, a current frame and a plurality of redundant frames.
23. The apparatus of claim 14 , wherein the transporting network is a packet switched network.
24. The apparatus of claim 21 , wherein the packet switched network is selected from a group consisting of Local Area Networks, Internet Protocol Networks, and frame relay networks, ATM networks, and Wide Area Networks.
25. The apparatus of claim 21 , wherein the packet switched network is an Internet.
26. The apparatus of claim 21 , wherein the packet switched network is an Intranet.
27. A system for transmitting real time media, the system comprising:
a first calling device for placing a call to a first processing hub, the first processing hub comprising:
an encoder for partitioning the call into a plurality of compressed frames, and
a packetizer for packetizing the frames into a data packet according to a redundancy variable;
a transporting network for transporting the data packet between the first hub and a second hub wherein the second hub comprises:
a decoder for decoding the data packet into the plurality of frames and ordering these frames within a buffer having a buffer depth according to a buffer length variable, the decoder generating a plurality of utility parameters based on the redundancy variable and the buffer length variable, wherein the decoder selects a preferred utility parameter from the generated utility parameters such that the redundancy variable and the buffer length variable are adjusted in accordance with a change in the dynamic characteristic; and
a second calling device connected to the second processing hub.
28. The apparatus of claim 25 , wherein the data packets contain a time stamp, a sequence number, a redundancy parameter, a current frame and a plurality of redundant frames.
29. The apparatus of claim 25 , wherein a function of the preferred utility parameter is packeted into a feedback packet and transported to the packetizer by the transporting network.
30. The apparatus of claim 25 , wherein the redundancy value is adjusted in accordance with a dynamic characteristic of the transporting network.
31. The apparatus of claim 25 wherein the data packets contain a time stamp, a sequence number, a redundancy parameter, a current frame and a plurality of redundant frames.
32. The apparatus of claim 25 wherein the transporting network is a packet switched network.
33. The apparatus of claim 30 wherein the packet switched network is selected from a group consisting of Local Area Networks, Internet Protocol Networks, frame relay networks, ATM networks, and Wide Area Networks.
34. The apparatus of claim 30 wherein the packet switched network is the Internet.
35. The apparatus of claim 30 wherein the packet switched network is an Intranet.
36. A method for communicating a real time media input comprising the steps of:
communicating the real time media input to a sending device;
encoding the media input into a plurality of frames, packetizing the frames into data packets, each data packet comprising an ordered plurality of the frames according to a first variable;
transporting the data packets to a receiving device;
unpacking the data packets at the receiving device;
buffering the unpacked data packets according to a second variable,
generating at least two utility parameters, the utility parameters representing a dynamic characteristic of a transporting network that transports the data packets from the sending device to the receiving device,
selecting a preferred utility parameter from the generated utility parameters; and
adjusting the first and the second variable according to the selected preferred utility parameter.
37. The method of claim 34 further comprising the step of adjusting the sending device and the receiving device to a change in the dynamic transporting characteristic.
38. The method of claim 34 further comprising the step of playing out the analog input.
39. The method of claim 34 wherein the real time media input is an audio waveform.
40. The method of claim 34 wherein the real time media input is a video waveform.
41. The method of claim 34 , further comprising the step of selecting the dynamic characteristic from a group consisting of data packet loss, data packet delay, packet burst loss, loss auto-correlation and delay variation.
42. The method of claim 34 , wherein the network comprises a packet switched network.
43. The method of above claim 40 , further comprising the step of selecting the packet switched network from a group consisting of Local Area Networks, Internet Protocol Networks, and frame relays.
44. The method of claim 40 , wherein the transporting network comprises an interconnected switched network of Local Area Networks, Internet Protocol Networks, frame relay networks, ATM networks, and wide Area Networks.
45. The method of claim 40 , further comprising the steps of packetizing a function of the preferred utility parameter into a feedback packet, and transporting the feedback packet to the encoding device by the transporting network.
46. A method for communicating a real time media input comprising the steps of:
partitioning and compressing the real time media input into a plurality of frames at a digital waveform encoder;
packetizing the frames into a plurality of data packets according to an actual redundancy variable;
transporting the data packets by a transporting network from the digital encoder to a receiver;
unpacking the transported data packets into the plurality of frames;
arranging the frames within a buffer according to a buffer length variable;
evaluating a plurality of utility parameters for evaluating a dynamic transporting characteristic of the transporting network;
selecting a preferred utility parameter from the utility parameters; and
adapting the buffer length variable according to a change in the transporting characteristic.
47. The method of claim 44 , further comprising the step of communicating the unpacked frames to a digital waveform decoder.
48. The method of claim 44 , further comprising the step of adopting the actual redundancy variable according to change in the transporting characteristic.
49. The method of claim 45 , further comprising the step of playing out the media input.
50. The method of claim 44 , wherein the real time media input is an audio waveform.
51. The method of claim 44 , wherein the real time media input is a video waveform.
52. The method of claim 44 , further comprising the step of selecting the dynamic characteristic from a group consisting of data packet loss, data packet delay, packet burst loss, loss auto-correlation and delay variation.
53. The method of claim 44 , wherein the network comprises a packet switched network.
54. The method of above claim 51 , further comprising the step of selecting the packet switched network from a group consisting of Local Area Networks, Internet Protocol Networks, and frame relays.
55. The method of claim 51 , wherein the transporting network comprises an interconnected switched network of Local Area Networks, Internet Protocol Networks, frame relay networks, ATM networks, and Wide Area Networks.
56. A method for transmitting real time media between a first and a second processing hub comprising the steps of:
placing a call to the first processing hub,
partitioning and compressing the call at the first processing hub into a plurality of frames,
packetizing the frames at the first processing hub into a data packet according to a redundancy value;
transporting the data packet between the first processing hub and the second processing hub by a transporting network;
decoding the data packet at the second processing hub into the plurality of frames;
ordering these frames within a buffer according to a buffer length variable;
generating a plurality of utility parameters based on the redundancy variable and the buffer length variable;
selecting a preferred utility parameter from the generated utility parameters;
adjusting the redundancy variable and the buffer length variable in accordance with a dynamic transporting characteristic of the transporting network; and
receiving the call from the second processing hub.
57. The method of claim 54 , further comprising the step of communicating the unpacked frames to a decoder from the second processing hub for a play out of the media input.
58. The method of claim 55 , further comprising the step of playing out the media input.
59. The method of claim 54 , wherein the real time media input is an audio waveform.
60. The method of claim 54 , wherein the real time media input is a video waveform.
61. The method of claim 54 , further comprising the step of selecting the dynamic characteristic from a group consisting of data packet loss, data packet delay, packet burst loss, loss auto-correlation and delay variation.
62. The method of claim 54 , wherein the network comprises a packet switched network.
63. The method of above claim 60 , further comprising the step of selecting the packet switched network from a group consisting of Local Area Networks, Internet Protocol Networks, frame relay networks, ATM networks, and Wide Area Networks.
64. The method of claim 60 , wherein the transporting network comprises an interconnected switched network of Local Area Networks, Internet Protocol Networks, frame relay networks, ATM networks, and Wide Area Networks.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/282,403 US20030105874A1 (en) | 1997-10-01 | 2002-10-28 | Method and apparatus for real time communication over packet networks |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US08/942,446 US6175871B1 (en) | 1997-10-01 | 1997-10-01 | Method and apparatus for real time communication over packet networks |
US09/692,499 US6487603B1 (en) | 1997-10-01 | 2000-10-19 | Method and apparatus for real time communication over switched networks |
US10/282,403 US20030105874A1 (en) | 1997-10-01 | 2002-10-28 | Method and apparatus for real time communication over packet networks |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US08/942,446 Continuation US6175871B1 (en) | 1997-10-01 | 1997-10-01 | Method and apparatus for real time communication over packet networks |
US09/692,499 Continuation US6487603B1 (en) | 1997-10-01 | 2000-10-19 | Method and apparatus for real time communication over switched networks |
Publications (1)
Publication Number | Publication Date |
---|---|
US20030105874A1 true US20030105874A1 (en) | 2003-06-05 |
Family
ID=25478078
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US08/942,446 Expired - Lifetime US6175871B1 (en) | 1997-10-01 | 1997-10-01 | Method and apparatus for real time communication over packet networks |
US09/692,499 Expired - Fee Related US6487603B1 (en) | 1997-10-01 | 2000-10-19 | Method and apparatus for real time communication over switched networks |
US10/282,403 Abandoned US20030105874A1 (en) | 1997-10-01 | 2002-10-28 | Method and apparatus for real time communication over packet networks |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US08/942,446 Expired - Lifetime US6175871B1 (en) | 1997-10-01 | 1997-10-01 | Method and apparatus for real time communication over packet networks |
US09/692,499 Expired - Fee Related US6487603B1 (en) | 1997-10-01 | 2000-10-19 | Method and apparatus for real time communication over switched networks |
Country Status (3)
Country | Link |
---|---|
US (3) | US6175871B1 (en) |
AU (1) | AU1064099A (en) |
WO (1) | WO1999017584A2 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060222073A1 (en) * | 2005-03-29 | 2006-10-05 | Guillaume Mercier | Authoring running marks in compressed data |
US20090285202A1 (en) * | 1998-02-25 | 2009-11-19 | John Jutten Lawser | Method for completing internet telephony calls |
US20140068008A1 (en) * | 2012-09-05 | 2014-03-06 | Meiron Tzhori | Method and system for flow controlling |
Families Citing this family (93)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6175871B1 (en) * | 1997-10-01 | 2001-01-16 | 3Com Corporation | Method and apparatus for real time communication over packet networks |
US6434606B1 (en) * | 1997-10-01 | 2002-08-13 | 3Com Corporation | System for real time communication buffer management |
US6366959B1 (en) * | 1997-10-01 | 2002-04-02 | 3Com Corporation | Method and apparatus for real time communication system buffer size and error correction coding selection |
US6744763B1 (en) | 1998-01-15 | 2004-06-01 | Apple Computer, Inc. | Method and apparatus for media data transmission |
US6134243A (en) | 1998-01-15 | 2000-10-17 | Apple Computer, Inc. | Method and apparatus for media data transmission |
US6453355B1 (en) * | 1998-01-15 | 2002-09-17 | Apple Computer, Inc. | Method and apparatus for media data transmission |
JP3589851B2 (en) * | 1998-02-20 | 2004-11-17 | 株式会社日立製作所 | Packet communication system and packet communication device |
SE513520C2 (en) * | 1998-05-14 | 2000-09-25 | Ericsson Telefon Ab L M | Method and apparatus for masking delayed packages |
FI106232B (en) * | 1998-05-19 | 2000-12-15 | Nokia Networks Oy | Audio / video signal transmission in a packet network with variable bit rate |
US6556565B1 (en) * | 1998-07-01 | 2003-04-29 | Nortel Networks Limited | Internet protocol (IP) telecommunication |
CN1126334C (en) * | 1998-12-02 | 2003-10-29 | 皇家菲利浦电子有限公司 | System and method for generoting a real-time signal |
KR100322015B1 (en) * | 1998-12-23 | 2002-03-08 | 윤종용 | Frame Structure Variable Method in Local Area Network |
US6771674B1 (en) * | 1998-12-28 | 2004-08-03 | 3Com Corporation | Method and system for forward error correction based on parallel streams |
JP2000196467A (en) * | 1998-12-28 | 2000-07-14 | Nec Corp | Error correction coder and error correction decoder |
US7215641B1 (en) | 1999-01-27 | 2007-05-08 | Cisco Technology, Inc. | Per-flow dynamic buffer management |
US6515963B1 (en) * | 1999-01-27 | 2003-02-04 | Cisco Technology, Inc. | Per-flow dynamic buffer management |
US7610607B1 (en) * | 1999-02-19 | 2009-10-27 | Chaincast Networks, Inc. | Chaincast method and system for broadcasting information to multiple systems within the internet |
US6901604B1 (en) * | 1999-02-19 | 2005-05-31 | Chaincast, Inc. | Method and system for ensuring continuous data flow between re-transmitters within a chaincast communication system |
US6973140B2 (en) * | 1999-03-05 | 2005-12-06 | Ipr Licensing, Inc. | Maximizing data rate by adjusting codes and code rates in CDMA system |
US6785323B1 (en) | 1999-11-22 | 2004-08-31 | Ipr Licensing, Inc. | Variable rate coding for forward link |
US7102998B1 (en) * | 1999-03-22 | 2006-09-05 | Lucent Technologies Inc. | Scaleable congestion control method for multicast communications over a data network |
FR2795548B1 (en) * | 1999-06-24 | 2001-09-21 | France Telecom | METHOD FOR MANAGING THE DECODING AND RESTORATION OF A SOUND SIGNAL IN AN ASYNCHRONOUS TRANSMISSION SYSTEM |
US6535521B1 (en) * | 1999-06-29 | 2003-03-18 | 3Com Corporation | Distributed speech coder pool system with front-end idle mode processing for voice-over-IP communications |
US6735192B1 (en) * | 1999-09-29 | 2004-05-11 | Lucent Technologies Inc. | Method and apparatus for dynamically varying a packet delay in a packet network based on a log-normal delay distribution |
US6859460B1 (en) * | 1999-10-22 | 2005-02-22 | Cisco Technology, Inc. | System and method for providing multimedia jitter buffer adjustment for packet-switched networks |
DE69934829T2 (en) | 1999-11-09 | 2007-06-28 | OpenTV, Inc., San Francisco | EVENT BOOKING |
GB2356537A (en) * | 1999-11-22 | 2001-05-23 | Mitel Corp | Silence suppression/insertion buffer management |
US6785234B1 (en) * | 1999-12-22 | 2004-08-31 | Cisco Technology, Inc. | Method and apparatus for providing user control of audio quality |
JP3841256B2 (en) * | 2000-02-15 | 2006-11-01 | 三菱電機株式会社 | Communication system, communication method, and transmission terminal |
US20010041981A1 (en) * | 2000-02-22 | 2001-11-15 | Erik Ekudden | Partial redundancy encoding of speech |
GB2360178B (en) * | 2000-03-06 | 2004-04-14 | Mitel Corp | Sub-packet insertion for packet loss compensation in Voice Over IP networks |
US6700895B1 (en) | 2000-03-15 | 2004-03-02 | 3Com Corporation | Method and system for computationally efficient calculation of frame loss rates over an array of virtual buffers |
ES2275481T3 (en) * | 2000-04-14 | 2007-06-16 | Alcatel Lucent | A SELF-ADAPTIVE FLUCTUATION MEMORY. |
US7373413B1 (en) * | 2000-06-28 | 2008-05-13 | Cisco Technology, Inc. | Devices and methods for minimizing start up delay in transmission of streaming media |
US7136577B1 (en) * | 2000-06-29 | 2006-11-14 | Tandberg Telecom As | RTP-formated media clips |
US6728672B1 (en) * | 2000-06-30 | 2004-04-27 | Nortel Networks Limited | Speech packetizing based linguistic processing to improve voice quality |
US7111049B1 (en) | 2000-08-18 | 2006-09-19 | Kyle Granger | System and method for providing internet based phone conferences using multiple codecs |
SG97934A1 (en) * | 2000-09-13 | 2003-08-20 | Mediaring Ltd | Quality of transmission across packet-based networks |
US7249185B1 (en) * | 2000-10-30 | 2007-07-24 | Cisco Technology, Inc. | Methods, devices and software for redundant transmission of voice data over a packet network connection established according to an unreliable communication protocol |
US6930982B2 (en) * | 2000-12-12 | 2005-08-16 | Cisco Technology, Inc. | Devices, software and methods for measuring packet loss burstiness to determine quality of voice data transmission through a network |
JP2002209234A (en) * | 2001-01-11 | 2002-07-26 | Fujitsu Ltd | Communication system |
FI118830B (en) * | 2001-02-08 | 2008-03-31 | Nokia Corp | Streaming playback |
US20040204935A1 (en) * | 2001-02-21 | 2004-10-14 | Krishnasamy Anandakumar | Adaptive voice playout in VOP |
US7016339B1 (en) * | 2001-02-22 | 2006-03-21 | 3Com Corporation | Method and apparatus for real time protocol feedback mixer traversal |
US20020120747A1 (en) * | 2001-02-23 | 2002-08-29 | Frerichs David J. | System and method for maintaining constant buffering time in internet streaming media delivery |
GB0104785D0 (en) * | 2001-02-27 | 2001-04-18 | Pace Micro Tech Plc | MPEG decoder video in the form of cue and/or review streams of data |
US7016315B2 (en) * | 2001-03-26 | 2006-03-21 | Motorola, Inc. | Token passing arrangement for a conference call bridge arrangement |
DE10118192A1 (en) * | 2001-04-11 | 2002-10-24 | Siemens Ag | Transmitting digital signals with various defined bit rates involves varying the number of frames in at least one packet depending on the length of at least one frame in packet |
CA2446617C (en) * | 2001-05-11 | 2010-03-23 | Scientific-Atlanta, Inc. | Channel buffering and display management system for multi-tuner set-top box |
US20020194609A1 (en) * | 2001-06-18 | 2002-12-19 | Tran Thanh T. | Video client with dynamically allocable video buffer for efficiently streaming video |
US6968396B1 (en) * | 2001-07-26 | 2005-11-22 | Openwave Systems Inc. | Reloading of hypermedia pages by sending only changes |
JP3659203B2 (en) * | 2001-08-13 | 2005-06-15 | 日産自動車株式会社 | Vehicle communication control device |
US20030041162A1 (en) * | 2001-08-27 | 2003-02-27 | Hochmuth Roland M. | System and method for communicating graphics images over a computer network |
US7047308B2 (en) * | 2001-08-31 | 2006-05-16 | Sharp Laboratories Of America, Inc. | System and method for simultaneous media playout |
US20030058248A1 (en) * | 2001-09-21 | 2003-03-27 | Hochmuth Roland M. | System and method for communicating graphics over a network |
US7559077B1 (en) * | 2001-11-16 | 2009-07-07 | Sprint Communications Company L.P. | Video system for use where the network transfer rate is slower than the video display rate |
US6754203B2 (en) | 2001-11-27 | 2004-06-22 | The Board Of Trustees Of The University Of Illinois | Method and program product for organizing data into packets |
US8565578B2 (en) * | 2001-12-06 | 2013-10-22 | Harold J. Plourde, Jr. | Dividing and managing time-shift buffering into program specific segments based on defined durations |
US7257308B2 (en) * | 2001-12-06 | 2007-08-14 | Scientific-Atlanta, Inc. | Converting time-shift buffering for personal video recording into permanent recordings |
US7962011B2 (en) * | 2001-12-06 | 2011-06-14 | Plourde Jr Harold J | Controlling substantially constant buffer capacity for personal video recording with consistent user interface of available disk space |
DE10227165A1 (en) * | 2002-06-18 | 2004-01-29 | Siemens Ag | Method and arrangement for coding or decoding a sequence of digital data |
US8181205B2 (en) | 2002-09-24 | 2012-05-15 | Russ Samuel H | PVR channel and PVR IPG information |
US8352991B2 (en) * | 2002-12-09 | 2013-01-08 | Thomson Licensing | System and method for modifying a video stream based on a client or network environment |
EP1579651A1 (en) * | 2002-12-20 | 2005-09-28 | Koninklijke Philips Electronics N.V. | Multi-track hinting for receiver-driven streaming system |
FR2849736A1 (en) * | 2003-01-02 | 2004-07-09 | Thomson Licensing Sa | DEVICE AND METHOD FOR ACQUIRING FILES BY ACCUMULATING POINTS AND RELATED PRODUCTS |
JP2004260658A (en) * | 2003-02-27 | 2004-09-16 | Matsushita Electric Ind Co Ltd | Wireless lan device |
US7474624B2 (en) * | 2003-05-29 | 2009-01-06 | Motorola, Inc. | Method and apparatus for reducing delay jitter |
WO2005050597A1 (en) * | 2003-11-14 | 2005-06-02 | Semiconductor Energy Laboratory Co., Ltd. | Light-emitting device and method for manufacturing the same |
US20050170643A1 (en) * | 2004-01-29 | 2005-08-04 | Semiconductor Energy Laboratory Co., Ltd. | Forming method of contact hole, and manufacturing method of semiconductor device, liquid crystal display device and EL display device |
US20050199642A1 (en) * | 2004-03-15 | 2005-09-15 | Maddock Cindie L. | Programmable snack dispenser and method |
US7542435B2 (en) * | 2004-05-12 | 2009-06-02 | Nokia Corporation | Buffer level signaling for rate adaptation in multimedia streaming |
US7447978B2 (en) * | 2004-11-16 | 2008-11-04 | Nokia Corporation | Buffering packets of a media stream |
EP1821442A4 (en) * | 2004-11-17 | 2011-07-20 | Sharp Kk | Transmitter |
JP4564350B2 (en) * | 2004-12-27 | 2010-10-20 | 株式会社東芝 | Signal processing device |
CN101138200A (en) * | 2005-02-04 | 2008-03-05 | 阿派伦特网络股份有限公司 | Method and apparatus for evaluation of service quality of a real time application operating over a packet-based network |
US20070022215A1 (en) * | 2005-07-19 | 2007-01-25 | Singer David W | Method and apparatus for media data transmission |
US7962637B2 (en) | 2006-11-03 | 2011-06-14 | Apple Computer, Inc. | Dynamic adjustments of video streams |
TWI337043B (en) * | 2007-03-30 | 2011-02-01 | Qisda Corp | Data transmission method and audio/video system capable of splitting and synchronizing audio/video data |
US20090025052A1 (en) * | 2007-07-18 | 2009-01-22 | General Instrument Corporation | Method and Apparatus for Controlling the Bandwidth of SDV Programming Supplied to an Edge Device in a n SDV System |
US20090138614A1 (en) * | 2007-11-28 | 2009-05-28 | Motorola, Inc. | Method and apparatus for streaming media in a communication network |
US9167007B2 (en) * | 2008-06-06 | 2015-10-20 | Amazon Technologies, Inc. | Stream complexity mapping |
US9047236B2 (en) * | 2008-06-06 | 2015-06-02 | Amazon Technologies, Inc. | Client side stream switching |
US20100011274A1 (en) * | 2008-06-12 | 2010-01-14 | Qualcomm Incorporated | Hypothetical fec decoder and signalling for decoding control |
US9692856B2 (en) * | 2008-07-25 | 2017-06-27 | Ca, Inc. | System and method for filtering and alteration of digital data packets |
US8401990B2 (en) * | 2008-07-25 | 2013-03-19 | Ca, Inc. | System and method for aggregating raw data into a star schema |
CN101924955A (en) * | 2009-06-16 | 2010-12-22 | 中兴通讯股份有限公司 | Method and system for improving play quality of mobile TV |
US8966110B2 (en) * | 2009-09-14 | 2015-02-24 | International Business Machines Corporation | Dynamic bandwidth throttling |
KR101262846B1 (en) * | 2009-12-15 | 2013-05-10 | 한국전자통신연구원 | Apparatus and method for measuring the performance of embedded devices |
US9521178B1 (en) | 2009-12-21 | 2016-12-13 | Amazon Technologies, Inc. | Dynamic bandwidth thresholds |
US20110193970A1 (en) * | 2010-02-11 | 2011-08-11 | Analogix Semiconductor, Inc. | Reducing Jitter in a Recovered Data Stream Clock of a Video DisplayPort Receiver |
WO2013187741A1 (en) * | 2012-06-15 | 2013-12-19 | Lg Electronics Inc. | Method and device for transmitting and receiving multi-input multi-output (mimo) signal in wireless communication system in which plural communication systems are present |
JP2014093584A (en) * | 2012-11-01 | 2014-05-19 | Sony Corp | Transmission device, transmission method, receiving device, reception method and computer program |
US12041303B1 (en) * | 2018-03-19 | 2024-07-16 | Amazon Technologies, Inc. | Bandwidth estimation for video encoding |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5446734A (en) * | 1990-04-13 | 1995-08-29 | Digital Equipment Corporation | Telecommunications network with plesiochronous transfer mode |
US5812790A (en) * | 1996-09-09 | 1998-09-22 | Sonic Solutions | Variable encoding rate plan generation |
US5859979A (en) * | 1993-11-24 | 1999-01-12 | Intel Corporation | System for negotiating conferencing capabilities by selecting a subset of a non-unique set of conferencing capabilities to specify a unique set of conferencing capabilities |
US5931928A (en) * | 1997-05-07 | 1999-08-03 | 3Com Corporaton | System for ISDN terminal adapter DCE for automatically negotiating data compression with it's PPP peer when DTE is unable or unwilling to negotiate compression |
US5949975A (en) * | 1997-03-12 | 1999-09-07 | Microsoft Corp. | Method and system for negotiating capabilities when sharing an application program with multiple computer systems |
US5953506A (en) * | 1996-12-17 | 1999-09-14 | Adaptive Media Technologies | Method and apparatus that provides a scalable media delivery system |
US6005846A (en) * | 1997-05-07 | 1999-12-21 | 3Com Corporation | Apparatus for an improved ISDN terminal adapter having automatic SPID configuration and methods for use therein |
US6031848A (en) * | 1997-05-07 | 2000-02-29 | 3Com Corporation | Apparatus for an improved ISDN terminal adapter having baud rate unblocking and methods for use therein |
US6175871B1 (en) * | 1997-10-01 | 2001-01-16 | 3Com Corporation | Method and apparatus for real time communication over packet networks |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AU1572995A (en) | 1994-02-11 | 1995-08-29 | Newbridge Networks Corporation | Method of dynamically compensating for variable transmission delays in packet networks |
US5825771A (en) | 1994-11-10 | 1998-10-20 | Vocaltec Ltd. | Audio transceiver |
US6366959B1 (en) * | 1997-10-01 | 2002-04-02 | 3Com Corporation | Method and apparatus for real time communication system buffer size and error correction coding selection |
US6233226B1 (en) * | 1998-12-14 | 2001-05-15 | Verizon Laboratories Inc. | System and method for analyzing and transmitting video over a switched network |
-
1997
- 1997-10-01 US US08/942,446 patent/US6175871B1/en not_active Expired - Lifetime
-
1998
- 1998-10-01 WO PCT/US1998/020506 patent/WO1999017584A2/en active Application Filing
- 1998-10-01 AU AU10640/99A patent/AU1064099A/en not_active Abandoned
-
2000
- 2000-10-19 US US09/692,499 patent/US6487603B1/en not_active Expired - Fee Related
-
2002
- 2002-10-28 US US10/282,403 patent/US20030105874A1/en not_active Abandoned
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5446734A (en) * | 1990-04-13 | 1995-08-29 | Digital Equipment Corporation | Telecommunications network with plesiochronous transfer mode |
US5859979A (en) * | 1993-11-24 | 1999-01-12 | Intel Corporation | System for negotiating conferencing capabilities by selecting a subset of a non-unique set of conferencing capabilities to specify a unique set of conferencing capabilities |
US5812790A (en) * | 1996-09-09 | 1998-09-22 | Sonic Solutions | Variable encoding rate plan generation |
US5953506A (en) * | 1996-12-17 | 1999-09-14 | Adaptive Media Technologies | Method and apparatus that provides a scalable media delivery system |
US5949975A (en) * | 1997-03-12 | 1999-09-07 | Microsoft Corp. | Method and system for negotiating capabilities when sharing an application program with multiple computer systems |
US5931928A (en) * | 1997-05-07 | 1999-08-03 | 3Com Corporaton | System for ISDN terminal adapter DCE for automatically negotiating data compression with it's PPP peer when DTE is unable or unwilling to negotiate compression |
US6005846A (en) * | 1997-05-07 | 1999-12-21 | 3Com Corporation | Apparatus for an improved ISDN terminal adapter having automatic SPID configuration and methods for use therein |
US6031848A (en) * | 1997-05-07 | 2000-02-29 | 3Com Corporation | Apparatus for an improved ISDN terminal adapter having baud rate unblocking and methods for use therein |
US6175871B1 (en) * | 1997-10-01 | 2001-01-16 | 3Com Corporation | Method and apparatus for real time communication over packet networks |
US6487603B1 (en) * | 1997-10-01 | 2002-11-26 | 3Com Corporation | Method and apparatus for real time communication over switched networks |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090285202A1 (en) * | 1998-02-25 | 2009-11-19 | John Jutten Lawser | Method for completing internet telephony calls |
US8520665B2 (en) * | 1998-02-25 | 2013-08-27 | At&T Intellectual Property Ii, L.P. | Method for completing internet telephony calls |
US9571327B2 (en) | 1998-02-25 | 2017-02-14 | At&T Intellectual Property Ii, L.P. | Method for completing internet telephony calls |
US10171676B2 (en) | 1998-02-25 | 2019-01-01 | At&T Intellectual Property Ii, L.P. | Method for completing internet telephony calls |
US20060222073A1 (en) * | 2005-03-29 | 2006-10-05 | Guillaume Mercier | Authoring running marks in compressed data |
US20140068008A1 (en) * | 2012-09-05 | 2014-03-06 | Meiron Tzhori | Method and system for flow controlling |
US9699103B2 (en) * | 2012-09-05 | 2017-07-04 | Flash Networks, Ltd | Method and system for flow controlling |
Also Published As
Publication number | Publication date |
---|---|
WO1999017584A2 (en) | 1999-04-08 |
US6175871B1 (en) | 2001-01-16 |
AU1064099A (en) | 1999-04-23 |
WO1999017584A3 (en) | 1999-05-20 |
US6487603B1 (en) | 2002-11-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6175871B1 (en) | Method and apparatus for real time communication over packet networks | |
US6366959B1 (en) | Method and apparatus for real time communication system buffer size and error correction coding selection | |
US6434606B1 (en) | System for real time communication buffer management | |
US7920492B1 (en) | Devices, softwares and methods for redundantly encoding a data stream for network transmission with adjustable redundant-coding delay | |
US6421720B2 (en) | Codec-independent technique for modulating bandwidth in packet network | |
US8830865B2 (en) | Adaptive packet size modification for packet networks | |
US8451723B2 (en) | Soft packet dropping during digital audio packet-switched communications | |
US7668968B1 (en) | Closed-loop voice-over-internet-protocol (VOIP) with sender-controlled bandwidth adjustments prior to onset of packet losses | |
US20050232309A1 (en) | In band signal detection and presentation for IP phone | |
US20080117906A1 (en) | Payload header compression in an rtp session | |
KR100941564B1 (en) | Transmission of a digital message interspersed throughout a compressed information signal | |
US20020018443A1 (en) | Method to have a real time data communication | |
US6661793B1 (en) | Method and apparatus for reconstructing media | |
JP2004535115A (en) | Dynamic latency management for IP telephony | |
JP4217121B2 (en) | Voice quality evaluation method and voice quality adjustment apparatus in IP network system | |
CA2442505C (en) | Lan communication method and system for transmitting and receiving packets with correction code data | |
US7299176B1 (en) | Voice quality analysis of speech packets by substituting coded reference speech for the coded speech in received packets | |
US6546009B1 (en) | Method of reducing delays in packet data transmission | |
Chin et al. | Enhancing the quality of Internet voice communication for Internet telephony systems | |
Francis‐Cobley et al. | Voice over IP versus voice over frame relay | |
Sherif et al. | CCITT/ANSI voice packetization protocol | |
Rosenberg et al. | Issues and Options for an Aggregation Service within RTP | |
Foo Siu Cheung Hui et al. | Enhancing the quality of low bit‐rate real‐time Internet communication services | |
Bolot et al. | Sound and Video on the Web | |
Chin et al. | An Internet telephone software system for real-time voice communication |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |