CA2804741A1 - Media server - Google Patents

Media server

Info

Publication number
CA2804741A1
CA2804741A1 CA 2804741 CA2804741A CA2804741A1 CA 2804741 A1 CA2804741 A1 CA 2804741A1 CA 2804741 CA2804741 CA 2804741 CA 2804741 A CA2804741 A CA 2804741A CA 2804741 A1 CA2804741 A1 CA 2804741A1
Authority
CA
Grant status
Application
Patent type
Prior art keywords
rtmp
server
client
bit rate
media
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
CA 2804741
Other languages
French (fr)
Inventor
Varadhan Venkataseshan
Seyed M. Sharif-Ahmadi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
MIMIK TECHNOLOGY Inc
Original Assignee
MIMIK TECHNOLOGY INC.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Application independent communication protocol aspects or techniques in packet data networks
    • H04L69/30Definitions, standards or architectural aspects of layered protocol stacks
    • H04L69/32High level architectural aspects of 7-layer open systems interconnection [OSI] type protocol stacks
    • H04L69/322Aspects of intra-layer communication protocols among peer entities or protocol data unit [PDU] definitions
    • H04L69/329Aspects of intra-layer communication protocols among peer entities or protocol data unit [PDU] definitions in the application layer, i.e. layer seven
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements or protocols for real-time communications
    • H04L65/60Media handling, encoding, streaming or conversion
    • H04L65/601Media manipulation, adaptation or conversion
    • H04L65/602Media manipulation, adaptation or conversion at the source
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements or protocols for real-time communications
    • H04L65/60Media handling, encoding, streaming or conversion
    • H04L65/608Streaming protocols, e.g. RTP or RTCP
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements or protocols for real-time communications
    • H04L65/80QoS aspects
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television, VOD [Video On Demand]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of content streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of content streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television, VOD [Video On Demand]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of content streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of content streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/23439Processing of video elementary streams, e.g. splicing of content streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements for generating different versions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television, VOD [Video On Demand]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2402Monitoring of the downstream path of the transmission network, e.g. bandwidth available
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L29/00Arrangements, apparatus, circuits or systems, not covered by a single one of groups H04L1/00 - H04L27/00 contains provisionally no documents
    • H04L29/02Communication control; Communication processing contains provisionally no documents
    • H04L29/06Communication control; Communication processing contains provisionally no documents characterised by a protocol
    • H04L29/08Transmission control procedure, e.g. data link level control procedure
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing packet switching networks
    • H04L43/08Monitoring based on specific metrics
    • H04L43/0876Network utilization
    • H04L43/0882Utilization of link capacity
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing packet switching networks
    • H04L43/08Monitoring based on specific metrics
    • H04L43/0876Network utilization
    • H04L43/0894Packet rate
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Application independent communication protocol aspects or techniques in packet data networks
    • H04L69/16Transmission control protocol/internet protocol [TCP/IP] or user datagram protocol [UDP]
    • H04L69/161Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
    • H04L69/162Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields involving adaptations of sockets based mechanisms

Abstract

A RTMP Server initiated bit rate switching in RTMP live media streaming is provided, directed to fully RTMP server originated Adaptive bit rate switching of live media stream containing H.264 video and AAC audio codecs served to RTMP based client players..

Description

MEDIA SERVER
TECHNICAL FIELD
[0001] The present disclosure is directed at a system and method for managing streaming media content delivery, and more particularly for managing the bit rate of such media content delivery between a server and a client.
BACKGROUND

[0002] The Internet allows the transmission and play of media content, and typically, a media server is responsible for obtaining the media content from a content source, and then transmitting the content to a client. Media servers may use HTTP, TCP, RTSP, or more commonly, RTMP or UDP, to deliver streaming content.

[0003] Media content is transmitted to mobile playback devices such as smart phones, tablets, laptop computers, and the like. Transmission of media content to such mobile devices offers additional challenges, as the bandwidth of the communication channel is typically reduced, and the processing power of the device itself is typically less than that of an ordinary computer or special purpose device.

[0004] In live streaming, the throughput of the communications channel used to transmit media content is highly variable and difficult to predict. Accordingly, media servers typically store several different versions of each media content, with each different version optimized for different communications channel throughput, and the appropriate version is selected for transmission based upon the communications channel throughput.

[0005] However, while the bit rate of most streamed media content is temporally constant, this is not the case where variable bit rate encoding is employed. With variable bit rate encoding, the media content is encoded using a bit rate that varies with the program material, and hence, varies with time as well. These temporal variations of the bit rate give rise to situations wherein the bandwidth of the communications channel is sufficient or more than is required for portions of the media program, but insufficient for other portions. Since decisions are made with respect to current throughput and the average bit rate of the media content stream, such variations cause {E6323648.DOC;1}

two problems: (1) the buffer becoming empty during the bitrate peak, resulting in choppy playback, and (2) the media program player may not switch up to a higher bitrate during a valley, which results in lower quality video displayed to the user who is capable of viewing a higher quality stream.

[0006] RTMP (Real Time Messaging Protocol) is a system for delivering on-demand and live media content to Adobe Flash applications or to any RTMP complaint players.
RTMP uses TCP
as the underlying protocol and maintains persistent connections between the server and a client.
To deliver streams smoothly and transmit as much information as possible, the protocol divides streams into "chunks" with a size negotiated dynamically between the client and server.
Fragments from different streams may be interleaved and multiplexed over a single connection.

[0007] RTMP defines several virtual channels on which packets may be sent and received, and which operate independently of each other. For example, there is a channel for handling RPC
requests and responses, a channel for a video stream, a channel for an audio stream and a channel for out-of-band control messages. During a typical RTMP session, several channels may be active simultaneously at any given time. When RTMP data is encoded, a packet header is generated. The packet header includes the ID of the channel on which the packet is to be sent, a timestamp of when the packet was generated (if necessary), and the size of the packet's payload.
This header is then followed by the actual payload content of the packet, which is fragmented according to the currently agreed-upon fragment size before it is sent over the connection. The packet header itself is never fragmented, and its size does not count towards the data in the packet's first fragment. In other words, only the actual packet payload (the media content data) is subject to fragmentation.

[0008] At a higher level, the RTMP encapsulates AAC audio and H.264 video multimedia streams with appropriate timing information. The RTMP includes remote procedure calls (RPCs) made using the Action Message Format.

[0009] Adaptive multi bit rate media streaming is known in the art and is the process of delivering streaming video to users by dynamically switching among different streams of varying quality and size without disrupting the continuous playback. Adaptive streaming may be the {E6323648.DOC; I }

single most important capability for delivering the optimum quality of service to a diverse range of viewers and devices.

[00010] In a typical prior art embodiment for RTMP based dynamic adaptive multi bit rate streaming for an RTMP based server and client, as shown in Figure 1, a streaming media player 210 is installed on a client device 240, typically as a web browser plugin.
The RTMP client 200 requests streaming media from a media server 110, which may be part of a gateway 150. The media server 110 provides the streaming media player 210 operating on the client 240, an initial request for the appropriate configuration file, such as SMIL (Synchronized Multimedia Integration Language) or for information including the RTMP connection URL and the available video bit-rates for the client.

[00011] An example of a small section from a SMIL sample showing the RTMP:URL and available bit rates follows:
<smil>
<head>
<meta base="RTMP://medisaserver.com/live" />
</head>
<body>
<switch>
<video src="chl -256" system-bitrate="1208" />
<video src="ch1-256" system-bitrate="256" />
<video src="ch1-5120" system-bitrate="5120" />
<video src="ch1-1024" system-bitrate="1024"/>
</switch>
</body>
</smil>

[00012] As shown in Figure 2, the RTMP client 200 first directly establishes a TCP
session with the RTMP server 120. After establishing the TCP connection, the RTMP client 200 initiates the RTMP Handshake process. Following a successful RTMP specific handshake process, the client and server exchanges a series of complex commands and Remote Procedure (E6323648.DOC;1) Calls, which are mostly Action Message Format (AMF) encoded. AMF is a compact binary format that is used to serialize ActionScript object graphs.

[00013] The RTMP client 200 then sends a RTMP Netconnection::connect() Command Message to the server requesting connection to a server application instance (step 1). The RTMP
server 120 sends the _result command message (step 2) informing the RTMP
client 200 of the connection status (success/fail) (step 3).

[00014] On a successful RTMP connect response, the RTMP client 200 then sends a Netconnection::createStream command (step 4) to the RTMP server 120 to create a logical channel for message communication. Audio, video, and metadata flows over this steam channel created using the createStream command.

[00015] Then, on a successful createStream response from the RTMP server 120 (step 5), the RTMP client 200 sends the NetStream::play command (step 6). The NetStream defines the channel through which the streaming audio, video, and data messages can flow over the NetConnection that connects the RTMP client 200 to the RTMP server 120. The "play"
command object contains a Stream Name field, which has the live channel or the media that has to be streamed by the RTMP server 120. On receiving the play command, the RTMP
server 120 sends a protocol message to set the chunk size (step 7).

[00016] Then RTMP server 120 sends a protocol message (user control) specifying the event 'StreamBegin', to indicate beginning of the streaming to the RTMP client 200 (step 8).
Then RTMP server 120 sends OnStatus command messages NetStream.Play.Start if the play command sent by the RTMP client 200 is successful (step 9). Then RTMP server 120 sends the relevant audio and video config (step 10) and then starts streaming the media content by continuously sending video and audio messages in chunks to RTMP client 200 (steps 12 and 13).

[00017] Streaming Media Player 210 and RTMP client 200 use heuristics based on factors such as device capabilities, estimated buffer occupancy and the estimated network bandwidth available, when deciding to switch to a different bit rate stream. RTMP client also chooses the appropriate bandwidth channel as provided in the initial configuration.
{E6323648.DOC; 1) [00018] RTMP client 200 then sends NetStream::play2 command requests (step 14) to RTMP server 120. Unlike the play command, play2 can switch to a different bit rate stream without changing the timeline of the content played. The command structure of play2 from RTMP client 200 contains "streamName" that has the name of new bit rate stream.

[00019] The RTMP server 120 receives the RTMP client 200 originated play2 bit rate change request. Based on the client rate change request, the RTMP server 120 handles the play2 request (steps 15 and 16) and then appropriately switches to the new bit rate stream for the video messages (the audio bit rate remains fixed) at the optimal point with minimal playback impact (steps 17 and 18). Then the RTMP server 120 continues to send the audio and video in chunks to RTMP client 200.

[00020] What is needed is a method and apparatus allowing selection, by a RTMP server, of media content bit-streams according to time-varying bit rate requirements.
The present invention satisfies this need.
SUMMARY OF THE INVENTION

[00021] A method of streaming multi bit rate media to a RTMP client is provided. In the system according to invention, a RTMP server implements a fully Adaptive Dynamic Multi Bitrate by not using the client initiated RTMP "NetStream::play2" invocation bit rate switching request.

[00022] In the system according to the invention, the RTMP streaming server completely initiates the bitrate switching of live streaming media, unlike the usual method wherein the RTMP client player originates a bit rate switch request for RTMP.

[00023] In the system according to the invention, the RTMP client is not provided with a multibit rate option to choose a bit rate and the RTMP client has no influence on the bandwidth requested.

[00024] As an example, suppose the RTMP client is provided with one simple connection URL such as RTMP://mediaserver_ip/live/ch[Chanrielid]. The client or RTMP
player is completely unaware of the available video quality or bit rates of the media before the request.
{E6323648.DOC; 1}

[00025] The use of fully server initiated adaptive multi bit rate switching for serving RTMP clients has some benefits for the RTMP media server. The advantages of this approach for the server include:

[00026] 1. Usually the configuration files involving adaptive multi bit rate support are very elaborate or complicated, and vary for different media player's plug-ins.
By using a server imitated, multi bit rate switching technique, media servers only need to provide a simple RTMP
stream access URL to the client.

[00027] 2. The system works with any RTMP players or flash based players (clients).

[00028] 3. The system provides improved Quality of Service (QoS) to the clients and a fair share of network bandwidth utilization by servers distributed in a carrier network.

[00029] 4. The RTMP streaming server need not have to rely on the client to choose the right bit rate stream and instead can decide the band width based on its own heuristics.
BRIEF DESCRIPTION OF THE FIGURES

[00030] Figure 1 is a block diagram showing an environment in which the RTMP media server and RTMP media player operate.

[00031] Figure 2 is a state diagram showing the prior art communication process between a prior art RTMP server and a RTMP client.

[00032] Figure 3 is a state diagram showing the communication between a RTMP server and RTMP client according to the invention.

[00033] Figure 4 is a block diagram of a media server and RTMP server according to the invention.
DETAILED DESCRIPTION

[00034] The following details the work flow of the system according to the invention to allow media server 110 initiated adaptive dynamic multi bitrate switching of a live stream that is transparent to the RTMP client 200.
(E6323648.DOC; I) [00035] As shown in Figures 1 and 3, streaming media player 210, installed mostly as a web browser plug in with a RTMP Client 200, requests media from Media Server 110. The RTMP Client 200 refers to an Adobe Flash Player or any other equivalent player that supports playback of audio and video streamed from RTMP servers. Media Server 110, just provides the Streaming Media Player 210 a simple RTMP media request URL, such as RTMP://medisaserver.com/live/ch 1 . Media server 110 may be within an Internet based server, or may be within a gateway device 150 connecting a user device to a local network, which in turn is connected to the Internet.

[00036] The steps taken in the communications between RTMP server 120 and RTMP
client 200 are similar to those in the prior art until the initial play command. RTMP client 200 first directly establishes a successful TCP session with RTMP Server 120 and initiates the RTMP
Handshake process. Upon a successful handshake process, RTMP client 200 and RTMP server 120 exchanges a series of complex commands and Remote Procedure Calls, which are mostly Action Message Format(AMF) encoded.

[00037] RTMP client 200 then sends the RTMP Netconnection::connect() Command Message (step 1) to RTMP server 120 requesting connection to a server application instance.
RTMP server sends the _result command message (step 2) informing the RTMP
client 200 of the connection status (success/fail) (step 3).

[00038] On a successful RTMP connect response, the RTMP client 200 then sends Netconnection::createStream command (step 4) to the RTMP server 120 to create a logical channel for message communication. Audio, video, and metadata flows over this stream channel created using the createStream command. On a successful createStream response (step 5) from RTMP server 120, the RTMP client 200 sends the NetStream::play command (step 6). The NetStream defines the channel through which the streaming audio, video, and data messages can flow over the NetConnection that connects the RTMP client 200 to the RTMP
server 120.

[00039] The "play" command object contains a Stream Name field, that has the live channel or the media that has to be streamed by the RTMP server 120, for example the Stream Name will contain something like "chi ". As the RTMP client 200 is not provided with available media bit rate options, RTMP client just issues the request for the channel provided, for {E6323648.DOC;1}

example, "chi". On receiving the play command, RTMP server 120 sends a protocol message to set the chunk size (step 7). Then RTMP server 120 sends protocol message (user control) specifying the event 'StreamBegin' (step 8), to indicate beginning of the streaming to the RTMP
client 200. Then RTMP server 120 sends OnStatus command messages NetStream.Play.Start (step 9) if the play command sent by the RTMP client 200 is successful.

[00040] RTMP server 120, based on its own Adaptive channel Bandwidth detection heuristics, along with other factors and logic, requests the transcoder for the media stream at the determined QoS. . Audio and video metadata obtained on reading the MPEGTS
stream is sent to RTMP client 200 as AMF encoded RTMP data with a notify "onMetadata" object (step 10).
When the media becomes available, then RTMP server 120 sends the relevant AAC
audio and AVC H.264 video configuration details (step 11) and the AAC audio and AVC
H.264 video data to the RTMP client 200 (steps 12 and 13). The RTMP server 120 typically supports H.264/MPEG-4 Part 10 or AVC(Advanced Video Coding) and Advanced Audio Coding (AAC) standards.

[00041] The RTMP server 120 then continuously monitors the network condition and when its unbiased estimated network bandwidth warrants a change in the video bit rate for the channel, RTMP server triggers the media transcoder to change the video bit rate beginning with an appropriate Group of Pictures ("GOP") of the video. Once the transcoded new bit rate stream is ready, RTMP server 120 sends the relevant audio and video config messages (steps 15 and 16) and then appropriately switches to the new bit rate stream for the video messages (the audio bit rate remains fixed) at the optimal point with minimal playback impact (steps 17 and 18). Then the RTMP server 120 continues to send the audio and video in chunks to RTMP
client 200. Of note, step 14, the Command (play2) message, is no longer needed, as the RTMP
server 120 selects the video bit rate.

[00042] Figure 4 is a block diagram of a system describing the views and embodiments of a dynamic adaptive multi bitrate RTMP server 120. System TC/IP stack 103 is an integral part of the underlying operating system's TCP/IP network stack software that implements Transmission Control Protocol (TCP) and the Internet Protocol (IP) Internet protocol suite.
{E6323648.DOC; 11 [00043] System Resource Monitor 102 is a software program that is a component of Media Server 110, and which monitors the system CPU and memory resources and advises on system available resources to carry out the requested advanced H.264 video compression.
Tunable Media Transcoder 101 is a software program that is a highly tunable stream media transcoder to achieve the desired speed, compression ratio, stream quality and desired bit rate stream.

[00044] Channel TCP KPI and Network Monitor 104 is a software program that interacts with System TC/IP stack 103 periodically or upon request or at other appropriate times, to get channel related various details such as TCP metadata, and network Key Performance Indicator (KPI), related to a TCP session between RTMP Server 120 and the RTMP Client 200.

[00045] Stream Bandwidth Estimator and Dynamic Tuner 107 is a software program that carries out various heuristics to predict the desired Bandwidth for switching stream and also decides on the optimal H.264 transcoding tuning parameters suitable for next stream switching.

[00046] Stream Channel Manager 106 is a software program that manages the RTMP
stream channel. RTMP Protocol handler 108 handles all the RTMP protocol related activities.
Channel Aware Stream Transport Scheduler 105 is a dynamic stream transport scheduler that actually sends media packets at channel Bandwidth through TCP stack to the RTMP client 200.

[00047] Optimally switching the bit rate of compressed video in synchronization with the dynamic network capacity is an important and non-trivial process in delivering an adaptive bit rate stream completely transparent to RTMP client 200. Adaptive Dynamic Streaming is used to maximize Quality of Service by switching from one bitrate to another depending on the changing network channel conditions. This is achieved by continuously monitoring CPU
and memory resource availability, dynamic network conditions, RTMP client 200 capabilities, and then using this information as feedback to tune the transcoder to get desired stream quality, compression efficiency and bandwidth.

[00048] Since RTMP uses TCP as the underlying transport protocol, exploiting the information provided by the TCP stack 103 on the connection is useful for predicting the optimal {E6323648.DOC; 1) stream Bandwidth time. TCP, with its built-in flow control, can already provide implicit feedback to the RTMP server 120 and thus no modification to the RTMP client 200 is necessary.

[00049] A simple flow of logic involved with adaptive bandwidth switching follows:
deciding on the initial stream bandwidth among the range of supported bandwidth by the transcoder 101 starts with a simple heuristic. Although the amount of data transferred back and forth to the RTMP client 200 before play is not significant, initial RTMP
protocol message exchanges prior to actual play command from RTMP client 200, such as the RTMP
handshake, Netconnection::connect, CreateStream, etc. are useful as a starting point of bandwidth estimation using the low pass filtered Round Trip Time (rtt) obtained by probing TCP
stack 103 for the channel. For example, in a LAN these initial rtts were found to be less than 5 milliseconds and in a heavily loaded Wi-Fi, in the range of few lOs of milliseconds, and in a 3G network, in the order of 100s of milliseconds. Based on the rtt and filtered smoothed rtt (srtt) obtained thus far, initial bandwidth for starting the stream may be decided, for example, by using a memory-less, continuous-time Markov process model. This assists in determining a stream bandwidth suitable for the current network condition.

[00050] Using the transcoding options, the starting media stream bandwidth is requested from transcoder 101. The transcoded stream is provided by transcoder 101 as a MPEGTS stream with AAC audio and AVC(H.264) video. This MPEGTS media stream in opened and parsed by RTMP server 120. First audio and video metadata are obtained from reading the MPEGTS
stream. This information is sent to RTMP client 200 as an AMF encoded RTMP
data notify "onMetadata" object. Then the RTMP/FLASH encoded AAC Audio Config and AVC
Video Config is sent to the RTMP client 200. Next, the the actual time synchronized audio and video data are sent to the RTMP client 200.

[00051] After the initial phase of sending some media to the RTMP client 200, TCP stack 130 is continuously probed, using the connected socket descriptor of the established client server TCP session, to get in-depth values of TCP congestion control specific information and TCP
state variables. These variables may include: rtt; smoothed low pass filtered rtt(srtt), slow start threshold (ssthresh); congestion window (cwnd); receiver window (rwnd);
maximum segment size (mss); amount of retransmission; TCP send buffer size; amount of unsent data in the socket (E6323648.DOC; 1) send queue; packets that are in flight; lost packets; flight size (the amount of data has been sent but has not been acknowledged); and others.

[00052] A number of heuristics, using the channel connection state variables obtained from TCP along with application of standard deviation and exponential smoothing techniques to time series data, are employed to obtain the optimal bandwidth estimation for switching the bit rate of next GOP of pictures (i.e. all frames starting from an I frame until the next I frame).

[00053] In one embodiment of determining bandwidth estimation, a simple conservative approach may be taken, to use the data acknowledged bandwidth instead of the transmit bandwidth, such as for example, [00054] Let Tx[i] be the total data sent by RTMP server 120 to RTMP client 200 at a given instance i. Let Ta[i] be the actual amount of data acknowledged to TCP stack 103 by RTMP
client 200 at a given instance i. Let At be the time elapsed to get Ta[i], then the acknowledgement based bandwidth estimate (AckBWe) is calculated as follows:
AckBWe[i] = Ta[i]/At [00055] In an alternative embodiment, the bandwidth estimate at a given instance i is computed as a function of the TCP congestion window (cwnd), maximum segment size (mss) and smoothed round trip time (srtt) as follows:
BWe[i] = (cwnd[i] * mss[i])/srtt[i]

[00056] The following time-varying discrete time filter can be used to smooth the oscillation in the bandwidth estimate:
ffiWe[i] = (1 - exp(-8t/T0)) * BWe[i] + exp(-St/T0) * fBWe[i-1 ]
where fBWe[i] is the infinite impulse response filtered bandwidth estimate at a given instance i;
fBWe[i-1] is the infinite impulse response filtered bandwidth estimate at a given instance prior to i; BWe[i] is the instantaneous bandwidth estimate obtained as described above;
To is a time constant, for example, 1 (one) second; St is the time interval between the last two estimates; and {E6323648.DOC; I}

the exp(x) function returns the value of e (the base of natural logarithms) raised to the power of x.

[00057] Such impulse response filtering reduces the frequency in switching up and down due to network bandwidth oscillations, for example from 768 Kbps to 1024 Kbps and back to 768 Kbps from 1024Kbps. In such cases it is prudent to stay at 768 Kbps to avoid the switching artifacts from going back and forth between 768 kbps and 1024 kbps. Also TCP
stack socket 103 sends the buffer occupancy, which when estimated properly, can reflect the RTMP client 200 buffer occupancy. Such embodiment of bandwidth estimation closely following the underlying TCP transport protocol, leads to an optimal and intelligent dynamic adaptive rate switching to provide a good QoS and smooth streaming to the RTMP client 200, completely transparent to client 200.

[00058] Based on the new change in stream bandwidth, optimal video transcoding options are determined and fed to the tunable transcoder 101 for getting the desired streaming bit rate.
When the new rate stream becomes available, then at the beginning of the video key frame, time synched properly with audio will be sent to the RTMP client 200. This process of dynamically adapting to the changing network condition continues until RTMP client 200 terminates the connection or when the media is finished.

[00059] Server 120 initiated bandwidth adaption of the stream is achieved for live RTMP
stream, without RTMP client 200 actually asking for change of bit rate, by switching to the new bandwidth stream at the next nearest matching H.264 keyframe video and also appropriately synchronizing audio and video.

[00060] During the initial connection to the Media Server 110, the RTMP
client 200 is not provided with any option to select the bit rate and is just given the live channel name along with the RTMP connection URL that is used to request the media.

[00061] The adaptive bit rate switching is made possible directly by the RTMP server 120 by changing the transmitted media bit rate when appropriate. The RTMP client 200, when using the system according to the invention, need not issue a play2 switch command to RTMP server 120.
(E6323648.DOC; 1}

[00062] For the sake of convenience, the embodiments above are described as various interconnected functional blocks or distinct software modules. This is not necessary, however, and there may be cases where these functional blocks or modules are equivalently aggregated into a single logic device, program or operation with unclear boundaries. In any event, the functional blocks and software modules or features of the flexible interface can be implemented by themselves, or in combination with other operations in either hardware or software.

[00063] While particular embodiments have been described in the foregoing, it is to be understood that other embodiments are possible and are intended to be included herein. It will be clear to any person skilled in the art that modifications of and adjustments to the foregoing embodiments, not shown, are possible.
{E6323648.DOC; 1)

Claims (3)

1. A media server, comprising a. a RTMP server configured to communicate streaming media content to a RTMP
client;
b. a stream channel monitor configured to evaluate the quality of the data stream between the RTMP server and the RTMP client;
c. a bandwidth estimator for providing an estimate of the available bandwidth available for streaming the media content;
d. a tunable transcoder for streaming the media content to the RTMP server at the estimated available bandwidth.
2. A method of providing dynamic adaptive multi bit rate live streaming for a RTMP client from an RTMP server, comprising:
a. said RTMP server determining a first video bit rate for said live streaming in consideration of network conditions;
b. said RTMP server sending said video stream to said RTMP client;
c. said RTMP server determining a second bit rate in consideration of changes in network conditions;
d. said RTMP server sending said video stream to said RTMP client at said second bit rate.
3. The method of claim 2 wherein said RTMP client is not provided with a choice of bit rates from said RTMP server.
CA 2804741 2013-01-21 2013-01-21 Media server Abandoned CA2804741A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CA 2804741 CA2804741A1 (en) 2013-01-21 2013-01-21 Media server

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CA 2804741 CA2804741A1 (en) 2013-01-21 2013-01-21 Media server
PCT/CA2014/000038 WO2014110670A1 (en) 2013-01-21 2014-01-21 Media server

Publications (1)

Publication Number Publication Date
CA2804741A1 true true CA2804741A1 (en) 2014-07-21

Family

ID=51208893

Family Applications (1)

Application Number Title Priority Date Filing Date
CA 2804741 Abandoned CA2804741A1 (en) 2013-01-21 2013-01-21 Media server

Country Status (2)

Country Link
CA (1) CA2804741A1 (en)
WO (1) WO2014110670A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104967884A (en) * 2015-04-17 2015-10-07 北京奇艺世纪科技有限公司 Code stream switching method and code stream switching device

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104602044A (en) * 2015-02-05 2015-05-06 秦永红 RTMP stream media public network live broadcast system and design method thereof

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3699910B2 (en) * 2000-10-31 2005-09-28 株式会社東芝 Data transmission apparatus, a data transmission method and a program
US8218657B2 (en) * 2005-09-02 2012-07-10 Netgear, Inc. System and method for automatic adjustment of streaming video bit rate
US20080040757A1 (en) * 2006-07-31 2008-02-14 David Romano Video content streaming through a wireless access point
US7962637B2 (en) * 2006-11-03 2011-06-14 Apple Computer, Inc. Dynamic adjustments of video streams
US7987285B2 (en) * 2007-07-10 2011-07-26 Bytemobile, Inc. Adaptive bitrate management for streaming media over packet networks
US8001260B2 (en) * 2008-07-28 2011-08-16 Vantrix Corporation Flow-rate adaptation for a connection of time-varying capacity
US20100121974A1 (en) * 2008-11-11 2010-05-13 Einarsson Torbjoem Stepwise probing for adaptive streaming in a packet communication network
JP2011009904A (en) * 2009-06-24 2011-01-13 Hitachi Ltd Wireless video distribution system, content bit rate control method, and computer readable recording medium having content bit rate control program stored therein
CN102131084A (en) * 2010-01-19 2011-07-20 深圳市在线通网络科技开发有限公司 RTMP (Real Time Messaging Protocol) pushing device and method for audio/video streaming media
US8904027B2 (en) * 2010-06-30 2014-12-02 Cable Television Laboratories, Inc. Adaptive bit rate for data transmission
US20120324122A1 (en) * 2011-06-20 2012-12-20 David Miles Method and apparatus for server-side adaptive streaming

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104967884A (en) * 2015-04-17 2015-10-07 北京奇艺世纪科技有限公司 Code stream switching method and code stream switching device
CN104967884B (en) * 2015-04-17 2018-01-26 北京奇艺世纪科技有限公司 One kind of stream switching method and apparatus

Also Published As

Publication number Publication date Type
WO2014110670A1 (en) 2014-07-24 application

Similar Documents

Publication Publication Date Title
Jacobs et al. Streaming video using dynamic rate shaping and tcp flow control
US7599307B2 (en) Quality of experience (QoE) method and apparatus for wireless communication networks
US6792449B2 (en) Startup methods and apparatuses for use in streaming content
US7002992B1 (en) Codec selection to improve media communication
US20030126277A1 (en) Apparatus and method for providing multimedia streaming service by using point-to-point connection
US20130297743A1 (en) Managed Adaptive Streaming
US7346007B2 (en) Bandwidth adaptation
US20130298170A1 (en) Video streaming quality of experience recovery using a video quality metric
US20100121974A1 (en) Stepwise probing for adaptive streaming in a packet communication network
US20040267956A1 (en) Method and device for proactive rate adaptation signaling
US20130286879A1 (en) Video streaming quality of experience degradation control using a video quality metric
US20020073238A1 (en) System and method for media stream adaptation
Oyman et al. Quality of experience for HTTP adaptive streaming services
US20130290492A1 (en) State management for video streaming quality of experience degradation control and recovery using a video quality metric
US20080133766A1 (en) Method and apparatus for streaming media to a plurality of adaptive client devices
US20130275615A1 (en) System and method for adapting video communications
US20130272121A1 (en) Systems and methods for application-aware admission control in a communication network
US20140149557A1 (en) Network-Capacity Optimized Adaptive HTTP Streaming
US20080162714A1 (en) Method and Apparatus for Reporting Streaming Media Quality
De Cicco et al. An experimental investigation of the Akamai adaptive video streaming
US20140013376A1 (en) Methods and devices for efficient adaptive bitrate streaming
US7047308B2 (en) System and method for simultaneous media playout
US20080114894A1 (en) Methods and Systems for HTTP Streaming Using an Intelligent HTTP Client
US20110225315A1 (en) Multi-stream bit rate adaptation
US20150215359A1 (en) Systems and methods for using client-side video buffer occupancy for enhanced quality of experience in a communication network

Legal Events

Date Code Title Description
FZDE Dead

Effective date: 20160121