US20120324122A1 - Method and apparatus for server-side adaptive streaming - Google Patents
Method and apparatus for server-side adaptive streaming Download PDFInfo
- Publication number
- US20120324122A1 US20120324122A1 US13/163,838 US201113163838A US2012324122A1 US 20120324122 A1 US20120324122 A1 US 20120324122A1 US 201113163838 A US201113163838 A US 201113163838A US 2012324122 A1 US2012324122 A1 US 2012324122A1
- Authority
- US
- United States
- Prior art keywords
- server
- data object
- client
- tcp connection
- bit rate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/163—In-band adaptation of TCP data exchange; In-band control procedures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
- H04L65/612—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for unicast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/762—Media network packet handling at the source
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/80—Responding to QoS
Definitions
- the invention relates generally to communication networks and, more specifically but not exclusively, to streaming content via communication networks.
- streaming of video content from a server to a client typically utilizes video encoding methods (e.g., one or more of the Moving Picture Experts Group (MPEG) standards, H.264/Advanced Video Coding (AVC), and the like) which allow for real-time delivery and presentation of content (e.g., video content and, optionally, audio content).
- video encoding methods e.g., one or more of the Moving Picture Experts Group (MPEG) standards, H.264/Advanced Video Coding (AVC), and the like
- MPEG Moving Picture Experts Group
- AVC Advanced Video Coding
- the video encoding methods attempt to deliver a target frame-rate and a target video bit rate (or video quality) for video content to be streamed.
- the video content once encoded, needs to be delivered at least as fast as the encoded bit rate or the presentation of frames of the video content to the end user may be negatively impacted.
- an apparatus for adaptive streaming of a data object from a server toward a client via a Transmission Control Protocol (TCP) connection where the server has access to a plurality of versions of the data object having a respective plurality of encoding bit rates associated therewith, is provided.
- the apparatus includes a processor and a memory.
- the processor is configured to determine, from information associated with a TCP congestion control mechanism running on the server for the TCP connection, a transmission bit rate expected to be supported by the client for the TCP connection.
- the processor also is configured to select, based on the transmission bit rate expected to be supported by the client and the encoding bit rates associated with the versions of the data object accessible to the server, one of the plurality of versions of the data object for propagation from the server toward the client via the TCP connection.
- a method for adaptive streaming of a data object from a server toward a client via a Transmission Control Protocol (TCP) connection where the server has access to a plurality of versions of the data object having a respective plurality of encoding bit rates associated therewith.
- the method includes determining, from information associated with a TCP congestion control mechanism running on the server for the TCP connection, a transmission bit rate expected to be supported by the client for the TCP connection, and selecting, based on the transmission bit rate expected to be supported by the client and the encoding bit rates associated with the versions of the data object accessible to the server, one of the plurality of versions of the data object for propagation from the server toward the client via the TCP connection.
- TCP Transmission Control Protocol
- a computer-readable storage medium stores instructions which, when executed by a computer, cause the computer to perform a method for adaptive streaming of a data object from a server toward a client via a Transmission Control Protocol (TCP) connection, where the server has access to a plurality of versions of the data object having a respective plurality of encoding bit rates associated therewith.
- TCP Transmission Control Protocol
- the method includes determining, from information associated with a TCP congestion control mechanism running on the server for the TCP connection, a transmission bit rate expected to be supported by the client for the TCP connection, and selecting, based on the transmission bit rate expected to be supported by the client and the encoding bit rates associated with the versions of the data object accessible to the server, one of the plurality of versions of the data object for propagation from the server toward the client via the TCP connection.
- FIG. 1 depicts an exemplary communication system illustrating use of a server-side adaptive streaming mechanism to stream data from a server to a client;
- FIG. 2 depicts an exemplary embodiment of the server-side adaptive streaming mechanism of the server of FIG. 1 ;
- FIG. 3 depicts one embodiment of a method by which a server selects, from among multiple versions of a data object available to the server, one of the versions of the data object from which to propagate a portion of the data object toward a client via a TCP connection;
- FIG. 4 depicts use of a server-side adaptive streaming mechanism on a server to produce an output data stream for a client from three encoded versions of the data object accessible to the server;
- FIG. 5 depicts a high-level block diagram of a computer suitable for use in performing the functions described herein.
- the server-side adaptive streaming capability enables a server to adaptively stream a data object to a client via a Transmission Control Protocol (TCP) connection by outputting a contiguous data stream formed by selecting from among multiple versions of the data object based on information available from a TCP congestion control mechanism operating on the server for the TCP connection.
- TCP Transmission Control Protocol
- the information available from the TCP congestion control mechanism is analyzed by the server to determine a transmission bit rate expected to be supported by the client based on analysis of the information available from the TCP congestion control mechanism.
- the multiple versions of the data object have respective encoding bit rates associated therewith.
- the server is configured to select from among the versions of the data object based on the transmission bit rate expected to be supported by the client as determined by the server and the encoding bit rates of the versions of the data object available to the server.
- the server is configured to select the version of the data object having a largest encoding bit rate which can be supported by the transmission bit rate expected to be supported by the client as determined by the server based on analysis of the information available from the TCP congestion control mechanism.
- the server may be configured to perform such functions periodically, in response to events, and the like, as well as various combinations thereof.
- the server is configured to dynamically switch between the versions of the data object as the transmission bit rate expected to be supported by the client changes.
- the server selects from among the versions of the data object, the server outputs a contiguous data stream formed by selecting from among the multiple versions of the data object, thereby enabling the client to receive the highest quality data stream, that the client is capable of receiving, at any given point in time.
- Various other functions and capabilities of embodiments of the server-side adaptive streaming capability may be better understood by way of reference to the following description and the associated drawings.
- FIG. 1 depicts an exemplary communication system illustrating use of a server-side adaptive streaming mechanism to stream data from a server to a client.
- the exemplary communication system 100 includes a communication network 101 , a server 110 , and a client 120 .
- a TCP connection 130 is established between the server 110 and client 120 , via communication network 101 , for enabling the server 110 to stream data to the client 120 .
- the communication network 101 may include any communication network configured for supporting TCP-based communications.
- the server 110 is configured for streaming data to clients using TCP connections (illustratively, to client 120 via TCP connection 130 ).
- the server 110 has access to data available to be streamed from server 110 to client 120 .
- the data which is streamed from the server 110 to the client 120 via the TCP connection 130 may be accessed locally (e.g., in a local database 115 L of server 110 ) and/or remotely (e.g., from a remote database 115 R that is remote from server 110 ).
- the data which is streamed from the server 110 to the client 120 via the TCP connection 130 may include any type(s) of data suitable for being streamed using TCP.
- the data may include audio, video, multimedia, text, software, and the like, as well as various combinations thereof.
- the data which is streamed from the server 110 to the client 120 via the TCP connection 130 may be maintained, accessed, and controlled in the form of data objects.
- a data object may be a song, a music album, a television show, a portion of a television show, a movie, a portion of a movie, a data stream requiring real-time delivery, and the like.
- the server 110 has access to multiple versions of a data object to be streamed to client 110 .
- the multiple versions of the data object correspond to different encoding bit rates and, thus, have different levels of quality associated therewith.
- the multiple versions of the data object may be maintained as separate files including the same data encoded at different bit rates.
- low, medium, and high quality versions of the data object e.g., corresponding to encoding at associated low, medium and high bit rates, respectively
- encoding bit rates of the low, medium, and high quality versions of the data object may be 1 Mbps, 2 Mbps, and 4 Mbps, respectively.
- these data object versions are merely exemplary, and that fewer for more data object versions may be supported for a data object (i.e., two or more versions of a data object may be used for providing the server-side adaptive streaming capability).
- these encoding bit rates are merely exemplary, and that any suitable encoding bit rates may be used for providing different versions of a data object.
- the multiple versions of a data object may be stored in one storage location (e.g., either in local database 115 L or in remote database 115 R ) or distributed across multiple storage locations (e.g., using local database 115 L and remote database 115 R and/or any other suitable data storage locations).
- FIG. 1 An exemplary data object 140 , maintained and available as multiple data object versions 140 1 - 140 N , is depicted in FIG. 1 .
- data object version 140 1 may correspond to the lowest quality version of the data object and data object version 140 N may correspond to the highest quality version of the data object.
- two or more versions of a data object may be used for providing the server-side adaptive streaming capability.
- the client 120 is configured to receive streaming data via a TCP connection (illustratively, via TCP connection 130 from server 110 ).
- the client 120 also may be configured for providing one or more additional functions related to the streaming data (e.g., presenting the streaming data at the client 120 , processing the received data at client 120 , propagating the received data toward one or more other devices (e.g., for presentation, processing, and/or other related functions) and the like, as well as various combinations thereof).
- the client 120 may be a desktop computer, a laptop computer, a tablet, a smartphone, a set top box (STB), a television, or any other device which may receive streaming data via a TCP connection.
- STB set top box
- the server 110 is configured to support a server-side adaptive streaming capability. As depicted in FIG. 1 , this capability is represented as server-side adaptive streaming mechanism 111 , which may be implemented on server 110 in any suitable manner.
- the server-side adaptive streaming mechanism 111 may provide various functions of the server-side adaptive streaming capability depicted and described herein.
- the server 110 is configured to propagate the data object toward client 120 in portions (or segments) and, for each portion of the data object to be propagated from the server 110 toward client 120 , the server 110 may select that portion of the data object from among the multiple versions of the data object available to the server 110 . As server 110 selects portions of the data object from among the multiple versions of the data object available to the server 110 , the server 110 propagates the selected portions of the data object toward client 120 via TCP connection 130 . In this manner, the server 110 is able to output a single contiguous data stream for propagation toward client 120 , even though the server 110 may have assembled the data stream from multiple different versions of the data object.
- the server 110 is configured to perform a process in order to select, for a given portion of the data object to be provided to client 120 , which version of the data object is used to provide that given portion of the data object.
- the server 110 determines a transmission bit rate expected to be supported by the client 120 , and selects which version of the data object is used to provide that given portion of the data object based on the determined transmission bit rate expected to be supported by the client 120 and the encoding bit rates of the versions of the data object available for selection by server 110 .
- the server 110 determines a transmission bit rate expected to be supported by the client 120 using information, associated with the TCP connection 130 , that is available on server 110 .
- the server 110 determines a transmission bit rate expected to be supported by the client 120 using information associated with the TCP congestion control mechanism running on the server 110 for the TCP connection 130 .
- TCP congestion control is a component of TCP.
- TCP congestion control has been defined in various Internet Engineering Task Force (IETF) Requests for Comment (RFCs), such as RFC 793, RFC 2001 (TCP Slow Start, Congestion Avoidance, Fast Retransmit, and Fast Recovery Algorithms), RFC 2581 (TCP Congestion Control), and the like.
- RFCs Internet Engineering Task Force (IETF) Requests for Comment
- RFC 793 RFC 793
- RFC 2001 TCP Slow Start, Congestion Avoidance, Fast Retransmit, and Fast Recovery Algorithms
- RFC 2581 TCP Congestion Control
- a TCP congestion control mechanism may be implemented using various features of one or more such RFCs.
- a TCP congestion control mechanism for a TCP connection is configured to adapt the current transmission bit rate for the TCP connection based on monitoring of information associated with the TCP connection.
- a TCP congestion control mechanism may be configured to perform monitoring of the TCP connection (e.g., monitoring one or more parameters and/or monitoring for one or more events in order to detect congestion associated with the TCP connection), and to set the transmission bit rate for the TCP connection based on such monitoring of the TCP connection.
- the TCP congestion control mechanism is able to control and adapt the current transmission bit rate for the TCP connection based on the current state of the TCP connection (e.g., by controlling the speed with which data is removed from a queue for transmission from the server toward a client via the TCP connection).
- information associated with the TCP congestion control mechanism running on the server 110 for the TCP connection 130 includes an effective transmission bit rate currently supported by the TCP congestion control mechanism running on the server 110 for the TCP connection 130 .
- the effective transmission bit rate currently supported by the TCP congestion control mechanism running on the server 110 may be used by the server 110 as the transmission bit rate expected to be supported by the client 120 .
- the information associated with the TCP congestion control mechanism running on the server 110 for the TCP connection 130 also may include additional information (e.g., one or more parameters and/or one or more events) associated with the TCP connection 130 that is available on server 110 .
- additional information associated with the TCP connection 130 which may be determined from the TCP congestion control mechanism running on the server 110 for the TCP connection 130 may include packet loss information, round trip delay information, and the like, as well as various combinations thereof.
- at least a portion of the additional information associated with the TCP connection 130 may be used, in combination with the effective transmission bit rate currently supported by the TCP congestion control mechanism running on the server 110 , to determine the transmission bit rate expected to be supported by the client 120 .
- the effective transmission bit rate currently supported by the TCP congestion control mechanism running on the server 110 may be modified based on the additional information to compute the transmission bit rate expected to be supported by the client 120 .
- the use of such additional information may result in determination of a more accurate transmission bit rate expected to be supported by the client, and may even result in determination of an optimal transmission bit rate expected to be supported by the client.
- the server 110 may be configured to select from among the multiple versions of the data object using a combination of (1) the information associated with the TCP connection 130 that is available on the server 110 and (2) information associated with the TCP connection 130 that is provided to the server 110 by the client 120 .
- the server 110 determines a transmission bit rate expected to be supported by the client 120 using a combination of (1) the information associated with the TCP connection 130 that is available on the server 110 and (2) information associated with the TCP connection 130 that is provided to the server 110 by the client 120 .
- the information associated with the TCP connection, provided to the server 110 by the client 120 may be provided in any suitable manner (e.g., using an adaptive streaming feedback capability of the client 120 and/or in any other suitable manner).
- the TCP congestion control mechanism running on the server 110 for TCP connection 130 may be implemented on the server 110 in a number of ways and, thus, that the information associated with the TCP congestion control mechanism running on the server 110 for TCP connection 130 may be determined.
- the information associated with the TCP congestion control mechanism running on the server 110 for the TCP connection 130 may be determined from the TCP stack of the server 110 .
- the information associated with the TCP congestion control mechanism running on the server 110 for the TCP connection 130 may be determined from the TCP congestion control mechanism as it is implemented on the server 110 . From the foregoing embodiments, it will be appreciated that the information associated with the TCP congestion control mechanism running on the server 110 for the TCP connection 130 may be determined in any suitable manner.
- the determination of the transmission bit rate expected to be supported by the client 120 is performed by an application running at an application layer of a communications stack implemented on the server 110 .
- the information associated with the TCP congestion control mechanism running on the server 110 e.g., specifying the transmission bit rate expected to be supported by the client 120 and/or for use in determining the transmission bit rate expected to be supported by the client 120
- the information associated with the TCP congestion control mechanism running on the server 110 may be received at the application layer of the communications stack from a process running at the application layer of the communications stack (e.g., such as where a version of the TCP stack of the transport layer of the communications stack is implemented at the application layer of the communications stack for enabling the application to obtain the information associated with the TCP congestion control mechanism running on the server 110 ).
- the information associated with the TCP congestion control mechanism running on the server 110 may be made available to the application in any other suitable manner. An exemplary embodiment is depicted and described with respect to FIG. 2 .
- FIG. 2 depicts an exemplary embodiment of the server-side adaptive streaming mechanism of the server of FIG. 1 .
- server-side adaptive streaming mechanism 111 of the server 110 is implemented in conjunction with a communication stack 210 of server 110 .
- the communication stack 210 includes a transport layer 220 and an application layer 230 A (among other layers which are omitted for purposes of clarity).
- the transport layer 220 includes a TCP stack 221 supporting a TCP congestion control mechanism 222 . It is noted that, as described hereinabove, the TCP congestion control mechanism 222 may be provided independent of the TCP stack 221 .
- the application layer 230 supports an application 231 configured to perform various functions of the server-side adaptive streaming capability.
- application 231 may be configured to receive information from TCP congestion control mechanism 222 for use in determining transmission bit rate expected to be supported by client 120 , determine the transmission bit rate expected to be supported by client 120 based on information received from TCP congestion control mechanism 222 , select portions of a data object from among multiple versions of the data object based on the transmission bit rate expected to be supported by the client 120 (and, optionally, other information), and the like, as well as various combinations thereof.
- FIG. 2 merely represents one exemplary implementation of the server-side adaptive streaming mechanism 111 , and that various other implementations may be provided.
- server 110 is configured to select, based on the determined transmission bit rate expected to be supported by the client 120 and the encoding bit rates of the versions of the data object available for selection by server 110 , which version of the data object is used to provide that given portion of the data object to the client 120 .
- the server 110 is configured to select the version of the data object having an associated encoding bit rate best suited for the determined transmission bit rate expected to be supported by the client 120 .
- the encoding bit rate best suited for the determined transmission bit rate is the largest encoding bit rate that is less than or equal to the transmission bit rate.
- the server 110 selects the medium quality version of the data object as the source of that portion of the data object to be provided to client 120 (since the 3.8 Mbps transmission bit rate is large enough to support the 2 Mbps encoding bit rate of the medium quality version of the data object, but not quite large enough to support the 4 Mbps encoding bit rate of the high quality version of the data object).
- the encoding bit rate best suited for the determined transmission bit rate is the largest encoding bit rate that is less than the transmission bit rate as long as the transmission bit rate is at least a threshold amount larger than that largest encoding bit rate; otherwise, the next lowest encoding bit rate is selected.
- the server 110 selects the low quality version of the data object as the source of that portion of the data object to be provided to client 120 (although the 2 Mbps transmission rate is just large enough to support the 2 Mbps encoding bit rate of the medium quality version of the data object, any slight decrease in the transmission bit rate supported by the client 120 may result in problems with transmission of the data object to the client 120 such that it may be prudent to, at least initially, select that low quality version of the data object that can definitely be supported by the determined transmission bit rate).
- server-side adaptive streaming mechanism 111 of server 110 in providing various functions of the server-side adaptive streaming capability may be better understood by way of reference to FIG. 3 .
- FIG. 3 depicts one embodiment of a method by which a server selects, from among multiple versions of a data object available to the server, one of the versions of the data object from which to propagate a portion of the data object toward a client via a TCP connection.
- the steps of method 300 of FIG. 3 may be better understood when read in conjunction with the description of FIG. 1 . Although the steps of method 300 of FIG. 3 are depicted and described as being performed serially, the steps of method 300 may be performed contemporaneously and/or in a different order than presented.
- step 310 method 300 begins.
- step 320 information associated with a TCP congestion control mechanism, running on the server for the TCP connection, is determined.
- a transmission bit rate expected to be supported by the client is determined using the information associated with a TCP congestion control mechanism running on the server.
- a version of the data object is selected based on the transmission bit rate expected to be supported by the client and encoding bit rates of the multiple versions of the data object available for selection.
- a portion of the data object from the selected version of the data object is propagated from the server toward the client via the TCP connection.
- step 360 method 300 ends.
- method 300 may continue to be repeated by the server for streaming the data object toward the client.
- the method 300 and/or portions of method 300 may be repeated in any suitable manner and upon any suitable basis.
- method 300 or portions of method 300 may be repeated periodically based on data object portions. For example, method 300 may be repeated once for each portion of the data object to be propagated from the server toward the client. For example, steps 320 and 330 of method 300 may be repeated once for every five contiguous portions of the data object to be propagated from the server toward the client and then the next four portions of the data object are selected from the same version of the data object based on that execution of steps 320 and 330 .
- method 300 or portions of method 300 may be repeated periodically based on time (e.g., once every 100 ms, once each second, and the like).
- method 300 may be repeated once every ten seconds where each portion of the data object to be propagated from the server toward the client is ten seconds in length.
- steps 320 and 330 of method 300 may be repeated once every twenty seconds where each portion of the data object to be propagated from the server toward the client is two seconds in length (such that, after selection of a version of the data object for the first two-second portion of the data object, the next nine two-second portions of the data object are selected from the same version of the data object based on that execution of steps 320 and 330 ).
- method 300 or portions of method 300 may be performed in response to one or more trigger conditions.
- steps 320 and 330 may be performed by the server continuously monitoring the TCP congestion control process for the TCP connection (as portions of the data object continue to be streamed toward the client from the previously selected version of the data object) until one or more parameters of the TCP congestion control process indicates that the version of the data object currently being propagated toward the client is no longer the optimal or desired version of the data object to be propagated toward the client, thereby triggering selection of a new version of the data object and propagation of portions of the newly selected version of the data object toward the client.
- portions of the data object may be streamed toward the client from the previously selected version of the data object (e.g., selected during the previous execution of method 300 ) until an event detected at the server indicates that the method 300 should be executed again (which may result in selection of the same or a different version of the data object from which portions of the data object are then streamed to the client).
- the previously selected version of the data object e.g., selected during the previous execution of method 300
- an event detected at the server indicates that the method 300 should be executed again (which may result in selection of the same or a different version of the data object from which portions of the data object are then streamed to the client).
- the server determines whether to switch between different versions of the data object and, ultimately, switches between different versions of the data object.
- the TCP stream output by the server for delivery to the client is a single contiguous data-stream even though the server may have assembled it from multiple different versions of the data object encoded at different encoding bit rates.
- the server-side adaptive streaming capability obviates the need for a client to include any special client software in order for the client to benefit from adaptive streaming.
- clients that support client-side adaptive streaming and clients that do not support client-side adaptive streaming each may benefit from the server-side adaptive streaming capability depicted and described herein.
- the server-side adaptive streaming capability supports adaptive streaming to any client device via any type of TCP connection.
- FIG. 4 depicts use of a server-side adaptive streaming mechanism on a server to produce an output data stream for a client from three encoded versions of the data object accessible to the server.
- three video sequences 410 1 - 410 3 are used, based on the transmission bit rate expected to be supported by the client for the TCP connection as represented by a graph 420 , to assemble an output data stream 430 .
- the three encoded versions of the data object include a high bit rate video sequence 410 1 , a medium bit rate video sequence 410 2 , and a low bit rate video sequence 410 3 (collectively, video sequences 410 ).
- the video sequences 410 each include the same video content (i.e., they are different encoded versions, encoded at different rates, of the same video content object).
- video sequences 410 are aligned temporally with respect to each other for purposes of clarity in illustrating the different quality levels of the video sequences 410 .
- the graph 420 represents the value of the transmission bit rate expected to be supported by the client for the TCP connection as determined by the server over time. As depicted in graph 420 , the value of the transmission bit rate expected to be supported by the client for the TCP connection changes over time (e.g., in response to various changing conditions).
- there are two threshold levels for the value of the transmission bit rate expected to be supported by the client for the TCP connection which define three ranges of transmission bit rates expected to be supported by the client for the TCP connection as follows: (1) a LOW range which indicates that, for a value of the transmission bit rate expected to be supported by the client for the TCP connection that is within this range, the low bit rate video sequence 410 3 is to be selected by the server for use in the output data stream 430 , (2) a MEDIUM range which indicates that, for a value of the transmission bit rate expected to be supported by the client for the TCP connection that is within this range, the medium bit rate video sequence 410 2 is to be selected by the server for use in the output data stream 430 , and (3) a HIGH range which indicates that, for a value of the transmission bit rate expected to be supported by the client for the TCP connection that is within this range, the high bit rate video sequence 410 1 is to be selected by the server for use in the output data stream 430 .
- the graph 420 is aligned temporally with respect to each of the video sequences 410 and the output data stream 430 .
- the values of the transmission bit rates marked on the graph 420 are merely exemplary and, similarly, that the definition of the LOW, MEDIUM, and HIGH ranges on the graph 420 also are merely exemplary (e.g., any suitable expected transmission bit rate values may be determined by the server, the ranges of expected transmission bit rate values used for determining selection between available video sequences may be defined in any suitable manner, any suitable number of ranges of expected transmission bit rate values used to determine selection between available video sequences may be employed, and the like, as well as various combinations thereof).
- the output data stream 430 is assembled using various portions of the video sequences 410 .
- the server-side adaptive streaming mechanism of the server assembles the output data stream 430 by evaluating information, associated with a TCP congestion control mechanism running on the server for the TCP connection, for appropriately selecting video content from video sequences 410 for use as the output data stream 430 .
- the selection of video content from the video sequences 410 based on the values of transmission bit rate expected to be supported by the client for the TCP connection as represented by graph 420 , for use as the output data stream 430 is depicted in FIG. 4 .
- server-side adaptive streaming mechanism controls selection of video content from video sequences 410 for use as the output data stream 430 as follows: (a) before time t 1 , the transmission bit rate expected to be supported by the client for the TCP connection is in the LOW range and, thus, the output data stream 430 includes video from low bit rate video sequence 410 3 , (b) at time t 1 , the transmission bit rate expected to be supported by the client for the TCP connection crosses the threshold between the LOW and MEDIUM ranges and, thus, the server switches from using video from the low bit rate video sequence 410 3 to using video from the medium bit rate video sequence 410 2 , (c) between times t 1 and t 2 , the output data stream 430 includes video from medium bit rate video sequence 410 2 , (d) at time t 2 , the transmission bit rate expected to be supported by the client for the TCP connection crosses the threshold between the MEDIUM and HIGH ranges and, thus, the server switches from using video from the medium bit
- server-side adaptive streaming mechanism produces an output data stream 430 for the TCP connection to the client, based on information associated with a TCP congestion control mechanism running on the server for the TCP connection, that is optimized for the transmission bit rate expected to be supported by the client for the TCP connection.
- server-side adaptive streaming capability may be used to provide adaptive streaming for TCP connections between other types of devices.
- references to such devices also may be considered to be references to the programs, running on the devices, for which the TCP connection is used (e.g., application programs or any other suitable type(s) of programs which may utilize a TCP connection for transfer of data).
- FIG. 5 depicts a high-level block diagram of a computer suitable for use in performing functions described herein.
- computer 500 includes a processor element 502 (e.g., a central processing unit (CPU) and/or other suitable processor(s)) and a memory 504 (e.g., random access memory (RAM), read only memory (ROM), and the like).
- processor element 502 e.g., a central processing unit (CPU) and/or other suitable processor(s)
- memory 504 e.g., random access memory (RAM), read only memory (ROM), and the like.
- the computer 500 also may include a cooperating module/process 505 and/or various input/output devices 506 (e.g., a user input device (such as a keyboard, a keypad, a mouse, and the like), a user output device (such as a display, a speaker, and the like), an input port, an output port, a receiver, a transmitter, and storage devices (e.g., a tape drive, a floppy drive, a hard disk drive, a compact disk drive, and the like)).
- a user input device such as a keyboard, a keypad, a mouse, and the like
- a user output device such as a display, a speaker, and the like
- an input port such as a display, a speaker, and the like
- an output port such as a receiver, a transmitter
- storage devices e.g., a tape drive, a floppy drive, a hard disk drive, a compact disk drive, and the like
- cooperating process 505 can be loaded into memory 504 and executed by processor 502 to implement functions as discussed herein.
- cooperating process 505 (including associated data structures) can be stored on a computer readable storage medium, e.g., RAM memory, magnetic or optical drive or diskette, and the like.
- computer 500 depicted in FIG. 5 provides a general architecture and functionality suitable for implementing functional elements described herein and/or portions of functional elements described herein.
- the computer 500 provides a general architecture and functionality suitable for implementing one or more of the server 110 , a portion of the server 110 , the server-side adaptive streaming mechanism 111 , a portion of the server-side adaptive streaming mechanism 111 , the client 120 , a portion of the client 120 , and the like.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Description
- The invention relates generally to communication networks and, more specifically but not exclusively, to streaming content via communication networks.
- In general, streaming of video content from a server to a client typically utilizes video encoding methods (e.g., one or more of the Moving Picture Experts Group (MPEG) standards, H.264/Advanced Video Coding (AVC), and the like) which allow for real-time delivery and presentation of content (e.g., video content and, optionally, audio content). The video encoding methods attempt to deliver a target frame-rate and a target video bit rate (or video quality) for video content to be streamed. The video content, once encoded, needs to be delivered at least as fast as the encoded bit rate or the presentation of frames of the video content to the end user may be negatively impacted.
- Various deficiencies in the prior art are addressed by embodiments for providing server-side adaptive streaming.
- In one embodiment, an apparatus for adaptive streaming of a data object from a server toward a client via a Transmission Control Protocol (TCP) connection, where the server has access to a plurality of versions of the data object having a respective plurality of encoding bit rates associated therewith, is provided. The apparatus includes a processor and a memory. The processor is configured to determine, from information associated with a TCP congestion control mechanism running on the server for the TCP connection, a transmission bit rate expected to be supported by the client for the TCP connection. The processor also is configured to select, based on the transmission bit rate expected to be supported by the client and the encoding bit rates associated with the versions of the data object accessible to the server, one of the plurality of versions of the data object for propagation from the server toward the client via the TCP connection.
- In one embodiment, a method for adaptive streaming of a data object from a server toward a client via a Transmission Control Protocol (TCP) connection, where the server has access to a plurality of versions of the data object having a respective plurality of encoding bit rates associated therewith, is provided. The method includes determining, from information associated with a TCP congestion control mechanism running on the server for the TCP connection, a transmission bit rate expected to be supported by the client for the TCP connection, and selecting, based on the transmission bit rate expected to be supported by the client and the encoding bit rates associated with the versions of the data object accessible to the server, one of the plurality of versions of the data object for propagation from the server toward the client via the TCP connection.
- In one embodiment, a computer-readable storage medium is provided, where the computer-readable storage medium stores instructions which, when executed by a computer, cause the computer to perform a method for adaptive streaming of a data object from a server toward a client via a Transmission Control Protocol (TCP) connection, where the server has access to a plurality of versions of the data object having a respective plurality of encoding bit rates associated therewith. The method includes determining, from information associated with a TCP congestion control mechanism running on the server for the TCP connection, a transmission bit rate expected to be supported by the client for the TCP connection, and selecting, based on the transmission bit rate expected to be supported by the client and the encoding bit rates associated with the versions of the data object accessible to the server, one of the plurality of versions of the data object for propagation from the server toward the client via the TCP connection.
- The teachings herein can be readily understood by considering the following detailed description in conjunction with the accompanying drawings, in which:
-
FIG. 1 depicts an exemplary communication system illustrating use of a server-side adaptive streaming mechanism to stream data from a server to a client; -
FIG. 2 depicts an exemplary embodiment of the server-side adaptive streaming mechanism of the server ofFIG. 1 ; -
FIG. 3 depicts one embodiment of a method by which a server selects, from among multiple versions of a data object available to the server, one of the versions of the data object from which to propagate a portion of the data object toward a client via a TCP connection; -
FIG. 4 depicts use of a server-side adaptive streaming mechanism on a server to produce an output data stream for a client from three encoded versions of the data object accessible to the server; and -
FIG. 5 depicts a high-level block diagram of a computer suitable for use in performing the functions described herein. - To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures.
- In general, a server-side adaptive streaming capability is depicted and described herein, although various other capabilities also may be presented herein. In at least some embodiments, the server-side adaptive streaming capability enables a server to adaptively stream a data object to a client via a Transmission Control Protocol (TCP) connection by outputting a contiguous data stream formed by selecting from among multiple versions of the data object based on information available from a TCP congestion control mechanism operating on the server for the TCP connection. The information available from the TCP congestion control mechanism is analyzed by the server to determine a transmission bit rate expected to be supported by the client based on analysis of the information available from the TCP congestion control mechanism. The multiple versions of the data object have respective encoding bit rates associated therewith. The server is configured to select from among the versions of the data object based on the transmission bit rate expected to be supported by the client as determined by the server and the encoding bit rates of the versions of the data object available to the server. In one embodiment, the server is configured to select the version of the data object having a largest encoding bit rate which can be supported by the transmission bit rate expected to be supported by the client as determined by the server based on analysis of the information available from the TCP congestion control mechanism. The server may be configured to perform such functions periodically, in response to events, and the like, as well as various combinations thereof. The server is configured to dynamically switch between the versions of the data object as the transmission bit rate expected to be supported by the client changes. As the server selects from among the versions of the data object, the server outputs a contiguous data stream formed by selecting from among the multiple versions of the data object, thereby enabling the client to receive the highest quality data stream, that the client is capable of receiving, at any given point in time. Various other functions and capabilities of embodiments of the server-side adaptive streaming capability may be better understood by way of reference to the following description and the associated drawings.
-
FIG. 1 depicts an exemplary communication system illustrating use of a server-side adaptive streaming mechanism to stream data from a server to a client. - The
exemplary communication system 100 includes acommunication network 101, aserver 110, and aclient 120. A TCPconnection 130 is established between theserver 110 andclient 120, viacommunication network 101, for enabling theserver 110 to stream data to theclient 120. - The
communication network 101 may include any communication network configured for supporting TCP-based communications. - The
server 110 is configured for streaming data to clients using TCP connections (illustratively, toclient 120 via TCP connection 130). - The
server 110 has access to data available to be streamed fromserver 110 toclient 120. - The data which is streamed from the
server 110 to theclient 120 via the TCPconnection 130 may be accessed locally (e.g., in a local database 115 L of server 110) and/or remotely (e.g., from a remote database 115 R that is remote from server 110). - The data which is streamed from the
server 110 to theclient 120 via the TCPconnection 130 may include any type(s) of data suitable for being streamed using TCP. For example, the data may include audio, video, multimedia, text, software, and the like, as well as various combinations thereof. - The data which is streamed from the
server 110 to theclient 120 via the TCPconnection 130 may be maintained, accessed, and controlled in the form of data objects. For example, a data object may be a song, a music album, a television show, a portion of a television show, a movie, a portion of a movie, a data stream requiring real-time delivery, and the like. - The
server 110 has access to multiple versions of a data object to be streamed toclient 110. The multiple versions of the data object correspond to different encoding bit rates and, thus, have different levels of quality associated therewith. The multiple versions of the data object may be maintained as separate files including the same data encoded at different bit rates. For example, for a given data object, low, medium, and high quality versions of the data object (e.g., corresponding to encoding at associated low, medium and high bit rates, respectively) may be supported. For example, for this given data object, encoding bit rates of the low, medium, and high quality versions of the data object may be 1 Mbps, 2 Mbps, and 4 Mbps, respectively. It is noted that these data object versions are merely exemplary, and that fewer for more data object versions may be supported for a data object (i.e., two or more versions of a data object may be used for providing the server-side adaptive streaming capability). Similarly, it is noted that these encoding bit rates are merely exemplary, and that any suitable encoding bit rates may be used for providing different versions of a data object. The multiple versions of a data object may be stored in one storage location (e.g., either in local database 115 L or in remote database 115 R) or distributed across multiple storage locations (e.g., using local database 115 L and remote database 115 R and/or any other suitable data storage locations). - An
exemplary data object 140, maintained and available as multiple data object versions 140 1-140 N, is depicted inFIG. 1 . For theexemplary data object 140, it is noted thatdata object version 140 1 may correspond to the lowest quality version of the data object anddata object version 140 N may correspond to the highest quality version of the data object. As noted above, two or more versions of a data object may be used for providing the server-side adaptive streaming capability. - The
client 120 is configured to receive streaming data via a TCP connection (illustratively, via TCPconnection 130 from server 110). Theclient 120 also may be configured for providing one or more additional functions related to the streaming data (e.g., presenting the streaming data at theclient 120, processing the received data atclient 120, propagating the received data toward one or more other devices (e.g., for presentation, processing, and/or other related functions) and the like, as well as various combinations thereof). For example, theclient 120 may be a desktop computer, a laptop computer, a tablet, a smartphone, a set top box (STB), a television, or any other device which may receive streaming data via a TCP connection. - In one embodiment, the
server 110 is configured to support a server-side adaptive streaming capability. As depicted inFIG. 1 , this capability is represented as server-sideadaptive streaming mechanism 111, which may be implemented onserver 110 in any suitable manner. The server-sideadaptive streaming mechanism 111 may provide various functions of the server-side adaptive streaming capability depicted and described herein. - The
server 110 is configured to propagate the data object towardclient 120 in portions (or segments) and, for each portion of the data object to be propagated from theserver 110 towardclient 120, theserver 110 may select that portion of the data object from among the multiple versions of the data object available to theserver 110. Asserver 110 selects portions of the data object from among the multiple versions of the data object available to theserver 110, theserver 110 propagates the selected portions of the data object towardclient 120 via TCPconnection 130. In this manner, theserver 110 is able to output a single contiguous data stream for propagation towardclient 120, even though theserver 110 may have assembled the data stream from multiple different versions of the data object. - The
server 110 is configured to perform a process in order to select, for a given portion of the data object to be provided toclient 120, which version of the data object is used to provide that given portion of the data object. - In one embodiment, the
server 110 determines a transmission bit rate expected to be supported by theclient 120, and selects which version of the data object is used to provide that given portion of the data object based on the determined transmission bit rate expected to be supported by theclient 120 and the encoding bit rates of the versions of the data object available for selection byserver 110. - In one embodiment, the
server 110 determines a transmission bit rate expected to be supported by theclient 120 using information, associated with theTCP connection 130, that is available onserver 110. - In one embodiment, the
server 110 determines a transmission bit rate expected to be supported by theclient 120 using information associated with the TCP congestion control mechanism running on theserver 110 for theTCP connection 130. - In general, TCP congestion control is a component of TCP. For example, TCP congestion control has been defined in various Internet Engineering Task Force (IETF) Requests for Comment (RFCs), such as RFC 793, RFC 2001 (TCP Slow Start, Congestion Avoidance, Fast Retransmit, and Fast Recovery Algorithms), RFC 2581 (TCP Congestion Control), and the like. As a result, a TCP congestion control mechanism may be implemented using various features of one or more such RFCs.
- In general, a TCP congestion control mechanism for a TCP connection is configured to adapt the current transmission bit rate for the TCP connection based on monitoring of information associated with the TCP connection. For example, a TCP congestion control mechanism may be configured to perform monitoring of the TCP connection (e.g., monitoring one or more parameters and/or monitoring for one or more events in order to detect congestion associated with the TCP connection), and to set the transmission bit rate for the TCP connection based on such monitoring of the TCP connection. In this manner, the TCP congestion control mechanism is able to control and adapt the current transmission bit rate for the TCP connection based on the current state of the TCP connection (e.g., by controlling the speed with which data is removed from a queue for transmission from the server toward a client via the TCP connection).
- In one embodiment, information associated with the TCP congestion control mechanism running on the
server 110 for theTCP connection 130 includes an effective transmission bit rate currently supported by the TCP congestion control mechanism running on theserver 110 for theTCP connection 130. In one such embodiment, the effective transmission bit rate currently supported by the TCP congestion control mechanism running on theserver 110 may be used by theserver 110 as the transmission bit rate expected to be supported by theclient 120. - In one embodiment, the information associated with the TCP congestion control mechanism running on the
server 110 for theTCP connection 130 also may include additional information (e.g., one or more parameters and/or one or more events) associated with theTCP connection 130 that is available onserver 110. For example, additional information associated with theTCP connection 130 which may be determined from the TCP congestion control mechanism running on theserver 110 for theTCP connection 130 may include packet loss information, round trip delay information, and the like, as well as various combinations thereof. In one such embodiment, at least a portion of the additional information associated with theTCP connection 130 may be used, in combination with the effective transmission bit rate currently supported by the TCP congestion control mechanism running on theserver 110, to determine the transmission bit rate expected to be supported by theclient 120. For example, the effective transmission bit rate currently supported by the TCP congestion control mechanism running on theserver 110 may be modified based on the additional information to compute the transmission bit rate expected to be supported by theclient 120. The use of such additional information may result in determination of a more accurate transmission bit rate expected to be supported by the client, and may even result in determination of an optimal transmission bit rate expected to be supported by the client. - In one embodiment, although primarily depicted and described with respect to embodiments in which the
server 110 selects from among the multiple versions of the data object based only on information available at theserver 110, theserver 110 also may be configured to select from among the multiple versions of the data object using a combination of (1) the information associated with theTCP connection 130 that is available on theserver 110 and (2) information associated with theTCP connection 130 that is provided to theserver 110 by theclient 120. In one such embodiment, for example, theserver 110 determines a transmission bit rate expected to be supported by theclient 120 using a combination of (1) the information associated with theTCP connection 130 that is available on theserver 110 and (2) information associated with theTCP connection 130 that is provided to theserver 110 by theclient 120. The information associated with the TCP connection, provided to theserver 110 by theclient 120, may be provided in any suitable manner (e.g., using an adaptive streaming feedback capability of theclient 120 and/or in any other suitable manner). - It is noted that the TCP congestion control mechanism running on the
server 110 forTCP connection 130 may be implemented on theserver 110 in a number of ways and, thus, that the information associated with the TCP congestion control mechanism running on theserver 110 forTCP connection 130 may be determined. In one embodiment, for example, in which the TCP congestion control mechanism running onserver 110 is implemented as part of a TCP stack of theserver 110, the information associated with the TCP congestion control mechanism running on theserver 110 for theTCP connection 130 may be determined from the TCP stack of theserver 110. In one embodiment, for example, in which the TCP congestion control mechanism running on theserver 110 is implemented such that at least a portion of the TCP congestion control mechanism is not implemented as part of the TCP stack of theserver 110, the information associated with the TCP congestion control mechanism running on theserver 110 for theTCP connection 130 may be determined from the TCP congestion control mechanism as it is implemented on theserver 110. From the foregoing embodiments, it will be appreciated that the information associated with the TCP congestion control mechanism running on theserver 110 for theTCP connection 130 may be determined in any suitable manner. - In one embodiment, the determination of the transmission bit rate expected to be supported by the
client 120 is performed by an application running at an application layer of a communications stack implemented on theserver 110. In one embodiment, for example, the information associated with the TCP congestion control mechanism running on the server 110 (e.g., specifying the transmission bit rate expected to be supported by theclient 120 and/or for use in determining the transmission bit rate expected to be supported by the client 120) may be received at the application layer of the communications stack from a transport layer of the communications stack (e.g., where the information is received from a TCP stack implemented on the server 110). In one embodiment, for example, the information associated with the TCP congestion control mechanism running on the server 110 (e.g., specifying the transmission bit rate expected to be supported by theclient 120 and/or for use in determining the transmission bit rate expected to be supported by the client 120) may be received at the application layer of the communications stack from a process running at the application layer of the communications stack (e.g., such as where a version of the TCP stack of the transport layer of the communications stack is implemented at the application layer of the communications stack for enabling the application to obtain the information associated with the TCP congestion control mechanism running on the server 110). The information associated with the TCP congestion control mechanism running on theserver 110 may be made available to the application in any other suitable manner. An exemplary embodiment is depicted and described with respect toFIG. 2 . -
FIG. 2 depicts an exemplary embodiment of the server-side adaptive streaming mechanism of the server ofFIG. 1 . - As depicted in
FIG. 2 , server-sideadaptive streaming mechanism 111 of theserver 110 is implemented in conjunction with acommunication stack 210 ofserver 110. - The
communication stack 210 includes atransport layer 220 and an application layer 230 A (among other layers which are omitted for purposes of clarity). - The
transport layer 220 includes aTCP stack 221 supporting a TCPcongestion control mechanism 222. It is noted that, as described hereinabove, the TCPcongestion control mechanism 222 may be provided independent of theTCP stack 221. - The
application layer 230 supports anapplication 231 configured to perform various functions of the server-side adaptive streaming capability. For example,application 231 may be configured to receive information from TCPcongestion control mechanism 222 for use in determining transmission bit rate expected to be supported byclient 120, determine the transmission bit rate expected to be supported byclient 120 based on information received from TCPcongestion control mechanism 222, select portions of a data object from among multiple versions of the data object based on the transmission bit rate expected to be supported by the client 120 (and, optionally, other information), and the like, as well as various combinations thereof. - It is noted that
FIG. 2 merely represents one exemplary implementation of the server-sideadaptive streaming mechanism 111, and that various other implementations may be provided. - As described hereinabove, and returning now to
FIG. 1 ,server 110 is configured to select, based on the determined transmission bit rate expected to be supported by theclient 120 and the encoding bit rates of the versions of the data object available for selection byserver 110, which version of the data object is used to provide that given portion of the data object to theclient 120. - In one embodiment, the
server 110 is configured to select the version of the data object having an associated encoding bit rate best suited for the determined transmission bit rate expected to be supported by theclient 120. - In one embodiment, for example, the encoding bit rate best suited for the determined transmission bit rate is the largest encoding bit rate that is less than or equal to the transmission bit rate. For example, where the data object to be provided to the
client 120 is available in low, medium, and high quality versions having associated encoding bit rates of 1 Mbps, 2 Mbps, and 4 Mbps, respectively, and the transmission bit rate expected to be supported by theclient 120 is 3.8 Mbps, theserver 110 selects the medium quality version of the data object as the source of that portion of the data object to be provided to client 120 (since the 3.8 Mbps transmission bit rate is large enough to support the 2 Mbps encoding bit rate of the medium quality version of the data object, but not quite large enough to support the 4 Mbps encoding bit rate of the high quality version of the data object). - In one embodiment, for example, the encoding bit rate best suited for the determined transmission bit rate is the largest encoding bit rate that is less than the transmission bit rate as long as the transmission bit rate is at least a threshold amount larger than that largest encoding bit rate; otherwise, the next lowest encoding bit rate is selected. For example, where the data object to be provided to the
client 120 is available in low, medium, and high quality versions having associated encoding bit rates of 1 Mbps, 2 Mbps, and 4 Mbps, respectively, and the transmission bit rate expected to be supported by theclient 120 is 2 Mbps, theserver 110 selects the low quality version of the data object as the source of that portion of the data object to be provided to client 120 (although the 2 Mbps transmission rate is just large enough to support the 2 Mbps encoding bit rate of the medium quality version of the data object, any slight decrease in the transmission bit rate supported by theclient 120 may result in problems with transmission of the data object to theclient 120 such that it may be prudent to, at least initially, select that low quality version of the data object that can definitely be supported by the determined transmission bit rate). - The operation of server-side
adaptive streaming mechanism 111 ofserver 110 in providing various functions of the server-side adaptive streaming capability may be better understood by way of reference toFIG. 3 . -
FIG. 3 depicts one embodiment of a method by which a server selects, from among multiple versions of a data object available to the server, one of the versions of the data object from which to propagate a portion of the data object toward a client via a TCP connection. The steps ofmethod 300 ofFIG. 3 may be better understood when read in conjunction with the description ofFIG. 1 . Although the steps ofmethod 300 ofFIG. 3 are depicted and described as being performed serially, the steps ofmethod 300 may be performed contemporaneously and/or in a different order than presented. - At
step 310,method 300 begins. - At
step 320, information associated with a TCP congestion control mechanism, running on the server for the TCP connection, is determined. - At
step 330, a transmission bit rate expected to be supported by the client is determined using the information associated with a TCP congestion control mechanism running on the server. - At
step 340, a version of the data object is selected based on the transmission bit rate expected to be supported by the client and encoding bit rates of the multiple versions of the data object available for selection. - At
step 350, a portion of the data object from the selected version of the data object is propagated from the server toward the client via the TCP connection. - At
step 360,method 300 ends. - Although depicted and described as ending (for purposes of clarity), it is noted that the
method 300 may continue to be repeated by the server for streaming the data object toward the client. Themethod 300 and/or portions ofmethod 300 may be repeated in any suitable manner and upon any suitable basis. - In one embodiment, for example,
method 300 or portions ofmethod 300 may be repeated periodically based on data object portions. For example,method 300 may be repeated once for each portion of the data object to be propagated from the server toward the client. For example, steps 320 and 330 ofmethod 300 may be repeated once for every five contiguous portions of the data object to be propagated from the server toward the client and then the next four portions of the data object are selected from the same version of the data object based on that execution ofsteps - In one embodiment, for example,
method 300 or portions ofmethod 300 may be repeated periodically based on time (e.g., once every 100 ms, once each second, and the like). - For example,
method 300 may be repeated once every ten seconds where each portion of the data object to be propagated from the server toward the client is ten seconds in length. For example, steps 320 and 330 ofmethod 300 may be repeated once every twenty seconds where each portion of the data object to be propagated from the server toward the client is two seconds in length (such that, after selection of a version of the data object for the first two-second portion of the data object, the next nine two-second portions of the data object are selected from the same version of the data object based on that execution ofsteps 320 and 330). - In one embodiment, for example,
method 300 or portions ofmethod 300 may be performed in response to one or more trigger conditions. For example, steps 320 and 330 may be performed by the server continuously monitoring the TCP congestion control process for the TCP connection (as portions of the data object continue to be streamed toward the client from the previously selected version of the data object) until one or more parameters of the TCP congestion control process indicates that the version of the data object currently being propagated toward the client is no longer the optimal or desired version of the data object to be propagated toward the client, thereby triggering selection of a new version of the data object and propagation of portions of the newly selected version of the data object toward the client. For example, portions of the data object may be streamed toward the client from the previously selected version of the data object (e.g., selected during the previous execution of method 300) until an event detected at the server indicates that themethod 300 should be executed again (which may result in selection of the same or a different version of the data object from which portions of the data object are then streamed to the client). - It is noted that various combinations of such embodiments may be used for controlling the manner in which the server determines whether to switch between different versions of the data object and, ultimately, switches between different versions of the data object. In this manner, as noted herein, the TCP stream output by the server for delivery to the client is a single contiguous data-stream even though the server may have assembled it from multiple different versions of the data object encoded at different encoding bit rates.
- In at least some embodiments, the server-side adaptive streaming capability obviates the need for a client to include any special client software in order for the client to benefit from adaptive streaming. For example, clients that support client-side adaptive streaming and clients that do not support client-side adaptive streaming each may benefit from the server-side adaptive streaming capability depicted and described herein. As a result, the server-side adaptive streaming capability supports adaptive streaming to any client device via any type of TCP connection.
-
FIG. 4 depicts use of a server-side adaptive streaming mechanism on a server to produce an output data stream for a client from three encoded versions of the data object accessible to the server. - As depicted in
FIG. 4 , three video sequences 410 1-410 3 (collectively, video sequences 410) are used, based on the transmission bit rate expected to be supported by the client for the TCP connection as represented by agraph 420, to assemble anoutput data stream 430. - The three encoded versions of the data object include a high bit
rate video sequence 410 1, a medium bitrate video sequence 410 2, and a low bit rate video sequence 410 3 (collectively, video sequences 410). Thevideo sequences 410 each include the same video content (i.e., they are different encoded versions, encoded at different rates, of the same video content object). InFIG. 4 ,video sequences 410 are aligned temporally with respect to each other for purposes of clarity in illustrating the different quality levels of thevideo sequences 410. - The
graph 420, as indicated above, represents the value of the transmission bit rate expected to be supported by the client for the TCP connection as determined by the server over time. As depicted ingraph 420, the value of the transmission bit rate expected to be supported by the client for the TCP connection changes over time (e.g., in response to various changing conditions). As further depicted ingraph 420, there are two threshold levels for the value of the transmission bit rate expected to be supported by the client for the TCP connection, which define three ranges of transmission bit rates expected to be supported by the client for the TCP connection as follows: (1) a LOW range which indicates that, for a value of the transmission bit rate expected to be supported by the client for the TCP connection that is within this range, the low bitrate video sequence 410 3 is to be selected by the server for use in theoutput data stream 430, (2) a MEDIUM range which indicates that, for a value of the transmission bit rate expected to be supported by the client for the TCP connection that is within this range, the medium bitrate video sequence 410 2 is to be selected by the server for use in theoutput data stream 430, and (3) a HIGH range which indicates that, for a value of the transmission bit rate expected to be supported by the client for the TCP connection that is within this range, the high bitrate video sequence 410 1 is to be selected by the server for use in theoutput data stream 430. InFIG. 4 , thegraph 420 is aligned temporally with respect to each of thevideo sequences 410 and theoutput data stream 430. It is noted that the values of the transmission bit rates marked on thegraph 420 are merely exemplary and, similarly, that the definition of the LOW, MEDIUM, and HIGH ranges on thegraph 420 also are merely exemplary (e.g., any suitable expected transmission bit rate values may be determined by the server, the ranges of expected transmission bit rate values used for determining selection between available video sequences may be defined in any suitable manner, any suitable number of ranges of expected transmission bit rate values used to determine selection between available video sequences may be employed, and the like, as well as various combinations thereof). - The
output data stream 430 is assembled using various portions of thevideo sequences 410. The server-side adaptive streaming mechanism of the server assembles theoutput data stream 430 by evaluating information, associated with a TCP congestion control mechanism running on the server for the TCP connection, for appropriately selecting video content fromvideo sequences 410 for use as theoutput data stream 430. The selection of video content from thevideo sequences 410, based on the values of transmission bit rate expected to be supported by the client for the TCP connection as represented bygraph 420, for use as theoutput data stream 430 is depicted inFIG. 4 . - As depicted in
FIG. 4 , server-side adaptive streaming mechanism controls selection of video content from video sequences 410 for use as the output data stream 430 as follows: (a) before time t1, the transmission bit rate expected to be supported by the client for the TCP connection is in the LOW range and, thus, the output data stream 430 includes video from low bit rate video sequence 410 3, (b) at time t1, the transmission bit rate expected to be supported by the client for the TCP connection crosses the threshold between the LOW and MEDIUM ranges and, thus, the server switches from using video from the low bit rate video sequence 410 3 to using video from the medium bit rate video sequence 410 2, (c) between times t1 and t2, the output data stream 430 includes video from medium bit rate video sequence 410 2, (d) at time t2, the transmission bit rate expected to be supported by the client for the TCP connection crosses the threshold between the MEDIUM and HIGH ranges and, thus, the server switches from using video from the medium bit rate video sequence 410 2 to using video from the high bit rate video sequence 410 1, (e) between times t2 and t3, the output data stream 430 includes video from high bit rate video sequence 410 1, (f) at time t3, the transmission bit rate expected to be supported by the client for the TCP connection crosses the threshold between the HIGH and MEDIUM ranges and, thus, the server switches from using video from the high bit rate video sequence 410 1 to using video from the medium bit rate video sequence 410 2, (g) between times t3 and t4, the output data stream 430 includes video from medium bit rate video sequence 410 2, (h) at time t4, the transmission bit rate expected to be supported by the client for the TCP connection crosses the threshold between the MEDIUM and HIGH ranges and, thus, the server switches from using video from the medium bit rate video sequence 410 2 to using video from the high bit rate video sequence 410 1, (i) between times t4 and t5, the output data stream 430 includes video from high bit rate video sequence 410 1, (j) at time t5, the transmission bit rate expected to be supported by the client for the TCP connection crosses the threshold between the HIGH and MEDIUM ranges and, thus, the server switches from using video from the high bit rate video sequence 410 1 to using video from the medium bit rate video sequence 410 2, and (k) after time t5, the output data stream 430 includes video from medium bit rate video sequence 410 2. - In this manner, server-side adaptive streaming mechanism produces an
output data stream 430 for the TCP connection to the client, based on information associated with a TCP congestion control mechanism running on the server for the TCP connection, that is optimized for the transmission bit rate expected to be supported by the client for the TCP connection. - Although primarily depicted and described herein with respect to embodiments in which the server-side adaptive streaming capability is provided for a TCP connection between a server and a client, it is noted that embodiments of the server-side adaptive streaming capability may be used to provide adaptive streaming for TCP connections between other types of devices. As such, the terms “server” and “client” may be referred to more generally, e.g., data source and data destination, as source computer and destination computer, or using any other suitable terms. Furthermore, references to such devices also may be considered to be references to the programs, running on the devices, for which the TCP connection is used (e.g., application programs or any other suitable type(s) of programs which may utilize a TCP connection for transfer of data).
-
FIG. 5 depicts a high-level block diagram of a computer suitable for use in performing functions described herein. - As depicted in
FIG. 5 ,computer 500 includes a processor element 502 (e.g., a central processing unit (CPU) and/or other suitable processor(s)) and a memory 504 (e.g., random access memory (RAM), read only memory (ROM), and the like). Thecomputer 500 also may include a cooperating module/process 505 and/or various input/output devices 506 (e.g., a user input device (such as a keyboard, a keypad, a mouse, and the like), a user output device (such as a display, a speaker, and the like), an input port, an output port, a receiver, a transmitter, and storage devices (e.g., a tape drive, a floppy drive, a hard disk drive, a compact disk drive, and the like)). - It will be appreciated that the functions depicted and described herein may be implemented in software (e.g., via implementation of software on one or more processors) and/or hardware (e.g., using a general purpose computer, one or more application specific integrated circuits (ASIC), and/or any other hardware equivalents).
- It will be appreciated that the functions depicted and described herein may be implemented in software for executing on a general purpose computer (e.g., via execution by one or more processors) so as to implement a special purpose computer, and/or may be implemented in hardware (e.g., using one or more application specific integrated circuits (ASIC) and/or one or more other hardware equivalents).
- In one embodiment, the cooperating
process 505 can be loaded intomemory 504 and executed byprocessor 502 to implement functions as discussed herein. Thus, cooperating process 505 (including associated data structures) can be stored on a computer readable storage medium, e.g., RAM memory, magnetic or optical drive or diskette, and the like. - It will be appreciated that
computer 500 depicted inFIG. 5 provides a general architecture and functionality suitable for implementing functional elements described herein and/or portions of functional elements described herein. For example, thecomputer 500 provides a general architecture and functionality suitable for implementing one or more of theserver 110, a portion of theserver 110, the server-sideadaptive streaming mechanism 111, a portion of the server-sideadaptive streaming mechanism 111, theclient 120, a portion of theclient 120, and the like. - It is contemplated that some of the steps discussed herein as software methods may be implemented within hardware, for example, as circuitry that cooperates with the processor to perform various method steps. Portions of the functions/elements described herein may be implemented as a computer program product wherein computer instructions, when processed by a computer, adapt the operation of the computer such that the methods and/or techniques described herein are invoked or otherwise provided. Instructions for invoking the inventive methods may be stored in fixed or removable media, transmitted via a data stream in a broadcast or other signal bearing medium, and/or stored within a memory within a computing device operating according to the instructions.
- Although various embodiments which incorporate the teachings of the present invention have been shown and described in detail herein, those skilled in the art can readily devise many other varied embodiments that still incorporate these teachings.
Claims (22)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/163,838 US20120324122A1 (en) | 2011-06-20 | 2011-06-20 | Method and apparatus for server-side adaptive streaming |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/163,838 US20120324122A1 (en) | 2011-06-20 | 2011-06-20 | Method and apparatus for server-side adaptive streaming |
Publications (1)
Publication Number | Publication Date |
---|---|
US20120324122A1 true US20120324122A1 (en) | 2012-12-20 |
Family
ID=47354659
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/163,838 Abandoned US20120324122A1 (en) | 2011-06-20 | 2011-06-20 | Method and apparatus for server-side adaptive streaming |
Country Status (1)
Country | Link |
---|---|
US (1) | US20120324122A1 (en) |
Cited By (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130060904A1 (en) * | 2011-07-15 | 2013-03-07 | Shmuel Ur | User-controlled download duration time |
US20130091249A1 (en) * | 2011-10-07 | 2013-04-11 | Kevin McHugh | Http adaptive streaming server with automatic rate shaping |
US20140201333A1 (en) * | 2013-01-17 | 2014-07-17 | Electronics And Telecommunications Research Institute | Method of adaptively delivering media based on reception status information from media client and apparatus using the same |
WO2014110670A1 (en) * | 2013-01-21 | 2014-07-24 | Mimik Technology Inc. | Media server |
WO2014117775A1 (en) * | 2013-01-31 | 2014-08-07 | Codemate A/S | Network content delivery method using a delivery helper node |
US20140304377A1 (en) * | 2011-10-17 | 2014-10-09 | Telefonaktiebolaget Lm Ericsson (Publ) | Method for adaptive streaming, local storing and post-storing quality increase of a content file |
US20160019397A1 (en) * | 2014-07-16 | 2016-01-21 | Theplatform, Llc | Managing Access Rights To Content Using Social Media |
US20160212189A1 (en) * | 2015-01-16 | 2016-07-21 | Boe Technology Group Co., Ltd | Streaming media data transmission method, client and server |
EP2973228A4 (en) * | 2013-07-26 | 2016-08-31 | Huawei Tech Co Ltd | Spatial adaptation in adaptive streaming |
US9462032B2 (en) | 2013-07-24 | 2016-10-04 | Google Inc. | Streaming media content |
US9479807B1 (en) * | 2011-09-29 | 2016-10-25 | Arris Enterprises, Inc. | Gateway-based video client-proxy sub-system for managed delivery of A/V content using fragmented method in a stateful system |
US20170324642A1 (en) * | 2016-05-04 | 2017-11-09 | Microsoft Technology Licensing, Llc | Initial and periodic slowdowns for background connections |
US10091264B2 (en) * | 2015-12-26 | 2018-10-02 | Intel Corporation | Technologies for streaming device role reversal |
US10708667B1 (en) * | 2016-06-28 | 2020-07-07 | Amazon Technologies, Inc. | Combining fragments with different encodings |
US11157556B2 (en) * | 2019-12-20 | 2021-10-26 | Sling Media Pvt Ltd | Method and apparatus for thumbnail generation for a video device |
US11558480B2 (en) | 2014-07-16 | 2023-01-17 | Comcast Cable Communications Management, Llc | Tracking content use via social media |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5943316A (en) * | 1995-05-17 | 1999-08-24 | Roke Manor Research Limited | Credit bandwidth allocator for a radio system |
US6654417B1 (en) * | 1998-01-26 | 2003-11-25 | Stmicroelectronics Asia Pacific Pte. Ltd. | One-pass variable bit rate moving pictures encoding |
US20080259796A1 (en) * | 2008-04-17 | 2008-10-23 | Glen Patrick Abousleman | Method and apparatus for network-adaptive video coding |
US20100131671A1 (en) * | 2008-11-24 | 2010-05-27 | Jaspal Kohli | Adaptive network content delivery system |
US20110225315A1 (en) * | 2010-03-09 | 2011-09-15 | Mobixell Networks Ltd. | Multi-stream bit rate adaptation |
US20110292884A1 (en) * | 2010-05-26 | 2011-12-01 | Qualcomm Incorporated | Methods and apparatus supporting load balancing in a wireless communications system |
US20120311174A1 (en) * | 2010-02-19 | 2012-12-06 | Guillaume Bichot | Multipath delivery for adaptive streaming |
-
2011
- 2011-06-20 US US13/163,838 patent/US20120324122A1/en not_active Abandoned
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5943316A (en) * | 1995-05-17 | 1999-08-24 | Roke Manor Research Limited | Credit bandwidth allocator for a radio system |
US6654417B1 (en) * | 1998-01-26 | 2003-11-25 | Stmicroelectronics Asia Pacific Pte. Ltd. | One-pass variable bit rate moving pictures encoding |
US20080259796A1 (en) * | 2008-04-17 | 2008-10-23 | Glen Patrick Abousleman | Method and apparatus for network-adaptive video coding |
US20100131671A1 (en) * | 2008-11-24 | 2010-05-27 | Jaspal Kohli | Adaptive network content delivery system |
US20120311174A1 (en) * | 2010-02-19 | 2012-12-06 | Guillaume Bichot | Multipath delivery for adaptive streaming |
US20110225315A1 (en) * | 2010-03-09 | 2011-09-15 | Mobixell Networks Ltd. | Multi-stream bit rate adaptation |
US20110292884A1 (en) * | 2010-05-26 | 2011-12-01 | Qualcomm Incorporated | Methods and apparatus supporting load balancing in a wireless communications system |
Cited By (32)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9319453B2 (en) * | 2011-07-15 | 2016-04-19 | Shmuel Ur | User-controlled download duration time |
US20130060904A1 (en) * | 2011-07-15 | 2013-03-07 | Shmuel Ur | User-controlled download duration time |
US9479807B1 (en) * | 2011-09-29 | 2016-10-25 | Arris Enterprises, Inc. | Gateway-based video client-proxy sub-system for managed delivery of A/V content using fragmented method in a stateful system |
US20130091249A1 (en) * | 2011-10-07 | 2013-04-11 | Kevin McHugh | Http adaptive streaming server with automatic rate shaping |
US8751679B2 (en) * | 2011-10-07 | 2014-06-10 | Ericsson Television Inc. | HTTP adaptive streaming server with automatic rate shaping |
US9628532B2 (en) | 2011-10-07 | 2017-04-18 | Ericsson Ab | HTTP adaptive streaming server with automatic rate shaping |
US9674266B2 (en) * | 2011-10-17 | 2017-06-06 | Telefonaktiebolaget Lm Ericsson (Publ) | Method for adaptive streaming, local storing and post-storing quality increase of a content file |
US20140304377A1 (en) * | 2011-10-17 | 2014-10-09 | Telefonaktiebolaget Lm Ericsson (Publ) | Method for adaptive streaming, local storing and post-storing quality increase of a content file |
US20140201333A1 (en) * | 2013-01-17 | 2014-07-17 | Electronics And Telecommunications Research Institute | Method of adaptively delivering media based on reception status information from media client and apparatus using the same |
US9654533B2 (en) * | 2013-01-17 | 2017-05-16 | Electronics And Telecommunications Research Institute | Method of adaptively delivering media based on reception status information from media client and apparatus using the same |
WO2014110670A1 (en) * | 2013-01-21 | 2014-07-24 | Mimik Technology Inc. | Media server |
WO2014117775A1 (en) * | 2013-01-31 | 2014-08-07 | Codemate A/S | Network content delivery method using a delivery helper node |
US9462032B2 (en) | 2013-07-24 | 2016-10-04 | Google Inc. | Streaming media content |
EP2973228A4 (en) * | 2013-07-26 | 2016-08-31 | Huawei Tech Co Ltd | Spatial adaptation in adaptive streaming |
US10447746B2 (en) | 2013-07-26 | 2019-10-15 | Futurewei Technologies, Inc. | System and method for spatial adaptation in adaptive streaming |
US11558480B2 (en) | 2014-07-16 | 2023-01-17 | Comcast Cable Communications Management, Llc | Tracking content use via social media |
US9633124B2 (en) * | 2014-07-16 | 2017-04-25 | Theplatform, Llc | Managing access rights to content using social media |
US11741178B2 (en) * | 2014-07-16 | 2023-08-29 | Comcast Cable Communications Management, Llc | Managing access rights to content using social media |
US20180032624A1 (en) * | 2014-07-16 | 2018-02-01 | Theplatform, Llc | Managing Access Rights To Content Using Social Media |
US20160019397A1 (en) * | 2014-07-16 | 2016-01-21 | Theplatform, Llc | Managing Access Rights To Content Using Social Media |
US10127326B2 (en) * | 2014-07-16 | 2018-11-13 | Comcast Cable Communications Management, Llc | Managing access rights to content using social media |
US20190266199A1 (en) * | 2014-07-16 | 2019-08-29 | Comcast Cable Communications Management, Llc | Managing access rights to content using social media |
US10671687B2 (en) * | 2014-07-16 | 2020-06-02 | Comcast Cable Communications Management, Llc | Managing access rights to content using social media |
US20160212189A1 (en) * | 2015-01-16 | 2016-07-21 | Boe Technology Group Co., Ltd | Streaming media data transmission method, client and server |
US10516712B2 (en) * | 2015-01-16 | 2019-12-24 | Boe Technology Group Co., Ltd. | Streaming media data transmission method, client and server |
US10091264B2 (en) * | 2015-12-26 | 2018-10-02 | Intel Corporation | Technologies for streaming device role reversal |
US11405443B2 (en) | 2015-12-26 | 2022-08-02 | Intel Corporation | Technologies for streaming device role reversal |
US20230047746A1 (en) * | 2015-12-26 | 2023-02-16 | Intel Corporation | Technologies for streaming device role reversal |
US20170324642A1 (en) * | 2016-05-04 | 2017-11-09 | Microsoft Technology Licensing, Llc | Initial and periodic slowdowns for background connections |
US10708667B1 (en) * | 2016-06-28 | 2020-07-07 | Amazon Technologies, Inc. | Combining fragments with different encodings |
US11157556B2 (en) * | 2019-12-20 | 2021-10-26 | Sling Media Pvt Ltd | Method and apparatus for thumbnail generation for a video device |
US11714850B2 (en) | 2019-12-20 | 2023-08-01 | Dish Network Technologies India Private Limited | Method and apparatus for thumbnail generation for a video device |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20120324122A1 (en) | Method and apparatus for server-side adaptive streaming | |
US8732274B2 (en) | Method and apparatus for generating and handling streaming media quality-of-experience metrics | |
US10110650B2 (en) | Client side stream switching | |
EP3200423B1 (en) | Media host transmitting media stream with adapted bit rate | |
US9167007B2 (en) | Stream complexity mapping | |
US9060207B2 (en) | Adaptive video streaming over a content delivery network | |
EP3539295B1 (en) | Initial bitrate selection for a video delivery session | |
US9306994B2 (en) | Stabilization of adaptive streaming video clients through rate limiting | |
CA2718720C (en) | Method and system for fast channel switching using standard rtsp messages | |
US11477257B2 (en) | Link-aware streaming adaptation | |
US10205984B1 (en) | Dynamic bitrate selection for streaming media | |
US20140095593A1 (en) | Method and apparatus for transmitting data file to client | |
US10469627B2 (en) | Rapid optimization of media stream bitrate | |
Bentaleb et al. | Performance analysis of ACTE: A bandwidth prediction method for low-latency chunked streaming | |
US20160036883A1 (en) | Systems and methods for selective transport accelerator operation | |
CN104581340A (en) | Client-side, streaming media data receiving method and streaming media data transmission system | |
US11777871B2 (en) | Delivery of multimedia components according to user activity | |
US20150350700A1 (en) | Media requests for trickplay | |
WO2013074335A1 (en) | Optimizing streaming of a group of videos | |
Pelayo | Bandwidth prediction for adaptive video streaming |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: ALCATEL-LUCENT USA INC., NEW JERSEY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MILES, DAVID, MR.;HEMIGE, VENUGOPAL;SIGNING DATES FROM 20110616 TO 20110617;REEL/FRAME:026481/0210 |
|
AS | Assignment |
Owner name: ALCATEL LUCENT, FRANCE Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:ALCATEL-LUCENT USA INC.;REEL/FRAME:028620/0682 Effective date: 20120723 |
|
AS | Assignment |
Owner name: CREDIT SUISSE AG, NEW YORK Free format text: SECURITY AGREEMENT;ASSIGNOR:LUCENT, ALCATEL;REEL/FRAME:029821/0001 Effective date: 20130130 Owner name: CREDIT SUISSE AG, NEW YORK Free format text: SECURITY AGREEMENT;ASSIGNOR:ALCATEL LUCENT;REEL/FRAME:029821/0001 Effective date: 20130130 |
|
AS | Assignment |
Owner name: ALCATEL LUCENT, FRANCE Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG;REEL/FRAME:033868/0555 Effective date: 20140819 |
|
AS | Assignment |
Owner name: PROVENANCE ASSET GROUP LLC, CONNECTICUT Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:NOKIA TECHNOLOGIES OY;NOKIA SOLUTIONS AND NETWORKS BV;ALCATEL LUCENT SAS;REEL/FRAME:043877/0001 Effective date: 20170912 Owner name: NOKIA USA INC., CALIFORNIA Free format text: SECURITY INTEREST;ASSIGNORS:PROVENANCE ASSET GROUP HOLDINGS, LLC;PROVENANCE ASSET GROUP LLC;REEL/FRAME:043879/0001 Effective date: 20170913 Owner name: CORTLAND CAPITAL MARKET SERVICES, LLC, ILLINOIS Free format text: SECURITY INTEREST;ASSIGNORS:PROVENANCE ASSET GROUP HOLDINGS, LLC;PROVENANCE ASSET GROUP, LLC;REEL/FRAME:043967/0001 Effective date: 20170913 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: NOKIA US HOLDINGS INC., NEW JERSEY Free format text: ASSIGNMENT AND ASSUMPTION AGREEMENT;ASSIGNOR:NOKIA USA INC.;REEL/FRAME:048370/0682 Effective date: 20181220 |
|
AS | Assignment |
Owner name: PROVENANCE ASSET GROUP LLC, CONNECTICUT Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CORTLAND CAPITAL MARKETS SERVICES LLC;REEL/FRAME:058983/0104 Effective date: 20211101 Owner name: PROVENANCE ASSET GROUP HOLDINGS LLC, CONNECTICUT Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CORTLAND CAPITAL MARKETS SERVICES LLC;REEL/FRAME:058983/0104 Effective date: 20211101 Owner name: PROVENANCE ASSET GROUP LLC, CONNECTICUT Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:NOKIA US HOLDINGS INC.;REEL/FRAME:058363/0723 Effective date: 20211129 Owner name: PROVENANCE ASSET GROUP HOLDINGS LLC, CONNECTICUT Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:NOKIA US HOLDINGS INC.;REEL/FRAME:058363/0723 Effective date: 20211129 |
|
AS | Assignment |
Owner name: RPX CORPORATION, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:PROVENANCE ASSET GROUP LLC;REEL/FRAME:059352/0001 Effective date: 20211129 |