Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
  • H04L65/60—Network streaming of media packets
  • H04L65/75—Media network packet handling
  • H04L65/752—Media network packet handling adapting media to network capabilities
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/146—Data rate or code amount at the encoder output
  • H04N19/15—Data rate or code amount at the encoder output by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
  • H04L65/60—Network streaming of media packets
  • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
  • H04L65/612—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for unicast
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
  • H04L65/60—Network streaming of media packets
  • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
  • H04L65/613—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for the control of the source by the destination
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
  • H04L65/80—Responding to QoS
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L67/00—Network arrangements or protocols for supporting network services or applications
  • H04L67/01—Protocols
  • H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
  • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
  • H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
  • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
  • H04N21/23439—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements for generating different versions
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/239—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
  • H04N21/2393—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests involving handling client requests
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
  • H04N21/2402—Monitoring of the downstream path of the transmission network, e.g. bandwidth available
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
  • H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
  • H04N21/26258—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
  • H04N21/44209—Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
  • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
  • H04N21/845—Structuring of content, e.g. decomposing content into time segments
  • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments

Definitions

• the present disclosure relates generally to the field of HTTP streaming and more specifically to just-in-time encoding of data files in response to a received streaming request.
• HTTP based streaming systems e.g. HTTP Live Streaming (HLS) [1] or MPEG Dynamic Adaptive Streaming over HTTP (DASH) [2]
• manifest files corresponding to various bit rates and resolutions (variants) are created and advertised to the clients.
• a client device connects to the server over a network and downloads the segments (chunks) of the video content that are listed in the manifest file.
• the client device regularly measures the network bandwidth and based on that asks for the content segment having the bit rate to accommodate the network bandwidth available at that time.
• the traditional HTTP streaming system generates or stores multiple variants of the same content at various resolutions and bit rates even though at a given time a client requests only one of them. This adds cost associated with generating and/or storing the all the variants. What is needed is an approach utilizing Pseudo Manifest file which provides an alternative to traditional HTTP streaming system, without requiring any changes on the client side, and can allow significantly lower storage and/or encoding cost in various systems.
• Fig.1 depicts a graphical representation 100 of a latency issue that results in a client ultimately requesting a less-than-optimal desired delivery rate.
• the x-axis 102 is the chunk number being downloaded and the y-axis 104 is the bit rate.
• the first curve 110 shows the available bandwidth (in units of bits per sec) as measured by a client at a given time, the second curve 108 shows the bit rate requested by a client and the third curve 106 shows the bit rate served by the server.
• the requested bit rate is lower than the measured bandwidth by an amount that is controlled by the client based on its internal algorithm on how aggressive it is in coming close to the measured available bandwidth.
• client asks for the bit rate that is 50% of measured available bandwidth if corresponding variant is advertised in the manifest file.
• the served bit rate also increases with some delay.
• the measured available bandwidth decreases from 4 Mbps to 3Mbps and the requested bit rate drops from 2 Mbps to 1.5 Mbps the client is expecting to get the bits corresponding to the advertised bit rate of 1.5 Mbps.
• the server will serve chunks that are at created at 2 Mbps.
• a system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions.
• One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.
• One general aspect includes a method of data delivery including: accessing a data file, publishing a pseudo-manifest of two or more available data bit rates, receiving a request for delivery of one of said two or more available data bit rates, encoding a first chunk of said data file in response to said request for delivery of said one of said two or more available data bit rates, and transmitting said chuck, where said encoding of said first chunk is processed at a rate greater than a real-time transcoding rate.
• Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.
• Implementations may include one or more of the following features.
• the method further including: determining an encoding rate for said first chunk of said data file based at least in part on said request.
• the method where said encoding of said first chunk occurs at a rate at least four times a real time transcoding rate.
• the method further including: determining a transmission time of said encoded first chunk of said data file, and determining an encoding rate for said first chunk of said data file based at least in part on said request and said transmission time.
• the method where said encoding of said first chunk occurs at a rate at least four times a real-time transcoding rate.
• the method where said encoding of said first chunk occurs at a rate of at least eight times a real-time transcoding rate.
• the method further including: receiving a second request for delivery of one of said two or more available data bit rates; in response to at least said request for delivery of one of said two or more available data bit rates and said second request for delivery of one of said two or more available data bit rates, encoding a second chunk of said data file; where said encoding rate of said second chunk can be greater than, less than or the same as the encoding rate for said first chunk.
• the method where the encoding rate of said first chunk is at least four times a real-time transcoding rate.
• the method where the encoding rate of said second chunk is at least four times a real-time transcoding rate.
• the method where the encoding rate of said second chunk is at least four times a real-time transcoding rate.
• Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.
• Fig. 1 depicts a graph showing deficiencies with the prior art, just-in-time data delivery systems.
• Fig. 2 depicts a system and method of just-in-time data delivery.
• Fig. 3 depicts transcoding/encoding and transmission times in view of a chunk duration.
• Fig. 4 depicts a table showing client-side measured bandwidth based upon a network bandwidth and a transcoding speed.
• Fig. 5 depicts a graphical representation of the data provided in Fig. 4.
• Fig. 6 depicts a graphical representation of transcoding/encoding and transmission times relative to the duration of a chuck, when operations are conducted in parallel.
• Fig. 7 depicts a graphical representation of transcoding/encoding and transmission times relative to the duration of a chuck, when operations are conducted in parallel.
• Fig. 8 depicts a graphical representation of HLS client behavior with transcoding/encoding operating at four times real-time transcoding speed and variable client aggressiveness, for serial operation.
• Fig. 9 depicts a graphical representation of HLS client behavior with transcoding/encoding operating at eight times real-time transcoding speed and variable client aggressiveness, for serial operation.
• Fig. 10 depicts a graphical representation of HLS client behavior with transcoding/encoding operating at four times real-time transcoding speed and 50% client aggressiveness, for serial operation.
• Fig. 11 depicts a graphical representation of HLS client behavior with transcoding/encoding operating at eight times real-time transcoding speed and 90% client aggressiveness, for serial operation.
• Fig. 12 depicts a graphical representation of a requested client bit rate based on a transcoding/encoding speed operating at four times real-time transcoding speed and 50% client aggressiveness, for parallel operation in a first case.
• Fig. 13 depicts a graphical representation of a requested client bit rate based on a transcoding/encoding speed operating at four times real-time transcoding speed and 50% client aggressiveness, for parallel operation in a second case.
• Fig. 14 depicts a graphical representation of a requested client bit rate based on a transcoding/encoding speed operating at four times real-time transcoding speed, 50% client aggressiveness and a large gap in advertised bit rate, for parallel operation in a second case.
• Fig. 15 depicts a graphical representation of a requested client bit rate based on a transcoding/encoding speed operating at four times real-time transcoding speed and 50% client aggressiveness, for parallel operation in a hybrid case.
• FIG. 16 depicts an embodiment of a computer system adapted and/or configured to process a method of transcoding/encoding for just-in-time data delivery.
• FIG. 17 depicts an embodiment of a coder/decoder system in a JVET (or any other codec standard like AVC/HEVC) encoder/decoder.
• Fig. 2 depicts a system and method of just-in-time data delivery 200.
• a data file can be received in step 202.
• the data file can be a video file.
• the data file can be any form or type of data file.
• pseudo manifest data can be generated based on desired and/or convenient data bit rates at which the data file can be encoded and delivered to client devices.
• a server can receive a request from a client device which can be related to one of the published bit rates indicated in the pseudo manifest data. Based upon a request received in step 206, a server can determine an appropriate coding rate 208 and then encode a chuck of the data file at the requested bit rate at an encoding rate faster-than-real -time encoding in step 210.
• the encoding rate can be performed at the rate determined in step 208, which can be higher, lower or the same as an encoding rate for a previous chunk.
• the faster-than-real - time encoding rate can be four times a real-time encoding rate.
• the faster-than-real-time encoding rate can be eight times a real-time encoding rate.
• the faster-than-real-time encoding rate can be any known, convenient and/or desired encoding rate that is faster than real-time encoding.
• Fig. 3 depicts transcoding/encoding time 306 and transmission times 304 in view of a chunk duration 302.
• a chunk gets created when a client requests it and the transmission starts after the creation is complete.
• this can happen when mp4 fragments are used.
• fragmented mp4 files contain header information with byte range offsets that can be placed prior to the media data and this information can be generated after the entire segment has been created.
• the video chunks (segments) needs to be fully generated before a packager can start its operation.
• this can happen when the packager is separated from the transcoding unit and may need all the bits in the chunk before creating and advertising their availability to a client.
• s be the faster than real time speed factor—that is, for real time transcoding/encoding s is 1, for the transcoder/encoder running at twice the speed, s is 2 and so on.
• T the segment duration. Accordingly, it will take the transcoder/encoder T/s sees to create the bits corresponding to a chunk of that duration. Hence, T - T/s seconds remain to transmit all the bits corresponding to that chunk. If it is assumed that Tt is the time it takes to send the chunk at the available bandwidth.
• BWa the available network bandwidth in bits/sec
• BWm the bandwidth measured by a client.
• Tt for a chunk is then given by:
• BWm is a function of BWa, but also of br. (the number of bits in the variant being transmitted). From this it can be noted that lower values of br result in lower values of BWm.
• VBR Variable Bit Rate
• FIG. 4 depicts a table 400 showing client-side measured bandwidth 402 based upon a variant bitrate (bitrate of the chunk) 404 and a encoding/transcoding speed, s.
• Figs. 4 and 5 demonstrate non-limiting examples of the BWm values calculated for a fixed network bandwidth of 10 Mbps and chunk duration of 2 seconds.
• Fig. 4 demonstrates that the client can measure more available bandwidth for a chunk with a higher bit rate variant at a given transcoding speed.
• a client can measure available bandwidth to be 1.07 Mbps 406 while for the 3.5 Mbps bit rate variant with the same transcoding speed of 4x and bandwidth BWa, it can measure at 5.8 Mbps 408. This behavior is due to the fact that more number of bits are being transmitted at higher bit rates in duration Tt.
• Fig. 5 depicts a graphical representation 500 of the data provided in Fig. 4.
• the x-axis 502 is the chunk rate, br in Kbps and the y-axis represented client-side measured bandwidth, BWm.
• each of the lines 506 508 510 512 514 516 518 520 represent the relevant data plotted corresponding to measured bandwidth, BWm, for a given encoding/transcoding speed, s, wherein the encoding/transcoding incrementally from 1 - 8 for each of the lines 506 508 510 512 514 516 518 520, respectively.
• bit rate requested by a client is given by [0044]
• A is the factor of aggressiveness of a client in requesting the bit rate closer to the measured bandwidth and B r is the bit rate requested by the client.
• [0050] is less than 1 and represents the Pseudo Manifest Penalty.
• a client can then request a bit rate that is smaller by this factor than would be requested (A x BW a ) in a system in which all variants were available, such as a traditional HLS system.
• This equation shows that as s increases the Pseudo Manifest Penalty tends to 1. Accordingly, with a greater value of s, a system utilizing a Pseudo Manifest file can more closely approximate the performance of a traditional HLS system, in which all published bitrates are generated ahead of time.
• a maximum spacing between advertised bit rates can be determined.
• an advertised and used variant bit rate in the manifest filed can be Bman(n) and the number of bits transferred for downloading the chunk of duration T from the manifest file can be
• bits br can be downloaded at the available bandwidth, BWa, during the transfer time, Tt.
• bitrate Bman(n+i) of the next variant that is advertised in the manifest file can be such that
• the client may not request the next higher bit rate even though network bandwidth may be available. This results in spacing between the advertised bit rates to be such that
• bit rate at the next variant Bman(n+i) should be such that
• next requested bit rate can be the highest allowed by the network and there is not significant penalty of using a Pseudo Manifest file for high bit rate variants and/or high transcoding speed s.
• Br is a monotonically increasing function of Tt.
• B r is maximum when Tt is maximum.
• the maximum chunk transfer time Tt max can be the entire remainder of the chunk duration calculated as the time from the end of transcode to the end of the chunk duration. If the chunk transfer time exceeds this, the client's buffer level can drop causing underflows leading to a decision by the client to switch to a lower bitrate.
• Figs. 6 and 7 depict graphical representations 600 700 of transcoding/encoding times 604 and transmission times 602 relative to the duration of a chunk 302, when operations are conducted in parallel. In such a case, the transcoding time and the transmission time overlap as shown in Figs. 6 and 7. Depending on the speed of transcoding s and the available bandwidth, two possible scenarios shown in Figs. 6 and 7 can happen.
• the transmission time, Tt, needed to send the br bits in a particular variant is greater than the transcoding time. Otherwise, the scenario shown in Fig. 7 applies.
• the system will behave the same as it does in the traditional ULS system when all the variants advertised in the manifest file are available ahead of time.
• the bandwidth measure by a client will be
• bit rate of the current variant Bman(n),that is being requested and transmitted is smaller than B r , then substituting B man(n) for br /T and Bman(n+i) for B r , results in a next requested bit rate of
• the case depicted in Fig. 6 can generally be applied for conditions under which there are high advertised bit rates and the case depicted in Fig. 7 can generally be applied for conditions under which there are low advertised bit rates.
• the maximum bit rate separation in situations in which there are low advertised bit rates, variants can follow:
• the maximum requested bit rate will approach and can eventually be the same as a traditional HLS system.
• the rate at which the system will reach the traditional HLS system levels is slower than that in traditional HLS systems.
• Figs. 8 - 15 depict a series of experimental results for various conditions of Aggressiveness, A, and speed factor, s.
• the setup consisted of the following:
• a headless HLS client was utilized. The client downloaded the chunks like a standard HLS client, but did not decode or display them. If the download finished in less than the chunk duration of T seconds then like a standard iPad client, the client waits till the end of the chunk duration before requesting more data. Additionally, the client was implemented with the following [00149] ABR algorithm:
• the client will switch to the first higher rate that is just lower than A times the measured bandwidth. This switch to a higher rate will occur only if there is no buffer underflow in the client due to download of a chunk taking longer than the duration of the chunk itself. By way of non-limiting example, if the chunk has a duration of 2 seconds, it must be downloaded within 2 seconds.
• Fig. 8 depicts a graphical representation 800 of client behavior with transcoding/encoding operating at four times real-time transcoding speed and variable client aggressiveness, for serial operation.
• the x-axis 802 represents the chunk number and the y-axis 804 represents the bit rate.
• Client behavior was observed to see if there was a bound the highest manifest variant that would be requested by a client for given values of s and A for the serial operation of transcoding and transmission.
• Fig. 9 depicts a graphical representation of HLS client behavior with transcoding/encoding operating at eight times real-time transcoding speed and variable client aggressiveness, for serial operation.
• the x-axis 802 represents the chunk number and the y-axis 804 represents the bit rate.
• Figs. 8 and 9 confirm the behavior of the system as predicted by equations. They show highest bit rates B r max requested by a client is a function of A as well as the transcoding speed s. The client starts with lower bit rate and starts requesting variants with higher and higher bit rates until a plateau is reached. That plateau in the highest requested bit rate is close to the one predicted. As the aggressiveness A of the client and the transcoding speed s increase, the level of plateau also increases to the levels close to the values obtained by mathematical prediction.
• Fig. 10 depicts a graphical representation of HLS client behavior with transcoding/encoding operating at four times real-time transcoding speed and 50% client aggressiveness, for serial operation.
• the x-axis 802 represents the chunk number and the y-axis 804 represents the bit rate and line 1002 represents Bmani, line 1004 represents A x BWm, line 1006 represents Bman2 and line 1008 represents BWm.
• the second set of experiments included removing variants from the manifest file to investigate if the rule of maximum spacing between the bit is violated then that will cause a client to get stuck at the lower bitrate.
• the results are shown in Fig. 10.
• Bmani represents the first variant bitrate in the manifest file and had the bit rate of 300 Kbps.
• Bman2 represents the next available variant and had bit rate of 700 Kbps. With a faster than real time transcoding of 4x and aggressiveness factor A of 0.5, the next variant should not be above about 600 kbps.
• the x-axis 802 shows the chunk number that is downloaded and the y-axis 804 shows the bit rate of that chunk. As it can be seen in Fig.
• the client is stuck at 300 kbps and does not ask for 700 kbps.
• the client can be stuck playing back content at 300 Kbps even if the available bandwidth BW a is high as 10 Mbps.
• Fig. 11 depicts a graphical representation of HLS client behavior with transcoding/encoding operating at eight times real-time transcoding speed and 90% client aggressiveness, for serial operation.
• the x-axis 802 represents the chunk number and the y-axis 804 represents the bit rate and line 1102 represents Bmani, line 1104 represents A x BWm, line 1106 represents BWm and line 1008 represents Bman2.
• Bmani was the first available variant of 300 Kbps and B ma n2, the next published variant of 2200 Kbps.
• the maximum bandwidth measured by the client BWm is about 1900 Kbps. If the aggressiveness of the client is 0.9, then the next bitrate that a client will ask for will be less than about 1710 Kbps.
• the client can again be stuck at playing the 300 Kbps variant even if the available bandwidth BW a is 10 Mbps.
• the next experiment included observing system behavior for the parallel operation of transcoding and transmission for various values of s and A.
• Fig. 12 shows the results obtained for one case of parallel processing. Various values of the parameters are shown on the top of the figure. As expected the requested bit rate jumped from 600kbps to 1Mbps and skipped 700kbps and 800 kbps variants as the measured and available bandwidths were more than 2 Mbps.
• Fig. 12 depicts a graphical representation of a requested client bit rate based on a transcoding/encoding speed operating at four times real-time transcoding speed and 50% client aggressiveness, for parallel operation in a first case.
• the x- axis 802 represents the chunk number and the y-axis 804 represents the bit rate and line 1202 represents B r and line 1204 represents BWa.
• Fig. 13 depicts a graphical representation of a requested client bit rate based on a transcoding/encoding speed operating at four times real-time transcoding speed and 50% client aggressiveness, for parallel operation in a second case.
• the x-axis 802 represents the chunk number and the y-axis 804 represents the bit rate and line 1302 represents B r and line 1304 represents BWa.
• Fig. 13 shows the results obtained for a second case of parallel processing. Various values of the parameters are shown on the top of the figure. As expected the requested bit rates now did not jump from 600kbps straight to 2Mbps. As predicted, it reached the value by taking intermediate jumps to 1 and 1.5 Mbps.
• the client may be stuck at lower bit rate. This is shown in Fig. 14, when the variant with 1Mbps was removed in the above manifest file. The client got stuck at 600 kbps and did not request any variant at higher bit rates even though the available bandwidth was 10 Mbps.
• Fig. 14 depicts a graphical representation of a requested client bit rate based on a transcoding/encoding speed operating at four times real-time transcoding speed, 50% client aggressiveness and a large gap in advertised bit rate, for parallel operation in a second case.
• the x-axis 802 represents the chunk number and the y-axis 804 represents the bit rate and line 1402 represents B r and line 1404 represents BW a .
• Fig. 15 depicts a graphical representation of a requested client bit rate based on a transcoding/encoding speed operating at four times real-time transcoding speed and 50% client aggressiveness, for parallel operation in a hybrid case.
• Fig. 15 shows the results obtained for the hybrid case where a first case was valid in the beginning where the system started with lower bit rates and as the bit rates increased the behavior corresponds to a second case for parallel processing.
• the x-axis 802 represents the chunk number and the y-axis 804 represents the bit rate and line 1502 represents B r and line 1504 represents BW a .
• execution of the sequences of instructions required to practice the embodiments can be performed by a computer system 1600 as shown in Fig. 16.
• execution of the sequences of instructions is performed by a single computer system 1600.
• two or more computer systems 1600 coupled by a communication link 1615 can perform the sequence of instructions in coordination with one another.
• a description of only one computer system 1600 will be presented below, however, it should be understood that any number of computer systems 1600 can be employed to practice the embodiments.
• FIG. 16 is a block diagram of the functional components of a computer system 1600.
• the term computer system 1600 is broadly used to describe any computing device that can store and independently run one or more programs.
• Each computer system 1600 can include a communication interface 1614 coupled to the bus 1606.
• the communication interface 1614 provides two-way communication between computer systems 1600.
• the communication interface 1614 of a respective computer system 1600 transmits and receives electrical, electromagnetic or optical signals, that include data streams representing various types of signal information, e.g., instructions, messages and data.
• a communication link 1615 links one computer system 1600 with another computer system 1600.
• the communication link 1615 can be a LAN, in which case the communication interface 1614 can be a LAN card, or the communication link 1615 can be a PSTN, in which case the communication interface 1614 can be an integrated services digital network (ISDN) card or a modem, or the communication link 1615 can be the Internet, in which case the communication interface 1614 can be a dial-up, cable or wireless modem.
• ISDN integrated services digital network
• a computer system 1600 can transmit and receive messages, data, and instructions, including program, i.e., application, code, through its respective communication link 1615 and communication interface 1614.
• Received program code can be executed by the respective processor(s) 1607 as it is received, and/or stored in the storage device 1610, or other associated non-volatile media, for later execution.
• the computer system 1600 operates in conjunction with a data storage system 1631, e.g., a data storage system 1631 that contains a database 1632 that is readily accessible by the computer system 1600.
• the computer system 1600 communicates with the data storage system 1631 through a data interface 1633.
• a data interface 1633 which is coupled to the bus 1606, transmits and receives electrical, electromagnetic or optical signals, that include data streams representing various types of signal information, e.g., instructions, messages and data.
• the functions of the data interface 1633 can be performed by the communication interface 1614.
• Computer system 1600 includes a bus 1606 or other communication mechanism for communicating instructions, messages and data, collectively, information, and one or more processors 1607 coupled with the bus 1606 for processing information.
• Computer system 1600 also includes a main memory 1608, such as a random access memory (RAM) or other dynamic storage device, coupled to the bus 1606 for storing dynamic data and instructions to be executed by the processor(s) 1607.
• the main memory 1608 also can be used for storing temporary data, i.e., variables, or other intermediate information during execution of instructions by the processor(s) 1607.
• the computer system 1600 can further include a read only memory (ROM) 1609 or other static storage device coupled to the bus 1606 for storing static data and instructions for the processor(s) 1607.
• ROM read only memory
• a storage device 1610 such as a magnetic disk or optical disk, can also be provided and coupled to the bus 1606 for storing data and instructions for the processor(s) 1607.
• a computer system 1600 can be coupled via the bus 1606 to a display device 1611, such as, but not limited to, a cathode ray tube (CRT) or a liquid-crystal display (LCD) monitor, for displaying information to a user.
• An input device 1612 e.g., alphanumeric and other keys, is coupled to the bus 1606 for communicating information and command selections to the processor(s) 1607.
• an individual computer system 1600 performs specific operations by their respective processor(s) 1607 executing one or more sequences of one or more instructions contained in the main memory 1608. Such instructions can be read into the main memory 1608 from another computer-usable medium, such as the ROM 1609 or the storage device 1610. Execution of the sequences of instructions contained in the main memory 1608 causes the processor(s) 1607 to perform the processes described herein.
• hardwired circuitry can be used in place of or in combination with software instructions. Thus, embodiments are not limited to any specific combination of hardware circuitry and/or software.
• Non-volatile media i.e., media that can retain information in the absence of power
• Volatile media i.e., media that cannot retain information in the absence of power
• Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise the bus 1606.
• Transmission media can also take the form of carrier waves; i.e., electromagnetic waves that can be modulated, as in frequency, amplitude or phase, to transmit information signals. Additionally, transmission media can take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
• the present invention can be implemented in a variety of computer systems.
• the various techniques described herein can be implemented in hardware or software, or a combination of both.
• the techniques are implemented in computer programs executing on programmable computers that each include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device.
• Program code is applied to data entered using the input device to perform the functions described above and to generate output information.
• the output information is applied to one or more output devices.
• Each program is preferably implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the programs can be implemented in assembly or machine language, if desired.
• the language can be a compiled or interpreted language.
• Each such computer program is preferably stored on a storage medium or device (e.g., ROM or magnetic disk) that is readable by a general or special purpose programmable computer for configuring and operating the computer when the storage medium or device is read by the computer to perform the procedures described above.
• the system can also be considered to be implemented as a computer- readable storage medium, configured with a computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner.
• the storage elements of the exemplary computing applications can be relational or sequential (flat file) type computing databases that are capable of storing data in various combinations and configurations.
• FIG. 17 is a high level view of a source device 1712 and destination device 1710 that may incorporate features of the systems and devices described herein.
• example video coding system 1710 includes a source device 1712 and a destination device 1714 where, in this example, the source device 1712 generates encoded video data. Accordingly, source device 1712 may be referred to as a video encoding device.
• Destination device 1714 may decode the encoded video data generated by source device 1712. Accordingly, destination device 1714 may be referred to as a video decoding device.
• Source device 1712 and destination device 1714 may be examples of video coding devices.
• Destination device 1714 may receive encoded video data from source device 1712 via a channel 1716.
• Channel 1716 may comprise a type of medium or device capable of moving the encoded video data from source device 1712 to destination device 1714.
• channel 1716 may comprise a communication medium that enables source device 1712 to transmit encoded video data directly to destination device 1714 in real-time.
• source device 1712 may modulate the encoded video data according to a communication standard, such as a wireless communication protocol, and may transmit the modulated video data to destination device 1714.
• the communication medium may comprise a wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines.
• the communication medium may form part of a packet-based network, such as a local area network, a wide-area network, or a global network such as the Internet.
• the communication medium may include routers, switches, base stations, or other equipment that facilitates communication from source device 1712 to destination device 1714.
• channel 1716 may correspond to a storage medium that stores the encoded video data generated by source device 1712.
• source device 1712 includes a video source 1718, video encoder 1720, and an output interface 1722.
• output interface 1728 may include a modulator/demodulator (modem) and/or a transmitter.
• video source 1718 may include a source such as a video capture device, e.g., a video camera, a video archive containing previously captured video data, a video feed interface to receive video data from a video content provider, and/or a computer graphics system for generating video data, or a combination of such sources.
• Video encoder 1720 may encode the captured, pre-captured, or computer-generated video data.
• An input image may be received by the video encoder 1720 and stored in the input frame memory 1721.
• the general purpose processor 1723 may load information from here and perform encoding.
• the program for driving the general purpose processor may be loaded from a storage device, such as the example memory modules depicted in FIG. 17.
• the general purpose processor may use processing memory 1722 to perform the encoding, and the output of the encoding information by the general processor may be stored in a buffer, such as output buffer 1726.
• the video encoder 1720 may include a resampling module 1725 which may be configured to code (e.g., encode) video data in a scalable video coding scheme that defines at least one base layer and at least one enhancement layer. Resampling module 1725 may resample at least some video data as part of an encoding process, wherein resampling may be performed in an adaptive manner using resampling filters.
• code e.g., encode
• Resampling module 1725 may resample at least some video data as part of an encoding process, wherein resampling may be performed in an adaptive manner using resampling filters.
• the encoded video data may be transmitted directly to destination device 1714 via output interface 1728 of source device 1712.
• destination device 1714 includes an input interface 1738, a video decoder 1730, and a display device 1737.
• input interface 1738 may include a receiver and/or a modem.
• Input interface 1738 of destination device 1714 receives encoded video data over channel 1716.
• the encoded video data may include a variety of syntax elements generated by video encoder 1720 that represent the video data. Such syntax elements may be included with the encoded video data transmitted on a communication medium, stored on a storage medium, or stored a file server.
• the encoded video data may also be stored onto a storage medium or a file server for later access by destination device 1714 for decoding and/or playback.
• the coded bitstream may be temporarily stored in the input buffer 1731, then loaded in to the general purpose processor 1733.
• the program for driving the general purpose processor may be loaded from a storage device or memory.
• the general purpose processor may use a process memory 1732 to perform the decoding.
• the video decoder 1730 may also include a resampling module 1735 similar to the resampling module 1725 employed in the video encoder 1720.
• FIG. 17 depicts the resampling module 1735 separately from the general purpose processor 1733, but it would be appreciated by one of skill in the art that the resampling function may be performed by a program executed by the general purpose processor, and the processing in the video decoder may be accomplished using one or more processors.
• the decoded image(s) may be stored in the output frame buffer 1736 and then sent out to the input interface 1738.
• Display device 1737 may be integrated with or may be external to destination device 1714.
• destination device 1714 may include an integrated display device and may also be configured to interface with an external display device.
• destination device 1714 may be a display device.
• display device 1737 displays the decoded video data to a user.
• video encoder 1720 and video decoder 1730 may operate according to other proprietary or industry standards that function with the disclosed JVET features.
• other standards such as the ITU-T H.264 standard, alternatively referred to as MPEG-4, Part 10, Advanced Video Coding (AVC), or extensions of such standards.
• AVC Advanced Video Coding
• techniques of this disclosure are not limited to any particular coding standard or technique.
• Other examples of video compression standards and techniques include MPEG-2, ITU- T H.263 and proprietary or open source compression formats and related formats.
• Video encoder 1720 and video decoder 1730 may also, additionally or alternately operate according to a video compression standard.
• ITU-T VCEG (Q6/16) and ISO/IEC MPEG (JTC 1/SC 29/WG 11) are studying the potential need for standardization of future video coding technology with a compression capability that significantly exceeds that of the current High Efficiency Video Coding HEVC standard (including its current extensions and near-term extensions for screen content coding and high-dynamic-range coding).
• the groups are working together on this exploration activity in a joint collaboration effort known as the Joint Video Exploration Team (JVET) to evaluate compression technology designs proposed by their experts in this area.
• JVET Joint Video Exploration Team
• Video encoder 1720 and video decoder 1730 may be implemented in hardware, software, firmware or any combination thereof.
• the video encoder 1720 and decoder 1730 may employ one or more processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, or any combinations thereof.
• DSPs digital signal processors
• ASICs application specific integrated circuits
• FPGAs field programmable gate arrays
• a device may store instructions for the software in a suitable, non-transitory computer- readable storage medium and may execute the instructions in hardware using one or more processors to perform the techniques of this disclosure.
• Each of video encoder 1720 and video decoder 1730 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder/decoder (CODEC) in a respective device.
• CDEC combined encoder/decoder
• aspects of the subject matter described herein may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer, such as the general purpose processors 1723 and 1733 described above.
• program modules include routines, programs, objects, components, data structures, and so forth, which perform particular tasks or implement particular abstract data types.
• aspects of the subject matter described herein may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network.
• program modules may be located in both local and remote computer storage media including memory storage devices.
• RAM random access memory
• ROM read only memory
• Memory may store instructions, such as source code or binary code, for performing the techniques described above. Memory may also be used for storing variables or other intermediate information during execution of instructions to be executed by a processor, such as processor 1723 and 1733.
• a storage device may also store instructions, instructions, such as source code or binary code, for performing the techniques described above.
• a storage device may additionally store data used and manipulated by the computer processor.
• a storage device in a video encoder 1720 or a video decoder 1730 may be a database that is accessed by computer system 1723 or 1733.
• Other examples of storage device include random access memory (RAM), read only memory (ROM), a hard drive, a magnetic disk, an optical disk, a CD-ROM, a DVD, a flash memory, a USB memory card, or any other medium from which a computer can read.
• a memory or storage device may be an example of a non-transitory computer-readable storage medium for use by or in connection with the video encoder and/or decoder.
• the non- transitory computer-readable storage medium contains instructions for controlling a computer system to be configured to perform functions described by particular embodiments.
• the instructions when executed by one or more computer processors, may be configured to perform that which is described in particular embodiments.
• Particular embodiments may be implemented in a non-transitory computer-readable storage medium for use by or in connection with the instruction execution system, apparatus, system, or machine.
• the computer-readable storage medium contains instructions for controlling a computer system to perform a method described by particular embodiments.
• the computer system may include one or more computing devices.
• the instructions, when executed by one or more computer processors, may be configured to perform that which is described in particular embodiments

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Computer Networks & Wireless Communication (AREA)
• Databases & Information Systems (AREA)
• Communication Control (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=66213042

Family Applications (1)

Country Status (1)

• 2017
  • 2017-09-19 EP EP17780572.8A patent/EP3488613A1/de active Pending

Similar Documents

Legal Events