CN103650526B - Playlists for real-time or near real-time streaming - Google Patents

Abstract

Methods, systems and machine readable storage medium for allowing playback of streaming media at playback rates of other than IX are described. In one embodiment, a method can determine different sets of I-frames that are available before a display deadline, where each set can have a different cadence, and one of the sets can be selected for download and display to achieve playback at other than IX. Byte range requests from a client device can be used to retrieve each of the I-frames. Other methods, system and media are also described.

Description

For real-time or near real-time streaming playlist

Related application

The application is according to 35U.S.C. § 119（e）Advocate the U.S. Provisional Application No.61/ of on June 3rd, 2011 submission The rights and interests of 493329 submitting day.This U.S. Patent application is also related to following U.S. Patent application, and each of which is all passed through to quote It is herein incorporated：

（1）The Shen of entitled " REAL-TIME OR NEAR REAL-TIME STREAMING " that on June 5th, 2009 submits to Please No.12/479690（File number P7437US1）；

（2）Entitled " the VARIANT STREAMS FOR REAL-TIME OR NEAR REAL- that on June 5th, 2009 submits to The application No.12/479698 of TIME STREAMING "（File number P7437US2）；

（3）Entitled " the UPDATABLE REAL-TIME OR NEAR REAL-TIME that on June 5th, 2009 submits to The application No.12/479,732 of STREAMING "（File number P7437US3）；And

（4）Entitled " the PLAYLISTS FOR REAL-TIME OR NEAR REAL-TIME that on June 5th, 2009 submits to The application No.12/479735 of STREAMING "（File number P7437US4）.

Technical field

Embodiments of the invention are related to data transferring technique.More particularly, embodiments of the invention are related to allow using all As HTTP（HTTP）Etc non-flow host-host protocol streaming is carried out to data（streaming）Technology.

Background technology

It is many that the streaming of content is generally referred to as constantly sent from server apparatus and received by client device Media content.Content is generally streamed while server delivers at it and is presented to terminal use.This title refers to matchmaker The delivering method of body rather than media itself.

Current streaming service generally requires special server to distribute " live " content to terminal use.Any In large scale deployment, this may result in huge cost, and needs professional and technical personnel to set up and to run.This led to less than the phase The content library hoped can be used for streaming.

Content of the invention

One of one embodiment method can use with meet HTTP fact flow host-host protocol（Referring to http:// tools.ietf.org/html/draft-pantos-http-live-streaming-06）URL be associated label middle finger Fixed bytes range required parameter fetches streaming content from the file of server storage.FTP client FTP receives played column Table, this playlist comprises URL and associated label, its appointed program can be according to this agreement come the part of streaming, this A little labels include byte range parameter（For example, the side-play amount the length of scope and a certain position from file）；URL is from client End is sent to server, and server is fetched using byte range parameter and sent this part of program.So, media content can It is stored as single big file rather than multiple less file, and each several part of content can be passed through using bytes range request HTTP fact flow host-host protocol, to fetch, is then presented with 1X playback mode by the player at client.

Also describe the streaming media being to deliver by HTTP flow host-host protocol（Such as film）There is provided " special play-back " Mode playback（Such as fast forward or reverse）Method.The method is related to I frame using inclusion（I frame is can be independent of other videos The key frame of frame decoding or intra coded frame）URL I frame playlist.Player device or client can be downloaded and use Normal playlist, it includes being related to the URL of all the elements and allowing with normal（Such as 1X）Speed is played back, player Equipment or client can be downloaded and use I frame playlist, and it includes the URL with regard to I frame, outside with normal speed Speed（8 speeds of such as 8X or normal playback rate）Played back.I frame playlist can include as with playlist in URL be associated a part for parameter byte range parameter, the following data area in its specified file：This data is to use Data in the specific I frame of video data；This scope can pass through data length（For example in units of byte）Hereof Starting position（Side-play amount for example from file beginning）To specify.So, I frame playlist may include on given server Those parameters of the bytes range pertaining only to I frame in file, to assume playback with special-effect mode on a client device.Visitor Family end equipment can recover normal playback by using the normal playlist file comprising the URL to following file：This literary composition Part is just included in that file of the I frame that I frame playlist is used for fetching during the special play-back providing streaming media.One In individual embodiment（Such as MPEG-DASH play list file）, the normal playlist file for 1X playlist also includes The data of specified I frame position.In one embodiment, the label in I frame playlist may include and is related to current I frame and next I frame Between time span parameter, this time span can be used for determine for given playback rate（Such as 8X or 16X）Using which A little I frames.Additionally, in one embodiment, client device can use variant playlist, and this variant playlist relates to save Mesh（Such as film）At least one normal playlist（1X）And the I frame broadcasting of the trickplay modes for same program List.

Phrase " bytes range " will be appreciated that the number including other sizes or group by byte number or by data refers to Fixed scope.Byte can be for example 8 bits or 9 bits or 16 bits or 32 bits or 36 bits.

In one embodiment, a kind of method of " special play-back " mode playback providing streaming media can receive first and broadcast Emplace table, this first playlist includes the URL for the second playlist and URL for the 3rd playlist；Second broadcasts Emplace the label that table may include multiple URL and associates, the part comprising I frame of its specified file（For example, specified with bytes range Part）, the 3rd playlist may include for 1X playback speed show video multiple URL（Or single URL）.First broadcasts Emplacing table can be variant playlist, and this variant playlist includes the multiple URL for 1X playback（Every in these URL One provides different resolution ratio（Such as 640 pixels take advantage of 480 pixels etc.）Or quality（Such as color bit-depth））Be used for Only contain multiple URL of the playlist of I frame（Each of these URL for I frame playlist provide different resolution ratio Or quality, for the trickplay modes playback beyond 1X）.The label being associated with the multiple URL in the 3rd playlist May also comprise bytes range, each of which is specified for fetching content and the data area with 1X playback rate playing back content.The party Method may also include：（a）Determine the playback speed different from 1X（For example there are 8X or 16X or 32X for F.F., or for refunding There are -8X or -16X or -32X）；（b）Send the URL for the second playlist in response to determination playback speed；（c）Receive the Two playlists；（d）Send the request to I frame using the URL in the second playlist with the label associating；And（e）With institute The playback speed of choosing presents（For example show on the display device）The I frame receiving in response to the request to I frame.

In one embodiment, a kind of method of the present invention can be using based on rhythm（cadence）I frame select to select can With the subset of I frame so that the I frame of download and display is at least uniformly distributed in time to a certain extent；In addition, the method can be by This subset is restricted to the number less than available I frame sum.The method may include：（a）Determine one group of parameter, for example playback rate and Can be used for downloading the time period of at least one I frame；（b）Determine the first set of available I frame, wherein this first set is based on can In time the parameter of dynamic change and determine；（c）Determine multiple subsets of I frame, each subset in plurality of subset is The subset of the I frame in this first set；（d）Determine the estimation download time of each subset in multiple subsets of I frame；（e）Base In one of the multiple subsets estimating download time and this group parameter selection I frame subset；And（f）Receive and show I frame The I frame in selected subset in multiple subsets.In an embodiment of the method, the display to I frame is provided to regarding in I frame The special play-back playback mode according to playback rate of frequency content, each I frame can be fetched with the label associating using URL（Example As downloaded）, label include identify file the part comprising corresponding I frame bytes range.This group parameter may include the display time limit The current movie time of the current I frame of parameter and just display；The time period that can be used for downloading at least one I frame can be according to current electricity Shadow time and playback rate and display live parameter, to draw, show that live parameter can be set to fixed duration（Example As 1/2 second or 1 second or 2 seconds）.In one embodiment, show live parameter（Such as 1 second）It is multiplied by playback rate and taken advantage of with producing one Long-pending（Such as 1X（8X））, this product is a time value, and this time value was added with the current movie time be can be used for downloading to draw The time period of at least one I frame.In one embodiment, the time period that can be used for downloading at least one I frame is exactly to arrive from now on The time of display deadline time.The use in display time limit can ensure that user will be appreciated that at least one I frame in each period in time limit （For example, at least one I frame per second）.In one embodiment, each subset of I frame has predetermined rhythm, and this rhythm is in I frame It is different between collection, thus each subset has the rhythm different with respect to the rhythm of other subsets.In one embodiment, I The predetermined cadence of each subset in multiple subsets of frame is one below：（a）Each in available I frame in first set Individual；（b）In available I frame every one；（c）In available I frame every two, etc..In one embodiment, this group parameter bag Include the restriction to the I frame number in each subset, from can have I frames more more than predetermined number without subset.

Describe additive method, and also describe system and machine readable non-transient storage media.

Foregoing invention content does not include the exhaustive list to all aspects of the invention.It is understood that the present invention include can root All systems according to all appropriately combined enforcement of each side disclosed in each side outlined above and specific embodiment And method.

Brief description

With example, non-limited way illustrates the present invention in the accompanying drawings, and similar reference instruction in accompanying drawing is similar to Element.

Fig. 1 be can send and receive in real time or near real-time content an embodiment of server and client side block diagram.

Fig. 2A is that one or more server apparatus utilize non-flow host-host protocol to support a reality of the technology of media content The flow chart applying example.

Fig. 2 B is that one or more server apparatus provide the dynamic played column updating to one or more client devices The flow chart of one embodiment of the technology of table.

Fig. 2 C is that one or more server apparatus utilize multiple bit rates to provide the skill of media content to client device The flow chart of one embodiment of art.

Fig. 3 A is that client device utilizes non-flow host-host protocol to support an embodiment of the technology of streaming of content Flow chart.

Fig. 3 B is that client device utilizes multiple bit rates to support the stream of an embodiment of the technology of streaming of content Cheng Tu.

Fig. 4 is the block diagram of an embodiment of server flows agency.

Fig. 5 is the block diagram of an embodiment of client stream agency.

Fig. 6 illustrates an embodiment of the play list file with multiple labels.

Fig. 7 is the flow chart of an embodiment of the playback technology for assembling stream as described herein.

Fig. 8 is the block diagram of an embodiment of electronic system.

Fig. 9 A is to illustrate how client device can switch between the replacement content in variant playlist to show The flow chart of example.

Fig. 9 B is to illustrate another stream how client device can switch between the content in two playlists Cheng Tu.

Fig. 9 C is to illustrate how client device can be utilized the example that audio mode coupling switches between content Another flow chart.

Fig. 9 D diagrammatically show the how method to realize Fig. 9 C using audio mode coupling.

Figure 10 shows and comprises for 1X（Normal speed）The example of the variant playlist of multiple URL of playback, these Each of URL provides the playlist of the 1X playback for different resolution or quality, and this variant playlist also includes Multiple URL for the playback of speed beyond 1X.

Figure 11 illustrates the method for creating I frame playlist that can execute at server.

Figure 12 is the client device that can be used for processing the method shown in I frame playlist and executable Figure 14 or 15 Block diagram represents.

Figure 13 A shows the example of video file, such as film or order video, has I frame in this video file.

Figure 13 B shows the example of the I frame playlist for the video file shown in Figure 13 A.

Figure 14 is the flow chart illustrating the method example for processing I frame playlist.

Figure 15 is the flow chart illustrating the other method example for processing I frame playlist.

Figure 16 show the video file with I frame example and how can service limits excluding the limit more than I frame The rhythm of degree（cadence）.

Figure 17 A, 17B, 17C and 17D show the example of the different rhythm of different subsets representing available I frame.

Figure 18 is the flow process being shown in the method switch between different resolution and/or the I frame playlist of quality Figure.

Figure 19 illustrates the block diagram of spendable exemplary API architecture in some embodiments of the invention.

Figure 20 shows the one exemplary embodiment of spendable software stack in some embodiments of the invention.

Specific embodiment

In the following description, many details are elaborated.However, not having these details can also implement the present invention Embodiment.In other cases, it is not illustrated in detail known circuit, structure and technology, in order to avoid obscuring to this specification Understand.

This specification includes the material by copyright protection, the such as diagram to graphical user interface image.Copyright institute Someone（Including assignee of the present invention）Hereby retain its right to these materials, including copyright.Copyrighter is not Oppose that anyone is open to patent document or patent to carry out replicating reproduction, as long as it occurs in patent and trademark office file or record In, but retain all copyrights in addition.Copyright Apple Inc.2009.

The one side of this specification is related to ask using bytes range in playlist described here, and this theory The another aspect of bright book is related to provide the fast forward or reverse from I frame in streaming media（Fall back）The method of playback, and right There is the use of the playlist of timing information, timing information allows to select the rhythm of I frame when providing fast forward or reverse.In knot After Fig. 1 9D offer background information is provided, in conjunction with Figure 10 18, these aspects will be described.

In one embodiment, technology described herein and assembly may include using non-flow host-host protocol（Such as HTTP）And its His technology（Such as motion characteristics planning（Motion Picture Expert Group, MPEG）Stream）To realize streaming body The mechanism tested.For example, the streaming of near real-time can be provided to experience using HTTP, thus broadcasting " live " music or physical culture thing Part, live news, IP Camera feeding etc..In one embodiment, agreement incoming media data can be segmented into multiple Media file, and by the media file storage of these segmentations on the server.Agreement also can build play list file, this broadcasting Listing file includes client guides the Uniform Resource Identifier of the segment media file on the server to storage （Uniform Resource Identifiers, URI）.Work as basis（One or more）Play list file is playing back segmentation matchmaker During body file, client can provide a user with the near real-time broadcast of " live " event.Can provide pre-recorded in a similar manner Content.

In one embodiment, the media content that server can will supplement or replace（For example, advertisement and sport event have The additional media content that the statistics of pass, master present）It is dynamically introduced in broadcast event.For example, in client to media During the playback of event, server can add extra URI to play list file, and these URI can identify client and can download The position of supplementing media file.Client can be instructed and periodically fetch one or more updated played column from server List file so as access server introduce any supplement or additional（Or both）Media content.

In one embodiment, server can operate in accumulation mode or rolling mode.In accumulation mode, server Play list file can be created and media file identifier is attached to the end of play list file.Then client can be from list Individual play list file（When downloading）Access all parts of stream（For example, user can be from the beginning of the centre of program）.Rolling In pattern, server can be by removing media file identifier come restricting media with roll mode from the beginning of play list file The availability of file, thus provide the sliding window of the addressable media content of client device.Server also can be to played column Table adds media file identifier, and in rolling mode, the availability of media file can be restricted to recently by server Those to playlist added.Client subsequently repeatedly downloads the updated copy of play list file to continue to see See.The roll mode that playlist is downloaded is when content may be unlimited in time（For example, from the network of continuous operation The content of camera）It is probably useful.Client can continue repeatedly to ask playlist with rolling mode, until it is being play Till finding end-tag in list.

In one embodiment, by providing the same variant stream presenting, this mechanism supports that bit rate switches.For example, Several versions presenting providing can be stored on server.Each version can have essentially identical content, but with not Same bit rate coding.This can allow client device foundation that for example the detection of available bandwidth is come with switching between bit rate, Continuity without infringement playback.

In one embodiment, it is possible to provide protection feature exempts from unwarranted use protecting content.For example, it is possible to make Numbered with non-sequential media file to prevent from predicting.The encryption of media file can be used.Can be using part of media file row Table.Extra and/or different protection features also can be provided.

Fig. 1 be can send and receive in real time or near real-time content an embodiment of server and client side block diagram. The example of Fig. 1 provides simple client-server to connect, and two of which client is via network and Coupled processors.Using Technology described herein and mechanism can support any number of client.In addition, according to technology described herein and mechanism, multiple Server can provide content and/or can operate to provide content together.For example, a server can create content, create and play List and create multiple media（Such as file）, other server storages simultaneously send created content.

Network 110 can be any kind of network, either wired, wireless（For example, IEEE 802.11, 802.16）Or its any combinations.For example, network 100 can be internet or Intranet.As another example, network 110 can To be cellular network（For example, 3G, CDMA）.In one embodiment, client device 150 and 180 can pass through multiple Network type is communicated（For example, each equipment can be communicated by WiFi WLAN, also can pass through wireless cell phone Network is communicated）.For example, client device 150 and 180 can be to pass through cellular radio telephone network and data Smart phone or the personal digital assistant possessing cellular capacity that network is communicated.These equipment can be in any sort Utilize streaming mechanism described herein on the network of type, or even switch between networks as needed.

Server 120 can operate as HTTP server in any manner known in the art.That is, server 120 include providing the HTTP server agency 145 of content using http protocol.Although the example of Fig. 1 is to describe according to HTTP , but other agreements can also be utilized in a similar manner.Sectionaliser 130 and index 135 are resident in server 120（Or Multiple servers）Come up and utilize play list file to provide the agency of the content in media file as described herein.Can be utilized Http protocol acts on behalf of 145 via HTTP server（Or via other servers）By network 110 provide these media files and Play list file.Here the agency discussing can achieve as hardware, software, firmware or a combination thereof.

Sectionaliser 130 can be used for for media data flow being divided into the multiple media files that can transmit via http protocol.Rope Lead device 135 can have following function：Create the play list file corresponding with segment media file so that client device can These media files of re-assemble come to provide the content being provided by server 120 in real time or near real-time transmission.In response to from visitor One or more requests of family end equipment, HTTP server agency 145（Or other servers）Can send and be generated by index 135 One or more play list file and the content being generated by sectionaliser 130 media file.Server 120 may also include Optional security agency 140, the security agency 140 one or more security functions of discussing here of offer（For example encrypt）.Service Device 120 may also include the unshowned additional assemblies of Fig. 1.

Client device 150 and 180 can receive play list file and the media from server 120 by network 110 File.Client device can be can to receive the data transmitting by network and using the data next life receiving via network Become any kind of electronic equipment of output, such as wireless mobile apparatus, PDA, amusement equipment, consumer-elcetronics devices etc..Output Can be the combination of any medium type or medium type, for example, include audio frequency, video or its any combinations.

Client device 150 may include assembler agency 160 and output maker agency 165.Similarly, client device 180 may include assembler agency 190 and output maker agency 195.Assembler agency 160 and 180 receives from server 120 and broadcasts Emplace list file and accessed and downloads of media file from server 120 using play list file.Output maker agency 165 and 195 outputs generating client device 150 and 160 respectively using the media file downloaded.Output can be by one or many Individual loudspeaker, the combination of one or more display screen, loudspeaker and display screen or any other input or output equipment are carrying For.Client device may also include memory（Such as flash memory or DRAM etc.）To serve as the storage matchmaker when media file is received Body file（For example compressed media file or decompressed media file）Buffer（buffer）；Buffer can carry For beyond seconds many beyond the time of content being currently being rendered can presenting content, thus buffer content subsequently can be interior newly Hold shown while being just downloaded.This buffer just can attempted taking by intermittent slow network connection in client device Return provide while content can presenting content, therefore buffer can hide network delay or connectivity problem.

Client device 150 and 180 also can include optional security agency 170 and 185 respectively, and its offer is one or more Here the security function of discussing.Client device 150 and 180 may also include the unshowned additional assemblies of Fig. 1.

In one embodiment, technology described in this application can be used for by non-flow host-host protocol（Such as HTTP）Transmission Unlimited multimedia data stream.Embodiment may also include the encryption of media data and/or the offer of the replacement version of stream（For example, The bit rate substituting is provided）.Because media data can be sent after creation quickly, data can be connect by near real-time Receive.Provide the Example data formats of file and the server of multimedia data stream（Sender）And client（Recipient）Will The action taken；However, extended formatting also can be supported.

Can be used as emulation real-time streams（Or near real-time stream）The media presentation of transmission is by the unified money indicating play list file Source identifier（URI）Specify.In one embodiment, play list file is the ordered list of extra URI.Play list file In each URI be incorporated as the media file of fragment that flows, this stream can be the single continuous of the media data of specific program Stream.

In order to play media data flow, client device obtains play list file from server.Client also obtains simultaneously Play each media data file being indicated by play list file.In one embodiment, client can dynamically or instead Reload play list file again to find extra and/or different media fragments.

Play list file can be for example extension M3U play list file.In one embodiment, using effectively expanding The additional tags of exhibition M3U form.M3U refers to the 3rd layer of URL of motion characteristics planning audio frequency（Moving Picture Experts Group Audio Layer 3 Uniform Resource Locator, MP3URL）, it is for depositing The form of storage multimedia play list.M3U file is the position comprising the one or more media files for media renderer plays The text put.

The extension M3U format text file that play list file is made up of each row in one embodiment.These row May terminate at single line feed character or the CR character being followed by line feed character.Every a line can be URI, blank line or to annotate Character（Such as " # "）Start.URI mark media file to be play.Blank line can be ignored.

The row being started with annotating character can be annotation or label.Label can begin at #EXT, and comment line can start In #.Comment line usual being serviced device and client are ignored.In one embodiment, play list file is to be compiled with UTF-8 form Code.UTF-8（8 bit Unicode transform format）It is variable length character coded format.In an alternate embodiment, it is possible to use Other character encoding formats.

In ensuing example, using the extension M3U form including following two labels：EXTM3U and EXTINF.Expand Exhibition M3U file can be distinguished with basic M3U file by including the first row of " #EXTM3U ".

EXTINF is the recording mark describing the media file by the URI mark followed after this label.In a reality Apply in example, be EXTINF label before each media file URI, for example：

#EXTINF:<duration>,<title>

Wherein " duration " specifies the duration of media file, and " title " is the title of target media file.

In one embodiment, it is possible to use following label is managing transmission and the playback of media file：

EXT-X-TARGETDURATION

EXT-X-MEDIA-SEQUENCE

EXT-X-KEY

EXT-X-PROGRAM-DATE-TIME

EXT-X-ALLOW-CACHE

EXT-X-STREAM-INF

EXT-X-ENDLIST

These labels each of are described more detail above.Although describing specific form for each new label And attribute, but it is possible with different attributes, title, form etc. to support alternative embodiment.

EXT-X-TARGETDURATION label may indicate that and will be added to the substantially lasting of next media file presenting Time.It can be included in play list file, and form can be：

#EXT-X-TARGETDURATION:<seconds>

Wherein " seconds " indicates the duration of media file.In one embodiment, the actual duration can be with this The target time duration of label instruction is slightly different.In one embodiment, each URI of instruction fragment will be big with this fragment The duration is caused to be associated；For example, with regard to a fragment URI can before be embroidered with indicate this fragment the substantially duration label.

Each media file URI in play list file can have unique sequence number.In one embodiment, URI Sequence number（If there is）Sequence number equal to the URI before it adds one.EXT-X-MEDIA-SEQUENCE label may indicate that Occur in the sequence number of the URI in play list file, form can be：

#EXT-X-MEDIA-SEQUENCE:<number>

Wherein " number " is the sequence number of URI.If play list file does not include #EXT-X-MEDIA-SEQUENCE Label, then it is considered that the Serial No. 1 of a URI in playlist.In one embodiment, sequence number can right and wrong Sequential；For example, such as 1,5,7, the non-sequential sequence number of 17 etc is so that be difficult in forecasting sequence Next Serial Number, and this can help protection content to exempt from piracy.Another option helping protect content is at any given time Only disclose a part for playlist.

Some media files can be encrypted.EXT-X-KEY label provides and can be used for media file followed by is carried out The information of deciphering, form can be：

#EXT-X-KEY:METHOD=<method>[,URI=″<URI>″]

METHOD parameter specifies encryption method, URI parameter（If there is）Specify how to obtain key.

The encryption method instruction of NONE is not encrypted.Various encryption methods can be used, such as AES-128, AES-128 refer to Show using Advanced Encryption Standard encryption [referring to RFC3852] with 128 bit keys and PKCS7 filling.New EXT-X-KEY Label replaces any previous EXT-X-KEY label.

There is the EXT-X-KEY tag identifier key file of URI parameter.Key file can comprise for playlist literary composition The key that the subsequent media file listed in part is decrypted.For example, AES-128 encryption method uses the close of 16 eight bytes Key.The form of key file can be the compression array of 16 eight bytes of binary format.

Usually required that using AES-128 provides the initialization vector of 16 eight bytes of identical in encryption and deciphering （IV）.Change IV to can be used for increasing the intensity of password.When being encrypted using AES-128, media file is being encrypted or is solving The sequence number of media file can be used as IV when close.

EXT-X-PROGRAM-DATE-TIME label can be by the beginning of next media file and absolute date and/or time Associated, and may include or indicate time zone.In one embodiment, date/time represents is ISO/IEC 8601:2004. Tag format can be：

EXT-X-PROGRAM-DATE-TIME:<YYYY-MM-DDThh:mm:ssZ>

EXT-X-ALLOW-CACHE label may be used to indicate whether client can cache to supply by the media file of download Play back later.Tag format can be：

EXT-X-ALLOW-CACHE:<YES|NO>

EXT-X-ENDLIST label indicates that more multimedia file can not be added to playlist in one embodiment File.Tag format can be：

EXT-X-ENDLIST

In one embodiment, if playlist comprises final fragment or media file, this playlist will have There is EXT-X-ENDLIST label.

Next URI that EXT-X-STREAM-INF label may be used to indicate in play list file identifies another playlist File.Tag format can be in one embodiment：

EXT-X-STREAM-INF:[attribute=value][,attribute=value]*<URI>Wherein can make In order to properties.Attribute BANDWIDTH=<n>It is the substantially upper limit of the stream bit rates being expressed as BPS.Attribute PROGRAM-ID=<i>It is the specific numbering presenting uniquely identifying in the range of this play list file.Play list file May include and there are multiple EXT-X-STREAM-INF URI of identical PROGRAM-ID to describe the same variant stream presenting.Variant Stream and variant playlist have further description in the disclosure（For example, see Fig. 9 A-9D）.

Aforementioned label and attribute being serviced device equipment be used for organizing, send and process the media representing original media content File.Client device uses the information to provide real-time or near real-time streaming experience to the user of client device（Example As watched the live broadcast of such as music or sport event etc）Mode re-assemble and assume media file.

Each media file URI mark in play list file presents as original（That is, original media content）Piece The media file of section.In one embodiment, each media file be formatted as mpeg 2 transport stream, MPEG-2 program stream or MPEG-2 audio stream.Form can be specified by specifying codec, and playlist can be referred to by specifying codec Fix formula.In one embodiment, one present in all media files all there is identical form；However, in other enforcements Multiple format can be supported in example.Transfer stream file should comprise single MPEG-2 program in one embodiment, and at each File beginning should have Program Association Table and Program Map Table.The file comprising video should have at least one key frame Intactly to initialize Video Decoder with enough information.Client should be ready for by selecting rational subset Particular type（Such as audio or video）A plurality of tracks.Client should ignore in one embodiment in transport stream its not The privately owned stream identifying.For in the stream within a media file and between the respective streams of multiple media files samples Coding parameter should be consistent.However, client should tackle code change when running into code change, its mode is, for example, Change resolution is adapted to by scaling video content.

Fig. 2A is that one or more server apparatus utilize non-flow host-host protocol to support a reality of the technology of media content The flow chart applying example.The example of Fig. 2A to provide according to HTTP；However, it is possible to non-be spread using other in a similar fashion Defeated agreement.The example of Fig. 2A executes some tasks to provide according to individual server.However, it is possible to using any number of Server.For example, providing the server of media file to client device can be to become multiple media files with by content section The different equipment of server.

Server apparatus receive content to be provided in operation 200.Content can represent live audio and/or video（Example As sport event, live news, IP Camera feeding）.Content may also indicate that prerecorded content（For example, the sound of record Happy meeting, training class etc.）.Content being serviced device receive according to any form known in the art and agreement, regardless of whether being stream Transformat.In one embodiment, content being serviced device in the form of MPEG-2 stream receives；However, other lattice also can be supported Formula.

Server subsequently operation 210 in can interim storage content at least some part.Inside perhaps content at least one A little parts can be temporarily stored in such as storage device（Such as hard disk in storage area network etc.）Above or in memory.Or Person, can be via storage medium（For example, compact disk, flash disk）Receive content, content can be sent to storage from this storage medium and set Standby or memory.In one embodiment, server has encoder, and Content Transformation is become one or many by encoder as needed Individual stream（Such as MPEG-2）.This conversion can occur in the case of not permanently storing received content, and real at some Apply in example, the storage that storage operation 210 can be omitted or it can be longer-term in other embodiments（For example filing is deposited Storage）.

In operation 220, content to be provided is segmented into multiple media files.In one embodiment, server will Circulation changes into and separates and different media files（That is, fragment）, these media files can using the web server of standard come point Send out.In one embodiment, server is in the point of the efficient decoding supporting individual media file（For example, in packet and key frame On border, such as PES boundaries of packets and i frame boundaries）Segmentation is carried out to Media Stream.Media file can be that having of primary flow is big Cause the part of equal duration.Server also creates URI for each media file.These URI allow client device to visit Ask media file.

Because fragment is to be provided using the HTTP server inherently delivering whole file, in complete segmentation Before media file is provided to client, server should make it can use.Thus, client can（In time）Delayed wide Broadcast, hysteresis is the length of at least one media file.In one embodiment, media file size be based on lag time and There is balance between too many file.

In one embodiment, two kinds of conversation type are supported（Live session and event session）.For live session, only protect Stay the fixed size part of stream.In one embodiment, out-of-date content media files are by from program play list file Remove, and can be removed from server.Equations of The Second Kind session is event session, and wherein client may be tuned to any the one of broadcast Point（For example, start from the outset, from the beginning of intermediate point）.This kind of session for example can be used for broadcasting again.

In operation 230, media file is stored in server memory.Before storage file in operation 230, can Media file is protected by the security feature such as encrypted etc.Media file is stored as being ready to utilize server apparatus On Web server application supported（Or another equipment being transmitted is supported）Procotol（For example, HTTP or HTTPS）The file sending.

In operation 240, generate one or more play list file and should assemble media literary composition in what order to indicate Part is to rebuild original contents.（One or more）Play list file can using extension M3U label and label described herein Lai Offer information supplies client device access and re-assemble media file to provide streaming to experience on a client device.With regard to every The URI of individual media file is included according to the order of media file to be play（One or more）In play list file.Clothes Business device can also be（One or more）Play list file creates one or more URI to allow client device access to be somebody's turn to do （One or more）Play list file.

In operation 250,（One or more）Play list file can be stored on server.Although media file and （One or more）Being created and stored in of play list file is presented by particular order in Fig. 2A, but also can be using difference Order.For example, it is possible to create before establishment or storage media file（One or more）Play list file.As another Example,（One or more）Play list file and media file can either of which stored before be created.

If media file is to be encrypted,（One or more）Play list file definable URI, this URI allow warp The client device authorizing obtains the key file comprising encryption key media file is decrypted.Available secure connection （Such as HTTPS）To transmit encryption key.As another example, can be transmitted using HTTPS（One or more）Playlist File.As another example, media file can be arranged by uncertain order, so that client is not having（One or many Individual）Can not be flow-reconstituted in the case of play list file.

If encryption method is AES-128, such as AES-128 CBC encryption can be applied to each media file.? In one embodiment, whole file is encrypted.In one embodiment, generally not across media file application CBC.Matchmaker The sequence of body file is used as IV as mentioned above.In one embodiment, server adds tool to the end of play list file There is the EXT-X-KEY label of key URI.Server all subsequent media file are encrypted followed by this key, Zhi Daozuo Till going out the change of encryption configuration.

In order to be switched to new encryption key, server can be via previous key URI all used in presenting with this Different new URI come to make new key can use.Server also adds the EXT- with new key URI to the end of play list file X-KEY label, and using this new key, all follow-up media files are encrypted.

In order to terminate to encrypt, server can add the EXT- with encryption method NONE at the end of play list file X-KEY label.This label（Using " NONE " as method）Do not include URI parameter in one embodiment.As described above, after all Continuous media file is not encrypted, till making the change of encryption configuration.Server does not remove from play list file EXT-X-KEY label, if the URI that this play list file comprises to go to using the media file of this key encryption.? In operation 270, server be may be in response to client request and sent by network（One or more）Play list file and media File, contact Fig. 3 A to be more fully described by this.

In one embodiment, server is in response to receiving the request to play list file from client device And send this play list file to client device.Client device can using have been provided to the URI of this client device Lai Access/request play list file.URI indicates play list file position on the server.As response, server can be by Play list file is supplied to client device.Client device can be using the label in play list file and URI（Or other Identifier）To access multiple media files.

In one embodiment, the availability of media file can be limited to be added to recently by server（One or more） Those of play list file.For this reason, each play list file may include only one EXT-X-MEDIA-SEQUENCE label, And for each media file URI removing from play list file, this value can be incremented by 1.Media file URI can be by it The order being added by from（One or more）Remove in play list file.In one embodiment, when server from（One Or it is multiple）When removing media file URI in play list file, this media file kept to client in a period of time can With the length of this time comes across the longest played column therein equal to the duration of this media file plus this media file The duration of list file.

The duration of play list file is the summation of the duration of the media file in this play list file.? Other duration can be used.In one embodiment, server can maintain at least three masters to present all the time in the play list , unless there is EXT-X-ENDLIST label in media file.

Fig. 2 B is that one or more server apparatus provide the dynamic played column updating to one or more client devices The flow chart of one embodiment of the technology of table.Played column can be updated using accumulation mode described herein or rolling mode Table.The example of Fig. 2 B to provide according to HTTP；However, it is possible to utilize other non-flow host-host protocols in a similar fashion（Example As HTTPS etc.）.The example of Fig. 2 B executes some tasks to provide according to a server.However, it is possible to using any Number destination server.For example, the server providing media file to client device becomes multiple media files with by content section Server can be different equipment.

Server apparatus receive content to be provided in operation 205.Server subsequently can interim storage in operation 215 At least some part of content.Operation 215 can be similar with the operation 210 in Fig. 2A.In operation 225, content quilt to be provided It is segmented into multiple media files.In operation 235, media file can be stored in server memory.Operation 235 is deposited Before storage file, media file can be protected by the security feature such as encrypted etc.

In operation 245, generate one or more play list file and should assemble media literary composition in what order to indicate Part is to rebuild original contents.In operation 255,（One or more）Play list file can be stored on server.Although Media file and（One or more）Being created and stored in of play list file is given by particular order in Fig. 2 B, but Can make in differing order.

In operation 275, server（Or another server）May be in response to client request and sent by network（One Or it is multiple）Play list file and media file, contact Fig. 3 A-3B to be more fully described by this.

Server can update for various reasons（One or more）Play list file.In operation 285, server The excessive data of client device to be supplied to can be received.Excessive data can be stored in operation 255（One or more） Receive after play list file.Excessive data can be for example the extra section that fact presents, or presents with regard to existing Extraneous information.Excessive data may include advertisement or statistics（For example, the score relevant with sport event or data）.Additionally Data can（By translucent）It is coated over and present or present in sidebar user interface.Excessive data can by with original The data identical mode receiving is segmented.If excessive data constitutes the section represented by advertisement or playlist to be inserted into Other guide in mesh, then excessive data can operation 215 in quilt（At least temporarily with）Storage, is segmented simultaneously in operation 225 And stored in operation 235；Before the excessive data of memory segment, the fragment of excessive data can be encrypted.Then in behaviour Make in 245, the playlist being updated over will be generated, it comprises program and excessive data.Playlist is based on excessive data by more Stored in operation 255 newly and again.Right（One or more）The change of play list file is from the angle of client device Should be carried out in the way of atom from the point of view of degree.Updated playlist substitutes previous playlist in one embodiment.As As discussed in further detail below, client device can repeatedly ask playlist.These requests enable client device Using nearest playlist.In one embodiment, excessive data can be metadata；In the case, playlist is not Needs are updated, but fragment can be updated to including metadata.For example, can comprise can be with the timestamp in fragment for metadata The timestamp joined, and metadata can be added to the fragment of the timestamp with coupling.

Updated playlist may also lead to the removal of media file.In one embodiment, server should be according to URI with regard to media file is added to the order of playlist and removes the URI with regard to media file from playlist.One In individual embodiment, if server removes entirely presented, it makes（One or more）Play list file is for client Equipment is unavailable.In one embodiment, server maintain media file and（One or more）Play list file, maintenance Time is comprise media file the longest（One or more）The duration of play list file, to allow current client End equipment completes to the access presenting.Therefore, each media file URI in play list file can before be embroidered with EXT-X- STREAM-INF label is to indicate the substantially cumulative duration of the media file of this play list file instruction.Implement substituting Example in, can remove immediately media file and（One or more）Play list file.

The request to playlist being subsequently derived from client device leads to server offer in operation 275 to be updated over Playlist.In one embodiment, playlist is regularly updated, and for example relevant with the target time duration time is all Phase.The periodically updating of play list file allow for server and provide the access presenting to dynamic change.

Fig. 2 C is that one or more server apparatus utilize multiple bit rates to provide the skill of media content to client device The flow chart of one embodiment of art, this is using a kind of form replacing stream.The example of Fig. 2 C to provide according to HTTP； However, it is possible to utilize other non-flow host-host protocols in a similar fashion.The example of Fig. 2 C is to execute some tasks according to server Come to provide.However, it is possible to utilize any number of server.For example, the server of media file is provided to client device Can be different equipment from the server that content section is become multiple media files.

In one embodiment, server can provide multiple play list file or have in single play list file There is the single play list file of multiple media file tabulars, to provide the same different coding presenting.Provided that it is different Coding, then（One or more）Play list file may include each variant stream providing different bit rates to allow client to set Dynamically switch between coding（Contact Fig. 9 A-9D to be further described by this）.There is the play list file of variant stream EXT-X-STREAM-INF label be may include for each variant stream.Same each EXT-X-STREAM-INF label presenting can There is identical PROGRAM-ID property value.The PROGRAM-ID value that each presents is unique in variant stream.

In one embodiment, constrain below server satisfaction when producing variant stream.Each variant stream can be by identical interior Hold and constitute, including the optional content not being the part that master presents.In the precision of the minimum target duration of stream, clothes Business device can make the content of identical period can be used for all variant streams.The media file of variant stream has in one embodiment The mpeg 2 transport stream of sample time stamp or MPEG-2 program stream, sample time stamp for the corresponding contents in all variant streams is Coupling.In addition, all variant streams should comprise identical audio coding in one embodiment.This allow that client device Switching between variant stream and not lost content.

With reference to Fig. 2 C, server apparatus receive content to be provided in operation 202.Server is subsequently in operation 212 Can at least interim storage content.In operation 222, content to be provided is segmented into multiple media files.In operation 232, For selected bit rate（Or the selected value of other coding parameters）Each media file is encoded and is stored it in On server.For example, media file can connect for high bandwidth, middle bandwidth and low bandwidth.Media file can before storing by Encryption.For various types of connections, media file is carried out with coding and be selectable to provide streaming on target bandwidth level Experience.

In one embodiment, generate in operation 242 and there is the label indicating various code levels as described herein Variant playlist.Label for example may include EXT-X-STREAM-INF label for each code levels, wherein carries and goes URI to respective media play list file.

This variant playlist may include the URI of the media play list file going to various code levels.Thus, client Selection target bit rate and taking among the alternative that end equipment can provide from the variant playlist of instruction code levels Return corresponding play list file.In one embodiment, client device can change during playing back between bit rate（Example As described in contact Fig. 9 A-9D）.Indicate that the variant playlist of various code levels is stored in server in operation 252 On.In operation 242, each playlist being related in variant playlist also can be generated, then quilt in operation 252 Storage.

In response to the request from client device, server can send in operation 272 and indicate various code levels Variant playlist.Server operation 282 in can receive in the media play list that variant playlist is specified with selected The request of the corresponding media play list of bit rate.In response to this request, server sends and from client in operation 292 The corresponding media play list file of request of end equipment.Client device may then use that this media play list comes to clothes Business device request media file.Media file is supplied to client device in response to request in operation 297 by server.

Fig. 3 A is that client device utilizes non-flow host-host protocol to support an embodiment of the technology of streaming of content Flow chart.The example of Fig. 3 A to provide according to HTTP；However, it is possible to utilize other non-streaming associations in a similar fashion View.Method shown in Fig. 3 A-3B can by a client device or several separate client device execution.For example, exist In the case of any one of these methods, single client equipment can perform all operations（For example, request playlist literary composition Part, using in play list file URI request media file, assembling media file present/export to generate and to provide）Or Several different client devices can perform some but not all operations（For example, the first client device can ask to play These media file storage simultaneously using the URI request media file in play list file and can be supplied second by listing file Client device uses, and the second client device can process media file and present/export to generate and to provide）.

Client device in operation 300 can be to server request play list file.In one embodiment, this request It is to be made according to the agreement deferring to HTTP.This request is using going to storage initial play file on the server URI.In alternative embodiments, other non-flow host-host protocols can be supported.In response to this request, server will be by network corresponding Play list file be sent to client.As described above, network can be wired or wireless and can be wired or Any combinations of wireless network.In addition, network can be data network（Such as IEEE 802.11, IEEE 802.16）Or honeycomb Telephone network（Such as 3G）.

Client device can receive play list file in operation 310.Play list file can be deposited in operation 320 Storage is in the memory of client device.Memory can be for example hard disk, flash memory, random access memory.Implement at one In example, every time when playlist URI loads or reloads play list file, client is just checked to determine this broadcasting Listing file starts from #EXTM3U label, if this label does not exist, does not continue.As described above, play list file includes One or more labels and the one or more URI going to media file.

Client device may include assembler agency, and assembler agency is by asking play list file in operation 330 In the media file indicated by URI to carry out re-assemble original contents using play list file.In one embodiment, assemble Device agency is the card module of the part as standard web browsers application.In another embodiment, assembler agency is permissible It is independent application, it is interacted with Web browser to utilize（One or more）Play list file receives and assembles media literary composition Part.As another example, assembler agency can be built-in the specialized hardware in client device or fastener components.

Assembler makes media file from the play list file server indicated by from URI download.As carpostrote Emplace list file and comprise EXT-X-ENDLIST label, then any media file indicated by play list file can be broadcast first Put.Any matchmaker if EXT-X-ENDLIST label does not exist, in addition to last and penultimate media file Body file can be play first.Subsequent media literary composition once have selected the first media file to be play, in play list file The order that part just occurs according to them in play list file in one embodiment is loaded（Otherwise content by out of order is in Existing）.In one embodiment, client device attempted to loading medium file before needing media file（And they are deposited Storage is in a buffer）Compensate with the continual playback of offer and to the temporary variations of network delay and handling capacity.

In operation 340, download（One or more）Media file can be stored in the memory on client device In.Can the memory of storage content can be any kind of memory on client device, such as random access memory, Hard disk or video buffer.Storage can be interim allowing to play back, or can be permanent.If play list file Comprise EXT-X-ALLOW-CACHE label and its value is NO, then client is not deposited after the media file downloaded is played Store up these media files.If playlist comprises EXT-X-ALLOW-CACHE label and its value is YES, client sets Standby can indefinitely storage media file so that later reset.Client device can use EXT-X-PROGRAM-DATE-TIME The value of label is displaying to the user that program initial time.In one embodiment, the multiple media file of client available buffer, so that Obtain it and be less easily subject to network interim card（jitter）Impact to provide more preferable Consumer's Experience.

In one embodiment, if decryption method is AES-128, AES-128 CBC deciphering is applied to each matchmaker Body file.Whole file is decrypted.In one embodiment, not across media file application CBC.The sequence of media file Row number can be used as initialization vector as mentioned above.

In operation 350, from memory, content can be exported from client device.Output or present can be for example through Audio output by built-in loudspeaker or headphone.Output may include via screen output or from client device projection Video.Any kind of output known in the art can be utilized.In operation 351, client device determines in storage Whether there is media file that is any more not yet played or otherwise presenting in current play list.If there is this The media file of sample（And if they are not yet requested）, then process and return to operation 330, one or more in this operation Media file is requested, and this process repeats.Without such media file（That is, all in current play list Media file is all played）, then process proceeds to operation 352, and this operation determines whether play list file includes terminating mark Sign.

If playlist includes end-tag in operation 352（For example, EXT-X-ENDLIST）, then work as playlist When media file indicated by file is played, playback stops.If being not over label, client in the play list Again to server request playlist and return to operation 300 with obtain this program another or be updated over played column Table.

As contacted Fig. 2 B and be discussed in greater detail, the renewable play list file of server is to introduce supplemental content（Example As the additional media file identifier corresponding with the additional media content in live broadcast）Or extra content（For example, in stream Ensuing content）.In order to access supplemental content or extra content, the played column that client can be updated over from server reloading Table.This can provide a kind of mechanism, by this mechanism, even if during the playback of the media content being associated with play list file Also play list file can dynamically be updated.Client can ask the reloading of play list file based on multiple triggers.Knot The shortage of bundle label is a this trigger.

In one embodiment, client reloading（One or more）Play list file, unless Play list file comprises EXT-X-ENDLIST label.When client device first time loaded playlists file or again plus When carrying play list file and finding that this play list file changes from last time is loaded, client is being attempted weighing again A period of time is may wait for before loaded playlists file.This period is referred to as initially minimum reloading and postpones.It is from client Measurement from the time that end starts loaded playlists file.

In one embodiment, initially minimum reloading postpones to be continuing of the last media file in play list file In the target time duration of time or three times less that.The media file duration is specified by EXTINF label.If objective Family end reloads play list file and finds that it is not changed in, then client may wait for a period of time before retrying.One Minimum delay in individual embodiment be in the three times of target time duration or multiple that initially minimum reloading postpones less that Individual.In one embodiment, this multiple was attempted being 0.5 for first time, attempted being 1.5 for second, and for follow-up Trial be 3.0；However, it is possible to use other multiples.

When play list file is loaded or reloads every time, client device just checks that play list file will to determine Next media file loading.First file to be loaded is the media file being selected as broadcasting first as mentioned above.If First media file to be play has been loaded and play list file does not comprise EXT-X-MEDIA-SEQUENCE label, then Client can verify that and comprises at the side-play amount of the URI in the media file originally finding last loading for the current play list file This URI, then stops playback without finding this document.Next media file to be loaded can be in play list file with With the first media file URI after the last URI loading.

If the first file to be play has been loaded and play list file comprises EXT-X-MEDIA-SEQUENCE Label, then next media file to be loaded can be the lowest series with the sequence number more than the last media file loading Number that.If play list file comprises to specify the EXT-X-KEY label of key file URI, client device obtains The key file and media file followed after EXT-X-KEY label being solved using the key in key file Close, till running into another EXT-X-KEY label.

In one embodiment, client device downloads play list file using with previously used identical URI. Thus, if play list file has been made with change, client device can be taken using the play list file being updated over Return media file and provide output based on these media files.

The change of play list file for example be may include delete and go to the URI of media file, add and go to new media literary composition The URI of part, is substituted with the URI going to alternate medium file.When change is made to play list file, renewable one or Multiple labels are to reflect（One or more）Change.For example, if the change to media file leads to play list file institute The change of the duration of the playback of media file indicating, then renewable duration label.

Fig. 3 B is that client device utilizes multiple bit rates to support the stream of an embodiment of the technology of streaming of content Cheng Tu, this is a kind of form replacing stream.The example of Fig. 3 B to provide according to HTTP；However, it is possible in a similar fashion Using other non-flow host-host protocols.

Client device can ask play list file in operation 370.Set as described above, can be utilized and being supplied to client Standby URI is fetching play list file.In one embodiment, play list file includes the row of the variant stream of media file Table to provide identical content with different bit rates；In other words, single play list file is included with regard to each variant stream The URI of media file.Example shown in Fig. 3 B uses this embodiment.In another embodiment, variant stream can be by being provided separately Represent to client, the respective multiple different play list file providing identical content with different bit rates, and variant Playlist can provide the URI for each of different play list file.This allows client device to be based on client End condition is selecting bit rate.

Client device retrieval in operation 375（One or more）Play list file.In operation 380（One Or it is multiple）Play list file can be stored in the memory of client device.In operation 385, client device can be based on Current network connection speed is selecting bit rate to be used.Operation 390 is broadcast using corresponding with selected bits rate Emplace the URI that list file includes to come to server request media file.The media file fetched can be stored in client and set In standby memory.In operation 394, client device utilizes media file to provide output, and client device determines whether Change bit rate.

In one embodiment, client device initially selects minimum Available Bit Rate.While playing media, client End equipment can monitor available bandwidth（For example current network connects bit rate）To determine whether available bandwidth is supported using higher ratio Special rate is playing back.If it is, client device may be selected playout length and accesses by playout length media play list Media file indicated by file.Also contrary situation can be supported.If playback consumes too many bandwidth, client device can Select more low bit rate and access the media file indicated by more low bit rate media play list file.

If client device change for example in response to available bandwidth or in response to user input in operation 394 Change bit rate, then client device may be selected different bit rates in operation 385.In one embodiment, in order to select Select different bit rates, client device can be utilized the play list file corresponding with the new bit rate selecting to include The different lists of URI.In one embodiment, client device can change ratio during accessing the media file in playlist Special rate.

If bit rate does not change in operation 394, whether client device determines to have in current play list appoints The media file do not play what is more not yet retrieved and presents.If there is such media file, then process and return To operation 390, and one or more media file is utilized the URI with regard to these files in playlist to fetch.If There is no such media file（That is, all media files in current play list are all played）, then process proceeds to behaviour Make 396, determine whether playlist includes end-tag in this operation.If terminated including, the playback of program and This process completes；If not including, processing and returning to operation 370, and client device requests reload with regard to this program Playlist, and this process repeats the method shown in Fig. 3 B.

Fig. 4 is the block diagram of an embodiment of server flows agency.It will be understood that, the element of server flows agency 400 can divide Cloth is on several server apparatus.For example, first server equipment may include sectionaliser 430, index 440 and security mechanism 450, but do not include file server 460, and second server equipment may include file server 450, but do not include sectionaliser 430th, index 440 and security mechanism 450.In this example, first server equipment will prepare playlist and media file, But client device will not be sent them to, and one or more second server equipment will receive and optionally stored broadcasting List and media file, and playlist and media file will be sent to client device.Server flows agency 400 includes Realize logic function control with Boot Server stream agency 400 operation control logic device 410, and with Boot Server stream The hardware that the operation of agency 400 is associated.Logic device can be hardware logic electric circuit or software routines or firmware.In a reality Apply in example, server flows agency 400 includes one or more applications 412, these are applied 412 to represent and carry to control logic device 410 Code sequence for instruction and/or program.

Server flows agency 400 includes memory 414, and memory 414 represents memory devices or to memory resource Access, for data storage or instruction.Memory 414 may include server flows and acts on behalf of 400 local memories, and/or Person alternatively includes the memory of the host computer system that server flows agency 400 is resided in.Server flows agency 400 also includes one Individual or multiple interfaces 416, these interfaces 416 represent the entity outside server flows agency 400（Electronics or the mankind）Contact Access interface in server flows agency 400（Input/output interface）.

Server flows agency 400 may also include server flows engine 420, and server flows engine 420 expression makes server Stream agency 400 can provide real-time as described herein or near real-time streaming one or more functions.The example of Fig. 4 provides May include several assemblies in server flows engine 420；However, may also comprise different or extra assembly.Carrying The exemplary components that can relate to during for streaming environment include sectionaliser 430, index 440, security mechanism 450 and file server 460.Each of these assemblies may also include other assemblies to provide other functions.Assembly used herein refers to example Journey, subsystem etc., are either realized with hardware, software, firmware or its certain combination.

430 division of teaching contents to be provided of sectionaliser become media file, and these media files can utilize Web server agreement （Such as HTTP）To transmit as file.For example, division of teaching contents can be become to take the predetermined of predetermined file form by sectionaliser 430 Fixed size data block.

Index 440 can provide one or more play list file, and these play list file provide and go to by segmentation The address of media file or URI that device 430 creates.Index 440 for example can create one or more files, and these files have The ordered list of the identifier corresponding with each file being created by sectionaliser 430.These identifiers can by sectionaliser 430 or Index 440 is creating or to assign.It is right to support that index 440 also can include one or more labels in play list file The access of media file and/or utilization.

Security mechanism 450 can provide security feature（For example encrypt）, such as those discussed above.Web server 460 can There is provided and be supplied to the relevant Web server function of remote client devices with the file that will be stored in host computer system.Web service Device 460 can be supported for example to defer to the agreement of HTTP.

Fig. 5 is the block diagram of an embodiment of client stream agency.It will be understood that, the element of client stream agency can be distributed in On several client devices.For example, the first client device may include assembler 530 and security mechanism 550 and can by through The media file stream of deciphering is supplied to the second client device, and the second client device includes exporting maker 540（But do not include Assembler 530 and security mechanism 550）.In another example, master client device retrieval playlist being supplied to Secondary client device, secondary client device is fetched the media file specified in playlist and is generated output to assume these matchmakers Body file.The control that client stream agency 500 includes realizing the operation to guide client stream agency 500 for the logic function control is patrolled Collect device 510, and the hardware being associated with the operation of guiding client stream agency 500.Logic device can be hardware logic electric circuit Or software routines or firmware.In one embodiment, client stream agency 500 includes one or more applications 512, and these should Represented with 512 provides code sequence or the program of instruction to control logic device 510.

Client stream agency 500 includes memory 514, and memory 514 represents memory devices or to memory resource Access, for data storage and/or instruction.Memory 514 may include client stream and acts on behalf of 500 local memories, and/ Or the memory of the host computer system being resided in including client stream agency 500.Client stream agency 500 also include one or Multiple interfaces 516, these interfaces 516 represent the entity outside client stream agency 500（Electronics or the mankind）It is to and from visitor The access interface of family end stream agency 500（Input/output interface）.

Client stream agency 500 may also include client stream engine 520, and client stream engine 520 expression makes client Stream agency 500 can provide real-time as described herein or near real-time streaming one or more functions.The example of Fig. 5 provides May include several assemblies in client stream engine 520；However, may also comprise different or extra assembly.Carrying The exemplary components that can relate to during for streaming environment include assembler 530, output maker 540 and security mechanism 550.These groups Each of part may also include other assemblies to provide other functions.Used here as when, assembly refers to routine, subsystem etc. Deng either with hardware, software, firmware or its certain combination realization.

Assembler 530 can be using the play list file receiving from server come via Web server agreement（For example HTTP）From server access media file.In one embodiment, assembler 530 can make the URI institute in play list file The media file indicating is downloaded.Assembler 530 can respond to the label that play list file includes.

Output maker 540 can provide the media file receiving as the audio frequency in host computer system or visual output（Or Person's audio frequency and vision）.Output maker 540 for example can make audio frequency be output to one or more loudspeakers and video It is output to display device.Security mechanism 550 can provide security feature, such as those discussed above.

Fig. 6 illustrates an embodiment of the play list file with multiple labels.The example playlist of Fig. 6 includes Given number and orderly label.This simply provides for the purpose of description.Some play list file may include more, less Or the label of various combination, and label can be by the order arrangement different from shown in Fig. 6.

Start the beginning that label 610 may indicate that play list file.In one embodiment, starting label 610 is # EXTM3U label.Duration label 620 may indicate that the duration of playback list.That is, playback list 600 instruction matchmaker The duration of the playback of body file.In one embodiment, duration label 620 is EXT-X-TARGETDURATION mark Sign；It is also possible, however, to use other labels.

Date/time label 625 can provide the date of the content being provided with the media file indicated by playback list 600 The information relevant with the time.In one embodiment, date/time label 625 is EXT-X-PROGRAM-DATE-TIME mark Sign；It is also possible, however, to use other labels.Sequence label 630 may indicate that play list file 600 in the sequence of playlist Sequentially.In one embodiment, sequence label 630 is EXT-X-MEDIA-SEQUENCE label；It is also possible, however, to use other marks Sign.

Security label 640 can provide with indicated by play list file 600 the security of media file application and/or plus Close relevant information.For example, security label 640 may specify for being decrypted to the file specified by media file designator Decruption key.In one embodiment, security label 640 is EXT-X-KEY label；It is also possible, however, to use other labels.Variant List tab 645 may indicate that whether playlist 600 provides variant stream and the information relevant with variant stream（For example, how many Individual, bit rate）.In one embodiment, list of variants label 645 is EXT-X-STREAM-INF label.

Media file designator 650 can provide the information relevant with media file to be play.In one embodiment, matchmaker Body file indicator 650 includes going to the URI of multiple media files to be play.In one embodiment, in playlist 600 The order of URI correspond to and should access and/or play the order of media file.Subsequent playlist designator 660 can provide The relevant information of the one or more play list file to be used with after play list file 600.In an embodiment In, subsequent playlist designator 660 may include to go to and will use after having play the media file of playlist 600 One or more play list file URI.

Memory tag 670 may indicate that whether client device can storage media literary composition after the playback of contents of media files Part and/or how long can store.In one embodiment, memory tag 670 is EXT-X-ALLOW-CACHE label.Terminate mark Sign last play list file whether 680 instruction play list file 600 are for presenting.In one embodiment, End-tag 680 is EXT-X-ENDLIST label.

Following part comprises several example playlist files according to an embodiment.

Simple play list file

#EXTM3U

#EXT-X-TARGETDURATION:10

#EXTINF:5220,

http://media.example.com/entire.ts

#EXT-X-ENDLIST

----------------------------------------------------

Sliding window playlist, using HTTPS

#EXTM3U

#EXT-X-TARGETDURATION:8

#EXT-X-MEDIA-SEQUENCE:2680

#EXTINF:8,

https://priv.example.com/fileSequence2680.ts

#EXTINF:8,

https://priv.example.com/fileSequence2681.ts

#EXTINF:8,

https://priv.example.com/fileSequence2682.ts

--- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- the media of --- ----have encryption The play list file of file

#EXTM3U

#EXT-X-MEDIA-SEQUENCE:7794

#EXT-X-TARGETDURATION:15

#EXT-X-KEY:METHOD=AES-128,URI=″

https://priv.example.com/key.php?r=52″

#EXTINF:15,

http://media.example.com/fileSequence7794.ts

#EXTINF:15,

http://media.example.com/fileSequence7795.ts

#EXTINF:15,

http://media.example.com/fileSequence7796.ts

#EXT-X-KEY:METHOD=AES-128,URI=″

https://priv.example.com/key.php?r=53″

#EXTINF:15,

http://media.example.com/fileSequence7797.ts

-------------------------------------------------------

Variant play list file

#EXTM3U

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1280000

http://example.com/low.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2560000

http://example.com/mid.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=7680000

http://example.com/hi.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=65000,CODECS=″mp4a.40.

5″

http://example.com/audio-only.m3u8

----------------------------------------------------------------

Fig. 7 is the flow chart of an embodiment of the playback technology for assembling stream as described herein.Implement at one In example, the playback that user can control the media file receiving starts, stops, refunding etc..Client device in operation 700 Receive play list file.Fetch the media file indicated by play list file in operation 710.Operation 720 is based on The media file receiving generates output.Receive and output is generated based on media file and can complete as described above.

If control input is detected in operation 730, whether client device can determine that this input in operation 740 Instruction stops.If this input is off, this process terminates and plays back stopping.If this input instruction in operation 750 Refund or advance request, then client device can be based on the media being previously played being still stored in memory in operation 760 File is generating output.If these files are no longer in the buffer, process and return to operation 710 to fetch media file and to repeat This process.In an alternate embodiment, playback can be supported to suspend feature, and suspending feature makes playback suspend, but unlike stopping inputting that Sample terminates to play back.

To further describe the method for changing to another stream from a circulation with reference to Fig. 9 A-9D.One client sets Standby can perform each of these methods, or the operation of each of these methods can be distributed in as described herein multiple On client device；For example, in distribution situation, a client device retrieval variant playlist and two media play List, and these are supplied to another client device, this another client device is fetched specified by two media play lists Media file and two streams being provided in the media file fetched between switch.It will also be understood that in alternative embodiment In, the order of shown operation can be changed, or can have than the more or less of operation shown in these in figures.These Method can select different streams using variant playlist.Retrieval process variant playlist with true in operation 901 Fixed available program（Such as sport event）Stream.Operation 901 can be completed by client device.Operation 903 can be broadcast from variant Emplace and select first-class in table, and the subsequent retrieval of client device is with regard to first media play list.Client sets Standby can process in operation 905 with regard to first media play list and also measurement or otherwise in operation 907 Determine the bit rate for first network connection.It will be understood that, the sequence of operation can be by the order different from shown in Fig. 9 A Execution；For example, operation 907 can execute during operation 903, etc..In operation 911, client device is based on from operation 907 bit rate measuring selects to replace media play list from variant playlist；This replaces media play list can It is in second bit rate higher than first existing bit rate.This often means that replacement stream will have higher than first-class Resolution ratio.If based on conditions present（The bit rate of measurement for example in operation 907）Replace media play list with regard to It is more preferable coupling that first current play list is compared, then may be selected to replace media play list.In operation 913, take Return and process the replacement media play list with regard to replacing stream.This often means that client device receiving and may process the First-class and replacement flows both, and therefore both can be used in presenting；One is presented, and another is ready to be presented simultaneously.Client End equipment subsequently select in operation 915 transition point and stopping of switching between the version of stream present first-class and start be in Now replace stream.The example how realizing this switching to be provided in conjunction with Fig. 9 B-9D.In certain embodiments, client device can be Stop reception first-class before making switching.

Fig. 9 B shows that client device is fetched, stores and presented by the first media play list in operation 921 and 923 The content specified（For example first-class）, and while the content specified in the first playlist is just presented, client device Operation 925 is also fetched and stores the content specified by the second media play list（Such as second）.Presenting from first Fetching and storing to the content specified by the second media play list while the content that media play list obtains（For example deposit Storage is in temporary buffer）Create programme content overlap 955 in time（Shown in Fig. 9 D）, this overlap permission client End equipment switches between the version of program, without there being the substantial interruption of program.So, can achieve section in many cases Switching between mesh version, and user will not notice that and there occurs switching（Although user can after the handover in some cases Notice the image of higher resolution）Or presenting of program does not have substantial interruption.In operation 927, client sets The standby transition point determining that the content specified from the first media play list is switched to the content that the second media play list is specified；Turn The example of height（Transition point 959）Shown in Fig. 9 D.Subsequently assume the second media play row after the handover in operation 931 The content that table is specified.

Method shown in Fig. 9 C and 9D represents an embodiment determining transition point；This embodiment depends on from two The pattern match of the audio sample of stream 951 and 953 is determining transition point.It will be understood that, alternative embodiment can use video sample Pattern match or transition point can be determined using the timestamp etc. in two stream.The method may include will in operation 941 Content specified by first media play list（For example flow 951）Storage is in a buffer；What this buffer can be used for content is in Existing, and can be additionally used in pattern matching operation.Stream 951 includes audio sample 951A and video sample 951B.Video sample Can be using the compress technique depending on i frame or key frame, i frame or key frame have being necessary needed for the single frame of video of display Content.Content in stream 951 may include the specified time（Elapsed time for example from program starts）Timestamp, and these Timestamp can mark the beginning of each sample（For example, starting of each audio sample 951A and opening of each video sample 951B Begin）.In some cases, the comparison of the timestamp between two streams is otiose possibly for determination transition point, because it May be insufficient to accurately or because the borderline difference of the sample in two streams；However, the comparison to timestamp scope Can be used for verifying the overlap 955 in existence time between two stream.In operation 943, client device is deposited in a buffer Store up the content specified by the second media play list；This content is with regard to same with the content obtaining from the first media play list Program and it may also comprise timestamp.In one embodiment, if timestamp does not exist in stream, can be added to Playlist with regard to stream；For example, in one embodiment, the ID3 label including one or more timestamps can be added to Entry in the such as playlist of variant playlist or media play list etc.This entry for example can be with regard to audio frequency In the URI of first sample of stream.Fig. 9 D shows the example of the content 953 obtaining from the second media play list, and its bag Include audio sample 953A and video sample 953B.In operation 945, client device can be to the audio frequency in two stream 951 and 953 To select transition point 959 from overlapping 955, transition point 959 can be coupling to sample execution pattern coupling in one embodiment Audio fragment（Such as fragment 957）The independent frame of video of the next one afterwards（Such as i frame 961）.From i frame 961（And its association Audio sample）Start, presenting of program uses the second obtaining from the second media play list.Preceding method is implemented at one Can be not only used for from changing it can also be used to from relatively near the change of slower bit rate to very fast bit rate more slowly in example, but another In one embodiment, the method can be only used for from more slowly to the change of very fast bit rate, and other method（For example, do not attempt positioning Transition point, and it is an attempt to store as quickly as possible and present the content from slower bit rate stream）Can be used for from relatively near slower The change of bit rate.

Following the relating in part in playlist described herein of this specification is asked using bytes range.In addition, this This part of specification describes to provide the fast forward or reverse from I frame in streaming media（Fall back）The method of playback, with And the use to the playlist with timing information, timing information permission is in the fast forward or reverse playback of offer streaming media When select I frame rhythm.Available bytes range request fetches I frame from server.

In the playlist playing back for 1X or in specifying the playlist for the I frame of fast forward or reverse, make Allow for supplier with bytes range request and define multiple logic segments in single big transport stream or elementary audio stream file.This The sum of the file that must manage at server or other guide distribution network can be reduced, and provide pass to caching server More relationship information in set of segments.Have multiple fragments and have using bytes range request fetch data from file Ability media file（Such as film）Example illustrate in figure 13a.In one embodiment, CBC （Cipher block chaining, CBC）Can often keep to fragment so that CBC will be in segment boundaries with password filling（Example The segment boundaries between seg1.ts and seg2.ts as shown in FIG. 13A）Restart.In one embodiment, in Figure 13 A Shown big file is substantially the concatenation of all independent fragment files.

It is as follows using the playlist syntax example of bytes range request in one embodiment.Bytes range label It is added to each fragment indicator and there is following form：#EXT-X-BYTERANGE:<length>[@<offset>], Wherein<length>It is the byte number in fragment, optional parameter<offset>It is the side-play amount from file beginning.If Omit<offset>, then document misregistration amount be equal to previous entries side-play amount add its length.For the first label it is assumed that< offset>For 0.In one embodiment it is intended that the example of the playlist of bytes range fragment is：

#EXTINF:10,

#EXT-X-BYTERANGE:2390240@0

entire_movie.ts

#EXTINF:10,

#EXT-X-BYTERANGE:670416

entire_movie.ts

#EXTINF:10,

#EXT-X-BYTERANGE:465120

entire_movie.ts

Can be with normal（1X）Speed provides the conventional playlist of playback can ask come from single big using bytes range Transport stream or elementary audio stream file fetch content.In addition, can be used for providing trickplay modes playback（Such as fast forward or reverse Playback（Such as 8X or 16X or 32X or -8X or -16X or -32X））Only contain I frame playlist also can use bytes range Request fetches content from single big transport stream or basic stream file.

In one embodiment, variant playlist may include URL and the pass playing back playlist with regard at least one 1X URL at least one I frame playlist；In typical implementation, variant playlist can have multiple to be broadcast with regard to 1X Emplace the URL of table, each has different bandwidth or other changes, for example different codec etc., and variant are play List also can comprise multiple with regard to have different bandwidth or other change（For example different codec etc.）I frame play The URL of list.Figure 10 shows the example of variant playlist 1001, and variant playlist 1001 had both comprised to return with regard to traditional 1X Put the URL of playlist, also comprise multiple URL with regard to I frame playlist, these I frame playlists are only in the case Playlist containing I frame.Specifically, URL1003 is the URL playing back playlist with regard to the 1X under bandwidth X, and URL1005 is to close 1X under bandwidth Y plays back the URL of playlist.URL1007 is with regard to the URL only containing the playlist of I frame under bandwidth X, URL1009 is with regard to the URL only containing the playlist of I frame under bandwidth Y.This whole play list file 1001 is in response to There is provided from the request of the viewing certain movie of client or video request program or other guide, and in response to this request, take Business device provides the play list file 1001 comprising different two group URL, thus allowing 1X playback or fast forward or reverse playback. This allow that client device receives variant playlist and stores this variant playlist, then can be by optionally selecting The suitable play list file in variant playlist is selected to switch between 1X playback or fast forward or reverse playback.Generally, I Frame playlist is by using the bytes range feature discussed here.Bytes range feature is provided with regard to the I frame position in fragment Information, this allow client device only by obtain and show I frame present in video request program and fact in promptly backward or Scan forward, and it does not require special media file to come for different playback speeds.In one embodiment, client Equipment adaptively can be switched when showing I frame to be provided based on current network speed between the variant of I frame playlist Good Consumer's Experience；In other words, network speed can be dynamically determined while executing fast forward or reverse playback, and client Adapt between the variant of I frame playlist that the network speed that system may be in response to be dynamically determined comprises in variant playlist Property ground switching.The example of the variant playlist comprising I frame playlist is：

#EXTM3U

#EXT-X-VERSION:4

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=326448

BTBW0.2Mb/prog_index.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1255635

BTBW0.8Mb/prog_index.m3u8

#EXT-X-I-FRAME-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=75915

BTBW0.2Mb/iframes.m3u8

#EXT-X-I-FRAME-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=380752

BTBW0.8Mb/iframes.m3u8

#EXT-X-I-FRAME-STREAM-INF label has and one group of attribute of #EXT-X-STREAM-INF identical （PROGRAM-ID、BANDWIDTH、CODECS）.Can be seen that variant playlist and provide from this variant playlist and be used for Two versions of the 1X playback under different bandwidth, and two versions of the also I frame playlist under two different bandwidths.? In one embodiment, the bytes range request of the I frame that I frame playlist comprises to be only used in media file, therefore B frame and P frame In these bytes range request specified not included in the playlist only containing I frame.

The example of playlist only containing I frame is（Note EXT-X-I-FRAMES-ONLY label）：

#EXTM3U

#EXT-X-TARGETDURATION:10

#EXT-X-VERSION:4

#EXT-X-MEDIA-SEQUENCE:0

#EXT-X-I-FRAMES-ONLY

#EXTINF:1.2489,

#EXT-X-BYTERANGE:1505@376

segment0.ts

#EXTINF:4.9633,

#EXT-X-BYTERANGE:1505@125584

segment0.ts

#EXTINF:5.005,

#EXT-X-BYTERANGE:37413@539184

segment0.ts

#EXTINF:5.005,

#EXT-X-BYTERANGE:30645@152844

segment1.ts

Note, bytes range only includes I frame；In order to read PAT/PMT, client by from the beginning of a URL non- The side-play amount of an I frame is read in continuous domain.In this embodiment, bytes range request is adjacent in its corresponding URL Specify in label, but be not a part of this URL, but in another embodiment, bytes range request can be this URL A part.Can be seen that new label from this I frame playlist and be added to conventional playlist it is indicated that fragment is specified Symbol is related to I frame, and the EXTINF duration is related to the time span between this I frame and next I frame.To contact in Figure 14 and 15 Shown method is describing the use to this temporal information.

Figure 11 is to illustrate to be used in one embodiment creating the variant playlist shown in Figure 10 or comprising to I frame The flow chart of the method for other variant playlists quoted of playlist.In operation 1101, data handling system determines I frame in video content.I frame in file or transport stream can be determined using known technology.In operation 1103, data Processing system can create I frame playlist, for example, only contain the playlist of I frame, additionally, subsequent establishment comprises to play for I frame List and the variant playlist of the URL optionally for 1X playlist.Except variant playlist and I frame playlist In addition, data handling system also can create 1X playlist.I frame playlist may include multiple with regard to multiple 1X playlists URL, and may also include the multiple URL with regard to multiple I frame playlists accordingly.The example of this variant playlist is in figure Shown in 10.In response to the request to viewing program from client device, at this data handling system or other data Reason system can send the change comprising the URL with regard to one or more 1X playlists with regard to one or more I frame playlists Body playlist.The transmission of variant playlist is illustrated as operating 1105 in fig. 11.Then, same server system or another Server system waits the client request to I frame playlist or 1X playlist.In operation 1107 with operation 1109 In, send server and determine that client requests I frame playlist or 1X played column based on the request receiving from client Table, and as response, send server operation 1113 in send I frame playlist or operation 1111 in send 1X broadcast Emplace table.

Figure 12 shows showing of the architecture of the client device of any one method shown in achievable Figure 14 and 15 Example.Assembly shown in client device 1201 can be component software（In data handling system（System for example shown in Fig. 8） Upper operation）Or nextport hardware component NextPort or the combination of software and hardware assembly.In one embodiment, user application 1207 be by with Family is run and to be shown film or baseball game or the software application listening program of radio station etc..For example, in one embodiment, user Application can be to provide to watch the application of baseball game by Zhi Bang Major Leagues of the U.S..Application 1207 can be with player module 1209 Interaction, player module 1209 can also be in data handling system（System for example shown in Fig. 8）The software journey of upper operation Sequence.User's application 1207 to interact with player module 1209 in an embodiment by the API further describing here.Class As, player module 1209 can be interacted with downloading data rate measurement module 1211 by API.Player module 1209 exists Playlist can be processed so that instruction download module downloads content from the URL being supplied to download module in one embodiment.Download Module 1211 can execute down operations by one or more network interfaces 1213, these network interfaces 1213 be coupled to one or Multiple networks, such as network 1203, and network 1203 is coupled to one or more servers 1205, for example content server or Server of playlist and content etc. is provided.Download data rate measurement module 1211 and also fetch playlist middle finger Execution data rate measurement during fixed content.Data rate measurement can be wanted for determining in the method with reference to Figure 14 and 15 description The suitable rhythm of the I frame downloaded is to provide fast forward or reverse（Fall back）Playback, or also referred to as special play-back playback mode.

Referring now to Figure 13 A and 13B, the relation between media file I associated with it frame playlist is described.To see Go out, I frame playlist includes the metadata with regard to media file, it allows the method shown in execution Figure 14 and 15 to select The subset of I frame or rhythm play back providing fast forward or reverse（Such as 8X or 16X or 32X playback or -8X or -16X or -32X Refund playback）.Media file 1301 can be film or video request program or live broadcast or according to as is generally known in the art HTTP fact flow host-host protocol create alternative document.As it is known in the art, media file 1301 includes multiple I frames.I Frame 1305 includes I frame I₀、I₁、I₂、I₃、I₄、I₅And I₆.As it is known in the art, each I frame is followed by B frame and P frame, B Frame and P frame need I frame that the content in B frame and P frame is decoded.As can be seen from Figure 13A, due to the motion in video originally Matter, I frame is not equally distributed, as known in the art in whole file in time.In media file 1301 Metadata may include piece segment information 1303, it specifies temporal different fragments, and these fragments are used for CBC and close Code filling, as known in the art.In one embodiment, CBC will restart in segment boundaries. Media file 1301 is substantially the concatenation of all individuality clip file.Media file 1301 may include time metadata 1307, Time metadata 1307 represents the time span between the beginning of each a sequence of I frame.For example, I frame I₁Beginning at time The meaning of 7.3 seconds is I frame I₀And its B the and P frame of association occupies the duration of 7.3 seconds in time.Under I frame in Figure 13 A Time metadata 1307 shown in face is the value of time；For example, display in the 26th second of the normal 1X playback duration in film for the I frame 6. Similarly, when film is play with 1X playback rate, I frame 4 presenting for the 16.2nd second in movie time.In one embodiment, The time metadata that I frame in Figure 13 A is illustrated below can be included in media file 1301, and it can in other embodiments Draw from media file, and do not store together with media file, but time metadata will be used in I frame as described below In playlist, such as in the I frame playlist shown in Figure 13 B, and this time metadata can be used on Figure 14 or 15 side In method.

I frame playlist shown in Figure 13 B represents the I frame played column with regard to the media file 1301 shown in Figure 13 A A part for table.This playlist includes the only indicator of frame containing I 1315, and it indicates that this playlist is only with respect to I frame.Mark Sign #EXTINF:7.3 is I frame I₀Span during；In other words, 1317 expression I frame I during span₀Temporal span and Coupling is in I frame I₁Beginning at illustrate 7.3.Bytes range request 1319 includes specifying bytes range A shown in Figure 13 A Byte range parameter；In other words, the byte range parameter in bytes range request 1319 specifies the scope of data to come from media literary composition Part 1301 only fetches I frame I₀.The specific fragment that fragment indicator 1321 and 1327 specified bytes are located at.Bytes range is asked During 1323 also include span, it is 2.5 in the case, the difference between 9.8 seconds and 7.3 seconds shown in this and Figure 13 A Coupling.Specifically, 2.5 seconds specified I frame I₁Beginning and I frame I₂Beginning between duration.Bytes range request 1325 bag Include byte range parameter, bytes range B shown in its specified Figure 13 A is to fetch I frame I₁Video content.It will be appreciated that, In certain embodiments, voice data in the range of identical bytes can with video data multiplex, but this voice data generally without In fast forward or reverse playback.During span, 1317 and 1323 may be used to determine in one embodiment and can be used for downloading different sections The time quantum of the I frame played, this further describes hereinafter in connection with Figure 14 or Figure 15.

Figure 14 shows the method for determining I frame set from available all available I frames before the display time limit.Figure Method shown in 14 is in client device（Such as client device 1201）Upper execution, this client device has been received by I frame playlist, such as I frame playlist 13B, and it is in this I frame playlist of process to play by using I frame During list is to provide fast forward or reverse playback.Can be so that I frame exists from all available I frames before the display time limit On time, at least equally distributed mode selects to gather to a certain extent.In operation 1401, client device determines that I frame is play Multiple set of the available I frame in list；These set are based on current movie time, selected playback mode（Such as 8X or 16X Or -8X etc.）And the display time limit.In one embodiment, the display time limit can be added based on the real-time clock of current time Upper regular time value, such as half second or one second or two seconds from the current real-time clock time.Player time or movie time Can be drawn from this display time limit based on playback rate；For example, if playback rate is 16X, can be by the when span of current I frame 16 seconds are added to provide the player time under this display time limit during degree.For example, if playback rate is 8X and currently shows The I frame showing is the I shown in Figure 13 A₀, then 7.3 seconds can be added 8 seconds（Corresponding with 8X playback rate）To draw this display phase The player time under limit（Or movie time）.Setting on player chronergy under the display time limit can before the display time limit The maximum number of the I frame fetched；This maximum number of I frame be currently displaying I frame with the display time limit before finally complete I frame it Between I frame number.Different playback rates will produce the different display time limits, as shown in Figure 16.Figure 16 shows tool There is the example of at least media file of 17 I frames, this 17 I frames start from I frame I in time₀, and end at I frame I₁₇, such as Shown in Figure 16.Currently displaying I frame is I frame I₀And the time advances from left to right.If playback rate is set to 16X, Then show the time limit in I frame I₁₀Or in I frame I₁₀Before, and if playback rate is set to 32X, then show that the time limit is set In I frame I₁₇Or in I frame I₁₇Before.Therefore, in the case of Figure 16, if playback rate is 16X, before the display time limit The maximum number of retrievable I frame is 9 I frames（I frame I₁–I₉）, and can be used for download when playback rate is set to 32X The maximum number of frame is 16 I frames（I frame I₁–I₁₆）.Therefore, the operation 1401 in operation 14 determines in the given display time limit and works as Front selected playback rate and in the case of the current movie time Downloadable I frame maximum number, then and then determine these The subset of available I frame.In one embodiment, these subsets can be temporal every an I frame or temporal every Every two I frames etc..

Then, in operation 1403, client device is that each set of available I frame determines download time.Generally, can use Each set of I frame will have different number of I frame, and therefore its download time will be different.For example, client device can be true Fix the download time every an I frame carrying in available the maximum number of I frame, and determine available before the display time limit The download time every two I frames in the maximum number of I frame downloaded, etc..Then, in operation 1405, client Equipment choice one of set of Downloadable available I frame before the display time limit, then in operation 1407, under client device Carry selected available I frame collection merging and show them.In one embodiment, client device will download each I one at a time The process operation of frame and the repetition of the actual download time based on last I frame 1401,1403 and 1405.It will be understood that, in F.F. or During refunding playback, user may wish to stop playback or can change the speed of playback, and this is represented as operating 1409, at this In operation, system determines whether content terminates or whether be stopped or whether user changes playback speed, etc..Client End equipment will be responded based on user input.

In one embodiment, the method for the present invention constantly can be commented while assuming I frame with fast forward or reverse modes Estimate download time, and client device can automatically determine whether to change resolution ratio or matter in the case of not having user intervention Amount.For example, if user have selected 8X and network speed is very fast, I frame is sufficiently rapidly downloaded so that having nothing to do Can be used for downloading the I frame of higher resolution, then system can be cut after determining change resolution ratio or quality in operation 1411 automatically Change to different I frame playlists, then switch in operation 1413, client device changes to difference in this operation I frame playlist, this I frame playlist has higher resolution ratio or better image quality in the case.Figure 18 shows Go out the example of the method for operation 1801,1803 and 1805 including switching between I frame playlist.Implement at one In example, client device can require user to confirm resolution ratio or quality to user's present user interface before being changed Change.The change of resolution ratio or quality can be the raising of resolution ratio or quality when the network bandwidth is higher, or in Netowrk tape It can be the change to more low resolution or quality when wide poor or relatively low.Method shown in Figure 14 is worked on 1415, determine the next current movie time in this operation, and determine the next display time limit, then this process is passed through back Repeat to operation 1401.

Figure 15 shows and processes I frame playlist to provide relevant more thin of method of fast forward or reverse playback Section.It will be understood that, the method for Figure 15 may include user stop playback or determine to change playback rate or system change resolution ratio or The operation of quality；For example, any time execution in the method that operation 1409 and 1411 also can be shown in fig .15.In addition, will Understand, the order of operation in an alternate embodiment can be different from the order shown in Figure 15.The method of Figure 15 supposes client device Download I frame playlist, for example, played back in variant（For example, the playlist shown in Figure 10）In the I frame specified play List.It will also be apparent that the method for Figure 15 can be realized using the client device shown in Figure 12, this client device includes downloading Measurement module, the downloading rate of this module measurement data, such as with specific I frame（The last I frame for example downloaded）Corresponding The data of bytes range.In operation 1501, client device determines the current movie time of the currently displaying beginning of I frame, This current movie time can be designated as T_C.In the example depicted in fig. 10, I frame I₀It is the current I frame showing, client sets Standby will determine I frame I₀Beginning the current movie time.Then, in operation 1503, client device determines current playback mould Formula, it can be multiple different fast forward or reverse speed（Such as 8X or 16X or -8X etc.）Any one of.Then exist In operation 1505, client device determines the currently displaying time limit.In one embodiment, this can pass through current playback rate Multiple（Such as 1 times or 2 times）With the execution of current movie time phase Calais.For example, if this multiple is one second and current playback Speed is 8X, then the product being multiplied by for a second 8 seconds produce 8 seconds and this product by with current movie time T₀It is added current to produce The display time limit.This currently displaying time limit and can set border in movie time, and this border determined in the currently displaying phase The maximum number of available I frame before limit.This operation is shown as operating 1507 in fig .15.In example shown in Figure 16, if Playback rate is 16X, then client device will determine available all I frames pair before the currently displaying time limit in operation 1507 Should I frame I shown in Figure 16₁To I₉.Then, in operation 1509, client device determines the institute determining in operation 1507 There is the subset of these I frames.In other words, client device by determine before the currently displaying time limit those I frames available various Subset.In one embodiment, each subset in these subsets can have different rhythm, and each rhythm can have not Same I frame number.In one embodiment, rhythm may be designed such that I frame is uniformly distributed in time, or at least can It is uniformly distributed in sequence with I frame（And be not necessarily completely homogeneously distributed in time）.

Figure 17 A, 17B, 17C show the example of four different rhythm with 17D.Figure 17 A is expressed as follows rhythm：In operation Each of I frame in the set of all I frames determining in 1507 is all considered for downloading.Rhythm shown in Figure 17 B It is expressed as follows rhythm（It is the subset of I frame in the case）：Operation 1507 determines in the I frame that can be used for downloading every one Individual be considered for download.Similarly, the rhythm shown in Figure 17 C is every two I frames, and the rhythm shown in Figure 17 D is Operation 1507 in determine available I frame group in every three I frames.In one embodiment, as in operation 1511 Shown, client device can alternatively by the number of the I frame in each subset be restricted to fixation or predetermined number or It is restricted to the number that can be changed by user or system.Restriction to I frame number can improve in certain embodiments by this way Consumer's Experience.Then, in operation 1513, client device is that each subset of I frame determines download time, in one embodiment Each subset has different rhythm, the such as different rhythm shown in Figure 17 A to 17D.Under each rhythm in rhythm set The load time can be from last two or three or four of the download time of last the I frame being currently displaying or download Or more I frames the rolling average exponentially decaying in time or download last two or three or more The smooth of I frame averagely to draw.Measurement to download time can the module 1211 shown in Figure 12 execute, and these downloads Time is provided to player module 1209, player module 1209 one of executable subset to I frame in operation 1515 Selection.Client device will determine whether each rhythm can be downloaded before the display time limit in operation 1515, and For those can not, as possible solution, it will abandon these rhythm, and by select there is highest I frame number Mesh and the rhythm that can download within the time period before the currently displaying time limit.Equally, in one embodiment, selected rhythm The maximum number of I frame in a period of time, not more than 10 I frames for example per second can be limited in operation 1511, this can improve The visual experience of special play-back.After have selected particular cadence in operation 1515, client device can start to download selected subset In an I frame.In one embodiment, client device can alternatively first check whether too late so that it cannot under Carry this I frame, if it is, jumping to next the I frame in the rhythm selecting in operation 1515.In one embodiment, player mould Selection in block 1209 execution operation 1515, then request download module 1211 download the first I frame in operation 1517.Download mould Block subsequently can measure downloading rate or the download time of this I frame, and this information subsequently can be used on the method for Figure 15 next time In repetition, when this repeats to occur to followed by operation 1501 after operation 1517 next time.If network condition changes（Example As the network bandwidth declines so that download time becomes longer）, then can hold again before all I frames in rhythm selected by downloading The method of row Figure 15, therefore completes rhythm alterable before the download of I frame in the rhythm finally selecting in client device.As Really this continues to occur, then client device can determine to change the resolution ratio of I frame as shown in figure 18 or quality slows down to be adapted to Network.Or, if network speed improves, the method for Figure 18 can also be used for changing resolution ratio or the quality of I frame.As above institute State, the method for Figure 18 can automatically be executed at any time by system or execute in response to user's request in some cases.

Fig. 8 is the block diagram of an embodiment of electronic system.Electronic system shown in Fig. 8 is intended to indicate that the electricity of a certain scope Subsystem（Wired or wireless）, for example, include desktop computer system, laptop system, cell phone, individual Personal digital assistant（PDA）（Including the PDA possessing cellular capacity）, Set Top Box, entertainment systems or other consumer-elcetronics devices.Alternative Electronic system may include more, less and/or different assemblies.The electronic system of Fig. 8 can be used for providing client to set Standby and/or server apparatus.

Electronic system 800 includes bus 805 or other communication equipments carry out transmission information, and includes being coupled to bus 805 Can processing information processor 810.Although electronic system 800 is shown to have single processor, electronic system 800 also may be used Including multiple processors and/or coprocessor.Electronic system 800 may also include random access memory（RAM）Or other are dynamic Storage facilities 820（Referred to as main storage）, it is coupled to bus 805, and can store the instruction that can be executed by processor 810 and Information.Main storage 820 can be additionally used in storage temporary variable or other average informations during processor 810 execute instruction.

Electronic system 800 may also include the read-only storage being coupled to bus 805（ROM）And/or other static storage set Standby 830, it can be processor 810 storage static information and instruction.Data Holding Equipment 840 can be coupled to bus 805 to store Information and instruction.The Data Holding Equipment 840 of such as flash memory or disk or CD and corresponding driver etc can be coupled to electricity Subsystem 800.

Electronic system 800 also can be coupled to display device 850, such as cathode-ray tube via bus 805（CRT）Or liquid crystal Display（LCD）, to display to the user that information.Electronic system 800 may also include character input device 860, and it includes alphabetical number Keyboard and other keys, can be coupled to bus 805 with to processor 810 transmission information and command selection.Another kind of user input sets Standby is cursor control device 870, and such as touch pad, mouse, trace ball or cursor direction key, with to processor 810 side of transmission To information and command selection and control the cursor on display 850 to move.

Electronic system 800 may also include one or more network interfaces 880 to provide to network（Such as LAN）Connect Enter.Network interface 880 for example may include the radio network interface with antenna 885, and antenna 885 can represent one or more skies Line.Electronic system 800 may include multiple radio network interfaces, the such as combination of WiFi, bluetooth and cellular telephone interface.Network connects Mouth 880 may also include such as wired network interface, and to communicate with remote equipment via network cable 887, network cable 887 is for example Can be Ethernet cable, coaxial cable, optical cable, serial cable or parallel cable.

In one embodiment, network interface 880 can be for example by meeting IEEE802.11b and/or IEEE 802.11g Standard providing the access of local area network, and/or radio network interface can for example by meet bluetooth standard provide right The access of personal area network.Also other radio network interfaces and/or agreement can be supported.

In addition to the communication via Wireless LAN standard, or as the replacement to it, network interface 880 can such as profit Use time division multiple acess（TDMA）Agreement, global system for mobile communications（GSM）Agreement, CDMA（CDMA）Agreement and/or any its The wireless communication protocol of his type is providing radio communication.

One or more API can be used in certain embodiments（API）.API be by program code components or Nextport hardware component NextPort（Hereinafter referred to as " API realizes assembly "）The interface realized, it allows another different program code components or hardware Assembly（Hereinafter referred to as " API Calls assembly "）Access and using by API implement assembly provide one or more functions, method, Process, data structure, class and/or other services.API definable implements one of transmission between assembly in API Calls assembly and API Individual or multiple parameters.

API allows the developer of API Calls assembly（It can be third party developer）There is provided using assembly is implemented by API Specific characteristic.Can there is an API Calls assembly, or can have more than one such assembly.API can be source code Interface, computer system or program library provide this source code interface to support the request to service of self-application.Operation system System（OS）Can have multiple API to allow one or more of these API of application call of operation on OS, and service （Such as program library）Can have multiple API to allow one or more of these API of application call using this service.API Can be specified according to the programming language that can be explained when building application or compile.

In certain embodiments, API implements assembly can provide more than one API, and wherein each API provides and implemented by API The different views of the function that assembly is realized, or there is the difference accessing the different aspect being implemented the function that assembly is realized by API Aspect.For example, an API of API enforcement assembly can provide first group of function and can be exposed to third party developer, and Another API that API implements assembly can be hidden（It is not exposed）And the subset of first group of function is provided, and also provides another Group function, the such as not test in first group of function or debugging function.In other embodiments, API enforcement assembly itself can Via the one or more other assemblies of lower floor's API Calls, thus be API Calls assembly be also API implement assembly.

Language and the ginseng using during the specific characteristic of assembly implemented by API definition API Calls assembly accessing with using API Number.For example, API Calls assembly by one or more API Calls of being exposed by API or is transferred（For example adjusted by function or method With realizing）Implement the specific characteristic of assembly to access API, and via these API Calls or transfer and transmit number using parameter According to and control information.In response to the API Calls from API Calls assembly, API implements assembly can be by API come return value.Although The grammer of API definition API Calls and result（For example, how to transfer API Calls and API Calls do anything）, but API can not take off How dew API Calls complete the function specified by API Calls.Via one or more API in called side（API adjusts Use assembly）With API implements to transmit various API Calls between assembly.Transmission API Calls may include issue, initiate, transfer, calling, Receive, return or receptance function are called or message；In other words, transmission can describe API Calls assembly or API implements what assembly was carried out Action.Function call or other of API are transferred and can be sent by parameter list or other structures or receive one or more Parameter.Parameter can be constant, key, data structure, object, object class, variable, data type, pointer, array, list or Person points to the pointer of function or method or quote will be via the another way of the data of API transmission or sundry item.

In addition, data type or class can be provided by API and implement assembly by API realizing.Thus, API Calls assembly can The variable of this type or class is stated in definition by using providing in API, using the pointer pointing to this type or class, makes With or instantiate the constant value of this type or class.

Usually, API can be used for accessing the service being provided by API enforcement assembly or data or initiates to implement group by API Operation or the execution of calculating that part provides.As an example, API implement assembly and API Calls assembly can be individually operating system, Any one in storehouse, device driver, API, application program or other modules（It should be understood that API implements assembly and API Calls Assembly can be to be same to each other or different to each other the module of type）.API implement assembly in some cases can at least partly with firmware, Microcode or other hardware logic devices are realized.In certain embodiments, API can allow client-side program using by software development kit （Software Development Kit, SDK）The service that storehouse provides.In other embodiments, application or other client journeys Sequence can use by application framework（Application Framework）The API providing.In these embodiments, application or client End program can merge to function that is being provided by SDK and being provided by API or method call or using defined in SDK and by Data type or object that API provides.Application framework can provide main event circulation for program in these embodiments, and this circulation rings Should be in events various defined in this framework.API allows applications exploiting application framework to specify event and the response to event.? In some implementations, API Calls can to the ability of application report hardware device or state, including with such as input capability and shape State, fan-out capability are relevant with the aspect of state, disposal ability, power supply status, memory capacity and state, communication capacity etc Those, and API can other low level logic devices that be executed on a hardware component by firmware, microcode or a part of part come Realize.

API Calls assembly can be local component（That is, implement assembly in same data handling system with API）Or warp The remote component that API and API implements assembly communication is passed through by network（That is, implement assembly in different data handling systems from API On）.It should be understood that API implements assembly also acts as API Calls assembly（That is, it can expose to implementing assembly by another difference API API make API Calls）, and API Calls assembly also can be by implementing to be exposed to the API of another difference API Calls assembly Implement assembly to serve as API.

API can allow to be communicated with API enforcement assembly with multiple API Calls assemblies that different programming languages is write （Thus API may include for implementing to convert, in API, the feature called and return between assembly and API Calls assembly）；But API Can realize according to specific programming language.API Calls assembly can call the API from different suppliers in one embodiment, For example it is derived from one group of API and another group of API from plugged-in provider of OS supplier and carry from the another of another group of API Donor（The supplier of such as software library）Or another group of API of founder.

Figure 19 is the block diagram illustrating spendable exemplary API architecture in some embodiments of the invention.As figure Shown in 19, the API that API architecture 1800 includes implementing API1820 implements assembly 1810（For example, operating system, storehouse, equipment Driver, API, application program, software or other modules）.API1820 specify API implement assembly can be by API Calls assembly The one or more functions of 1830 uses, method, class, object, agreement, data structure, form and/or other features.API1820 May specify at least one calling convention, this calling convention specifies API to implement how the function in assembly receives from API Calls assembly Parameter and how this function is to API Calls assembly returning result.API Calls assembly 1830（For example, operating system, storehouse, equipment Driver, API, application program, software or other modules）Make API Calls by API1820 to access and use API to implement The feature specified by API1820 of assembly 1810.API implements assembly 1810 can be by API1820 to API in response to API Calls Invocation component 1830 return value.

It will be understood that, API implement assembly 1810 may include do not pass through API1820 specify and to API Calls assembly 1830 not Available additional functions, method, class, data structure and/or other features.It should be understood that API Calls assembly 1830 can be real with API Apply assembly 1810 on same system, or can be located at long-range and utilize API1820 to access API enforcement assembly by network 1810.Although Figure 19 show single API Calls assembly 1830 interact with API1820 it should be appreciated that with API Calls assembly 1830 different language（Or identical language）Other API Calls assemblies write can use API1820.

API implements assembly 1810, API1820 and API Calls assembly 1830 and can be stored in the non-transient storage of machine readable In medium, this medium is included for machine（For example, computer or other data handling systems）Readable form storage information Any mechanism.For example, machine readable media includes disk, CD, random access memory；Read-only storage, flash memory device Deng.

In Figure 20（" software stack "）In, in an exemplary embodiment, application can be using several services API to service 1 or 2 Make and calling, and using several OS API to operating system（OS）Make and calling.Service 1 and 2 can utilize several OS API makes to OS and calling.

Note, service 2 has two API, one of（Service 2API1）Receive calling and to application 1 of self-application 1 Return value, another（Service 2API2）Receive calling and to applying 2 return values of self-application 2.Service 1（It can be for example Software library）OS API1 is made and calls and receive return value from OS API1, service 2（It can be for example software library）To OS API1 and OS API2 makes and calls and receive return value from OS API1 and OS API2.Application 2 is to OS API2 Make and call and receive return value from OS API2.

Refer in this specification " embodiment " or " embodiment " the meaning be contact this embodiment description specific spy Levy, structure or characteristic are included at least one embodiment of the invention.Various in this manual places occur phrase " In one embodiment " it is not necessarily all referring to same embodiment.

In the foregoing specification, the specific embodiment by reference to the present invention describes the present invention.It may be evident, however, that In the case of the wider spirit and scope of the present invention, various modifications and changes can be carried out to it.Specification and drawings because This is considered as exemplary rather than restricted.

Annex

Following annex is the draft specification of agreement according to a particular embodiment of the invention.It will be understood that, right in this annex Some keywords（For example it is necessary to, must not, should, should not wait）Use be applied to this specific embodiment, and bel not applied to the disclosure Described in other embodiment.

HTTP fact streaming

draft-pantos-http-live-streaming-06

Summary

This document describes for the agreement transmitting unlimited multimedia data stream.Which specify file data form and The server of stream（Sender）And client（Recipient）Action to be taken.Which depict the 3rd version of this agreement.

Catalogue

1. brief introduction

2. summary

3. play list file

3.1. brief introduction

3.2. attribute list

3.3. new label

3.3.1.EXT-X-TARGETDURATION

3.3.2.EXT-X-MEDIA-SEQUENCE

3.3.3.EXT-X-KEY

3.3.4.EXT-X-PROGRAM-DATE-TIME

3.3.5.EXT-X-ALLOW-CACHE

3.3.6.EXT-X-PLAYLIST-TYPE

3.3.7.EXT-X-ENDLIST

3.3.8.EXT-X-STREAM-INF

3.3.9.EXT-X-DISCONTINUITY

3.3.10.EXT-X-VERSION

4. media file

5. key file

5.1. brief introduction

5.2. it is used for the IV of AES-128

6. client/server action

6.1. brief introduction

6.2. server process

6.2.1. brief introduction

6.2.2. sliding window playlist

6.2.3. encrypted media file

6.2.4. providing variant stream

6.3. client process

6.3.1. brief introduction

6.3.2. loaded playlists file

6.3.3. play play list file

6.3.4. reload play list file

6.3.5. determine next file to be loaded

6.3.6. encrypted media file is decrypted

7. protocol edition compatible

8. example

8.1. brief introduction

8.2. simple playback listing file

8.3. sliding window playlist, using HTTPS

8.4. there is the play list file of encrypted media file

8.5. variant play list file

9. security considers

10. bibliography

10.1. normative bibliography

10.2. informedness bibliography

1. brief introduction

This document describes for the agreement transmitting unlimited multimedia data stream.This agreement supports the encryption of media data Replacement version with stream（Such as bit rate）Offer.Media data can be transmitted after it is created quickly, so that It can be play by near real-time.Data is transported typically by HTTP [RFC2616].

The external reference document of the relevant criterion of description such as HTTP etc is listed in Section 11.

2. summary

Multimedia presents is specified by the URI [RFC3986] going to play list file, and play list file is media URI Ordered list with information labels.Each media URI is incorporated as the media file of the fragment of single continuous stream.

In order to play stream, client obtains play list file first, then obtains and plays each in playlist Media file.It reloads play list file to find extra fragments as described in this document.

Keyword " necessary " in this document, " must not ", " requirement ", " answering ", " not should ", " should ", " not should ", " push away Recommend ", "available" and " optional " should explain as described in RFC 2119 [RFC 2119].

3. play list file

3.1. brief introduction

Playlist must be extension M3U play list file [M3U].This document is extended by defining extra label M3U file format.

The text that M3U playlist is made up of each row.Row may terminate at single line feed character or is followed by There is the CR character of line feed character.Every a line is URI, blank or is started with annotating character " # ".Blank line is ignored.Must not exist Space character, in addition to the element wherein clearly specified.

URI line identifier media file or variant play list file（Referring to 3.3.8 section）.

URI can be relative.Relative URI must compare the URI of the play list file comprising it to parse.

The row being started with annotating character " # " is annotation or label.Label starts from #EXT.Every other started with " # " Row is all annotation and should be ignored.

The duration of play list file is the summation of the duration of its interior media file.

That title is ended up with .m3u8 and/or there is HTTP content type " application/vnd.apple.mpegurl " M3U play list file encoded with UTF-8 [RFC3629].That title is ended up with .m3u and/or there is HTTP content class The file of type [RFC2616] " audio/mpegurl " is encoded with US-ASCII [US_ASCII].

Play list file must have with the title of .m3u8 ending and/or have content type " application/ vnd.apple.mpegurl”（If transmitted by HTTP）, or have with the title of .m3u ending and/or have in HTTP Hold type " audio/mpegurl "（For compatibility）.

Extension M3U file format defines two labels：EXTM3U and EXTINF.Extension M3U file pass through its first row with Basic M3U file phase is distinguished, and its first row must be #EXTM3U.

EXTINF is the recording mark of the media file that description URI followed by is identified.Each media file URI it Before must be EXTINF label.Its form is as follows.

#EXTINF:<duration>,<title>

" duration " is integer or the floating number of the duration specifying media file in seconds.When integer continues Between should be rounded to nearest integer.If the protocol version of play list file is less than 3, the duration must be whole Number.After comma, the remainder of this row is the title of media file, and it is the human readable information of optional media fragment Title.

The following new label of this document definition：EXT-X-TARGETDURATION、EXT-X-MEDIA-SEQUENCE、EXT-X- KEY、EXT-X-PROGRAM-DATE-TIME、EXT-X-ALLOW-CACHE、EXT-X-PLAYLIST-TYPE、EXT-X- STREAM-INF, EXT-X-ENDLIST, EXT-X-DISCONTINUITY and EXT-X-VERSION.3.2. attribute list

It is the value of attribute list that some extension M3U labels have.Attribute list is that do not have the attribute/value of space character to teasing Number separate list.

Attribute/value is to having following grammer.

AttributeName=AttributeValue

AttributeName is the character string without quotation marks comprising the character from set [A-Z].

AttributeValue is one below.

Zero decimal integer：From the character string without quotation marks of the character of set [0-9], in the arithmetic with 10 as radix Middle expression integer.

Zero hexadecimal integers：From the character string without quotation marks of the character of set [0-9] and [A-F], its prefix is 0x or 0X and it expresses integer in the arithmetic with 16 as radix.

Zero decimal floating point：From the character string without quotation marks of the character of set [0-9] and ". ", with 10 as radix Arithmetic in express floating number.

Zero character string with quotation marks：A pair of double quotation marks（″）Interior character string.The set of character allowing in character string and use Any rule in escape spcial character to be specified by attribute definition, but any double quotation marks (") character and any carriage return or line feed All substituted by escape sequence all the time.

Zero enumeration type character string：Carry out the character string without quotation marks of the clearly defined set of free attribute.Enumerate ocra font ocr String never comprises double quotation marks（″）, comma（,）Or space character.

Zero decimal system resolution ratio：Two decimal integers being separated by " x " character, indicate horizontal and vertical pixel chi Very little.

The type of the AttributeValue of given AttributeName to be specified by attribute definition.

Given AttributeName must not occur more than once in given attribute list.

The attribute/value of AttributeName with None- identified is to must be ignored by client.

The attribute/value of the enumeration type character string type of the value comprising None- identified is to should be ignored by client.

3.3. new label

3.3.1.EXT-X-TARGETDURATION

The maximum media file duration specified by EXT-X-TARGETDURATION label.Each in play list file The EXTINF duration of media file is necessarily less than or is equal to target time duration.This label is necessary in play list file Occur once.Its form is：

#EXT-X-TARGETDURATION:<s>

Wherein s is the integer indicating target time duration in seconds.

3.3.2.EXT-X-MEDIA-SEQUENCE

Each media file URI in playlist has unique integer sequence number.It is front that the sequence number of URI is equal to it The sequence number of URI adds one.EXT-X-MEDIA-SEQUENCE label indicates the sequence of the URI occurring in play list file Number.Its form is：

#EXT-X-MEDIA-SEQUENCE:<number>

Play list file must not comprise more than one EXT-X-MEDIA-SEQUENCE label.If play list file Do not comprise EXT-X-MEDIA-SEQUENCE label, then the sequence number of the URI in playlist will be considered as 0.

The sequence number not requiring media file occurs in its URI.

With regard to processing the information of EXT-X-MEDIA-SEQUENCE label, referring to 6.3.2 section and 6.3.5 section.

3.3.3.EXT-X-KEY

Media file can be encrypted.EXT-X-KEY label is provided needed for media file followed by is decrypted Information.Its form is：

#EXT-X-KEY:<attribute-list>

Define with properties：

METHOD attribute specifies encryption method.It is enumeration type character string type.Define two methods：NONE and AES-128.

It is not encrypted that encryption method NONE refers to media file.If encryption method is NONE, URI and IV attribute is not Must exist.

Encryption method AES-128 refer to using have 128 bit keys and PKCS7 filling [RFC5652] senior plus Data Encryption Standard [AES_128] is encrypted to media file.If encryption method is AES-128, URI attribute there must be.IV Attribute there may be；Referring to Section 5.2.

URI attribute specifies how to obtain key.Its value be comprise URI [RFC3986] with regard to this key with quotation marks Character string.

IV attribute is if there is the then specified initialization vector will being used together with key.Its value is hexadecimal integers. IV attribute occurs in protocol version 2.

New EXT-X-KEY replaces any previous EXT-X-KEY.

If play list file does not comprise EXT-X-KEY label, media file is not encrypted.

With regard to the form of key file, referring to Section 5, with regard to the extraneous information of media file encryption, referring to Section 5.2, 6.2.3 section and 6.3.6 section.

3.3.4.EXT-X-PROGRAM-DATE-TIME

The beginning of next media file is closed by EXT-X-PROGRAM-DATE-TIME label with absolute date and/or time Connection gets up.It is ISO/IEC8601 that date/time represents:2004 [ISO_8601] and time zone should be indicated.For example：

#EXT-X-PROGRAM-DATE-TIME:<YYYY-MM-DDThh:mm:ssZ>

With regard to the more information of EXT-X-PROGRAM-DATE-TIME label, referring to 6.2.1 section and 6.3.3 section.

3.3.5.EXT-X-ALLOW-CACHE

EXT-X-ALLOW-CACHE label indicate client be can or must not cache the media file of download for after Reset.It can occur in play list file Anywhere；It must not occur more than once.EXT-X-ALLOW- All fragments be applied to playlist for the CACHE label.Its form is：

#EXT-X-ALLOW-CACHE:<YES|NO>

With regard to the more information of EXT-X-ALLOW-CACHE label, referring to 6.3.3 section.

3.3.6.EXT-X-PLAYLIST-TYPE

EXT-X-PLAYLIST-TYPE label provides the mutability information with regard to play list file.It is optional.Its Form is：

#EXT-X-PLAYLIST-TYPE:<EVENT|VOD>

6.2.1 section defines the connotation of EXT-X-PLAYLIST-TYPE label.

3.3.7.EXT-X-ENDLIST

EXT-X-ENDLIST label indicates that more multimedia file can not be added to play list file.It is permissible Occur in play list file Anywhere；It must not occur more than once.Its form is：

#EXT-X-ENDLIST

3.3.8.EXT-X-STREAM-INF

EXT-X-STREAM-INF label indicates that next URI in play list file identifies another play list file. Its form is：

#EXT-X-STREAM-INF:<attribute-list>

<URI>

Define with properties：

BANDWIDTH

This value is the decimal integer representing BPS.It must be occurred or will appear in playlist The upper limit of the overall bit rate of each media file, is calculated as including capsule overhead.

Each EXT-X-STREAM-INF label must include BANDWIDTH attribute.

PROGRAM-ID

This value is the specific decimal integer presenting uniquely identifying in the range of play list file.

Play list file can comprise to have multiple EXT-X-STREAM-INF labels of identical PROGRAM-ID with Identify the same different coding presenting.These variant playlists can comprise extra EXT-X-STREAM-INF label.

CODECS

This value is the band quotation marks character string of the comma separated list comprising form, and wherein each form is specified in playlist Media sample type present in media file in file.Valid format identifier is to be defined by RFC 4281 [RFC4281] ISO file format name space in those.

Each EXT-X-STREAM-INF label should include CODECS attribute.

RESOLUTION

This value is the decimal system resolution ratio of the horizontal and vertical resolution ratio after describing the coding substantially of video in flowing.

3.3.9.EXT-X-DISCONTINUITY

Between EXT-X-DISCONTINUITY label instruction media file followed by and the media file before it Encryption discontinuity.The feature collection that can change is：

Zero file format

The number of zero track and type

Zero coding parameter

Zero coded sequence

Zero timestamp sequence

Its form is：

#EXT-X-DISCONTINUITY

With regard to the more information of EXT-X-DISCONTINUITY label, referring to Section 4,6.2.1 section and 6.3.3 section.

3.3.10.EXT-X-VERSION

EXT-X-VERSION label indicates the compatible version of play list file.Play list file, the matchmaker of its association Body and its server must be complied with describing all regulations of the most recent version of this document of protocol version of label value instruction.

Its form is：

#EXT-X-VERSION:<n>

Wherein n is the integer of instruction protocol version.

Play list file must not comprise more than one EXT-X-VERSION label.Do not comprise EXT-X-VERSION label Play list file must be complied with the version 1 of this agreement.

4. media file

Each media file URI in play list file must identify the media file as the overall fragment presenting. Each media file must be formatted as mpeg 2 transport stream or MPEG-2 audio stream [ISO_13818].

Transfer stream file must comprise single MPEG-2 program.The beginning of each file should have Program Association Table and Program Map Table.The file comprising video should have at least one key frame and enough information to initialize video solution completely Code device.

Media file in playlist must be the encoding stream at the end of the media file with previous sequence number Continue, unless it is to be embroidered with EXT-X- before the first media file or its once occurring in play list file DISCONTINUITY label.

Client should be ready for particular type（Such as audio or video）Multiple tracks.There is no other preferences What client should select that it can play has that of minimum numerical value PID.

Client must ignore its in transport stream unidentified go out privately owned stream.

The encryption parameter of in the stream within media file and between the respective streams of multiple media files sample should It is consistent.However, client should tackle this change when running into code change, for example, fitted by scaling video content Answer change resolution.

5. key file

5.1. brief introduction

There is the EXT-X-KEY tag identifier key file of URI attribute.Key file comprises to follow-up in playlist Media file deciphering has to use for encryption key.

AES-128 encryption method uses the key of 16 eight bytes.The form of key file is exactly this of binary format The compression array of 16 eight bytes.

5.2. it is used for the IV of AES-128

128 bit A ES require to provide the initialization vector of 16 eight bytes of identical in encryption and deciphering（IV）.Change This IV will increase the intensity of password.

If EXT-X-KEY label has IV attribute, implementation must when being encrypted or being deciphered using this key This property value must be used as IV.This value must be construed as 128 bits hexadecimal number and must before be embroidered with 0x or 0X.

If EXT-X-KEY label does not have IV attribute, implementation is necessary when to media file encryption or deciphering The sequence number that this media file is used is as IV.The big end binary representation of sequence number should be placed on the buffering of 16 eight bytes In device and（In left side）It is filled with zero.

6. client/server action

6.1. brief introduction

This section describes how server generates playlist and media file and how client should be downloaded and broadcast Put them.

6.2. server process

6.2.1. brief introduction

Outside the scope of this document, this document just simply supposes a continuous stream comprising to present for the generation of MPEG-2 stream Source.

Stream must be divided into the individual media literary composition that the duration is less than or equal to constant target time duration by server Part.Server should attempt the point in the efficient decoding supporting individual media file（For example in packet and crucial frame boundaries）Draw Shunting.

Server is necessary for each media file and creates URI, and this URI will allow its client to obtain this document.

Server must create play list file.Play list file has to comply with the form described in Section 3.With regard to Server wishes that the URI being allowed to each media file available must be occurred in the play list with its order to be played. Whole media file must can use to client, if its URI is in play list file.

Play list file must comprise EXT-X-TARGETDURATION label.Its value be necessarily equal to or more than occur or EXTINF value by any media file appearing in play list file.Its value must not change.Typical target Continuous Time is 10 seconds.

Play list file should comprise an EXT-X-VERSION label, the compatible version of this label instruction stream.Its Value must be the minimum protocol version all deferred to of media file of server, play list file and association.

Server is necessary for play list file and creates URI, and this URI will allow its client to obtain this document.

If play list file is distributed by HTTP, server should be supported to use the visitor of " gzip " research content Family end is asked.

Change to play list file must be carried out from the perspective of client device in the way of atom.

Server must not change play list file, except in order to：

To its additional row（6.2.1 saves）；

The order being occurred with it from playlist removes media file URI, and is only applicable to these media files Any label（6.2.2 saves）；

Change the value of EXT-X-MEDIA-SEQUENCE label（6.2.2 saves）；

Add or remove EXT-X-STREAM-INF label（6.2.4 saves）It is noted that not requiring client reloading variant Play list file, therefore changing their possibility will not be effective immediately；

Add EXT-X-ENDLIST label to playlist（6.2.1 saves）.

In addition, play list file can comprise the EXT-X-PLAYLIST-TYPE label that value is EVENT or VOD.If This label exists and is worth for EVENT, then server must not change or delete any part of play list file（Although it can With to its additional row）.If this label exists and is worth for VOD, play list file must not change.

Each media file URI in playlist must before be embroidered with indicate this media file duration EXTINF Label.

Server can be by will be exhausted with EXT-X-PROGRAM-DATE-TIME label by the URI prefix of media file Date and time is associated with this media file.The value of date and time provides the timeline of media to suitable wall clock The informedness mapping of time, this can serve as searching, the benchmark or for other purposes of display.If server provides this to reflect Penetrate, then EXT-X-PROGRAM-DATE-TIME label should be placed on each EXT-X- in play list file by it After DISCONTINUITY label.

If playlist comprises last media file presenting, play list file must comprise EXT-X- ENDLIST label.

If playlist does not comprise EXT-X-ENDLIST label, server must make to comprise at least one new matchmaker The play list file of the redaction of body file URI can use.Must be available with respect to the play list file making previous version Time makes following available：It is no earlier than the half of the target time duration after this time, and target after being not later than this time 1.5 times of duration.

If server wishes to remove entirely presented, it must make play list file unavailable to client.It should When guaranteeing that all media files in the play list file when removing are available to client holding, the retention time is at least plays The duration of listing file.

6.2.2. sliding window playlist

The availability of media file can be limited to be added to those of playlist recently by server.For this reason, playing Listing file must comprise just what a EXT-X-MEDIA-SEQUENCE label all the time.For by from play list file Each media file URI removing, its value must be incremented by 1.

Media file URI must be removed from play list file by the order that it is added.

If the duration that the duration of play list file deducts media file is less than the three of target time duration Times, then server must not remove media file URI from play list file.

When server removes media file URI from playlist, media file should be to client in a period of time End keeps available, and the amount of this time is equal to the longest broadcasting that the duration of media file occurs wherein plus media file The duration of listing file.

If server is planned to remove this media file after media file is delivered to client by HTTP, its It should be ensured that http response comprises the expired head of the life span of reflection plan.

6.2.3. encrypting to media file

If media file is to be encrypted, server must define a URI, and this URI will allow authorized client Obtain the key file comprising decruption key.Key file has to comply with the form described in Section 5.

Server can set the expired head of HTTP in the key response that instruction key can be buffered.

If encryption METHOD is AES-128, reply individual media file application AES-128CBC encryption.Must be to whole Individual file encryption.Must not across media file application CBC.For encryption IV must be media file sequence number or The value of the IV attribute of person's EXT-X-KEY label, as described in Section 5.2.

Server must be using the method specified by following EXT-X-KEY label and other attributes to every in playlist Individual media file encryption：This EXT-X-KEY label is that being close in play list file before its URI.It is above METHOD is the EXT-X-KEY label of NONE or does not above have the media file of any EXT-X-KEY label must not be encrypted.

Server must not remove EXT-X-KEY label from play list file, if play list file comprises to go to If the URI of the media file of this key encryption.

6.2.4. providing variant stream

Server can provide multiple play list file to provide the same different coding presenting.If its do so, Then it should provide the variant play list file listing each variant stream to allow client dynamically to switch between coding.

Variant playlist must comprise EXT-X-STREAM-INF label for each variant stream.Present with regard to same Each EXT-X-STREAM-INF label must have identical PROGRAM-ID property value.The PROGRAM- presenting with regard to each ID value must be unique in variant playlist.

If EXT-X-STREAM-INF label comprises CODECS attribute, this property value must be included by [RFC4281] Being present in of definition any occurs or by each of the media file appearing in play list file form.

Server must is fulfilled for following constraint when producing variant stream：

Each variant stream must assume identical content, locates including stream is discontinuous；

Each variant play list file must have identical target time duration；

Occur in a variant play list file but the content that do not appear in another must be present in this broadcasts Emplace beginning or the end of list file, and target time duration must not be longer than；

Matching content in variant stream must have the timestamp of coupling, this allow that client convection current synchronizes；

Elementary audio stream file must be by adding owner's identifier before ID3 PRIV label [ID3] " com.apple.streaming.transportStreamTimestamp " is signaling first sample in file Timestamp, binary data must be the 33 bit MPEG-2 Program Elementary Stream timestamps being expressed as holding greatly eight eight byte numbers, 31 high-order bits are set to zero.

Additionally, all variant streams should comprise identical coded audio bit stream.This allow that client between streams Switching, without audible burr.

6.3. client process

6.3.1. brief introduction

How client obtains the URI going to play list file outside the scope of this document；It is assumed that this has completed.

Client must obtain play list file from URI.If the play list file being achieved in that is variant play List, then client must obtain play list file from variant playlist.

This document does not specify the disposal to variant stream for the client.

6.3.2. loaded playlists file

Every time when playlist URI loads or reloads play list file：

Client must assure that this play list file starts from EXTM3U label, and guarantees that EXT-X-VERSION marks Sign if there is the protocol version that then given client end is supported；If it is not, then client must not be attempted using this playlist；

Client should ignore its unidentified go out any label and attribute；

Client must determine next media file to be loaded as described in 6.3.5 section.

If playlist comprises EXT-X-MEDIA-SEQUENCE label, client it shall be assumed that the following time its In each media file will become unavailable：This time is the time of loaded playlists file to add play list file Duration.The duration of play list file is the summation of the duration of its interior media file.

6.3.3. play play list file

Client should select which media file to be play from playlist first when playback starts.If EXT-X- ENDLIST label does not exist and client is wanted in a usual manner（That is, according to playlist order, with nominal playback speed Rate）Play media, then client should not select from the end of play list file be less than three target time duration when Between start file.Do so can trigger playback and pause.

In order to realize conventional playback, the played in order that media file must be occurred in play list file by them. Client can assume useable medium in its desired any mode, including conventional playback, random access and special-effect mode.

Client must be prepared to reset before broadcasting is above the media file of EXT-X-DISCONTINUITY label Its（One or more）Resolver and（One or more）Decoder.

Client should attempt to loading medium file before needing media file in case carry out continual playback with The temporary variations of delay and handling capacity are compensated.

If play list file comprises EXT-X-ALLOW-CACHE label and its value is NO, in the media downloaded After file is played, client must not cache the media file of download.Otherwise, client can indefinitely cache download Media file reset for later.

When client can display to the user that program initiates using the value of EXT-X-PROGRAM-DATE-TIME label, Between.If this value includes time-zone information, client should take into account, but if not including, then client must not be inferred Rise time zone.

Client must not depend on correctness or the uniformity of the value of EXT-X-PROGRAM-DATE-TIME label.

6.3.4. reload play list file

Client periodically must reload play list file, unless it comprises EXT-X-ENDLIST label.

However, client must not attempt the more frequently reloading play list file than this section regulation.

When client first time loaded playlists file or reloading play list file find it from last time quilt Load when changing, client had to wait for a period of time before attempting reloading play list file again.Now Section is referred to as initially minimum reloading and postpones.It is measurement from the time that client starts loaded playlists file.

Initially minimum reloading postpones to be the duration of last media file in playlist.Media file is held The continuous time is specified by EXTINF label.

If client reloads play list file and finds that it is not changed in, client is necessary before retrying Wait for a period of time.Minimum delay is the multiple of target time duration.This multiple was attempted being 0.5 for first time, for the Secondary is 1.5, for later be 3.0.

In order to mitigate server load, client should not reload current uncivilian in the playlist of the variant stream play Part.If playback is switched to another different variants by its decision, its should stop reloading the playlist of old variant and Start to load neomorphic playlist.It can determine phase using the constraint in EXTINF duration and 6.2.4 section Answer the approximate location of media.Once being loaded with from neomorphic media it is possible to use the timestamp in media file Come accurately synchronous old times top-stitching and new timeline.

6.3.5. determine next file to be loaded

When play list file is loaded or reloads every time, client has to check for this play list file will to determine Next media file loading.

First file to be loaded must be client select will broadcasting first file, described in such as 6.3.3 saves.

If the first file to be play has been loaded and play list file does not comprise EXT-X-MEDIA- SEQUENCE label, then client must verify that current play list file in the media file originally finding last loading Comprising this URI at the side-play amount of URI, if do not comprised, stopping playback.Next media file to be loaded must be played column The first media file URI after the last URI loading is followed in table.

If the first file to be play has been loaded and play list file comprises EXT-X-MEDIA-SEQUENCE Label, then next media file to be loaded can be the lowest series with the sequence number more than the last media file loading Number that.

6.3.6. encrypted media file is deciphered

If play list file comprises to specify the EXT-X-KEY label of key file URI, client must obtain this Key file and using its interior key come to all media files deciphering followed after EXT-X-KEY label, until Till running into another EXT-X-KEY label.

If encryption METHOD is AES-128, reply individual media file application AES-128CBC deciphering.Must be to whole Individual file decryption.Must not across media file application CBC.For deciphering IV must be media file sequence number or The value of the IV attribute of person's EXT-X-KEY label, as described in Section 5.2.

If encryption METHOD is NONE, client must follow all media after EXT-X-KEY label File is considered as in plain text（Unencryption）, till running into another EXT-X-KEY label.

7. protocol edition compatible

Client and server must realize protocol version 2 or more highest version to use：

The IV attribute of zero EXT-X-KEY label.

Client and server must realize protocol version 3 or more highest version to use：

Zero floating-point EXTINF duration value.

8. example

8.1. brief introduction

This section comprises several example playlist files.

8.2. simple playback listing file

#EXTM3U

#EXT-X-TARGETDURATION:5220

#EXTINF:5220,

http://media.example.com/entire.ts

#EXT-X-ENDLIST

8.3. sliding window playlist, using HTTPS

#EXTM3U

#EXT-X-TARGETDURATION:8

#EXT-X-MEDIA-SEQUENCE:2680

#EXTINF:8,

https://priv.example.com/fileSequence2680.ts

#EXTINF:8,

https://priv.example.com/fileSequence2681.ts

#EXTINF:8,

https://priv.example.com/fileSequence2682.ts

8.4. there is the play list file of encrypted media file

#EXTM3U

#EXT-X-MEDIA-SEQUENCE:7794

#EXT-X-TARGETDURATION:15

#EXT-X-KEY:METHOD=AES-128,URI=″https://priv.example.com/key.php?r=52″

#EXTINF:15,

http://media.example.com/fileSequence52-1.ts

#EXTINF:15,

http://media.example.com/fileSequence52-2.ts

#EXTINF:15,

http://media.example.com/fileSequence52-3.ts

#EXT-X-KEY:METHOD=AES-128,URI=″https://priv.example.com/key.php?r=53″

#EXTINF:15,

http://media.example.com/fileSequence53-1.ts

8.5. variant play list file

#EXTM3U

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1280000

http://example.com/low.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2560000

http://example.com/mid.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=7680000

http://example.com/hi.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=65000,CODECS=″mp4a.40.5″

http://example.com/audio-only.m3u8

9. security considers

Because agreement generally uses HTTP to transmit data, so most of identical security considers it is all applicable.Ginseng See Section 15 of RFC 2616 [RFC2616].

Media file resolver is typically easy to suffer from " investigation（fuzzing）" attack.Client is receiving to from server File care should be taken to, to refuse incongruent file when being parsed.

Play list file comprises URI, and client will make the network request of any entity using these URI.Client End should carry out range check to prevent buffer from overflowing to response.See also " the Security of RFC 3986 [RFC3986] This section of Considerations ".

Client should idly load the resource being identified by URI, to avoid facilitating Denial of Service attack.

HTTP request often includes session status（“cookie”）, this session status can comprise secret user data.Realize Mode must comply with cookie restriction and the rule of expiring that RFC 2965 [RFC2965] specifies.See also RFC 2965 " Security Considerations " this section, and RFC 2964 [RFC2964].

Encryption key is specified by URI.Should be by the HTTP [RFC5246] such as based on TLS（It was SSL in the past）Etc Mechanism to protect the delivery of these keys with reference to security fields or session cookie.

10. bibliography

10.1. normative bibliography

[AES_128]U.S.Department of Commerce/National Institute of Standards and Technology,″Advanced Encryption Standard(AES),FIPS PUB197″,November 2001,

<http://csrc.nist.gov/publications/fips/fips197/fips-197.pdf

<http://csrc.nist.gov/publications/fips/fips197/fips-197.pdf>>.

[ISO_13818]International Organization for Standardization,″ISO/IEC International Standard13818;Generic coding of moving pictures and associated audio information″,October 2007,

<http://www.iso.org/iso/catalogue_detail?csnumber=44169>.

[ISO_8601]International Organization for Standardization,″ISO/IEC International Standard8601:2004;Data elements andinterchange formats-- Information interchange--Representation of dates and times″,December2004,

<http://www.iso.org/iso/catalogue_detail?csnumber=40874>.

[RFC2046]Freed,N.and N.Borenstein,″Multipurpose Internet Mail Extensions(MIME)Part Two:Media Types″,RFC2046,November1996.

[RFC2119]Bradner,S.,″Key words for use in RFCs to Indicate Requirement Levels″,BCP 14,RFC 2119,March 1997.

[RFC2616]Fielding,R.,Gettys,J.,Mogul,J.,Frystyk,H.,Masinter,L.,Leach, P.,and T.Berners-Lee,″Hypertext Transfer Protocol--HTTP/1.1″,RFC 2616, June1999.

[RFC2964]Moore,K.and N.Freed,″Use of HTTP State Management″,BCP 44, RFC 2964,October 2000.

[RFC2965]Kristol,D.and L.Montulli,″HTTP State Management Mechanism″, RFC 2965,October 2000.

[RFC3629]Yergeau,F.,″UTF-8,a transformation format of ISO 10646″,STD 63,RFC 3629,November 2003.

[RFC3986]Berners-Lee,T.,Fielding,R.,and L.Masinter,″Uniform Resource Identifier(URI):Generic Syntax″,STD66,RFC3986,January 2005.

[RFC4281]Gellens,R.,Singer,D.,and P.Frojdh,″The Codecs Parameter for″ Bucket″Media Types″,RFC 4281,November 2005.

[RFC5246]Dierks,T.and E.Rescorla,″The Transport Layer Security(TLS) Protocol Version1.2″,RFC 5246,August 2008.

[RFC5652]Housley,R.,″Cryptographic Message Syntax(CMS)″,STD70,RFC 5652,September2009.

[US_ASCII]American National Standards Institute,″ANSI X3.4-1986, Information Systems--Coded Character Sets7-Bit American National Standard Code for Information Interchange(7-Bit ASCII)″,December1986.

10.2. informedness bibliography

[ID3]ID3.org<http://ID3.org/>,″The ID3audio file data taggingformat″, <http://www.id3.org/Developer_Information>.

[M3U]Nullsoft,Inc.,″The M3U Playlist format,originally invented for the Winamp media player″,<http://wikipedia.org/wiki/M3U>.

Claims (25)

1. a kind of method that machine is implemented, including：

Receive the first playlist, this first playlist is included with regard to the URL of the second playlist with regard to the 3rd played column The URL of table, described second playlist includes multiple URL of the part comprising I frame of specified file, described second playlist In multiple parts specified there is identical bit rate, and described 3rd playlist includes regarding for showing with 1X speed Multiple URL of frequency and associated label；

Determine the playback speed different from 1X；

Send the URL with regard to described second playlist in response to the described playback speed of determination；

Receive described second playlist；

Send the request to I frame using the URL in described second playlist；

The I frame receiving in response to the described request to I frame is assumed with described playback speed；And

Wherein, the URL in described second playlist comprises the data area of each I frame in the described file of I frame with specifying Parameter is associated.

2. the label being the method for claim 1, wherein associated with URL in described 3rd playlist includes specifying The parameter of the data area of video is comprised in described file.

3. the method for claim 1, wherein described first playlist is included with regard to multiple 1X playlists accordingly , in order under different resolution ratio or different quality with 1X speed show video multiple URL, and described first broadcasting List is included with regard to multiple accordingly playlist, under different resolution or different quality multiple URL only containing I frame.

4. method as claimed in claim 3, wherein, described second playlist has label, and described label is its corresponding URL be associated or its corresponding URL adjacent it is intended that the part only comprising I frame of described file.

5. method as claimed in claim 3, wherein, each URL in described second playlist is had and is associated with this URL Movie time metadata.

6. method as claimed in claim 5, wherein, the data area of each I frame is specified by bytes range, each bytes range Including voice data untapped when being played back with described playback speed.

7. a kind of data handling system, including：

For receiving the device of the first playlist, this first playlist include with regard to the second playlist URL and with regard to The URL of the 3rd playlist, described second playlist includes multiple URL of the part comprising I frame of specified file, and described Multiple parts specified in two playlists have identical bit rate, and described 3rd playlist is included for 1X speed Multiple URL of display video and associated label；

For determining the device of the playback speed different from 1X；

For sending the device of the URL with regard to described second playlist in response to the described playback speed of determination；

For receiving the device of described second playlist；

For sending the device of the request to I frame using the URL in described second playlist；

For assuming the device of the I frame receiving in response to the described request to I frame with described playback speed；And

Wherein, the URL in described second playlist and the parameter specifying the data area of each I frame in the file comprising I frame Associated.

8. system as claimed in claim 7, wherein, the label being associated with URL in described 3rd playlist includes specifying The parameter of the data area of video is comprised in described file.

9. system as claimed in claim 7, wherein, described first playlist is included with regard to multiple 1X playlists accordingly , in order under different resolution ratio or different quality with 1X speed show video multiple URL, and described first broadcasting List is included with regard to multiple accordingly playlist, under different resolution or different quality multiple URL only containing I frame.

10. system as claimed in claim 9, wherein, described second playlist has label, and described label is its corresponding URL be associated or its corresponding URL adjacent it is intended that the part only comprising I frame of described file.

11. systems as claimed in claim 9, wherein, each URL in described second playlist has related to this URL The movie time metadata of connection, the data area of each I frame is specified by bytes range, and each bytes range is included when with described time Untapped voice data when putting speed playback.

The method that a kind of 12. machines are implemented, including：

Determine one group of parameter, including playback rate, current movie time, display deadline time and can be used for downloading at least one I The time period of frame；

Determine the first set of available I frame, the I frame in described first set is selected from many to display deadline time from current time All available I frame in items of media content and be based on described one group of parameter determination；

Determine multiple I frame subsets, each in the plurality of I frame subset has the I frame in the first set of described available I frame Subset；

Determine for each in the plurality of I frame subset and estimate download time；

One of the plurality of I frame subset is selected based on described estimation download time and described one group of parameter；

Receive and show the I frame in selected in the plurality of I frame subset.

13. methods as claimed in claim 12, wherein, the display to I frame is provided to the video content in I frame according to difference In the special play-back playback mode of the playback rate of 1X, and each shown I frame is to be taken using the label being associated with URL Return, this label includes identifying the bytes range of the part comprising corresponding I frame of file.

14. methods as claimed in claim 13, wherein, the described time period is according to described current movie time and described playback Speed and described display deadline time draw.

15. methods as claimed in claim 14, wherein, the scope that described display deadline time is set as about 1/2 second to about 2 seconds In fixed duration, and described display deadline time be multiplied by described playback rate to produce a product, this product is One time value, this time value is added with the described current movie time to draw the described time period.

16. methods as claimed in claim 15, wherein, each in the plurality of I frame subset has predetermined cadence, this rhythm It is different between the plurality of subset.

17. methods as claimed in claim 16, wherein, the I frame number in each subset in the plurality of subset is subject to predetermined The maximum number of restriction.

18. methods as claimed in claim 17, wherein, the described predetermined cadence of each subset in the plurality of I frame subset It is one below：Each of available I frame in (a) described first set；(b) can use I frame in every one；And (c) can use I frame in every two.

A kind of 19. machines, including the processor of the memory being coupled to instruction programming, described instruction ought be held by described processor Operated during row, described operation includes：

Determine one group of parameter, under described one group of parameter includes playback rate, current movie time, display deadline time and can be used for Carry the time period of at least one I frame；

Determine the first set of available I frame, the I frame in described first set is selected from from the current movie time to display deadline time Content of multimedia item in all available I frame and be based on described one group of parameter determination；

Determine multiple I frame subsets, each in the plurality of I frame subset has the I frame in the first set of described available I frame Subset；

Determine for each subset in the plurality of I frame subset and estimate download time；

One of the plurality of I frame subset subset is selected based on described estimation download time and described one group of parameter；

Receive and show the I frame in the selected a subset in the plurality of I frame subset.

20. machines as claimed in claim 19, wherein, the display to I frame is provided to the video content in I frame according to difference In the special play-back playback mode of the playback rate of 1X, and each shown I frame is to be taken using the label being associated with URL Return, this label includes identifying the bytes range of the part comprising corresponding I frame of file.

21. machines as claimed in claim 20, wherein, described one group of parameter also include show deadline time and just display ought The current movie time of front I frame, and the described time period be according to described current movie time and described playback rate and institute State what display deadline time drew.

22. machines as claimed in claim 21, wherein, the scope that described display deadline time is set as about 1/2 second to about 2 seconds In fixed duration, and described display deadline time be multiplied by described playback rate to produce a product, this product is One time value, this time value is added with the described current movie time to draw the described time period.

23. machines as claimed in claim 22, wherein, each subset in the plurality of I frame subset has predetermined cadence, should Rhythm is different between the plurality of subset.

24. machines as claimed in claim 23, wherein, the number of the I frame in each subset in the plurality of subset is subject in advance Fixed the maximum number of restriction.

25. machines as claimed in claim 24, wherein, the described predetermined cadence of each subset in the plurality of I frame subset It is one below：Each of available I frame in (a) described first set；(b) can use I frame in every one；And (c) can use I frame in every two.