WO2016119735A1

WO2016119735A1 - Method and device for adaptive video content delivery using http

Info

Publication number: WO2016119735A1
Application number: PCT/CN2016/072698
Authority: WO
Inventors: Shan Liu; Chenghao Liu
Original assignee: MediaTek Inc
Current assignee: MediaTek Inc
Priority date: 2015-01-30
Filing date: 2016-01-29
Publication date: 2016-08-04
Anticipated expiration: 2017-07-30

Abstract

A method of switching Representation for a client device and a server is disclosed. For the client device, a current Representation and one or more switched-to Representations associated with media-streaming are identified. The client device then transmits a single command to the server to request switching Representation to the one or more switched-to Representations. The single command comprises switch-Representation information to identify each switched-to Representation or to identify a media presentation time list associated with each switched-to Representation. For the server, a single command related to switching Representation associated with media-streaming from a client device is received. The server parses the single command to determine switch-Representation information for identifying the one or more switched-to Representations associated with the media-streaming, or for identifying a media presentation time list associated with each switched-to Representation. The server then pushes the one or more switched-to Representations associated with the media-streaming to the client device.

Description

METHOD AND DEVICE FOR ADAPTIVE VIDEO CONTENT DELIVERY USING HTTP

CROSS REFERENCE TO RELATED APPLICATIONS

The present invention claims priority to U.S. Provisional Patent Application, Serial No. 62/109,813, filed on January 30, 2015. The U.S. Provisional Patent Application is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

The present invention relates to media streaming over Internet. In particular, the present invention relates to method and device to expedite the switching Representation process between a client and a server using WebSocket protocol or HTTP/2.

BACKGROUND

HyperText Transfer Protocol (HTTP) is a set of standards that allow users of the World Wide Web to exchange information found on web pages. It has been de facto standard for Internet access today. Browsers from various developers all support the HTTP as the communication protocol to connect a client to Web servers on the Internet. Through HTTP, a connection between a user and a server can be established so that HTML (HyperText Markup Language) pages can be sent to the user's browser. The protocol can also be used to download files from the server either to the browser or to any other requesting application that uses the HTTP.

In recent years, video streaming over Internet has been an important application. Today, video streaming contributes to the most Internet traffic. Various multimedia streaming protocols have been widely used and some of the protocols are based on the HTTP. Dynamic Adaptive Streaming over HTTP (DASH) , also known as MPEG-DASH, is an adaptive bitrate streaming technique that enables media content delivery over the Internet based on the conventional HTTP web servers.

In MPEG-DASH (Bouazizi, et al, ISO/IECJTC1/SC29/WG11 MPEG2014/m33623, “DASH over HTTP/2 and WebSocket, ” July 2014, Sapporo, Japan, document: M33623) , a method of DASH over HTTP/2 and WebSocket is disclosed. In particular, “dash” sub protocol in Websocket and HTTP/2 for DASH is used. In Websocket, the keyword “dash” and protocol upgrade request are included in the Sec-WebSocket-Protocol header field. Upon completing upgrade, the DASH data frames are exchanged between a client and a server. The MPEG-DASH standard is intended to provide desired functionalities available to the DASH, including:

· DASH client is able to minimize amount of requests to server；

· DASH client is able to do prompt rate adaptation；

· DASH client is able to minimize delay, e.g. in the case of live streaming, where content is being generated on the fly； and

· DASH/Web server is able to prioritize the data from different Representations based on their importance to the playback.

In M33623, a DASH frame format is defined, which includes DASH command denoted as CMD_CODE and is sent by the request/response. The defined DASH commands include Get the MPD, MPD update, Get all segments of a Representation starting from a given segment or timestamp, reply to a segment request, cancel the transmission of the current resource, request to make a decision, with command code equal to 0 to 6, respectively. MPD represents Media Presentation Description. The Representation refers to collection and encapsulation of one or more media streams in a delivery format and associated with descriptive metadata. The list command codes are shown in Table 1. In command 3, several parameters are defined including “repid” , “segnum” , and “timestamp” , denoting Representation identifier, segment number from which to start streaming and the timestamp from which to start stream. In command 5, no parameter is defined, and reception of command code 5 results in the server to stop the transmission at a reasonable point in the stream with STREAM_ID. For example, the end of a movie fragment may correspond to a point. In the DASH frame format, F is defined to provide a list of flags associated with the command in the DASH frame.

Table 1.

In table 1, the “json parameters” represents JavaScript Object Notation (JSON) parameters, where parameters are formatted in the message body using java script.

In DASH related specifications DASH-part 1 (ISO/IEC 23009-1: 2012, “Information technology -Dynamic adaptive streaming over HTTP (DASH) -Part 1: Media presentation description and segment formats” ) and 3GP-DASH (3GPP TS 26.247, “Transparent end-to-end Packet-switched Streaming Service (PSS) ； Progressive Download and Dynamic Adaptive Streaming over HTTP (3GP-DASH)) , Media Presentation Description (MPD) describes Media Presentation. MPD is hierarchically organized metadata and consisting of one or multiple periods. Each period consists of one or more Adaptation Sets and each Adaptation Set consists of one or multiple Representations. Each Representation may further include one or multiple Sub-Representations and each Representation consists of one or more Segments.

In DASH, @startWithSAP is one of the common attributes in Adaptation Set, Representation, Sub-Representation, which specifies that each Media Segment in the associated Representations starts with a SAP not bigger than the attribute value in case it is present and bigger than 0. SAP represents Stream Access Point, which corresponds to a selected position in a Representation to enable starting playback of a media stream using only the information contained in Representation data. The play back will use the information in the Representation starting from the selected position onwards, which may include information preceded by initializing data in the Initialization Segment if any. According to XML (EXtensible Markup Language) convention, attributes are identified by a lower-case first letter and are preceded by a ‘@’ -sign. The convention is used in this disclosure.

In DASH, @segmentAligment, @sub-segment Alignment, and @startWithSAP are used jointly in switching Representations. Usage examples of those attributes are provided in appendix A. 6 of the DASH specification in both DASH-part 1 and 3GP-DASH.

In DASH, @mediaStreamStructureId is one of the attributes in Representation, which indicates a list of identifier values of media stream structure. According to the description of the attribute in DASH specification, sharing same media stream structure identifier has a specific indication with regard to SAP (Stream Access Point) as specified in the description of @mediaStreamStructureId. The attribute is used to provide bitstream structure identifier values list, which is used for the purpose of switching between different streams. With the bitstream structure identifier values list, whether two bitstreams have a same bitstream structure can be determined. Bitstream switching can occur at a certain RAP (Random Access Point) . A use case of multiple media stream structure identifier values has been noted in the description of @mediaStreamStructureId as described below. Specifically, in a certain bitstream switching use case using multiple bitstream structure identifier values, if Representation B shares a bitstream structure identifier value, which is present in Representation A and Representation C, then bitstream switching between Representations A and B, and switching between Representations B and C are allowed. However, if Representations A and C do not share a same bitstream structure ID, then switching between different Representations A and C are not allowed. In other words, bitstream switching between Representations A and C across B could be realized using multiple media stream structure identifier values.

In DASH, a set of Quality of Experience (QoE) metrics have been specified, where Representation switch events define a list of Representation switch events. In the Representation switch events, mt is specified which denotes the media presentation time of the earliest access unit of the Representation being switched to.

SUMMARY

A method of switching Representation for a client device and a server is disclosed. For the client device, a current Representation and one or more switched-to Representations associated with media-streaming are identified. The client device then transmits a single command to the server to request switching Representation to the one or more switched-to Representations. The single command comprises switch-Representation information to identify the one or more switched-to Representations or to identify a media presentation time list associated with the one or more switched-to Representations. For the server pushing a current representation associated with media-streaming to a client device, a single command related to switching Representation associated with the media-streaming from the client device is received. The server parses the single command to determine switch-Representation information for identifying the one or more switched-to Representations associated with the media-streaming, or for identifying a media presentation time list associated with the one or more switched-to Representations. The server then pushes the one or more switched-to Representations associated with the media-streaming to the client device. The single command can be transmitted from the client device to the server using Websocket protocol or HTTP/2 (Hypertext Transfer Protocol 2) .

The one or more switched-to Representations may comprise a target Representation and one or more intermediary Representations between the current Representation and the target Representation, and wherein the server pushes said one or more intermediary Representations before the target Representation. The switch-Representation information may comprise ordered list of intermediary Representation IDs for the intermediary Representations and target Representation ID for the target Representation. Alternatively, the switch-Representation information may comprise an ordered list of one or more first parameter pairs for the intermediary Representations and a second parameter pair for the target Representation. Each first parameter pair consists of one intermediary Representation ID and one media unit number of one media unit of a corresponding intermediary Representation to be pushed by the server. The second parameter pair consists of a target Representation ID and a media unit number of a media unit of the target Representation to be pushed by the server. The media unit may correspond to one segment or one sub-segment depending on a level-switch parameter.

The media presentation time list may comprise a list of media presentation times of earliest access units of the intermediary Representations and the target Representation. Alternatively, the media presentation time list may comprise a list of paired parameters associated with media presentation times of earliest access units of the intermediary Representations and the target Representation. Each paired parameter consists of one media presentation time of one earliest access unit of one corresponding intermediary Representation and one corresponding switched-to Representation ID, or consists of an earliest access unit of the target Representation and a target Representation ID. The intermediary Representations can be determined based on Media Presentation Description (MPD) .

BRIEF DESCRIPTION OF DRAWINGS

Fig. 1 illustrates an example of communication protocol between a server and a client for switching between two Representations.

Fig. 2 illustrates an exemplary scenario of “server push” , where the server initially pushes segment 11 to the client, where the bandwidth of the Representation corresponds to 1,800 kbps.

Fig. 3 illustrates an exemplary switching Representation process according to one embodiment the present invention.

Fig. 4 illustrates an exemplary flowchart for switching Representation in a client device according to an embodiment of the present invention.

DETAILED DESCRIPTION

The following description is of the best-contemplated mode of carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.

In low latency streaming, fast Representation switching is a very important and critical feature for preventing buffer draining-up and playback interruptions. In low latency streaming, buffered media time is typically very limited to provide low latency in buffering media data to reach a certain level for example prior to start playback. Limited buffered media data imposes a strict requirement on the delay for switching Representation upon detecting congestion. The delay for switching Representation is referring to the media playback time duration from the moment to decide switching Representation to the moment to reach the target level of Representation.

The specification in M33623 may be used to provide basic switching functionalities between two Representations as discussed below. For example, a client may send command 5 to stop the transmission of the resource corresponding to a certain Representation, (e.g. Representation with repid equal to 1) and then send command 3 to get all segments of Representation by setting “repid” equal to taget_repid (e.g. 3) starting from a certain segment number by setting “segnum” equal to a target_segnum or alternatively setting “timestamp” equal to a target_timestamp. Fig. 1 illustrates an example of communication protocol 100 between a server and a client for switching between two Representations. The client sends command 3 to get the segments of Representation with repid equal to 1 as indicated by reference number 110. The service will acknowledge the request with “OK” (120) . In order to switch to another Representation, the client sends command 5 to stop the transmission of Representation with repid equal to 1 as indicated by reference number 130. The server will acknowledge the request with “OK” (140) . The client sends command 3 to get the segments of Representation with repid equal to 2 as indicated by reference number 150. The service will acknowledge the request with “OK” (160) . However, such a switching process has several shortcomings in preventing from fast switching as described below.

First, in the scenario of “server push” , where the server pushes the frames of a certain stream, the Representation associated with the stream is unknown to the client, when the start streaming command identified by command 3 arrives at the server due to transmission delay. Therefore, the client has to set a “segnum” or “timestamp” to a conservative value (i.e., a larger value) . Hence, the time of switching is delayed and results in late switching.

Fig. 2 illustrates an exemplary scenario of “server push” , where the server initially pushes segment 11 (i.e., segnum＝11) to the client as indicated by reference number 200, where the bandwidth of Representation corresponds to 1800 (i.e., 1,800 kbps) . The client desires to switch to transmission at bandwidth 500 (i.e., 500 kbps) . The corresponding switching Representation process according to the conventional MPEG-DASH or 3GP-DASH is shown by reference number 210. During data transmission of segment 12, the client sends a request to cancel transmission, i.e. to stop the transmission once segment 12 is transmitted, followed by another request with push-next directive (i.e., segment 13) corresponding to the bandwidth of Representation at 1200 (i.e., 1,200 kbps) as indicated by reference number 220. Next, the client sends a request to cancel transmission, i.e. to stop the transmission once segment 13 is transmitted, followed by another request with push-next directive (i.e., segment 14) corresponding to the bandwidth of Representation at 800 (i.e., 800 kbps) as indicated by reference number 230. Further next, the client sends a request to cancel transmission, i.e. to stop the transmission once segment 14 is transmitted, followed by another request with push-next directive (i.e., segment 15) corresponding to the bandwidth of Representation at 500 (i.e., 500 kbps) as indicated by reference number 240. As shown in Fig. 2, multiple segment requests have to be initiated using the existing MPEG-DASD or 3GP-DASH. The process is time consuming.

Second, it is not possible to provide an efficient way of signaling of switching Representations across one or multiple Representations to reach the final target Representation between the server and the client with a single command. However, the above feature is necessary for allowing the server to quickly switching to target Representation.

Third, the information available for client to select a stream access point is much limited for the client as the client typically relies on the MPD information to provide switching. Instead, the server can access the segment to provide more flexible Representation switching functions.

Switching Representations across one or multiple Representations provides an efficient way of switching Representations quickly to reach target Representation by efficiently utilizing @mediaStreamStructureId as described in MPEG-DASH, 3GP-DASH. By using multiple bitstream structure identifier values, it is possible to switch from Representation A to Representation C across Representation B at certain RAPs (Random Access Points) . Otherwise, it is not possible to directly switch from Representation A to Representation C as disclosed in MPEG-DASH and 3GP-DASH. In addition, single command based multiple Representations switching could save time to send multiple commands for reach target Representations in Representation switching.

It is desirable to provide a feature for signaling of switching Representations across one or multiple Representations to reach the final target Representation between the server and the client with a single command. Therefore, the server can across one or multiple Representations at RAPs by efficiently utilizing the mediaStreamStructureId attribute of DASH. The above feature is very important to allow the server to switch Representations quickly and avoid buffer draining.

To cope with the above mentioned problems, the present invention discloses two embodiments of fast signaling of switching Representations in Websocket protocol or HTTP/2 between the server and the DASH client. The server may be a HTTP/2 server or a Websocket server that provides DASH segment delivery function.

According to the first embodiment, signaling of Representation switching from the DASH client to the server in Websocket protocol or HTTP/2 to enable the server to switch to one or multiple intermediary Representations prior to switching to target Representation is disclosed. In one example, a new command code, denoted as switchRepresentation, is specified to indicate the signaling of switching Representation. The new command may be included in the dash-aware sub protocol of the Websocket protocol. According to the first embodiment for fast Representation switching, three new parameters, denoted as targetRepresentationID, intermediaryRepresentationIDList and SwitchLevel are signaled together with the command code from the DASH client to the server. The targetRepresentationID denotes the target Representation ID that the server will switch to. The intermediaryRepresentationIDList denotes an ordered list of intermediary Representation IDs that the server needs to switch to prior to reach the target Representation ID as specified in targetRepresentationID. Parameter switchLevel denotes that the server may switch at segment boundary or sub-segment boundary. Or alternatively, the intermediaryRepresentationIDList may denote an ordered list of paired parameters including intermediary Representation IDs and pushing media unit numbers in the corresponding Representations. For example, the media unit may be interpreted as a segment or sub-segment depending on the value of switchLevel. If switchLevel is set to 0 indicating the switching may occur at each segment boundary by the server. The media unit may be interpreted as segment. Otherwise, if switchLevel is set to 1 indicating that the switching may occur at each sub-segment boundary by the server, then the media unit may be interpreted as sub-segment.

In another example of the first embodiment, the command code of signaling of Representation switching and the required parameters associated with the command code from the DASH client to the server are described as follows.

Table 2.

As mentioned in Table 2, the intermediaryRepresentationIDList may denote an ordered list of paired parameters including intermediary Representation IDs and pushing media unit numbers in the corresponding Representations. The media unit may be interpreted as a segment or sub-segment depending on the value of switchLevel as mentioned above. The switching Representation protocol as shown in Table 2 can be applied to the case shown in Fig. 2. An example of fast switching Representation process according to the above embodiment is shown in Fig. 3. The conditions prior to the switching Representation process are the same as Fig. 2, where segment 12 is being pushed when the directive arrives at the server. According to the embodiment, single push-Representation-switch directive with proper parameters are transmitted. In order to result in smooth switch as Fig. 2, the directive may contain two parameter pairs for two intermediary Representations and one parameter pair for the target Representation. The two paired parameters for the intermediary Representations correspond to one segment (i.e., 13) from Rp@bw1200 (i.e., ID ＝ 5) and one segment (i.e., 14) from Rp@bw800 (i.e., ID ＝ 4) . The target Representation corresponds to one segment (i.e., 15) from Rp@bw500 (i.e., ID ＝ 3) . Accordingly, the single push-Representation-switch directive can be represented by {5 1, 4 1} ； 3 1 as shown in Fig. 3, or can be represented by {5 1, 4 1, 31} .

The parameter of switchLevel may be specified as follows.

Table 3.

In the first embodiment, the DASH client may set the intermediaryRepresentationIDList and switchLevel based on the information of Media Presentation Description (MPD) and signal those parameters together with the associated command code (i.e., switchRepresentation) to the server. One example is described as follows to illustrate how a DASH client can set the intermediaryRepresentationIDList and switchLevel based on the information of MPD specified in DASH. For example, the switchLevel may be set to 0 to indicate that the switching Representation may occur at each segment boundary by the DASH client if each segment starts with SAP indicated by @startWithSAP and each segment is aligned to switch between different Representations indicated by @segmentAligmentas specified bythe DASH. Otherwise, if each sub-segment starts with SAP indicated by @subsegmentStartsWithSAPand each sub-segment is aligned to switch between different Representations as indicated by @subsegmentAlignmentas specified bythe DASH, then the switchLevel may be set to 1 to indicate the switching Representation may occur at each sub-segment boundary by the DASH client.

In case that @startWithSAP is equal to 3 and/or @subsegmentStartsWithSAP is equal to 3, the DASH client further checks @mediaStreamStructureId, as specified in DASH, of Representations between the current Representation and the target Representation. The DASH can then identify a list of intermediary Representations that share an identical @mediaStreamStructureId value. Accordingly, Representations can be switched from a current Representation to the target Representation across the list of intermediary Representations. The DASH client may set @intermediaryRepresentationIDList as the identified list of intermediary Representations. In the case that @subsegmentsStartsWithSAP is equal to 1 or 2, the DASH client may set @intermediaryRepresentationIDList to empty to indicate that the server can directly switch from the current Representation to the target Representation without reaching any intermediary Representation prior to switching to target Representation. Alternatively, in the case that @subsegmentStartsWithSAP is equal to 1 or 2, the DASH client may set @intermediaryRepresentationIDList to a list of Representations between the current Representation and the target Representation. The setting is chosen if the DASH client prefers step-wise Representation switching for smooth quality change or and relies on a rate adaptation algorithm. The DASH client may also further set @intermediaryRepresentationIDList to a list of paired values including Representations and pushing media unit numbers in the corresponding Representations for providing more smooth quality change. Then, the DASH client sends the Representation switching command code with the associated parameters targetRepresentationID, intermediaryRepresentationIDList, and SwitchLevel to the server for providing fast switching Representations.

In the first embodiment, the server parses the targetRepresentationID to identify the target Representation, the intermediaryRepresentationIDList to identify the list of intermediary Representations to be reached before the target Representation, and switchLevel to identify whether the switching Representations are at each segment boundary or sub-segment boundary according to the above description and tables. Or additionally, the server parses intermediaryRepresentationIDList to identify the list of intermediary Representations to be reached before the target Representation and to push media unit numbers in the corresponding Representations. The server may push segments or sub-segments for intermediary Representations and target Representation in the stream of Websocket protocol or HTTP/2, which are used to push segments or sub-segments of the original Representation. The server may also create a new stream in Websocket protocol or HTTP/2 for pushing segments or sub-segments of intermediary Representations or target Representation. The server may close the streams that are used to push media data of previous Representations. Alternatively, the server may close original streams that are used to push the current Representation and create a new stream which is used to push segments or sub-segments of the intermediary Representations and target Representation.

In the second embodiment, signaling of a list of media presentation times of the earliest access units in a list of switched-to Representations from the server to the DASH client in Websocket protocol or HTTP/2 is disclosed. The request for transmitting a list of media presentations times is initiated /sent from the client to the server, so that the server can send the corresponding list of media presentations back to the client. In other words, the signaling (request) is from client to server, whilst the actual data (i.e., media of various presentations) are from server to client. The signaling will take place when the server may switch to one or multiple intermediary Representations and then switch to the target Representation. The one or multiple intermediary Representations and the target Representation are denoted as switched-to Representations. The server may be a HTTP/2 server or a Websocket server that provides DASH segment delivery function. In the second embodiment, the signaling of a list of media presentation times of the earliest access units in the list of switched-to Representations may include enough information to allow the client to identify or derive the list of media presentation times of the earliest access units in the list of switched-to Representations. The list of media presentation times of the earliest access units in the list of switched-to Representations may be used to specify the media presentation time of the earliest access unit played from the switched-to Representation denoted as mt in the process of creating the Representation switch events of QoE in DASH.

In one example of the second embodiment, a new command code may be signaled to indicate a list of media presentation times of the earliest access unit in the list of switched-to Representations. The new command may be included in the dash-aware sub protocol of Websocket protocol. A new parameter, denoted as mediaTimesListToReprsentations, can be signaled together with the new command code from the client to the DASH server. The new parameter may include a list of media presentation times of the earliest access units in the list of switched-to Representations. Alternatively, the parameter mediaTimesListToReprsentationsmay include a list of URLs of the earliest segment or sub-segment in the list of switched-to Representations. The process of including a list of URLs of the earliest segments or sub-segments in mediaTimesListToReprsentations may depend on the targetRepresentationID, intermediaryRepresentationIDList, and SwitchLevel discussed in the first embodiment. Based on the signaled list of URLs of the earliest segments or sub-segments in the list of switched-to Representations, the DASH client may specify the media presentation times of the earliest access units in the list of switched-to Representations.

The command code of signaling of the list of media presentation times of the earliest access units in the switched-to Representations and the required parameters associated with the command code from the server to the DASH client are described in Table 4.

Table 4.

The mediaTimesListToReprsentations may be used to identify a list of media presentation times of the earliest access units in the list of switched-to Representations. The identified list of presentation times of the earliest access units in the list of switched-to Representations may be used to specify the media presentation time, mt of the earliest access unit played from the switched-to Representation in the process of creating the Representation switch events of QoE in DASH. The mt of the Representation switch event may be set to the media presentation time of the earliest access unit in the target Representation. Alternatively, mt and tr of the Representation switch event may be modified so that mt denotes a list of media presentation times of the earliest access units in all switched-to Representations, including all intermediary Representations and target Representation, and tr denotes a list of the switched to representations. Then, the mt and tr may be set accordingly based on the signaled mediaTimesListToReprsentations. For example, the signaled mediaTimesListToReprsentations may correspond to a list of presentation times of the earliest access units in the list of switched-to Representations and list of the corresponding switched-to Representations.

The server may switch to one or multiple intermediary Representations prior to switching to the target Representation at the boundary of segments or sub-segments based on the targetRepresentationID, intermediaryRepresentationIDList, and SwitchLevel according to the description of the first embodiment. The server may add the media presentation time of the earliest access unit of the new Representation to the mediaTimesListToReprsentations each time upon switching to a new intermediary or target Representations. The server may send the mediaTimesListToReprsentations together with the command code to the DASH client upon switching to target Representation.

The DASH client may parse the mediaTimesListToReprsentations to identify or derive a list of media presentation times of the earliest access units in the switched-to Representations, which may be used to specify the media presentation time, mt of the earliest access unit played from the switched-to Representation in the process of creating the Representation switch events of QoE in DASH.

Please note, the names of the foregoing parameters are for illustration purpose only, but not for limiting the scope of the present invention.

Fig. 4 illustrates an exemplary flowchart for switching Representation in a client device according to an embodiment of the present invention. The client device determines a current representation associated with media-streaming received from a server in step 410. One or more switched-to representations associated with the media-streaming are determined in step 420. The client device then transmits a single command to the server to request switch representation to said one or more switched-to representations in step 430, wherein the single command comprises switch-representation information to identify said one or more switched-to representations, or to identify media presentation time list associated with said two or more switched-to representations.

The flowchart shown above is intended to illustrate examples of fast switching Representation for media streaming incorporating an embodiment of the present invention. A person skilled in the art may modify each step, re-arranges the steps, split a step, or combine the steps to practice the present invention without departing from the spirit of the present invention.

The above description is presented to enable a person of ordinary skill in the art to practice the present invention as provided in the context of a particular application and its requirement. Various modifications to the described embodiments will be apparent to those with skill in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features herein disclosed. In the above detailed description, various specific details are illustrated in order to provide a thorough understanding of the present invention. Nevertheless, it will be understood by those skilled in the art that the present invention may be practiced.

Embodiment of the present invention as described above may be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present invention can be a circuit integrated into a video compression chip or program code integrated into video compression software to perform the processing described herein. An embodiment of the present invention may also be program code to be executed on a Digital Signal Processor (DSP) to perform the processing described herein. The invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA) . These processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention. The software code or firmware code may be developed in different programming languages and different formats or styles. The software code may also be compiled for different target platforms. However, different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.

The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims

A method of switching representation for a client device, the method comprising:

determining a current representation associated with media-streaming received from a server；

determining one or more switched-to representations associated with the media-streaming； and

transmitting a single command to the server to request switch representation to said one or more switched-to representations, wherein the single command comprises switch-representation information to identify said one or more switched-to representations, or to identify a media presentation time list associated with said one or more switched-to representations.
The method of Claim 1, wherein the single command is transmitted to the server using Websocket protocol or HTTP/2 (Hypertext Transfer Protocol 2) .
The method of Claim 1, said one or more switched-to representations comprise a target representation and one or more intermediary representations between the current representation and the target representation, and wherein the server pushes said one or more intermediary representations before the target representation.
The method of Claim 3, wherein the switch-representation information comprises an ordered list of intermediary representation IDs (identifications) for said one or more intermediary representations and target representation ID (identification) for the target representation.
The method of Claim 3, wherein the switch-representation information comprises an ordered list of one or more first parameter pairs for said one or more intermediary representations and a second parameter pair for the target representation, wherein each first parameter pair consists of one intermediary representation ID (identification) and one media unit number of one media unit of a corresponding intermediary representation to be pushed by the server and the second parameter pair consists of a target representation ID and a media unit number of a media unit of the target representation to be pushed by the server.
The method of Claim 5, wherein one media unit of the corresponding intermediary representation and the media unit of the target representation correspond to one segment or one sub-segment.
The method of Claim 6, wherein a level-switch parameter is signaled to indicated whether switching representation occurs at a segment boundary or a sub-segment boundary.
The method of Claim 3, wherein the media presentation time list comprises a list of media presentation times of earliest access units of said one or more intermediary representations and the target representation.
The method of Claim 3, wherein the media presentation time list comprises a list of paired parameters associated with media presentation times of earliest access units of aid one or more intermediary representations and the target representation, wherein each paired parameter consists of one media presentation time of one earliest access unit of one corresponding intermediary representation and one corresponding switched-to representation ID (identifier) , or consists of an earliest access unit of the target representation and a target representation ID.
The method of Claim 3, wherein said one or more intermediary representations are determined based on Media Presentation Description.
The method of Claim 1, wherein the media presentation time list comprises a list of media presentation times of earliest access units of said one or more switched-to representations.
The method of Claim 1, wherein the media presentation time list comprises a list of paired parameters associated with media presentation times of earliest access units of said one or more switched-to representations, wherein each paired parameter consists of one media presentation time of one earliest access unit of one corresponding switched-to representation and one corresponding switched-to representation ID (identifier) .
The method of Claim 1, wherein the media presentation time list is determined based on a second media presentation time list received from the server.
The method of Claim 1, wherein the media presentation time list is used to derive media presentation times of earliest access units played from the switched-to representations respectively in a process of creating representation switch events of Quality of Experience (QoE) in Dynamic Adaptive Streaming over HTTP.
A client device for receiving media streaming from a server, the client device comprising:

an interface arranged to establish a communication connection with a server； and

one or more electronic circuits configured to:

determine a current representation associated with media-streaming received from the server；

determine one or more switched-to representations associated with the media-streaming； and

transmit a single command to the server to request switch representation to said one or more switched-to representations, wherein the single command comprises switch-representation information to identify said one or more switched-to representations, or to identify a media presentation time list associated with said one or more switched-to representations.
A method of switching representation for a server, the method comprising:

pushing a current representation associated with media-streaming to a client device；

receiving a single command related to switching representation associated with the media-streaming from the client device；

parsing the single command to determine switch-representation information for identifying one or more switched-to representations associated with the media-streaming, or for identifying a media presentation time list associated with said one or more switched-to representations； and

pushing said one or more switched-to representations associated with the media-streaming to the client device.
The method of Claim 16, wherein the single command is received from the client device using Websocket protocol or HTTP/2 (Hypertext Transfer Protocol 2) .
The method of Claim 16, said one or more switched-to representations comprise a target representation and one or more intermediary representations between the current representation and the target representation, and wherein the server pushes said one or more intermediary representations before the target representation.
The method of Claim 18, wherein the switch-representation information comprises an ordered list of intermediary representation IDs (identifications) for said one or more intermediary representations and target representation ID (identification) for the target representation.
The method of Claim 18, wherein the switch-representation information comprises an ordered list of one or more first parameter pairs for said one or more intermediary representations and a second parameter pair for the target representation, wherein each first parameter pair consists of one intermediary representation ID (identification) and one media unit number of one media unit of a corresponding intermediary representation to be pushed by the server and the second parameter pair consists of a target representation ID and a media unit number of a media unit of the target representation to be pushed by the server.
The method of Claim 20, wherein one media unit of the corresponding intermediary representation and the media unit of the target representation correspond to one segment or one sub-segment.
The method of Claim 21, wherein a level-switch parameter is signaled to indicated whether switching representation occurs at a segment boundary or a sub-segment boundary.
The method of Claim 18, wherein the media presentation time list comprises a list of media presentation times of earliest access units of said one or more intermediary representations and the target representation.
The method of Claim 18, wherein the media presentation time list comprises a list of paired parameters associated with media presentation times of earliest access units of aid one or more intermediary representations and the target representation, wherein each paired parameter consists of one media presentation time of one earliest access unit of one corresponding intermediary representation and one corresponding switched-to representation ID (identifier) , or consists of an earliest access unit of the target representation and a target representation ID.
The method of Claim 18, wherein said one or more intermediary representations are determined based on Media Presentation Description.
The method of Claim 16, wherein the media presentation time list comprises a list of media presentation times of earliest access units of said one or more switched-to representations.
The method of Claim 16, wherein the media presentation time list comprises a list of paired parameters associated with media presentation times of earliest access units of said one or more switched-to representations, wherein each paired parameter consists of one media presentation time of one earliest access unit of one corresponding switched-to representation and one corresponding switched-to representation ID (identifier) .
The method of Claim 16, wherein the media presentation time list is determined based on a second media presentation time list received from the server.
The method of Claim 16, wherein the media presentation time list is used to derive the media presentation times of earliest access units played from the switched-to representations respectively in a process of creating representation switch events of Quality of Experience (QoE) in Dynamic Adaptive Streaming over HTTP.
A method of media communication for a server and a client device, the method comprising:

transmitting a current representation associated with media-streaming from a server to a client device；

determining, by the client device, one or more switched-to representations associated with the media-streaming；

transmitting a single command from the client device to the server to request switch representation to said one or more switched-to representations, wherein the single command comprises switch-representation information to identify said one or more switched-to representations, or to identify a media presentation time list associated with said one or more switched-to representations；

parsing the single command, at the client device, to determine the switch-representation information for identifying said one or more switched-to representations associated with the media-streaming, or for identifying said media presentation time list associated with said one or more switched-to representations； and

pushing said one or more switched-to representations associated with the media-streaming from the server to the client device.