WO2017063574A1

WO2017063574A1 - Streaming media adaptive transmission method and device

Info

Publication number: WO2017063574A1
Application number: PCT/CN2016/102032
Authority: WO
Inventors: 黄成�
Original assignee: 中兴通讯股份有限公司
Priority date: 2015-10-14
Filing date: 2016-10-13
Publication date: 2017-04-20
Also published as: CN106604077B; CN106604077A

Abstract

Provided are a streaming media adaptive transmission method and device. The method comprises: transmitting to a server an acquisition request requesting first media data, wherein the acquisition request carries an instruction to push second media data; receiving the first media data returned by the server and a push response associated with the second media data, and receiving a push promise associated with the second media data and transmitted by the server according to the push response; and receiving the second media data pushed by the server according to the push promise. The invention solves a problem in which accessibility is limited when a server push feature is used to perform video streaming media transmission, thereby realizing high efficiency and low delay full-duplex transmission protocol-based streaming media adaptive transmission.

Description

Adaptive streaming media transmission method and device

Technical field

The present invention relates to the field of communications, and in particular to an adaptive streaming media transmission method and apparatus.

Background technique

In recent years, the content of video transmitted through the Internet has experienced explosive growth, which is due to the increase in access bandwidth, including the scale application of 3G and 4G systems, Digital Subscriber Line (xDSL) and fiber optic connection. A large number of deployments for home users have effectively increased the access bandwidth of the last mile; on the other hand, the progress of video coding has improved the compression ratio of video, H.264/AVC, H.265/HEVC are all aspects of this. representative. According to statistics, in 2014, HTTP video data has accounted for more than 50% of Internet traffic in North America. It is estimated that by 2020, mobile video data traffic will account for more than 70% of mobile data traffic.

HTTP streaming (HTTP Streaming) is widely used for its compatibility and scalability, and has become the de facto standard for Internet video transmission. HTTP streaming is a rate-adaptive media transmission technology based on the HTTP protocol, which uses the traditional HTTP web server on the Internet to provide high-quality streaming media content. HTTP streaming divides a video into chunks of files on many small HTTP servers, each of which contains a video content with a shorter playback time. These small video clips may be part of a few hours of movie (on demand) or live video (live) of a sports event. Among them, the video can be encoded into a plurality of video segments of different code rates to cover different network bandwidth requirements.

However, the HTTP protocol was originally designed for file transfer (such as Web pages). It does not consider the real-time transmission characteristics of video, so it cannot meet the requirements of low-latency video transmission: First, HTTP streaming is a unit of media segment (segment). File download technology, a segment contains several seconds of video frames, which will lead to unavoidable media segmentation and encoding delays; secondly, although the Content Delivery Network (CDN) has been widely deployed to provide users with Near access service, but asynchronous data acquisition based on client polling (Poll) will still bring additional delay; finally, Figure 1 is a schematic diagram of PULL mode transmission in HTTP streaming technology according to the related art, as shown in Figure 1. The existing HTTP 1.1 protocol provides a pull mode (PULL) transmission mechanism. The client must send an HTTP request for each media segment. The HTTP request has a large overhead and is inversely proportional to the duration of the media segment playback. Limited mobile terminal. Therefore, efficient (low latency, reasonable request) video transmission under Internet conditions, especially Live Video applications, is still a huge challenge for HTTP streaming.

Currently, an upgraded version of the HTTP protocol, HTTP 2.0, provides a viable solution. Compared with HTTP 1.1, HTTP 2.0 improves the transmission performance of the protocol, supporting request-response multiplexing, protocol header compression, and server "Server Push" features. 2 is a schematic diagram of PUSH mode transmission in the HTTP streaming technology according to the related art. As shown in FIG. 2, using the HTTP 2.0 server “push”, the server can actively push the latest available fragment to the client without the client. Additional requests can greatly reduce transmission overhead and latency. Specifically, after processing a certain media segment HTTP request, the HTTP 2.0 server will agree with the client on the subsequent media segment push strategy (Push Strategy), and actively push subsequent available media segments to the client according to the push policy.

The Media Presentation Description (MPD) defined by the Extensible Markup Language (XML) format provides location information and time information for the client to acquire and play media segments in the video content. The MPD file describes the duration of the media segment, the Uniform Resource Locator (URL), and media properties such as video resolution, adaptation bitrate, and so on. When the client chooses to play a video, it can select and download the media segment adapted to the currently available network bandwidth and terminal decoding and rendering capabilities according to the information in the MPD file.

The client can access the corresponding media segment based on the URL information provided in the MPD file only within the media clip availability time range. The @availabilityStartTime field and the @availabilityEndTime field are defined in the MPD file to provide the earliest start time and the latest end time of streaming media playback, respectively. For the live video application, the client can calculate the availability start time and the availability termination time of each media segment according to the time information in the MPD file, and any media segment can only be accessed between the two time points.

In order to provide a smooth video streaming experience, the client usually needs to obtain and parse the MPD file to create the current client-local time (NOW) based on the local clock time and the server-side global clock time synchronization. A list of media segments available for access, and a media segment acquisition request is sent to the HTTP server according to the list. FIG. 3 is a schematic diagram 1 of media segment accessibility in the HTTP streaming media technology according to the related art. As shown in FIG. 3, when requesting to acquire a media segment, the client should according to the local clock time value tnow and the MPD file synchronized with the server end. Medium time information, for example, the start time, duration, etc. of the media segment relative to the period (Period), the calculation availability start time is less than or equal to tnow and the availability termination time is greater than or equal to the tnow media segment, and the currently accessible media segment list is obtained. Further determining the next request to obtain the media segment URL (ie, the media segment corresponding to the tnext time number #N in the figure), and triggering the corresponding HTTP request operation.

Differently, with the server "push" feature, the streaming media system implemented by HTTP 2.0 protocol binding allows the server to actively push subsequent media segments according to the "pushing policy" reached with the client, so as to reduce the live broadcast delay and reduce the request. The purpose of the overhead. 4 is a second schematic diagram of media segment accessibility in the HTTP streaming media technology according to the related art. As shown in FIG. 4, the client may request to push subsequent K media segments or push a subsequent media segment with a playback time of T seconds. . However, the "availability" and the "pushing policy" of the media segment transmitted in the push mode are independent of each other, that is, the media segment requested to be pushed in the "pushing policy" may not be all available at the time of sending the request by the client (ie, the number in the figure is #N+J+1~#N+K media clips). Therefore, how to achieve the video streaming media transmission while fully utilizing the "push" feature of the HTTP 2.0 server, satisfying the accessibility limitation of the media segment is a problem that needs to be studied and solved at present.

In view of the related art, the problem of accessibility limitation of push media data existing when video streaming is performed by using the server "push" feature has no effective solution at present.

Summary of the invention

The present invention provides an adaptive streaming media transmission method and apparatus to at least solve the problem of accessibility limitation of push media data existing when video streaming media transmission is performed by using the server "push" feature in the related art.

According to an aspect of the present invention, an adaptive streaming media transmission method is provided, including:

Sending a first media data acquisition request to the server, where the obtaining request carries a push indication of the second media data;

Receiving a push response of the first media data and the second media data returned by the server, and receiving a push commitment of the second media data sent by the server according to the push response;

Receiving, by the server, the second media data that is pushed according to the push commitment.

Optionally, before sending the first media data acquisition request to the server, the method includes:

The push indication of the second media data includes push condition information, and the push condition information is used to request the server to send a push commitment for the media data that is not accessible in the push indication of the second media data.

Optionally, the method includes:

The push condition information is set as a push wait time, and the push wait time is used to set an upper limit of a waiting time for the push commitment of the second media data to be sent by the server after receiving the first media data acquisition request.

Optionally, the method includes:

The push condition information is set to a push wait time with a value of zero, and the push wait time is used to request the server to push all the media data referenced in the push indication of the second media data.

Optionally, the method includes:

Setting the push condition information as a push threshold, where the push threshold is used to set the server to use the first media data as an initial index, and push a lower limit of the quantity of media data in the second media data; or, the push The threshold is used to set a lower limit of the media time of the media data in the second media data from the media time of the first media data.

Optionally, receiving a push response of the second media data returned by the server includes:

Receiving a push response of the second media data returned by the server, and the push response of the second media data includes push condition information requested in the second media data push instruction.

Optionally, receiving a push commitment of the second media data that is sent by the server according to the push response includes:

Receiving, by the server, a push commitment of the second media data that is sent according to a push parameter value returned by the push response.

Receiving, by the server, the push commitment of the second media data that is sent according to the push condition information returned by the push response.

According to another aspect of the present invention, an adaptive streaming media transmission method is further provided, including:

Receiving an acquisition request of the first media data sent by the client, where the obtaining request carries a push indication of the second media data;

Returning a push response of the first media data and the second media data to the client, and sending a push commitment of the second media data to the client according to the push response;

Pushing the second media data to the client according to the push commitment.

Optionally, before returning the push response of the second media data to the client, the method includes:

And setting, in the push response of the second media data, push condition information including a push instruction request of the second media data, and sending, for sending, the inaccessible media data referenced in the push instruction of the second media data committed to.

Optionally, sending the push commitment of the second media data to the client according to the push response includes:

Transmitting, according to the push parameter value returned in the push response, a push commitment of the second media data to the client;

Sending a push commitment of the second media data to the client according to the push condition information returned in the push response.

Optionally, sending the push commitment of the second media data to the client according to the push condition information returned in the push response includes:

When the push condition information is the push waiting time, the push commitment of the second media data is sent to the client within the push waiting time from receiving the first media data acquisition request.

When the push condition information is a push waiting time with a value of zero, all media data referenced in the push instruction of the second media data is pushed to the client.

When the push condition information is a push threshold, the first media data is used as an initial index, and at least the media data in the second media data with the push threshold is a lower limit is pushed to the client; Or, at least the media data in the second media data with the push threshold being the media time lower limit is pushed to the client from the media time of the first media data.

According to another aspect of the present invention, an adaptive streaming media transmission device is further provided on the terminal side, including:

The first sending module is configured to send a first media data obtaining request to the server, where the obtaining request carries a push indication of the second media data;

a first receiving module, configured to receive a push response of the first media data and the second media data returned by the server, and receive a push of the second media data sent by the server according to the push response committed to;

The second receiving module is configured to receive the second media data that is sent by the server according to the push commitment.

Optionally, the device further includes:

a first setting module, configured to set, before sending the first media data acquisition request to the server, push condition information included in the push indication of the second media data, where the push condition information is used to request the server to be the first The non-accessible media data referenced in the push indication of the second media data sends a push commitment.

Optionally, the first setting module includes:

a first setting unit, configured to set the push condition information to be a push wait time, where the push wait time is used to set the server to send the push of the second media data after receiving the first media data acquisition request The maximum waiting time for commitment.

Optionally, the first setting module includes:

a second setting unit, configured to set the push condition information to a push waiting time with a value of zero, the push waiting time for requesting the server to push all the media data referenced in the push indication of the second media data .

Optionally, the first setting module includes:

a third setting unit, configured to set the push condition information to be a push threshold, where the push threshold is used to set the server to use the first media data as an initial index, and push the quantity of media data in the second media data The lower limit is used; or the push threshold is used to set a lower limit of the media time of the media data in the second media data from the media time of the first media data.

Optionally, the first receiving module includes:

The first receiving unit is configured to receive a push response of the second media data returned by the server, and the push response of the second media data includes push condition information requested in the second media data push instruction.

Optionally, the first receiving module includes:

The second receiving unit is configured to receive a push commitment of the second media data that is sent by the server according to the push parameter value returned by the push response.

Optionally, the first receiving module includes:

The third receiving unit is configured to receive a push commitment of the second media data that is sent by the server according to the push condition information returned by the push response.

According to another aspect of the present invention, an adaptive streaming media transmission device is further provided on the server side, including:

a third receiving module, configured to receive a request for acquiring the first media data sent by the client, where the obtaining request carries a push indication of the second media data;

a fourth receiving module, configured to return a push response of the first media data and the second media data to the client, and send a push commitment of the second media data to the client according to the push response End

The second sending module is configured to push the second media data to the client according to the push commitment.

Optionally, the device further includes:

a second setting module, configured to set, before returning the push response of the second media data to the client, a push condition information including a push indication request of the second media data in a push response of the second media data And sending a push commitment for the inaccessible media data referenced in the push indication of the second media data.

Optionally, the second sending module includes:

The first sending unit is configured to send a data push commitment of the second media to the client according to the push parameter value returned in the push response;

The second sending unit is configured to send a push commitment of the second media data to the client according to the push condition information returned in the push response.

Optionally, the second sending module includes:

The third sending unit is configured to, when the push condition information is the push waiting time, send the push commitment of the second media data in the push waiting time from receiving the first media data obtaining request To the client.

Optionally, the second sending module includes:

The fourth transmitting unit is configured to push all the media data referenced in the push instruction of the second media data to the client when the push condition information is a push waiting time with a value of zero.

Optionally, the second sending module includes:

The fifth sending unit is configured to: when the push condition information is a push threshold, use the first media data as an initial index to push at least the media in the second media data with the push threshold as a lower limit Data is sent to the client; or, at least from the media time of the first media data, the media data in the second media data with the push threshold being the media time lower limit is pushed to the client.

Another embodiment of the present invention provides a computer storage medium storing execution instructions for performing the method in the above embodiments.

According to the present invention, the first media data acquisition request is sent to the server, and the acquisition request carries a push indication of the second media data; receiving the first media data returned by the server and the push response of the second media data, and receiving the Receiving, by the server, the push commitment of the second media data sent according to the push response; receiving the second media data pushed by the server according to the push commitment, and solving the push media data existing when the video streaming media is transmitted by using the server “push” feature The problem of accessibility limitation enables efficient, low-latency adaptive streaming media transmission based on full-duplex transmission protocol binding.

DRAWINGS

The drawings described herein are intended to provide a further understanding of the invention, and are intended to be a part of the invention. In the drawing:

FIG. 1 is a schematic diagram of PULL mode transmission according to the related art HTTP streaming technology;

2 is a schematic diagram of PUSH mode transmission in an HTTP streaming media technology according to the related art;

3 is a schematic diagram 1 of media segment accessibility in an HTTP streaming media technology according to the related art;

4 is a schematic diagram 2 of media segment accessibility in an HTTP streaming media technology according to the related art;

FIG. 5 is a flowchart 1 of an adaptive streaming media transmission method according to an embodiment of the present invention;

6 is a second flowchart of an adaptive streaming media transmission method according to an embodiment of the present invention;

7 is a structural block diagram 1 of an adaptive streaming media transmission apparatus according to an embodiment of the present invention;

FIG. 8 is a structural block diagram 2 of an adaptive streaming media transmission apparatus according to an embodiment of the present invention; FIG.

9 is a structural diagram of an adaptive streaming media system according to an embodiment of the present invention;

10 is a flowchart 1 of an adaptive streaming media transmission method according to an alternative embodiment of the present invention;

11 is a second flowchart of an adaptive streaming media transmission method according to an alternative embodiment of the present invention;

12 is a flowchart 3 of an adaptive streaming media transmission method according to an alternative embodiment of the present invention;

FIG. 13 is a flowchart 4 of an adaptive streaming media transmission method according to an alternative embodiment of the present invention; FIG.

14 is a flowchart 5 of an adaptive streaming media transmission method according to an alternative embodiment of the present invention;

15 is a flowchart 6 of an adaptive streaming media transmission method according to an alternative embodiment of the present invention;

16 is a flowchart 7 of an adaptive streaming media transmission method according to an alternative embodiment of the present invention;

17 is a flowchart 8 of an adaptive streaming media transmission method according to an alternative embodiment of the present invention;

FIG. 18 is a flowchart IX of an adaptive streaming media transmission method according to an alternative embodiment of the present invention; FIG.

19 is a flowchart 10 of an adaptive streaming media transmission method according to an alternative embodiment of the present invention;

20 is a flow chart 1 of message interaction of an adaptive streaming media transmission method according to an embodiment of the present invention;

21 is a flow chart 2 of message interaction of an adaptive streaming media transmission method according to an embodiment of the present invention;

FIG. 22 is a flowchart 3 of message interaction of an adaptive streaming media transmission method according to an embodiment of the present invention; FIG.

23 is a message interaction flowchart 4 of an adaptive streaming media transmission method according to an embodiment of the present invention;

FIG. 24 is a structural block diagram 3 of an adaptive streaming media transmission apparatus according to an embodiment of the present invention.

detailed description

The invention will be described in detail below with reference to the drawings in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict.

It is to be understood that the terms "first", "second" and the like in the specification and claims of the present invention are used to distinguish similar objects, and are not necessarily used to describe a particular order or order.

An adaptive streaming media transmission method is provided in this embodiment. FIG. 5 is a flowchart 1 of an adaptive streaming media transmission method according to an embodiment of the present invention. As shown in FIG. 5, the process includes the following steps:

Step S502, sending a first media data acquisition request to the server, where the acquisition request carries a push indication of the second media data;

Step S504: Receive a push response of the first media data and the second media data returned by the server, and receive a push commitment of the second media data sent by the server according to the push response.

Step S506, receiving the second media data that the server pushes according to the push commitment.

And sending, by the foregoing step, the first media data acquisition request to the server, where the acquisition request carries a push indication of the second media data; receiving the first media data returned by the server and the push response of the second media data, and receiving the Receiving, by the server, the push commitment of the second media data sent according to the push response; receiving the second media data pushed by the server according to the push commitment, and solving the push media data existing when the video streaming media is transmitted by using the server “push” feature The problem of accessibility limitation enables efficient, low-latency adaptive streaming media transmission based on full-duplex transmission protocol binding.

In this embodiment, before sending the first media data acquisition request to the server, the push indication of the second media data is set to include push condition information, where the push condition information is used to request the server to be a push indication of the second media data. The push data promised in the media data referenced in the reference.

In this embodiment, the push condition information is set as a push waiting time, and the push waiting time is used to set an upper limit of the waiting time for the push commitment of the second media data to be sent by the server after receiving the first media data obtaining request.

In this embodiment, the push condition information is set to a push waiting time with a value of zero, and the push waiting time is used to request the server to push all the media data referenced in the push indication of the second media data.

In this embodiment, the push condition information is set as a push threshold, and the push threshold is used to set the server to use the first media data as an initial index to push a lower limit of the quantity of media data in the second media data; or, the push The threshold is used to set a lower limit of the media time of the media data in the second media data from the media time of the first media data.

In this embodiment, receiving the push response of the second media data returned by the server includes: receiving a push response of the second media data returned by the server, where the push response of the second media data includes the second media data push indication Push condition information requested in .

In this embodiment, receiving the push commitment of the second media data sent by the server according to the push response comprises: receiving a push commitment of the second media data sent by the server according to the push parameter value returned by the push response; or And receiving a push commitment of the second media data sent by the server according to the push condition information returned by the push response.

FIG. 6 is a second flowchart of an adaptive streaming media transmission method according to an embodiment of the present invention. As shown in FIG. 6, the process includes the following steps:

Step S602, receiving an acquisition request of the first media data sent by the client, where the acquisition request carries a push indication of the second media data;

Step S604, returning the first media data and the push response of the second media data to the client, and sending a push commitment of the second media data to the client according to the push response;

Step S606, pushing the second media data to the client according to the push commitment.

Receiving, by the foregoing step, an acquisition request of the first media data sent by the client, where the acquisition request carries a push indication of the second media data, and returns a push response of the first media data and the second media data to the client, And sending, according to the push response, the push commitment of the second media data to the client; and pushing the second media data to the client according to the push commitment, and solving the existence of the video streaming media transmission by using the server “push” feature The problem of restricting the accessibility of push media data enables efficient, low-latency adaptive streaming media transmission based on full-duplex transmission protocol binding.

In this embodiment, before returning the push response of the second media data to the client, setting a push condition information including a push instruction request of the second media data in the push response of the second media data, for The push data commitment is not transmitted by the media data that is not accessible in the push indication of the second media data.

In this embodiment, sending the push commitment of the second media data to the client according to the push response includes: sending a push commitment of the second media data to the client according to the push parameter value returned in the push response; Sending a push commitment of the second media data to the client according to the push condition information returned in the push response.

In this embodiment, according to the push condition information returned in the push response, sending the push commitment of the second media data to the client includes:

When the push condition information is the push waiting time, the push commitment of the second media data is transmitted to the client within the push waiting time from the receipt of the first media data acquisition request.

In this embodiment, sending the push commitment of the second media data to the client according to the push condition information returned in the push response includes:

When the push condition information is a push threshold, the first media data is used as an initial index, and at least the media data in the second media data with the push threshold is a lower limit is pushed to the client; or The media data of the media data is at least pushed to the client in the second media data with the push threshold being the media time lower limit.

An adaptive streaming media transmission device is also provided in the embodiment, and the device is used to implement the foregoing embodiments and preferred embodiments, and details are not described herein. As used below, the term "module" may implement a combination of software and/or hardware of a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware, is also possible and contemplated.

FIG. 7 is a structural block diagram of an adaptive streaming media transmission apparatus according to an embodiment of the present invention. As shown in FIG. 7, the apparatus is located on the terminal side, and includes:

The first sending module 72 is configured to send a first media data obtaining request to the server, where the obtaining request carries a push indication of the second media data;

The first receiving module 74 is configured to receive the first media data returned by the server and the push response of the second media data, and receive a push commitment of the second media data sent by the server according to the push response;

The second receiving module 76 is configured to receive the second media data that the server pushes according to the push commitment.

The first sending module 72 sends a first media data obtaining request to the server, where the obtaining request carries a push indication of the second media data, and the first receiving module 74 receives the first media data returned by the server and the first Transmitting a response of the second media data, and receiving a push commitment of the second media data sent by the server according to the push response, the second receiving module 76 receiving the second media data that is pushed by the server according to the push commitment, and solving the utilization server The "push" feature enables the accessibility limitation of push media data when video streaming is performed, and realizes efficient and low-latency adaptive streaming media transmission based on full-duplex transmission protocol binding.

In this embodiment, the device further includes:

The first setting module is configured to: before sending the first media data acquisition request to the server, set the push condition information included in the push indication of the second media data, where the push condition information is used to request the server to be the second media data A push commitment is sent by the inaccessible media data referenced in the push indication.

In this embodiment, the first setting module includes:

The first setting unit is configured to set the push condition information to be a push waiting time, where the push waiting time is used to set an upper waiting time limit for the push commitment of the second media data to be sent by the server after receiving the first media data obtaining request .

In this embodiment, the first setting module includes:

The second setting unit is configured to set the push condition information to a push waiting time with a value of zero, and the push waiting time is used to request the server to push all the media data referenced in the push indication of the second media data.

In this embodiment, the first setting module includes:

a third setting unit, configured to set the push condition information to be a push threshold, where the push threshold is used to set the server to use the first media data as an initial index to push a lower limit of the quantity of media data in the second media data; or The push threshold is used to set a lower limit of the media time of the media data in the second media data from the media time of the first media data.

In this embodiment, the first receiving module 74 includes:

FIG. 8 is a structural block diagram 2 of an adaptive streaming media transmission apparatus according to an embodiment of the present invention. As shown in FIG. 8, the apparatus is located on the server side, and includes:

The third receiving module 82 is configured to receive a request for acquiring the first media data sent by the client, where the obtaining request carries a push indication of the second media data;

The fourth receiving module 84 is configured to return a push response of the first media data and the second media data to the client, and send a push commitment of the second media data to the client according to the push response;

The second sending module 86 is configured to push the second media data to the client according to the push commitment.

The third receiving module 82 receives the request for acquiring the first media data sent by the client, where the acquiring request carries the push indication of the second media data, and the fourth receiving module 84 returns the first media data and the second The push of the media data is sent to the client, and the push commitment of the second media data is sent to the client according to the push response, and the second sending module 86 pushes the second media data to the client according to the push commitment. The problem of accessibility limitation of push media data existing when using video "push" feature for video streaming media transmission achieves efficient, low-latency adaptive streaming media transmission based on full-duplex transmission protocol binding.

In this embodiment, the device further includes:

The second setting module is configured to: before returning the push response of the second media data to the client, setting a push condition information of the push instruction request including the second media data in the push response of the second media data, The inaccessible media data referenced in the push indication of the second media data sends a push commitment.

In this embodiment, the second sending module 86 includes:

The third sending unit is configured to, when the push condition information is the push waiting time, send the push commitment of the second media data to the client in the push waiting time from receiving the first media data obtaining request .

The fifth sending unit is configured to: when the push condition information is a push threshold, use the first media data as an initial index to push at least the media data in the second media data with the push threshold as a lower limit to the client Or; for at least pushing the media data in the second media data with the push threshold as a media time lower limit from the media time of the first media data to the client.

The invention will now be described in detail in conjunction with the preferred embodiments and embodiments.

A preferred embodiment of the present invention provides an adaptive streaming media transmission system. FIG. 9 is a structural diagram of an adaptive streaming media system according to an embodiment of the present invention. As shown in FIG. 9, the terminal 100 includes a streaming media client. 200, HTTP/2 client 300, network 400, and server 500. The system shown in Fig. 9 will be described below.

The terminal 100 provides a user-side hardware device of the running environment for the streaming media client 200. The deployment example of the terminal 100 may include: a set-top box, a desktop computer, a tablet computer, a smart phone, and the like;

The streaming media client 200 provides access, decoding, caching, and playback control for the streaming media resources in the server 500, including: a code rate adaptation module 210, a media playback engine 220, a push module 230, and a pull module 240.

The streaming client 200 instructs the HTTP client 300 to establish a media transport channel with the server 500, and the server 500 actively pushes streaming media data to the terminal 100 using the media channel. The media channel can be established through an HTTP/1.1 protocol upgrade mechanism. The streaming client 200 transmits a media segment or MPD file access request to the server 500 via the HTTP client 300. The media segment access request may carry a uniform resource identifier URI of the requested media segment and a “push policy” of the subsequent media segment. The "push policy" is used by the streaming client 200 to negotiate with the server 500 for the push mechanism of subsequent media segments. In addition to returning the requested media segment data, after receiving the media segment push request, the server 500 will return a push response according to the availability of the media segment and the local policy, once the streaming media client 200 and the server 500 reach a subsequent media segment "push strategy". Consistently, the server 500 will initiate a media segment push session to actively push subsequent available media segments to the terminal 100. The streaming client 200 can start playing the video content after buffering a sufficient amount of media segment data.

The rate adaptation module 210 detects the current available network bandwidth and the media segment buffer status in the media playback engine 220, performs a code rate adaptation algorithm, and sends a code rate switching indication to the push module 230 and the pull module 240.

The media playback engine 220 decodes the received media segments, media data buffers, and video playback controls.

The push module 230 initiates a session function based on the server supported by the HTTP 2.0 protocol, and requests the server 500 to push the streaming media resource to the terminal 100 through the HTTP client 300.

In the solution of the present invention, the push module 230 supports the following "Push Behavior":

Push module 230 supports "Instant Push" behavior:

In the context of the present invention, "Instant Push" refers specifically to the push behavior that all push commitments that the server will send in the push session have been determined once the server returns a push response to the client. In the "instant push" behavior, the server dominates the push policy negotiation, which completely determines how to push subsequent media segments.

1. According to the local policy, if the server can fully satisfy the push indication carried in the push request, for example, the push indication only refers to the currently accessible media segment, the server can directly return to the same as the push indication by performing “immediate push”. Push response of push parameter values

2. If the server cannot fully satisfy the push indication carried in the push request, for example, the push indication refers to the media clip that is currently inaccessible, and the push indication does not directly request the server to push the currently unreachable media segment, the server can also Returns a push response with a different push parameter value in the push indication by performing an "immediate push"

3. In particular, when it is judged that any media clip cannot be pushed, the server can use "immediate push" to return a "reject push" push response.

Push module 230 supports "Progressive Push" behavior:

In the context of the present invention, "Progressive Push" or "Long Push" on behalf of the server returning a push response does not preclude the subsequent progressive, conditional transmission beyond the push response. The push behavior of the push promise of the push parameter value returned in . During the "progressive push" process, the server will push the promise for the newly-accessible media segment until the media segment referenced by the push indication has all been pushed, or the push condition set in the push indication is not met.

1. According to the local policy, if the server cannot fully satisfy the push indication carried in the push request, for example, the push indication refers to a media segment that is not yet accessible, but the push instruction explicitly requests the server to push the media segment that is not yet accessible, the server may Perform "gradual push"

1) In the initial stage, the server returns the push parameter value according to the currently accessible media segment in the returned push response, and confirms that the push commitment will be sent for the newly emerging accessible media segment.

2) In the subsequent stage of the push session, the server will push the promise for the newly-accessible media segment until the media segments referenced in the push indication have all been pushed, or the push conditions set in the push indication have expired

The "Progressive Push" behavior enables the server to send push commitments to media segments that are currently inaccessible (ie, temporarily unable to determine the access address) based on the push conditions set by the client:

1. The client sets push condition information in the second media data push indication for requesting the server to push The second media data push indication is not yet accessible media data;

2. The server will push the promise for the newly-accessible media segment until the media segment referenced in the push indication has all been pushed, or the push condition set in the push indication has expired

Push condition failure is usually accompanied by the occurrence of a specific event or state in the push session. For example, the push wait time for pushing a promised media segment push promise has expired, or the number of pushed media segments or media clip time has reached the push threshold, etc. .

The basic process of "Progressive Push" behavior is as follows:

1. The client carries a second media data push indication including push condition information in the sent push request

2. The server returns a second media data push response, and does not exclude that a push commitment that exceeds the push parameter value returned in the push response will be sent according to the push condition information requested in the push indication.

3. The server sends the push promise of the currently accessible media segment to the client immediately after returning the push response. In addition, the server will push the promise for the newly available accessible media segment.

It should be noted that although the progressive push process naturally involves an instant push process, the push condition information mentioned above will not impose any restrictions on the instant push process.

The pull module 240 requests the downloading of the streaming media resource in the server 500 to the terminal 100 through the HTTP client 300 based on the client initiated session function supported by the HTTP 1.1 protocol or the HTTP 2.0 protocol.

The HTTP client 300 includes: an HTTP/2 client, and an HTTP/1.1 client. The HTTP client 300 sends an HTTP request to the server 500 according to the media segment access request of the internal push module 230 or the pull module 240 of the streaming media client 200 and the code rate switching instruction, and requests the acquisition of the server 500 that is most suitable for the current network bandwidth and the cache status. Streaming media resources. The cache 330 provides local storage of the media resources downloaded by the HTTP client 300.

The network 400 may include: a digital subscriber line (X Digital Subscriber Line, abbreviated as xDSL), a fiber-optic access (Fiber-to-the-x, referred to as FTTx), a cable communication cable, and the like, and a Wi-Fi network. A wireless communication network such as 2G, 3G, and Long Term Evolution (LTE).

The server 500 completes the generation of the slice of the original video and the generation of the MPD file. The content source 520 is responsible for content source capture, encoding compression, and segmentation processing, and divides the original video content into a plurality of video segments having a fixed time interval. The HTTP server 510 stores each video clip as an HTTP resource that is independently accessible by HTTP requests and responses, that is, a video file, and stores it in a hierarchy of periods, representations, and segments. Upon request, the HTTP server 510 will return the appropriate media shards to the terminal.

The server 500 also provides MPD file storage in dynamic adaptive streaming. The MPD file describes the duration, URL, media attributes of each media segment, such as video resolution, adaptation bit rate, and the like. The MPD file provides enough HTTP-URL information for the streaming client to access the streaming content resource. When the streaming client starts playing a video, it selects the download according to the information in the MPD file to adapt to the currently available network bandwidth. And the media segment of the terminal cache status is played and played.

In the embodiment of the present invention, the adaptive streaming media (or dynamic adaptive streaming media) involved may be implemented in various manners, for example, dynamic adaptive streaming over HTTP (Dynamic Adaptive Streaming over HTTP) may be adopted. Referred to as DASH). The following takes DASH as an example to describe the MPD server and the streaming media server. The following embodiments are equally applicable to other adaptive streaming technologies.

FIG. 10 is a flowchart 1 of an adaptive streaming media transmission method according to an alternative embodiment of the present invention. As shown in FIG. 10, the process includes the following steps:

Step S1002: Send a first media data acquisition request to the server, where the acquisition request carries a push indication of the second media data;

Step S1004: Receive a push response of the first media data and the second media data returned by the server, and receive a push commitment of the second media data sent by the server according to the push response.

Step S1006: Receive the second media data that is sent by the server according to the push commitment.

Through the above steps, the client may carry a push indication of the second media data in the first media data acquisition request, and set the second media data push indication to include push condition information, where the server is requested to be the second The push data promised by the inaccessible media data referenced in the push indication of the media data. Therefore, the problem of accessibility limitation of push media data existing when the server "push" feature is used for video streaming transmission is solved in the related art, and an efficient, low-latency adaptive flow based on full-duplex transmission protocol binding is realized. Media transfer.

In the above step S1002, it is described that the first media data acquisition request is sent to the server, and the request carries the push instruction of the second media data; this will be described below in conjunction with an optional implementation.

FIG. 11 is a second flowchart of an adaptive streaming media transmission method according to an alternative embodiment of the present invention. As shown in FIG. 11, the process includes the following steps:

Step S1102, determining a first media data uniform resource identifier URI that is requested to be acquired;

In step S1104, it is determined whether the server is requested to push the second media data. If yes, go to step S1106; if no, go to step S1105, send the first media data acquisition request to the server, and end.

Step S1105: Send a first media data acquisition request to the server;

Step S1106, setting a push indication for requesting the server to push the second media data;

Step S1108: Determine whether the push indication refers to media data that is not yet accessible in the second media data. If yes, go to step S1110, if no, go to step S1114;

In step S1110, it is determined whether to request to push the media data that is not yet accessible in the second media data. If yes, step S1112 is performed, and if no, step S1114 is performed.

In step S1112, the push indication of the second media data includes push condition information, and is used to request the server to push the media data that is not yet accessible in the second media data;

Step S1114: Send a first media data acquisition request to the server, where the acquisition request carries a push indication of the second media data.

In the above step S1112, it is described that the second media data push instruction includes the push condition information, and is used to request the server to push the media data that is referenced in the push instruction of the second media data; The implementation is described.

The following is a detailed description of the "PushConditions" proposed in the present invention:

As part of the push indication in the push request, the "push condition" can be used by the client to explicitly request the server to send a push commitment under certain conditions in the push session, such as sending a push promise for a newly emerging accessible media segment. A "push condition" is a string consisting of a series of (keyword, value) pairs, where each pair (keyword, value) represents a specific push condition for equal evaluation of "push conditions" in a push session Effectiveness.

The Bacchus Paradigm (ABNF) for the "push condition" string extension is defined as follows:

PUSH_CONDITIONS=key BWS"="BWS value

*(OWS","[OWS parameter])

Parameter=key BWS"="BWS value

Table 1 below provides formal keyword and value definitions for "push conditions":

Table 1

The "push-wait" push condition defined in Table 1 is used to indicate:

1. The client is willing to wait for a certain maximum time value in the push session to receive the push commitment sent by the server.

2. The client may wait for a certain maximum time value to send a push request again for those media segments that have not received the push commitment.

The Push Wait indicator is used to request that the server keep the push request flow in an "open" state for a certain maximum time value even if the push commitment of all accessible media segments has been sent to the client. In some cases, such as in multiple consecutive push transactions, the server is unable to send a push commitment for subsequent media segments that the client requests to push. Clients can use the Push Waiting mechanism to reduce the need to create new request flows, thereby avoiding unnecessary request overhead.

The "push-threshold" push condition defined in Table 1 is used to indicate:

1. The client requests a media segment that is pushed at least by a certain number of lower limits or media time limits in the push session.

2. The client may send a push request again for media segments that have not received the push promise after the media segment that has been pushed to a certain lower limit or media time limit.

The "push threshold" indicator is used to request that the server keep the push request stream in an "open" state even if the push commitment of all accessible media segments has been sent to the client until the client is pushed a certain lower media segment.

The following is an example of the "PushConditions" proposed in the present invention as follows:

(1) Non-progressive push

1) The example of "push wait" is shown in Table 2:

Table 2

2) The "push threshold" usage example is shown in Table 3:

table 3

3) The example of "push wait" combined with "push threshold" is shown in Table 4:

Table 4

(2) Progressive push

1) The example of "push wait" is shown in Table 5:

table 5

2) The "push threshold" usage example is shown in Table 6:

Table 6

3) The example of "push wait" combined with "push threshold" is shown in Table 7:

Table 7

FIG. 12 is a third flowchart of an adaptive streaming media transmission method according to an alternative embodiment of the present invention. As shown in FIG. 12, the process includes the following steps:

Step S1202, setting a push indication of the second media data to include a push waiting time;

Step S1204: The push waiting time is used to set an upper limit of the waiting time for the push commitment of the second media data to be sent by the server after receiving the first media data obtaining request.

Specifically, when the push waiting time included in the push indication of the second media data set by the client is a preset value, for example, a push waiting time with a value of zero, the push waiting time is used for the requesting The server pushes all media data referenced in the push indication of the second media data.

FIG. 13 is a flowchart 4 of an adaptive streaming media transmission method according to an alternative embodiment of the present invention. As shown in FIG. 13, the process includes the following steps:

Step S1302, setting a push indication of the second media data to include a push threshold;

Step S1304: The push threshold is used to set the server to use the first media data as an initial index, and push a lower limit of the quantity of media data in the second media data.

FIG. 14 is a flowchart 5 of an adaptive streaming media transmission method according to an alternative embodiment of the present invention. As shown in FIG. 14, the process includes the following steps:

Step S1402, setting a push indication of the second media data to include a push threshold;

Step S1404: The push threshold is used to set a media time lower limit of the media data in the second media data from the media time of the first media data.

In the above step S1004, it is described that the push response of the first media data and the second media data returned by the server is received, and the push commitment of the second media data sent by the server according to the push response is received; The implementation is described.

FIG. 15 is a flowchart 6 of an adaptive streaming media transmission method according to an alternative embodiment of the present invention. As shown in FIG. 15, the process includes the following steps:

Step S1502: Receive a push response of the second media data returned by the server.

Step S1504, determining whether the push response refers to all media data in the second media data. If yes, go to step S1506, otherwise go to step S1510;

Step S1506: The sending server sends a push commitment of the second media data according to the push response.

Step S1508, receiving the second media data that the server pushes according to the push commitment;

Step S1510: Receive a push commitment of the second media data that is sent by the server according to the push parameter value returned by the push response.

In step S1512, it is determined whether the push response includes push condition information. If yes, go to step S1514, otherwise go to step 1516;

Step S1514: Receive a push commitment of the second media data that is sent by the server according to the push condition information returned by the push response.

Step S1516: Receive the second media data that is sent by the server according to the push commitment.

It should be noted that if the server does not return the second media data push response, the client may think that the server will not push the second media data; if the server returns a "reject push" push response, the client may determine that the server will not push the first Two media data. If the server returns a push response containing push condition information, the client can assume that the server will push the media data that is not yet accessible referenced in the push indication of the second media data.

16 is a flowchart 7 of an adaptive streaming media transmission method according to an alternative embodiment of the present invention. As shown in FIG. 16, the process includes the following steps:

Step S1602: Receive a first media data acquisition request sent by the client, where the acquisition request carries a push indication of the second media data.

Step S1604, returning a push response of the first media data and the second media data to the client, and sending a push commitment of the second media data to the client according to the push response;

Step S1606: Push the second media data to the client according to the push commitment.

Through the above steps, the server may set the push condition information of the push indication request of the second media data in the push response of the second media data, and the reference for the push indication of the second media data is not yet The accessed media data sends a push commitment, thereby solving the problem of the accessibility limitation of the push media data existing in the related art in the use of the server "push" feature for video streaming media transmission, and realizing the binding based on the full duplex transmission protocol. Efficient, low-latency adaptive streaming media transmission.

In the above step S1604, a push response that returns the first media data and the second media data is sent to the client, and a push commitment of the second media data is sent to the client according to the push response. This is explained below in conjunction with an optional implementation.

FIG. 17 is a flowchart 8 of an adaptive streaming media transmission method according to an alternative embodiment of the present invention. As shown in FIG. 17, the process includes the following steps:

Step S1702: Receive a first media data acquisition request sent by the client, where the acquisition request carries a push indication of the second media data.

Step S1704, determining whether the second media data referenced in the push indication is all accessible. If yes, go to step S1706, otherwise go to step 1712;

Step S1706, returning a push response of the first media data and the second media data, and setting a push parameter value returned in the push response to refer to all media data referenced in the push indication;

Step S1708: Send a push commitment of the second media data to the client according to the push parameter value returned by the push response.

Step S1710: Push the second media data to the client according to the push commitment.

In step S1712, it is determined whether to continue pushing the second media data. If yes, go to step S1714, otherwise go to step S1716;

In step S1714, it is determined whether the push condition information is included in the push instruction. If yes, go to step S1718, otherwise go to step S1722;

Step S1716, returning the first media data and "rejecting push" push response;

In step S1718, it is judged whether or not the push condition is valid. If yes, go to step S1720, otherwise go to step S1722;

Step S1720, setting the push condition information requested in the push instruction including the second media data in the push response;

Step S1722, returning a push response of the first media data and the second media data, and setting a push parameter value returned in the push response to reference the accessible media data referenced in the push indication;

Step S1724: Send a push commitment of the second media data to the client according to the push parameter value returned by the push response.

Step S1726, determining whether the push response includes push condition information. If yes, go to step S1728, otherwise go to step S1730;

Step S1728: Send a push commitment of the second media data to the client according to the push condition information returned by the push response.

Step S1730: Push the second media data to the client according to the push commitment.

It should be noted that, if the first media data acquisition request carries the push indication of the second media data, and the server supports the push indication, the server must return a push response of the second media data in response. Wherein, if the server does not support the push indication, the server may explicitly return a "reject push" push response in response. If the first media data acquisition request does not carry the push indication of the second media, the server is prohibited from returning a push response of the second media data in response.

In the above step S1728, the push commitment information returned by the push response is transmitted, and the push commitment of the second media data is transmitted to the client; this will be described below in conjunction with an optional implementation.

FIG. 18 is a flowchart 9 of an adaptive streaming media transmission method according to an alternative embodiment of the present invention. As shown in FIG. 18, in a case where the push condition information is a push waiting time, the server receives the first After the media data acquisition request, the push commitment of the second media data is sent to the client in the push waiting time, and the process includes the following steps:

Step S1802, waiting to push the media data that is not accessible in the push indication of the second media data;

Step S1804: Determine whether there is new accessible media data in the media data referenced by the push indication of the second media data. If yes, go to step S1806, otherwise go to step S1810;

Step S1806, sending a push commitment of the new accessible media data in the second media data to the client;

Step S1808, determining whether the second media data has all sent a push commitment. If yes, it ends, otherwise step S1810 is performed.

At step S1810, it is judged whether or not the push waiting time is exceeded. If yes, then it ends, otherwise step S1802 is performed;

In particular, in a case where the push condition information is a push waiting time with a value of zero, it is not necessary to determine whether the push wait time is exceeded in step S1810, and the server will push all the media referenced in the push instruction of the second media data. Data to the client.

FIG. 19 is a flowchart 10 of an adaptive streaming media transmission method according to an alternative embodiment of the present invention. As shown in FIG. 19, in a case where the push condition information is a push threshold, the server uses the first media. The data is an initial index, and at least the media data in the second media data that is the lower limit of the push threshold is pushed to the client; or the server pushes the at least the media time from the media time of the first media data. The second media data is sent to the client by the media data whose push threshold is the lower limit of the media time. The process includes the following steps:

Step S1902, waiting to push the media data that is not accessible in the push indication of the second media data;

Step S1904: Determine whether there is new accessible media data in the media data referenced by the push indication of the second media data. If yes, go to step S1906, otherwise go to step S1908;

Step S1906, sending a push commitment of the new accessible media data in the second media data to the client;

In step S1908, it is determined whether the push threshold is reached. If yes, then it ends, otherwise step S1902 is performed;

In the above steps S1002 to S1006, it is described that the client may carry the push instruction of the second media data in the first media data acquisition request, and the second media data push instruction includes the push condition information, and is configured to request the server to be The inaccessible media data referenced in the push indication of the second media data sends a push commitment. This is explained below in conjunction with an alternative implementation.

FIG. 20 is a flow chart 1 of message interaction of an adaptive streaming media transmission method according to an embodiment of the present invention.

Step 1: The client sends a first media segment acquisition request to the server using a standard HTTP GET message (a HEADERS frame in the HTTP/2 protocol). The "path" header field in the HTTP GET message is set as the first media segment uniform resource identifier URI, and is used to request the server to obtain the first media segment, that is, the path in the example in step 1: path="../rep1/ Segment1.mp4". In addition, the HTTP GET message also includes a "dash-push" header field for requesting the server to push the second media segment, that is, the example in step 1.

:dash-push="urn:mpeg:dash:fdh:2015",push-template,

"../rep1/segment$Number$.mp4":{2-6},push-wait=2000

The method further includes: 1) a push protocol scheme uniform resource identifier (SchemeID) field for identifying a push protocol version; and 2) a push directive (PushDirective) for sending the push policy requested in the push session, including: a push type ( The PushType field is used to describe the representation type of the pushed media segment in the Push Policy. For example, "push-next" indicates that a subsequent specific number of media segments are requested to be pushed, and "push-time" indicates that the request is pushed for a subsequent specific playing time. The media segment, "push-template", represents the media segment explicitly specified in the request push template, and the push parameter (PushParams) field, carrying the push parameter value of the pushed media segment in the "push policy".

Optionally, the "dash-push" header field may also carry: 3) PushCoditions field, which is used to explicitly request the server to send a push commitment under certain conditions in the push session, for example, for a newly appearable The accessed media segment sends a push commitment, including: a "push-wait" keyword, which is used to request the server to send a push commitment within a certain maximum time value in the push session, and a "push-threshold" keyword for requesting the server to push A media segment that is pushed at least by a certain number of lower limits or media time limits in the session.

In this embodiment, the client requests to acquire the "segment1" media segment in the "Representation" "rep1", and requests to push the 5 media segments of the subsequent numbers "segment2" to "segment6" in "rep1" and pass the " The push-wait keyword indicates that the server sends a push commitment within a maximum of 2000 milliseconds after receiving the request.

In the embodiment, the related push policy description information for requesting the server to push the second media segment may also be carried by other header fields such as “Accept-Push-Policy” in the HTTP GET message.

Step 2: The server returns an HTTP 200 OK response message (HEADERS frame in the HTTP/2 protocol) as the first media segment acquisition response. The HTTP 200 OK response message includes a "dash-push" header field for returning a second media segment push response, which is an example in step 2.

:dash-push="urn:mpeg:dash:fdh:2015",push-template,

"../rep1/segment$Number$.mp4":{2-6}

At least: 1) a push protocol scheme uniform resource identifier (SchemeID) field for identifying a push protocol version; 2) a push response (PushAck) for returning a push policy received by the server in the push session, and carrying and requesting There are the same type of push type field, push parameter field, and push condition field.

In this embodiment, the push response information returned by the server indicates that the push commitment can be immediately sent for the five media segments of "segment2" to "segment6" in the "rep1" requested to be pushed by the push indication, so there is no need to return the push in the push response. Conditional information, there is no "progressive push" process in the push session.

Steps 3 to 5: The server sends a push commitment message (HTTP/) to each of the media segments that are confirmed to be pushed (ie, 5 media segments of "segment2" to "segment6") according to the push parameter value returned by the push response in step 2. 2 PUSH_PROMISE frame in the protocol). The server sets the "path" header field in the PUSH_PROMISE frame to each of the acknowledgment pushed media segment uniform resource identifier URIs, and transmits the PUSH_PROMISE frame in the request stream initiated by the client in step 1.

Step 6: The server returns the media segment requested by the client, that is, "../rep1/segment1.mp4" in this embodiment.

Steps 7 to 12: According to the push commitment message sent in steps 3 to 5, the server creates a push flow initiated by the server for each media segment that is confirmed to be pushed, and pushes 5 media of the number "segment2" to "segment6" in "rep1". Fragment to the client.

In the above steps S1202 and S1204, it is described that the push instruction for setting the second media data includes a push waiting time, and the push waiting time is used to set the server to send the first media data acquisition request. The upper limit of the waiting time for the push commitment of the second media data. This is explained below in conjunction with an alternative implementation.

FIG. 21 is a second flowchart of message interaction of an adaptive streaming media transmission method according to an embodiment of the present invention.

Step 1: Similar to step 1 in the embodiment of Fig. 20, the client sends a first media segment acquisition request to the server using a standard HTTP GET message (a HEADERS frame in the HTTP/2 protocol).

Step 2: Similar to step 2 in the embodiment of Fig. 20, the server returns an HTTP 200 OK response message containing a "dash-push" header field for returning a second media segment push response.

In this embodiment, the push parameter field value returned by the push response is "../rep1/segment$Number$.mp4": {2-4}, indicating that the server only confirms the "rep1" requested in the push indication. The 3 media segments of segment2" to "segment4" send push commitments. Therefore, the push condition information requested in the push indication is still valid, and the push response also returns the push condition information requested by the push indication, that is, "push-wait=2000", indicating that the server will be up to 2000 milliseconds after receiving the request. The emerging accessible media segment sends a push commitment, which is a "progressive push" process in the push session.

Steps 3 to 4: Similar to steps 3 to 5 in the embodiment of FIG. 20, the server confirms the pushed media segment according to the push parameter value returned by the push response in step 2 (ie, "segment2" to "segment4" The 3 media segments) send a push commitment message (PUSH_PROMISE frame in the HTTP/2 protocol).

Steps 5-6: In this embodiment, it is assumed that the accessible media segments "segment5" and "segment6" appear newly during the maximum waiting time of the push waiting, and the server will send a push commitment message for them (PUSH_PROMISE frame in the HTTP/2 protocol) ).

Step 7: The server returns the media segment requested by the client, that is, "../rep1/segment1.mp4" in this embodiment.

Steps 8-11: According to the push commitment message sent in steps 3-6, the server creates a push flow initiated by the server for each media segment that is confirmed to be pushed, and pushes 5 media of the number "segment2" to "segment6" in "rep1". Fragment to the client.

In the above steps S1402 and S1404, it is described that the push instruction for setting the second media data includes a push threshold for setting the server to push the second from the media time of the first media data. The media time limit of the media data in the media data, which is described below in conjunction with an optional implementation.

FIG. 22 is a third flowchart of message interaction of an adaptive streaming media transmission method according to an embodiment of the present invention.

Step 1: Similar to step 1 in the embodiment of Fig. 20, the client sends a first media segment acquisition request to the server using a standard HTTP GET message (a HEADERS frame in the HTTP/2 protocol). The HTTP GET message includes a "dash-push" header field for requesting the server to push the second media segment, that is, the example in step 1.

:dash-push="urn:mpeg:dash:fdh:2015",push-template,

"../rep1/segment$Time$.mp4":{5000},push-threshold=5000

In this embodiment, the client requests to acquire the "segment1" media segment in the "Representation" "rep1", and requests to push the media segment of the subsequent 5000 milliseconds media time in "rep1", and passes the "push-threshold" keyword. Instruct the server to push at least the media segment with a lower limit of 5000 milliseconds of media time.

In this embodiment, the push parameter field value returned by the push response is "../rep1/segment$Time$.mp4": {3000}, indicating that the server only confirms the subsequent 3000 milliseconds of media in the "rep1" requested in the push indication. The media clip of the time sends a push promise. Therefore, the push condition information requested in the push indication is still valid, and the push response also returns the push condition information requested in the push indication, that is, "push-threshold=5000", indicating that the server pushes at least the media segment of 5000 milliseconds of media time. That is, there is a "progressive push" process in the push session.

Steps 3 to 4: Similar to steps 3 to 5 in the embodiment of FIG. 20, the server confirms the pushed media segment according to the push parameter value returned by the push response in step 2 (assuming the preset duration of each media segment) A push commitment message (PUSH_PROMISE frame in the HTTP/2 protocol) is sent for 1000 milliseconds, ie, 3 media segments of "segment2" to "segment4".

Step 5: The server returns the media segment requested by the client, that is, "../rep1/segment1.mp4" in this embodiment, while keeping the client request flow in the "Open" state.

Steps 6-7: The server will also send a push commitment message (PUSH_PROMISE frame in the HTTP/2 protocol) for the media segments (ie, "segment5" and "segment6") of the remaining 2000 milliseconds of media time.

Steps 8 to 11: According to the push commitment message sent in steps 3 to 4 and steps 6 to 7, the server creates a push flow initiated by the server for each media segment that is confirmed to be pushed, that is, pushes the number "segment2" in the "rep1" to "" 5 media segments of segment6" to the client.

In the above steps S1302 and S1304, it is described that the push instruction for setting the second media data includes a push threshold; the push threshold is used to set the server to use the first media data as an initial index to push the second media. The lower limit on the number of media data in the data, which is described below in conjunction with an optional implementation.

FIG. 23 is a flow chart 4 of message interaction of an adaptive streaming media transmission method according to an embodiment of the present invention.

:dash-push="urn:mpeg:dash:fdh:2015",push-template,

"../rep1/segment$Number$.mp4":{2~6}, push-wait=2000, push-threshold=4

In this embodiment, the client requests to obtain the "segment1" media segment in the representation "rep1". At the same time, it requests to push the 5 media segments of the subsequent numbers "segment2" to "segment6" in "rep1", and the "push-wait" keyword instructs the server to send the push commitment within 2000 milliseconds after receiving the request, and passes the " The push-threshold keyword indicates that the server pushes at least 4 media segments with a lower limit.

In this embodiment, the push parameter field value returned by the push response is "../rep1/segment$Number$.mp4": {2-4}, indicating that the server only confirms that "p1" is requested in the "rep1" for the push indication. The 3 media clips to "segment4" send push commitments. Therefore, the push condition information requested in the push instruction is still valid, and the push response returns the push condition information requested by the push instruction, that is, "push-wait=2000" and "push-threshold=4", indicating that the server is in the above dual condition. A push commitment is sent for the newly emerging accessible media segment, ie there is a "progressive push" process in the push session.

Step 6: This embodiment assumes that the accessible media segment "segment5" newly appears before the maximum push latency (i.e., 2000 milliseconds) expires, and the server sends a push commitment message (PUSH_PROMISE frame in the HTTP/2 protocol) for it.

Steps 7 to 10: According to the push commitment message sent in steps 3 to 4 and step 6, the server creates a server-initiated push stream for each media segment that is confirmed to be pushed, and pushes the number "segment2" to "segment5" in "rep1". 4 media clips to the client.

FIG. 24 is a structural block diagram 3 of an adaptive streaming media transmission apparatus according to an embodiment of the present invention. As shown in FIG. 24, the apparatus includes an obtaining module 242, a first determining module 244, and a second determining module 246. The adaptive media stream processing device will be described.

The obtaining module 242 is configured to obtain the location information and the time information of the adaptive streaming media carried in the transport stream. The first determining module 244 is connected to the obtaining module 242, and is configured to determine, according to the location information, the need for the adaptive streaming media. The transport stream is time-axis aligned media segment; the second determining module 246 is coupled to the first determining module 244, and configured to determine, according to the time information, the media presentation time of the media segment in the adaptive streaming media relative to the time axis of the transport stream Timestamp.

A preferred embodiment of the present invention provides an adaptive streaming media transmission method and apparatus to address at least the problem of accessibility limitations of push media segments in existing HTTP streaming media technologies.

Through the solution of the invention, the problem of the accessibility limitation of the push media segment in the existing HTTP streaming media technology is solved, and the efficient and low-latency adaptive streaming media transmission based on the HTTP 2.0 protocol binding is realized.

Through the description of the above embodiments, those skilled in the art can clearly understand the square according to the above embodiments. The method can be implemented by means of software plus the necessary general hardware platform, of course, it can also be through hardware, but in many cases the former is a better implementation. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, The optical disc includes a number of instructions for causing a terminal device (which may be a cell phone, a computer, a server, or a network device, etc.) to perform the methods described in various embodiments of the present invention.

It should be noted that each of the above modules may be implemented by software or hardware. For the latter, the foregoing may be implemented by, but not limited to, the foregoing modules are all located in the same processor; or, the modules are located in multiple In the processor.

Embodiments of the present invention also provide a storage medium. Alternatively, in the present embodiment, the above storage medium may be arranged to store program code for performing the method steps of the above-described embodiments.

Optionally, in this embodiment, the foregoing storage medium may include, but not limited to, a USB flash drive, a Read-Only Memory (ROM), a Random Access Memory (RAM), a mobile hard disk, and a magnetic memory. A variety of media that can store program code, such as a disc or a disc.

Optionally, in this embodiment, the processor performs the method steps of the foregoing embodiments according to the stored program code in the storage medium.

It will be apparent to those skilled in the art that the various modules or steps of the present invention described above can be implemented by a general-purpose computing device that can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device such that they may be stored in the storage device by the computing device and, in some cases, may be different from the order herein. The steps shown or described are performed, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps thereof are fabricated as a single integrated circuit module. Thus, the invention is not limited to any specific combination of hardware and software.

The above description is only the preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes can be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Industrial applicability

As described above, an adaptive streaming media transmission method and apparatus provided by an embodiment of the present invention have the following beneficial effects: solving the accessibility limitation of push media data existing when video streaming media transmission is performed by using a server "push" feature. The problem is to achieve efficient, low-latency adaptive streaming media transmission based on full-duplex transmission protocol binding.

Claims

An adaptive streaming media transmission method includes:

Sending a first media data acquisition request to the server, where the obtaining request carries a push indication of the second media data;

Receiving a push response of the first media data and the second media data returned by the server, and receiving a push commitment of the second media data sent by the server according to the push response;

Receiving, by the server, the second media data that is pushed according to the push commitment.
The method of claim 1, wherein before the transmitting the first media data acquisition request to the server, the method comprises:

The push indication of the second media data includes push condition information, and the push condition information is used to request the server to send a push commitment for the media data that is not accessible in the push indication of the second media data.
The method of claim 2 wherein said method comprises:

The push condition information is set as a push wait time, and the push wait time is used to set an upper limit of a waiting time for the push commitment of the second media data to be sent by the server after receiving the first media data acquisition request.
The method of claim 2 or 3, wherein the method comprises:

The push condition information is set to a push wait time with a value of zero, and the push wait time is used to request the server to push all the media data referenced in the push indication of the second media data.
The method of claim 2 wherein said method comprises:

Setting the push condition information as a push threshold, where the push threshold is used to set the server to use the first media data as an initial index, and push a lower limit of the quantity of media data in the second media data; or, the push The threshold is used to set a lower limit of the media time of the media data in the second media data from the media time of the first media data.
The method of claim 1 or 2, wherein receiving a push response of the second media data returned by the server comprises:

Receiving a push response of the second media data returned by the server, and the push response of the second media data includes push condition information requested in the second media data push instruction.
The method of claim 1, wherein receiving a push commitment of the second media data sent by the server according to the push response comprises:

Receiving, by the server, a push commitment of the second media data that is sent according to a push parameter value returned by the push response.
The method of claim 1, wherein receiving a push commitment of the second media data sent by the server according to the push response comprises:

Receiving, by the server, the push commitment of the second media data that is sent according to the push condition information returned by the push response.
An adaptive streaming media transmission method includes:

Receiving an acquisition request of the first media data sent by the client, where the obtaining request carries a push indication of the second media data;

Returning a push response of the first media data and the second media data to the client, and sending a push commitment of the second media data to the client according to the push response;

Pushing the second media data to the client according to the push commitment.
The method of claim 9, wherein before returning the push response of the second media data to the client, the method comprises:

And setting, in the push response of the second media data, push condition information including a push instruction request of the second media data, and sending, for sending, the inaccessible media data referenced in the push instruction of the second media data committed to.
The method according to claim 9 or 10, wherein the sending the push commitment of the second media data to the client according to the push response comprises:

Transmitting, according to the push parameter value returned in the push response, a push commitment of the second media data to the client;

Sending a push commitment of the second media data to the client according to the push condition information returned in the push response.
The method according to claim 11, wherein, according to the push condition information returned in the push response, sending the push commitment of the second media data to the client comprises:

When the push condition information is the push waiting time, the push commitment of the second media data is sent to the client within the push waiting time from receiving the first media data acquisition request.
The method of claim 11, wherein the transmitting the push commitment of the second media data to the client according to the push condition information returned in the push response comprises:

When the push condition information is a push waiting time with a value of zero, all media data referenced in the push instruction of the second media data is pushed to the client.
The method of claim 11, wherein the transmitting the push commitment of the second media data to the client according to the push condition information returned in the push response comprises:

When the push condition information is a push threshold, the first media data is used as an initial index, and at least the media data in the second media data with the push threshold is a lower limit is pushed to the client; Or, at least the media data in the second media data with the push threshold being the media time lower limit is pushed to the client from the media time of the first media data.
An adaptive streaming media transmission device is located on the terminal side and includes:

The first sending module is configured to send a first media data obtaining request to the server, where the obtaining request carries a push indication of the second media data;

a first receiving module, configured to receive a push response of the first media data and the second media data returned by the server, and receive a push of the second media data sent by the server according to the push response committed to;

The second receiving module is configured to receive the second media data that is sent by the server according to the push commitment.
The apparatus of claim 15 wherein said apparatus further comprises:

a first setting module, configured to set, before sending the first media data acquisition request to the server, push condition information included in the push indication of the second media data, where the push condition information is used to request the server to be the first The non-accessible media data referenced in the push indication of the second media data sends a push commitment.
The apparatus of claim 16, wherein the first setting module comprises:

a first setting unit, configured to set the push condition information to be a push wait time, where the push wait time is used to set the server to send the push of the second media data after receiving the first media data acquisition request The maximum waiting time for commitment.
The apparatus according to claim 16 or 17, wherein the first setting module comprises:

a second setting unit, configured to set the push condition information to a push waiting time with a value of zero, the push waiting time for requesting the server to push all the media data referenced in the push indication of the second media data .
The apparatus of claim 16, wherein the first setting module comprises:

a third setting unit, configured to set the push condition information to be a push threshold, where the push threshold is used to set the server to use the first media data as an initial index, and push the quantity of media data in the second media data The lower limit is used; or the push threshold is used to set a lower limit of the media time of the media data in the second media data from the media time of the first media data.
The apparatus of claim 15, wherein the first receiving module comprises:

The first receiving unit is configured to receive a push response of the second media data returned by the server, and the push response of the second media data includes push condition information requested in the second media data push instruction.
The apparatus of claim 15, wherein the first receiving module comprises:

The second receiving unit is configured to receive a push commitment of the second media data that is sent by the server according to the push parameter value returned by the push response.
The apparatus of claim 15, wherein the first receiving module comprises:

The third receiving unit is configured to receive a push commitment of the second media data that is sent by the server according to the push condition information returned by the push response.
An adaptive streaming media transmission device is located on the server side and includes:

a third receiving module, configured to receive a request for acquiring the first media data sent by the client, where the obtaining request carries a push indication of the second media data;

a fourth receiving module, configured to return a push response of the first media data and the second media data to the client, and send a push commitment of the second media data to the client according to the push response end;

The second sending module is configured to push the second media data to the client according to the push commitment.
The device of claim 23, wherein the device further comprises:

a second setting module, configured to set, before returning the push response of the second media data to the client, a push condition information including a push indication request of the second media data in a push response of the second media data And sending a push commitment for the inaccessible media data referenced in the push indication of the second media data.
The apparatus according to claim 23 or 24, wherein the second transmitting module comprises:

The first sending unit is configured to send a data push commitment of the second media to the client according to the push parameter value returned in the push response;

The second sending unit is configured to send a push commitment of the second media data to the client according to the push condition information returned in the push response.
The apparatus of claim 25, wherein the second transmitting module comprises:

The third sending unit is configured to, when the push condition information is the push waiting time, send the push commitment of the second media data in the push waiting time from receiving the first media data obtaining request To the client.
The apparatus of claim 25, wherein the second transmitting module comprises:

The fourth transmitting unit is configured to push all the media data referenced in the push instruction of the second media data to the client when the push condition information is a push waiting time with a value of zero.
The apparatus of claim 25, wherein the second transmitting module comprises:

The fifth sending unit is configured to: when the push condition information is a push threshold, use the first media data as an initial index to push at least the media in the second media data with the push threshold as a lower limit Data is sent to the client; or, at least from the media time of the first media data, the media data in the second media data with the push threshold being the media time lower limit is pushed to the client.