WO2016145913A1 - 自适应流媒体处理方法及装置 - Google Patents

自适应流媒体处理方法及装置 Download PDF

Info

Publication number
WO2016145913A1
WO2016145913A1 PCT/CN2015/096589 CN2015096589W WO2016145913A1 WO 2016145913 A1 WO2016145913 A1 WO 2016145913A1 CN 2015096589 W CN2015096589 W CN 2015096589W WO 2016145913 A1 WO2016145913 A1 WO 2016145913A1
Authority
WO
WIPO (PCT)
Prior art keywords
media
time
event
value
timeline
Prior art date
Application number
PCT/CN2015/096589
Other languages
English (en)
French (fr)
Inventor
黄成�
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Priority to US15/558,008 priority Critical patent/US10715571B2/en
Priority to EP15885282.2A priority patent/EP3270601B1/en
Publication of WO2016145913A1 publication Critical patent/WO2016145913A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43074—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/242—Synchronization processes, e.g. processing of PCR [Program Clock References]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4622—Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8547—Content authoring involving timestamps for synchronizing content

Definitions

  • the present invention relates to the field of communications, and in particular to an adaptive streaming media processing method and apparatus.
  • the user accesses the service provider and/or the content provider to develop the deployed communication service through a fixed mobile communication terminal such as a television, a desktop computer, a tablet computer, a mobile phone, etc., by accessing the customized communication client capability at the user terminal
  • the communication link is established between the service servers (content servers), and the above communication links are usually composed of, for example, various types of digital subscriber lines (X Digital Subscriber Line, xDSL), and fiber access (Fiber-to-the).
  • -x referred to as FTTx
  • cable communication cable cable and other wired communication networks and Wi-Fi, 2G, 3G, Long Term Evolution (LTE) and other wireless communication networks.
  • Digital Video Broadcasting is one of the world's mainstream digital TV standards. Based on MPEG-2 audio and video data compression algorithms and multiplexing technology, the DVB standard provides a complete set and is suitable for different applications.
  • the specification of the digital TV system of the medium defines three channels of coding and modulation for the transmission of media data streams (DVB-C), satellite (DVB-S) and terrestrial (DVB-T), and extends MPEG- 2 standard system layer, enriching the network interface.
  • DASH Dynamic Adaptive Streaming over HTTP
  • DASH is an adaptive bit rate streaming technology that provides high-quality streaming media content using traditional HTTP web servers on the Internet.
  • DASH divides a video into chunks of files on many small HTTP servers, each of which contains a video content with a shorter playback time. These small video chunks may be part of a live video content for hours of movies (on demand) or sports events.
  • Video may be encoded into video clips of multiple bitrates to cover different network bandwidth requirements.
  • OTT technology represented by adaptive streaming media has the flexibility to consume video anytime and anywhere, and IPTV, digital Traditional broadcasting technologies such as television and satellite television have high definition and quality assurance of large screens. Therefore, the integration of OTT technology and broadcasting technologies such as digital television can provide a comprehensive solution suitable for a variety of scenarios.
  • Second Screen applications are currently receiving attention, users utilize "TV buddies" such as smartphones, tablets, etc., to consume personalized media information associated with TV screen broadcast programs.
  • broadcast networks such as DVB-C digital TV networks
  • OTT mode Internet Broadband Network
  • the above hybrid delivery mode has significant features such as content source independence, transport network heterogeneity, and differentiated playback terminals. Among them, is it possible to establish a time correlation between a broadcast program or a primary media content (Primary Media) and a broadband program or an auxiliary media content (Ancillary Media), that is, whether different content sources, different transmission formats, and media of different playback terminals can be guaranteed? Synchronous playback between content has become the key to whether such applications can be accepted by users.
  • Primary Media Primary media content
  • Ancillary Media auxiliary media content
  • the audio and video feature recognition technology and its application are currently a research direction to solve the problem of media content synchronization in the above hybrid delivery mode, and utilize the digital features of the audio and video media content as the time synchronization reference between media.
  • digital watermarking technology The digital watermark of the audio signal in the television program is collected, and the program channel identifier and time stamp information in the digital watermark can be used to detect the time progress of the program being played, thereby obtaining appropriate auxiliary media content and playing synchronously; 2) digital fingerprint technology.
  • the digital fingerprint information is extracted from the audio and video samples, and the fingerprint retrieval database is used to obtain the media synchronization information.
  • Another research direction for media synchronous playback in mixed delivery mode is to use a single reference clock for broadcast network and broadband network media delivery.
  • One of the methods is to include the PCR/PTS (Program Reference Clock/Display Time Stamp) clock information carried in the transport stream, for example, the MPEG-2 TS code stream, such as Real-time Transport Protocol (Real-time Transport Protocol).
  • RTP is transmitted in the Internet Protocol (IP) packet.
  • IP Internet Protocol
  • Embodiments of the present invention provide an adaptive streaming media processing method and apparatus, to at least solve the problem of media synchronous playback in a hybrid delivery mode of a broadcast network and a broadband network in the related art.
  • an adaptive streaming media processing method including: acquiring a media presentation MP timeline alignment event message carried in a media segment; determining an aligned media segment according to the MP timeline alignment event message
  • the aligned media segment is a media segment in which the media presentation time MPT in the MP time axis is aligned with an external time axis; calculating, after the MP time axis establishes a mapping relationship with the external time axis, The time that the media time segment in the MP timeline alignment event elapses during the media presentation time.
  • the InbandEventStream element includes: an event flow scheme identifier schemeIdUri attribute and an event stream value value attribute, and is set to indicate an MP time axis alignment event.
  • the MP timeline alignment event message includes at least one of: a presentation time increment presentation_time_delta field set to provide an MP time axis alignment event; an event duration event_duration field, set to indicate a media presentation remaining time period; message data message_data field, arranged to provide the alignment with respect to the media segment period period start time period where presentation time T p.
  • the MP timeline alignment event message further includes: an event message plan identifier scheme_id_uri field and an event value value field; the event message scheme identifier scheme_id_uri field value is equal to the event stream scheme identifier schemeIdUri attribute value, and the event value value field The value is equal to the event stream value value attribute value.
  • Determining the aligned media segments based on the MP timeline alignment event message includes determining a media segment having a presentation_time_delta field value in the MP timeline alignment event message as a predetermined value as the aligned media segment.
  • Calculating a time period of the media segment at the media presentation time including: determining that the media segment having the same identifier id field value in the MP time axis alignment event message is a media segment in the MP time axis alignment event; The time increment indication_time_delta field value and the time scale timescale field value are presented in the MP timeline alignment event message, and the time elapsed during the media presentation time of the media segment in the MP timeline alignment event is calculated.
  • the media presentation time of the aligned media segment and the subsequent media segment in the event; emsg.presentation_time_delta and emsg.timescale respectively represent the media rendering time increment value and the time scale value in the MP timeline alignment event message.
  • the method also includes before the media carried in the media segment presents the MP timeline alignment event message. Include: obtaining the media segment carrying the MP timeline alignment event message.
  • Obtaining the media segment carrying the MP timeline alignment event message includes: acquiring an event stream plan identifier field and an event stream value field; determining the media presentation description by using the event stream scheme identifier field value and the event stream value field value An in-band event stream InbandEventStream element in the MPD; a media segment corresponding to the in-band event stream InbandEventStream element is obtained as the media segment.
  • the external time axis includes a transport stream time axis.
  • an adaptive streaming media processing apparatus including: a first obtaining module, configured to acquire a media presentation MP timeline alignment event message carried in a media segment; and a determining module configured to The MP timeline alignment event message determines an aligned media segment, wherein the aligned media segment is a media segment in which the media presentation time MPT in the MP timeline is aligned with an external timeline; a computing module configured to calculate After the MP time axis establishes a mapping relationship with the external time axis, the MP time axis aligns the time that the media segment in the event elapses during the media presentation time.
  • the first obtaining module includes: a first access unit configured to access an in-band event stream in the MPD to describe an in-band event stream in the MPD; and a first acquiring unit configured to acquire a media segment corresponding to the in-band event stream InbandEventStream element
  • the carried media presents an MP timeline alignment event message.
  • the InbandEventStream element includes: an event flow scheme identifier schemeIdUri attribute and an event stream value value attribute, and is set to indicate an MP time axis alignment event.
  • the MP event axis alignment event message includes at least one of: a presentation time increment presentation_time_delta field set to provide an MP time axis alignment event; an event duration event_duration field, set to indicate a media presentation remaining time period; message data message_data field, arranged to provide the alignment with respect to the media segment period period start time period where presentation time T p.
  • the MP timeline alignment event message further includes: an event message plan identifier scheme_id_uri field and an event value value field; the event message scheme identifier scheme_id_uri field value is equal to the event stream scheme identifier schemeIdUri attribute value, and the event value value field The value is equal to the event stream value value attribute value.
  • the determining module includes: a first determining unit, configured to determine, as the aligned media segment, a media segment in which the presentation_time_delta field value in the MP time axis alignment event message is a predetermined value.
  • the calculation module includes: a second determining unit configured to determine the MP time axis alignment event message a media segment having the same identity id field value is a media segment in the MP timeline alignment event; a computing unit configured to present a time increment presentation_time_delta field value in accordance with the MP timeline alignment event message, and a time scale timescale A field value that calculates the time elapsed for the media segment in the MP timeline alignment event over the media presentation time.
  • the apparatus further includes a second acquisition module configured to acquire the media segment carrying the MP timeline alignment event message.
  • the second obtaining module includes: a second acquiring unit, configured to acquire an event stream scheme identifier field and an event stream value field; and a third determining unit configured to use the event stream scheme identifier field value and the event stream value field value, Determining that the media presentation describes an in-band event stream InbandEventStream element in the MPD; and a third obtaining unit, configured to acquire a media segment corresponding to the in-band event stream InbandEventStream element as the media segment.
  • the external time axis includes a transport stream time axis.
  • yet another adaptive streaming media processing method including:
  • a location of the external media resource is determined based on the MP timeline alignment event message, and the external media resource is mapped to an external media presentation time EMPT on the MP timeline.
  • the MP timeline alignment event message includes at least:
  • the message data message_data field is set to provide a URL of the external media resource location in the MP timeline alignment event.
  • the MP timeline alignment event message includes at least:
  • a presentation time increment presentation_time_delta field is provided to provide an increment of the media presentation time of the external media resource on the MP time axis relative to the earliest presentation time of the media segment in the MP timeline alignment event.
  • the MP timeline alignment event message further includes:
  • the event message scheme identifies a scheme_id_uri field, and is set to provide an event message scheme identifier
  • the event value value field is set to provide the event value.
  • Determining the location of the external media resource according to the MP timeline alignment event message including:
  • the value of the message_data field in the MP timeline alignment event message is determined as the location of the external media resource.
  • Determining, according to the MP timeline alignment event message, that the external media resource is mapped to an external media presentation time on the MP timeline including:
  • Determining, according to the MP timeline alignment event message, that the external media resource is mapped to an external media presentation time on the MP timeline further includes:
  • the media presentation time of the external media resource on the MP timeline is equal to the earliest presentation time of the aligned media segment.
  • Determining, according to the MP time axis alignment event message, that the external media resource is mapped to an external media presentation time on the MP time axis further comprising:
  • the MP timeline alignment event message has the same message identification id field value and satisfies:
  • the EMPT id is an external media presentation time of the external media resource in the MP time axis alignment event on the MP time axis after the mapping between the MP time axis and the external time axis;
  • the i is a positive integer.
  • Obtaining the media presentation MP timeline alignment event message carried in the media segment further includes:
  • the access media presentation describes an in-band event stream InbandEventStream element in the MPD;
  • the event stream scheme of the InbandEventStream element identifies that the schemeIdUri attribute value is equal to the value of the scheme_id_uri field in the MP timeline alignment event message, and the event stream value value attribute value of the InbandEventStream element is equal to the value field in the MP timeline alignment event message. value;
  • yet another adaptive streaming media processing apparatus including:
  • the third obtaining module is configured to obtain an MP timeline alignment event message for the media carried in the media segment
  • a third determining module is configured to determine a location of the external media resource based on the MP timeline alignment event message, and the external media resource is mapped to an external media presentation time EMPT on the MP timeline.
  • the MP timeline alignment event message includes at least:
  • the message data message_data field is set to provide a URL of the external media resource location in the MP timeline alignment event.
  • the MP timeline alignment event message includes at least:
  • a presentation time increment presentation_time_delta field is provided to provide an increment of the media presentation time of the external media resource on the MP time axis relative to the earliest presentation time of the media segment in the MP timeline alignment event.
  • the MP timeline alignment event message further includes:
  • the event message scheme identifies a scheme_id_uri field, and is set to provide an event message scheme identifier
  • the event value value field is set to provide the event value.
  • the third determining module includes:
  • a fourth determining unit configured to determine a value of the message_data field in the MP timeline alignment event message as a location of the external media resource.
  • the third determining module includes:
  • a fifth determining unit configured to determine, according to an earliest presentation time EPT of the media segment, and a presentation_time_delta field value in the MP timeline alignment event message, to determine that the external media resource is mapped to an external media presentation time on the MP time axis .
  • the third determining module further includes:
  • a sixth determining unit configured to determine that the media segment whose presentation_time_delta field has a value of 0 in the MP time axis alignment event message is a media segment whose media presentation time MPT in the MP time axis is aligned with an external time axis;
  • the media presentation time of the external media resource on the MP timeline is equal to the earliest presentation time of the aligned media segment.
  • the third determining module further includes:
  • a seventh determining unit configured to confirm the same external media resource in the MP timeline alignment event, the MP timeline alignment event message having the same message identifier id field value, and satisfying:
  • the EMPT id is an external media presentation time of the external media resource in the MP time axis alignment event on the MP time axis after the mapping between the MP time axis and the external time axis;
  • the i is a positive integer.
  • the third obtaining module further includes: a second access unit configured to access an in-band event stream InbandEventStream element in the media presentation description MPD; wherein the event stream scheme identifier of the InbandEventStream element identifies a schemeIdUri attribute value equal to the MP timeline Aligning the value of the scheme_id_uri field in the event message, and the event stream value value attribute value of the InbandEventStream element is equal to the value field value in the MP timeline alignment event message; and the fourth obtaining unit is configured to obtain the medium corresponding to the InbandEventStream element The MP timeline alignment event message carried in the fragment.
  • an MP timeline alignment event message by the media carried in the media segment by using the solution of the embodiment of the present invention; determining an aligned media segment according to the MP time axis alignment event message, wherein the aligned media segment is the MP time a media segment in the in-axis media presentation time MPT aligned with an external time axis; calculating a media segment in the MP time axis alignment event at a media presentation time after the MP time axis establishes a mapping relationship with the external time axis
  • the elapsed time solves the problem caused by the single reference clock in the prior art, and realizes the synchronous play of the media in the mixed delivery mode of the broadcast network and the broadband network.
  • FIG. 1 is a schematic diagram of a first broadcast/broadband hybrid delivery media synchronization system architecture in accordance with an embodiment of the present invention
  • FIG. 2 is a flowchart of a dynamic adaptive streaming media processing method according to an embodiment of the present invention
  • FIG. 3 is a schematic diagram of acquiring an adaptive streaming media location descriptor according to an alternative embodiment of the present invention.
  • FIG. 4 is a schematic diagram of another method for acquiring an adaptive streaming media location according to an alternative embodiment of the present invention.
  • FIG. 5 is a diagram showing an example of determining media segments that require time axis alignment, in accordance with an embodiment of the present invention
  • FIG. 6 is another example diagram of determining that a time-axis aligned media segment is needed, in accordance with an embodiment of the present invention.
  • FIG. 7 is an exemplary diagram for determining that a time axis alignment of a media segment is required, in accordance with an embodiment of the present invention.
  • FIG. 8 is a flowchart of a method for processing a transport stream according to an embodiment of the present invention.
  • FIG. 9 is a structural block diagram of an adaptive streaming media processing apparatus according to an embodiment of the present invention.
  • FIG. 10 is a structural block diagram of another adaptive streaming media processing apparatus according to an embodiment of the present invention.
  • FIG. 11 is a flowchart of another adaptive streaming media processing method according to an embodiment of the present invention.
  • FIG. 12 is a schematic diagram of a second broadcast/broadband hybrid transmission medium synchronization system architecture according to an embodiment of the present invention.
  • FIG. 13 is a flowchart of a dynamic adaptive streaming media processing method according to an alternative embodiment of the present invention.
  • FIG. 14 is a diagram showing an example of determining media segments that require time axis alignment, in accordance with an embodiment of the present invention.
  • FIG. 15 is a flowchart of still another adaptive streaming media processing apparatus according to an embodiment of the present invention.
  • FIG. 16 is a structural block diagram of a first acquiring module 152 in another adaptive streaming media processing apparatus according to an embodiment of the present invention.
  • FIG. 17 is a structural block diagram of a determining module 154 in still another adaptive streaming media processing apparatus according to an embodiment of the present invention.
  • FIG. 18 is a structural block diagram of a computing module 156 in still another adaptive streaming media processing apparatus according to an embodiment of the present invention.
  • FIG. 19 is a block diagram showing a preferred structure of still another adaptive streaming media processing apparatus according to an embodiment of the present invention.
  • FIG. 20 is a structural block diagram of a second acquiring module 192 in another adaptive streaming media processing apparatus according to an embodiment of the present invention.
  • 21 is a flowchart of another adaptive streaming media processing method according to an embodiment of the present invention.
  • 22 is a diagram showing an example of determining media segments that require time axis alignment, in accordance with an embodiment of the present invention.
  • FIG. 23 is a structural block diagram of another adaptive streaming media processing apparatus according to an embodiment of the present invention.
  • FIG. 24 is a structural block diagram of a third determining module of another adaptive streaming media processing device 234 according to an embodiment of the present invention.
  • FIG. 25 is a structural block diagram of a third acquisition module of another adaptive streaming media processing device 232 according to an embodiment of the present invention.
  • FIG. 1 is a schematic diagram of a first broadcast/broadband hybrid transmission media synchronization system architecture according to an embodiment of the present invention, as shown in FIG.
  • the system shown in Fig. 1 will be described below.
  • the broadcast content source 100 may use the MPEG-2 standard, or other standards may be used.
  • MPEG-2 will be described as an example.
  • the broadcast content source 100 inputs audio data and video data to respective encoders for compression and encoding at the transmitting end to obtain an elementary stream (Elemental Stream, referred to as ES), ES of the audio and video.
  • ES elementary stream
  • the packet is divided into groups to form a packet stream (Packet Elemental Stream, PES for short), and the PES enters the stream multiplexer together with the program content information and the conditional information to generate a program stream (Program Stream) , referred to as PS) or Transport Stream (TS).
  • PS Program Stream
  • TS Transport Stream
  • the MPEG-2TS code stream payload sent by the broadcast content source 100 carries a PES packet code stream having a different packet identifier (Packet Identification, abbreviated as PID).
  • PES Packet Identification
  • the TS stream having a PID value of 0x0000 corresponds to a Program Association Table (PAT), and the PAT defines all programs in the TS, and provides program initialization information, which is a complete list of all programs in the TS, and The program number of the different programs and the PID of the Program Map Table (PMT) related to the program content are indicated.
  • PMT indicates the correspondence between the program number and the transmission code stream of audio and video and other content contained therein.
  • the program is fully defined, including the program number of the program, PCR PID, stream type (such as audio, video and other data, etc.) and elementary stream PID (pointing to the TS carrying the audio and video content) Package PID) and other information.
  • the MPEG-2 TS stream can also carry a TEMI PES code stream called Timeline and External Media Information (TEMI PES), which is set to indicate the external media resources carried in the MPEG-2 TS stream. Time information and location information.
  • TEMI PES Timeline and External Media Information
  • the adaptive streaming media (or dynamic adaptive streaming media) involved may be implemented in various manners.
  • dynamic adaptive streaming over HTTP Dynamic Adaptive Streaming over HTTP
  • DASH Dynamic Adaptive Streaming over HTTP
  • the following embodiments are equally applicable to other adaptive streaming technologies.
  • the MPD server 200 provides MPD file storage in dynamic adaptive streaming media.
  • the MPD file describes the duration, URL, media attributes of each media segment, such as video resolution, adaptation bit rate, and the like.
  • the MPD file provides enough HTTP-URL information for the DASH client to access the streaming media content resource.
  • the DASH client starts playing a video, it selects the download according to the information in the MPD file to adapt to the currently available network bandwidth and the terminal. Cache the media clips and play them.
  • the streaming server 300 completes the generation of the slice of the original video and the generation of the MPD file. Video files are segmented and stored in a hierarchy of periods, representations, and segments. Upon request, the streaming server 300 will return the appropriate media shards to the terminal.
  • the terminal 400 may include a first terminal 410, such as a digital television set top box, or a broadcast ⁇ broadband dual mode set top box; and a second terminal 420, such as a tablet computer, a smart phone, a handheld television device, and the like.
  • a first terminal 410 such as a digital television set top box, or a broadcast ⁇ broadband dual mode set top box
  • second terminal 420 such as a tablet computer, a smart phone, a handheld television device, and the like.
  • the terminal 400 after receiving the MPEG-2 TS code stream, splits the code stream into different parsing channels according to different packet identifiers PID, The audio PES and the video PES are parsed, and the ES stream is reorganized into the corresponding cache of the audio and video.
  • a channel of the demultiplexing module is bound to a TS stream with a PID value of 0x0000, and the PAT is parsed therefrom to obtain the packet identifier PID of the PMT of the desired program.
  • the program mapping table PMT is found, and the system reference clock PCR in the code stream and the PTS and other time information of the audio and video PES code stream of the related program are extracted.
  • the dynamic adaptive streaming media processing method provided by this embodiment is described below with reference to the architecture in FIG. 1:
  • the terminal 400 may further extract a specific type of adaptation domain descriptor (AF Descriptor) carried in the TEMI PES code stream and the corresponding PES.
  • AF Descriptor a specific type of adaptation domain descriptor carried in the TEMI PES code stream and the corresponding PES.
  • the code stream header field displays a time stamp (Presence Time Stamp, or PTS for short), and is set to obtain dynamic self-adaptation. Location information and time information should be streamed.
  • the terminal may request the MPD server 200 to acquire the MPD file of the dynamic adaptive streaming media according to the location information.
  • the terminal 400 parses the acquired MPD file to determine a media segment in the dynamic adaptive streaming media that needs to be time aligned with the MPEG-2 TS code stream.
  • the terminal 400 may determine, according to the time information, a media presentation time (Media Presentation Time, for example, a time axis aligned media segment and a subsequent media segment in the same time period) in the dynamic adaptive streaming media. Show timestamp PTS for MPT) relative to the MPEG-2 TS stream timeline.
  • Media Presentation Time for example, a time axis aligned media segment and a subsequent media segment in the same time period
  • the terminal may periodically send an HTTP request through a dynamic adaptation (Quality Adaptation) algorithm to obtain a media segment of a suitable code rate or quality version in the dynamic adaptive streaming media, and according to the MPEG-2 TS code.
  • a dynamic adaptation Quality Adaptation
  • the audio and video PES code stream PTS information in the stream related program and the media segment PTS information in the dynamic adaptive streaming media realize audio and video synchronization processing.
  • FIG. 2 is a flowchart of a dynamic adaptive streaming media processing method according to an embodiment of the present invention. As shown in FIG. 2, the process includes the following steps:
  • Step S202 Obtain location information and time information of the adaptive streaming media carried in the transport stream.
  • Step S204 determining, according to the location information, a media segment in the adaptive streaming media that needs time axis alignment with the transport stream;
  • Step S206 determining, according to the time information, a display timestamp of the media presentation time of the media segment in the adaptive streaming media with respect to the time axis of the transport stream.
  • the location information and the time information of the adaptive streaming media carried in the transport stream are used to determine the media segments in the adaptive streaming media that need to be time-aligned with the transport stream and their corresponding display time stamps, thereby solving the problem.
  • the problem caused by using a single reference clock for the mixed network transmission of the broadcast network and the broadband network makes the media synchronization processing in the hybrid delivery mode more efficient, and also improves the scalability of the service deployment.
  • step S202 The acquisition of the location information and the time information is described in step S202, and the acquisition of the location information and the time information will be described below in conjunction with an optional implementation.
  • the location information in step S202 may be a location descriptor (Location Descriptor).
  • FIG. 3 is a schematic diagram of acquiring an adaptive streaming media location descriptor according to an alternative embodiment of the present invention, as shown in FIG.
  • the child may also be an AF descriptor, which may be carried in the PES data payload, and the PTS time value may be carried in the PES extension header field.
  • the terminal first parses the program mapping table PMT in the obtained transport stream, and determines a packet identifier PID of the TEMI PES code stream whose stream type value is a predetermined value (for example, 0x26).
  • the terminal binds the packet identifier PID to the demultiplexing module channel, and filters out the TS packet containing the TEMI PES code stream, where the stream identifier value (streamID) of the TEMI PES code stream is private_stream_1.
  • the terminal extracts the adaptation domain AF descriptor of the TEMI PES code stream data payload TEMI_AU carrying the label field (af_descr_tag) value as a predetermined value (for example, 0x05) and the service type field (service_type) value is a predetermined value (for example, 0x01), That is, the location descriptor of the dynamic adaptive streaming media (Location Descriptor), thereby obtaining the location information of the dynamic adaptive streaming media.
  • the terminal acquires a TEMI PES code stream in the transport stream, and extracts a PTS included in the TEMI PES code stream extension header field as a PTS value corresponding to the dynamic adaptive streaming media location descriptor, thereby acquiring dynamic adaptive streaming media. Time information.
  • FIG. 4 is a schematic diagram of another method for obtaining an adaptive streaming media location descriptor according to an alternative embodiment of the present invention. As shown in FIG. 4, the AF descriptor may be carried in the TS stream. Adaptation domain.
  • the stream ID indicates whether the PES payload is audio, video or data stream.
  • the terminal acquires a specific media component (video or audio) TS code stream whose extension descriptor (Extension_descriptor_tag) value in the transport stream has a predetermined value (for example, 0x04).
  • the terminal extracts an AF descriptor whose tag field (af_descr_tag) value in the Adaptation Field of the specific media component is a predetermined value (0x05) and whose service type field (service_type) value is a predetermined value (for example, 0x01). That is, the location descriptor of the dynamic adaptive streaming media (Location Descriptor), thereby obtaining the location information of the dynamic adaptive streaming media.
  • the terminal acquires a specific media component (video or audio) TS code stream in the transport stream, and extracts the PTS included in the PES code stream extension header field carried in the TS code stream payload of the specific media component (video or audio) as a
  • the PTS value corresponding to the dynamic adaptive streaming location descriptor is obtained, thereby obtaining time information of the dynamic adaptive streaming media.
  • the terminal acquires a subsequent TS code stream having the same PID value as the specific media component (video or audio) TS code stream and a payload data unit start indicator (payload_unit_start_indicator) in the TS code stream header field.
  • the PTS included in the extended header field of the PES stream carried in the TS stream payload is extracted as a PTS value corresponding to the dynamic adaptive streaming location descriptor to obtain time information of the dynamic adaptive streaming media.
  • Table 1 shows a syntax structure chart of the adaptive streaming media location descriptor according to an embodiment of the present invention.
  • the location descriptor (Location Descriptor) of the adaptive streaming media in this embodiment is extended.
  • the definition is as follows, taking the service type as DASH as an example:
  • Timeline_id the unique identifier of the position descriptor in the MPEG-2TS code stream
  • Force_reload a 1-bit flag indicating whether to reload the MPD before DASH synchronization (ie, media segment timeline alignment);
  • Is_announcement a 1-bit flag indicating that the DASH media segment corresponding to the position descriptor has not been in a play (activated) state;
  • Time_before_activation indicates the remaining time of the resource (DASH media segment) referenced by the location descriptor into the playing state
  • Timescale the time stamp unit, set to represent the time value of the time_before_activation field
  • Service_type indicates the service type of the external resource referenced by the location descriptor.
  • the service type is "MPEG-DASH”.
  • the location information carried in the location descriptor will be set to achieve mapping between the media presentation time of the DASH media segment and the PTS in the MPEG-2 TS code stream, that is, between the completion of the DASH streaming media and the MPEG-2 TS code stream. Synchronize.
  • Event Stream the location information of the above-mentioned DASH streaming media time information and transport stream time information mapping relationship is referred to as a Timeline Alignment event stream (Event Stream), and is defined as follows:
  • URN Uniform Resource Name
  • URL Uniform Resource Locator
  • the event message plan identification field together with the event stream value field can uniquely identify the time axis aligned event stream.
  • the event ID field is set to uniquely identify a specific timeline alignment event in the timeline alignment event stream.
  • URL uniform resource locator
  • the terminal acquires the adaptive streaming media included in the program map table PMT.
  • the MPD descriptor can be defined as shown in Table 2, using DASH as an example:
  • the mpd_url takes the value of the URL of the MPD corresponding to the DASH of the adaptive streaming location descriptor.
  • step S204 describes that the location information determines a media segment in the adaptive streaming media that needs to be time-aligned with the transport stream; this will be described below in conjunction with an optional implementation.
  • the MPD is a document that contains the metadata required by the DASH client to construct the media segment HTTP-URL, so that the client can access the media segment and provide streaming services for the user.
  • the MPD may contain an event set to notify the DASH client or the DASH application of non-periodic information.
  • Events are time-controlled, that is, events start at a particular media presentation time point and usually last for a while. Events include DASH-specific notification events and application-specific events.
  • the Period element in the MPD may contain a Timeline Alignment Event Stream (EventStream) child element, where:
  • the EventStream@schemeIdUri property is set to a message scheme (Scheme) that uniquely identifies the timeline alignment event stream.
  • the EventStream@value property is used as the value of the timeline alignment event stream whose value space will be defined by the message schema of the timeline alignment event stream identified by event_schemeIdUri.
  • the EventStream element contains a series of timeline alignment event Event elements of the same type, uniquely identified by the Event.id property value.
  • the terminal may acquire the corresponding MPD according to the location information according to different methods.
  • the terminal extracts the MPD descriptor of the dynamic adaptive streaming media included in the PMT, and obtains the URL of the MPD of the dynamic adaptive streaming media corresponding to the location information according to the value of the mpd_url field included therein. or,
  • the terminal extracts the dynamic adaptive streaming media location descriptor, and obtains the URL of the MPD of the dynamic adaptive streaming media corresponding to the location information according to the mpd_url field value included therein.
  • FIG. 5 is a diagram showing an example of determining a media segment that needs time axis alignment, as shown in FIG. 5, according to an embodiment of the present invention, including:
  • the terminal retrieves and determines the Period element in the MPD according to the event stream scheme identifier field with the value of "urn:uuid:XYZY" carried in the location descriptor and the event stream value field with the value of "ad", that is, the @id attribute in FIG. A Period element with a value of "1".
  • the terminal replaces the $EventStream of the @media attribute in the media fragment template (SegmentTemplate) sub-element of the Period element by using the event stream scheme identifier field value "urn:uuid:XYZY" carried in the location descriptor and the event stream value field value "ad".
  • $ identifier replacing the $EventID$ identifier of the @media attribute in the MediaTemplate Template (SegmentTemplate) sub-element of the Period element using the event identifier (event_id) field value "1" carried in the position descriptor.
  • the next constructed media segment HTTP URL of the dynamic adaptive streaming media that needs time axis alignment with the transport stream Is " http://www.example.com/uuid:XYZY/ad_00002.ts ", and so on.
  • FIG. 6 is another example diagram for determining that a time axis alignment of a media segment is required, as shown in FIG. 6, according to an embodiment of the present invention, including:
  • the terminal retrieves and determines the Period element in the MPD according to the event stream scheme identifier field of the value "urn:uuid:XYZY" carried in the location descriptor and the event stream value field with the value of "ad", that is, the @id attribute in FIG. A Period element with a value of "1".
  • the terminal retrieves, according to the event identifier field of the value "1" carried in the location descriptor, the media segment URL (SegmentURL) sub-element under the Period element in the MPD, that is, the media segment whose @eventID attribute value is "1" in FIG. URL (SegmentURL) element.
  • FIG. 7 is a diagram showing another example of determining that a time-axis aligned media segment needs to be performed, as shown in FIG. 7, including:
  • the terminal retrieves and determines the Period element in the MPD according to the event stream scheme identifier field of the value "urn:uuid:XYZY" carried in the location descriptor and the event stream value field with the value of "ad", that is, the @id attribute in FIG. A Period element with a value of "1".
  • the terminal retrieves an event (Event) sub-element under the Period element in the MPD according to the event identifier field of the value "1" carried in the position descriptor, that is, an event in which the @id attribute value is "1" in FIG. element.
  • Event an event sub-element under the Period element in the MPD according to the event identifier field of the value "1" carried in the position descriptor, that is, an event in which the @id attribute value is "1" in FIG. element.
  • BaseURL base address
  • the method when determining a media segment that needs to be time aligned with the transport stream, the method further includes:
  • the method when determining a media segment that needs to be time aligned with the transport stream, the method further includes:
  • the terminal extracts the forced reload (force_reload) flag in the dynamic adaptive streaming media location descriptor; if the value of the forced reload (force_reload) flag is 0x01, the MPD of the dynamic adaptive streaming media is reloaded.
  • the method when determining a media segment that needs to be time aligned with the transport stream, the method further includes:
  • the terminal extracts an is_announcement flag in the dynamic adaptive streaming media location descriptor. If the value of the is_announcement flag is 0x01, the time_before_activation field in the dynamic adaptive streaming location descriptor is extracted. Timescale field.
  • the method when determining a media segment that needs to be time-aligned with the transport stream, the method further includes: extracting a forced reload flag from the location information; and if the value of the forced reload flag is a predetermined value Reload the MPD of the above DASH.
  • the method when determining a media segment that needs to be time-aligned with the transport stream, the method further includes: extracting a pre-announcement flag from the location information; and if the value of the pre-announcement flag is a predetermined value, The active countdown field and the timestamp field are extracted from the location information.
  • step S206 the PTS of the media presentation time of the media segment in the adaptive streaming media relative to the time axis of the transport stream is determined based on the time information; this will be described below in conjunction with an alternative embodiment.
  • One of the main features of DASH is that the encoded versions of different media components share a globally uniform timeline.
  • the presentation time of the access units in the media content is mapped to a global unified presentation timeline, facilitating synchronization of different media components and enabling the same seamless switching of different encoded versions of the same media component.
  • the presentation time in each time period is a value corresponding to the PeriodStart time of the corresponding time period minus the @presentationTimeOffset of the included representation, that is, T 0 .
  • the media presentation time of each media segment in the time period has a predetermined relationship, for example, may be continuous, and therefore, the time period may be derived according to the media presentation time of a media segment.
  • the media presentation time of subsequent media segments may be derived according to the media presentation time of a media segment.
  • the terminal determines the media presentation time of the subsequent media segments in the period in which the media segment is located according to the media presentation time of the media segment that needs time axis alignment with the transport stream.
  • the media presentation time of a media segment requiring time axis alignment is expressed as a PTS with respect to the time axis of the transport stream, the following relationship exists.
  • Segment s PTS(in seconds) [(S-Ssyn)*@duration]/SegmentBase.timescale+Segment syn PTS(in seconds).
  • Ssyn is the number of the media segments of the dynamic adaptive streaming media that need time axis alignment in the time period
  • S is the number of subsequent media segments in the time period
  • the Segment syn PTS is the PTS of the media presentation time of the media segment of the dynamically adaptive streaming media that needs time axis alignment relative to the time axis of the transport stream.
  • the Segment PTS is the PTS of the media presentation time of subsequent media segments in the time period relative to the time axis of the transport stream.
  • @duration and SegmentBase.timescale are the @duration attribute value and the @timescale attribute value of the time zone element of the media segment, respectively;
  • the terminal determines, according to the PTS value corresponding to the dynamic adaptive streaming location descriptor, the media presentation time of the media segment of the dynamic adaptive streaming media that needs time axis alignment relative to the transport stream time.
  • the PTS of the axis ie:
  • the PTS i is a PTS value corresponding to the dynamic adaptive streaming location descriptor.
  • the Segment syn PTS is the PTS of the media presentation time of the media segment of the dynamically adaptive streaming media that needs time axis alignment relative to the time axis of the transport stream.
  • FIG. 8 is a flowchart of a method for processing a transport stream according to an embodiment of the present invention. As shown in FIG. 8, the process includes the following steps:
  • Step S802 adding location information and/or time information of the adaptive streaming media to the transport stream, where the location information is set to determine a media segment in the adaptive streaming media that needs to be time-aligned with the transport stream, and the time information is set. Determining a display timestamp of a media presentation time of a media segment in the adaptive streaming media relative to a time axis of the transport stream;
  • Step S804 the foregoing transport stream is sent to the user terminal.
  • the following method may be adopted: adding a PID in the PMT of the transport stream, where the PID is set to obtain the stream type value as a predetermined value.
  • a valued TEMI PES code stream carrying location information may be adopted: adding a PID in the PMT of the transport stream, where the PID is set to obtain the stream type value as a predetermined value.
  • adding the time information carrying the adaptive streaming media to the transport stream includes: The TEMI PES stream extends the header field to carry time information.
  • the following method may also be adopted: carrying the specific media component TS code of the extended description subtag with a value of a predetermined value in the transport stream. a stream, wherein the TS code stream adaptation domain carries location information.
  • adding the time information carrying the adaptive streaming media to the transport stream includes:
  • the PTS extended header field of the code stream payload carries a PTS, wherein the PTS value is set to determine time information of the adaptive streaming media.
  • Adding the time information carrying the adaptive streaming media in the transport stream may also adopt the following method: setting a TS stream corresponding to a specific media component TS in the transport stream and having the same PID value and TS header field in the specific media component TS code stream
  • the payload data unit start indicator takes a value of a predetermined value; the PTS is carried in the PES extended header field of the subsequent TS stream payload, wherein the PTS value determines time information of the adaptive streaming medium.
  • the AF descriptor carrying the label field value as a predetermined value and the service type field value is a predetermined value in the transport stream is used as a position descriptor of the adaptive streaming media, and the location descriptor includes location information.
  • the location descriptor further includes at least one of the following: a unique identifier of the location descriptor in the transport stream, a forced reload flag, a pre-announcement flag, an activation countdown flag, a time stamp unit, and a service.
  • the forced reload flag is set to indicate whether a media presentation description MPD needs to be reloaded before the media segment timeline alignment is performed, the pre-announcement flag being set to indicate that the media segment of the adaptive media corresponding to the position descriptor is not yet playing
  • the activation countdown flag is set to indicate a remaining time of the media segment of the adaptive media corresponding to the location descriptor to enter a playing state
  • the time stamp unit is set to indicate a remaining time identification field time value
  • the service type is set to indicate that the location descriptor is referenced The type of business for external resources.
  • the foregoing transport stream may further carry information about the media presentation description MPD that is set to determine the adaptive streaming media corresponding to the location information.
  • the MPD descriptor is carried in the PMT in the foregoing transport stream; wherein the MPD descriptor includes a URL of the MPD corresponding to the location information; or the URL of the MPD is obtained from the location information.
  • An adaptive streaming media processing device is also provided in this embodiment.
  • the device is configured to implement the foregoing embodiments and preferred embodiments, and details are not described herein.
  • the term "module” may implement a combination of software and/or hardware of a predetermined function.
  • the apparatus described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware, is also possible and contemplated.
  • FIG. 9 is a structural block diagram of an adaptive streaming media processing apparatus according to an embodiment of the present invention. As shown in FIG. 9, the apparatus includes an obtaining module 92, a first determining module 94, and a second determining module 96. The adaptive media stream processing device will be described.
  • the obtaining module 92 is configured to obtain the location information and the time information of the adaptive streaming media carried in the transport stream.
  • the first determining module 94 is connected to the acquiring module 92, and is configured to determine the need for the adaptive streaming media according to the location information. Transmitting a time-axis aligned media segment; the second determining module 96 is coupled to the first determining module 94, and configured to determine, according to the time information, the media presentation time of the media segment in the adaptive streaming media relative to the time axis of the transport stream Timestamp.
  • FIG. 10 is a structural block diagram of another adaptive streaming media processing apparatus according to an embodiment of the present invention, as shown in FIG. As shown, the apparatus includes an add module 102 and a transmit module 104, the adaptive media stream processing apparatus being described below.
  • the adding module 102 is configured to add location information and/or time information of the adaptive streaming media in the transport stream, wherein the location information is set to determine a media segment in the adaptive streaming media that needs to be time-aligned with the transport stream.
  • the time information is set to determine the media presentation time of the media segment in the adaptive streaming media relative to the time axis of the transport stream;
  • the sending module 104 is coupled to the adding module 102 and configured to send the transport stream to the user terminal.
  • EventStream EventStream
  • EventStream EventStream element in the Period element.
  • an in-band event notification can also be passed.
  • event information can be added to the Segment as part of the media segment to stream the event through Representation.
  • the event stream can exist in one or more Representations associated with an AdaptationSet, or it can exist in all Representations.
  • the Representation carrying the in-band event should be identified in the MPD to inform the client.
  • the MPD uses an in-band event stream (InbandEventStream) element to specify an in-band event stream that can appear in the AdaptationSet and Representation levels. If a Representation contains multiple in-band event streams, each in-band event stream should be specified by a separate InbandEventStream element.
  • InbandEventStream in-band event stream
  • FIG. 11 is a flowchart of another adaptive streaming media processing method according to an embodiment of the present invention. As shown in FIG. 11, the method includes the following steps:
  • Step S1102 Acquire a media presentation MP timeline alignment event message carried in the media segment.
  • Step S1104 Determine an aligned media segment according to the MP timeline alignment event message, wherein the aligned media segment is a media segment in which the media presentation time MPT in the MP time axis is aligned with an external time axis;
  • Step S1106 calculating a time elapsed during the media presentation time of the media segment in the MP time axis alignment event after the MP time axis establishes a mapping relationship with the external time axis.
  • the media segment whose media presentation time MPT is aligned with the external time axis is determined according to the MP time axis alignment event message carried in the media segment, and the MP is calculated after the MP time axis establishes a mapping relationship with the transport stream time axis.
  • the time elapsed in the media presentation time of the media segment in the time axis alignment event can effectively solve the problem caused by adopting a single reference clock for the mixed network transmission of the broadcast network and the broadband network in the related art, so that the media synchronization processing in the mixed delivery mode More efficient, while also improving the scalability of business deployment.
  • the MPEG-2 TS PCR/PTS Program Reference Clock/Display Time Stamp
  • the audio and video coding methods of other formats can also be used as the external time axis.
  • the MPEG-2TS code stream payload sent by the broadcast content source carries a PES packet code stream having a different packet identifier (Packet Identification, abbreviated as PID).
  • PID Packet Identification
  • the terminal After receiving the MPEG-2TS code stream, the terminal offloads the code stream to different parsing channels according to different packet identifiers PID, parses out the audio PES and the video PES, and reconstructs the ES stream into the corresponding cache of the audio and video.
  • a channel of the demultiplexing module is bound to a TS stream with a PID value of 0x0000, and the PAT is parsed therefrom to obtain the packet identifier PID of the PMT of the desired program.
  • the program mapping table PMT is found, and the system reference clock PCR in the code stream and the PTS and other time information of the audio and video PES code stream of the related program are extracted.
  • the MPEG-2TS code stream may carry a TEMI PES code stream called Timeline and External Media Information (TEMI PES), which is set to indicate external media synchronized with the MPEG-2 TS code stream. Resources.
  • the terminal can obtain the location information of the external media resource by extracting a specific type of adaptation field descriptor (AF Descriptor) carried in the TEMI PES code stream, and establish the MPEG and MPEG.
  • AF Descriptor adaptation field descriptor
  • DASH is taken as an example for description.
  • One of the main characteristics of DASH is that the coded versions of different media components share a global unified time axis.
  • the presentation time of the access units in the media content is mapped to a global unified presentation timeline, facilitating synchronization of different media components and enabling the same seamless switching of different encoded versions of the same media component.
  • the presentation time in each time period is the value of the Period starting time relative to the corresponding period minus the @presentationTimeOffset of the included representation, that is, T O .
  • T M T P -T O .
  • An obvious way to solve timeline alignment is to use a single reference clock for adaptive streaming and external media streaming.
  • the above MPEG-2 TS stream PCR/PTS (Program Reference Clock/Display Time Stamp) clock information is also included in an IP packet such as RTP, HTTP, and the like.
  • the main drawback of this method is that the broadcast media and broadband media source are bundled and deployed, which is not conducive to system expansion, and the PCR clock information may be changed during demultiplexing. The law maintains the continuity of the clock.
  • FIG. 12 is A schematic diagram of a second broadcast/broadband hybrid transmission medium synchronization system architecture according to an embodiment of the present invention is shown in FIG. 12: wherein, regarding the broadcast content source 100, the MPD server 200, the streaming media server 300, the first terminal 410, and the second The description of the terminal 420 has been described in detail in the related description of FIG. 1, and details are not described herein again.
  • This embodiment defines a "Media Presentation Timeline Alignment” event, referred to as an "MP Timeline Alignment” event, which is set to notify the client that the media presentation time (MPT) of the media segment will be mapped to an external timeline, such as : MPEG-2TS PCR/PTS (Program Reference Clock/Display Time Stamp) clock information, or vice versa.
  • MPT media presentation time
  • the "Timeline Alignment Event Flow” carries a series of timeline alignment event time schedules (or locations) Tj , by adding Tj signaling in the media stream and establishing time information in the respective timelines.
  • the mapping relationship with T j that is, (PTS i , T j ) and (MPT k , T j ) in FIG. 12, can establish a mapping relationship between the media presentation (MP) time axis and the external time axis.
  • the media presentation (MP) timeline aligns with the external timeline key elements, as shown in Table 3:
  • a media presentation (MP) timeline alignment signaling mechanism needs to be defined for adaptive streaming, supporting timeline alignment progress (location) information and corresponding media presentation time (MPT).
  • the information of the MP time axis alignment event may be an MP time axis alignment event message.
  • the media carried in the obtained media segment in the above step S1102 is displayed in the MP time axis alignment.
  • the event message may include the steps of: accessing the media presentation description in-band event stream InbandEventStream element in the MPD; and acquiring the media presentation MP timeline alignment event message carried in the media segment corresponding to the inband event stream InbandEventStream element.
  • the above InbandEventStream element may include: an event flow scheme identifier schemeIdUri attribute field and an event stream value attribute, and is set to indicate an MP time axis alignment event.
  • the foregoing MP time axis alignment event message includes at least one of: a presentation time increment presentation_time_delta field, a time set to provide an MP time axis alignment event, and an event duration event_duration field, configured to represent the media the remaining presentation time; message_data message data field arranged to provide alignment with respect to the media segment period period start time period where the presentation time T p.
  • the foregoing MP time axis alignment time message further includes: an event message plan identifier scheme_id_uri field and an event value value field; the event message scheme identifier scheme_id_uri field value is equal to the event stream scheme identifier schemeIdUri attribute value, and the event value value field value is equal to the event stream value Value attribute value.
  • determining the media segment may include determining that the presentation_time_delta field value in the MP timeline alignment event message is predetermined.
  • the following manner may be adopted: determining that the media segment having the same identifier id field value in the MP time axis alignment event message is the media segment in the MP time axis alignment event; The time increment presentation_time_delta field value is presented in the axis alignment event message, and the time scale timescale field value is calculated to calculate the time elapsed during the media presentation time of the media segment in the MP timeline alignment event.
  • MPT MPT 0 +emsg.presentation_time_delta/emsg.timescale
  • MPT 0 and MPT represent MP time axis alignment events, respectively.
  • Media presentation time of aligned media segments and subsequent media segments; emsg.presentation_time_delta and emsg.timescale represent media rendering time increment values and time scale values in the MP timeline alignment event message, respectively.
  • the method before acquiring the media carried in the media segment to present the MP timeline alignment event message, the method further includes: acquiring the media segment carrying the MP timeline alignment event message.
  • the media segment carrying the MP time axis alignment event message may be obtained by: acquiring an event stream scheme identifier field and an event stream value field; determining the media presentation description by using the event stream scheme identifier field value and the event stream value field value.
  • the external time axis described above includes a transport stream timeline.
  • step S1102 in FIG. 11 Obtained in step S1102 in FIG. 11 is an MP timeline alignment event message.
  • the MP timeline alignment event message may be implemented using an MP timeline alignment event message, although it may be used. Other ways to achieve.
  • An alternative embodiment of using the MP Timeline Alignment Time message ⁇ is illustrated in FIG.
  • FIG. 13 is a flowchart of a dynamic adaptive streaming media processing method according to an alternative embodiment of the present invention. As shown in FIG. 13, the process includes the following steps:
  • step S1302 the client obtains the media presentation MP timeline alignment event message carried in the media segment.
  • the InbandEventStream element in the MPD that specifies the "MP Timeline Alignment" in-band event stream contains an @schemeIdUri attribute that is set to provide a URI to specify the event flow scheme, for example: "urn:avs:dash:event:2014"; and an optional
  • the @value attribute is set to the value of the specified event stream element, for example: "3".
  • the URI of the identification mechanism may be a URN or a URL.
  • Event Message box (‘emsg’) is set to notify general in-band events related to media presentation time.
  • the general syntax is defined as follows:
  • Scheme_id_uri Identifies the event message scheme.
  • Value specifies the value of the event.
  • Timescale specifies the time scale.
  • Presentation_time_delta Specifies the media rendering time increment.
  • Event_duration Specifies the time period of the event on the media presentation time.
  • Id Identifies the message instance.
  • Message_data[] The body of the message.
  • the "scheme_id_uri” field in the event message is set to a predetermined value, for example: “urn:avs:dash:event:2014” and the "value” field is set to a predetermined value, for example: “3”
  • the event message ⁇ is the "MP Timeline Alignment” event message, and the message field carried by it follows the following rules:
  • the "presentation_time_delta" field provides the time of the MP timeline alignment event, indicating that the media segment (with the same "id” field value) in the MP timeline alignment event is in the media since the MP timeline is mapped to the external timeline. Renders an increment in time.
  • Event_duration represents the time remaining from the time of the MP timeline alignment event and the media presentation. If “event_duration” takes a value of "0”, the media presentation is terminated after the MP timeline is aligned with the event time; if “event_duration” takes the value "0xFFFF", the remaining time period of the media presentation is unknown.
  • Step S1304 The client determines, according to information carried in the MP timeline alignment event message (for example, time information), a media segment whose media presentation time MPT is aligned with an external time axis in the MP time axis, and calculates an MP time axis and an external time axis. After the mapping relationship is established, the MP timeline aligns the time that the media segment in the event elapses during the media presentation time.
  • information carried in the MP timeline alignment event message for example, time information
  • the MP timeline aligns the time that the media segment in the event elapses during the media presentation time.
  • the segment of the event message "presentation_time_delta" field with a value of "0" is the segment in which the media presentation time in the MP time axis is aligned with the external time axis.
  • the event message carried in different segments in the MP timeline alignment event may have the same "id” field value and different "presentation_time_delta” field values. Calculating the media presentation time increment of the segment corresponding to the message instance having the same "id” field value, indicating that the MP time axis and the external time axis establish a time information mapping relationship, and the related segments in the MP time axis alignment event are on the media presentation time.
  • the time elapsed namely:
  • MPT MPT 0 +emsg.presentation_time_delta/emsg.timescale, where
  • MPT 0 and MPT represent the aligned segments in the MP time axis alignment event and the subsequent segment media presentation time
  • Emsg.presentation_time_delta and emsg.timescale represent media rendering time increment values and time scale values in the MP timeline alignment event message.
  • FIG. 14 is a diagram showing an example of determining a media segment that requires time axis alignment, as shown in FIG. 14, according to an embodiment of the present invention, including:
  • the terminal identifies the field value according to the event flow scheme carried in the location descriptor, for example: "urn: avs:dash:event:2014”, and the event stream value field, for example, the value is "3", and the retrieval determines the Period element in the MPD.
  • the InbandEventStream element in the AdaptationSet and Representation levels that is, the InbandEventStream element contained in the Period element with the @id attribute values of "1" and "2" in Figure 14.
  • the terminal uses the MediaTemplate template (SegmentTemplate) sub-element or the Media Fragment URL (SegmentURL) sub-element of the Period element, and combines the resource URL address base address (BaseURL) element defined in other levels of the MPD, for example: "http://www .example.com/", thereby constructing the HTTP URL of the media segment corresponding to the in-band event stream, ie the different pointing URLs in Figure 14.
  • MediaTemplate SegmentTemplate
  • SegmentURL Media Fragment URL
  • the terminal acquires the content of the media segment according to the above HTTP URL request, and extracts an event message ⁇ carried therein, that is, the “MP time axis alignment” event information defined by the present invention.
  • the terminal identifies the "id" field of the "MP Timeline Alignment” event message according to the event identifier field value carried in the location descriptor, for example, "123”, and sets the value of the "presentation_time_delta" field in the search result to "0".
  • the corresponding media segment is determined to be a media segment of the dynamically adaptive streaming media that needs to be time aligned with the transport stream, ie, the first media segment from the left in FIG.
  • the terminal determines the media presentation time of the subsequent media segments in the period in which the media segment is located according to the media presentation time of the media segment that needs time axis alignment with the transport stream.
  • the media presentation time of a media segment requiring time axis alignment is expressed as a PTS relative to the time axis of the transport stream, the following relationship exists:
  • Segment s PTS(in seconds) Segment syn PTS(in seconds)+emsg.presentation_time_delta/emsg.timescale, where:
  • the subscript syn is the number of the media segment (the "presentation_time_delta” field value "0" in the "MP Timeline Alignment” event message) that needs time axis alignment;
  • the subscript s is the number of the subsequent media segment (the "MP Timeline Alignment" event message carried with the same "id” field value);
  • Emsg.presentation_time_delta is the media rendering time increment value in the "MP Timeline Alignment" event message carried by the media segment Segment s ;
  • Emsg.timescale is the time scale value in the "MP Timeline Alignment" event message
  • Segment syn PTS and Segment s PTS are respectively the media segments that need time axis alignment and the media presentation time of the subsequent media segments relative to the PTS of the transport stream time axis.
  • FIG. 21 is a flowchart of another adaptive streaming media processing method according to an embodiment of the present invention. As shown in FIG. 21, the method includes the following steps:
  • Step S2102 Acquire a media presentation (MP) timeline alignment event message carried in the media segment;
  • MP media presentation
  • Step S2104 Determine a location of the external media resource according to the MP timeline alignment event message, and the external media resource is mapped to an external media presentation time (EMPT) on the MP timeline.
  • EMPT external media presentation time
  • the media segment whose media presentation time MPT is aligned with the external time axis is determined according to the MP time axis alignment event message carried in the media segment, and the MP is calculated after the MP time axis establishes a mapping relationship with the transport stream time axis.
  • the time elapsed in the media presentation time of the media segment in the time axis alignment event can effectively solve the problem caused by adopting a single reference clock for the mixed network transmission of the broadcast network and the broadband network in the related art, so that the media synchronization processing in the mixed delivery mode More efficient, while also improving the scalability of business deployment.
  • step S2102 in FIG. 21 is an MP timeline alignment event message.
  • the MP timeline alignment event message may be implemented using an MP timeline alignment event message, although it may be used. Other ways to achieve.
  • An alternative embodiment of using the MP Timeline Alignment Time message is described below.
  • the Media Presentation timeline alignment event is set to notify the client of the location of the external media asset and the external media resource is mapped to the media presentation time on the MP timeline.
  • the "presentation_time_delta" field provides an increment of the media presentation time of the external media resource on the MP timeline in the MP timeline alignment event relative to the earliest presentation time of the segment;
  • the "message_data[]" field provides the URL of the external media resource location in the MP timeline alignment event.
  • the segment of the event message "presentation_time_delta" field with a value of "0" is the segment in which the media presentation time in the MP time axis is aligned with the external time axis, that is, the media presentation time of the external media resource on the MP time axis is equivalent to alignment.
  • the earliest presentation time of the fragment is the segment in which the media presentation time in the MP time axis is aligned with the external time axis, that is, the media presentation time of the external media resource on the MP time axis is equivalent to alignment.
  • the MP Timeline Alignment Event Messages instance must have the same "id" field value and satisfy:
  • ⁇ EMPT id represents the external media presentation time of the external media resources on the MP time axis after the MP time axis is mapped to the external time axis.
  • ⁇ Represents the earliest rendering time of segment i in the MP timeline alignment event
  • ⁇ versus Represents the MP timeline alignment event message, the media rendering time increment value and the time scale value in the instance i, respectively.
  • FIG. 22 is a diagram showing an example of determining a media segment that needs time axis alignment, as shown in FIG. 22, according to an embodiment of the present invention, including:
  • the multiple Representation elements of the MPD document contain an InbandEventStream element corresponding to this event scheme.
  • MPT media presentation time
  • the information of the external media resource relative to the fragment's earliest presentation time increment (presentation_time_delta) and the location of the external media resource (message_data[]) will be Add to the corresponding MP timeline alignment event message ⁇ .
  • MP timeline alignment event messages can appear in multiple segments.
  • the terminal identifies the field value according to the event flow scheme, for example, "urn:avs:dash:event:2014”, and the event stream value field value, for example, "3”, and retrieves the InbandEventStream element in the AdaptationSet and Representation levels of the Period element in the MPD. That is, the InbandEventStream element included in the Period element whose @id attribute values are "1" and "2" in Fig. 22, respectively.
  • the terminal uses the MediaTemplate template (SegmentTemplate) sub-element or the Media Fragment URL (SegmentURL) sub-element of the Period element, and combines the resource URL address base address (BaseURL) element defined in other levels of the MPD, for example: "http://www .example.com/", thereby constructing the HTTP URL of the media segment corresponding to the in-band event stream, ie the different pointing URLs in Figure 22.
  • MediaTemplate SegmentTemplate
  • SegmentURL Media Fragment URL
  • the terminal acquires the content of the media segment according to the above HTTP URL request, and extracts an event message ⁇ carried therein, that is, the “MP time axis alignment” event information defined by the present invention.
  • the terminal retrieves the "id" field of the "MP Timeline Alignment” event message according to the event identification field value, for example: “123”, and determines the media segment corresponding to the "presentation_time_delta” field value in the search result to be "0" event message ⁇ A media segment of a dynamically adaptive streaming media that will be time aligned with an external media resource, such as a transport stream, ie, a second media segment from the left in FIG.
  • an external media resource such as a transport stream, ie, a second media segment from the left in FIG.
  • FIG. 15 is a flowchart of still another adaptive streaming media processing apparatus according to an embodiment of the present invention. As shown in FIG. 15, the apparatus includes a first obtaining module 152, a determining module 154, and a calculating module 156. Be explained.
  • the first obtaining module 152 is configured to obtain a media presentation MP time axis alignment event message carried in the media segment; the determining module 154 is connected to the first obtaining module 152, and is configured to determine the aligned media segment according to the MP time axis alignment event message.
  • the aligned media segment is the media presentation time in the MP timeline a media segment aligned with the external time axis by the MPT; the calculation module 156 is coupled to the determining module 154, configured to calculate that the media segment in the MP time axis alignment event is after the mapping between the MP time axis and the transport stream time axis The media presents the time elapsed over time.
  • FIG. 16 is a structural block diagram of a first acquisition module 152 in another adaptive streaming media processing apparatus according to an embodiment of the present invention.
  • the first acquisition module 152 includes a first access unit 162 and a first acquisition. Unit 164, the first acquisition module 152 is described below.
  • the first access unit 162 is configured to access an in-band event stream InbandEventStream element in the media presentation description MPD.
  • the first obtaining unit 164 is connected to the first access unit 162, and is configured to acquire media corresponding to the inband event stream element of the inband event stream.
  • the InbandEventStream element may include: an event flow scheme identifier schemeIdUri attribute and an event stream value attribute, and is set to indicate an MP time axis alignment event.
  • the foregoing MP time axis alignment event message includes at least one of: a presentation time increment presentation_time_delta field, a time set to provide an MP time axis alignment event, and an event duration event_duration field, configured to represent the media the remaining presentation time; message_data message data field, aligned to provide the above presentation media segment is located with respect to the time period T p period start time period.
  • the MP time axis alignment event message may further include: an event message plan identifier scheme_id_uri field and an event value value field; the event message scheme identifier scheme_id_uri field value is equal to the event stream scheme identifier schemeIdUri attribute value, and the event value value field value is equal to the event Stream value attribute value.
  • FIG. 17 is a structural block diagram of a determining module 154 in another adaptive streaming media processing apparatus according to an embodiment of the present invention. As shown in FIG. 17, the determining module 154 includes a first determining unit 172, and the determining module 154 Be explained.
  • the first determining unit 172 is configured to determine a media segment in which the presentation_time_delta field value in the MP time axis alignment event message is a predetermined value as the aligned media segment.
  • FIG. 18 is a structural block diagram of a computing module 156 in another adaptive streaming media processing apparatus according to an embodiment of the present invention. As shown in FIG. 18, the computing module 156 includes a second determining unit 182 and a computing unit 184. The calculation module 156 is described.
  • the second determining unit 182 is configured to determine that the media segment having the same identifier id field value in the MP time axis alignment event message is a media segment in the MP time axis alignment event; the calculating unit 184 is connected to the second determining unit 182, and is configured to To calculate the time elapsed during the media presentation time of the media segment in the MP timeline alignment event, according to the presentation time increment presentation_time_delta field value in the MP timeline alignment event message and the time scaled timescale field value.
  • FIG. 19 is a block diagram showing a preferred structure of an adaptive streaming media processing apparatus according to an embodiment of the present invention. As shown in FIG. 19, the apparatus includes a second obtaining module 192 in addition to all the modules shown in FIG. The device will be described below.
  • the second obtaining module 192 is connected to the first acquiring module 152, and is configured to acquire a media segment carrying the MP time axis alignment event message.
  • FIG. 20 is a structural block diagram of a second obtaining module 192 in another adaptive streaming media processing apparatus according to an embodiment of the present invention.
  • the second obtaining module 192 includes a second obtaining unit 202, and a third determining.
  • the unit 204 and the third obtaining unit 206 will be described below for the second obtaining module 192.
  • the second obtaining unit 202 is configured to obtain an event stream plan identifier field and an event stream value field.
  • the third determining unit 204 is connected to the second acquiring unit 202, and is configured to use the event stream scheme identifier field value and the event stream value field value. Determining that the media presentation describes an in-band event stream InbandEventStream element in the MPD; the third obtaining unit 206 is coupled to the third determining unit 204, and configured to acquire a media segment corresponding to the in-band event stream InbandEventStream element as the media segment.
  • FIG. 23 is a structural block diagram of another adaptive streaming media processing apparatus according to an embodiment of the present invention. As shown in FIG. 23, the apparatus includes a third obtaining module 232 and a third determining module 234. .
  • the third obtaining module 232 is configured to obtain the media presentation MP time axis alignment event message carried in the media segment
  • the third determining module 234 is connected to the third acquiring module 232, and is configured to determine according to the MP time axis alignment event message.
  • the location of the external media asset, and the external media asset is mapped to the external media presentation time on the MP timeline.
  • the MP timeline alignment event message at least includes: a message data message_data field, configured to provide a URL of the external media resource location in the MP timeline alignment event.
  • the MP timeline alignment event message at least includes: a presentation time increment presentation_time_delta field, configured to provide a media presentation time of the external media resource on the MP time axis in the MP timeline alignment event relative to the The media clip is the first to show an increment of time.
  • a presentation time increment presentation_time_delta field configured to provide a media presentation time of the external media resource on the MP time axis in the MP timeline alignment event relative to the The media clip is the first to show an increment of time.
  • the MP time axis alignment event message further includes: an event message plan identifier scheme_id_uri field, configured to provide an event message plan identifier; and an event value value field, configured to provide an event value.
  • FIG. 24 is a structural block diagram of a third determining module 234 of another adaptive streaming media processing apparatus according to an embodiment of the present invention. As shown in FIG. 24, the third determining module 234 includes a fourth determining unit 2342. The third determining module 234 is described.
  • the fourth determining unit 2342 is configured to determine a value of the message_data field in the MP time axis alignment event message as a location of the external media resource.
  • the third determining module may also include:
  • the fifth determining unit 2344 is configured to determine that the external media resource is mapped to an external media presentation on the MP time axis according to the earliest presentation time EPT of the media segment and the presentation_time_delta field value in the MP time axis alignment event message. time.
  • the third determining module 234 may further include:
  • the sixth determining unit 2346 is connected to the fifth determining unit 2344, and is configured to determine that the media segment whose presentation_time_delta field has a value of 0 in the MP time axis alignment event message is the media presentation time MPT and the external time axis in the MP time axis. Aligned media segments; wherein the media presentation time of the external media asset on the MP timeline is equal to the earliest presentation time of the aligned media segments.
  • the third determining module may further include:
  • a seventh determining unit 2348 coupled to the fifth determining unit 2344 and/or the sixth determining unit 2346, is configured to confirm the same external media resource in the MP timeline alignment event, the MP timeline alignment event message having the same message Identifies the id field value and satisfies:
  • the EMPT id is an external media presentation time of the external media resource in the MP time axis alignment event on the MP time axis after the mapping between the MP time axis and the external time axis;
  • the i is a positive integer.
  • FIG. 25 is a structural block diagram of a third obtaining module 232 of another adaptive streaming media processing apparatus according to an embodiment of the present invention.
  • the third obtaining module 232 includes a second access unit 2322 and a fourth obtaining. Unit 2324, the third acquisition module 232 is described below.
  • a second access unit 2322 configured to access an in-band event stream InbandEventStream element in the media presentation description MPD;
  • the event stream scheme identifier of the InbandEventStream element identifies that the schemeIdUri attribute value is equal to the value of the scheme_id_uri field in the MP timeline alignment event message, and the event stream value value attribute value of the InbandEventStream element is equal to the MP time axis alignment event message. Value field value;
  • the fourth obtaining unit 2324 is connected to the second access unit 2322, and is configured to acquire an MP timeline alignment event message carried in the media segment corresponding to the InbandEventStream element.
  • the external time axis includes a transport stream time axis.
  • modules or steps of the present invention described above can be implemented by a general-purpose computing device that can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device such that they may be stored in the storage device by the computing device and, in some cases, may be different from the order herein.
  • the steps shown or described are performed, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps thereof are fabricated as a single integrated circuit module.
  • the invention is not limited to any specific combination of hardware and software.
  • an adaptive streaming media processing method and apparatus provided by an embodiment of the present invention have the following beneficial effects: solving the problem caused by using a single reference clock in the prior art, and implementing hybrid delivery of a broadcast network and a broadband network.
  • the media is played synchronously in mode.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

本发明提供了一种自适应流媒体处理方法及装置，其中，该方法包括：获取媒体片段中携带的媒体呈现（MP）时间轴对齐事件消息（S1102）；根据该MP时间轴对齐事件消息确定对齐的媒体片段，其中，该对齐的媒体片段是该MP时间轴中媒体呈现时间（MPT）与外部时间轴对齐的媒体片段（S1104）；计算在MP时间轴与外部时间轴建立映射关系之后，该MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间（S1106）。本发明解决了相关技术中为广播网络和宽带网络混合媒体传输采用单一参考时钟所导致的问题，使得混合传输模式下的媒体同步处理更加高效。

Description

自适应流媒体处理方法及装置 技术领域
本发明涉及通信领域，具体而言，涉及一种自适应流媒体处理方法及装置。
背景技术
用户通过电视、台式机、平板电脑、手机等固定移动通信终端访问业务提供商和/或内容提供商开发部署的通信服务，该访问是通过借助于定制化的通信客户端能力，在用户终端与业务服务器(内容服务器)之间建立通信链路来实施完成的，上述通信链路通常由诸如各种类型数字用户线路(X Digital Subscriber Line，简称为xDSL)、光纤接入(Fiber-to-the-x，简称为FTTx)、有线电视电缆Cable等有线通信网络，以及Wi-Fi、2G、3G、长期演进(Long Term Evolution，简称为LTE)等无线通信网络建立。
数字视频广播(Digital Video Broadcasting，简称为DVB)是世界主流数字电视标准之一，在MPEG-2音视频数据压缩算法和复用技术的基础上，DVB标准提供了一套完整的、适用于不同媒介的数字电视系统的规范，定义了有线(DVB-C)、卫星(DVB-S)和地面(DVB-T)三种供媒体数据流传输用的信道编码和调制方式，同时扩展了MPEG-2标准的系统层，丰富了网络接口。
DASH(Dynamic Adaptive Streaming over HTTP)是一种自适应比特率流媒体技术，利用互联网上传统的HTTP Web服务器提供高品质的流媒体内容。DASH将一段视频切分成许多小的HTTP服务器上的文件分块，每个文件分块包含一个较短播放时间的视频内容。这些小的视频分块可能是数个小时的电影(点播)或体育赛事的现场直播视频内容的一部分。视频可能被编成多个码率的视频片段，以覆盖不同网络带宽需求。
在上述通信服务发展过程中，终端能力和网络环境不断提升，而用户对业务体验的需求也在提高，以自适应流媒体为代表的OTT技术具备随时随地消费视频的灵活性，而IPTV、数字电视、卫星电视等传统广播技术具有大屏幕的高清晰度和质量保证。因此，将OTT技术与数字电视等广播技术相互融合，才能提供适合多种场景的综合方案。典型地，如第二屏(Second Screen)应用目前广受关注，用户利用智能手机、平板电脑等“TV buddies”，消费与TV屏广播节目相关联的个性化媒体信息。其间，业务运营商利用广播网络(譬如DVB-C数字电视网络)为广大用户传输传统电视节目，同时借助互联网宽带网络(OTT模式)为特定用户群体额外递送具有长尾效应的互联网点播内容。这种混合递送(Hybrid Delivery)模式能够节约网络投资成本，综合广播网络和宽带网络技术优势，从而为用户提供内容丰富、个性化的业务体验。
上述混合递送模式具有内容来源独立、传输网络异构，以及差异化的播放终端等显著特征。其中，能否建立广播节目或者主要媒体内容(Primary Media)与宽带节目或者辅助媒体内容(Ancillary Media)之间时间关联，即，能否保证不同内容来源、不同传输格式、以及不同播放终端的媒体内容之间的同步播放，已成为此类应用能否被用户接受的关键。
音视频特征识别技术及其应用是当前解决上述混合递送模式下媒体内容同步问题的一个研究方向，它利用音视频媒体内容自身的数字特征作为媒体间时间同步参考。其中，包括：1)数字水印技术。采集电视节目中音频信号的数字水印，利用数字水印中节目频道标识符和时间戳信息，可以检测正在播放节目的时间进度，从而获取适当的辅助媒体内容与之同步播放；2)数字指纹技术。从音视频样本中提取数字指纹信息，利用指纹检索数据库，获取媒体同步信息。
然而，数字水印和数字指纹属于计算密集型技术实现，都依赖于第二屏终端捕获音视频样本质量和特征提取算法性能；同时，此类技术受环境噪音和终端能力影响较大，适用场景存在一定的局限性。
针对混合递送模式下媒体同步播放问题的另一个研究方向是为广播网络和宽带网络媒体递送采用单一的参考时钟。其中一个方法是将传输流，例如，MPEG-2TS码流，中携带的PCR/PTS(节目参考时钟/显示时间戳)时钟信息同时也包含在诸如实时传输协议(Real-time Transport Protocol，简称为RTP)的互网络协议(Internet Protocol，简称为IP)报文中传输。该方案的主要缺陷是传输流中的PCR时钟信息在解复用时可能被改变(但是，宽带运营商无法获知这一点)，从而无法维护时钟的延续性。
针对上述广播网络和宽带网络中混合递送模式下的媒体同步播放问题，需要研究一种改进的信令机制。
发明内容
本发明实施例提供了一种自适应流媒体处理方法及装置，以至少解决相关技术中广播网络和宽带网络混合递送模式下的媒体同步播放问题。
根据本发明的一个实施例，提供了一种自适应流媒体处理方法，包括：获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息；根据所述MP时间轴对齐事件消息确定对齐的媒体片段，其中，所述对齐的媒体片段是所述MP时间轴中媒体呈现时间MPT与外部时间轴进行对齐的媒体片段；计算在所述MP时间轴与所述外部时间轴建立映射关系之后，所述MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间。
获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息，包括：访问媒体呈现描述MPD中带内事件流InbandEventStream元素；获取对应于所述带内事件流InbandEventStream元素的媒体片段中携带的MP时间轴对齐事件消息。
所述InbandEventStream元素包括：事件流方案标识schemeIdUri属性和事件流值value属性，设置为指示MP时间轴对齐事件。
所述MP时间轴对齐事件消息包括以下至少之一：呈现时间增量presentation_time_delta字段，设置为提供MP时间轴对齐事件的时间；事件持续时间event_duration字段，设置为表示媒体呈现剩余时间段；消息数据message_data字段，设置为提供所述对齐的媒体片段相对于所在时段Period时段开始时间的呈现时间Tp。
所述MP时间轴对齐事件消息还包括：事件消息方案标识scheme_id_uri字段以及事件值value字段；所述事件消息方案标识scheme_id_uri字段值等于所述事件流方案标识schemeIdUri属性值，并且所述事件值value字段值等于所述事件流值value属性值。
根据所述MP时间轴对齐事件消息确定对齐的媒体片段，包括：确定所述MP时间轴对齐事件消息中presentation_time_delta字段值为预定值的媒体片段作为所述对齐的媒体片段。
所述方法还包括：基于所述MP时间轴对齐事件消息中的message_data字段中呈现时间Tp以及所述对齐的媒体片段所在表示Representation的呈现时间偏移TO属性值，计算所述对齐的媒体片段的MPT，即TM＝Tp-TO。
计算所述媒体片段在媒体呈现时间上历经的时间，包括：确定所述MP时间轴对齐事件消息中具有相同标识id字段值的媒体片段为所述MP时间轴对齐事件中的媒体片段；根据所述MP时间轴对齐事件消息中呈现时间增量presentation_time_delta字段值，和时间标度timescale字段值，计算所述MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间。
根据所述MP时间轴对齐事件消息中呈现时间增量presentation_time_delta字段值，和时间标度timescale字段值，计算所述MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间，包括：根据以下公式计算所述MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间：MPT＝MPT0+emsg.presentation_time_delta/emsg.timescale，其中，MPT0与MPT分别代表所述MP时间轴对齐事件中的对齐媒体片段以及后续媒体片段的媒体呈现时间；emsg.presentation_time_delta与emsg.timescale分别代表所述MP时间轴对齐事件消息中的媒体呈现时间增量值及时间标度值。
在获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息之前，所述方法还包 括：获取携带有所述MP时间轴对齐事件消息的所述媒体片段。
获取携带有所述MP时间轴对齐事件消息的所述媒体片段包括：获取事件流方案标识字段以及事件流值字段；使用所述事件流方案标识字段值以及事件流值字段值，确定媒体呈现描述MPD中的带内事件流InbandEventStream元素；获取对应于所述带内事件流InbandEventStream元素的媒体片段作为所述媒体片段。
所述外部时间轴包括传输流时间轴。
根据本发明的另一实施例，提供了一种自适应流媒体处理装置，包括：第一获取模块，设置为获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息；确定模块，设置为根据所述MP时间轴对齐事件消息确定对齐的媒体片段，其中，所述对齐的媒体片段是所述MP时间轴中媒体呈现时间MPT与外部时间轴进行对齐的媒体片段；计算模块，设置为计算在所述MP时间轴与所述外部时间轴建立映射关系之后，所述MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间。
所述第一获取模块包括：第一访问单元，设置为访问媒体呈现描述MPD中带内事件流InbandEventStream元素；第一获取单元，设置为获取对应于所述带内事件流InbandEventStream元素的媒体片段中携带的媒体呈现MP时间轴对齐事件消息。
所述InbandEventStream元素包括：事件流方案标识schemeIdUri属性和事件流值value属性，设置为指示MP时间轴对齐事件。
所述MP事件轴对齐事件消息包括以下至少之一：呈现时间增量presentation_time_delta字段，设置为提供MP时间轴对齐事件的时间；事件持续时间event_duration字段，设置为表示媒体呈现剩余时间段；消息数据message_data字段，设置为提供所述对齐的媒体片段相对于所在时段Period时段开始时间的呈现时间Tp。
所述MP时间轴对齐事件消息还包括：事件消息方案标识scheme_id_uri字段以及事件值value字段；所述事件消息方案标识scheme_id_uri字段值等于所述事件流方案标识schemeIdUri属性值，并且所述事件值value字段值等于所述事件流值value属性值。
所述确定模块包括：第一确定单元，设置为确定所述MP时间轴对齐事件消息中presentation_time_delta字段值为预定值的媒体片段作为所述对齐的媒体片段。
所述装置还包括：基于所述MP时间轴对齐事件消息中的message_data字段中呈现时间Tp以及所述对齐的媒体片段所在表示Representation的呈现时间偏移TO属性值，计算所述对齐的媒体片段的MPT，即TM＝Tp-TO。
所述计算模块包括：第二确定单元，设置为确定所述MP时间轴对齐事件消息中 具有相同标识id字段值的媒体片段为所述MP时间轴对齐事件中的媒体片段；计算单元，设置为根据所述MP时间轴对齐事件消息中呈现时间增量presentation_time_delta字段值，和时间标度timescale字段值，计算所述MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间。
所述计算单元包括：根据以下公式计算所述MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间：MPT＝MPT0+emsg.presentation_time_delta/emsg.timescale，其中，MPT0与MPT分别代表所述MP时间轴对齐事件中的对齐媒体片段以及后续媒体片段的媒体呈现时间；emsg.presentation_time_delta与emsg.timescale分别代表所述MP时间轴对齐事件消息中的媒体呈现时间增量值及时间标度值。
所述装置还包括：第二获取模块，设置为获取携带有所述MP时间轴对齐事件消息的所述媒体片段。
所述第二获取模块包括：第二获取单元，设置为获取事件流方案标识字段以及事件流值字段；第三确定单元，设置为使用所述事件流方案标识字段值以及事件流值字段值，确定媒体呈现描述MPD中的带内事件流InbandEventStream元素；第三获取单元，设置为获取对应于所述带内事件流InbandEventStream元素的媒体片段作为所述媒体片段。
所述外部时间轴包括传输流时间轴。
根据本发明的另一实施例，还提供了又一种自适应流媒体处理方法，包括：
获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息；
根据所述MP时间轴对齐事件消息确定外部媒体资源的位置，以及所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间EMPT。
所述MP时间轴对齐事件消息至少包括：
消息数据message_data字段，设置为提供MP时间轴对齐事件中的所述外部媒体资源位置的URL。
所述MP时间轴对齐事件消息至少包括：
呈现时间增量presentation_time_delta字段，设置为提供MP时间轴对齐事件中的所述外部媒体资源在MP时间轴上的媒体呈现时间相对于所述媒体片段最早呈现时间的增量。
所述MP时间轴对齐事件消息还包括：
事件消息方案标识scheme_id_uri字段，设置为提供事件消息方案标识；
事件值value字段，设置为提供事件值。
根据所述MP时间轴对齐事件消息确定外部媒体资源的位置，包括：
将所述MP时间轴对齐事件消息中message_data字段值确定为所述外部媒体资源的位置。
根据所述MP时间轴对齐事件消息确定所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间，包括：
根据所述媒体片段的最早呈现时间EPT，以及所述MP时间轴对齐事件消息中presentation_time_delta字段值，确定所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间。
根据所述MP时间轴对齐事件消息确定所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间还包括：
确定所述MP时间轴对齐事件消息中presentation_time_delta字段取值为0的媒体片段为MP时间轴中的媒体呈现时间MPT与外部时间轴对齐的媒体片段；
所述外部媒体资源在MP时间轴上的媒体呈现时间等于所述对齐媒体片段的最早呈现时间。
根据所述MP时间轴对齐事件消息确定所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间，还包括：
针对MP时间轴对齐事件中相同的外部媒体资源，所述MP时间轴对齐事件消息具有相同消息标识id字段值，且满足：
Figure PCTCN2015096589-appb-000001
其中，
所述EMPTid为MP时间轴与外部时间轴建立映射关系之后，MP时间轴对齐事件中的外部媒体资源在MP时间轴上的外部媒体呈现时间；
所述
Figure PCTCN2015096589-appb-000002
为MP时间轴对齐事件中第i个媒体片段的最早呈现时间；
所述
Figure PCTCN2015096589-appb-000003
为第i个MP时间轴对齐事件消息中媒体呈现时间增量字段值；
所述
Figure PCTCN2015096589-appb-000004
为第i个MP时间轴对齐事件消息中时间标度字段值；
所述i为正整数。
获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息还包括：
访问媒体呈现描述MPD中的带内事件流InbandEventStream元素；
所述InbandEventStream元素的事件流方案标识schemeIdUri属性值等于所述MP时间轴对齐事件消息中scheme_id_uri字段值，并且所述InbandEventStream元素的事件流值value属性值等于所述MP时间轴对齐事件消息中value字段值；
获取对应于所述InbandEventStream元素的媒体片段中携带的MP时间轴对齐事件消息。
根据本发明的另一实施例，还提供了又一种自适应流媒体处理装置，包括：
第三获取模块，设置为获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息；
第三确定模块，设置为根据所述MP时间轴对齐事件消息确定外部媒体资源的位置，以及所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间EMPT。
所述MP时间轴对齐事件消息至少包括：
消息数据message_data字段，设置为提供MP时间轴对齐事件中的所述外部媒体资源位置的URL。
所述MP时间轴对齐事件消息至少包括：
呈现时间增量presentation_time_delta字段，设置为提供MP时间轴对齐事件中的所述外部媒体资源在MP时间轴上的媒体呈现时间相对于所述媒体片段最早呈现时间的增量。
所述MP时间轴对齐事件消息还包括：
事件消息方案标识scheme_id_uri字段，设置为提供事件消息方案标识；
事件值value字段，设置为提供事件值。
所述第三确定模块包括：
第四确定单元，设置为将所述MP时间轴对齐事件消息中message_data字段值确定为所述外部媒体资源的位置。
所述第三确定模块包括：
第五确定单元，设置为根据所述媒体片段的最早呈现时间EPT，以及所述MP时间轴对齐事件消息中presentation_time_delta字段值，确定所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间。
所述第三确定模块还包括：
第六确定单元，设置为确定所述MP时间轴对齐事件消息中presentation_time_delta字段取值为0的媒体片段为MP时间轴中的媒体呈现时间MPT与外部时间轴对齐的媒体片段；
其中所述外部媒体资源在MP时间轴上的媒体呈现时间等于所述对齐的媒体片段的最早呈现时间。
所述第三确定模块还包括：
第七确定单元，设置为确认针对MP时间轴对齐事件中相同的外部媒体资源，所述MP时间轴对齐事件消息具有相同消息标识id字段值，且满足：
Figure PCTCN2015096589-appb-000005
其中，
所述EMPTid为MP时间轴与外部时间轴建立映射关系之后，MP时间轴对齐事件中的外部媒体资源在MP时间轴上的外部媒体呈现时间；
所述
Figure PCTCN2015096589-appb-000006
为MP时间轴对齐事件中第i个媒体片段的最早呈现时间；
所述
Figure PCTCN2015096589-appb-000007
为第i个MP时间轴对齐事件消息中媒体呈现时间增量字段值；
所述
Figure PCTCN2015096589-appb-000008
为第i个MP时间轴对齐事件消息中时间标度字段值；
所述i为正整数。
所述第三获取模块还包括：第二访问单元，设置为访问媒体呈现描述MPD中的带内事件流InbandEventStream元素；其中，所述InbandEventStream元素的事件流方案标识schemeIdUri属性值等于所述MP时间轴对齐事件消息中scheme_id_uri字段值，并且所述InbandEventStream元素的事件流值value属性值等于所述MP时间轴对齐事件消息中value字段值；第四获取单元，设置为获取对应于所述InbandEventStream元素的媒体片段中携带的MP时间轴对齐事件消息。
通过本发明实施例方案，获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息；根据所述MP时间轴对齐事件消息确定对齐的媒体片段，其中，所述对齐的媒体片段是所述MP时间轴中媒体呈现时间MPT与外部时间轴进行对齐的媒体片段；计算在所述MP时间轴与所述外部时间轴建立映射关系之后，所述MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间，从而解决了现有技术中采用单一参考时钟所导致的问题，实现了广播网络和宽带网络混合递送模式下媒体同步播放。
附图说明
此处所说明的附图用来提供对本发明的进一步理解，构成本申请的一部分，本发明的示意性实施例及其说明用于解释本发明，并不构成对本发明的不当限定。在附图中：
图1是根据本发明实施例的第一种广播/宽带混合递送媒体同步系统架构示意图；
图2是根据本发明实施例的一种动态自适应流媒体处理方法的流程图；
图3是根据本发明可选实施例的一种获取自适应流媒体位置描述子的示意图；
图4是根据本发明可选实施例的另一种获取自适应流媒体位置描述子的示意图；
图5是根据本发明实施例的一种确定需要进行时间轴对齐的媒体片段的示例图；
图6是根据本发明实施例的另一种确定需要进行时间轴对齐媒体片段的示例图；
图7是根据本发明实施例的再一种确定需要进行时间轴对齐媒体片段的示例图；
图8是根据本发明实施例的一种传输流处理方法的流程图；
图9是根据本发明实施例的一种自适应流媒体处理装置的结构框图；
图10是根据本发明实施例的另一种自适应流媒体处理装置的结构框图；
图11是根据本发明实施例的另一种自适应流媒体处理方法的流程图；
图12是根据本发明实施例的第二种广播/宽带混合传输媒体同步系统架构示意图；
图13是根据本发明可选实施例的一种动态自适应流媒体处理方法的流程图；
图14是根据本发明实施例的一种确定需要进行时间轴对齐的媒体片段的示例图；
图15是根据本发明实施例的再一种自适应流媒体处理装置的流程图；
图16是根据本发明实施例的再一种自适应流媒体处理装置中第一获取模块152的结构框图；
图17是根据本发明实施例的再一种自适应流媒体处理装置中确定模块154的结构框图；
图18是根据本发明实施例的再一种自适应流媒体处理装置中计算模块156的结构框图；
图19是根据本发明实施例的再一种自适应流媒体处理装置的优选结构框图；
图20是根据本发明实施例的再一种自适应流媒体处理装置中第二获取模块192的结构框图；
图21是根据本发明实施例的另一种自适应流媒体处理方法的流程图；
图22是根据本发明实施例的一种确定需要进行时间轴对齐的媒体片段的示例图；
图23是根据本发明实施例的另一种自适应流媒体处理装置的结构框图；
图24是根据本发明实施例的另一种自适应流媒体处理装置234的第三确定模块的结构框图；
图25是根据本发明实施例的另一种自适应流媒体处理装置232的第三获取模块的结构框图。
具体实施方式
下文中将参考附图并结合实施例来详细说明本发明。需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。
本发明实施例提供了一种广播/宽带混合递送(也称传输)媒体同步系统，图1是根据本发明实施例的第一种广播/宽带混合传输媒体同步系统架构示意图，如图1所示，包括：广播内容源100、MPD服务器200、流媒体服务器300、第一终端410、第二终端420，其中第一终端和第二终端可以是独立的两个终端，这两个终端的功能也可以放在一个物理终端中实现。下面对图1中示出的系统进行说明。
广播内容源100可以使用MPEG-2标准，或者也可以使用其他标准，以下以MPEG-2为例进行说明。根据MPEG-2标准编码模型，广播内容源100在发送端，将音频数据和视频数据输入到各自的编码器进行压缩和编码，得到音视频的基本码流(Elemental Stream，简称为ES)，ES被送到打包器分割编组形成长度不等的分组码流(Packet Elemental Stream，简称为PES),PES再与节目内容信息、条件接收信息等一起进入码流复用器，生成节目流(Program Stream，简称为PS)或者传输流(Transport Stream，简称为TS)。
广播内容源100发送的MPEG-2TS码流载荷中携带具有不同的包标识符(Packet Identification，简称为PID)的PES分组码流。其中，PID值为0x0000的TS码流对应节目关联表(Program Association Table，简称为PAT)，PAT定义了TS中所有的节目，提供了节目初始化信息，是TS中所有节目的一个完整列表，并指示了不同节目的节目号以及与节目内容相关的节目映射表(Program Map Table，简称为PMT)的PID。PMT表明了节目号与其所包含的音视频及其他内容的传输码流之间的对应关系，对一 套节目进行了完整的定义，其中包含了节目的节目号(program number)，PCR PID，流类型(stream type，如音频、视频和其他数据等)以及基本流PID(指出携带音视频内容的TS包PID)等信息。此外，MPEG-2TS码流还可以携带称作时间轴和外部媒体信息(Timeline and External Media Information，简称为TEMI PES)的TEMI PES码流，设置为指示MPEG-2TS码流中携带的外部媒体资源的时间信息和位置信息。
在以下实施例中，涉及到的自适应流媒体(或者称为动态自适应流媒体)有多种实现方式，例如，可以采用基于HTTP的动态自适应流媒体(Dynamic Adaptive Streaming over HTTP，简称为DASH)。下面以DASH为例对MPD服务器和流媒体服务器进行说明。以下实施例对其他自适应流媒体技术也同样适用。
MPD服务器200，提供动态自适应流媒体中MPD文件存储。MPD文件描述了每个媒体片段的持续时间，URL，媒体属性，如视频分辨率、适配比特率等。MPD文件给DASH客户端提供了足够的HTTP-URL信息设置为访问流媒体内容资源，当DASH客户端开始播放一个视频的时候，它根据MPD文件中的信息选择下载适应于当前可用网络带宽和终端缓存状况的媒体片段并进行播放。
流媒体服务器300，完成对原始视频的切片和MPD文件的生成。视频文件被切分，并以时段(period)、表示(representation)和媒体片段(segment)的层次存储。根据请求，流媒体服务器300将向终端返回适当的媒体分片。
终端400可以包括：第一终端410，例如数字电视机顶盒，或者广播\宽带双模机顶盒；以及第二终端420，例如平板电脑，智能手机、手持电视设备等。
终端400(第一终端410和/或第二终端420，为了描述下文中均使用终端)在接收到MPEG-2TS码流后，按照不同的包标识符PID将码流分流到不同的解析通道，解析出音频PES、视频PES，并重组成ES流存到音视频各自对应的缓存中。首先，指定解复用模块的一个通道绑定PID值为0x0000的TS流，并从中解析出PAT，获得所需节目的PMT的包标识符PID。通过通道绑定，找到节目映射表PMT，提取码流中的系统参考时钟PCR,以及相关节目中音视频PES码流的PTS等时间信息。
下述实施例及可选实施方式在申请号为：201410856623.9中的专利申请中已经进行了保护，现摘录如下。需要说明的是，这些实施例即可选实施中的部分内容也可以应用本发明实施例中。
下面结合图1中的架构对本实施例提供的动态自适应流媒体处理方法进行说明：
在一个可选的实施方式中，终端400在接收到MPEG-2TS码流后，还可以将提取TEMI PES码流中携带的特定类型的适配域描述子(AF Descriptor)以及与之对应的PES码流头域显示时间戳(Presence Time Stamp，简称为PTS)，设置为获取动态自适 应流媒体的位置信息和时间信息。
在一个可选的实施方式中，终端可以根据位置信息请求MPD服务器200获取动态自适应流媒体的MPD文件。终端400解析获取到的MPD文件确定动态自适应流媒体中需要与MPEG-2TS码流进行时间轴对齐的媒体片段。
在一个可选的实施方式中，终端400可以根据时间信息确定动态自适应流媒体中媒体片段(包括：时间轴对齐媒体片段以及同一时段中后续媒体片段)的媒体呈现时间(Media Presentation Time，简称为MPT)相对于MPEG-2TS码流时间轴的显示时间戳PTS。
在一个可选的实施方式中，终端可以通过动态自适应(Quality Adaptation)算法周期性的发出HTTP请求，获得动态自适应流媒体中适当码率或质量版本的媒体片段，并根据MPEG-2TS码流相关节目中的音视频PES码流PTS信息，以及动态自适应流媒体中媒体片段PTS信息，实现音视频同步处理。
图2是根据本发明实施例的一种动态自适应流媒体处理方法的流程图，如图2所示，该流程包括如下步骤：
步骤S202，获取传输流中携带的自适应流媒体的位置信息和时间信息；
步骤S204，根据上述位置信息确定自适应流媒体中需要与传输流进行时间轴对齐的媒体片段；
步骤S206，根据上述时间信息确定自适应流媒体中媒体片段的媒体呈现时间相对于传输流时间轴的显示时间戳。
通过上述步骤，利用传输流中携带的自适应流媒体的位置信息和时间信息来确定该自适应流媒体中需要与传输流进行时间轴对齐的媒体片段及其对应的显示时间戳，从而解决了相关技术中为广播网络和宽带网络混合媒体传输采用单一参考时钟所导致的问题，使得混合递送模式下媒体同步处理更加高效，同时也提高业务部署的可扩展性。
步骤S202中记载了获取位置信息和时间信息，下面结合可选实施来对位置信息和时间信息的获取来进行说明。
步骤S202中的位置信息可以是位置描述子(Location Descriptor)，图3是根据本发明可选实施例的一种获取自适应流媒体位置描述子的示意图，如图3所示，包括：位置描述子也可以是AF描述子，其可以携带在PES数据载荷中，PTS时间值可以携带在PES扩展头域中。
在一个可选的实施方式中，终端首先解析获得传输流中的节目映射表PMT，从中确定流类型(Stream Type)值为预定值(例如，0x26)的TEMI PES码流的包标识符PID，终端将包标识符PID绑定到解复用模块通道，过滤出含有TEMI PES码流的TS包，其中，TEMI PES码流的流标识(streamID)值为private_stream_1。
终端提取TEMI PES码流数据载荷TEMI_AU中携带的标签字段(af_descr_tag)值为预定值(例如，0x05)且业务类型字段(service_type)值为预定值(例如，0x01)的适配域AF描述子，即动态自适应流媒体的位置描述子(Location Descriptor)，从而获取动态自适应流媒体的位置信息。
参照图3，终端获取传输流中TEMI PES码流，提取TEMI PES码流扩展头域中包含的PTS，作为与动态自适应流媒体位置描述子对应的PTS值，从而获取动态自适应流媒体的时间信息。
位置信息还可以通过另外一种方式携带，图4是根据本发明可选实施例的另一种获取自适应流媒体位置描述子的示意图，如图4所示，AF描述子可以携带在TS流的适配域中。
根据MPEG-2标准PES语法说明，流标识(streamID)表明了PES载荷中是音频、视频还是数据流。参照本发明中图4，终端获取传输流中扩展描述子标签(Extension_descriptor_tag)值为预定值(例如，0x04)的特定媒体组件(视频或者音频)TS码流。
终端提取特定媒体组件TS码流适配域(Adaptation Field)中携带的标签字段(af_descr_tag)值为预定值(0x05)且业务类型字段(service_type)值为预定值(例如，0x01)的AF描述子，即动态自适应流媒体的位置描述子(Location Descriptor)，从而获取动态自适应流媒体的位置信息。
参照图4，终端获取传输流中特定媒体组件(视频或者音频)TS码流，提取特定媒体组件(视频或者音频)TS码流载荷中携带的PES码流扩展头域中包含的PTS，作为与动态自适应流媒体位置描述子对应的PTS值，从而获取动态自适应流媒体的时间信息。
或者，终端获取后续第一个与特定媒体组件(视频或者音频)TS码流具有相同PID值且TS码流头域中载荷数据单元起始指示符(payload_unit_start_indicator)取值为1的TS码流，提取TS码流载荷中携带的PES码流扩展头域中包含的PTS，作为与动态自适应流媒体位置描述子对应的PTS值，从而获取动态自适应流媒体的时间信息。
在上述实施例及可选的实施方式中，对现有的位置描述子进行扩展，表1所示的是根据本发明实施例的自适应流媒体位置描述子的语法结构图表。
表1
Figure PCTCN2015096589-appb-000009
如表1所示，本实施例中自适应流媒体的位置描述子(Location Descriptor)，扩展 定义如下，以业务类型为DASH为例进行说明：
timeline_id:MPEG-2TS码流中位置描述子的唯一标识；
force_reload:1比特标志，指示DASH同步(即进行媒体片段时间轴对齐)前是否重新加载MPD；
is_announcement:1比特标志，指示与位置描述子对应的DASH媒体片段尚未处于播放(激活)状态；
time_before_activation:指示位置描述子所引用资源(DASH媒体片段)进入播放状态的剩余时间；
timescale:时标单位，设置为表示time_before_activation字段时间值；
service_type:指示位置描述子所引用外部资源的业务类型。当service_type＝＝1时，业务类型为“MPEG-DASH”。此时，该位置描述子中所携带的位置信息将设置为实现DASH媒体片段的媒体呈现时间与MPEG-2TS码流中PTS之间的映射，即完成DASH流媒体与MPEG-2TS码流之间同步。
如表1所示，本发明实施例将上述建立DASH流媒体时间信息与传输流时间信息映射关系的位置信息称为时间轴对齐(Timeline Alignment)事件流(Event Stream)，定义如下：
event_stream_schemeIdUri：当service_type＝＝1时，位置描述子包含事件流方案标识event_stream_schemeIdUri字段，遵循统一资源名(URN)或者统一资源定位符(URL)语法规范，设置为唯一标识时间轴对齐事件流的方案(Scheme)；
event_stream_value：当service_type＝＝1时，位置描述子包含事件流值event_stream_value字段，其取值空间将由event_schemeIdUri标识的时间轴对齐事件流的消息方案定义，设置为标识事件流方案中的事件流；
event_id：当service_type＝＝1时，位置描述子包含事件标识event_id字段，设置为标识时间轴对齐事件流中的事件；
其中，事件消息方案标识字段与事件流值字段一起可以唯一标识时间轴对齐事件流。事件标识字段则设置为唯一标识时间轴对齐事件流中的某个特定时间轴对齐事件。
可选的，mpd_url：当service_type＝＝1时，mpd_url取值为位置描述子所对应的DASH的MPD的统一资源定位符(URL)。
在一个可选的实施例中，终端获取节目映射表PMT中包含的自适应流媒体的 MPD描述子，可以定义如表2所示，以DASH为例进行说明：
表2
Figure PCTCN2015096589-appb-000010
其中，mpd_url取值为对应于自适应流媒体位置描述子的DASH的MPD的URL。
上述步骤S204中记载了位置信息确定自适应流媒体中需要与传输流进行时间轴对齐的媒体片段；下面结合可选实施来对此来进行说明。
在动态自适应流媒体技术中，MPD是一个文档，该文档中包含了DASH客户端构建媒体片段HTTP-URL所需的元数据，以便客户端访问媒体片段，为用户提供流媒体服务。
MPD中可以包含事件(Event)，设置为向DASH客户端或DASH应用通知非周期性的信息。事件是时控的，亦即，事件在某一特定的媒体呈现时间点开始，通常会持续一段时间。事件包括DASH特有的通知事件和应用特有的事件。
与上述位置描述子中所包含的时间轴对齐事件相对应，MPD中时段(Period)元素可以包含时间轴对齐事件流(EventStream)子元素，其中：
EventStream@schemeIdUri属性设置为唯一标识时间轴对齐事件流的消息方案(Scheme)。EventStream@value属性作为时间轴对齐事件流的值，其取值空间将由event_schemeIdUri标识的时间轴对齐事件流的消息方案定义。EventStream元素中包含一系列相同类型的时间轴对齐事件Event元素，由Event.id属性值唯一标识。
在本实施例中，终端可以按照不同方法根据位置信息获取对应的MPD。
终端提取PMT中包含的动态自适应流媒体的MPD描述子，根据其中包含的mpd_url字段值获取与位置信息对应的动态自适应流媒体的MPD的URL。或者，
终端提取动态自适应流媒体位置描述子，根据其中包含的mpd_url字段值获取与位置信息对应的动态自适应流媒体的MPD的URL。
在根据动态自适应流媒体的位置信息以及对应的MPD确定需要与传输流进行时间轴对齐的动态自适应流媒体的媒体片段时，可以有多种方式，下面结合不同的方式 进行说明：
图5是根据本发明实施例的一种确定需要进行时间轴对齐的媒体片段的示例图，如图5所示，包括：
终端根据位置描述子中携带的值为"urn:uuid:XYZY"的事件流方案标识字段以及值为"ad"的事件流值字段，检索确定MPD中的Period元素，即图6中@id属性值为"1"的Period元素。
终端使用位置描述子中携带的事件流方案标识字段值"urn:uuid:XYZY"和事件流值字段值"ad"替换该Period元素的媒体片段模板(SegmentTemplate)子元素中@media属性的$EventStream$标识符，使用位置描述子中携带的事件标识(event_id)字段值"1"替换该Period元素的媒体片段模板(SegmentTemplate)子元素中@media属性的$EventID$标识符。同时结合MPD其他层次中定义的资源URL地址基址(BaseURL)元素，譬如："http://www.example.com/"，从而构造需要与传输流进行时间轴对齐的动态自适应流媒体的媒体片段的HTTP URL，即"http://www.example.com/uuid:XYZY/ad_00001.ts"。
对应传输流中携带的下一个相同类型位置描述子中的事件标识(event_id)字段值"2"，则构造的下一个需要与传输流进行时间轴对齐的动态自适应流媒体的媒体片段HTTP URL为"http://www.example.com/uuid:XYZY/ad_00002.ts"，以此类推。
图6是根据本发明实施例的另一种确定需要进行时间轴对齐媒体片段的示例图，如图6所示，包括：
终端根据位置描述子中携带的值为"urn:uuid:XYZY"的事件流方案标识字段以及值为"ad"的事件流值字段，检索确定MPD中的Period元素，即图7中@id属性值为"1"的Period元素。
终端根据位置描述子中携带的值"1"的事件标识字段，检索确定MPD中该Period元素下的媒体片段URL(SegmentURL)子元素，即图7中@eventID属性值为"1"的媒体片段URL(SegmentURL)元素。
终端根据该媒体片段URL(SegmentURL)元素的@media及@mediaRange属性值，同时结合MPD其他层次中定义的资源URL地址基址(BaseURL)元素，譬如："http://www.example.com/"，从而构造需要与传输流进行时间轴对齐的动态自适应流媒体的媒体片段的HTTP URL，即"http://www.example.com/seg201.mp4"，以及可选的区间请求头域"Range:bytes＝0-499"。
对应传输流中携带的下一个相同类型位置描述子中的事件标识(event_id)字段值 "2"，则构造的下一个需要与传输流进行时间轴对齐的动态自适应流媒体的媒体片段HTTP URL为"http://www.example.com/seg205.mp4"，以及可选的区间请求头域"Range:bytes＝500-999"，以此类推。
图7是根据本发明实施例的再一种确定需要进行时间轴对齐媒体片段的示例图，如图7所示，包括：
终端根据位置描述子中携带的值为"urn:uuid:XYZY"的事件流方案标识字段以及值为"ad"的事件流值字段，检索确定MPD中的Period元素，即图7中@id属性值为"1"的Period元素。
终端根据位置描述子中携带的值"1"的事件标识字段，检索确定MPD中该Period元素下的事件(Event)子元素，即图7中@id属性值为"1"的事件(Event)元素。
终端根据该事件(Event)元素的@media及@mediaRange属性值，同时结合MPD其他层次中定义的资源URL地址基址(BaseURL)元素，譬如："http://www.example.com/"，从而构造需要与传输流进行时间轴对齐的动态自适应流媒体的媒体片段的HTTP URL，即"http://www.example.com/seg301.mp4"，以及可选的区间请求头域"Range:bytes＝0-499"。
对应传输流中携带的下一个相同类型位置描述子中的事件标识(event_id)字段值"2"，则构造的下一个需要与传输流进行时间轴对齐的动态自适应流媒体的媒体片段HTTP URL为"http://www.example.com/seg305.mp4"，以及可选的区间请求头域"Range:bytes＝500-999"，以此类推。
在一个可选的实施例中，在确定需要与传输流进行时间轴对齐的媒体片段时，还包括：
在一个可选的实施例中，在确定需要与传输流进行时间轴对齐的媒体片段时，还包括：
终端提取动态自适应流媒体位置描述子中的强制重载(force_reload)标志；如果强制重载(force_reload)标志的值为0x01，则重新加载动态自适应流媒体的MPD。
在一个可选的实施例中，在确定需要与传输流进行时间轴对齐的媒体片段时，还包括：
终端提取动态自适应流媒体位置描述子中的预宣告(is_announcement)标志，如果预宣告(is_announcement)标志的值为0x01，则提取动态自适应流媒体位置描述子中的激活倒计时(time_before_activation)字段以及时标(timescale)字段。
在一个可选的实施例中，在确定需要与传输流进行时间轴对齐的媒体片段时，还包括：从位置信息中提取强制重载标志；在强制重载标志的值为预定值的情况下，重新加载上述DASH的MPD。
在另一个可选的实施例中，在确定需要与传输流进行时间轴对齐的媒体片段时，还包括：从位置信息中提取预宣告标志；在预宣告标志的值为预定值的情况下，从位置信息中提取激活倒计时字段和时标字段。
步骤S206中记载了根据时间信息确定自适应流媒体中媒体片段的媒体呈现时间相对于传输流时间轴的PTS；下面结合可选实施例来对此来进行说明。
DASH的主要特性之一是不同媒体组件的编码版本共享全局统一的时间轴。媒体内容中的接入单元的呈现时间被映射到全局统一呈现时间轴，便于同步不同的媒体组件并实现相同媒体组件的不同的编码版本的相同的无缝切换。
各个时段内的呈现时间都是相对相应时段的PeriodStart时间减去所包含表示的@presentationTimeOffset即T0的值。这意味着，对于媒体流中呈现时间为TP的接入单元，其媒体呈现时间相对PeriodStart的值为TM＝TP–TO。
根据MPD定义的动态自适应流媒体的媒体片段时间轴规则，时段中各个媒体片段的媒体呈现时间存在预定的关系，譬如可以是连续的，因此，根据一个媒体片段的媒体呈现时间可以推算出时段中后续媒体片段的媒体呈现时间。
在本发明一个可选的实施例中，终端根据需要与传输流进行时间轴对齐的媒体片段的媒体呈现时间确定该媒体片段所在时段中后续媒体片段的媒体呈现时间。特别地，当需要进行时间轴对齐的媒体片段的媒体呈现时间表示为相对于传输流时间轴的PTS时，存在以下关系，
Segments PTS(in seconds)＝[(S-Ssyn)*@duration]/SegmentBase.timescale+Segmentsyn PTS(in seconds)。
其中，Ssyn为需要进行时间轴对齐的动态自适应流媒体的媒体片段在时段中的编号，S为时段中后续其他媒体片段的编号。
Segmentsyn PTS为需要进行时间轴对齐的动态自适应流媒体的媒体片段的媒体呈现时间相对于传输流时间轴的PTS。Segment PTS为时段中后续其他媒体片段的媒体呈现时间相对于传输流时间轴的PTS。
其中，@duration和SegmentBase.timescale分别为媒体片段所在时段元素的@duration属性值和@timescale属性值；
在本发明一个可选的实施例中，终端根据与动态自适应流媒体位置描述子对应的PTS值确定需要进行时间轴对齐的动态自适应流媒体的媒体片段的媒体呈现时间相对于传输流时间轴的PTS，即：
Segmentsyn PTS(in seconds)＝PTSi/90000。
其中，PTSi为与动态自适应流媒体位置描述子对应的PTS值。
Segmentsyn PTS为需要进行时间轴对齐的动态自适应流媒体的媒体片段的媒体呈现时间相对于传输流时间轴的PTS。
图8是根据本发明实施例的一种传输流处理方法的流程图，如图8所示，该流程包括如下步骤：
步骤S802，在传输流中添加自适应流媒体的位置信息和/或时间信息，其中，该位置信息设置为确定自适应流媒体中需要与传输流进行时间轴对齐的媒体片段，该时间信息设置为确定自适应流媒体中媒体片段的媒体呈现时间相对于传输流时间轴的显示时间戳；
步骤S804，将上述传输流发送给用户终端。
通过上述步骤，通过将添加了预定类型码流的传输流发送给用户终端，可以有效解决相关技术中为广播网络和宽带网络混合媒体传输采用单一参考时钟所导致的问题，使得混合递送模式下媒体同步处理更加高效，同时也提高业务部署的可扩展性。
在一个可选的实施例中，在传输流中添加携带自适应流媒体的位置信息时，可以采用如下方法：在传输流的PMT中添加PID，其中，该PID设置为获取流类型值为预定值的TEMI PES码流，该TEMI PES码流中携带有位置信息。
在添加了自适应流媒体的位置信息后，还可以添加该自适应流媒体的时间信息，在一个可选的实施例中，在传输流中添加携带自适应流媒体的时间信息包括：在该TEMI PES码流扩展头域中携带时间信息。
在一个可选的实施例中，在传输流中添加携带自适应流媒体的位置信息时，还可以采用如下方法：在传输流中携带扩展描述子标签的值为预定值的特定媒体组件TS码流，其中，该TS码流适配域中携带位置信息。
在添加了自适应流媒体的位置信息后，还可以添加该自适应流媒体的时间信息，在一个可选的实施例中，在传输流中添加携带自适应流媒体的时间信息包括：在TS码流载荷的PES扩展头域中携带PTS，其中该PTS值设置为确定自适应流媒体的时间信息。
在传输流中添加携带自适应流媒体的时间信息还可以采用如下方法：设置传输流中的特定媒体组件TS后续的TS码流与该特定媒体组件TS码流具有相同PID值且TS头域中载荷数据单元起始指示符取值为预定值；在后续的TS码流载荷的PES扩展头域中携带PTS，其中，该PTS值确定自适应流媒体的时间信息。
在一个可选的实施例中，在传输流中携带标签字段值为预定值且业务类型字段值为预定值的AF描述子作为自适应流媒体的位置描述子，该位置描述子中包括位置信息。
在一个可选的实施例中，上述的位置描述子还包括以下至少之一：在传输流中位置描述子的唯一标识、强制重载标志、预宣告标志、激活倒计时标志、时标单位、服务类型，其中，该强制重载标志设置为指示在进行媒体片段时间轴对齐之前是否需要重新加载媒体呈现描述MPD，预宣告标志设置为指示与位置描述子对应的自适应媒体的媒体片段尚未处于播放装置，激活倒计时标志设置为指示与位置描述子对应的自适应媒体的媒体片段进入播放状态的剩余时间，时标单位设置为指示剩余时间标识字段时间值，服务类型设置为指示位置描述子所引用外部资源的业务类型。
在一个可选的实施例中，上述传输流中还可以携带有设置为确定位置信息对应的自适应流媒体的媒体呈现描述MPD的信息。
在一个可选的实施例中，在上述传输流中的PMT中携带MPD描述子；其中，该MPD描述子中包括与位置信息对应的MPD的URL；或者，从位置信息中获取该MPD的URL。
在本实施例中还提供了一种自适应流媒体处理装置，该装置设置为实现上述实施例及优选实施方式，已经进行过说明的不再赘述。如以下所使用的，术语“模块”可以实现预定功能的软件和/或硬件的组合。尽管以下实施例所描述的装置较佳地以软件来实现，但是硬件，或者软件和硬件的组合的实现也是可能并被构想的。
图9是根据本发明实施例的一种自适应流媒体处理装置的结构框图，如图9所示，该装置包括获取模块92、第一确定模块94和第二确定模块96，下面对该自适应媒体流处理装置进行说明。
获取模块92，设置为获取传输流中携带的自适应流媒体的位置信息和时间信息；第一确定模块94，连接至上述获取模块92，设置为根据上述位置信息确定自适应流媒体中需要与传输流进行时间轴对齐的媒体片段；第二确定模块96，连接至上述第一确定模块94，设置为根据时间信息确定自适应流媒体中媒体片段的媒体呈现时间相对于传输流时间轴的显示时间戳。
图10是根据本发明实施例的另一种自适应流媒体处理装置的结构框图，如图10 所示，该装置包括添加模块102和发送模块104，下面对该自适应媒体流处理装置进行说明。
添加模块102，设置为在传输流中添加自适应流媒体的位置信息和/或时间信息，其中，该位置信息设置为确定自适应流媒体中需要与传输流进行时间轴对齐的媒体片段，该时间信息设置为确定自适应流媒体中媒体片段的媒体呈现时间相对于传输流时间轴的；发送模块104，连接至上述添加模块102，设置为将传输流发送给用户终端。在上述实施例及可选实施方式在申请号为：201410856623.9中的专利申请中已经进行了保护，摘录如上。
事件可通过MPD通知。相同类型的事件归结为一个事件流(EventStream)，并在Period元素中的EventStream元素描述。事件在相应的Period结束时终结。
在一个可选实施例中，也可以通过带内事件通知。例如，可以将事件信息作为媒体片段的一部分，添加到Segment中，从而将事件流通过Representation传输。事件流可以存在于某一个或多个自适应集(AdaptationSet)关联的Representation中，也可以存在于所有的Representation中。
运载带内事件的Representation应当在MPD中标识，以通知客户端。MPD中使用带内事件流(InbandEventStream)元素指定一个带内事件流，该元素可以出现在AdaptationSet和Representation级别中。如果一个Representation包含多个带内事件流，每个带内事件流应当由一个单独的InbandEventStream元素指定。
图11是根据本发明实施例的另一种自适应流媒体处理方法的流程图，如图11所示，该方法包括如下步骤：
步骤S1102，获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息；
步骤S1104，根据MP时间轴对齐事件消息确定对齐的媒体片段，其中，该对齐的媒体片段是该MP时间轴中媒体呈现时间MPT与外部时间轴对齐的媒体片段；
步骤S1106，计算在MP时间轴与外部时间轴建立映射关系之后，该MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间。
通过上述步骤，根据媒体片段中携带的MP时间轴对齐事件消息来确定媒体呈现时间MPT与外部时间轴对齐的媒体片段，并计算出在MP时间轴与传输流时间轴建立映射关系之后，该MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间，可以有效的解决相关技术中为广播网络和宽带网络混合媒体传输采用单一参考时钟所导致的问题，使得混合递送模式下媒体同步处理更加高效，同时也提高业务部署的可扩展性。
上述步骤中涉及到外部时间轴、MP时间轴以及时间轴的对齐，以下对此进行说明。
外部时间轴
在本实施例中以MPEG-2TS PCR/PTS(节目参考时钟/显示时间戳)时钟信息为例进行说明，其他格式的音视频编码方式也可以作为外部时间轴。
根据MPEG-2标准编码模型，广播内容源发送的MPEG-2TS码流载荷中携带具有不同的包标识符(Packet Identification，简称为PID)的PES分组码流。终端在接收到MPEG-2TS码流后，按照不同的包标识符PID将码流分流到不同的解析通道，解析出音频PES、视频PES，并重组成ES流存到音视频各自对应的缓存中。首先，指定解复用模块的一个通道绑定PID值为0x0000的TS流，并从中解析出PAT，获得所需节目的PMT的包标识符PID。通过通道绑定，找到节目映射表PMT，提取码流中的系统参考时钟PCR,以及相关节目中音视频PES码流的PTS等时间信息。
此外，MPEG-2TS码流中可以携带称作时间轴和外部媒体信息(Timeline and External Media Information，简称为TEMI PES)的TEMI PES码流，设置为指示与该MPEG-2TS码流同步的外部媒体资源。相对应地，终端在接收到MPEG-2TS码流后，通过提取TEMI PES码流中携带的特定类型适配域描述子(AF Descriptor)，可以获取外部媒体资源的位置信息，并建立其与MPEG-2TS时间轴中时间信息(TEMI PES码流头域PTS值)的对应关系。
媒体呈现(MP)时间轴
在本实施例中以DASH为例进行说明，DASH的主要特性之一是不同媒体组件的编码版本共享全局统一的时间轴。媒体内容中的接入单元的呈现时间被映射到全局统一呈现时间轴，便于同步不同的媒体组件并实现相同媒体组件的不同的编码版本的相同的无缝切换。
各个时段内的呈现时间都是相对相应时段的PeriodStart时间减去所包含表示的@presentationTimeOffset即TO的值。这意味着，对于媒体流中呈现时间为TP的接入单元，其媒体呈现时间相对PeriodStart的值为TM＝TP–TO。
时间轴对齐
解决时间轴对齐的一个显见方法是为自适应流与外部媒体流递送采用单一的参考时钟。例如，上述MPEG-2TS码流PCR/PTS(节目参考时钟/显示时间戳)时钟信息同时也包含在诸如RTP、HTTP等IP报文中传输。该方法主要缺陷在于广播媒体和宽带媒体源绑定部署，不利于系统扩展，而且PCR时钟信息在解复用时可能被改变，无 法维护时钟的延续性。
为此，本实施例提出了一种独立于特定媒体流时钟信息的“时间轴对齐事件流”方案，建立自适应流的媒体呈现(MP)时间轴与外部时间轴的映射关系，图12是根据本发明实施例的第二种广播/宽带混合传输媒体同步系统架构示意图，如图12所示：其中，关于广播内容源100、MPD服务器200、流媒体服务器300、第一终端410、第二终端420的描述，在图1的相关描述中已经进行了详细的阐述，在此不再赘述。
本实施例定义“媒体呈现时间轴(Media Presentation timeline)对齐”事件，简称“MP时间轴对齐”事件，设置为通知客户端媒体片段的媒体呈现时间(MPT)将被映射到外部时间轴，例如：MPEG-2TS PCR/PTS(节目参考时钟/显示时间戳)时钟信息，或者相反。在图12中，“时间轴对齐事件流”中携带一系列代表时间轴对齐事件时间进度(或位置)Tj，通过在媒体流中添加携带Tj信令，并建立各自时间轴中时间信息与Tj的映射关系，即图12中(PTSi，Tj)和(MPTk，Tj)，可以建立媒体呈现(MP)时间轴与外部时间轴的映射关系。
媒体呈现(MP)时间轴与外部时间轴对齐关键要素，如表3所示：
表3
Figure PCTCN2015096589-appb-000011
据此，需针对自适应流媒体定义一种媒体呈现(MP)时间轴对齐信令机制，支持携带时间轴对齐进度(位置)信息以及与之对应媒体呈现时间(MPT)。
在上述的实施例中，MP时间轴对齐事件的信息可以是MP时间轴对齐事件消息匣，在一个可选的实施例中，上述步骤S1102中的获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息可以包括如下步骤：访问媒体呈现描述MPD中带内事件流InbandEventStream元素；获取对应于带内事件流InbandEventStream元素的媒体片段中携带的媒体呈现MP时间轴对齐事件消息。
上述的InbandEventStream元素可以包括：事件流方案标识schemeIdUri属性字段和事件流值value属性，设置为指示MP时间轴对齐事件。
在一个可选的实施例中，上述MP时间轴对齐事件消息包括以下至少之一：呈现时间增量presentation_time_delta字段，设置为提供MP时间轴对齐事件的时间；事件持续时间event_duration字段，设置为表示媒体呈现剩余时间段；消息数据message_data字段，设置为提供对齐的媒体片段相对于所在时段Period时段开始时间的呈现时间Tp。
上述的MP时间轴对齐时间消息还包括：事件消息方案标识scheme_id_uri字段以及事件值value字段；事件消息方案标识scheme_id_uri字段值等于事件流方案标识schemeIdUri属性值，并且该事件值value字段值等于事件流值value属性值。
在根据MP时间轴对齐事件消息确定对齐的媒体片段时，可以有多种确定方式，在一个可选的实施例中，确定媒体片段可以包括：确定MP时间轴对齐事件消息中presentation_time_delta字段值为预定值的媒体片段作为对齐的媒体片段。
在一个可选的实施例中，上述方法还可以包括：基于MP时间轴对齐事件消息中的message_data字段中呈现时间Tp以及对齐的媒体片段所在表示Representation的呈现时间偏移TO属性值，计算该对齐的媒体片段的MPT，即TM＝Tp-TO。
在计算上述媒体片段在媒体呈现时间上历经的时间时可以采用如下方式：确定MP时间轴对齐事件消息中具有相同标识id字段值的媒体片段为MP时间轴对齐事件中的媒体片段；根据MP时间轴对齐事件消息中呈现时间增量presentation_time_delta字段值，和时间标度timescale字段值，计算MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间。
其中，在根据MP时间轴对齐事件消息中呈现时间增量presentation_time_delta字段值，和时间标度timescale字段值，计算MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间时，可以采用如下方式：根据以下公式计算MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间：MPT＝MPT0+emsg.presentation_time_delta/emsg.timescale，其中，MPT0与MPT分别代表MP时间轴对齐事件中的对齐媒体片段以及后续媒体片段的媒体呈现时间；emsg.presentation_time_delta与emsg.timescale分别代表MP时间轴对齐事件消息中的媒体呈现时间增量值及时间标度值。
在一个可选的实施例中，在获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息之前，该方法还包括：获取携带有MP时间轴对齐事件消息的媒体片段。
其中，获取携带有MP时间轴对齐事件消息的媒体片段可以采用如下方式：获取事件流方案标识字段以及事件流值字段；使用上述事件流方案标识字段值以及事件流值字段值，确定媒体呈现描述MPD中的带内事件流InbandEventStream元素；获取对应于该带内事件流InbandEventStream元素的媒体片段作为媒体片段。
在一个可选的实施例中，上述的外部时间轴包括传输流时间轴。
在图11中的步骤S1102中获取到的是MP时间轴对齐事件消息，在一个可选的实施方式中，MP时间轴对齐事件消息可以使用MP时间轴对齐事件消息匣来实现，当然也可以使用其他的方式实现。图13中对使用MP时间轴对齐时间消息匣的可选实施例进行了说明。
图13是根据本发明可选实施例的一种动态自适应流媒体处理方法的流程图，如图13所示，该流程包括如下步骤：
步骤S1302，客户端获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息匣。
MPD中指定“MP时间轴对齐”带内事件流的InbandEventStream元素包含一个@schemeIdUri属性，设置为提供URI以指定事件流方案，例如：“urn:avs:dash:event:2014”；和一个可选的@value属性，设置为指定事件流元素值，例如：“3”。其中，标识机制的URI可以是URN或者URL。
1.在本可选实施例中使用了事件消息匣
在动态自适应流媒体技术中，事件消息匣(Event Message box,‘emsg’)设置为通知与媒体呈现时间相关的一般带内事件，其通用语法定义如下：
Figure PCTCN2015096589-appb-000012
其中：
scheme_id_uri：标识事件消息方案。
value：指定事件的值。
timescale：指定时间标度。
presentation_time_delta：指定媒体呈现时间增量。
event_duration：指定在媒体呈现时间上事件的时间段。
id：标识消息实例。
message_data[]：消息主体。
根据本实施例中的方案，如果事件消息匣中“scheme_id_uri”字段设置为预定值，例如：“urn:avs:dash:event:2014”且“value”字段设置为预定值，例如：“3”，则该事件消息匣是“MP时间轴对齐”事件消息匣，其携带的消息字段遵循以下规定：
—“presentation_time_delta”字段提供MP时间轴对齐事件的时间，表示自MP时间轴与外部时间轴建立时间信息映射关系起，MP时间轴对齐事件中的媒体片段(具有相同“id”字段值)在媒体呈现时间上的增量。
—“event_duration”字段代表自MP时间轴对齐事件时间起，媒体呈现剩余时间段。如果“event_duration”取值“0”，则MP时间轴对齐事件时间后，媒体呈现终止；如果“event_duration”取值“0xFFFF”，则媒体呈现剩余时间段未知。
—“message_data[]”提供MP时间轴中媒体呈现时间与外部时间轴对齐的片段相对于所在Period时段开始时间(PeriodStart time)的呈现时间(presentation time,Tp)。
步骤S1304，客户端根据MP时间轴对齐事件消息匣中携带的信息(例如：时间信息)确定MP时间轴中媒体呈现时间MPT与外部时间轴对齐的媒体片段，并计算MP时间轴与外部时间轴建立起映射关系之后，MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间。
事件消息匣中“presentation_time_delta”字段取值为“0”的片段就是MP时间轴中的媒体呈现时间与外部时间轴对齐的片段。基于“message_data[]”字段值呈现时间Tp以及所在Representation的呈现时间偏移属性值(@presentationTimeOffset，TO)，可以计算MP时间轴对齐事件中对齐片段的媒体呈现时间MPT，即TM＝Tp-TO。
MP时间轴对齐事件中不同片段所携带的事件消息匣中可以具有相同“id”字段值，不同“presentation_time_delta”字段值。计算具有相同“id”字段值的消息实例所对应片段的媒体呈现时间增量，表示MP时间轴与外部时间轴建立起时间信息映射关系之后，MP时间轴对齐事件中相关片段在媒体呈现时间上历经的时间，即：
MPT＝MPT0+emsg.presentation_time_delta/emsg.timescale，其中，
MPT0与MPT代表MP时间轴对齐事件中对齐片段以及后续片段媒体呈现时间；
emsg.presentation_time_delta与emsg.timescale代表MP时间轴对齐事件消息匣中的媒体呈现时间增量值及时间标度值。
图14是根据本发明实施例的一种确定需要进行时间轴对齐的媒体片段的示例图，如图14所示，包括：
终端根据位置描述子中携带的事件流方案标识字段值，例如："urn:avs:dash:event:2014"，以及事件流值字段，例如：值为"3"，检索确定MPD中Period元素的AdaptationSet和Representation级别中的InbandEventStream元素，即图14中@id属性值分别为"1"和"2"的Period元素所包含的InbandEventStream元素。
终端使用该Period元素的媒体片段模板(SegmentTemplate)子元素或者媒体片段URL(SegmentURL)子元素，并结合MPD其他层次中定义的资源URL地址基址(BaseURL)元素，譬如："http://www.example.com/"，从而构造带内事件流所对应媒体片段的HTTP URL，即图14中不同指向的URL。
终端根据上述HTTP URL请求获取媒体片段内容，并提取其中携带的事件消息匣，即本发明定义的“MP时间轴对齐”事件信息。终端根据位置描述子中携带的事件标识字段值，例如："123"，检索“MP时间轴对齐”事件消息匣的“id”字段，将检索结果中“presentation_time_delta”字段值为“0”事件消息匣对应的媒体片段确定为需要与传输流进行时间轴对齐的动态自适应流媒体的媒体片段，即图14中左起第一个媒体片段。
在本发明一个可选的实施例中，终端根据需要与传输流进行时间轴对齐的媒体片段的媒体呈现时间确定该媒体片段所在时段中后续媒体片段的媒体呈现时间。特别地，当需要进行时间轴对齐的媒体片段的媒体呈现时间表示为相对于传输流时间轴的PTS时，存在以下关系：
Segments PTS(in seconds)＝Segmentsyn PTS(in seconds)+emsg.presentation_time_delta/emsg.timescale，其中：
·下标syn为需要进行时间轴对齐的媒体片段(所携带“MP时间轴对齐”事件消息匣中“presentation_time_delta”字段值为“0”)的编号；
·下标s为后续媒体片段(所携带“MP时间轴对齐”事件消息匣具有相同“id”字段值)的编号；
·emsg.presentation_time_delta为媒体片段Segments所携带“MP时间轴对齐”事件消息匣中的媒体呈现时间增量值；
·emsg.timescale为“MP时间轴对齐”事件消息匣中时间标度值；
·Segmentsyn PTS与Segments PTS分别为需要进行时间轴对齐的的媒体片段以及后续媒体片段的媒体呈现时间相对于传输流时间轴的PTS。
图21是根据本发明实施例的另一种自适应流媒体处理方法的流程图，如图21所示，该方法包括如下步骤：
步骤S2102，获取媒体片段中携带的媒体呈现(MP)时间轴对齐事件消息；
步骤S2104，根据所述MP时间轴对齐事件消息确定外部媒体资源的位置，以及所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间(EMPT)。
通过上述步骤，根据媒体片段中携带的MP时间轴对齐事件消息来确定媒体呈现时间MPT与外部时间轴对齐的媒体片段，并计算出在MP时间轴与传输流时间轴建立映射关系之后，该MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间，可以有效的解决相关技术中为广播网络和宽带网络混合媒体传输采用单一参考时钟所导致的问题，使得混合递送模式下媒体同步处理更加高效，同时也提高业务部署的可扩展性。
在图21中的步骤S2102中获取到的是MP时间轴对齐事件消息，在一个可选的实施方式中，MP时间轴对齐事件消息可以使用MP时间轴对齐事件消息匣来实现，当然也可以使用其他的方式实现。下面对使用MP时间轴对齐时间消息匣的可选实施例进行了说明。
MP时间轴(Media Presentation timeline)对齐事件设置为通知客户端外部媒体资源的位置，以及外部媒体资源被映射到MP时间轴上的媒体呈现时间。
若事件消息匣中“scheme_id_uri”字段设置为“urn:avs:dash:event:2014”，“value”字段设置为“3”，则是MP时间轴对齐事件消息匣，其所携带的消息字段遵循以下规定：
—“presentation_time_delta”字段提供MP时间轴对齐事件中的外部媒体资源在MP时间轴上的媒体呈现时间相对于片段最早呈现时间的增量；
—“message_data[]”字段提供MP时间轴对齐事件中外部媒体资源位置的URL。
事件消息匣中“presentation_time_delta”字段取值为“0”的片段就是MP时间轴中的媒体呈现时间与外部时间轴对齐的片段，即指外部媒体资源在MP时间轴上的媒体呈现时间等同于对齐片段的最早呈现时间。
针对MP时间轴对齐事件中相同的外部媒体资源，MP时间轴对齐事件消息匣实例须具有相同“id”字段值，且满足：
Figure PCTCN2015096589-appb-000013
其中，
·EMPTid代表MP时间轴与外部时间轴建立映射关系之后，MP时间轴对齐事件中的外部媒体资源在MP时间轴上的外部媒体呈现时间
·
Figure PCTCN2015096589-appb-000014
代表MP时间轴对齐事件中片段i的最早呈现时间
·
Figure PCTCN2015096589-appb-000015
与
Figure PCTCN2015096589-appb-000016
分别代表MP时间轴对齐事件消息匣实例i中的媒体呈现时间增量值及时间标度值
图22是根据本发明实施例的一种确定需要进行时间轴对齐的媒体片段的示例图，如图22所示，包括：
MPD文档的多个Representation元素中包含对应于本事件方案的InbandEventStream元素。当需要建立媒体呈现时间(MPT)与外部时间轴时间信息的映射关系时，外部媒体资源相对于片段最早呈现时间增量(presentation_time_delta)，以及外部媒体资源的位置(message_data[])等信息将被添加到对应的MP时间轴对齐事件消息匣。MP时间轴对齐事件消息可以出现在多个片段中。
终端根据事件流方案标识字段值，例如"urn:avs:dash:event:2014"，以及事件流值字段值，例如"3"，检索确定MPD中Period元素的AdaptationSet和Representation级别中的InbandEventStream元素，即图22中@id属性值分别为"1"和"2"的Period元素所包含的InbandEventStream元素。
终端使用该Period元素的媒体片段模板(SegmentTemplate)子元素或者媒体片段URL(SegmentURL)子元素，并结合MPD其他层次中定义的资源URL地址基址(BaseURL)元素，譬如："http://www.example.com/"，从而构造带内事件流所对应媒体片段的HTTP URL，即图22中不同指向的URL。
终端根据上述HTTP URL请求获取媒体片段内容，并提取其中携带的事件消息匣，即本发明定义的“MP时间轴对齐”事件信息。终端根据事件标识字段值，例如："123"，检索“MP时间轴对齐”事件消息匣的“id”字段，将检索结果中“presentation_time_delta”字段值为“0”事件消息匣对应的媒体片段确定为将与外部媒体资源，譬如：传输流进行时间轴对齐的动态自适应流媒体的媒体片段，即图22中左起第二个媒体片段。
图15是根据本发明实施例的再一种自适应流媒体处理装置的流程图，如图15所示，该装置包括第一获取模块152、确定模块154和计算模块156，下面对该装置进行说明。
第一获取模块152，设置为获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息；确定模块154，连接至上述第一获取模块152，设置为根据MP时间轴对齐事件消息确定对齐的媒体片段，其中，该对齐的媒体片段是该MP时间轴中媒体呈现时间 MPT与外部时间轴进行对齐的媒体片段；计算模块156，连接至上述确定模块154，设置为计算在MP时间轴与传输流时间轴建立映射关系之后，上述MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间。
图16是根据本发明实施例的再一种自适应流媒体处理装置中第一获取模块152的结构框图，如图16所示，该第一获取模块152包括第一访问单元162和第一获取单元164，下面对该第一获取模块152进行说明。
第一访问单元162，设置为访问媒体呈现描述MPD中带内事件流InbandEventStream元素；第一获取单元164，连接至上述第一访问单元162，设置为获取对应于上述带内事件流InbandEventStream元素的媒体片段中携带的MP时间轴对齐事件消息。
其中，上述InbandEventStream元素可以包括：事件流方案标识schemeIdUri属性和事件流值value属性，设置为指示MP时间轴对齐事件。
在一个可选的实施例中，上述MP时间轴对齐事件消息包括以下至少之一：呈现时间增量presentation_time_delta字段，设置为提供MP时间轴对齐事件的时间；事件持续时间event_duration字段，设置为表示媒体呈现剩余时间段；消息数据message_data字段，设置为提供上述对齐的媒体片段相对于所在时段Period时段开始时间的呈现时间Tp。
其中，上述MP时间轴对齐事件消息还可以包括：事件消息方案标识scheme_id_uri字段以及事件值value字段；该事件消息方案标识scheme_id_uri字段值等于事件流方案标识schemeIdUri属性值，并且事件值value字段值等于事件流值value属性值。
图17是根据本发明实施例的再一种自适应流媒体处理装置中确定模块154的结构框图，如图17所示，该确定模块154包括第一确定单元172，下面对该确定模块154进行说明。
第一确定单元172，设置为确定MP时间轴对齐事件消息中presentation_time_delta字段值为预定值的媒体片段作为对齐的媒体片段。
在一个可选的实施例中，上述装置还包括：基于MP时间轴对齐事件消息中的message_data字段中呈现时间Tp以及对齐的媒体片段所在表示Representation的呈现时间偏移TO属性值，计算该对齐的媒体片段的MPT，即TM＝Tp-TO。
图18是根据本发明实施例的再一种自适应流媒体处理装置中计算模块156的结构框图，如图18所示，该计算模块156包括第二确定单元182和计算单元184，下面对该计算模块156进行说明。
第二确定单元182，设置为确定MP时间轴对齐事件消息中具有相同标识id字段值的媒体片段为MP时间轴对齐事件中的媒体片段；计算单元184，连接至上述第二确定单元182，设置为根据MP时间轴对齐事件消息中呈现时间增量presentation_time_delta字段值，和时间标度timescale字段值，计算MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间。
在一个可选的实施例中，上述计算单元184包括：根据以下公式计算MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间：MPT＝MPT0+emsg.presentation_time_delta/emsg.timescale，其中，MPT0与MPT分别代表MP时间轴对齐事件中的对齐媒体片段以及后续媒体片段的媒体呈现时间；emsg.presentation_time_delta与emsg.timescale分别代表MP时间轴对齐事件消息中的媒体呈现时间增量值及时间标度值。
图19是根据本发明实施例的再一种自适应流媒体处理装置的优选结构框图，如图19所示，该装置除包括图15所示的所有模块外，还包括第二获取模块192，下面对该装置进行说明。
第二获取模块192，连接至上述第一获取模块152，设置为获取携带有上述MP时间轴对齐事件消息的媒体片段。
图20是根据本发明实施例的再一种自适应流媒体处理装置中第二获取模块192的结构框图，如图20所示，该第二获取模块192包括第二获取单元202、第三确定单元204和第三获取单元206，下面对该第二获取模块192进行说明。
第二获取单元202，设置为获取事件流方案标识字段以及事件流值字段；第三确定单元204，连接至上述第二获取单元202，设置为使用事件流方案标识字段值以及事件流值字段值，确定媒体呈现描述MPD中的带内事件流InbandEventStream元素；第三获取单元206，连接至上述第三确定单元204，设置为获取对应于带内事件流InbandEventStream元素的媒体片段作为媒体片段。
图23是根据本发明实施例的另一种自适应流媒体处理装置的结构框图，如图23所示，该装置包括第三获取模块232和第三确定模块234，下面对该装置进行说明。
第三获取模块232，设置为获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息；第三确定模块234，连接至上述第三获取模块232，设置为根据所述MP时间轴对齐事件消息确定外部媒体资源的位置，以及所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间。
可选地，所述MP时间轴对齐事件消息至少包括：消息数据message_data字段，设置为提供MP时间轴对齐事件中的所述外部媒体资源位置的URL。
可选地，所述MP时间轴对齐事件消息至少包括：呈现时间增量presentation_time_delta字段，设置为提供MP时间轴对齐事件中的所述外部媒体资源在MP时间轴上的媒体呈现时间相对于所述媒体片段最早呈现时间的增量。
可选地，所述MP时间轴对齐事件消息还包括：事件消息方案标识scheme_id_uri字段，设置为提供事件消息方案标识；事件值value字段，设置为提供事件值。
图24是根据本发明实施例的另一种自适应流媒体处理装置的第三确定模块234的结构框图，如图24所示，该第三确定模块234包括第四确定单元2342，下面对该第三确定模块234进行说明。
第四确定单元2342，设置为将所述MP时间轴对齐事件消息中message_data字段值确定为所述外部媒体资源的位置。
可选地，如图24所示，所述第三确定模块也可以包括：
第五确定单元2344，设置为根据所述媒体片段的最早呈现时间EPT，以及所述MP时间轴对齐事件消息中presentation_time_delta字段值，确定所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间。
可选地，如图24所示，所述第三确定模块234还可以包括：
第六确定单元2346，连接至上述第五确定单元2344，设置为确定所述MP时间轴对齐事件消息中presentation_time_delta字段取值为0的媒体片段为MP时间轴中的媒体呈现时间MPT与外部时间轴对齐的媒体片段；其中所述外部媒体资源在MP时间轴上的媒体呈现时间等于所述对齐的媒体片段的最早呈现时间。
可选地，如图24所示，所述第三确定模块还可以包括：
第七确定单元2348，连接至上述第五确定单元2344和/或第六确定单元2346，设置为确认针对MP时间轴对齐事件中相同的外部媒体资源，所述MP时间轴对齐事件消息具有相同消息标识id字段值，且满足：
Figure PCTCN2015096589-appb-000017
其中，
所述EMPTid为MP时间轴与外部时间轴建立映射关系之后，MP时间轴对齐事件中的外部媒体资源在MP时间轴上的外部媒体呈现时间；
所述
Figure PCTCN2015096589-appb-000018
为MP时间轴对齐事件中第i个媒体片段的最早呈现时间；
所述
Figure PCTCN2015096589-appb-000019
为第i个MP时间轴对齐事件消息中媒体呈现时间增量字段值；
所述
Figure PCTCN2015096589-appb-000020
为第i个MP时间轴对齐事件消息中时间标度字段值；
所述i为正整数。
图25是根据本发明实施例的另一种自适应流媒体处理装置的第三获取模块232的结构框图，如图25所示，该第三获取模块232包括第二访问单元2322和第四获取单元2324，下面对该第三获取模块232进行说明。
第二访问单元2322，设置为访问媒体呈现描述MPD中的带内事件流InbandEventStream元素；
其中，所述InbandEventStream元素的事件流方案标识schemeIdUri属性值等于所述MP时间轴对齐事件消息中scheme_id_uri字段值，并且所述InbandEventStream元素的事件流值value属性值等于所述MP时间轴对齐事件消息中value字段值；
第四获取单元2324，连接至上述第二访问单元2322，设置为获取对应于所述InbandEventStream元素的媒体片段中携带的MP时间轴对齐事件消息。
在一个可选的实施例中，上述外部时间轴包括传输流时间轴。
显然，本领域的技术人员应该明白，上述的本发明的各模块或各步骤可以用通用的计算装置来实现，它们可以集中在单个的计算装置上，或者分布在多个计算装置所组成的网络上，可选地，它们可以用计算装置可执行的程序代码来实现，从而，可以将它们存储在存储装置中由计算装置来执行，并且在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤，或者将它们分别制作成各个集成电路模块，或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样，本发明不限制于任何特定的硬件和软件结合。
以上所述仅为本发明的优选实施例而已，并不用于限制本发明，对于本领域的技术人员来说，本发明可以有各种更改和变化。凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。
工业实用性
如上所述，本发明实施例提供的一种自适应流媒体处理方法及装置，具有以下有益效果：解决了现有技术中采用单一参考时钟所导致的问题，实现了广播网络和宽带网络混合递送模式下媒体同步播放。

Claims (42)

  1. 一种自适应流媒体处理方法，包括：
    获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息；
    根据所述MP时间轴对齐事件消息确定对齐的媒体片段，其中，所述对齐的媒体片段是所述MP时间轴中媒体呈现时间MPT与外部时间轴进行对齐的媒体片段；
    计算在所述MP时间轴与所述外部时间轴建立映射关系之后，所述MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间。
  2. 根据权利要求1所述的方法，其中，获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息，包括：
    访问媒体呈现描述MPD中带内事件流InbandEventStream元素；
    获取对应于所述带内事件流InbandEventStream元素的媒体片段中携带的MP时间轴对齐事件消息。
  3. 根据权利要求2所述的方法，其中，所述InbandEventStream元素包括：
    事件流方案标识schemeIdUri属性和事件流值value属性，设置为指示MP时间轴对齐事件。
  4. 根据权利要求1所述的方法，其中，所述MP时间轴对齐事件消息包括以下至少之一：
    呈现时间增量presentation_time_delta字段，设置为提供MP时间轴对齐事件的时间；
    事件持续时间event_duration字段，设置为表示媒体呈现剩余时间段；
    消息数据message_data字段，设置为提供所述对齐的媒体片段相对于所在时段Period时段开始时间的呈现时间Tp。
  5. 根据权利要求4所述的方法，其中，所述MP时间轴对齐事件消息还包括：
    事件消息方案标识scheme_id_uri字段以及事件值value字段；
    所述事件消息方案标识scheme_id_uri字段值等于事件流方案标识schemeIdUri属性值，并且所述事件值value字段值等于事件流值value属性值。
  6. 根据权利要求4所述的方法，其中，根据所述MP时间轴对齐事件消息确定对齐的媒体片段，包括：
    确定所述MP时间轴对齐事件消息中presentation_time_delta字段值为预定值的媒体片段作为所述对齐的媒体片段。
  7. 根据权利要求4所述的方法，其中，还包括：
    基于所述MP时间轴对齐事件消息中的message_data字段中呈现时间Tp以及所述对齐的媒体片段所在表示Representation的呈现时间偏移TO属性值，计算所述对齐的媒体片段的MPT，即TM＝Tp-TO。
  8. 根据权利要求4所述的方法，其中，计算所述媒体片段在媒体呈现时间上历经的时间，包括：
    确定所述MP时间轴对齐事件消息中具有相同标识id字段值的媒体片段为所述MP时间轴对齐事件中的媒体片段；
    根据所述MP时间轴对齐事件消息中呈现时间增量presentation_time_delta字段值，和时间标度timescale字段值，计算所述MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间。
  9. 根据权利要求4或8所述的方法，其中，根据所述MP时间轴对齐事件消息中呈现时间增量presentation_time_delta字段值，和时间标度timescale字段值，计算所述MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间，包括：
    根据以下公式计算所述MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间：
    MPT＝MPT0+emsg.presentation_time_delta/emsg.timescale，其中，
    MPT0与MPT分别代表所述MP时间轴对齐事件中的对齐媒体片段以及后续媒体片段的媒体呈现时间；
    emsg.presentation_time_delta与emsg.timescale分别代表所述MP时间轴对齐事件消息中的媒体呈现时间增量值及时间标度值。
  10. 根据权利要求1至9中任一项所述的方法，其中，在获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息之前，所述方法还包括：
    获取携带有所述MP时间轴对齐事件消息的所述媒体片段。
  11. 根据权利要求10所述的方法，其中，获取携带有所述MP时间轴对齐事件消息的所述媒体片段包括：
    获取事件流方案标识字段以及事件流值字段；
    使用所述事件流方案标识字段值以及事件流值字段值，确定媒体呈现描述MPD中的带内事件流InbandEventStream元素；
    获取对应于所述带内事件流InbandEventStream元素的媒体片段作为所述媒体片段。
  12. 根据权利要求1至9中任一项所述的方法，其中，所述外部时间轴包括传输流时间轴。
  13. 一种自适应流媒体处理装置，包括：
    第一获取模块，设置为获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息；
    确定模块，设置为根据所述MP时间轴对齐事件消息确定对齐的媒体片段，其中，所述对齐的媒体片段是所述MP时间轴中媒体呈现时间MPT与外部时间轴进行对齐的媒体片段；
    计算模块，设置为计算在所述MP时间轴与所述外部时间轴建立映射关系之后，所述MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间。
  14. 根据权利要求13所述的装置，其中，所述第一获取模块包括：
    第一访问单元，设置为访问媒体呈现描述MPD中带内事件流InbandEventStream元素；
    第一获取单元，设置为获取对应于所述带内事件流InbandEventStream元素的媒体片段中携带的MP时间轴对齐事件消息。
  15. 根据权利要求14所述的装置，其中，所述InbandEventStream元素包括：
    事件流方案标识schemeIdUri属性和事件流值value属性，设置为指示MP时间轴对齐事件。
  16. 根据权利要求13所述的装置，其中，所述MP时间轴对齐事件消息包括以下至少之一：
    呈现时间增量presentation_time_delta字段，设置为提供MP时间轴对齐事件的时间；
    事件持续时间event_duration字段，设置为表示媒体呈现剩余时间段；
    消息数据message_data字段，设置为提供所述对齐的媒体片段相对于所在时段Period时段开始时间的呈现时间Tp。
  17. 根据权利要求16所述的装置，其中，所述MP时间轴对齐事件消息还包括：
    事件消息方案标识scheme_id_uri字段以及事件值value字段；
    所述事件消息方案标识scheme_id_uri字段值等于事件流方案标识schemeIdUri属性值，并且所述事件值value字段值等于事件流值value属性值。
  18. 根据权利要求16所述的装置，其中，所述确定模块包括：
    第一确定单元，设置为确定所述MP时间轴对齐事件消息中presentation_time_delta字段值为预定值的媒体片段作为所述对齐的媒体片段。
  19. 根据权利要求16所述的装置，其中，还包括：
    基于所述MP时间轴对齐事件消息中的message_data字段中呈现时间Tp以及所述对齐的媒体片段所在表示Representation的呈现时间偏移TO属性值，计算所述对齐的媒体片段的MPT，即TM＝Tp-TO。
  20. 根据权利要求16所述的装置，其中，所述计算模块包括：
    第二确定单元，设置为确定所述MP时间轴对齐事件消息中具有相同标识id字段值的媒体片段为所述MP时间轴对齐事件中的媒体片段；
    计算单元，设置为根据所述MP时间轴对齐事件消息中呈现时间增量presentation_time_delta字段值，和时间标度timescale字段值，计算所述MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间。
  21. 根据权利要求16或20所述的装置，其中，所述计算模块包括：
    根据以下公式计算所述MP时间轴对齐事件中的媒体片段在媒体呈现时间上历经的时间：
    MPT＝MPT0+emsg.presentation_time_delta/emsg.timescale，其中，
    MPT0与MPT分别代表所述MP时间轴对齐事件中的对齐媒体片段以及后续媒体片段的媒体呈现时间；
    emsg.presentation_time_delta与emsg.timescale分别代表所述MP时间轴对齐事件消息中的媒体呈现时间增量值及时间标度值。
  22. 根据权利要求13至21中任一项所述的装置，其中，所述装置还包括：
    第二获取模块，设置为获取携带有所述MP时间轴对齐事件消息的所述媒体片段。
  23. 根据权利要求22所述的装置，其中，所述第二获取模块包括：
    第二获取单元，设置为获取事件流方案标识字段以及事件流值字段；
    第三确定单元，设置为使用所述事件流方案标识字段值以及事件流值字段值，确定媒体呈现描述MPD中的带内事件流InbandEventStream元素；
    第三获取单元，设置为获取对应于所述带内事件流InbandEventStream元素的媒体片段作为所述媒体片段。
  24. 根据权利要求13至21中任一项所述的装置，其中，所述外部时间轴包括传输流时间轴。
  25. 一种自适应流媒体处理方法，包括：
    获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息；
    根据所述MP时间轴对齐事件消息确定外部媒体资源的位置，以及所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间EMPT。
  26. 根据权利要求25所述的方法，其中，所述MP时间轴对齐事件消息至少包括：
    消息数据message_data字段，设置为提供MP时间轴对齐事件中的所述外部媒体资源位置的URL。
  27. 根据权利要求25所述的方法，其中，所述MP时间轴对齐事件消息至少包括：
    呈现时间增量presentation_time_delta字段，设置为提供MP时间轴对齐事件中的所述外部媒体资源在MP时间轴上的媒体呈现时间相对于所述媒体片段最早呈现时间的增量。
  28. 根据权利要求26或27所述的方法，其中，所述MP时间轴对齐事件消息还包括：
    事件消息方案标识scheme_id_uri字段，设置为提供事件消息方案标识；
    事件值value字段，设置为提供事件值。
  29. 根据权利要求26所述的方法，其中，根据所述MP时间轴对齐事件消息确定外部媒体资源的位置，包括：
    将所述MP时间轴对齐事件消息中message_data字段值确定为所述外部媒体资源的位置。
  30. 根据权利要求27所述的方法，其中，根据所述MP时间轴对齐事件消息确定所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间，包括：
    根据所述媒体片段的最早呈现时间EPT，以及所述MP时间轴对齐事件消息中presentation_time_delta字段值，确定所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间。
  31. 根据权利要求27或30所述的方法，其中，根据所述MP时间轴对齐事件消息确定所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间还包括：
    确定所述MP时间轴对齐事件消息中presentation_time_delta字段取值为0的媒体片段为MP时间轴中的媒体呈现时间MPT与外部时间轴对齐的媒体片段；
    所述外部媒体资源在MP时间轴上的媒体呈现时间等于所述对齐的媒体片段的最早呈现时间。
  32. 根据权利要求27、30和31中任一项所述的方法，其中，根据所述MP时间轴对齐事件消息确定所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间，还包括：
    针对MP时间轴对齐事件中相同的外部媒体资源，所述MP时间轴对齐事件消息具有相同消息标识id字段值，且满足：
    Figure PCTCN2015096589-appb-100001
    其中，
    所述EMPTid为MP时间轴与外部时间轴建立映射关系之后，MP时间轴对齐事件中的外部媒体资源在MP时间轴上的外部媒体呈现时间；
    所述
    Figure PCTCN2015096589-appb-100002
    为MP时间轴对齐事件中第i个媒体片段的最早呈现时间；
    所述
    Figure PCTCN2015096589-appb-100003
    为第i个MP时间轴对齐事件消息中媒体呈现时间增量字段值；
    所述
    Figure PCTCN2015096589-appb-100004
    为第i个MP时间轴对齐事件消息中时间标度字段值；
    所述i为正整数。
  33. 根据权利要求25至32中任一项所述的方法，其中，获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息还包括：
    访问媒体呈现描述MPD中的带内事件流InbandEventStream元素；
    所述InbandEventStream元素的事件流方案标识schemeIdUri属性值等于所述MP时间轴对齐事件消息中scheme_id_uri字段值，并且所述InbandEventStream元素的事件流值value属性值等于所述MP时间轴对齐事件消息中value字段值；
    获取对应于所述InbandEventStream元素的媒体片段中携带的MP时间轴对齐 事件消息。
  34. 一种自适应流媒体处理装置，包括：
    第三获取模块，设置为获取媒体片段中携带的媒体呈现MP时间轴对齐事件消息；
    第三确定模块，设置为根据所述MP时间轴对齐事件消息确定外部媒体资源的位置，以及所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间EMPT。
  35. 根据权利要求34所述的装置，其中，所述MP时间轴对齐事件消息至少包括：
    消息数据message_data字段，设置为提供MP时间轴对齐事件中的所述外部媒体资源位置的URL。
  36. 根据权利要求34所述的装置，其中，所述MP时间轴对齐事件消息至少包括：
    呈现时间增量presentation_time_delta字段，设置为提供MP时间轴对齐事件中的所述外部媒体资源在MP时间轴上的媒体呈现时间相对于所述媒体片段最早呈现时间的增量。
  37. 根据权利要求35或36所述的装置，其中，所述MP时间轴对齐事件消息还包括：
    事件消息方案标识scheme_id_uri字段，设置为提供事件消息方案标识；
    事件值value字段，设置为提供事件值。
  38. 根据权利要求35所述的装置，其中，所述第三确定模块包括：
    第四确定单元，设置为将所述MP时间轴对齐事件消息中message_data字段值确定为所述外部媒体资源的位置。
  39. 根据权利要求36所述的装置，其中，所述第三确定模块包括：
    第五确定单元，设置为根据所述媒体片段的最早呈现时间EPT，以及所述MP时间轴对齐事件消息中presentation_time_delta字段值，确定所述外部媒体资源被映射到MP时间轴上的外部媒体呈现时间。
  40. 根据权利要求36或39所述的装置，其中，所述第三确定模块还包括：
    第六确定单元，设置为确定所述MP时间轴对齐事件消息中presentation_time_delta字段取值为0的媒体片段为MP时间轴中的媒体呈现时间MPT与外部时间轴对齐的媒体片段；
    其中所述外部媒体资源在MP时间轴上的媒体呈现时间等于所述对齐的媒体 片段的最早呈现时间。
  41. 根据权利要求36、39和40中任一项所述的装置，其中，所述第三确定模块还包括：
    第七确定单元，设置为确认针对MP时间轴对齐事件中相同的外部媒体资源，所述MP时间轴对齐事件消息具有相同消息标识id字段值，且满足：
    Figure PCTCN2015096589-appb-100005
    其中，
    所述EMPTid为MP时间轴与外部时间轴建立映射关系之后，MP时间轴对齐事件中的外部媒体资源在MP时间轴上的外部媒体呈现时间；
    所述
    Figure PCTCN2015096589-appb-100006
    为MP时间轴对齐事件中第i个媒体片段的最早呈现时间；
    所述
    Figure PCTCN2015096589-appb-100007
    为第i个MP时间轴对齐事件消息中媒体呈现时间增量字段值；
    所述
    Figure PCTCN2015096589-appb-100008
    为第i个MP时间轴对齐事件消息中时间标度字段值；
    所述i为正整数。
  42. 根据权利要求34至41中任一项所述的装置，其中，所述第三获取模块还包括：
    第二访问单元，设置为访问媒体呈现描述MPD中的带内事件流InbandEventStream元素；
    其中，所述InbandEventStream元素的事件流方案标识schemeIdUri属性值等于所述MP时间轴对齐事件消息中scheme_id_uri字段值，并且所述InbandEventStream元素的事件流值value属性值等于所述MP时间轴对齐事件消息中value字段值；
    第四获取单元，设置为获取对应于所述InbandEventStream元素的媒体片段中携带的MP时间轴对齐事件消息。
PCT/CN2015/096589 2015-03-13 2015-12-07 自适应流媒体处理方法及装置 WO2016145913A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US15/558,008 US10715571B2 (en) 2015-03-13 2015-12-07 Self-adaptive streaming medium processing method and apparatus
EP15885282.2A EP3270601B1 (en) 2015-03-13 2015-12-07 Self-adaptive streaming medium processing method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510113122.6A CN106034262B (zh) 2015-03-13 2015-03-13 自适应流媒体处理方法及装置
CN201510113122.6 2015-03-13

Publications (1)

Publication Number Publication Date
WO2016145913A1 true WO2016145913A1 (zh) 2016-09-22

Family

ID=56918284

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/096589 WO2016145913A1 (zh) 2015-03-13 2015-12-07 自适应流媒体处理方法及装置

Country Status (4)

Country Link
US (1) US10715571B2 (zh)
EP (1) EP3270601B1 (zh)
CN (1) CN106034262B (zh)
WO (1) WO2016145913A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108769723A (zh) * 2018-05-28 2018-11-06 广州虎牙信息科技有限公司 直播视频中优质内容的推送方法、装置、设备和存储介质

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20120034550A (ko) 2010-07-20 2012-04-12 한국전자통신연구원 스트리밍 컨텐츠 제공 장치 및 방법
US9467493B2 (en) * 2010-09-06 2016-10-11 Electronics And Telecommunication Research Institute Apparatus and method for providing streaming content
EP3262789B1 (en) * 2015-02-26 2020-01-01 Second Screen Ventures Ltd. System and method for associating messages with media during playing thereof
WO2016163772A2 (ko) * 2015-04-07 2016-10-13 엘지전자 주식회사 방송 신호 송신 장치, 방송 신호 수신 장치, 방송 신호 송신 방법, 및 방송 신호 수신 방법
US10986421B2 (en) * 2017-07-03 2021-04-20 Dolby Laboratories Licensing Corporation Identification and timing data for media content
CN113179262B (zh) * 2018-07-11 2022-12-06 上海交通大学 异构网络下媒体的传输方法及系统
JP6485935B1 (ja) * 2018-10-18 2019-03-20 ベルフェイス株式会社 オンラインコミュニケーションのレビューシステム、方法、及びコンピュータプログラム
CN110213633A (zh) * 2019-06-03 2019-09-06 重庆蓝岸通讯技术有限公司 一种流媒体广播电台列表的显示处理方法
US11490169B2 (en) * 2019-07-02 2022-11-01 Tencent America LLC Events in timed metadata tracks
US11303688B2 (en) * 2019-09-30 2022-04-12 Tencent America LLC Methods and apparatuses for dynamic adaptive streaming over HTTP
US11687386B2 (en) * 2020-10-07 2023-06-27 Tencent America LLC MPD validity expiration processing model
US11683355B2 (en) * 2021-01-05 2023-06-20 Tencent America LLC Methods and apparatuses for dynamic adaptive streaming over HTTP

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7953707B2 (en) * 2002-05-09 2011-05-31 International Business Machines Corporation System and program product for sequential coordination of external database application events with asynchronous internal database events
CN103947215A (zh) * 2011-09-23 2014-07-23 韩国电子通信研究院 用于活动画面专家组媒体传输系统的传送媒体数据的方法和设备、以及接收媒体数据的设备和方法
CN103988479A (zh) * 2011-10-13 2014-08-13 三星电子株式会社 用于在广播系统中配置控制消息的装置和方法
CN104185040A (zh) * 2013-05-25 2014-12-03 中兴通讯股份有限公司 应用同步方法、应用服务器及终端
CN104205859A (zh) * 2012-04-05 2014-12-10 汤姆逊许可公司 多媒体流的同步

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101378356B (zh) * 2008-06-10 2011-05-11 中兴通讯股份有限公司 一种ip实时流媒体的播放方法
US8996538B1 (en) * 2009-05-06 2015-03-31 Gracenote, Inc. Systems, methods, and apparatus for generating an audio-visual presentation using characteristics of audio, visual and symbolic media objects
US9323438B2 (en) * 2010-07-15 2016-04-26 Apple Inc. Media-editing application with live dragging and live editing capabilities
KR101651137B1 (ko) * 2010-07-19 2016-08-25 엘지전자 주식회사 미디어 세그먼트 송수신 방법 및 그를 이용한 송수신 장치
US9253233B2 (en) * 2011-08-31 2016-02-02 Qualcomm Incorporated Switch signaling methods providing improved switching between representations for adaptive HTTP streaming
CN104205895B (zh) * 2012-03-19 2018-07-13 诺基亚技术有限公司 用于建立提供网络辅助信息的条件的方法和设备
WO2013163477A1 (en) * 2012-04-25 2013-10-31 Huawei Technologies Co., Ltd. Systems and methods for segment integrity and authenticity for adaptive streaming
WO2014008637A1 (en) * 2012-07-11 2014-01-16 Nokia Corporation Method and apparatus for interacting with a media presentation description that describes a summary media presentation and an original media presentation
US20140019635A1 (en) * 2012-07-13 2014-01-16 Vid Scale, Inc. Operation and architecture for dash streaming clients
KR101843328B1 (ko) * 2012-10-26 2018-03-28 인텔 코포레이션 비디오 방향 조정(cvo)를 갖는 스트리밍
CN103036888B (zh) * 2012-12-19 2016-04-13 南京视海网络科技有限公司 自适应的流媒体播放方法及其自适应播放单元
US20150201253A1 (en) * 2014-01-10 2015-07-16 Samsung Electronics Co., Ltd. Methods and apparatus for universal presentation timeline alignment
KR101788066B1 (ko) * 2014-01-13 2017-11-15 엘지전자 주식회사 하나 이상의 네트워크를 통해 방송 컨텐츠를 송수신하는 장치 및 방법
US20150261425A1 (en) * 2014-03-14 2015-09-17 Apple Inc. Optimized presentation of multimedia content

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7953707B2 (en) * 2002-05-09 2011-05-31 International Business Machines Corporation System and program product for sequential coordination of external database application events with asynchronous internal database events
CN103947215A (zh) * 2011-09-23 2014-07-23 韩国电子通信研究院 用于活动画面专家组媒体传输系统的传送媒体数据的方法和设备、以及接收媒体数据的设备和方法
CN103988479A (zh) * 2011-10-13 2014-08-13 三星电子株式会社 用于在广播系统中配置控制消息的装置和方法
CN104205859A (zh) * 2012-04-05 2014-12-10 汤姆逊许可公司 多媒体流的同步
CN104185040A (zh) * 2013-05-25 2014-12-03 中兴通讯股份有限公司 应用同步方法、应用服务器及终端

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CHRISTOPHER KOHNEN ET AL.: "A DVB/IP Streaming Testbed for Hybrid Digital Media Content Synchronization", IEEE SECOND INTERNATIONAL CONFERENCE ON CONSUMER ELECTRONICS - BERLIN, 31 December 2012 (2012-12-31), XP032256541 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108769723A (zh) * 2018-05-28 2018-11-06 广州虎牙信息科技有限公司 直播视频中优质内容的推送方法、装置、设备和存储介质
CN108769723B (zh) * 2018-05-28 2020-12-15 广州虎牙信息科技有限公司 直播视频中优质内容的推送方法、装置、设备和存储介质

Also Published As

Publication number Publication date
CN106034262A (zh) 2016-10-19
US20180159909A1 (en) 2018-06-07
US10715571B2 (en) 2020-07-14
CN106034262B (zh) 2021-01-22
EP3270601A4 (en) 2018-01-17
EP3270601A1 (en) 2018-01-17
EP3270601B1 (en) 2020-10-21

Similar Documents

Publication Publication Date Title
WO2016145913A1 (zh) 自适应流媒体处理方法及装置
US9860611B2 (en) Broadcast service transmitting method, broadcasting service receiving method and broadcast service receiving apparatus
US9661371B2 (en) Method for transmitting a broadcast service, apparatus for receiving same, and method for processing an additional service using the apparatus for receiving same
US9596510B2 (en) Method for transmitting broadcast service, method for receiving broadcast service, and apparatus for receiving broadcast service
US9225443B2 (en) Method for transmitting broadcast service, method for receiving the broadcasting service, and apparatus for receiving the broadcasting service
US9667902B2 (en) Method for transmitting a broadcast service, method for receiving a broadcast service, and apparatus for receiving a broadcast service
CN104885473A (zh) 用于经由http的动态自适应流式传输(dash)的实况定时
US20130125187A1 (en) Method for transceiving media files and device for transmitting/receiving using same
US9723362B2 (en) Method for transmitting and receiving broadcast service and receiving device thereof
CA2948124C (en) Reception apparatus, reception method, transmission apparatus, and transmission method
KR102499231B1 (ko) 수신 장치, 송신 장치 및 데이터 처리 방법
JPWO2015137149A1 (ja) 受信装置、受信方法、送信装置、及び、送信方法
US20100262492A1 (en) Method and arrangement relating to a media structure
CN107925798B (zh) 接收设备、发送设备和数据处理方法
WO2016107192A1 (zh) 自适应流媒体处理方法及装置
CA2849064A1 (en) Method of processing an interactive service related to a broadcast program and apparatus therefor
Le Feuvre et al. Hybrid broadcast services using MPEG DASH
KR101710452B1 (ko) 멀티미디어 전송 시스템에서 서비스 발견 정보를 송수신하는 방법 및 장치와 이를 위한 파일 구조
Kang et al. Method of DASH segments into a MMTP stream for switching contents under a hybrid broadcasting environment
CA2822968C (en) Broadcast service transmitting method, broadcasting service receiving method and broadcast service receiving apparatus

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15885282

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

REEP Request for entry into the european phase

Ref document number: 2015885282

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 15558008

Country of ref document: US