CN112188256A

CN112188256A - Information processing method, information providing device, electronic device and storage medium

Info

Publication number: CN112188256A
Application number: CN202010627768.7A
Authority: CN
Inventors: 索达加伊拉吉
Original assignee: Tencent America LLC
Current assignee: Tencent America LLC
Priority date: 2019-07-02
Filing date: 2020-07-01
Publication date: 2021-01-05
Anticipated expiration: 2040-07-01
Also published as: CN112188256B

Abstract

The embodiment of the application provides an information processing method, an information providing device and a storage medium, wherein the information processing method comprises the steps of receiving a manifest file of a content stream; notifying an application of at least one event schema in a timed metadata track of the content stream in accordance with the received manifest file; receiving a subscription request for an event schema of the at least one event schema from the application; sending an event message associated with the event schema in the timed metadata track to the application in accordance with the received subscription request.

Description

Information processing method, information providing device, electronic device and storage medium

Cross-referencing

The present disclosure claims priority from united states provisional application No. 62/869,861 filed on 2.7.2019 and united states patent application No. 16/916,617 filed on 30.6.2020 at the united states patent and trademark office, the entire contents of which are incorporated herein by reference in their entirety.

Technical Field

This disclosure describes embodiments relating to timing metadata tracks.

Background

The background description provided herein is intended to be a general presentation of the background of the application. Work of the presently named inventors, to the extent it is described in this background section, as well as aspects of the description, is not admitted to be prior art by inclusion in this application, nor is it expressly or implied that it is prior art to the present application.

Moving Picture Experts Group Dynamic Adaptive Streaming over HTTP (MPEG-DASH: Moving Picture Experts Group Dynamic Adaptive Streaming over HTTP or DASH: Dynamic Adaptive Streaming over HTTP) provides a standard for Streaming multimedia content over an Internet Protocol (IP: Internet Protocol) network. Streaming multimedia content may include content streams of live or on-demand (pre-recorded) content.

In MPEG-DASH, a manifest called Media Presentation Description (MPD) describes Media data segments and provides information needed by a DASH client to render multimedia content. The MPD may indicate different representations (e.g., quality levels) of the available media, such as different video resolutions and bitrates of the same media segments. Further, although the MPD may signal that there are some types of events, the MPEG-DASH does not provide a signaling solution for the timing metadata track, and the information carried by the metadata track cannot be disclosed to the application, which reduces the efficiency of the application in reproducing the multimedia content.

Disclosure of Invention

Based on the above problems, embodiments of the present application provide an information processing method, an information providing apparatus, an electronic device, and a storage medium.

The information processing apparatus provided in the embodiment of the present application includes:

a receiving module, configured to receive a manifest file of a content stream;

a notification module for notifying an application of at least one event schema in a timed metadata track of the content stream according to the received manifest file;

the receiving module is further configured to receive a subscription request for an event schema of the at least one event schema from the application; and

a sending module to deliver an event message associated with the event schema in the timed metadata track to the application according to the received subscription request.

The information processing method provided by the embodiment of the application comprises the following steps: receiving a manifest file for a content stream; notifying an application of at least one event schema in a timed metadata track of the content stream in accordance with the received manifest file; receiving a subscription request for one of the at least one event schema from the application; and communicating to the application, in accordance with the received subscription request, an event message associated with the event schema in the timed metadata track.

The information providing method provided by the embodiment of the application comprises the following steps: generating a timed metadata track for a content stream, the timed metadata track comprising event messages associated with at least one event schema; generating a manifest file identifying at least one of the one or more event schema in the timed metadata track; and providing the manifest file to an information processing apparatus.

The electronic device provided by the embodiment of the application comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein when the processor executes the program, the method provided by the embodiment of the application is realized.

A non-volatile computer-readable storage medium provided in an embodiment of the present application is characterized by including: the storage medium stores a plurality of instructions that can cause at least one processor to perform the methods of the embodiments of the present application.

According to the technical scheme provided by the embodiment of the application, the application can be informed of the event scheme in the timing metadata track according to the manifest file of the content stream, and the event message associated with the event scheme in the timing metadata track is sent to the application according to the subscription request for the event scheme in the timing metadata track sent by the application, so that the problem that the MPEG-DASH does not provide signaling for the timing metadata track is solved, the information carried in the timing metadata track is disclosed to the application, namely, the media data segment and the information required for reproducing the multimedia content are provided for the application, and the efficiency of reproducing the multimedia by the application is improved.

Drawings

Other features, properties, and various advantages of the disclosed subject matter will be more apparent from the following detailed description and the accompanying drawings, in which:

FIG. 1 illustrates a system according to an embodiment of the present disclosure;

fig. 2 illustrates a dynamic adaptive streaming over HTTP (DASH) system in accordance with an embodiment of the present disclosure;

fig. 3 illustrates a DASH client architecture in accordance with an embodiment of the present disclosure;

FIG. 4 illustrates an embedded event message box in a timed metadata track according to an embodiment of the present disclosure;

FIG. 5 illustrates exemplary semantics of an embedded event stream according to an embodiment of the present disclosure;

FIGS. 6A and 6B illustrate exemplary common attributes and elements according to embodiments of the present disclosure;

fig. 7 illustrates an exemplary method of an information processing apparatus according to an embodiment of the present disclosure;

fig. 8 illustrates an exemplary method of an information providing apparatus according to an embodiment of the present disclosure;

fig. 9 illustrates a computer system according to some embodiments of the present disclosure.

Detailed Description

For a more complete understanding of this disclosure, reference is now made to the following brief description, taken in connection with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.

The present disclosure includes embodiments for signaling and processing of event information, such as event information included in a timing metadata track. The event information corresponds to media timed events in a media presentation (e.g., a continuous audio and/or video presentation) associated with a point in time or time period. For example, event information may be used for dynamic content replacement, advertisement insertion, presenting supplemental content with audio and/or video, changing web pages, and executing application code triggered at a particular point on a media timeline of a media presentation (e.g., an audio and/or video media stream).

Media timing events may be used to carry information intended to be synchronized with a media stream. For example, event information may include metadata (or timing metadata) describing the content of the media presentation, such as program or chapter titles, or geographical location information. Further, the event information may include control messages of the media player associated with a particular time during playback of the media presentation, such as an ad insertion cue.

A Media Presentation Description (MPD) event is an event that may be signaled in the MPD. In MPDs, the sequence of events assigned to a media presentation time is provided at a periodic level. Events of the same type may be specified by an event stream element (e.g., EventStream) in the period element. An event terminates at the end of a cycle even if the start time is after the cycle boundary or the duration of the event extends beyond the cycle boundary. The event stream element includes message scheme identification information (e.g., @ schemeIdUri) and an optional value (e.g., @ value) of the event stream element. Further, since the event stream contains timed events, a timestamp attribute (e.g., @ timestamp) may be provided to assign the event to a particular media presentation time within the period. The timed event itself may be described by an event element included in the event stream element.

The in-band event stream may be multiplexed with a Representation (replication) by adding the event message as part of the media segment. The event stream may be present in the selected representation, only in one or a few selected adaptation sets, or in all representations. For example, one possible configuration is that only the audio adaptation set contains inband events. The in-band event stream presented in the representation may be indicated by an in-band event stream element (e.g., InBandEventStream) at the adaptation set or representation level. Further, a representation may contain a plurality of in-band event streams, each indicated by a separate in-band event stream element.

Embodiments of the present disclosure may be implemented in MPEG-DASH. The timed metadata track may include an embedded event message box (emsg) that carries event information. The timed metadata track may be used to carry information similar to the MPD events and in-band events (Inband events) described above. The event information may include metadata that is time-synchronized with information provided in other tracks. Due to this synchronization, metadata in the timed metadata tracks may be provided irregularly or non-continuously, and such timed metadata tracks may be referred to as sparse timed metadata tracks.

DASH clients typically ignore event message boxes with scheme identifiers that are not defined in the manifest (e.g., MPD). If the timed metadata track carries an event message box and is part of the track, the event schema information for the event message box needs to be included in the MPD. The present disclosure includes embodiments that provide signaling for timing metadata tracks. A DASH client may discover the event message boxes and provide (for example) event schema information to the application based on signaling of the timing metadata track. The application may then subscribe to the event schema of interest based on the event schema information. The DASH client may parse the timed metadata track and dispatch event information based on a subscribed event scheme.

Fig. 1 shows a system (100) according to an embodiment of the present disclosure; the system (100) includes a content server (110) and an information processing apparatus (120). A content server (110) may provide a content stream including primary content (e.g., a main program) and at least one timed metadata track.

The information processing apparatus (120) may interface with the content server (110). For example, the information processing apparatus (120) can play back content received from the content server (110). Playback of content may be performed according to a manifest file (e.g., MPD) received by the information processing apparatus (120) (e.g., from the content server (110)). The manifest file may further comprise signaling for at least one timed metadata track.

Fig. 2 illustrates an exemplary DASH system. The DASH system (200) may include a content server (210), an advertisement server (220), and an information processing device (230) that are connected to a network (250). The DASH system (200) may also include at least one supplemental content server.

The content server (210) may provide the information processing apparatus (230) with main content (e.g., a main program) and a manifest file (e.g., an MPD). The manifest file may be generated by, for example, an MPD generator (214). In other embodiments, the primary content and manifest file may be provided by different servers.

The information processing device (230) receives the MPD and is capable of acquiring main content from an HTTP server (212) of the content server (210) on the basis of the MPD. The MPD may be processed by a DASH client (232) executing on an information processing device (230). Further, the DASH client (232) may obtain advertising content from the advertising server (220) or other content (e.g., interactive content) from at least one supplemental content server. The primary content and the advertising content may be processed by a DASH client (232) and output for display on a display device (236). The display device (236) may be integrated in the information processing apparatus (230) or external thereto. Further, the DASH client (232) may extract event information from the at least one timed metadata track and send the extracted event information to an application (234) for further processing. The application (234) may be used to display supplemental content, for example, based on event information.

The ad server (220) may store the ad content in an ad storage (e.g., memory). For example, the information processing apparatus (230) may request the stored advertisement content according to the event information.

Fig. 3 illustrates a DASH client architecture in accordance with an embodiment of the present disclosure. A DASH client (or DASH player) may be configured to communicate with an application (390) and handle various types of events, including (i) MPD events, (ii) in-band events, and (iii) timing metadata events.

The manifest parser (305) parses a manifest (e.g., MPD). The manifest is provided, for example, by a content server (110, 210). The manifest parser (305) extracts event information about MPD events, in-band events, and timing metadata events embedded in timing metadata tracks. The extracted event information is provided to DASH logic (310) (e.g., DASH player control, selection, and heuristics logic). DASH logic (310) may notify an application (390) of the event scheme signaled in the manifest according to the event information.

The event information may include event scheme information for distinguishing different event streams. The application (390) may subscribe to an event schema of interest using the event schema information. The application (390) may further indicate a desired assignment pattern for each subscribed schema through at least one subscription API. For example, the application (390) may send a subscription request to the DASH client identifying at least one event schema of interest and any desired corresponding dispatch pattern.

If an application (390) subscribes to at least one event schema that is communicated as part of at least one timed metadata track, an in-band event and "moof" parser (325) may stream the at least one timed metadata track to the timed metadata track parser (330). For example, the in-band event and "moof" parser (325) parses Movie Fragment Box (moof), then parses the timed metadata track based on control information from DASH logic (310).

The timed metadata track parser (330) may extract event messages embedded in the timed metadata track. The extracted event messages may be stored in an event buffer (335) (e.g., an event buffer). A synchronizer/dispatcher module (340) (e.g., an event and timing metadata synchronizer and dispatcher) can dispatch (or send) the subscribed events to an application (390).

MPD events described in the MPD may be parsed by the manifest parser (305) and stored in the buffer (335). For example, the manifest parser (305) parses each event stream element of the MPD and parses each event described in each event stream element. For each event signaled in the MPD, event information (e.g., presentation time and event duration) may be stored in association with the event in a buffer (335).

An in-band event and 'moof' parser (325) may parse the media segments to extract in-band event messages. Any such identified in-band events and associated presentation times and durations may be stored in a buffer (335).

Accordingly, MPD events, in-band events, and/or timing metadata events may be stored in the buffer (335). The buffer (335) may be, for example, a First-In-First-Out (FIFO) buffer. The buffer (335) may be managed in concert with the media buffer (350). For example, any event or timing metadata corresponding to a media segment may be stored in the buffer (335) as long as the media segment is present in the media buffer (350).

A DASH Access Application Programming Interface (API) (315) may manage the acquisition and reception of content streams (or data streams) including media content and various metadata through an HTTP protocol stack (320). The DASH access API (315) may separate the received content stream into different data streams. The data stream provided to the in-band event and moof parser may include a media fragment, at least one timing metadata track, and in-band event signaling included in the media fragment. In an embodiment, the data stream provided to manifest parser 305 may comprise an MPD.

The DASH access API (315) may forward the manifest to the manifest parser (305). In addition to describing events, the manifest may also provide information about the media segments to DASH logic (310), which may communicate with the application (390) and the in-band events and moof parser (325). The application (390) may be associated with media content handled by a DASH client. Control/synchronization signals exchanged between the application (390), DASH logic (310), manifest parser (305), and DASH access API (315) may control the fetching of media segments from the HTTP stack (320) according to information about the media segments provided in the manifest.

The in-band event and moof parser (325) may parse the media data stream into media segments that include media content, timing metadata in the timing metadata track, and any signaled in-band events in the media segments. Media segments comprising media content may be parsed by a file format parser (345) and stored in a media buffer (350).

Events stored in the buffer (335) may allow the synchronizer/dispatcher (340) to communicate available events (or events of interest) related to the application through the event/metadata API. The application may be configured to process available events (e.g., MPD events, in-band events, or timing metadata events) and subscribe to specific events or timing metadata through a notification synchronizer/dispatcher (340). Any events stored in the buffer (335) that are not related to an application, but are related to the DASH client itself, may be forwarded by the synchronizer/dispatcher (340) to the DASH logic (310) for further processing.

In response to an application (390) subscribing to a particular event, a synchronizer/dispatcher (340) can communicate an event instance (or timing metadata sample) to the application, the event instance corresponding to an event schema to which the application subscribes. The event instances can be delivered according to a dispatch mode (e.g., for a particular event scenario) or a default dispatch mode indicated by the subscription request. For example, in an on-receive dispatch mode, an event instance may be sent to an application (390) once received in a buffer (335). On the other hand, in a start dispatch mode, an event instance may be sent to an application (390) at a presentation time associated with the event instance, e.g., in synchronization with a timing signal from a media decoder (355).

The MPD describes a DASH media presentation and may be in the form of an Extensible Markup Language (XML) document. The MPD may include a sequence of at least one period. Each cycle may include at least one adaptation set. Each adaptation set may include at least one representation. Further, each representation may include at least one segment. The MPD may define a format to inform an HTTP Uniform Resource Locator (URL) or a network address for downloading the data segments.

The media presentation includes at least one period. The period may be defined by a period element in the MPD element. In some cases, the MPD is provided such that the next period is a continuation of content in a previous period, possibly an immediately next period, or a later period (e.g., after an advertisement period is inserted).

An adaptation set may be described by an adaptation set element (e.g., AdaptationSet). The adaptation set element may be included in the period element. The adaptation set may include alternative representations (i.e., only one representation in the adaptation set is desired to be presented at a time). All representations contained in one adaptation set may represent the same media content component and thus contain media streams that are considered to be perceptually equivalent. The adaptation sets and the included representations may contain sufficient information to enable seamless switching between different representations in one adaptation set.

A representation may be described by a representation element. As described above, the presentation element is contained in the adaptation element. The representation may be one of a complete set or a subset of media content components (including media content) of the defined period. The representation may begin at the beginning of the period (e.g., period start) and continue until the end of the period (i.e., the beginning of the next period or the end of the media presentation). Each representation may comprise at least one media stream, wherein each media stream is an encoded version of a media content component.

Each representation may include at least one segment. In one embodiment, the segment may be referenced by an HTTP-URL included in the MPD. For example, according to IETF RFC 3986, the HTTP-URL may be defined as an < absolute URI >, with a fixed scheme of "HTTP" or "HTTPs", possibly limited by byte range if a range attribute is provided with the web address. The byte range may be expressed as, for example, a byte-range-spec defined in the IET FRFC 2616. It may be limited to a single expression identifying a contiguous range of bytes. In an embodiment, the segment may be included in an MPD with a data URL, such as defined in IETF RFC (2397).

Each segment referenced by an HTTP-URL in the MPD may be associated with a segment availability interval (e.g., a time window in wall clock time that the segment may access via the HTTP-URL). The presentation may be assigned segment information by the presence of an element (e.g., a base URL, a segment base, a segment template, and/or a segment list). The segment information may provide information about the location availability and attributes of all segments contained in one representation.

The sub-representation may be embedded (or contained) in the regular representation and described by sub-representation elements (e.g., sub-representation). The sub-representation elements may describe attributes of one or more media content components embedded in the representation. For example, a sub-representation element may describe an embedded audio component (e.g., codec, sampling rate, etc.), attributes of an embedded subtitle (e.g., codec), or a sub-representation element may describe some embedded lower quality video layers (e.g., some lower frame rates, etc.). The child presentation element and the presentation element may share some common attributes and elements.

Fig. 4 shows an example of a media track and a timing metadata track that may be included in a content stream. The metadata track may include an event message box (emsg). The event message box may provide signaling for general events related to media presentation time as well as for events specific to DASH operations. In some embodiments, if the DASH client detects that the event message box has a scheme not defined in the MPD, the DASH client ignores the event message box.

The event message box may include message scheme identification information (e.g., scheme _ id _ uri), optional values of events contained in the event message box, timing information, and event data. The timing information may indicate a time scale (e.g., in number of ticks per second) of other time information, such as a media presentation time delta from a media presentation time of the event at a reference presentation time (e.g., the beginning of a segment or metadata sample), a media presentation time of the event, and an event duration (e.g., in media presentation time).

The metadata track may carry an event message box as part of a metadata sample included in the metadata track. Further, the metadata sample may include at least one event message box. Each event message box may belong to an event scheme defined by the scheme URI id and the value (optionally) of the respective event message box. Since event instances from multiple solutions are included in at least one metadata sample, event solutions need to be identified in the DASH manifest for DASH clients to discover these solutions.

DASH includes two elements that can be used to describe an event scheme in MPD. These two elements are the event stream element for MPD events (e.g., EventStream) and the inband event stream element for inband events (e.g., InBandEventStream). The two event scenarios may use the same configuration.

Embodiments of the present disclosure include a new MPD element that may be used to describe timed metadata events provided in a timed metadata track. The new MPD element may have the same or similar construction as the in-band event stream element. In one embodiment, the new MPD element is an embedded event stream element (or timed metadata event stream element) (e.g., EmbeddedEventStream). Exemplary semantics of embedded event stream elements are shown in fig. 5.

As shown in fig. 5, an embedded event stream element may be used to specify at least one timed metadata event (e.g., timed metadata instance/stream) in a timed metadata track. The embedded event stream element may include attributes such as reference information (e.g., @ xlink: href), processing information (e.g., @ xlink: act), message scheme identification information (e.g., @schemehduri), a value of the embedded event stream element (e.g., @value), timestamp (time scale) information (e.g., @ time), and presentation time offset information (e.g., @ presentationTimeOffset). Message scheme identification information, optionally paired with values of embedded event stream elements, may be used to identify different event schemes within the timing metadata track. For example, the message scheme identification information in the embedded event stream element is optionally paired with the value of the embedded event stream element) may be matched with the message scheme identification information (e.g., scheme _ id _ uri) (optionally paired with the value of the event) in the event message box.

A new MPD element (e.g., an embedded event stream element) for describing timing metadata events provided in the timing metadata track may be included as part of common elements and attributes in MPEG-DASH. The new MPD element may be included in a manner similar to the in-band event stream. The element adaptation sets, representations, and sub-representations may have assigned common attributes and elements. Common attributes and elements (e.g., the element CommonAttributesElements) may exist in all three elements. In some embodiments, an adaptation set element may contain default values for elements and attributes associated with the contained representation. In some embodiments, any common attribute may be present in only one of the adaptation set element and the representation element.

Exemplary semantics of common attributes and elements are provided in the tables shown in fig. 6A and 6B. In an embodiment, the "Use" column in the common attributes and elements table may be interpreted such that an attribute labeled "M" is available for representation, e.g., it exists either in the representation element or in the corresponding adaptation set element. Further, in one embodiment, there is no attribute labeled "0" in the presentation element and the adaptation set element.

FIGS. 6A and 6B illustrate exemplary tables of common adaptation sets, representations, and sub-representation attributes and elements. Similar to the in-band event stream element, an embedded event stream element may be included as part of a common attribute and element (e.g., common attribute element) of the MPD. Thus, in some embodiments, any timed metadata track carrying an embedded event has a corresponding adaptation set or representation element in the MPD. For embedded events, the MPD may include one embedded event stream element for each event schema. The DASH client may extract and process embedded events in the timed metadata track to obtain events with corresponding embedded event stream elements in the received MPD, while ignoring other events.

Fig. 7 shows an exemplary flowchart of an information processing method (700) that may be performed by the information processing apparatus (120). The method (700) begins at step (S710) where, at step S710, an information processing device (120) (e.g., from a content server (110)) receives a manifest file (e.g., MPD) for a content stream.

In some embodiments, the manifest file comprises a media presentation description, MPD.

In some embodiments, the media presentation description, MPD, comprises at least one element for identifying the at least one event scheme in the timed metadata track.

In step (S720), the information processing apparatus (120) notifies the application of at least one event scheme in the timed metadata track of the content stream according to the received manifest file. For example, the manifest file includes embedded event stream elements for each of the at least one event schema. The information processing apparatus (120) identifies at least one event schema from each embedded event stream element in the manifest file.

At step (S730), the information processing apparatus (120) receives a subscription request for an event schema of at least one event schema from an application. The subscription request may identify at least one event schema of interest to the application. The subscription request may further identify (e.g., through a subscription API) a subset of the at least one schema or a desired assignment pattern for each of the at least one schema.

At step (S740), the information processing device (120) delivers (or dispatches) to the application, in accordance with the received subscription request, an event message associated with the event schema in the timed metadata track. For example, a DASH client retrieves event messages associated with at least one subscription event schema and sends the retrieved event messages to an application. Event messages are dispatched according to a dispatch mode and/or a default dispatch mode indicated by the received subscription request.

In some embodiments, event messages associated with the event schema are dispatched to the application according to a dispatch mode indicated by the subscription request.

In some embodiments, the at least one element in the media presentation description, MPD, for identifying the at least one event scheme comprises a uniform resource identifier, URI, of the event scheme.

In some embodiments, the element of the at least one element in the media presentation description, MPD, indicates a value of the element, and the URI and the value of the element are used to identify the event scheme.

In some embodiments, the at least one event scheme comprises a plurality of event schemes, and the at least one element in the media presentation description, MPD, identifies only a subset of the plurality of event schemes.

In some embodiments, the at least one element in the media presentation description MPD is included in common attributes and elements of the media presentation description MPD.

Fig. 8 shows an exemplary flowchart of an information providing method (800) performed by an information providing apparatus, such as a content server (110, 210).

The method (800) begins at step (S810) and at step S810, the information providing device generates a timed metadata track for the content stream. In other examples, multiple timed metadata tracks may be generated. The timing metadata track includes event messages associated with at least one event schema. Each event schema may be associated with different schema identification information (e.g., @ schemeIdUri) or with different paired schema identification information (e.g., @ schemeIdUri) and values (e.g., @value).

At step (S820), a manifest file (e.g., MPD) is generated for the content stream. The manifest file is used to identify at least one event schema of the one or more event schemas in the timed metadata track. For example, the manifest file includes an embedded event stream element (e.g., EmbeddedEventStream) for each event schema included in the timed metadata track. In some embodiments, the manifest file may not identify each event schema included in the timed metadata track. For example, the manifest file may include only the timed metadata event elements of certain event schemas.

Further, in some embodiments, a plurality of manifest files are generated, each of which identifies a different event schema in the timed metadata track. In particular, a plurality of different manifest files may be generated, with different manifest files identifying different event scenarios (e.g., different individual event scenarios or combinations of event scenarios). Different manifest files may be provided to different end users for custom or purposeful targeting.

At step (S830), the manifest file is supplied to the information processing apparatus. In some embodiments, multiple different manifest files may be provided to different information processing devices or the same information processing device (e.g., for user selection).

The proposed methods can be used alone or in any order in combination. Further, each of the method or embodiment, the information processing apparatus, the information providing apparatus, and the server may be implemented by a processing circuit (e.g., at least one processor or at least one integrated circuit). In one example, at least one processor executes a program stored in a non-transitory computer readable medium.

An embodiment of the present application further provides an information processing apparatus corresponding to the information processing method, where the apparatus includes:

a receiving module, configured to receive a manifest file of a content stream;

In some embodiments, the sending module delivers an event message associated with the event schema to the application according to a dispatch mode indicated by the subscription request.

The embodiment of the present application further provides an information providing device corresponding to the information providing method, where the device includes:

a generation module to generate a timed metadata track for a content stream, the timed metadata track comprising event messages associated with at least one event schema;

the generating module is further configured to generate a manifest file identifying at least one of the one or more event schema in the timed metadata track; and

and the providing module is used for providing the manifest file to an information processing device.

In some embodiments, the generation module generates a plurality of manifest files, each of the manifest files identifying a different event schema in the timed metadata track.

The techniques described above may be implemented as computer software via computer readable instructions and physically stored in one or more computer readable media. For example, fig. 9 illustrates a computer system (900) suitable for implementing certain embodiments of the disclosed subject matter.

The computer software may be encoded in any suitable machine code or computer language, and by assembly, compilation, linking, etc., mechanisms create code that includes instructions that are directly executable by one or more computer Central Processing Units (CPUs), Graphics Processing Units (GPUs), etc., or by way of transcoding, microcode, etc.

The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablets, servers, smartphones, gaming devices, internet of things devices, and so forth.

The components illustrated in FIG. 9 for the computer system (900) are exemplary in nature and are not intended to limit the scope of use or functionality of the computer software implementing embodiments of the present application in any way. Neither should the configuration of components be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiments of the computer system (900).

The computer system (900) may include some human interface input devices. Such human interface input devices may respond to input from one or more human users through tactile input (e.g., keyboard input, swipe, data glove movement), audio input (e.g., sound, applause), visual input (e.g., gestures), olfactory input (not shown). The human-machine interface device may also be used to capture media that does not necessarily directly relate to human conscious input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from still-image cameras), video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

The human interface input device may include one or more of the following (only one of which is depicted): keyboard (901), mouse (902), touch pad (903), touch screen (910), data glove (not shown), joystick (905), microphone (906), scanner (907), camera (908).

The computer system (900) may also include certain human interface output devices. Such human interface output devices may stimulate the senses of one or more human users through, for example, tactile outputs, sounds, light, and olfactory/gustatory sensations. Such human interface output devices may include tactile output devices (e.g., tactile feedback through a touch screen (910), data glove (not shown), or joystick (905), but there may also be tactile feedback devices that do not act as input devices), audio output devices (e.g., speakers (909), headphones (not shown)), visual output devices (e.g., screens (910) including cathode ray tube screens, liquid crystal screens, plasma screens, organic light emitting diode screens, each with or without touch screen input functionality, each with or without haptic feedback functionality-some of which may output two-dimensional visual output or more than three-dimensional output by means such as stereoscopic picture output; virtual reality glasses (not shown), holographic displays and smoke boxes (not shown)), and printers (not shown).

The computer system (900) may also include human-accessible storage devices and their associated media, such as optical media including compact disc read-only/rewritable (CD/DVD ROM/RW) (920) or similar media (921) with CD/DVD, thumb drive (922), removable hard drive or solid state drive (923), conventional magnetic media such as magnetic tape and floppy disk (not shown), ROM/ASIC/PLD based proprietary devices such as secure dongle (not shown), and the like.

Those skilled in the art will also appreciate that the term "computer-readable medium" used in connection with the disclosed subject matter does not include transmission media, carrier waves, or other transitory signals.

The computer system (900) may also include an interface to one or more communication networks. For example, the network may be wireless, wired, optical. The network may also be a local area network, a wide area network, a metropolitan area network, a vehicular network, an industrial network, a real-time network, a delay tolerant network, and so forth. The network also includes ethernet, wireless local area networks, local area networks such as cellular networks (GSM, 3G, 4G, 5G, LTE, etc.), television wired or wireless wide area digital networks (including cable, satellite, and terrestrial broadcast television), automotive and industrial networks (including CANBus), and so forth. Some networks typically require external network interface adapters for connecting to some general purpose data port or peripheral bus (949) (e.g., a USB port of computer system (900)); other systems are typically integrated into the core of the computer system (900) by connecting to a system bus as described below (e.g., an ethernet interface to a PC computer system or a cellular network interface to a smart phone computer system). Using any of these networks, the computer system (900) may communicate with other entities. The communication may be unidirectional, for reception only (e.g., wireless television), unidirectional for transmission only (e.g., CAN bus to certain CAN bus devices), or bidirectional, for example, to other computer systems over a local or wide area digital network. Each of the networks and network interfaces described above may use certain protocols and protocol stacks.

The human interface device, human accessible storage device, and network interface described above may be connected to the core (940) of the computer system (900).

The core (940) may include one or more Central Processing Units (CPUs) (941), Graphics Processing Units (GPUs) (942), special purpose programmable processing units in the form of Field Programmable Gate Arrays (FPGAs) (943), hardware accelerators (944) for specific tasks, and so forth. These devices, as well as Read Only Memory (ROM) (945), random access memory (946), internal mass storage (e.g., internal non-user accessible hard drives, solid state disks, etc.) (947), etc., may be connected by a system bus (948). In some computer systems, the system bus (948) may be accessed in the form of one or more physical plugs, so as to be extensible through additional central processing units, graphics processing units, and the like. The peripheral devices may be attached directly to the system bus (948) of the core or connected through a peripheral bus (949). The architecture of the peripheral bus includes peripheral controller interface PCI, universal serial bus USB, etc.

The CPU (941), GPU (942), FPGA (943) and accelerator (944) may execute certain instructions, which in combination may constitute the computer code described above. The computer code may be stored in ROM (945) or RAM (946). Transitional data may also be stored in RAM (946) while persistent data may be stored in, for example, internal mass storage (947). Fast storage and retrieval of any memory device may be achieved through the use of cache memory, which may be closely associated with one or more CPUs (941), GPUs (942), mass storage (947), ROMs (945), RAMs (946), and so on.

The computer-readable medium may have computer code thereon for performing various computer-implemented operations. The media and computer code may be those specially designed and constructed for the purposes of the present application, or they may be of the kind well known and available to those having skill in the computer software arts.

By way of example, and not limitation, a computer system having an architecture (900), and in particular a core (940), may provide functionality as a processor (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media may be media associated with the user-accessible mass storage described above, as well as certain memory having a non-volatile core (940), such as core internal mass storage (947) or ROM (945). Software implementing various embodiments of the present application may be stored in such devices and executed by the core (940). The computer-readable medium may include one or more memory devices or chips, according to particular needs. The software may cause the core (940), and in particular the processors therein (including CPUs, GPUs, FPGAs, etc.), to perform certain processes or certain portions of certain processes described herein, including defining data structures stored in RAM (946) and modifying such data structures according to software defined processes. Additionally or alternatively, the computer system may provide functionality that is logically hardwired or otherwise embodied in circuitry (e.g., accelerator (944)) that may operate in place of or in conjunction with software to perform certain processes or certain portions of certain processes described herein. Where appropriate, reference to software may include logic and vice versa. Where appropriate, reference to a computer-readable medium may include circuitry (e.g., an Integrated Circuit (IC)) storing executable software, circuitry comprising executable logic, or both. The present application includes any suitable combination of hardware and software.

Claims

1. An information processing apparatus characterized by comprising:

a receiving module, configured to receive a manifest file of a content stream;

2. The information processing apparatus according to claim 1, wherein the manifest file includes a Media Presentation Description (MPD).

3. The information processing apparatus of claim 2, wherein the Media Presentation Description (MPD) comprises at least one element for identifying the at least one event scheme in the timed metadata track.

4. The information processing apparatus of claim 3, wherein the sending module dispatches the event message associated with the event schema to the application according to a dispatch mode indicated by the subscription request.

5. The information processing apparatus of claim 3, wherein the at least one element in the Media Presentation Description (MPD) for identifying the at least one event scheme comprises a Uniform Resource Identifier (URI) of the event scheme.

6. The information processing apparatus according to claim 5, characterized in that:

the element of the at least one element in the Media Presentation Description (MPD) indicates a value of the element, and

the URI and the value of the element are used to identify the event schema.

7. The information processing apparatus according to claim 3, wherein:

the at least one event schema includes a plurality of event schemas, and

the at least one element in the media presentation description, MPD, identifies only a subset of the plurality of event schemes.

8. The information processing apparatus according to claim 3, wherein the at least one element in the Media Presentation Description (MPD) is included in common attributes and elements of the MPD.

9. An information processing method, characterized in that the method comprises:

receiving a manifest file for a content stream;

notifying an application of at least one event schema in a timed metadata track of the content stream in accordance with the received manifest file;

receiving a subscription request for one of the at least one event schema from the application; and

communicating to the application, in accordance with the received subscription request, an event message associated with the event schema in the timed metadata track.

10. The method of claim 9, wherein the manifest file comprises a Media Presentation Description (MPD).

11. The method of claim 10, wherein the Media Presentation Description (MPD) comprises at least one element, and wherein the at least one element is used to identify the at least one event scheme in the timed metadata track.

12. The method of claim 11, wherein the delivering to the application, in accordance with the received subscription request, an event message associated with the event schema in the timed metadata track comprises:

dispatching an event message associated with the event schema to the application according to a dispatch mode indicated by the subscription request.

13. The method of claim 11, wherein the at least one element in the Media Presentation Description (MPD) that identifies the event scheme comprises a Uniform Resource Identifier (URI) of the event scheme.

14. The method of claim 13, wherein:

the element of at least one element in the media presentation description, MPD, indicates a value of the element, and

the URI and the value of the element are used to identify the event schema.

15. The method of claim 11, wherein:

the at least one event schema includes a plurality of event schemas, and

16. The method of claim 11, wherein the at least one element in the Media Presentation Description (MPD) is included in common attributes and elements of the MPD.

17. An information providing method, characterized in that the method comprises:

generating a timed metadata track for a content stream, the timed metadata track comprising event messages associated with at least one event schema;

generating a manifest file identifying at least one of the one or more event schema in the timed metadata track; and

the manifest file is provided to an information processing apparatus.

18. The method of claim 17, wherein generating the manifest file comprises:

a plurality of manifest files are generated, each of the manifest files identifying a different event schema in the timed metadata track.

19. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any of claims 9 to 18 when executing the program.

20. A non-transitory computer-readable storage medium, comprising: the storage medium stores a plurality of instructions that cause at least one processor to perform the method of any of claims 9-18.