WO2016127862A1 - 一种关联多媒体内容个性化呈现的实现方法及应用 - Google Patents

一种关联多媒体内容个性化呈现的实现方法及应用 Download PDF

Info

Publication number
WO2016127862A1
WO2016127862A1 PCT/CN2016/073167 CN2016073167W WO2016127862A1 WO 2016127862 A1 WO2016127862 A1 WO 2016127862A1 CN 2016073167 W CN2016073167 W CN 2016073167W WO 2016127862 A1 WO2016127862 A1 WO 2016127862A1
Authority
WO
WIPO (PCT)
Prior art keywords
content
information
mpu
media
multimedia
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/073167
Other languages
English (en)
French (fr)
Inventor
徐异凌
张文军
朱文婕
张良慧
陈浩
李博
孙军
柳宁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiao Tong University
Original Assignee
Shanghai Jiao Tong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201510080580.4A external-priority patent/CN105992061B/zh
Priority claimed from CN201510080011.XA external-priority patent/CN105988369B/zh
Priority claimed from CN201510401550.9A external-priority patent/CN106341740B/zh
Priority claimed from CN201510955611.6A external-priority patent/CN106899866B/zh
Priority claimed from CN201610031034.6A external-priority patent/CN106982376B/zh
Priority to JP2017541331A priority Critical patent/JP6567064B2/ja
Priority to US15/550,789 priority patent/US10433029B2/en
Priority to KR1020197003889A priority patent/KR102378107B1/ko
Application filed by Shanghai Jiao Tong University filed Critical Shanghai Jiao Tong University
Priority to CA3004644A priority patent/CA3004644C/en
Priority to KR1020177024204A priority patent/KR101988454B1/ko
Publication of WO2016127862A1 publication Critical patent/WO2016127862A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8549—Creating video summaries, e.g. movie trailer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/84—Generation or processing of descriptive data, e.g. content descriptors
    • H04N21/8402—Generation or processing of descriptive data, e.g. content descriptors involving a version number, e.g. version number of EPG data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/23439—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements for generating different versions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • H04N21/2355—Processing of additional data, e.g. scrambling of additional data or processing content descriptors involving reformatting operations of additional data, e.g. HTML pages
    • H04N21/2358—Processing of additional data, e.g. scrambling of additional data or processing content descriptors involving reformatting operations of additional data, e.g. HTML pages for generating different versions, e.g. for different recipient devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/61—Network physical structure; Signal processing
    • H04N21/6106—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network
    • H04N21/6125—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network involving transmission via Internet
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/84—Generation or processing of descriptive data, e.g. content descriptors
    • H04N21/8405—Generation or processing of descriptive data, e.g. content descriptors represented by keywords
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8451—Structuring of content, e.g. decomposing content into time segments using Advanced Video Coding [AVC]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8543—Content authoring using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML]

Definitions

  • the invention relates to a novel multimedia organization, storage and transmission method in an Internet (on-demand) system, and more particularly to an implementation method and application of an associated multimedia content personalized presentation.
  • multimedia content in the Internet has exploded, especially the high-definition, high-resolution video services, and the pressure brought by multimedia services to network bandwidth has become more prominent.
  • the user's personalized interest and the time-varying network conditions, today's multimedia services are characterized by aggregative features, and multimedia content is fragmented.
  • Most of the existing multimedia protocols only focus on the integrity and security of data during network transmission, and ensure the quality of multimedia services by guaranteeing the quality of the network (QoS).
  • QoS quality of the network
  • the integration of multimedia content is rarely considered. Subjective satisfaction is also considered very little.
  • some multimedia protocols basically describe and control the transmission and presentation of data through signaling information, but do not involve the organization level of multimedia content, and do not consider the matching of content organization and storage, transmission, and data optimization of the entire multimedia system.
  • the multimedia producer has produced a complete program with uniform content, after post-production such as editing, the program publisher and provider divide the original program into different versions, and each version is independent of each other. A separate media content. This surface seems to enrich the content of the program, which actually destroys the ecological environment of the entire media system and generates a lot of duplicate data.
  • the richness of media resources also increases the burden on users to consume digital media to a certain extent, and users must concentrate more on the choice of media consumption.
  • the traditional method is that the content provider provides rich media programs such as various types of movie resources, but these programs are generally full version, and users generally can only order some complete version of the media program for consumption. In this case, if the user wants to choose to watch the program they like, they need to enter multiple programs. Line preview, and the inefficiency of data buffering based on this mechanism requires users to wait for a long time, and the user experience is very bad.
  • a better approach is that the program issuer and provider only logically split the original program into different versions. Due to the widespread use of the media content fragmentation unit, the original program is still physically unified as a whole. Only the full version of the storage medium needs to be stored on the media content server, and corresponding signaling files are provided to describe the data units corresponding to different media versions.
  • the user orders the media data of the relevant version according to the needs of the user, and the server only transmits the corresponding data to the user according to the mapping of the media version and the data unit in the signaling information. For example, if the user only wants to preview a movie, he only needs to click on the link of the preview version. The server only transmits the preview version of the movie data unit to the client for decoding and rendering.
  • the user does not need to wait for the buffer of the entire movie content to get better.
  • the difficulty lies in: the system is difficult to know the correspondence between media units between different versions, and lacks relevant signaling mechanism guidance; each medium may have multiple versions of different lengths, how to distinguish and manage these versions, It is also an urgent problem to be solved.
  • the server transmits the mapping table of the multimedia content and the media description to the user, that is, the option of providing the user with the associated multimedia content. Users can request different versions of the same content according to their own needs, or request multiple presentation forms such as associated content of a certain media resource, and personalize according to the needs of the user.
  • This multimedia content personalization presentation mechanism puts forward a new requirement. Since the associated media resources are represented by different edit lists, different edit lists contain different media data units. And for different multimedia content, users also have different choice angles. For example, for a party, the user can choose to watch multiple angle videos or watch different singers' performances. Therefore, the edit list can also be personalized according to user needs and program types. To implement personalized transmission, a corresponding mechanism is needed to transmit the mapping relationship between the description information of the multimedia content and the edit list to the client that accepts the media resource.
  • the object of the present invention is to provide an implementation method for personalizing personalized multimedia content, and a flexible and efficient scalable organization, storage and transmission method in an Internet on-demand system, thereby adding existing multimedia.
  • the flexible organization mechanism of media content lacking in the protocol solves the problem of low transmission efficiency, waste of storage resources and poor user experience of the existing multimedia system.
  • a method for implementing personalized presentation of associated multimedia content is provided.
  • the method is specifically: the multimedia content provider uniformly completes the complete file content when creating the multimedia file, and uses the marking method according to the content.
  • the importance and/or degree of association ranks the segments to produce different versions of the multimedia Body files; users selectively preview and/or play based on these versions.
  • the method specifically includes the following steps:
  • the first step is to divide the multimedia file into multiple versions through version management of the content rating
  • a second step, generating a version-related differentiated MPD file generating an MPD (Media Presentation Description) file corresponding to the content rating, that is, a version-level version management label according to the plurality of versions of the multimedia file;
  • MPD Media Presentation Description
  • the user requests different versions of the multimedia file content according to the needs of the user, and the server transmits the unified MPD file.
  • the client parses the MPD, the client requests the corresponding media segment according to the network state, the device performance, and the requested version. Slice content.
  • the method for implementing the personalized presentation of the associated multimedia content includes the following steps:
  • the first step is to divide the multimedia file into multiple versions through version management of the content rating
  • the segment description of the MPD file adds the @scaleID attribute, that is, the version management label of the content rating
  • the user requests different versions of the multimedia file content according to the needs of the user, and the server transmits the unified MPD file.
  • the client parses the MPD, the client requests the corresponding media segment according to the network state, the device performance, and the requested version. Slice content.
  • the implementation method for the personalized presentation of the associated multimedia content according to the present invention can reduce the mechanism of flexible management of the media content in the existing multimedia protocol, resulting in low transmission efficiency, waste of storage resources, and poor user experience in the multimedia system, by adding media.
  • the scalable and flexible organization, storage and transmission method of content realizes the optimization and integration of media content on the Internet, logically stores the same media resource uniformly, performs differential transmission according to the media version requirement of the user, and performs specific media unit buffering on the client side.
  • decoding presentation not only the unified optimization of rich media resources, but also facilitate the supervision and control of multimedia content, improve the overall transmission efficiency of multimedia data in the network, reduce the waste of storage space, and bring a better service experience to users.
  • the present invention provides a method for implementing the multimedia content grading technology: multimedia is produced when the multimedia file is produced. The important segments of the content are marked, and each segment is divided into different levels according to the importance of the content of the multimedia file; when the user views, the multimedia content is selectively played according to the levels, instead of blindly randomly selecting; a new MUR is defined.
  • the information file is used to describe the correspondence between the multimedia file and the different version content, mpu_seq_number, and level_list.
  • the mpu_seq_number is a tag field in the MPU
  • the level_list is a set of mpu_seq_number, which has a one-to-one correspondence with different versions of the video.
  • a new MUR information file is newly defined for describing the multimedia file and different version content, mpu_seq_number Correspondence between level_list, which is different according to the corresponding rules of the level_list array and different version content time, and is divided into two types:
  • type 1 there is a one-to-one correspondence between the content of different versions and the level_list;
  • the content of different versions is a combination of different level_lists
  • the multimedia file is differentially transmitted according to the contents of the MUR information file in Type 1 and Type 2.
  • the multimedia content grading technology blank in the existing multimedia protocol may be caused, and the problem that the media content transmission efficiency is not high and the storage space room fee is caused, by increasing the scalable transmission of the media content
  • the storage and presentation mechanism implements association classification of different versions of media content on the Internet, increases management control of multimedia services, improves transmission efficiency of multimedia data, and reduces waste of storage space. Giving users more freedom to choose the media version to enhance the user experience, but also to provide network operators with better multimedia content rating, transmission and storage solutions.
  • an implementation method for personalized presentation of the associated multimedia content is applied, and a content-driven smart home control method is provided, which is based on image main color and/or based on key frames in the video.
  • the scene plot features mark the video fragments, or tag them according to the audio data features, and add corresponding tags.
  • the labeled tags will drive the smart home device to work.
  • the video fragment is marked by the main color of the image based on the key frame in the video, specifically: extracting and marking the main color of the key frame in the video MPU, and defining the color_tag by using the reserved field reserved in the MPUBox.
  • Different color_tags can be used for different colors.
  • the smart home control system can be automatically driven by the multimedia content, and most of the current needs to manually evaluate the multimedia content and then manually control the intelligent system, through the content-driven control of the present invention.
  • the smart home system can greatly simplify the cumbersome process of artificial control, and the invention makes the control system more flexible.
  • the service provider marks the media content according to the type of the media content or the importance thereof, the combination of the media data units of the same mark is a different edit list.
  • the tag information of the media content needs to be correspondingly described, so that the user understands the specific content of each edit list, realizes differential transmission and presentation, and provides accurate service.
  • the present invention provides a method for describing the personalized presentation information of the associated multimedia content, where the method is: association of the media resource Multimedia content
  • Different edit lists indicate that the media data units included in different edit lists are supplementary relationships or inclusion relationships; in order to present the correspondence of the media resources and the corresponding relationship of the description information to the user, in the transmitted information Add a description of the media resource feature information or associated information to implement a personalized presentation mechanism.
  • the implementation steps are:
  • the client parses the descriptor signaling information, and the user selects the multimedia content with the corresponding identifier or information representation according to the requirement, and requests the corresponding media data unit to be combined into a corresponding edit list;
  • the server parses the user request and sends the corresponding media data unit, and the client parses the information and implements personalized presentation.
  • the present invention is directed to an incompleteness of a presentation mechanism in an existing multimedia protocol, a user decision-based presentation mechanism of an on-demand service, a broadcast mechanism, a push mechanism of a live broadcast service, and a presentation service of associated content.
  • a timeline control method for personalized presentation of multimedia content is provided. The method: during the presentation of the multimedia resource, using the duration information of the media data unit and the initial time of the presentation, controlling the presentation time of the associated content, so that Different media resources or different associated versions of the same media resource have corresponding rendering timelines.
  • S1 acquiring, according to the media data unit included in the associated content version of the media resource selected by the user, each media data unit thereof with corresponding duration information;
  • S2 determining a start play time according to a time when the user selects the media resource in the VoD, determining, by the service provider, a play time in the broadcast service, and identifying a start play time in the signaling information;
  • S3 Obtain an absolute presentation time of the corresponding media data unit according to the start of the play time in S2 and the duration of the media data unit in the corresponding associated content version, and accordingly maintain the corresponding presentation timeline to guide the presentation of the media content.
  • the timeline control method for personalized presentation of multimedia content ensures the smooth viewing experience of the user on the basis of saving storage space while satisfying the individualized needs of the user and utilizing the relevance of the same set of programs. Since the presentation timeline information of the media is flexibly transmitted with the signaling information, the delay and the packet loss caused by the process of consuming the media resources cause the user experience to be reduced. If the packet loss event is detected before the presentation time, the media may be awaited. Re-acquisition of resources, or repeatedly presenting the contents of the previous media data unit to prevent playback and ensure user experience. The presentation continues according to the indication of the presentation timeline, and the specific implementation scheme is not specified here.
  • the above technical solution of the present invention can be used to address the blank of the associated multimedia content information in the existing multimedia protocol, and the user cannot understand the problem of the media resource provided by the server.
  • the user has a more intuitive and multi-faceted understanding of the multimedia content, describes the personalized mapping relationship to provide associated media services, and ensures that the user selects the most suitable multimedia content as much as possible.
  • This solution is not limited to the description of the multimedia content provided by the server, but also applies to the content description of multimedia resources such as multi-source or various classification methods, and even generates descriptions through user-defined tags.
  • the invention solves the problem that the user cannot select the associated multimedia content provided by the server, and realizes the personalized media presentation for different user requirements.
  • FIG. 1 is a schematic diagram of an example of media version and media data unit mapping of Embodiments 1, 2, and 5;
  • FIG. 2 is a schematic diagram of a hierarchical mechanism-mode 1 in Embodiment 1 of the present invention.
  • FIG. 3 is a schematic diagram of a hierarchical mechanism system framework-mode 2 according to Embodiment 1 of the present invention.
  • FIG. 5 is a system diagram of controlling a hue system by using an MPU according to Embodiment 3 of the present invention.
  • FIG. 6 is a schematic diagram showing a differentiated presentation of a device with low power in Embodiment 5 of the present invention.
  • Embodiment 7 is a schematic diagram of real-time live broadcast of multiple views in Embodiment 5 of the present invention.
  • FIG. 8 is a schematic diagram of a presentation timeline of different versions of related content in Embodiment 5 of the present invention.
  • Content provider is making video
  • the complete video content is unified and the tagging method is used to classify the segments according to the importance and relevance of the content, so that different versions of the video can be generated.
  • the presentation interface should provide version information related to the ranking to facilitate the user to select content consumption.
  • Optional previews and playbacks can be made based on these versions as the user views, rather than blindly randomly selecting.
  • the content creator only needs to complete a complete set of program content, and then the issuer can divide the video into multiple versions and provide it to the media service provider through content grading version management.
  • the first version of the video is a preview version, the duration is 5 minutes, the content only includes the wonderful picture in the video; the second version of the video is a cut version, the duration is 30 minutes, the content only includes the story line and important scene of the video; the third version
  • the video is a full version, with a duration of 120 minutes, including the full storyline of the video;
  • the fourth version is an enhanced version with a duration of 150 minutes, including expanded content beyond the full storyline, such as behind-the-scenes footage.
  • the MPEG-DASH (Dynamic Adaptive Streaming over HTTP) standard is taken as an example to introduce a version management mechanism for content classification.
  • this mechanism is not limited to the DASH standard, but can be applied to other standards and solutions.
  • DASH is a full-featured HTTP-based dynamic adaptive stream, which is developed by the international standard group MPEG. It can distribute media through HTTP protocol in an adaptive, progressive, download or streaming manner, supporting multiple networks under different network conditions. Adaptive media consumption on different performance terminals.
  • DASH has multiple fragmentation strategies, and there are corresponding signaling files to indicate its fragmentation details. Considering the versatility, the following two modes are described separately.
  • an MPD Media Presentation Description
  • a file corresponding to the content rating may be generated according to the four versions in the above video program.
  • the description of the segment list in the MPD is written, and other cases such as the segment template are handled similarly.
  • the segment list corresponding to the four versions in the above video program when these segments are generated, four MPD files matching the content version are automatically generated.
  • the MPD element adds the @scaleID attribute, which defines the level of content grading described by the MPD file.
  • the content grading version correspondence table is as shown in the following table. Furthermore, the segment list in the SegmentList sub-element in the Representation element is no longer a pure full-part slice list, but a specific slice list corresponding to MPD@scaleID.
  • the framework of the entire grading mechanism is basically the same as the original DASH framework.
  • the media fragmentation mechanism remains unchanged during the media preparation phase, and four MPD files corresponding to different hierarchical versions are generated. .
  • the user requests different versions of the video content according to their own needs, and only needs the server to transmit the relevant MPD file, and the client parses and then requests the corresponding media fragment content from the server.
  • the server when the user selects to play the preview movie program, the server sends an MPD file with MPD@scaleID of 0, and the client parses and requests the fragments listed by the SegmentList sub-element in the Representation element, namely channel1init.mp4, channel11.m4s, Channel14.m4s, channel17.m4s, channel19.m4s.
  • An example of this MPD document is shown below (the new parameter is that sascleID can have different new applications):
  • the DASH segment When the DASH segment is generated, it can be distinguished according to the four content classification versions, and the indication of the differentiation is presented in the MPD table generated at the same time. For the sake of simplicity and clarity, only the description of the segment list in the MPD is written. Others are treated in the same way as the segment template.
  • each segment in the SegmentList sub-element of the Representation element of the MPD file provides a @scaleID attribute, which is labeled. Use the minimum version level of this segment.
  • the content rating version correspondence table is consistent with mode 1. If the @scaleID of segment:channel13.m4s is 2, then this segment will only be transmitted by the server and consumed by the client when the user requests the full version of the video.
  • the system framework diagram of the entire grading mechanism is basically the same as the original DASH framework.
  • the generation mechanism of the media fragment remains unchanged, and only one corresponding MPD file is generated.
  • the segment description of the MPD file adds the @scaleID attribute, which is the version management label of the content rating.
  • the DASH client the user requests different versions of video content according to their own needs, and the server transmits a unified MPD file. After parsing the MPD, the client requests the corresponding media segment according to its network state, device performance, and requested version status. Slice content.
  • the server sends a unified MPD file, and the client parses and requests the media segment of the @scaleID attribute value listed in the SegmentList sub-element of the Representation element to be less than or equal to 2, that is, channel1init.mp4. , channel11.m4s, channel12.m4s, channel13.m4s, channel14.m4s, channel15.m4s, channel16.m4s, channel17.m4s, channel18.m4s, channel19.m4s, channel110.m4s, the server sends these fragments, by the client The end is decoded and presented to the user.
  • An example of this MPD document is as follows:
  • the above embodiment is a flexible and efficient scalable organization, storage and transmission method in an Internet on-demand system, thereby increasing the flexible organization mechanism of the media content lacking in the existing multimedia protocol, and solving the low transmission efficiency of the existing multimedia system.
  • Content providers can mark important segments of video content in some way when creating a video, or further, each video segment can be divided into different levels depending on the importance of the video content. Selective playback can be performed based on these ratings as the user views, rather than blindly randomly selecting.
  • the producer can provide multiple different versions of the video.
  • the video organization structure it can be divided into video versions of different durations such as Lite, Cut, Full and Extended (see Example 1).
  • MPU media processing units
  • Mpu_seq_number is the tag field in the MPU, as shown in Figure 1, assuming that a video is divided into mpu_seq_number
  • the MPU fragments are 0 to 11 respectively, and the different mpu_seq_number sequences are combined into different versions of the video.
  • the compact video is composed of MPUs with mpu_seq_number of 1 and 9.
  • a MUR (Media Unit Relationship) information file is newly defined to describe the correspondence between media resources and different versions of video, mpu_seq_number, level_list, and the like.
  • the level_list array is different from the corresponding rules of different versions of video time, and can be divided into two types. In Type 1, there is a one-to-one correspondence between different versions of video and level_list; in Type 2, different versions of video are combinations of different level_lists.
  • the contents of the MUR information file in Type 1 and Type 2 are separately explained, and how to carry out the differential transmission, and how the client can use the mediaSrc attribute in the CI file to flexibly present different versions of the video.
  • the level_list is composed of different mpus and corresponds to different video versions.
  • the corresponding rules are shown in the following table.
  • level_list is composed of different MPUs, and there is no overlapping MPU between each level_list[].
  • Different video versions are composed of different level_lists, and the corresponding rules are shown in the following table.
  • the reduced version video consists of level_list[0]
  • the cut version video consists of level_list[0] and level_list[1]
  • level_list[2] is the difference between the full version and the cut version.
  • the extended version consists of four level_lists. So when the user wants to watch the full version of the video after watching the cut version of the video, it only needs to differentially transfer the MPU fragments contained in level_list[2].
  • the values of the meiaSrc attribute for the different versions of the video in the CI file in Type 2 are shown in the following table.
  • the MUR information file contains important information about the content rating, which is described above. How to transmit these important information, there may be different methods, such as by defining a new signaling unit, signaling file, signaling message, descriptor, or header information added to the transport packet, etc., depending on the implementation method, through different The method method transmits and uses the information described by the MUR. Four schemes are proposed for the transmission mode of the MUR information file as a use case.
  • the methods for adding the MUR information to the signaling information include but are not limited to the following methods:
  • Method 1 add MUR information to the existing CI file
  • the function of the CI file is to guide the presentation of the media resources, and the level_list is also the information related to the resolution of the resources, so the level_list content can be added to the existing CI file.
  • the receiving end parses the CI file, it needs to support the resolution of the level_list information.
  • Mode 2 Add a MUR file describing the MUR information to the signaling information.
  • the generated MUR file can be transmitted in parallel with the CI and HTML files. According to the way the CI file is transmitted in the protocol, the appropriate transmission method of the MUR file is placed.
  • the MPI table is transmitted as part of the signaling message.
  • the part to be changed is in the description part of PI_content.
  • the value of PI_content_count is increased by 1.
  • the PI_content_type field describes the type of the MUR file. You can select the file format suitable for describing the MUR information according to the actual situation.
  • the PI_content_type_length value is the file type.
  • the length, PI_content_type_length_byte value is the type of the MUR file;
  • PI_content_name_length is the length of the MUR file name,
  • PI_content_name_byte is the value of the name of the MUR file;
  • PI_content_descriptores_length and PI_content_descriptors_byte are the length and content of the description information of the MUR file, respectively, which can be empty at present. Reserved for future expansion; PI_content_length and PI_content_byte respectively Is the length and content of the MUR file.
  • the MUR information can be transmitted.
  • Method 3 add a descriptor describing the MUR information
  • Some descriptive descriptors are defined in the signaling messages defined by the existing MMT protocol, and the descriptor is a descriptive information used by the signaling part of the MMT to define some fields or functions.
  • the dependency descriptor and the MPU_timestamp_descriptor therefore, a descriptor describing the MUR information can be defined, and the level information of the mpu can be described in the descriptor.
  • asset_descriptors field there is an asset_descriptors field. You can add related descriptors to the asset_descriptors as needed. When you need to describe the level information of the mpu in the signaling, you can add the descriptor to the asset_descriptors.
  • Method 4 Add a signaling table describing the MUR information.
  • a table can be added to the signaling information in addition to the existing several signaling tables to describe the MUR information. After generating a table describing the MUR information and adding a specific header, a message describing the MUR information is generated, and the message and the PA message, the MPI message, the MP messae, the CRI message, and the like together form a new signaling information transmission. After receiving the signaling message, the receiving end parses and obtains the relevant level_list information.
  • FIG. 4 is a method for constructing a MUR file according to the method described in Type 1 above, and transmitting the MUR file by using Mode 2 above, for example, describing a specific implementation flow.
  • the transmitting end sends signaling information
  • the receiving end receives the MUR, CI, and HTML files by determining the PI_content_type_length_byte field in the MPI table, and updates the generated MPU that is locally maintained.
  • the receiver checks the MUR file to find the lite version corresponding to level_list[0], and finds that the corresponding mpu_seq_number is 1,9, at which time the client requests these MPUs and stores them in Play locally to the user and update the local database.
  • the MPU file and the local database are used to obtain the MPUs that are not stored locally from the sender.
  • the MPUs are MPUs obtained by level_list[1]-level_list[0], where mpu_seq_number is 4. 6. After receiving the 4,6, the receiving end reorganizes with the locally stored 1,9 into a cut version video.
  • the embodiment provides a content-driven smart home control method.
  • Philips hue the product of the world's smartest LED bulb, gives a way to tag video and audio data to drive the Hue system.
  • Philips hue looks like a normal light bulb, but it can be bridged to the router at home so that users can have more personalized control over lighting.
  • Philips hue uses LED lighting technology and wireless interconnection technology to provide LED lighting on the basis of more convenient lighting for people's lives. Through mobile phone positioning function, hue can automatically go home or go out. Turn the lights on, turn off the lights, or change the color of the lights. By setting a timed reminder, hue can make everyday life more regular: for example, by letting the light in the room gradually light up in the morning, and the night light will remind people to fall asleep.
  • Philips hue offers "different shades of white, from warm to cool”, as well as pre-programmed over 16 million color options (red, green and blue), as well as relaxation, reading, Four different preset modes, including concentration and vitality. Create a specific lighting scene for a specific atmosphere, remotely control and monitor the lighting conditions in your home, set a timer to manage your daily lighting needs, help you with your lights, or wake your family, all of which can be done for you by Philips Hue. In addition, users can even use a photo from the device as a color palette for color picking, and set the program to activate a specific color for a specific time of day.
  • Hue's functions can be continuously enriched.
  • the smart home control system is automatically driven by the multimedia content, such as the above-mentioned Philips hue.
  • the multimedia content such as the above-mentioned Philips hue.
  • Add corresponding tags to the multimedia content used, and these tags will drive the smart device to work each time the multimedia content is played. The details are as follows, as shown in FIG. 5.
  • All media data is encapsulated in a fragmented MPU format in a heterogeneous network transport protocol (MMT), each MPU has a time length of about 0.5 seconds, and the video video and audio audio of the multimedia audio and video data are separately packaged. Based on this media data encapsulation format, it is proposed that the MPU can be marked in the following ways.
  • MMT heterogeneous network transport protocol
  • the color of the picture can be controlled by the color of the picture, so the main color of the key frame in the video MPU can be extracted and marked, using the reserved field 7bit reserved in the MPUBox.
  • color_tag you can use different color_tag for different colors, as shown in the following table.
  • color_tag can be assigned to the color_tag by the key main tone analysis in the video MPU, and the MPUBox should add the required new attribute. Or automatically assign an assignment to the main tone of the MPU by an algorithm.
  • the reserved 7 bits are used for the color_tag, and if more bits are needed in the future, the extension may be added.
  • the Philips hue system different lighting modes can be presented according to different environments in life, so this embodiment can extract patterns of a certain scene in the video, such as whether it is romantic or intense, and then mark the MPU.
  • the reserved field in the MPUBox can be defined as the scene_tag, and different algorithms can be used to extract the patterns in the scene.
  • the lighting mode can be controlled by the level of the musical tone, so some embodiments of the present invention can extract the tonal features in the audio MPU, thereby marking the MPU.
  • the reserved field in the MPUBox can be defined as tone_tag, which can sample a variety of different algorithms to extract the pitch of the audio data.
  • the lighting device Since the corresponding scene_tag and tong_tag are loaded in the media content, when the media content is in normal play, after the device reads the corresponding new attribute, the attribute information is transmitted to the lighting control device interface. At this point, the lighting device will parse out the commands and parameters that control the lighting system based on the received attribute information. Thus, the lighting adjusts the changes in real time as the media content is played.
  • MPUBox shows how to add new attributes to achieve the combination of media content and lighting.
  • new attributes can be added to other signaling locations according to actual needs.
  • the above embodiment of the present invention takes illumination as an example to illustrate how traditional multimedia and smart homes can be combined.
  • the idea and method of the present invention can also be applied to other smart homes and even smart city systems.
  • the intelligent media transmission system can classify multimedia content at different angles, that is, to put different logos, which provides a possibility for the user's personalized needs.
  • the service provider associates different multimedia content versions with different edit lists according to different labels, and the corresponding identifier is edit id. In order to enable the user to identify the content corresponding to the edit list provided by the service, it is necessary to transmit corresponding description information to ensure the personalized needs of the user.
  • the producer can provide multiple versions of the video, which can be divided into video versions of the video, such as Lite, Cut, Full and Extended.
  • Some embodiments of the present invention implement personalized presentation by adding description information (multimedia content feature information or association information of each version of the same media resource) to the transmitted information, and the following three schemes are taken as an example for introduction.
  • Solution 1 Add a description attribute to the existing Element (MediaSync) information of the CI (Composition Information) information to describe the description information of different versions of the content, as follows:
  • the CI information mainly includes a view element, an area element, a MediaSync element, and an Option element.
  • the view element provides time domain control information for layout transformation of a visual area; the area element is a child element of the view element, that is, a certain part of a visual view, corresponding to a div element in the HTML5 file matched with CI; MediaSync element A media resource used to indicate the HTML5 file that is paired with the CI; the Option element is used to indicate that the CI in this section is an option.
  • Each CI element can have some different attributes.
  • the id attribute is the identifier of the element; the style attribute is used to specify the CCS style of the element; the begin attribute indicates that the CI instruction starts to function. Time; the end attribute specifies the time at which the CI instruction stops functioning; the refDiv attribute indicates the identity of the div element in the HTML5 file corresponding to the element; the refId attribute indicates the identity of an element in the HTML5 file corresponding to the element; the mediaSrc attribute Indicate the address of the property; and the new description attribute.
  • the function of the CI file is to guide the presentation of the media resource, and the edit list is to parse the resource-related information, so the edit list content can be added to the existing CI file.
  • the receiving end parses the CI file, it needs to support the parsing of the edit list information.
  • the MediaSync element in the CI file is used to specify the media resource, so that the description information can also be added to the media resource while the media resource is specified. Therefore, the description attribute is added under the MediaSync element to describe the content description information corresponding to the media resource.
  • the media resources in the above program include video and audio, respectively asset1 and asset2.
  • the server recognizes the asset
  • the identifiers of the media data units are classified into different edit lists, and each edit list is identified by the corresponding edit id, and a corresponding CI file is generated, which includes the transmitted edit list and the corresponding media description information description.
  • the client parses the received CI file, selects the corresponding description according to the user's needs, parses the edit id information, and requests the corresponding media data unit.
  • Option 2 Add a new element EditDesp to the CI file for description.
  • the information of the media resource is placed in MediaSync, and a new element EditDesp with the same level as MediaSync is added in the CI to indicate the description information of all related contents in the media resource, and the edit element is a child element of the EditDesp element, and each edit element Characterize a variety of personalized presentation descriptions.
  • the edit element the edit_id attribute and the description attribute are added.
  • the edit_id attribute is used to identify a certain level or a part of the media resource; the description attribute is used to describe the content description information of the media resource.
  • the EditDesp element in the above program defines a description of the four associated content in the media resource and writes its corresponding edit_id.
  • the client parses the received CI file, selects the corresponding description according to the user's needs, parses the edit id information, and requests the corresponding media data unit.
  • the CI file generated by the server for the same media resource is consistent and complete, which satisfies the user's individual needs, and reduces the server to repeatedly generate related descriptions and corresponding edits. Id.
  • Descriptor_tag defines the tag of this descriptor type.
  • Descriptor_length defines the length of the descriptor.
  • Edit_list_number — defines the associated content of the media resource, and N1 represents the number of related content.
  • Edit_id – A label that defines the versions of the associated content of the property.
  • Edit_description_length The length of the media resource level description in bytes.
  • Edit_description_byte – specifies a byte in the message.
  • the server identifies the identifiers of the media data units in the media resource and classifies them into different edit lists, and each edit list is identified by the corresponding edit id.
  • the system In order to realize the interaction between the user and the system, identify the transmitted media resources and select relevant content for presentation according to the requirements, the system generates descriptor signaling. It contains the transmitted edit list and the corresponding media description information description.
  • the client parses the received signaling information, selects the corresponding description according to the user's needs, parses the edit id information, and requests its corresponding media data unit.
  • the relevant description of the content can be achieved by adding the descriptor to the asset_descriptors.
  • Embodiments 1 and 2 are different implementation methods for personalizing the two transmission protocols in DASH and MMT, and the media content is classified into different levels (ie, the edit list in this embodiment) for transmission.
  • Embodiment 3 is a specific application of different levels. The user does not distinguish between different edit lists, and the user can only understand the corresponding description information, such as different video versions corresponding to different edit lists, which is different from the above embodiment.
  • the CI file is constructed according to the method described in the foregoing scheme 2, and a specific implementation process is described by way of example: as can be seen from FIG. 4, when the receiving end logs in to the VoD service.
  • the sender sends signaling information, and the receiver picks up the PI_content_type_length_byte segment in the MPI table.
  • Receive CI and HTML5 files and update the generated MPU storage database MySQL and parse the CI and HTML5 files; provide personalized information for the user according to the control information in the CI and the newly added multimedia content description information.
  • the receiving end finds the lite version corresponding to edit_list[0] through the CI file, and finds that the corresponding mpu_seq_number media data identifier is 1, 9, and the client requests these MPUs and stores them. Play to the user locally and update the local database.
  • the MPU is obtained from the sender by checking the CI file and the local database.
  • the MPUs are MPUs corresponding to edit_list[1], and the mpu_seq_number media data identifiers are 4, 6. After receiving the 4,6, the receiving end reorganizes with the locally stored 1,9 into a cut version video.
  • this embodiment is directed to the incompleteness of the presentation mechanism in the existing multimedia protocol, fully considering the user decision-based presentation mechanism of the on-demand service, the push mechanism of the broadcast and real-time live broadcast service, and the related content.
  • Presentation service For the associated content version of the same media resource, the presentation time of each media data unit is different, by extracting the duration of the media data unit, according to the version selected by the user and different operations during the playback process, in the signaling information On the basis of the start time, the duration of the media data unit is accumulated to generate a presentation timeline of the corresponding version of the media content; or the absolute presentation time of the media data unit is generated while the signaling is generated, and the corresponding presentation is generated according to the user's selection. Timeline information.
  • the server transmits the mapping table of the media content and the media description to the user, that is, the option of providing the user with the associated media content.
  • the user can request different versions of the same content according to their own needs, or request multiple presentation forms such as associated content of a certain media resource.
  • different versions of the same set of media resources contain shared data units.
  • the existing system cannot control the presentation time of different versions of the content, which may result in a fragment of the media playing, which affects the user experience.
  • VoD Video on Demand
  • real-time live broadcast is synchronized on each device at the specified time, but still need to consider the problem of random access and real-time in real-time live broadcast. Therefore, in this embodiment, different personalized presentation timeline control mechanisms are introduced by taking the transmission network in three cases as an example.
  • the server responds to the request, reads the duration of the corresponding media data unit according to the version selected by the user, and generates a corresponding guidance presentation information file and signaling information.
  • the first version of the video is a streamlined version of the film, which lasts for 5 minutes. It only includes the wonderful pictures in the movie.
  • the second version of the video is a cut version with a duration of 30 minutes. It only includes the scenes of the movie story.
  • the version video is the full version, with a duration of 120 minutes, including a complete storyline; the fourth version is an extended version with a duration of 150 minutes, which includes expanded content in addition to the complete storyline, such as behind-the-scenes footage, etc. 1 is shown.
  • the server transmits the media data stream in accordance with a predetermined program list. Considering the random access problem of the user, the server needs to present related information to the carousel. The user can start watching the media content upon access based on the received current presentation timeline. At the same time, the viewing mode is switched in real time according to the status of the viewing device, such as the power level, and the like, and a better user experience is provided while adapting to the device state.
  • the media data unit is generated, the media data unit is classified according to the importance of the media data unit, such as a wonderful moment and a goal, etc.
  • the lens is a different related content of the program, and different identifiers are respectively given, as shown in FIG. 6.
  • the user's power status is roughly divided into full power, medium power and low battery levels.
  • the server When the remaining power of the device reaches the corresponding level, the corresponding request is sent to the server, and the server automatically switches the delivered video media content and the complete audio content, and controls the synchronization of the media content according to the transmitted timeline.
  • the time information inside the video data unit cannot be parsed. Therefore, the time information is obtained by analyzing the corresponding time line related signaling to implement program presentation that adapts to the condition of the device.
  • the media data is directly processed and transmitted to the client, so the requirements for real-time performance are very high.
  • the associated content of the media resources needs to quickly generate an independent time line for control.
  • the content provided by the service is transmitted to the user through the broadcast network, and the timeline information of all associated content of the media resource is transmitted. Since the total duration of each associated content and the presentation time of the included media data units are different, it is necessary to generate multiple sets of different timelines for rendering control.
  • the invention divides the media resource into independent solvable media units, and automatically generates a corresponding presentation timeline according to the different video versions selected by the user by using the association relationship of the media units. Therefore, the presentation mechanism of the present invention is more flexible than the prior art.
  • the intelligent media transmission system can classify the media content from different angles, that is, to put different labels, which provides the user's personalized needs.
  • different versions of the associated content contain shared media data units, but the presentation time of these data units in each version is different, therefore, different rendering timelines need to be generated for each version to control playback.
  • the presentation time of the MPU (media processing units) media data unit in each version is as shown in FIG.
  • duri represents the duration of the i-th MPU.
  • the presentation time of the same MPU is different. Therefore, different associated versions of the same media resource require separate timelines to guide presentation.
  • the invention realizes the transmission of the presentation timeline by adding a new descriptor or adding other guidance information such as message, table and other signaling information, and introduces the following three schemes as an example.
  • the MPU timestamp descriptor defined in the MMT is used.
  • the descriptor marks the corresponding label mpu_sequence_number and the corresponding UTC absolute presentation time of the MPU in a media resource; the syntax structure of the descriptor is as shown in the following table.
  • Descriptor_tag defines the tag of this descriptor type.
  • Descriptor_length defines the length of the descriptor.
  • Mpu_sequence_number lists the labels of all MPUs contained in the corresponding media resources in order, and N indicates the number of MPUs.
  • Mpu_presentation_time enumerates the UTC absolute rendering time of all MPUs contained in the media resource corresponding to the descriptor.
  • the server obtains the corresponding MPU according to the version selected by the user.
  • Duration information Combine the media play time selected by the user or the media play time specified by the system, and calculate the UTC absolute presentation time corresponding to each MPU by accumulating the durations of all MPUs before the MPU, which is the mpu_presentation_time in the descriptor.
  • Some descriptive descriptors are defined in the signaling messages defined by the existing MMT protocol, such as a dependency descriptor and an MPU_timestamp_descriptor. Therefore, a descriptor that provides media content and time information related to the corresponding presentation can be defined.
  • asset_descriptors field in the MP table you can add related descriptors in the asset_descriptors as needed. When using a multimedia service, the descriptor can be added to the asset_descriptors for rendering.
  • the on-demand service is user-driven and needs to consider the user's operations during the viewing process, such as pause and fast forward.
  • the server only provides the UTC absolute rendering time of each corresponding MPU
  • the playback cannot be resumed correctly, and the server needs to regenerate the presentation timeline information and then transmit, which causes a large computational burden and redundancy.
  • the balance will also cause a certain delay and affect the user experience.
  • UTC absolute presentation time, mpu_presentation_time in broadcast services is a convenient way because the playback time of selected media content in the live broadcast service is fixed.
  • the user receives a certain media resource and selects its associated service, and receives the corresponding MPU_timestamp_descriptor.
  • the media data unit of the content of each associated part may be presented at a fixed time according to the time information in the descriptor.
  • the edit list is defined according to the associated content of the same media resource and the corresponding set of MPUs, and the associated content of each version is given an independent edit id.
  • the mpu_sequence_number and corresponding presentation time information of all MPUs included in each edit list are described in the descriptor; the syntax structure of the descriptor is as shown in the following table.
  • Descriptor_tag defines the tag of this descriptor type.
  • Descriptor_length defines the length of the descriptor.
  • Edit_list_number defines the associated content of the media resource, and N1 indicates its number.
  • Edit_id – A label that defines the versions of the associated content of the property.
  • Mpu_sequence_number lists the labels of all MPUs contained in the corresponding media resources in order, and N indicates the number of MPUs.
  • Mpu_presentation_time enumerates the UTC absolute rendering time of all MPUs contained in the media resource corresponding to the descriptor.
  • the UTC absolute presentation time mpu_presentation_time of all the associated content of the selected media resource is written in the descriptor (the time obtaining manner refers to the scheme 1).
  • the server sends the above descriptor to the client along with the signaling information.
  • the user selects a certain version of the associated content, parses the media data unit MPU and the corresponding absolute presentation time mpu_presentation_time according to its corresponding edit_id, and generates a timeline of the corresponding version to control the presentation.
  • the presentation time of the media content of each associated version can be conveniently obtained, and the personalized user requirements are guided by the same descriptor information, which is more convenient to control.
  • the duration of each MPU in a certain media resource is described in the signaling, and the start time of the media resource is known from the CI (Composition Information), and the UTC absolute presentation time of each MPU can be calculated.
  • CI Composition Information
  • Descriptor_tag defines the tag of this descriptor type.
  • Descriptor_length defines the length of the descriptor.
  • the corresponding media data unit MPU is selected according to the media content selected by the user, and the duration information duration is analyzed, and the guidance presentation information, that is, the MPU label mpu_sequence_number and the corresponding duration information are generated.
  • such a scheme can acquire the corresponding duration information while generating and encapsulating the media data unit, so that the real-time requirement can be satisfactorily satisfied, and the real-time live application can be satisfied. demand.
  • the client By replacing the absolute presentation time by transmitting the duration information, the client can more self-organize the media content.
  • users can also support input operations at any time to meet the individual needs of users.
  • the above three schemes can cover multiple multimedia transmission systems including broadband networks and broadcast networks and even heterogeneous networks.
  • We can also add corresponding presentation timelines in the presentation guidance file CI or other locations where signaling information is transmitted.
  • Personalized rendering services can be implemented.
  • both the broadcast and the live broadcast service start playing from the user's current access time.
  • the presentation time information or the duration information of the MPU that has not been consumed are written in the above descriptor (hereinafter, the two types of time information are collectively referred to as related time information), and The relevant time of all MPUs corresponding to the media resources is not transmitted. In this way, although the complexity of generating signaling information is increased, the transmission network bandwidth and the limited computing power of the client are greatly saved.
  • the on-demand service does not have a random access problem
  • the related time information of the limited number of MPUs can be correspondingly transmitted, and the relevant time information of the subsequent MPUs can be timely transmitted according to the user's viewing progress.
  • the following figure generates an MPU_presentation_descriptor according to the description method of the above scheme 3, constructs a personalized presentation timeline mechanism in the VoD service, and describes a specific implementation process by way of example:
  • the foregoing scheme 3 is taken as an example to provide a timeline control method for personalized presentation of multimedia content.
  • the method is specifically as follows:
  • the transmitting end When the receiving end requests the VoD service, the transmitting end will send signaling information, and the receiving end receives the MUR, CI, and HTML files by determining the PI_content_type_length_byte field in the MPI table, and updates the generated local dimension.
  • the sender resolves the request to obtain the corresponding mpu_seq_number and parses the media data unit MPU to obtain its corresponding duration information, and generates MPU_presentation_descriptor signaling.
  • the receiving end receives the corresponding signaling, and generates an absolute presentation time of each MPU according to the start time in the MPU_presentation_descriptor and the CI, and maintains the presentation timeline.
  • the received media data is stored locally for playback to the user and the local database is updated.
  • the association of the same set of programs is utilized, and the smooth viewing experience of the user is ensured on the basis of saving storage space. Since the presentation timeline information of the media is flexibly transmitted with the signaling information, the delay and the packet loss caused by the process of consuming the media resources cause the user experience to be reduced. If the packet loss event is detected before the presentation time, the media may be awaited. Re-acquisition of resources, or repeatedly presenting the contents of the previous media data unit to prevent playback and ensure user experience.
  • the MMT is taken as an example to illustrate the proposed solution, but these solutions can also be used in other file encapsulation, transmission systems and protocols.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

一种关联多媒体内容个性化呈现的实现方法及应用，所述方法具体为：多媒体内容提供商在制作多媒体文件时统一完成完整的文件内容，同时采用标记方法依据内容的重要性和/或关联程度对其中的片段进行分级，从而产生不同版本的多媒体文件；用户根据这些版本进行选择性预览和/或播放。该方法是一种在互联网点播系统中灵活高效的可伸缩组织、存储与传输的方法，从而增加了现有多媒体协议缺乏的媒体内容灵活组织机制，解决了现有多媒体系统的传输效率低、存储资源浪费、用户体验差的问题。

Description

一种关联多媒体内容个性化呈现的实现方法及应用 技术领域
本发明涉及一种在互联网(Internet)点播系统中新型多媒体组织、存储与传输的方法，更具体的说，涉及一种关联多媒体内容个性化呈现的实现方法及应用。
背景技术
随着多媒体技术的快速发展，互联网中多媒体内容出现爆炸式的激增，特别是高清晰度、高分辨率视频业务的广泛深入，多媒体服务给网络带宽带来的压力愈加凸显。同时用户个性化兴趣的不同以及网络状况的时变性，如今的多媒体服务呈现聚合式的特征，多媒体内容呈现碎片化的趋势。现有的多媒体协议，大多只关注在网络传输过程中数据的完整性和安全性，通过保证网络的质量(QoS)来保证多媒体服务的质量，然而对多媒体内容整合却很少考虑，对用户的主观满意度也考虑甚少。
当前一些多媒体协议基本都是通过信令信息描述和控制数据的传输和呈现，但是没有涉及到多媒体内容组织级别，而且没有考虑内容组织与存储、传输的匹配和整个多媒体系统的数据优化。这样造成的结果是，虽然多媒体制作方制作了内容统一的完整节目，但经过剪辑等后期制作，节目发行方和提供商将原节目分割成不同版本，每一种版本之间都相互独立，成为一个单独的媒体内容。这样表面看起来丰富了节目内容，实际上破坏了整个媒体系统的生态环境，产生大量重复数据。这些数据不仅破坏了媒体内容天然的关联体系，不利于媒体内容的管理，增加了存储成本，造成了资源极大的浪费，而且逻辑上内容相同的数据单元需要多次在网络中重复传输，大大降低了数据传输的效率。同时，不同版本节目内容冗余的独立存在使原本丰富的媒体资源变得臃肿杂乱，加大了数字媒体内容监管的难度，许多盗版侵权的媒体内容也因此更加容易进入数字媒体市场，造成节目提供商和消费者利益的损害。
另一方面，媒体资源的丰富性在某种程度也增加了用户消费数字媒体的负担，用户必须更多精力用于媒体消费的选择。传统的方法是，内容提供商提供丰富的媒体节目如各种类型的电影资源，但这些节目一般都是完整版的，用户一般只能点播某种完整版本的媒体节目进行消费。这样的情况下，如果用户想选择观看自己喜欢的节目就需要对多个节目进 行预览，而基于该机制数据缓冲的效率低下，需要用户等待很长时间，用户体验非常糟糕。
更好的方法是，节目发行方和提供商只在逻辑上将原节目分割成不同版本，由于媒体内容碎片化单元的广泛应用，原节目在物理上仍旧是统一的天然整体。在媒体内容服务器上只需要存储媒体的完整版本，同时提供相应的信令文件描述不同媒体版本所对应的数据单元。这样用户根据自己需求点播相关版本的媒体数据，服务器根据信令信息中媒体版本与数据单元的映射只传输相应数据给用户。例如，用户只想预览一下某个电影，他只需要点播预览版本的链接，服务器只传输预览版的电影数据单元给客户端解码和呈现，用户不需要等待整个电影内容的缓冲，可以得到更好的用户体验。其困难在于：系统很难知道不同版本之间媒体单元的对应关系，并且缺少相关的信令机制指导；每个媒体都可能有多个长短不一的版本，如何对这些版本进行区分和管理，也是亟待解决的问题。
另外，虽然当前的多媒体系统可实现内容的初步分级，但用户无法获知多媒体内容的对应描述信息，无法实现自主地选择。甚至对于同样的节目，用户也有不同的选择需求。为了实现多媒体内容的个性化呈现，更高效的在媒体网络中传输媒体资源，需要多媒体内容个性化呈现机制。服务器通过将多媒体内容与媒体描述的映射表传输给用户，即给用户提供关联性的多媒体内容的选项。用户可根据自己的需求请求相同内容的不同版本，或是请求某一媒体资源的关联内容等多种呈现形式，并根据用户的需求进行个性化设置。
而这种多媒体内容个性化呈现机制提出了一个新的要求，由于关联的媒体资源用不同的edit list(编辑列表)表示，不同的edit list中包含不同的媒体数据单元。且对于不同的多媒体内容，用户也有不同的选择角度。例如，对于一场晚会，用户可以选择观看多角度的视频还是观看不同歌手的表演。所以edit list也能根据用户需求和节目类型等个性化生成。实现个性化传输需要相应的机制将多媒体内容的描述信息和edit list之间的映射关系传输给接受媒体资源的客户端。
发明内容
针对现有技术中的缺陷，本发明的目的是提供一种关联多媒体内容个性化呈现的实现方法，在互联网点播系统中灵活高效的可伸缩组织、存储与传输的方法，从而增加了现有多媒体协议缺乏的媒体内容灵活组织机制，解决了现有多媒体系统的传输效率低、存储资源浪费、用户体验差的问题。
根据本发明的第一目的，提供一种关联多媒体内容个性化呈现的实现方法，所述方法具体为：多媒体内容提供商在制作多媒体文件时统一完成完整的文件内容，同时采用标记方法依据内容的重要性和/或关联程度对其中的片段进行分级，从而产生不同版本的多媒 体文件；用户根据这些版本进行选择性预览和/或播放。
进一步的，所述方法具体包括如下步骤：
第一步，通过内容分级的版本管理将多媒体文件分为多个版本；
第二步，生成与版本相关的差异化MPD文件：根据上述多媒体文件的多个版本，生成与内容分级相对应的MPD(Media Presentation Description)文件，即内容分级的版本管理标签；
第三步，用户根据自己的需求请求不同版本的多媒体文件内容，服务器传输统一的MPD文件，客户端解析MPD之后根据自己的网络状态、设备性能以及请求的版本情况，向服务器请求相应的媒体分片内容。
进一步的，所述关联多媒体内容个性化呈现的实现方法，包括如下步骤：
第一步，通过内容分级的版本管理将多媒体文件分为多个版本；
第二步，对应于多个版本，只有一个相对应的MPD文件被生成，但MPD文件的segment描述增加了@scaleID属性，即内容分级的版本管理标签；
第三步，用户根据自己的需求请求不同版本的多媒体文件内容，服务器传输统一的MPD文件，客户端解析MPD之后根据自己的网络状态、设备性能以及请求的版本情况，向服务器请求相应的媒体分片内容。
采用本发明上述的关联多媒体内容个性化呈现的实现方法，可以针对现有多媒体协议缺乏媒体内容灵活管理的机制，造成多媒体系统中传输效率低、存储资源浪费、用户体验差的问题，通过增加媒体内容的可伸缩灵活组织、存储和传输方法实现对互联网上媒体内容的优化整合，逻辑上同一媒体资源统一存储，根据用户的媒体版本需求进行差异化传输，并且在客户端进行特定的媒体单元缓冲和解码呈现，既使丰富的媒体资源得到统一优化，也便于多媒体内容的监管控制，同时提高网络中多媒体数据的整体传输效率，减少存储空间的浪费，给用户带来更好的服务体验。
根据本发明的第二目的，为解决了现有的多媒体内容分级无法细化并且传输效率、存储空间浪费的问题，本发明提供一种多媒体内容分级技术的实现方法：在制作多媒体文件时对多媒体内容的重要片段进行标记，根据多媒体文件内容的重要程度将每个片段划分不同的等级；在用户观看时，根据这些等级对多媒体内容进行选择性播放，而不是盲目的随机选择；新定义一个MUR信息文件，用于描述多媒体文件与不同版本内容、mpu_seq_number、level_list之间的对应关系，mpu_seq_number是MPU中的标记字段，level_list是mpu_seq_number的集合，与不同版本的视频是一一对应关系。
优选地，对于某一多媒体内容，将其切成多个MPU碎片，内容分级成不同版本后与mpu_seq_number之间存在对应关系；新定义一个MUR信息文件，用于描述多媒体文件与不同版本内容、mpu_seq_number、level_list之间的对应关系，其中根据level_list数组与不同版本内容时间的对应规则不一样，分为两种类型：
类型1中，不同版本的内容与level_list之间是一一对应的；
类型2中，不同版本的内容是不同level_list的组合；
根据类型1和类型2中MUR信息文件的内容对多媒体文件进行差异化传输。
采用本发明的多媒体内容分级技术的实现方法，可以针对现有多媒体协议中多媒体内容分级技术空白，造成的媒体内容传输效率不高、存储空间房费的问题，通过增加媒体内容的可伸缩传送、存储和呈现机制实现对互联网上媒体内容不同版本的关联分类，增加对多媒体服务的管理控制，同时提高多媒体数据的传输效率，减少存储空间的浪费。给予用户更多自主选择媒体版本的权利提高用户体验的同时，也能给网络运营商提供更好的多媒体内容分级、传输及存储方案。
根据本发明的第三目的，将上述关联多媒体内容个性化呈现的实现方法进行应用，提供一种以内容驱动的智能家居控制方法，所述方法通过基于视频中关键帧的图像主色调和/或场景情节特征对视频碎片进行标记，或者依据音频数据特征进行标记，添加相应的标签，每次播放这些多媒体内容时，标记的这些标签都会驱动智能家居设备工作。
优选地，所述通过基于视频中关键帧的图像主色调对视频碎片进行标记，具体为：提取视频MPU中关键帧的主色调并对其进行标记，利用MPUBox中预留的reserved字段定义color_tag，对于不同的颜色可以用不一样的color_tag。
采用本发上述的以内容驱动的智能家居控制方法，可以通过多媒体内容自动的驱动智能家居控制系统，目前大多数需要人为评价多媒体内容之后再去手动的控制智能系统，通过本发明的内容驱动控制智能家居系统可以大大简化人为控制这一繁琐的过程，并且本发明使得控制系统更加灵活。
根据本发明的第四目的，当服务商依据媒体内容种类或其重要程度等信息对媒体内容进行标记后，同一标记的媒体数据单元的组合即为不同的edit list。为依据用户的个性化请求提供服务，需要对媒体内容的标记信息进行相应的描述，使得用户理解各个edit list的具体内容，实现差异化传输及呈现，提供准确的服务。为了解决用户无法选择服务器提供的关联多媒体内容的问题，实现针对不同用户需求的个性化媒体呈现，本发明提供一种关联多媒体内容个性化呈现信息的描述方法，所述方法为：媒体资源的关联多媒体内容用 不同的edit list(编辑列表)表示，不同的edit list中所含的媒体数据单元是补充关系或包含关系；为了将媒体资源的分类及其描述信息的对应关系呈现给用户，在传输的信息中添加对媒体资源特征信息或关联信息的描述，来实现个性化呈现机制。
进一步，所述关联多媒体内容个性化呈现信息的描述方法，实现步骤为：
定义同一媒体资源所包含的媒体数据单元的标识或对应的描述信息，生成descriptor信令信息，其中包含传输的媒体资源中的各类多媒体内容描述信息；
客户端解析descriptor信令信息，用户根据需求选择具有相应标识或信息表示的多媒体内容，并请求相应的媒体数据单元组合成对应的edit list；
服务端解析用户请求并发送获取相应的媒体数据单元，客户端解析信息并实现个性化呈现。
根据本发明的第五目的，针对现有多媒体协议中呈现机制的不完备，考虑到点播服务的基于用户决策的呈现机制和广播、实时直播服务的推送机制，以及关联内容的呈现服务，本发明提供一种多媒体内容个性化呈现的时间线控制方法，所述方法：在多媒体资源呈现的过程中，利用媒体数据单元的持续时间信息和呈现的初始时间，对关联内容的呈现时间进行控制，使得不同的媒体资源或是同一媒体资源的不同关联版本都有对应的呈现时间线。
进一步的，所述方法具体步骤包括：
S1：根据用户选择的媒体资源的关联内容版本所包含的媒体数据单元，对每个媒体数据单元获取其相应的持续时间信息；
S2：在VoD中根据用户选择媒体资源时的时间确定开始播放时间，在广播服务中由服务提供商确定开始播放时间，在信令信息中标识开始播放时间；
S3：根据S2中的开始播放时间和对应关联内容版本中的媒体数据单元持续时间的累加得出对应媒体数据单元的绝对呈现时间，据此维护相应的呈现时间线来指导媒体内容的呈现。
本发明所述的多媒体内容个性化呈现的时间线控制方法，在满足用户个性化需求的同时，利用同一套节目的关联性，在节约存储空间的基础上，保证用户的流畅的观看体验。由于媒体的呈现时间线信息随信令信息灵活地发送，针对消费媒体资源的过程中出现的延时及丢包造成用户体验降低的问题，若在呈现时间之前检测到丢包事件，可等待媒体资源的重新获取，或是重复呈现前一个媒体数据单元的内容以防止播放卡顿，保证用户体验。依据呈现时间线的指示继续呈现，具体实现方案不在此规定。
采用了本发明上述的技术方案，可以针对现有多媒体协议中关联多媒体内容信息描述的空白，造成的用户无法理解服务器提供的媒体资源的问题。通过增加对多媒体内容特征信息或关联信息的描述，使用户对该多媒体内容有更直观且多方面的理解，描述个性化的映射关系提供关联的媒体服务，尽可能保证用户选择最适合的多媒体内容。此方案不局限于对服务器提供的多媒体内容的描述，还适用于多源或各种分类方式等多媒体资源的内容描述，甚至可通过用户自定义标签生成描述。本发明解决了用户无法选择服务器提供的关联多媒体内容的问题，实现针对不同用户需求的个性化媒体呈现。
附图说明
通过阅读参照以下附图对非限制性实施例所作的详细描述，本发明的其它特征、目的和优点将会变得更明显：
图1为实施例1、2、5媒体版本与媒体数据单元映射实例示意图；
图2为本发明实施例1中分级机制-模式1示意图；
图3为本发明实施例1中分级机制系统框架-模式2示意图；
图4为本发明实施例3、4、5的实现流程图；
图5为本发明实施例3中利用MPU控制hue系统图；
图6为本发明实施例5中设备低电量情况下的差异化呈现示意图；
图7为本发明实施例5中多视角实时直播示意图；
图8为本发明实施例5中关联内容的不同版本的呈现时间线示意图。
具体实施方式
下面结合具体实施例对本发明进行详细说明。以下实施例将有助于本领域的技术人员进一步理解本发明，但不以任何形式限制本发明。应当指出的是，对本领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干变形和改进。这些都属于本发明的保护范围。
实施例1
如今，视频内容量呈现爆炸式增长，人们可支配的时间也越来越碎片化。当用户接触到一系列新视频时，往往不再是直接点播观看其完整内容，而是希望首先预览这些视频，然后根据用户自身喜好和当时可支配时间决定选择哪一个视频进行观看，是否观看全部视频内容。
针对此问题，可以通过如下方法有效的解决，提高用户体验。内容提供商在制作视频 时统一完成完整的视频内容，同时采用标记方法依据内容的重要性及关联程度对其中的片段进行分级，从而可以产生不同版本的视频。当然，在呈现界面应该提供与分级相关的版本信息，以方便用户进行内容选择消费。在用户观看时，可以根据这些版本进行选择性预览和播放，而不是盲目的随机选择。
以视频内容为例，内容制作商只需要完成一套完整的节目内容，随后在发行方可以通过内容分级的版本管理将视频分为多个版本，提供给媒体服务提供商。第一个版本视频为预览版，时长5分钟，内容只包括视频中的精彩画面；第二个版本视频为剪辑版，时长30分钟，内容只包括视频的故事主线和重要场景；第三个版本视频为完整版，时长120分钟，内容包括视频的完整故事情节；第四个版本为加强版，时长150分钟，里面包括除完整的故事情节之外的扩充内容，例如幕后花絮等等。
下面以MPEG-DASH(Dynamic Adaptive Streaming over HTTP)标准为例，介绍内容分级的版本管理机制。当然，此机制不限于DASH标准，也可以应用于其他标准和方案。DASH全称为基于HTTP的动态自适应流，由国际标准组MPEG制定，能够将媒体通过HTTP协议，以自适应、渐进式、下载或流的方式进行内容分发，支持在不同网络状况下、多种不同性能的终端上的自适应媒体消费。在内容组织上DASH具有多种分片策略，并且有对应的信令文件来指示其分片细节，考虑到通用性，下面按照两种模式分别进行描述。
模式1:
生成与版本相关的差异化MPD文件
在生成DASH segment时，可以根据上述视频节目中的四个版本，生成与内容分级相对应的MPD(Media Presentation Description)文件。为描述简单清晰，只写出了MPD中segment list这种描述情况，其他如segment template情况处理方式类似。
根据上述视频节目中的四个版本对应的segment列表，如图1所示，在生成这些segment时，自动生成四个匹配内容版本的MPD文件。其中MPD元素增加@scaleID属性，该属性定义了此MPD文件描述的内容分级的级别，内容分级版本对应表如下表所示。此外Representation元素中SegmentList子元素中的segment列表不再是单纯的全部分片列表，而是对应于MPD@scaleID的特定分片列表。
版本 预览版 剪辑版 完整版 加强版
scaleID 0 1 2 3
如图2所示，整个分级机制的框架图与DASH原有框架基本一致，在媒体准备阶段媒体分片(segment)的生成机制保持不变，而对应于不同分级版本的4个MPD文件被生成。 在DASH客户端，用户根据自己的需求请求不同版本的视频内容，只需要服务器传输相关的MPD文件，客户端解析之后再向服务器请求相应的媒体分片内容。
例如，当用户选择播放预览版电影节目时，服务器发送MPD@scaleID为0的MPD文件，客户端解析之后请求Representation元素中SegmentList子元素所列出的分片，即channel1init.mp4、channel11.m4s、channel14.m4s、channel17.m4s、channel19.m4s。此MPD文档实例如下所示(新加的参数是sascleID可以有不同的新应用)：
<？xml version＝"1.0"？>
<MPD
xmlns＝"urn:mpeg:dash:schema:mpd:2011"
minBufferTime＝"PT1.500000S"
type＝"static"
scaleID＝”0”
mediaPresentationDuration＝"PT0H26M26.92S"
profiles＝"urn:mpeg:dash:profile:full:2011">
<Period duration＝"PT0H26M26.92S">
<AdaptationSetsegmentAlignment＝"true"maxWidth＝"1920"maxHeight＝"1080"
maxFrameRate＝"30"par＝"16:9">
<ContentComponent id＝"1"contentType＝"video"/>
<ContentComponent id＝"2"contentType＝"audio"/>
<Representation id＝"1"mimeType＝"video/mp4"
codecs＝"avc1.640029,mp4a.6b"width＝"1920"height＝"1080"frameRate＝"30"
sar＝"1:1"audioSamplingRate＝"44100"startWithSAP＝"1"bandwidth＝"29437637"
sacleID＝"2">
<AudioChannelConfiguration
schemeIdUri＝"urn:mpeg:dash:23003:3:audio_channel_configuration:2011"value＝"2"/>
<SegmentList timescale＝"1000"duration＝"9754">
<Initialization sourceURL＝"channel1init.mp4"/>
<SegmentURL media＝"channel11.m4s"/>
<SegmentURL media＝"channel14.m4s"/>
<SegmentURL media＝"channel17.m4s"/>
<SegmentURL media＝"channel19.m4s"/>
</SegmentList>
</Representation>
</AdaptationSet>
</Period>
</MPD>
模式2：
遵循原有DASH机制，只生成一个MPD文件，其中在Representation元素中加入每个分片的@scaleID属性
在生成DASH segment时，可以按照四种内容分级版本进行区分，区分的指示在同时生成的MPD表里呈现。为描述简单清晰，只写出了MPD中segment list这种描述情况， 其他如segment template情况处理方式类似。
根据上述视频节目中的四个版本对应的segment列表，如图1所示，在生成这些segment时，在MPD文件的Representation元素的SegmentList子元素中的每一个segment提供@scaleID属性，该属性标注了使用此segment的最低版本级别。内容分级版本对应表与模式1保持一致。如segment：channel13.m4s的@scaleID为2，那么此segment只有在用户请求完整版视频时才会被服务器传输和客户端消费。
如图3所示，整个分级机制的系统框架图与DASH原有框架基本一致，在媒体准备阶段媒体分片(segment)的生成机制保持不变，同时也只有一个相对应的MPD文件被生成，但MPD文件的segment描述增加了@scaleID属性，即内容分级的版本管理标签。在DASH客户端，用户根据自己的需求请求不同版本的视频内容，服务器传输统一的MPD文件，客户端解析MPD之后根据自己的网络状态、设备性能以及请求的版本情况，向服务器请求相应的媒体分片内容。
例如，当用户选择播放完整版电影节目时，服务器发送统一的MPD文件，客户端解析之后请求Representation元素中SegmentList子元素所列出的@scaleID属性值小于等于2的媒体分片，即channel1init.mp4、channel11.m4s、channel12.m4s、channel13.m4s、channel14.m4s、channel15.m4s、channel16.m4s、channel17.m4s、channel18.m4s、channel19.m4s、channel110.m4s，服务器发送这些分片内容，由客户端解码并呈现给用户。此MPD文档实例如下所示：
<？xml version＝"1.0"？>
<MPD
xmlns＝"urn:mpeg:dash:schema:mpd:2011"
minBufferTime＝"PT1.500000S"
type＝"static"
mediaPresentationDuration＝"PT0H26M26.92S"
profiles＝"urn:mpeg:dash:profile:full:2011">
<Period duration＝"PT0H26M26.92S">
<AdaptationSetsegmentAlignment＝"true"maxWidth＝"1920"maxHeight＝"1080"
maxFrameRate＝"30"par＝"16:9">
<ContentComponent id＝"1"contentType＝"video"/>
<ContentComponent id＝"2"contentType＝"audio"/>
<Representation id＝"1"mimeType＝"video/mp4"
codecs＝"avc1.640029,mp4a.6b"width＝"1920"height＝"1080"frameRate＝"30"
sar＝"1:1"audioSamplingRate＝"44100"startWithSAP＝"1"bandwidth＝"29437637">
<AudioChannelConfiguration
schemeIdUri＝"urn:mpeg:dash:23003:3:audio_channel_configuration:2011"value＝"2"/>
<SegmentList timescale＝"1000"duration＝"9754">
<Initialization sourceURL＝"channel1init.mp4"/>
<SegmentURL media＝"channel11.m4s"scaleID＝"0"/>
<SegmentURL media＝"channel12.m4s"scaleID＝"1"/>
<SegmentURL media＝"channel13.m4s"scaleID＝"2"/>
<SegmentURL media＝"channel14.m4s"scaleID＝"0"/>
<SegmentURL media＝"channel15.m4s"scaleID＝"2"/>
<SegmentURL media＝"channel16.m4s"scaleID＝"1"/>
<SegmentURL media＝"channel17.m4s"scaleID＝"0"/>
<SegmentURL media＝"channel18.m4s"scaleID＝"1"/>
<SegmentURL media＝"channel19.m4s"scaleID＝"0"/>
<SegmentURL media＝"channel110.m4s"scaleID＝"1"/>
<SegmentURL media＝"channel111.m4s"scaleID＝"3"/>
<SegmentURL media＝"channel112.m4s"scaleID＝"3"/>
</SegmentList>
</Representation>
</AdaptationSet>
</Period>
</MPD>
上述实施例是一种在互联网点播系统中灵活高效的可伸缩组织、存储与传输的方法，从而增加了现有多媒体协议缺乏的媒体内容灵活组织机制，解决了现有多媒体系统的传输效率低、存储资源浪费、用户体验差的问题。
实施例2
用户在观看视频时，往往不会直接连续播放整段视频，大多数情况下会提前快进或者跳跃式的浏览这段视频，然后根据自己的喜好或者能够接受的视频时间长度决定是否进行完整播放。在用户跳跃式播放时，极有可能因为随机选择的跳跃时间点不合适，而错过视频内容的重要部分。
针对此问题，可以通过如下方法有效的解决，提高用户体验。内容提供商在制作视频时可以采用某种方式对视频内容的重要片段进行标记，或者更进一步，根据视频内容的重要程度将每个视频片段划分不同的等级。在用户观看时，可以根据这些等级进行选择性播放，而不是盲目的随机选择。
例如，对于某电影节目，制片方可以提供多个不同版本视频。按视频组织结构划分，可分为精简版、剪切版、完整版和扩展版等不同时长的视频版本(可以参见实施例1)。
对于某一多媒体资源，假设其为asset1，将其切成多个MPU(media processing units)碎片，内容分级成的不同版本视频后与mpu_seq_number之间的对应关系如图1所示。mpu_seq_number是MPU中的标记字段，如图1所示，假设某个视频被分成mpu_seq_number 分别是0至11的MPU碎片，不同的mpu_seq_number序列组合成不同版本的视频，例如精简版视频则由mpu_seq_number为1和9的MPU组成。
为了能够灵活的差异化传输及呈现，新定义一个MUR(Media Unit Relationship)信息文件，用于描述媒体资源与不同版本视频、mpu_seq_number、level_list等之间的对应关系。其中根据level_list数组与不同版本视频时间的对应规则不一样，又可以分为两种类型。在类型1中，不同版本的视频与level_list之间是一一对应的；在类型2中，不同版本的视频是不同level_list的组合。
下面根据以上示例，分别说明类型1和类型2中MUR信息文件的内容，以及如何进行差异化传输，还有在客户端如何利用CI文件中的mediaSrc属性对不同版本视频进行灵活呈现。
类型1：
level_list由不同的mpu组合而成，且与不同视频版本之间一一对应，其对应规则如下表所示。
Figure PCTCN2016073167-appb-000001
这个对应规则与图1是完全匹配的，但是图1是一个实例。针对不同的实例，对应规则也可以变化。下一个类型2中的也是同样的道理。
为提高传输效率，当用户需要更多的视频片段时，server仅需传输用户缺失的MPU。例如，假设用户点播看完level_list[i]对应的视频之后，想继续观看level_list[j]对应的视频，其中0≤i<j≤3。因为用户本地已经存放level_list[i]所包含的全部MPU，所以server只需要继续推送Δtf＝level_list[j]-level_list[i]中包含的MPU。
在客户端，用户根据本地的level_list[i]及接收的Δtf重新组成level_list[j]。所以其CI文件中针对不同版本视频的meiaSrc属性的值如下表所示。
Figure PCTCN2016073167-appb-000002
类型2:
与类型1不同的是，level_list由不同的MPU组合而成，并且每个level_list[]之间没有重叠的MPU出现。不同视频版本是由不同的level_list组合而成，其对应规则如下表所示。
Figure PCTCN2016073167-appb-000003
由上表可以看出，精简版视频由level_list[0]组成，剪切版视频由level_list[0]和level_list[1]组成，而level_list[2]则是完整版与剪切版视频的差值，扩充版是由四个level_list组成。所以当用户观看完剪切版视频之后想看完整版视频时，只需要差异化传输level_list[2]所包含的MPU碎片。类型2中的CI文件中针对不同版本视频的meiaSrc属性的值如下表所示。
Figure PCTCN2016073167-appb-000004
MUR信息文件包含了内容分级的重要信息，上文给予描述。如何传输这些重要信息，可以有不同的方法，如通过定义新的信令单元，信令文件，信令消息，descriptor,或者加入传输包的头信息等，可以根据实现方法的不同，通过不同的方式方法传输和使用MUR所描述的信息。针对MUR信息文件的传输方式提出了四个方案作为用例，把MUR信息加入信令信息中的方法包括但不限于以下几种方式：
方式1、把MUR信息加入到现有的CI文件中
CI文件的功能是指导媒体资源的呈现，level_list也是解析资源相关的信息，因此可以把level_list内容添加到现有的CI文件中。接收端解析CI文件的时候需要支持level_list信息的解析。
方式2、在信令信息中增加一个描述MUR信息的MUR文件
按照与方式1相同的思路，也可以定义一个与CI文件类似的描述MUR信息的文件，这种方法的优点是不需要对现有协议中CI进行改动。生成的MUR文件可以与CI和HTML文件并列传送，按照协议中传送CI文件的方式，MUR文件比较合适的传送方式是放在 MPI table中作为信令消息的一部分传送。
Figure PCTCN2016073167-appb-000005
将MUR文件加入MPI table中，需要改动的部分在PI_content的描述部分，PI_content_count的值加1，PI_content_type字段描述MUR文件的类型，可以根据实际情况选择适合描述MUR信息的文件格式，PI_content_type_length值为文件类型的长度，PI_content_type_length_byte值为MUR文件的类型；PI_content_name_length值为MUR文件name的长度，PI_content_name_byte值为MUR文件的name的值；PI_content_descriptores_length和PI_content_descriptors_byte分别是MUR文件的描述信息的长度和内容，当前可以为空，留待以后扩充；PI_content_length和PI_content_byte分别 是MUR文件的长度和内容。
按照以上说明将MUR文件写入MPI table中后，就可以把MUR信息传送出去。
方式3、增加一个描述MUR信息的descriptor
现有MMT协议定义的信令消息中定义了一些描述性的descriptor，descriptor是MMT中信令部分用于定义一些字段或者功能的描述性信息。例如dependency descriptor和MPU_timestamp_descriptor，因此，可以定义一个描述MUR信息的descriptor，在descriptor里可以描述mpu的level信息。在MP table中有asset_descriptors字段，可以根据需要在asset_descriptors中加入相关descriptor，当需要在信令中描述mpu的level信息时，就可以通过把该descriptor加入asset_descriptors中来实现。
方式4、增加一个描述MUR信息的信令表
依照以上思路，也可以在信令信息中除了现有几个信令表之外再增加一个table专门用来描述MUR信息。生成描述MUR信息的表后加上特定的头部就生成了描述MUR信息的message，该message和PA message、MPI message、MP messae、CRI message等一起组成新的信令信息传送。接收端接收到信令消息后解析即可得到相关的level_list信息。
为更加清楚的描述本发明提出的实现方法，图4为按照上文中类型1描述方法构造MUR文件，并且采用上文中方式2传输MUR文件，举例描述一个具体的实现流程。由图4可以看出，当接收端登录VoD服务时，发送端会发送信令信息，接收端通过判断MPI表中的PI_content_type_length_byte字段来接收MUR、CI和HTML文件，同时会更新生成本地维护的MPU存储数据库MySQL；当用户请求精简版视频时，接收端通过检查MUR文件找出精简版与level_list[0]对应，并且找出对应的mpu_seq_number为1,9，此时客户端请求这些MPU并存储在本地播放给用户，并且更新本地数据库。当用户接着请求剪切版视频时，通过检查MUR文件和本地数据库，从发送端获取本地未存储的MPU，这些MPU是level_list[1]-level_list[0]所得到的MPU，其中mpu_seq_number为4,6。接收端接收到4,6之后再与本地存储的1,9重组为剪切版视频。
上述实施例以MMT为例，阐明所提出的解决方案，但是这些方案同样也可以用于其它文件封装，传输系统和协议里。
实施例3
基于上述实施例1、2技术的一种应用，本实施例提供一种内容驱动的智能家居控制方法。
目前智能家居设备逐渐出现在每个人的家庭中，以飞利浦(Philips)公司推出的一款 被称为“世界上最聪明的LED灯泡”的产品——Philips hue为例，给出如何对视频和音频数据进行标记以驱动hue系统的方法。
1.飞利浦hue简介
飞利浦hue它表上面看起来和普通的灯泡一样，但是它可以通过桥接的方式连接到家里的路由器从而让用户可以对照明进行更为个性化的控制。飞利浦hue应用了LED照明技术和无线互联技术，在提供LED照明基础之上，让灯光在更多方面为人们的生活创造便利——通过手机定位功能，hue可以在回家或外出时，自动地开灯、关灯或是改变灯光颜色。通过设置定时提醒功能，hue可以让每天的生活更有规律：比如早上通过让屋里的灯光逐渐变亮，而晚上灯光会提醒人们入睡。飞利浦hue可以提供“深浅不同的白色色调，从暖色调到冷色调”，同时还有经过预编程设置好的超过1600万种颜色选择(红绿蓝组合)，同时还提供了包括放松、阅读、集中以及活力在内的四种不同预设模式。为特定氛围创建特定灯光场景、远程控制和监视家中的灯光情况、设置计时器管理日常灯光需求、通过灯光助眠或叫醒家人，这一切飞利浦hue都能帮你完成。除此之外，用户甚至还可以使用设备中的某张照片作为调色板来进行取色，并设置程序在一天的特定时间内激活特定的颜色。
hue的功能可以不断被丰富，有网络平台向开发者们开放API接口和软件开发工具包SDK，新的应用方案不断诞生；而用户也可以通过该平台分享其个性化的灯光配方，已经有超过40个新的应用程序被开发出来。这些程序让hue与更多的设备和应用程序相联：有的能够用灯光的变化来显示心跳；有的能够让人们通过声音来控制灯光；有的还能和电视屏幕的变化同步。
本实施例则通过多媒体内容自动的驱动智能家居控制系统，比如上述飞利浦hue。在使用的多媒体内容中添加相应的标签，每次播放这些多媒体内容时这些标签都会驱动智能设备工作。具体说明如下，如图5所示。
2.对媒体数据碎片进行标记
在异构网络传输协议Mpeg media transport(MMT)中对所有媒体数据以碎片化MPU格式进行封装，每个MPU时间长度大约为0.5秒，并且将多媒体音视频数据的视频video和音频audio分开封装。基于此媒体数据封装格式，提出可以按照如下几种方法对MPU进行标记。
a)对视频MPU按照关键帧主色调标记
在Philips hue系统中，可以通过图片的颜色来控制灯泡颜色的改变，所以可以提取视频MPU中关键帧的主色调并对其进行标记，利用MPUBox中预留的reserved字段7bit定 义color_tag，对于不同的颜色可以用不一样的color_tag，如下表所示。
Figure PCTCN2016073167-appb-000006
color_tag的取值，可以通过视频MPU中关键主色调分析而对color_tag进行赋值，MPUBox要加入所需新属性。或者通过算法自动对MPU的主色调提取赋值。目前本发明部分实施例中将reserved的7bit用于color_tag，将来如果需要更多bit位，可以增加扩展。
b)对视频MPU按照场景情节标记
在Philips hue系统中，可以根据生活中不同的氛围呈现不同的灯光模式，所以本实施例可以提取视频中某段场景的模式，例如是否属于浪漫或者激烈等等模式，进而对MPU进行标记。仿照上表，可以将MPUBox中的reserved字段定义为scene_tag，可以采用不同的算法提取场景中的模式。
Figure PCTCN2016073167-appb-000007
c)对音频MPU按照音调标记
在Philips hue系统中，可以利用音乐音调的高低控制灯光模式，所以本发明部分实施例可以提取音频MPU中的音调特征，进而对MPU进行标记。仿照上表，可以将MPUBox中的reserved字段定义为tone_tag，可以采样各种不同的算法提取音频数据的音调。
由于在媒体内容中加载了相应的scene_tag,tong_tag,当媒体内容在正常播放时，设备读取到相应的新属性后，会将此属性信息传送给照明的控制设备接口中。此时，照明设备将根据接收到的属性信息，解析出控制照明系统的指令和参数。从而实现灯光随着媒体内容的播放而实时调节变化。
这里以MPUBox为例，说明如何加入新的属性，以实现媒体内容与灯光的结合呈现。但是实际应用中，新属性可以根据实际需要加在其它信令位置。本发明上述实施例是以照明为例，说明传统的多媒体与智能家居如何结合。应用本发明的思路和方法，也可以扩展应用于其它智能家居，甚至智慧城市系统中。
实施例4
用户在观看多媒体内容时，往往不会对整段节目内容都感兴趣，可能只对节目中的某个人物或某个场景感兴趣。智能媒体传输系统可对多媒体内容进行不同角度的分类，即打上不同的标识，为用户的个性化需求提供了可能。服务商依据不同的标签，将不同的多媒体内容版本与不同的edit list对应起来，其对应的标识即为edit id。为使得用户识别服务提供的edit list对应的内容，需要传输相应的描述信息，保证用户的个性化需求。
以相同内容的分级呈现为例，对于某电影节目，假设其为asset1。制片方可以提供多个不同版本视频，按视频组织结构划分，可分为精简版、剪切版、完整版和扩展版等不同时长的视频版本。
为将多媒体内容的分级信息及其描述信息的对应关系呈现给用户，便于用户选择。本发明部分实施例通过在传输的信息中添加描述信息(多媒体内容特征信息或同一媒体资源的各版本的关联信息)等方式来实现个性化呈现，以以下三种方案为例进行介绍。
方案一：在已有的CI(Composition Information)信息的元素MediaSync中新增加description属性，用于介绍不同版本内容的描述信息，如下所示：
Figure PCTCN2016073167-appb-000008
CI信息主要包含view元素、area元素、MediaSync元素和Option元素。其中view元素提供对一个视觉区域进行布局变换的时域控制信息；area元素是view元素的子元素，即一个视觉view的某一部分，对应与CI搭配的HTML5文件中的某个div元素；MediaSync元素用于指明与CI搭配的HTML5文件的某个媒体资源；Option元素用于表明此部分的CI是一个可选项。
每一个CI元素可以拥有某些不同的属性，上表所涉及到的属性中，id属性是该元素的标识；style属性用于指定该元素的CCS样式；begin属性指示该条CI指令开始起作用的时间；end属性指定该条CI指令停止起作用的时间；refDiv属性指明该元素对应的HTML5文件中的div元素的标识；refId属性指明该元素对应的HTML5文件中的某元素的标识；mediaSrc属性指明媒体资源的地址；以及新增的description属性。
CI文件的功能是指导媒体资源的呈现，edit list是解析资源相关的信息，因此可以把edit list内容添加到现有的CI文件中。接收端解析CI文件的时候需要支持edit list信息的解析。CI文件中MediaSync元素用于指定媒体资源，于是可以在指定媒体资源的同时也对媒体资源添加描述信息。因此在MediaSync元素下增加description属性，用于描述媒体资源对应的内容描述信息。
上述程序中的媒体资源包括视频和音频，分别为asset1和asset2。服务端识别asset中 的媒体数据单元的标识并分类成不同的edit list，每个edit list由对应的edit id来识别，并生成相应的CI文件，其中包含传输的edit list和对应的媒体描述信息description。客户端解析收到的CI文件，根据用户需求选择相应的description，解析出edit id信息并请求其对应的媒体数据单元。
方案二：在CI文件中增加新的元素EditDesp进行描述。
Figure PCTCN2016073167-appb-000009
媒体资源的信息放在MediaSync中，在CI中新增与MediaSync同级别的新元素EditDesp，用于指明媒体资源中的所有关联内容的描述信息，edit元素为EditDesp元素的子元素，每一个edit元素表征一个种类的的个性化呈现描述信息。在edit元素下，新增edit_id属性和description属性，edit_id属性用于标识某一级别或者某一部分的媒体资源；description属性用于描述媒体资源的内容描述信息。
上述程序中的EditDesp元素中定义了媒体资源中的四种关联内容的描述，并写入其相应的edit_id。客户端解析收到的CI文件，根据用户需求选择相应的description，解析出edit id信息并请求其对应的媒体数据单元。服务器端对同样的媒体资源生成的CI文件是一致的且完整的，满足用户的个性化需求的同时，减少了服务器重复生成相关描述及对应的edit  id。
方案三：在信令中增加descriptor描述
Figure PCTCN2016073167-appb-000010
上表包含的对应元素的含义如下：
descriptor_tag–定义该descriptor类型的标签。
descriptor_length–定义该descriptor的长度。
edit_list_number–定义了媒体资源的关联内容，N1表示其关联内容的个数。
edit_id–定义了媒体资源的关联内容的各版本的标号。
edit_description_length–媒体资源级别描述信息的长度，以字节为单位。
edit_description_byte–具体描述信息中的一个字节。
服务端识别媒体资源中的媒体数据单元的标识并分类成不同的edit list，每个edit list由对应的edit id来识别。为了实现用户与系统的交互，识别传输的媒体资源并能根据需求选择相关的内容进行呈现，系统生成了descriptor信令。其中包含传输的edit list和对应的媒体描述信息description。客户端解析收到的信令信息，根据用户需求选择相应的description，解析出edit id信息并请求其对应的媒体数据单元。内容的相关的描述信息时，就可以通过把该descriptor加入asset_descriptors中来实现。
综上，实施例1、2是个性化呈现在DASH和MMT两种传输协议中的不同实现方法，将媒体内容分级成不同的level(即本实施例中的edit list)进行传输。实施例3则是不同level的具体应用。而用户不区分不同的edit list，用户只能明白其对应的描述信息，例如不同的edit list对应的不同的视频版本等，这就是本实施例不同于上述实施例之处。
为更加清楚的描述本发明提出的实现方案，如图4所示，按照上述方案二描述方法构造CI文件，举例描述一个具体的实现流程：由图4可以看出，当接收端登录VoD服务时，发送端会发送信令信息，接收端通过判断MPI表中的PI_content_type_length_byte段来接 收CI和HTML5文件，同时会更新生成本地维护的MPU存储数据库MySQL并解析CI和HTML5文件；根据CI中的控制信息和新增加的多媒体内容描述信息description，为用户提供个性化呈现的分级信息，当用户根据分级信息请求精简版视频时，接收端通过CI文件找出精简版与edit_list[0]对应，并且找出对应的mpu_seq_number媒体数据标识为1,9，此时客户端请求这些MPU并存储在本地播放给用户，并且更新本地数据库。当用户接着请求剪切版视频时，通过检查CI文件和本地数据库，从发送端获取本地未存储的MPU，这些MPU是edit_list[1]所对应的MPU，其中mpu_seq_number媒体数据标识为4,6。接收端接收到4,6之后再与本地存储的1,9重组为剪切版视频。
实施例5
本实施例在实施例1、2的前提下，针对现有多媒体协议中呈现机制的不完备，充分考虑到点播服务的基于用户决策的呈现机制和广播、实时直播服务的推送机制，以及关联内容的呈现服务。对于同一媒体资源的关联内容版本，其中的每个媒体数据单元的呈现时间都不相同，通过提取媒体数据单元的持续时间，根据用户选择的版本和播放过程中的不同操作，在信令信息中的开始时间的基础上累加媒体数据单元的持续时间生成对应版本的媒体内容的呈现时间线；或是在信令生成的同时生成媒体数据单元的绝对呈现时间，根据用户的选择，生成对应的呈现时间线信息。
具体的，服务器通过将媒体内容与媒体描述的映射表传输给用户，即给用户提供关联性的媒体内容的选项。用户可根据自己的需求请求相同内容的不同版本，或是请求某一媒体资源的关联内容等多种呈现形式。为了节省存储资源，实现差异化传输，同一套媒体资源的不同版本中包含有共用的数据单元。但是现有的系统无法对不同版本的内容的呈现时间进行控制，可能导致媒体播放出现空缺的片段，影响用户体验。利用现有的系统提供了每个数据单元的持续时间信息，考虑到不同传输网络的特性，例如，VoD(Video on Demand)是根据用户选择节目的时间来确定媒体资源的初始呈现时间，而广播和实时直播则是在规定的时间在各个设备上同步呈现，但仍需要考虑随机接入的问题和实时直播中的实时性。因此，本实施例中以三种情况下的传输网络为例介绍不同的个性化呈现时间线控制机制。
应用一：VoD
在VoD服务中，用户选择了某个多媒体内容后，服务器对该需求进行响应，根据用户选择的版本读取对应的媒体数据单元的持续时间，生成相应的指导呈现信息文件和信令信息。
以相同内容的分级呈现为例，对于某电影节目，假设其为asset1。第一个版本视频为精简版片花，时长5分钟，里面只包括电影中的精彩画面；第二个版本视频为剪切版，时长30分钟，里面只包括电影故事情节重要的场景；第三个版本视频为完整版，时长120分钟，里面包括完整的故事情节；第四个版本为扩充版，时长150分钟，里面包括除完整的故事情节之外的扩充内容，如幕后花絮等等，如图1所示。
应用二：广播节目
在广播服务中，服务器按照预定的节目列表传送媒体数据流。考虑到用户随机接入问题，服务器需要将呈现相关的信息轮播。用户可以在接入时依据接收到的当前的呈现时间线开始观看媒体内容。同时，根据观看设备的状态如电量等级等指标实时地切换观看的模式，在适应设备状态的情况下提供更佳的用户体验。
以通过广播传送的球赛节目为例，当用户在移动设备上观看球赛时，需考虑设备的剩余电量。在生成媒体数据单元时，根据媒体数据单元的重要性进行分类，如精彩瞬间和进球等等镜头是该节目的不同的关联内容，分别赋予不同的标识，如图6所示。用户的电量情况大致分为满电量，中等电量和低电量的等级。
当设备的剩余电量达到相应的等级，则发送相应的请求给服务器，服务器自动切换下发的视频媒体内容和完整的音频内容，并根据传送的时间线控制媒体内容的同步。在传送部分视频媒体内容的情况下，无法解析视频数据单元内部的时间信息。因此，通过解析相应的时间线相关的信令获得时间信息，实现适应设备状况的节目呈现。
应用三：实时直播
在实时直播的服务中，媒体数据录制处理之后直接传送至客户端，因此对实时性的要求非常高。为了保证在实时直播中用户的个性化需求也能实现，对于媒体资源的关联内容都需要快速生成独立的时间线进行控制。
例如，在观看实时直播时，大部分用户也希望通过多视角的服务观看节目内容(如图7所示)。为了节约网络带宽并提供稳定实时的多视角服务，服务提供的内容均通过广播网传输给用户，同时传输该媒体资源的所有关联内容的呈现时间线信息。由于各关联内容的总时长和包含的媒体数据单元的呈现时间均不同，所以需要生成多套不同的时间线进行呈现控制。
本发明将媒体资源分为独立可解的媒体单元，利用媒体单元的关联关系，依据用户选择的不同的视频版本自动生成相应的呈现时间线。因此，本发明的呈现机制与现有技术相比更为灵活。
以下对本发明的部分具体实施例进行详细说明。
用户在观看媒体内容时，往往不会对整段节目内容都感兴趣，可能只对节目中的某个人物或某个场景感兴趣。智能媒体传输系统可对媒体内容进行不同角度的分类，即打上不同的标签，为用户的个性化需求提供了可能。在个性化呈现时，不同版本的关联内容包含共用的媒体数据单元，但这些数据单元在各个版本中的呈现时间不同，因此，需要针对每个版本生成不同的呈现时间线来控制播放。
在上述的例子中，每个版本中的MPU(media processing units)媒体数据单元的呈现时间如图8所示。其中，duri表示第i个MPU的持续时间。如图所示，在一个媒体资源中，相同MPU的呈现时间不同。因此，对同一媒体资源的不同关联版本都需要独立的时间线以指导呈现。
为了保证个性化服务中的用户体验，给不同的媒体资源或是同一媒体资源的不同版本的内容提供相应的呈现时间线，使得媒体内容能够依据时间线逐一呈现用户选择的内容。本发明通过增加新的descriptor或新增其他指导信息如message、table等信令信息，来实现呈现时间线的传输，并以以下三种方案为例进行介绍。
方案一：
采用MMT中定义的MPU timestamp descriptor。该descriptor中标记了MPU在一个媒体资源中对应的标号mpu_sequence_number和对应的UTC绝对呈现时间；该descriptor的语法结构如下表所示。
Figure PCTCN2016073167-appb-000011
descriptor_tag–定义该descriptor类型的标签。
descriptor_length–定义该descriptor的长度。
mpu_sequence_number–按顺序列举了对应的媒体资源中包含的所有MPU的标号，N表示MPU的个数。
mpu_presentation_time–列举了该descriptor对应的媒体资源包含的所有MPU的UTC绝对呈现时间。
在传输媒体内容的过程中，根据用户选择的媒体内容，选择其对应的媒体数据单 元MPU，解析其持续时间信息duration。若选择媒体资源的关联内容，则各关联内容版本所包含的MPU或有重复，即同一MPU在不同关联内容版本中可能有不同的绝对呈现时间，则服务器根据用户选择的版本获取相应的MPU的持续时间信息。结合用户选择的媒体播放时间或是系统规定的媒体播放时间，通过累加指定MPU之前的所有MPU的持续时间，计算出每个MPU对应的UTC绝对呈现时间，即为descriptor中的mpu_presentation_time。
现有MMT协议定义的信令消息中定义了一些描述性的descriptor，例如dependency descriptor和MPU_timestamp_descriptor，因此，可以定义一个提供媒体内容和对应呈现相关的时间信息的descriptor。在MP table中有asset_descriptors字段，可以根据需要在asset_descriptors中加入相关descriptor。在使用多媒体服务时，可以将该descriptor加入asset_descriptors中来实现呈现。
但是，点播服务是用户主导的，需要考虑用户在观看过程中的操作，例如暂停和快进等。在这种情况下，若服务器只提供了对应的每个MPU的UTC绝对呈现时间，则无法正确地继续播放，需要服务器重新生成呈现时间线信息再进行传输，造成了较大的计算负担和冗余量，还会造成一定的延时而影响用户体验。在广播服务中采用UTC绝对呈现时间即mpu_presentation_time是一种便捷的方式，因为选定的媒体内容在直播服务中的播放时间是固定的。用户接收某一媒体资源并选择其关联服务，同时接收对应的MPU_timestamp_descriptor，各关联部分的内容的媒体数据单元依据描述子中的时间信息在固定的时间呈现即可。
方案二：
定义MPU timestamp descriptor。根据同一媒体资源的关联内容及其对应的MPU的集合定义edit list，给各个版本的关联内容赋予独立的edit id。在descriptor中描述每个edit list中包含的所有MPU的mpu_sequence_number和对应的呈现时间信息；该descriptor的语法结构如下表所示。
Figure PCTCN2016073167-appb-000012
Figure PCTCN2016073167-appb-000013
descriptor_tag–定义该descriptor类型的标签。
descriptor_length–定义该descriptor的长度。
edit_list_number–定义了媒体资源的关联内容，N1表示其个数。
edit_id–定义了媒体资源的关联内容的各版本的标号。
mpu_sequence_number–按顺序列举了对应的媒体资源中包含的所有MPU的标号，N表示MPU的个数。
mpu_presentation_time–列举了该descriptor对应的媒体资源包含的所有MPU的UTC绝对呈现时间。
在传输媒体内容的过程中，根据用户选择的媒体内容，将其选择的媒体资源的所有关联内容的UTC绝对呈现时间mpu_presentation_time都写在该descriptor中(时间获得方式参照方案一)。在消费媒体内容的过程中，服务器随信令信息发送上述descriptor给客户端。用户选择某一版本的关联内容，根据其对应的edit_id解析出其中的媒体数据单元MPU和对应的绝对呈现时间mpu_presentation_time，生成对应版本的时间线来控制呈现。通过这样的方法可以较方便的获得各个关联版本的媒体内容的呈现时间，且个性化的用户需求均通过相同的descriptor信息来指导呈现，更便于控制。
方案三：
在信令中描述某个媒体资源中的每个MPU的持续时间，并由CI(Composition Information)中得知该媒体资源的开始时间，可计算得到每个MPU的UTC绝对呈现时间。
Figure PCTCN2016073167-appb-000014
descriptor_tag–定义该descriptor类型的标签。
descriptor_length–定义该descriptor的长度。
mpu_sequence_number–按顺序列举了对应的媒体资源中包含的所有MPU的标 号，N表示MPU的个数。
mpu_duration–列举了该descriptor对应的媒体资源包含的所有MPU的UTC绝对呈现时间。
在传输媒体内容的过程中，根据用户选择的媒体内容，选择其对应的媒体数据单元MPU，解析其持续时间信息duration，同时生成指导呈现信息，即MPU的标号mpu_sequence_number和对应的duration信息。
考虑到各种不同的传输网络状况，此种方案可以在生成并封装媒体数据单元的同时，获取其对应的持续时间duration信息，因此可以较好地满足实时性需求，即可满足实时直播的应用需求。通过传输duration信息取代绝对呈现时间，使得客户端能够更加灵活地自组织媒体内容。同时，在宽带点播的业务中，也可支持用户随时进行输入操作，满足用户的个性化需求。
以上三种方案可涵盖包括宽带网和广播网甚至异构网络等多个多媒体传输系统，我们还可以通过在呈现指导文件CI中或是传输信令信息的其他位置加入相应的呈现时间线，同样可以实现个性化的呈现服务。
在控制呈现的过程中，由于广播和实时直播服务均是从用户当前的接入时间开始播放。为了节约带宽资源以及客户端的存储资源及计算消耗，在上述的descriptor中只写入还未消费的MPU的呈现时间信息或是duration信息(以下将这两类时间信息统称为相关时间信息)，而不传输媒体资源对应的所有MPU的相关时间。通过此种方式，虽然增加了生成信令信息的复杂度，但大大节约了传输网络带宽和客户端有限的计算能力。此外，由于点播服务不存在随机接入问题，因此在用户开始服务的时候可相应的传输有限个数的MPU的相关时间信息，并根据用户的观看进度及时传输后续的MPU的相关时间信息。为保证观看的流畅性，需要及时响应观看中用户的操作时，传输的相关时间信息，更新呈现时间线。
为更加清楚的描述本发明提出的实现方案，下图按照上文中方案三的描述方法生成MPU_presentation_descriptor，构造VoD服务中的个性化的呈现时间线机制，举例描述一个具体的实现流程：
如图4所示，以上述的方案三为例进行介绍，提供一种多媒体内容个性化呈现的时间线控制方法，该方法流程具体为：
当接收端请求VoD服务时，发送端将发送信令信息，接收端通过判断MPI表中的PI_content_type_length_byte字段来接收MUR、CI和HTML文件，同时会更新生成本地维 护的MPU存储数据库MySQL；当用户请求精简版视频时，接收端通过查询关联媒体内容的edit_list信令信息获得与精简版视频对应的edit_list的标识edit_id＝00，其中包含的媒体单元的mpu_seq_number为1,9，此时接收端请求edit_id＝00的媒体内容。发送端解析请求获得对应的mpu_seq_number并解析媒体数据单元MPU获得其对应的持续时间duration信息，生成MPU_presentation_descriptor信令。接收端收到相应信令，根据MPU_presentation_descriptor和CI中的开始时间生成每个MPU的绝对呈现时间，维护呈现时间线。同时接收媒体数据存储在本地播放给用户，并且更新本地数据库。当用户接着请求剪切版视频时，剪切版视频的edit_id＝00U01。通过检查关联媒体内容的edit_list信令和本地数据库存在的MPU，向发送端请求edit_id＝01的媒体资源。接收端收到信令和媒体数据之后解析并获得资源中的媒体数据单元mpu_seq_number，并根据MPU_presentation_descriptor信令中的duration信息重新计算剪切版视频所包含的MPU的呈现时间，更新呈现时间线。同时接收mpu_seq_number为4,6的MPU储存到本地。
本实施例在满足用户个性化需求的同时，利用同一套节目的关联性，在节约存储空间的基础上，保证用户的流畅的观看体验。由于媒体的呈现时间线信息随信令信息灵活地发送，针对消费媒体资源的过程中出现的延时及丢包造成用户体验降低的问题，若在呈现时间之前检测到丢包事件，可等待媒体资源的重新获取，或是重复呈现前一个媒体数据单元的内容以防止播放卡顿，保证用户体验。
本发明上述部分实施例中所述以MMT为例，阐明所提出的解决方案，但是这些方案同样也可以用于其它文件封装，传输系统和协议里。
以上对本发明的具体实施例进行了描述。需要理解的是，本发明并不局限于上述特定实施方式，本领域技术人员可以在权利要求的范围内做出各种变形或修改，这并不影响本发明的实质内容。

Claims (31)

  1. 一种关联多媒体内容个性化呈现的实现方法，其特征在于，所述方法具体为：多媒体内容提供商在制作多媒体文件时统一完成完整的文件内容，同时采用标记方法依据内容的重要性和/或关联程度对其中的片段进行分级，从而产生不同版本的多媒体文件；用户根据这些版本进行选择性预览和/或播放。
  2. 根据权利要求1所述的关联多媒体内容个性化呈现的实现方法，其特征在于，内容提供商在多媒体文件呈现界面提供与片段分级相关的版本信息，以方便用户进行内容选择消费。
  3. 根据权利要求1或2所述的关联多媒体内容个性化呈现的实现方法，其特征在于，所述方法具体包括如下步骤：
    第一步，通过内容分级的版本管理将多媒体文件分为多个版本；
    第二步，生成与版本相关的差异化MPD文件：根据上述多媒体文件的多个版本，生成与内容分级相对应的MPD文件，即内容分级的版本管理标签；
    第三步，用户根据自己的需求请求不同版本的多媒体文件内容，服务器传输统一的MPD文件，客户端解析MPD之后根据自己的网络状态、设备性能以及请求的版本情况，向服务器请求相应的媒体分片内容。
  4. 根据权利要求3所述的关联多媒体内容个性化呈现的实现方法，其特征在于，上述多媒体文件中的多个版本对应MPD中segment列表，在生成这些segment时，自动生成相应匹配内容版本的MPD文件，其中MPD元素增加@scaleID属性，该属性定义了此MPD文件描述的内容分级的级别；此外Representation元素中SegmentList子元素中的segment列表不再是单纯的全部分片列表，而是对应于MPD@scaleID的特定分片列表。
  5. 根据权利要求1或2所述的关联多媒体内容个性化呈现的实现方法，其特征在于，所述方法具体包括如下步骤：
    第一步，通过内容分级的版本管理将多媒体文件分为多个版本；
    第二步，对应于多个版本，只有一个相对应的MPD文件被生成，但MPD文件的segment描述增加了@scaleID属性，即内容分级的版本管理标签；
    第三步，用户根据自己的需求请求不同版本的多媒体文件内容，服务器传输统一的MPD文件，客户端解析MPD之后根据自己的网络状态、设备性能以及请求的版本情况，向服务器请求相应的媒体分片内容。
  6. 根据权利要求5所述的关联多媒体内容个性化呈现的实现方法，其特征在于，上述不同版本的多媒体文件对应MPD文件的segment列表，在生成这些segment时，在MPD文件的Representation元素的SegmentList子元素中的每一个segment提供@scaleID属性，该属性标注了使用此segment的最低版本级别。
  7. 一种多媒体内容分级技术的实现方法，其特征在于，在制作多媒体文件时对多媒体内容的重要片段进行标记，根据多媒体文件内容的重要程度将每个片段划分不同的等级；在用户观看时，根据这些等级对多媒体内容进行选择性播放，而不是盲目的随机选择；新定义一个MUR信息文件，用于描述多媒体文件与不同版本内容、mpu_seq_number、level_list之间的对应关系，mpu_seq_number是MPU中的标记字段，level_list是mpu_seq_number的集合，与不同版本的视频是一一对应关系。
  8. 根据权利要求7所述的多媒体内容分级技术的实现方法，其特征在于：对于某一多媒体内容，将其切成多个MPU碎片，内容分级成不同版本后与mpu_seq_number之间存在对应关系；新定义一个MUR信息文件，用于描述多媒体文件与不同版本内容、mpu_seq_number、level_list之间的对应关系，其中根据level_list数组与不同版本内容时间的对应规则不一样，分为两种类型：
    类型1中，不同版本的内容与level_list之间是一一对应的；
    类型2中，不同版本的内容是不同level_list的组合；
    根据类型1和类型2中MUR信息文件的内容对多媒体文件进行差异化传输。
  9. 根据权利要求8所述的多媒体内容分级技术的实现方法，其特征在于：所述类型1中：level_list由不同的MPU组合而成，且与不同视频版本之间一一对应；当用户需要更多的片段时，server仅需传输用户缺失的MPU；
    所述类型2中：level_list由不同的MPU组合而成，并且每个level_list[]之间没有重叠的MPU出现，不同版本是由不同的level_list组合而成；当用户观看完剪切版视频之后想看完整版视频时，只需要差异化传输特定level_list[]所包含的MPU碎片。
  10. 根据权利要求7-9任一项所述的多媒体内容分级技术的实现方法，其特征在于，所述MUR信息文件包含了内容分级的重要信息，通过将MUR信息加入信令信息中的方法传输这些重要信息。
  11. 根据权利要求10所述的多媒体内容分级技术的实现方法，其特征在于，所述将MUR信息加入信令信息中的方法，是指：将MUR信息加入到现有的CI文件中；或者在信令信息中增加一个描述MUR信息的MUR文件；或者增加一个描述MUR信息的 descriptor，descriptor是MMT中信令部分用于定义一些字段或者功能的描述性信息；或者增加一个描述MUR信息的信令表。
  12. 一种应用权利要求1-11所述方法的智能家居控制方法，其特征在于：所述方法通过基于视频中关键帧的图像主色调和/或场景情节特征对视频碎片进行标记，或者依据音频数据特征进行标记，添加相应的标签，每次播放这些多媒体内容时，标记的这些标签都会驱动智能家居设备工作。
  13. 根据权利要求12所述的智能家居控制方法，其特征在于，所述通过基于视频中关键帧的图像主色调对视频碎片进行标记，具体为：提取视频MPU中关键帧的主色调并对其进行标记，利用MPUBox中预留的reserved字段定义color_tag，对于不同的颜色可以用不一样的color_tag。
  14. 根据权利要求13所述的智能家居控制方法，其特征在于，通过视频MPU中关键主色调分析对color_tag进行赋值，或者通过算法自动对MPU的主色调提取赋值。
  15. 根据权利要求13所述的智能家居控制方法，其特征在于，将MPUBox中的reserved的7bit用于color_tag，该bit位根据需要可扩展。
  16. 根据权利要求12-15任一项所述的智能家居控制方法，其特征在于，所述通过基于视频中场景情节特征对视频碎片进行标记，具体为：提取视频中某段场景的模式，进而对MPU进行标记，将MPUBox中的reserved字段定义为scene_tag。
  17. 根据权利要求12-15任一项所述的智能家居控制方法，其特征在于，所述依据音频数据特征进行标记，具体为：提取音频MPU中的音调特征，进而对MPU进行标记，将MPUBox中的reserved字段定义为tone_tag。
  18. 一种用于权利要求1或7分级后内容传输的关联多媒体内容个性化呈现信息的描述方法，其特征在于，所述方法为：媒体资源的关联多媒体内容用不同的edit list(编辑列表)表示，不同的edit list中所含的媒体数据单元是补充关系或包含关系；为了将媒体资源的分类及其描述信息的对应关系呈现给用户，在传输的信息中添加对媒体资源特征信息或关联信息的描述，来实现个性化呈现机制。
  19. 根据权利要求18所述的关联多媒体内容个性化呈现信息的描述方法，其特征在于，所述对多媒体内容特征信息或关联信息的描述，具体为：在已有的CI(Composition Information)元素MediaSync中新增加description属性，description用于描述媒体资源对应的内容描述信息。
  20. 根据权利要求18所述的关联多媒体内容个性化呈现信息的描述方法，其特征在 于，所述对多媒体内容特征信息或关联信息的描述，具体为：在CI文件中增加新的元素EditDesp，用于介绍媒体资源中的不同关联内容的描述信息。
  21. 根据权利要求20所述的关联多媒体内容个性化呈现信息的描述方法，其特征在于，多媒体内容的信息放在MediaSync中，在CI中增加与MediaSync同级别的新元素EditDesp，专用于指明不同级别的媒体资源的描述信息；同时edit元素为EditDesp元素的子元素，每一个edit元素表征一个级别的个性化呈现描述信息；在edit元素下，新增edit_id属性和description属性，edit_id属性用于标识某一级别或者某一部分的媒体资源；description属性用于描述媒体资源中的不同关联内容的描述信息。
  22. 根据权利要求18所述的关联多媒体内容个性化呈现信息的描述方法，其特征在于，所述对多媒体内容特征信息或关联信息的描述，具体为：在信令中增加descriptor描述，description用于描述媒体资源中的不同关联内容的描述信息。
  23. 根据权利要求18所述的关联多媒体内容个性化呈现信息的描述方法，其特征在于，在MMT协议定义的信令信息中新定义一个descriptor，该descriptor提供关联多媒体内容和对应的描述信息，在MP table中有asset_descriptors字段，当用户某节目内容的相关的描述信息时，通过把该descriptor加入asset_descriptors中来实现。
  24. 根据权利要求18-23任一项所述的关联多媒体内容个性化呈现信息的描述方法，其特征在于，所述方法实现步骤为：
    定义同一媒体资源所包含的媒体数据单元的标识或对应的描述信息，生成descriptor信令信息，其中包含传输的媒体资源中的各类多媒体内容描述信息；
    客户端解析descriptor信令信息，用户根据需求选择具有相应标识或信息表示的多媒体内容，并请求相应的媒体数据单元组合成对应的edit list；
    服务端解析用户请求并发送获取相应的媒体数据单元，客户端解析信息并实现个性化呈现。
  25. 一种用于权利要求1或7的多媒体内容个性化呈现的时间线控制方法，其特征在于，所述方法具体为：在多媒体资源呈现的过程中，利用媒体数据单元的持续时间信息和呈现的初始时间，对关联内容的呈现时间进行控制，使得不同的媒体资源或是同一媒体资源的不同关联版本都有对应的呈现时间线。
  26. 根据权利要求25所述的一种多媒体内容个性化呈现的时间线控制方法，其特征在于，所述方法包括：
    S1：根据用户选择的媒体资源的关联内容版本所包含的媒体数据单元，对每个媒体数 据单元获取其相应的持续时间信息；
    S2：在VoD中根据用户选择媒体资源时的时间确定开始播放时间，在广播服务中由服务提供商确定开始播放时间，在信令信息中标识开始播放时间；
    S3：根据S2中的开始播放时间和对应关联内容版本中的媒体数据单元持续时间的累加得出对应媒体数据单元的绝对呈现时间，据此维护相应的呈现时间线来指导媒体内容的呈现。
  27. 根据权利要求26所述的多媒体内容个性化呈现的时间线控制方法，其特征在于，S1中，所述关联内容版本，是指：根据用户需求对媒体资源进行标识，根据其标识的edit id生成不同的关联版本内容，对应不同的edit list，即权利要求9所述的level list。
  28. 根据权利要求25所述的多媒体内容个性化呈现的时间线控制方法，其特征在于，所述呈现时间线，是指：所选中的媒体内容包含的媒体数据单元的绝对呈现时间信息的组合。
  29. 根据权利要求25-28任一项所述的多媒体内容个性化呈现的时间线控制方法，其特征在于，采用MMT中定义的MPU timestamp descriptor，descriptor中标记了MPU在一个媒体资源中对应的标号mpu_sequence_number和对应的UTC绝对呈现时间；当用户消费一个媒体资源的关联内容版本时，服务器根据用户请求的关联版本，获取该版本中包含的媒体数据单元的标号和对应的持续时间信息，生成相应的descriptor。
  30. 根据权利要求25-28任一项所述的多媒体内容个性化呈现的时间线控制方法，其特征在于，定义MPU timestamp descriptor，根据同一媒体资源的关联内容及其对应的MPU的集合定义edit list，给各个版本的关联内容赋予独立的edit id，在descriptor中描述每个edit list中包含的所有MPU的mpu_sequence_number和对应的呈现时间信息；用户选择某一版本的关联内容，根据对应的edit_id解析出其中的媒体数据单元MPU和对应的绝对呈现时间mpu_presentation_time，生成对应版本的时间线来控制呈现。
  31. 根据权利要求25-28任一项所述的多媒体内容个性化呈现的时间线控制方法，其特征在于，在信令中描述某个媒体资源中的每个MPU的持续时间，并由信令信息中得知该媒体资源的开始时间，计算得到每个MPU的UTC绝对呈现时间；在传输媒体内容的过程中，根据用户选择的媒体内容，选择其对应的媒体数据单元MPU，解析其持续时间信息duration，同时生成指导呈现信息，即MPU的标号mpu_sequence_number和对应的duration信息。
PCT/CN2016/073167 2015-02-13 2016-02-02 一种关联多媒体内容个性化呈现的实现方法及应用 Ceased WO2016127862A1 (zh)

Priority Applications (5)

Application Number Priority Date Filing Date Title
KR1020177024204A KR101988454B1 (ko) 2015-02-13 2016-02-02 멀티미디어 컨텐츠 개성화 프레젠테이션의 실현 방법 및 응용
CA3004644A CA3004644C (en) 2015-02-13 2016-02-02 Implementing method and application of personalized presentation of associated multimedia content
JP2017541331A JP6567064B2 (ja) 2015-02-13 2016-02-02 関連マルチメディアコンテンツのカスタマイズ表示の実現方法およびその応用
KR1020197003889A KR102378107B1 (ko) 2015-02-13 2016-02-02 멀티미디어 컨텐츠 개성화 프레젠테이션의 실현 방법 및 응용
US15/550,789 US10433029B2 (en) 2015-02-13 2016-02-02 Implemental method and application of personalized presentation of associated multimedia content

Applications Claiming Priority (10)

Application Number Priority Date Filing Date Title
CN201510080011.X 2015-02-13
CN201510080580.4A CN105992061B (zh) 2015-02-13 2015-02-13 一种自适应动态的多媒体分级传送播放管理方法
CN201510080580.4 2015-02-13
CN201510080011.XA CN105988369B (zh) 2015-02-13 2015-02-13 一种内容驱动的智能家居控制方法
CN201510401550.9A CN106341740B (zh) 2015-07-09 2015-07-09 一种多媒体内容分级技术的实现方法
CN201510401550.9 2015-07-09
CN201510955611.6A CN106899866B (zh) 2015-12-17 2015-12-17 一种关联多媒体内容个性化呈现信息的描述方法
CN201510955611.6 2015-12-17
CN201610031034.6 2016-01-18
CN201610031034.6A CN106982376B (zh) 2016-01-18 2016-01-18 一种多媒体内容个性化呈现的时间线控制方法

Publications (1)

Publication Number Publication Date
WO2016127862A1 true WO2016127862A1 (zh) 2016-08-18

Family

ID=56615455

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/073167 Ceased WO2016127862A1 (zh) 2015-02-13 2016-02-02 一种关联多媒体内容个性化呈现的实现方法及应用

Country Status (5)

Country Link
US (1) US10433029B2 (zh)
JP (1) JP6567064B2 (zh)
KR (2) KR102378107B1 (zh)
CA (1) CA3004644C (zh)
WO (1) WO2016127862A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114007122A (zh) * 2021-10-13 2022-02-01 深圳Tcl新技术有限公司 一种视频播放方法、装置、电子设备和存储介质

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2963892A1 (en) * 2014-06-30 2016-01-06 Thomson Licensing Method and apparatus for transmission and reception of media data
KR102126257B1 (ko) * 2015-02-13 2020-06-24 에스케이텔레콤 주식회사 멀티뷰 스트리밍 서비스 지원 방법 및 이를 지원하는 장치
WO2017182365A1 (en) * 2016-04-22 2017-10-26 Philips Lighting Holding B.V. Controlling a lighting system
US20170344523A1 (en) * 2016-05-25 2017-11-30 Samsung Electronics Co., Ltd Method and apparatus for presentation customization and interactivity
GB201714592D0 (en) * 2017-09-11 2017-10-25 Provost Fellows Found Scholars And The Other Members Of Board Of The College Of The Holy And Undivid Determining representative content to be used in representing a video
US10771842B2 (en) * 2018-04-09 2020-09-08 Hulu, LLC Supplemental content insertion using differential media presentation descriptions for video streaming
WO2019238834A1 (en) * 2018-06-15 2019-12-19 Signify Holding B.V. Method and controller for selecting media content based on a lighting scene
US10791376B2 (en) 2018-07-09 2020-09-29 Spotify Ab Media program having selectable content depth
US10419786B1 (en) * 2018-07-20 2019-09-17 Fubotv Inc. Systems and methods for securely generating live previews
CN112740713B (zh) * 2018-09-21 2023-08-22 三星电子株式会社 用于提供多媒体内容中的关键时刻的方法及其电子设备
US10924776B2 (en) * 2019-03-27 2021-02-16 Rovi Guides, Inc. Systems and methods for media content navigation and filtering
US10897642B2 (en) 2019-03-27 2021-01-19 Rovi Guides, Inc. Systems and methods for media content navigation and filtering
CN113114608B (zh) * 2020-01-10 2022-06-10 上海交通大学 点云数据封装方法及传输方法
CN111259257A (zh) * 2020-02-14 2020-06-09 北京达佳互联信息技术有限公司 一种信息展示方法、系统、装置、电子设备及存储介质
US11570517B2 (en) 2020-06-23 2023-01-31 Tencent America LLC Application intended interactive selection information for interactive playback of dash content
CN114661386B (zh) * 2020-12-22 2024-11-22 腾讯科技（深圳）有限公司 点云视窗的呈现方法、装置、计算机可读介质及电子设备
CN115150368B (zh) * 2021-03-31 2023-11-03 腾讯科技（深圳）有限公司 媒体文件的关联处理方法、装置、介质及电子设备
CN115278326B (zh) * 2021-04-29 2025-04-25 腾讯科技（深圳）有限公司 视频展示方法、装置、计算机可读介质及电子设备
CN120343222A (zh) * 2021-07-12 2025-07-18 腾讯科技（深圳）有限公司 容积媒体的数据处理方法、装置、设备以及存储介质
CN116112473B (zh) * 2021-11-10 2025-10-31 腾讯科技（深圳）有限公司 媒体文件处理方法、装置及电子设备
US20230221768A1 (en) * 2022-01-13 2023-07-13 Asustek Computer Inc. Electronic device and screen control method thereof

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102217313A (zh) * 2011-05-26 2011-10-12 华为技术有限公司 重排、抽取分片中媒体数据的方法、设备及系统
CN102447818A (zh) * 2010-09-30 2012-05-09 上海维阔信息技术有限公司 视频版权保护和分级观看的播放方法
CN103491457A (zh) * 2012-06-11 2014-01-01 Rgb网络有限公司 用于组合用于流式视频信号的资产的方法及设备
CN103974147A (zh) * 2014-03-07 2014-08-06 北京邮电大学 一种基于mpeg-dash协议的带有码率切换控制和静态摘要技术的在线视频播控系统
US20140282792A1 (en) * 2013-03-15 2014-09-18 Cygnus Broadband, Inc. Video streaming with buffer occupancy prediction based quality adaptation
CN104093240A (zh) * 2014-06-30 2014-10-08 广东九联科技股份有限公司 一种智能调节环境灯光的系统
US20140366070A1 (en) * 2013-06-10 2014-12-11 Samsung Electronics Co., Ltd. Method and apparatus for assigning video bitrate in mobile communication system

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000350182A (ja) * 1999-06-01 2000-12-15 Ricoh Co Ltd 動画配信システム
US20030219708A1 (en) * 2002-05-23 2003-11-27 Koninklijke Philips Electronics N.V. Presentation synthesizer
PT3413558T (pt) * 2005-12-29 2022-03-14 Rovi Guides Inc Um sistema interativo de orientação de média com múltiplos dispositivos
US9386064B2 (en) * 2006-06-09 2016-07-05 Qualcomm Incorporated Enhanced block-request streaming using URL templates and construction rules
KR100820350B1 (ko) * 2007-02-08 2008-04-08 (주)센텍 다양한 파일 컨테이너 포멧을 지원하기 위한 통합 스트리밍서버 및 스트리밍 구현방법
KR20100008777A (ko) * 2008-07-16 2010-01-26 한국전자통신연구원 실감 효과 표현 방법 및 그 장치 및 실감 기기 제어 메타데이터가 기록된 컴퓨터로 읽을 수 있는 기록 매체
KR20100008775A (ko) * 2008-07-16 2010-01-26 한국전자통신연구원 실감 효과 표현 방법 및 그 장치 및 사용자 환경 정보 메타데이터가 기록된 컴퓨터로 읽을 수 있는 기록 매체
JP2010225003A (ja) * 2009-03-25 2010-10-07 Kddi Corp コンテンツ受信再生装置、コンテンツ受信再生方法、およびプログラム
KR101180127B1 (ko) * 2009-11-30 2012-09-05 이두한 멀티미디어파일 재생방법 및 멀티미디어 재생기기
JP2011029948A (ja) * 2009-07-27 2011-02-10 Hitachi Consumer Electronics Co Ltd コンテンツ処理装置
US9602849B2 (en) * 2010-09-17 2017-03-21 Futurewei Technologies, Inc. Method and apparatus for scrub preview services
KR20120119790A (ko) * 2011-04-22 2012-10-31 삼성전자주식회사 미디어 데이터 전송 방법 및 장치와 미디어 데이터 수신 방법 및 장치
JP5921852B2 (ja) * 2011-10-21 2016-05-24 シャープ株式会社 配信装置
US9380282B2 (en) * 2012-03-26 2016-06-28 Max Abecassis Providing item information during video playing
GB2512310A (en) * 2013-03-25 2014-10-01 Sony Corp Media Distribution

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102447818A (zh) * 2010-09-30 2012-05-09 上海维阔信息技术有限公司 视频版权保护和分级观看的播放方法
CN102217313A (zh) * 2011-05-26 2011-10-12 华为技术有限公司 重排、抽取分片中媒体数据的方法、设备及系统
CN103491457A (zh) * 2012-06-11 2014-01-01 Rgb网络有限公司 用于组合用于流式视频信号的资产的方法及设备
US20140282792A1 (en) * 2013-03-15 2014-09-18 Cygnus Broadband, Inc. Video streaming with buffer occupancy prediction based quality adaptation
US20140366070A1 (en) * 2013-06-10 2014-12-11 Samsung Electronics Co., Ltd. Method and apparatus for assigning video bitrate in mobile communication system
CN103974147A (zh) * 2014-03-07 2014-08-06 北京邮电大学 一种基于mpeg-dash协议的带有码率切换控制和静态摘要技术的在线视频播控系统
CN104093240A (zh) * 2014-06-30 2014-10-08 广东九联科技股份有限公司 一种智能调节环境灯光的系统

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114007122A (zh) * 2021-10-13 2022-02-01 深圳Tcl新技术有限公司 一种视频播放方法、装置、电子设备和存储介质
CN114007122B (zh) * 2021-10-13 2024-03-15 深圳Tcl新技术有限公司 一种视频播放方法、装置、电子设备和存储介质

Also Published As

Publication number Publication date
US10433029B2 (en) 2019-10-01
KR20190016618A (ko) 2019-02-18
KR102378107B1 (ko) 2022-03-25
US20180041820A1 (en) 2018-02-08
JP2018509065A (ja) 2018-03-29
KR20170110112A (ko) 2017-10-10
KR101988454B1 (ko) 2019-06-12
CA3004644A1 (en) 2016-08-18
CA3004644C (en) 2021-03-16
JP6567064B2 (ja) 2019-08-28

Similar Documents

Publication Publication Date Title
WO2016127862A1 (zh) 一种关联多媒体内容个性化呈现的实现方法及应用
KR101885852B1 (ko) 컨텐트 전송 및 수신 방법 및 장치
CN103974147A (zh) 一种基于mpeg-dash协议的带有码率切换控制和静态摘要技术的在线视频播控系统
CN104081759A (zh) 接收设备，接收方法和程序
CN103493499A (zh) 提供使用表示的流媒体内容的装置及方法
JP2005503628A (ja) メタデータ処理装置
CN107534793B (zh) 接收装置、传输装置以及数据处理方法
US11410199B2 (en) Reception apparatus, transmission apparatus, and data processing method
CN102364956A (zh) 一种iptv系统的开机广告的实现方法
JP2014534695A (ja) コンテンツディスプレイ方法、コンテンツ同期化方法、放送コンテンツディスプレイ方法及びディスプレイ装置
CN100542250C (zh) 实现数字媒体组合播放的方法、装置及系统
CN102395053B (zh) 一种iptv系统
KR100809641B1 (ko) 이종 시스템간의 컨텐츠 교환 방법 및 그 방법을 수행하는컨텐츠 관리 시스템
CN106982376B (zh) 一种多媒体内容个性化呈现的时间线控制方法
CN101221752B (zh) Iptv系统实现个性卡拉ok业务的方法、装置及系统
CN105992061B (zh) 一种自适应动态的多媒体分级传送播放管理方法
CN105653530B (zh) 一种高效可伸缩的多媒体传送、存储和呈现方法
CN107770601A (zh) 一种面向多媒体内容组件个性化呈现的方法及系统
KR100687416B1 (ko) 컨텐츠의 재생 구간 정보를 이용한 멀티미디어 컨텐츠 제공시스템 및 방법
CN106899866B (zh) 一种关联多媒体内容个性化呈现信息的描述方法
Bomcke et al. An interactive video streaming architecture for H. 264/AVC compliant players
CA2981270C (en) Reception apparatus, transmission apparatus, and data processing method
WO2025230719A1 (en) Streaming multiple-perspective audio and video
CN100530101C (zh) 组件识别方法与使用该组件识别方法的定向方法
CN107018452A (zh) 多媒体服务中内容组件关系的描述及个性化显示方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16748663

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2017541331

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 15550789

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 20177024204

Country of ref document: KR

Kind code of ref document: A

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 14/12/2017)

ENP Entry into the national phase

Ref document number: 3004644

Country of ref document: CA

122 Ep: pct application non-entry in european phase

Ref document number: 16748663

Country of ref document: EP

Kind code of ref document: A1