WO2011144097A2 - Method, apparatus and system for rearrangement and extraction of media data in segment - Google Patents

Abstract

A method, an apparatus and a system for rearrangement and extraction of media data in segment are disclosed in the present invention which belongs to the field of media data processing. The rearrangement method includes: segmenting media content; during the segmenting process, continuously and physically storing media data fulfilled with a rearrangement attribute in the fragments of the segment, and composing sub-fragments corresponding to the rearrangement attribute, wherein, the media data is composed with samples or sub-samples; adding index information of the sub-fragments, wherein, the index information includes the rearrangement attribute information and location information of the sub-fragment; obtaining the index information of the sub-fragments, wherein the index information includes the rearrangement attribute information and location information of the sub-fragment; calculating the range of byte of the sub-fragments corresponding to a predetermined rearrangement attribute according to the index information; and requesting and receiving the sub-fragments corresponding to the predetermined rearrangement attribute according to the range of the byte. It enables the client to obtain the media data corresponding to the rearrangement attribute in the segments quickly and precisely.

Description

 Method, device and system for rearranging and extracting media data in slices

 The present invention relates to the field of media data processing, and in particular, to a method, a device and a system for rearranging and extracting media data in a slice.

 Say

Background technique

 With the rapid development of the Internet, the transmission of multimedia information (such as animation, audio and video, etc.) on the network is more and more frequent and important. Generally, the server compresses and encodes multimedia information through MVC (Multiview Video Coding), SVC (Scalable Video Coding), or AVC (Advanced Video Coding) encoding techniques. One or more video streams are stored in the multimedia file, and the multimedia file can be generated into different segments, so that the client selects and downloads the corresponding segment according to the actual situation, so as to improve the transmission efficiency of the multimedia file in the network.

 See Figure 1, which is a schematic diagram of the structure of the Segment. The Segment includes several Movie Fragments. Among them, Movie Fragment is composed of Moof (Movie Fragment box, collectively referred to as description structure) and Media Data (Media Data). Moof is used to describe the time information and storage byte size of Media Data. The storage unit of Media Data is Samples are stored one by one and stored at the decoding time. The Sample consists of a NAL (Network Abstraction Layer) representing a single video frame or a series of video frames in the video stream. In a Sample, several consecutive NAL units with the same specified attributes form a Sub-sample.

 When the client performs certain operations, it only needs to request some discretely stored media data in the fragment. For example, when performing fast forward operations, the client only needs to display media data at a certain rate (for example, 4 times). However, since these media data are discretely stored in the slice according to the decoding time. One way of processing is that the client requests the entire fragment from the server, and then proposes the media data that it needs. The method requests a large amount of redundant media data other than the media data required by the client, resulting in waste of bandwidth and time. The other is that the client first calculates the byte range of each discrete media data, and then requests the required media data from the server according to the calculated byte range. This method will require multiple requests to get the media data you need, which is slower and less efficient. Summary of the invention

In order to solve a large amount of redundant data or multiple requests in order to partially discard the stored media data in the request fragment The present invention provides a method, device, and system for rearranging and extracting media data in a slice. The technical solution is as follows:

 A method for rearranging media data in a tile, the method comprising:

 Obtaining, in the fragment, a media data that satisfies a reordering attribute, where the media data is composed of a sample or a subsample; and media data that satisfies the rearrangement attribute in the fragment in the fragment is physically and continuously stored, and is configured and configured Rearranging the sub-segments corresponding to the attributes;

 The index information of the sub-segment is added, and the index information includes rearrangement attribute information and location information of the sub-segment.

 A server for rearranging media data in a fragment, the server includes: a slicing module, a rearrangement module, and an adding module;

 The slicing module is configured to slice media content;

 The rearrangement module is configured to physically store the media data in the segment that meets the rearrangement attribute in the process of the slice, and form a sub-segment corresponding to the rearrangement attribute, where the media data is And the adding module is configured to add index information of the sub-segment, where the index information includes rearrangement attribute information and location information of the sub-segment.

 A method for extracting media data in a tile, the method comprising:

 And acquiring index information of the sub-segment, where the index information includes re-arrangement attribute information and location information of the sub-segment; and calculating, according to the index information, a byte range of the sub-segment corresponding to the predetermined re-arrangement attribute;

 And requesting, according to the byte range, the sub-segment corresponding to the predetermined rearrangement attribute;

 Obtaining the media data in the sub-segment corresponding to the predetermined rearrangement attribute.

 A client, the client includes: an acquisition module, a calculation module, a request module, and a receiving module

 The obtaining module is configured to obtain index information of a sub-segment, where the index information includes re-arrangement attribute information and location information of the sub-segment;

 The calculating module is configured to calculate, according to the index information, a byte range of a sub-segment corresponding to a predetermined re-arrangement attribute, where the requesting module is configured to obtain, according to the byte range request, the predetermined re-arrangement attribute The receiving module is configured to receive the media data in the sub-segment corresponding to the predetermined rearrangement attribute.

 A system for rearranging and extracting media data in a tile, the system comprising a server and a client;

 The server includes:

 a slicing module for slicing media content;

a rearrangement module, configured to physically store the media data in the segment within the segment that satisfies the rearrangement attribute in a process of slicing, to form a sub-segment corresponding to the rearrangement attribute, where the media data is sampled Or subsample composition; An adding module, configured to add index information of the sub-segment, where the index information includes re-arrangement attribute information and location information of the sub-segment;

 a sending module, configured to send, to the client, the media data in the sub-segment corresponding to the predetermined re-sorting attribute; the client includes:

 An obtaining module, configured to acquire, by the server, index information of a sub-segment, where the index information includes rearrangement attribute information and location information of the sub-segment;

 a calculation module, configured to calculate a byte range of the sub-segment corresponding to the predetermined re-arrangement attribute according to the index information; and a requesting module, configured to request, according to the byte range, the obtained re-arrangement attribute from the server Corresponding sub-segment;

 And a receiving module, configured to receive media data in the sub-segment corresponding to the predetermined re-sorting attribute sent by the server.

 The technical solution provided by the embodiment of the present invention has the beneficial effects of: constituting a sub-segment corresponding to the rearrangement attribute by physically storing the media data satisfying the rearrangement attribute among the segments in the fragment; Implementing the technical solution of the index information of the sub-segment, and arranging the media data in the segment in the fragment according to different rearrangement attributes to form a sub-segment corresponding to different rearrangement attributes; acquiring index information of the sub-segment; The index information calculates a byte range of the sub-segment corresponding to the predetermined rearrangement attribute; the sub-segment corresponding to the predetermined rearrangement attribute is obtained according to the byte range request; and the sub-segment corresponding to the predetermined rearrangement attribute is obtained The implementation of the technical solution of the media data can quickly and accurately extract the media data corresponding to the predetermined rearrangement attribute in the slice. DRAWINGS

 1 is a schematic structural view of a segment provided in the background art of the present invention;

 2 is a flowchart of a method for rearranging media data in a slice according to Embodiment 1 of the present invention;

 3 is a block diagram of a server device according to Embodiment 2 of the present invention;

 4 is a flowchart of a method for extracting media data in a slice according to Embodiment 3 of the present invention;

 FIG. 5 is a flowchart of a method for extracting media data in a slice according to Embodiment 4 of the present invention; FIG.

 6 is a block diagram of a device of a client according to Embodiment 5 of the present invention;

 FIG. 7 is a schematic diagram of a system for rearranging and extracting media data in a slice according to Embodiment 6 of the present invention. detailed description

The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Example 1

 Referring to FIG. 2, a method for re-arranging media data in a fragment, the execution body of the method includes a content preparation module, an HTTP streaming server, or a distribution server. The present invention does not specifically limit this, as shown in FIG. 2, Specific steps are as follows:

 Step 101: Slice the media content, and package the slice content into a format that meets the requirements (such as 3GP format) to form a slice;

 Specifically, the media content is sliced according to a predetermined data block size;

 Referring to Figure 1, each Movie Fragment in Segment is composed of Moof (Movie Fragment Box) and Media Data of media data, and the media data is Sample or Sub-sample. (Subsample) Composition, in the implementation, Sample or Sub-sample is stored in Track.

 The description item Traf (Track Fragment Box) of the track fragment in Moof is used to describe the media data in the corresponding Track Fragment. Specifically, the track fragment operation description item Trun (Track Fragment Run Box) in Traf is used. Describes the time information of the Sample in the Track Fragment, such as the decoding time and the synthesis time. The sub-sample information item Subs (Sub-sample Information Box) is used to describe the storage byte size of all Sub-samples that make up a Sample in the Track Fragment.

 In the following, this embodiment uses the example of rearranging the media data in a Track Fragment in the Movie Fragment as an example, and the method of rearranging the media data in the other Track Fragment in the Movie Fragment is the same;

 Step 102: Physically reordering the media data satisfying the rearrangement attribute in the Movie Fragment of the Segment, so that the media data satisfying the rearrangement attribute is physically stored continuously, and the level corresponding to the rearrangement attribute is formed ( a sub-segment), wherein the media data is composed of Sample or Sub-sample;

 The reordering attribute is a single attribute or a combination of any attributes. Specifically, the attributes include a video view, a video quality, and a frame rate.

 Here is an example of how to physically reorder the Samples in the Track Fragment that satisfy the reordering properties, as follows:

The original Movie Fragment structure includes the moof and the Track Fragment, which stores a sample of 2D video; then, according to a single attribute, such as 2, 4 and 8 frame rate, the sample stored in the Track Fragment Performing rearrangement, specifically, physically storing the I frame samples of the 2x frame rate stored in the Track Fragment continuously to form a sub-segment corresponding to the 2x frame rate, and storing the 4x frame rate I frame samples stored in the Track Fragment Physically contiguous storage, forming a sub-segment corresponding to a frame rate of 4 times, physically storing the I frame samples of the 8x frame rate stored in the Track Fragment continuously, forming a sub-segment corresponding to the 8x frame rate, in the Track Fragment The remaining samples stored are treated as a sub-segment.

 An example is given to how to physically reorder Sub-samples in the Track Fragment that satisfy the reordering attribute, so that the Sub-samples that satisfy the reordering attributes are physically stored continuously, and constitute sub-segments corresponding to the re-arranged attributes, as follows:

 The segment stores media data of multiple views as a Track Fragment, and each Sample contains multiple view video frames at the same time, and each view video frame can constitute a Sub-sample;

 The Sample in the Track Fragment is stored in the decoding order. Where the Sample consists of multiple Sub-samples, the Sample structure in the Track Fragment is as follows:

 { [sub_sample (0, 0) , sub_sample (0, 1), - " 31113_3 leg 16 (0, k)],

 [sub-sample (1, 0) , sub-sample (1, 1) , " ' 31113_3 leg 16 (1, m)],

[sub-sample (N, 0) , sub-sample (N, 1) ,•••sub-sample (N, n)] }

 Where [sub-sample (0, 0) , sub-sample (0, 1) , ···, sub-sample (0, k)] is SampleO, M code time is the first (seconds); [sub- Sample (1, 0) , sub-sample (1, 1) , ..., sub-sample (1, m)] is Samplel, and the decoding time is L is;, ..., [ sub-sample (N, 0) , Sub-sample (Ν, 1) , sub-sample (Ν, η)] is SampleN, and the decoding time is xth seconds; where Sub-sample (i, j) represents the (i+1)th sample (j) +1) Sub-samples, where k, m, n may or may not be equal, and the present invention does not specifically limit this.

 Here, the Trun in the description item Traf corresponding to the Track fragment describes the decoding time of the Sample in the Track fragment according to the decoding order, that is, the decoding time of all Sub-samples belonging to the same Sample, and the media data Sample in Trun. The description order is as follows: Trun [sampleO, samplel, sample2, , sampleN] ; Subs in Traf describes the storage byte size of each Sub-sample in each Sample according to the decoding order of Sample. The contents of Subs are as follows:

 Subs [

 Sub-sample (0, 0) . size, sub-sample (0, 1) . size, •••sub-sample (0, k) . size, sub-sample (1, 0) . size, sub-sample (1, 1) . size,•••sub-sample (1, m) . size, sub-sample (N, 0) . size, sub-sample (N, 1) . size, sub-sample (N, n) . size

] Assuming that the reordering properties of the first Sub-samples of all Samples in the Track Fragment are the same, then the first Sub-samples in all Samples are physically reordered so that the first Sub-samples in all Samples are physically consecutive. The storage constitutes a sub-segment;

 Assume that the reordering properties of all Sub-samples except the first Sub-sample in all Samples are the same, and all other Sub-samples except the first Sub-sample in Sample are physically reordered, so that the first Sub- is removed from all Samples. The other Sub-samples of the sample are physically stored continuously to form another sub-segment;

 It should be noted that the Sub-sample belonging to the same Sample has the same time information. For example, the sampleO decoding time is the first (seconds), then the subsample sub-sample (0, 0), sub- in SampleO. The decoding time of sample (0, 1) , ..., sub-sample (0, k) is the first Is, and the time information of Samplel is 1. Is, the subsample sub-sample (1, 0) in Samplel , sub-sample (1, 1) , •••sub-sample (1, m) decoding time is 1. Is, , subsample sub-sample (N, 0), sub-sample ( in SampleN The decoding time of N, 1) and -sub-sample (N, n) is the xth second;

 Then, the sub-samples of the same reordering attribute are physically contiguously stored to form a sub-segment, and the sub-samplings in the sub-segment are still stored according to the decoding order of the original samples, for example, as described above, the same weight is The sub-sample (0, 0) of the row attribute belongs to SampleO, the decoding time is 1 second; the sub-sample (1, 0) belongs to the 1. 1 second; ...; sub-sample (Ν, Ο) belongs to SampleN, The decoding time is the xth second; in the sub-segment, the sub-samples with the same rearrangement attribute are stored in the order of sub-sample (0, 0), sub-sample (1, 0), •••sub -sample (N, 0);

 The following example describes the storage structure of the above two sub-segments:

 { ( sub-sample (0, 0) , sub-sample (1, 0) ,•••sub-sample (N, 0) ),

 (sub-sample (0, 1) ,•••sub-sample (0, k) , sub-sample (1, 1) ,•••sub-sample (1, m) , sub-sample (N, 1 ),•••sub-sample (N, n) ) }

 Where a pair () represents a sub-segment;

 Specifically, when the Sample in the Track fragment includes multiple Sub-samples, the step further includes: using a first description item in the description structure Moof in the fragment, such as a track fragment operation description item, according to a decoding order of the samples to which the sub-sample belongs Describe the decoding order of the subsamples;

Specifically, since the Sub-sample after the physical rearrangement changes in the physical storage structure, a corresponding information description needs to be added to the Sub-sample, as follows: (1) The first description item Trim describes the structure unchanged, and still follows The decoding order of the original Sample describes the time information of the Sample, that is, all Sub-samples belonging to the Sample have the same time information, the storage order of the Sub-sample in a sub-segment and the storage order of the Sample before the sorting. Still consistent. And the Subs can obtain the correspondence between Sample and Sub-sample, so use the Sample time letter. Information describing the time information of the sub-sample to which it belongs;

 And Trun points to the physical location of the first Sub-sample in the first Sample.

 The second description item in the description structure Moof in the fragment, such as the sub-sample information description item, describes the storage byte size of the sub-sample according to the decoding order of the samples to which the sub-sample belongs;

 Specifically, the second description item Subs describes the structure unchanged, and still describes the byte storage size of each Sub-sample in the Sample according to the decoding order of the original Sample.

 Identifying the physical storage structure of the subsample by the third description item added in the description structure Moof in the fragment; specifically, since the physical storage structure of the rearranged Sub-sample changes, it is necessary to add the third in Moof Description item, the third description item is used to identify the physical storage structure of the Sub-sample, and is defined as follows:

 Class SampleOrderBox extends Ful lBox ( ' sprd, , version=0, flags)

 { unsigned int (2) type; where type is 0 means that the Sub-sample is physically rearranged according to a reordering attribute;

 Through the description structure in the fragment, the fourth new description item in Moof describes the correspondence between the sub-sample and the sub-segment to which the sub-sample belongs according to the decoding order of the samples to which the sub-sample belongs.

 Specifically, the fourth description item is used to describe the correspondence between the Sub-sample and the sub-segment level to which it belongs. The fourth description item is as follows:

 Al igned (8) class SubsToLevelBox extends Ful lBox ( ' stlf, , version, 0) { unsigned int (32) entry— count ;

 Int i, j;

 For (i=0; i < entry- count ; i++) {

 Unsigned int (32) sample—delta ;

 Unsigned int (16) Subsample count;

 If (Subsample count > 0) {

 For (j=0 ; j < Subsample count; j++) {

If (version == 1) unsigned int (32) level— identify ; ; else Unsigned int (16) level- identify ;

}} Among them, the fourth description item is similar to the structure of Subs, and the order of description of sub-sample is the same, and is still described in the order in which Sub-samples are stored in the Sample. The structure parameter level_ident defines the identifier of the level to which each Sub-sample belongs. The identifier can start from a predetermined value (such as 1) and is incremented by the predetermined value in the order in which the order is stored. It should be noted that, when there is a Sub-sample that does not belong to any level, the level- identify corresponding to the Sub-sample that does not belong to any level is 0.

 For example, assuming that the parameter level_ identify takes a value from 1, the following structure describes the correspondence between each Sub-sample and level of the two levels obtained above, that is, the first Sub-sample in each Sample. Belonging to the first level, the other Sub-sample belongs to the second level, then the structure of the fourth description item is as follows:

 Stlf [levell, level2, — level2,

 Levell, level2, — level2, levell, level2, — level2, ];

 It should be noted that the foregoing fourth description item is an optional item, and may be undefined when the level order of each Sub-sample in all Samples is consistent;

 For example, SampleO contains subsamples Sub-sample (0, 1) and Sub-sample (0, 2), and these two subsamples correspond to sub-segments levell and level2, respectively, and Samplel contains sub-samples Sub-sample (1, 1) and Sub-sample (1, 2), and these two subsamples correspond to the sub-segments levell and level2 respectively. At this time, since there are only two Sub-samples in all Samples, and the two sub-samples correspond to two levels, When Subs describes the decoding time of the subsample, it also describes the information of the subsegment to which the subsample belongs. Then, at this time, the fourth description item may not be defined.

 In addition, it should be noted that the embodiment of the present invention supports the case of rearranging media data according to a combination of any attributes.

 In the Segment, the media data of multiple views is stored in a Track Fragment, and each Sample contains multiple view video frames at the same time, and each view video frame can constitute a Sub-sample.

Then, in order to meet the needs of the 2D terminal fast forward and rewind, the number of media that meet the requirements of the two angles of view and frame rate will be satisfied. According to physical rearrangement, a plurality of sub-segments are formed; for example, a sub-sample of an I frame representing a frame rate of 2 times of the view angle 1 is physically stored continuously to form a sub-segment, and an I representing a frame rate of 4 times of the angle of view 1 is formed. The sub-sample of the frame is physically stored contiguously, forming another sub-segment, and so on.

 Step 103: Add index information to the sub-segment, where the index information includes attribute information and location information of the sub-segment; what is needed is that, in a Segment, some tracks of the Movie Fragment may be followed according to certain rearrangement attributes.

The Sample in Fragment performs physical reordering, and the Sample that satisfies the reordering property continuously stored after physical reordering constitutes several sub-segments level; and Sub-sample in other Track Fragment of Movie Fragment can also be followed according to some reordering attributes. Physical reordering, Sub-Sample satisfying the rearrangement attribute continuously stored after physical reordering may constitute several sub-segment levels;

 In addition, a slice contains a number of Movie Fragments, and may also include a slice index item Sidx.

(Segment IndexBox) Sub-slice index item Ssix (SubsSubsegmentlndexBox) and other index items, which are used to describe the index information of the Movie Fragment in the fragment. For example, Ssix can be used to describe the location information and reordering attribute information of level; in the specific operation, these index items can also be extracted separately and stored in an index file.

 Then, adding index information for the sub-segment specifically includes:

 Add index information of the sub-segment level to the index entry in the slice; or

 Adding the reordering attribute information of the sub-segment level in the MPD description to the MPD, and adding the position information of the sub-segment level to the index file; or

 Adding the reordering attribute information of the sub-segment level in the MPD description to the MPD, and adding the position information of the sub-segment level to the index item in the fragment; or

 Add the index information of the sub-segment level to the index file.

 Among them, Table 1 exemplarily shows the position information of the level described in the sub-slice index item Ssix, as follows: Table 1

 Correspondingly, add the reordering attribute of level in Ssix.

Al igned (8) class SubsegmentlndexBox extends Ful lBox ( 'Ssix' , 0, 0) Unsigned int (32) Subsegment count;

 Unsigned int (8) Subsegment level count; for ( i=l; i <= Subsegment count; i++)

 For ( j=l; j <= Subsegment level count; j++) unsigned int (32) accumulated—level—size;

 〃 increase level specific attribute information

 Unsigned int (8) attribute- count ;

 For ( k=0; k< attribute_ count ; k++) {

 String attrName ; // level corresponds to the attribute name;

 Unsigned int (32) attrvalue ; // level corresponding attribute value

Table 2 exemplarily shows the reordering attribute information describing level in Ssix in the fragment, as follows:

 Table 2

Similarly, the position information of the sub-segment level and the method of adding the rearrangement attribute letter in the index item Sidx in the slice may be the same as the position information and the rearrangement attribute information added in the Ssix level, and will not be described here. . Table 3 exemplarily shows the attribute information of the level added in the MPD, as follows:

table 3

Representation

SubRepresentation

 Level required

 Bandwidth Required level bandwidth

 trickModeAPR Optional Frame rate for shortcut operations

 Viewld Optional Level Viewpoint Correspondence

 Qualitylevel Optional Video quality identifier corresponding to Level

Since the information described by the boxes Sidx and Ssix can be extracted separately and stored in an index file. Therefore, the index information of the level can also be added in the index file. The addition method is similar to the method of adding the level index information in Ssix, and will not be described here.

 The embodiment of the present invention physically stores the media data satisfying the rearrangement attribute in the segments in the fragment to form a sub-segment corresponding to the rearrangement attribute; and implements the technical solution of adding the index information of the sub-segment The media data in the segment in the segment is arranged according to different rearrangement attributes to form a sub-segment corresponding to different rearrangement attributes, and the rearrangement attribute of the sub-segment is a single attribute or a combination of attributes, so that the user can quickly and accurately According to the index information of the sub-segments of the different rearrangement attributes, the sub-segments of the required rearrangement attributes are found, and the media data is obtained therefrom. Example 2

 Referring to FIG. 3, a server may be a media data preparation module, an HTTP streaming server, or a distribution server. The present invention does not specifically limit the present invention, and includes: a slicing module 200, a reordering module 201, and an adding Module 202, the following describes each functional module in detail:

 a slicing module 200, configured to slice media content;

 The rearrangement module 201 is configured to physically store the media data satisfying the rearrangement attribute in the segments in the fragment in a slicing process to form a sub-segment corresponding to the rearrangement attribute;

 Wherein the media data consists of samples or subsamples;

 The reordering attribute is a single attribute or a combination of any attributes, and specifically one attribute includes a video view, a video quality, and a frame rate.

The adding module 202 is configured to add index information of the sub-segment, where the index information includes re-arrangement attribute information and location information of the sub-segment. Specifically, the adding module 202 includes:

 a first adding unit 2021, configured to add index information of the sub-segment in the index item in the fragment; or

 a second adding unit 2022, configured to add reordering attribute information of the sub-segment in the media presentation description MPD, and add location information of the sub-segment in the index file; or

 a third adding unit 2023, configured to add reordering attribute information of the sub-segment in the media presentation description MPD, and add location information of the sub-segment in the index item in the fragment; or

 The fourth adding unit 2024 is configured to add index information of the sub-segment in the index file.

 Specifically, when the rearrangement module 201 is configured to physically store the media data satisfying the rearrangement attribute among the segments in the fragment to form a sub-segment corresponding to the rearrangement attribute, and the media data is composed of the sub-sample, the server Also includes:

 a first description module 203, configured to describe, by using a first description item in the description structure Moof in the fragment, a decoding order of the sub-sample according to a decoding order of the samples to which the sub-sample belongs;

 a second description module 204, configured to describe, by using a second description item in the description structure Moof in the fragment, a storage byte size of the subsample according to a decoding order of the samples to which the subsample belongs;

 The third description module 205 is configured to identify a physical storage structure of the subsample by using a third description item added in the description structure Moof in the fragment.

 The fourth description unit 206 is configured to describe, by using the fourth description item in the description structure in the fragment, the correspondence between the sub-sample and the sub-segment to which the sub-sample belongs according to the decoding order of the sample to which the sub-sample belongs.

 The embodiment of the present invention physically stores the media data satisfying the rearrangement attribute in the segments in the fragment to form a sub-segment corresponding to the rearrangement attribute; and implements the technical solution of adding the index information of the sub-segment , the media data in the segment in the fragment is made into a sub-segment of different rearrangement attributes, and the rearrangement attribute of the sub-segment is a single attribute or a combination of attributes, so that the user can quickly and accurately according to the child of the different rearrangement attribute. The index information of the fragment finds the sub-segment of the desired reordering attribute and obtains the media data therefrom. Example 3

 Referring to FIG. 4, a method for extracting media data in a fragment, the execution body of the method includes any client, such as a mobile phone, a PC, etc., and the present invention does not specifically limit this, and the details are as follows:

 Step 301: Obtain index information of the sub-segment, where the index information includes re-arrangement attribute information and location information of the sub-segment; Step 302: calculate, according to the index information, a byte range of the sub-segment corresponding to the predetermined re-arrangement attribute;

 Step 303: Request a sub-segment corresponding to the predetermined reordering attribute according to the byte range;

 Step 304: Receive media data in a sub-segment corresponding to the predetermined rearrangement attribute.

Wherein the media data consists of samples or subsamples; The reordering attribute is a single attribute or a combination of any attributes. Specifically, the attributes include a video view, a video quality, and a frame rate.

 The embodiment of the present invention calculates a byte range of a sub-segment corresponding to a predetermined rearrangement attribute according to the index information; and obtains the sub-segment corresponding to the predetermined rearrangement attribute according to the byte range request; receiving the predetermined The implementation of the technical solution of the media data in the sub-segment corresponding to the rearrangement attribute, the client can quickly and accurately extract the media data corresponding to the predetermined rearrangement attribute in the fragment. Example 4

 Referring to FIG. 5, a method for extracting media data in a fragment, the execution body of the method includes any client, such as a mobile phone, a PC, etc., and the present invention does not specifically limit this, and the specific steps are as follows:

 Step 401: Send an MPD request to the server;

 Specifically, the client obtains a URL (Uniform Resource Locator) of the MPD (Media Presentation Description) through the external connection, and sends an MPD request to the distribution server through the URL; the server receives the MPD request. After that, the MPD information is sent to the client;

 Step 402: Receive MPD information sent by the server, and obtain a segment address from the MPD information. Step 403: Obtain index information of a sub-segment level in the slice, where the index information includes level re-arrangement attribute information and location. Information

 The reordering attribute is a single attribute or a combination of any attributes. Specifically, the attributes include a video view, a video quality, and a frame rate.

 Specifically, obtaining index information of the level in the fragment includes:

 Sending a fragmentation request to the server according to the obtained fragmentation address and the preset byte range to obtain an index entry in the fragment, and obtaining index information of the level from the index entry, where the preset byte is in the embodiment of the present invention. The range is preferably a byte range of 0 to 1000;

 Or

 Obtaining the attribute information of the level from the MPD, sending a fragmentation request to the server according to the obtained fragmentation address and the preset byte range to obtain an index item in the fragment, and obtaining location information of the level described in the index item; or

 Obtaining the attribute information of the level from the MPD, obtaining the index file according to the address of the obtained index file according to the obtained address file, and obtaining the position information of the level stored in the index file, thereby obtaining Index information of the sub-segment; or

Obtaining the index file according to the address of the index file in the MPD, and acquiring the index file from the server according to the obtained address file to obtain the index information of the level stored in the index file. Step 404: Obtain a byte range of a level corresponding to the predetermined rearrangement attribute according to the index information. Specifically, obtaining a byte range of the level corresponding to the predetermined rearrangement attribute according to the index information, including: rearranging the acquired sub-segment In the attribute information, the re-arrangement attribute information of the sub-segment level matching the predetermined re-arrangement attribute is found;

 In the location information of the obtained sub-segment, the location information of the level is searched according to the re-arrangement attribute information of the level matching the predetermined re-arrangement attribute;

 And calculating, according to the location information of the level matching the reordering attribute, the byte range of the level matching the predetermined reordering attribute.

 Step 405: Request a level corresponding to the predetermined reordering attribute from the server according to the calculated byte range.

 Specifically, the request is sent to the server according to the calculated byte range and the fragment address; after receiving the request, the server sends the level in the byte range corresponding to the fragment address to the client.

 Step 406: Receive the level returned by the server and obtain media data from it.

 The embodiment of the present invention calculates a byte range of a sub-segment corresponding to a predetermined rearrangement attribute according to the index information; and requests the sub-segment corresponding to the predetermined rearrangement attribute according to the byte range; receiving the predetermined weight The implementation of the technical solution of the media data in the sub-segment corresponding to the row attribute can quickly and accurately acquire the media data corresponding to the predetermined rearrangement attribute in the fragment. Example 5

 Referring to FIG. 6, a client, which may be a user terminal such as a mobile phone or a PC, is not specifically limited in the present invention, and includes: an obtaining module 501, a calculating module 502, a requesting module 503, and a receiving module 504. The following describes each function module in detail:

 The obtaining module 501 is configured to obtain index information of the sub-segment, where the index information includes re-arrangement attribute information and location information of the sub-segment;

 The calculating module 502 is configured to calculate, according to the index information, a byte range of the sub-segment corresponding to the predetermined re-arrangement attribute; the requesting module 503, configured to obtain, according to the byte range request, the sub-segment corresponding to the predetermined re-arrangement attribute;

 The receiving module 504 is configured to receive media data in a sub-segment corresponding to the predetermined re-arrangement attribute.

 The reordering attribute is a single attribute or a combination of any attributes. Specifically, the attributes include a video view, a video quality, and a frame rate.

 Specifically, the obtaining module 501 includes:

a first obtaining unit 5011, configured to acquire index information of a sub-segment stored in an index item in the fragment; or a second obtaining unit 5012, configured to acquire re-arrangement attribute information of the sub-segment stored in the media presentation description MPD, and Acquiring the location information of the sub-segment stored in the index entry in the fragment, so as to obtain the index information of the sub-segment; or the third obtaining unit 5013, configured to acquire the re-arrangement attribute information of the sub-segment stored in the media presentation description MPD, and obtain Indexing the location information of the sub-segments stored in the file, thereby obtaining index information of the sub-segments; or

 The fourth obtaining unit 5014 is configured to obtain index information of the sub-segments stored in the index file.

 The calculation module 502 includes:

 The first searching unit 5021 is configured to: in the reordering attribute information of the sub-segment, search for re-arrangement attribute information of the sub-segment matching the predetermined re-arrangement attribute;

 The second searching unit 5022 is configured to: in the location information of the sub-segment, search for location information of the sub-segment matching the predetermined re-arrangement attribute according to the re-arrangement attribute information of the sub-segment matching the predetermined re-arrangement attribute;

 The calculating unit 5023 is configured to calculate a byte range of the sub-segment matching the predetermined rearrangement attribute according to the position information of the sub-segment matching the predetermined re-arrangement attribute.

 The embodiment of the present invention calculates a byte range of a sub-segment corresponding to a predetermined rearrangement attribute according to the index information; and requests the sub-segment corresponding to the predetermined rearrangement attribute according to the byte range; receiving the predetermined weight The implementation of the technical solution of the media data in the sub-segment corresponding to the row attribute can quickly and accurately acquire the media data corresponding to the predetermined rearrangement attribute in the fragment. Example 6

 Referring to FIG. 7, an embodiment of the present invention provides a system for rearranging and extracting media data in a fragment. The system includes a server 61 and a client 62. The server 61 may specifically be a media data preparation module and an HTTP stream. For the server such as the media server or the distribution server, the client can be a user terminal such as a mobile phone or a PC. The following system is described in detail below:

 Specifically, the server 61 includes:

 a slicing module 610, configured to slice media content;

 The rearrangement module 611 is configured to physically store the media data in the segment that meets the rearrangement attribute in the process of the slice, to form a sub-segment corresponding to the rearrangement attribute, and the media data is sampled or subsampled Constitute

 The adding module 612 is configured to add index information of the sub-segment, where the index information includes re-arrangement attribute information and location information of the sub-segment;

 The sending module 613 is configured to send, to the client, the media data in the sub-segment corresponding to the predetermined re-sorting attribute; the client 62 includes:

The obtaining module 621 is configured to obtain index information of the sub-segment from the server, where the index information includes re-arrangement attribute information and location information of the sub-segment; a calculation module 622, configured to calculate, according to the index information, a byte range of the sub-segment corresponding to the predetermined re-arrangement attribute; the requesting module 623, configured to request, from the server, the sub-segment corresponding to the predetermined re-arrangement attribute according to the byte range; The module 624 is configured to receive media data in the sub-segment corresponding to the predetermined re-sorting attribute sent by the server. The embodiment of the present invention physically stores the media data satisfying the rearrangement attribute in the segments in the fragment to form a sub-segment corresponding to the rearrangement attribute; and implements the technical solution of adding the index information of the sub-segment And arranging the media data in the segment in the fragment according to different rearrangement attributes to form a sub-segment corresponding to different rearrangement attributes; acquiring index information of the sub-segment; and calculating, according to the index information, a corresponding to the predetermined rearrangement attribute a byte range of the sub-segment; requesting the sub-segment corresponding to the predetermined re-arrangement attribute according to the byte range; implementing the technical solution of receiving the media data in the sub-segment corresponding to the predetermined re-arrangement attribute, the client The media data corresponding to the predetermined rearrangement attribute can be obtained in the slice quickly and accurately.

 All or part of the technical solutions provided by the above embodiments may be implemented by software programming, and the software program is stored in a readable storage medium such as a hard disk, an optical disk or a floppy disk in a computer. The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., which are within the spirit and scope of the present invention, should be included in the protection of the present invention. Within the scope.

Claims

Claims

 A method for rearranging media data in a slice, the method comprising:

 Slice media content;

 In the process of slicing, the media data in the segment within the segment that satisfies the rearrangement attribute is physically stored continuously to form a sub-segment corresponding to the rearrangement attribute, and the media data is composed of samples or sub-samples;

 The index information of the sub-segment is added, and the index information includes rearrangement attribute information and location information of the sub-segment.

The method according to claim 1, wherein when the media data is composed of sub-samples, the media data satisfying the rearrangement attribute is physically continuously stored to form a correspondence with the rearrangement attribute. After the sub-segment, the method further includes:

 Decoding the decoding order of the subsample according to the decoding order of the samples to which the subsample belongs according to the first description item in the description structure Moof in the fragment;

 Describe the storage byte size of the subsample according to the decoding order of the samples to which the subsample belongs according to the second description item in the description structure in the fragment;

 The physical storage structure of the subsample is identified by the third description item added in the description structure in the fragment.

The method according to claim 2, wherein the method further comprises:

 Through the description structure in the fragment, the fourth new description item in Moof describes the correspondence between the sub-sample and the sub-segment to which the sub-sample belongs according to the decoding order of the samples to which the sub-sample belongs.

The method according to claim 1, wherein the adding the index information of the sub-segment comprises: adding index information of the sub-segment to an index item in the fragment; or

 Adding rearrangement attribute information of the sub-segment in the media presentation description MPD, and adding location information of the sub-segment in the index file; or

 Adding the rearrangement attribute information of the sub-segment in the media presentation description MPD, and adding the location information of the sub-segment in the index item in the fragment; or

 The index information of the sub-segment is added in the index file.

The method according to any one of claims 1 to 5, wherein the rearrangement attribute is a single a combination of attributes or arbitrary attributes;

 Among them, attributes include video perspective, video quality, and frame rate.

A server, the server includes: a slicing module, a rearrangement module, and an adding module; the slicing module is configured to slice the media content;

 The rearrangement module is configured to physically store the media data in the segment that meets the rearrangement attribute in the process of the slice, and form a sub-segment corresponding to the rearrangement attribute, where the media data is And the adding module is configured to add index information of the sub-segment, where the index information includes rearrangement attribute information and location information of the sub-segment.

The server according to claim 6, wherein the rearrangement module is configured to physically store the media data satisfying the rearrangement attribute in the segments in the fragment, and constitute the weight When the media data is composed of sub-samples, the server further includes:

 a first description module, configured to describe, by using a first description item in the description structure Moof in the fragment, a decoding order of the sub-sample according to a decoding order of the samples to which the sub-sample belongs;

 a second description module, configured to describe, by using a second description item in the description structure Moof in the fragment, a storage byte size of the subsample according to a decoding order of the samples to which the subsample belongs;

 And a third description module, configured to identify a physical storage structure of the subsample by using a third description item added in the description structure in the fragment.

The server according to claim 7, wherein the server further comprises:

 And a fourth description unit, configured to describe, by using a description item in the fragment, a fourth description item added in the Moof, according to a decoding order of the samples to which the sub-sample belongs, a correspondence relationship between the sub-sample and the sub-segment to which the sub-sample belongs. The server according to claim 6, wherein the adding module comprises:

 a first adding unit, configured to add index information of the sub-segment in an index item in the fragment; or

 a second adding unit, configured to add rearrangement attribute information of the sub-segment in the media presentation description MPD, and add location information of the sub-segment in the index file; or

a third adding unit, configured to add rearrangement attribute information of the sub-segment in the media presentation description MPD, and add location information of the sub-segment in an index item in the fragment; or And a fourth adding unit, configured to add index information of the sub-segment in the index file.

The server according to any one of claims 6-9, wherein the rearrangement attribute is a single attribute or a combination of any attributes;

 Among them, attributes include video perspective, video quality, and frame rate.

A method for extracting media data in a slice, the method comprising:

 And acquiring index information of the sub-segment, where the index information includes re-arrangement attribute information and location information of the sub-segment; and calculating, according to the index information, a byte range of the sub-segment corresponding to the predetermined re-arrangement attribute;

 And requesting, according to the byte range, the sub-segment corresponding to the predetermined rearrangement attribute;

 Receiving the media data in the sub-segment corresponding to the predetermined rearrangement attribute.

The method of claim 11, wherein the acquiring the index information of the sub-segment comprises: acquiring index information of the sub-segment stored in the index item in the fragment; or

 Obtaining a media presentation description of the reordering attribute information of the sub-segment stored in the MPD, and acquiring location information of the sub-segment stored in the index item in the fragment, thereby obtaining index information of the sub-segment; or

 Obtaining the media presentation description of the reordering attribute information of the sub-segments stored in the MPD, and obtaining the location information of the sub-segments stored in the index file, thereby obtaining index information of the sub-segments; or

 Get index information of the sub-segments stored in the index file.

The method according to claim 11, wherein the calculating a byte range of the sub-segment corresponding to the predetermined rearrangement attribute according to the index information comprises:

 In the rearrangement attribute information of the sub-segment, searching for reordering attribute information of the sub-segment matching the predetermined re-arrangement attribute; in the position information of the sub-segment, according to the weight of the sub-segment matching the predetermined re-arrangement attribute The row attribute information finds the location information of the sub-segment matching the predetermined rearrangement attribute;

 And calculating, according to the position information of the sub-segment matching the predetermined rearrangement attribute, the byte range of the sub-segment matching the predetermined rearrangement attribute.

The method according to any one of claims 12-13, wherein the rearrangement attribute is a single attribute or a combination of any attributes; 15. A client, wherein the client comprises: an acquisition module, a calculation module, a request module, and a receiving module

 The obtaining module is configured to obtain index information of a sub-segment, where the index information includes re-arrangement attribute information and location information of the sub-segment;

 The calculating module is configured to calculate, according to the index information, a byte range of a sub-segment corresponding to a predetermined re-arrangement attribute, where the requesting module is configured to request, according to the byte range, the content corresponding to a predetermined re-arrangement attribute a sub-segment; the receiving module, configured to receive the media data in the sub-segment corresponding to the predetermined re-arrangement attribute.

The client according to claim 15, wherein the obtaining module comprises:

 a first acquiring unit, configured to acquire index information of a sub-segment stored in an index item in the fragment; or

 a second obtaining unit, configured to acquire reordering attribute information of the sub-segment stored in the media presentation description MPD, and obtain location information of the sub-segment stored in the index item in the fragment, thereby obtaining index information of the sub-segment; or

 a third obtaining unit, configured to obtain reordering attribute information of the sub-segment stored in the media presentation description MPD, and obtain location information of the sub-segment stored in the index file, thereby obtaining index information of the sub-segment; or

 The fourth obtaining unit is configured to obtain index information of the sub-segments stored in the index file.

 The client according to claim 15, wherein the calculation module comprises:

 a first searching unit, configured to search, in the reordering attribute information of the sub-segment, re-arrangement attribute information of the sub-segment matching the predetermined re-arrangement attribute;

 a second searching unit, configured to: in the location information of the sub-segment, search for the location information of the sub-segment matching the predetermined re-arrangement attribute according to the re-arrangement attribute information of the sub-segment matching the predetermined re-arrangement attribute;

 And a calculating unit, configured to calculate, according to the location information of the sub-segment matching the predetermined reordering attribute, the byte range of the sub-segment matching the predetermined re-sequencing attribute.

The client according to any one of claims 16-17, wherein the rearrangement attribute is a single attribute or a combination of any attributes;

 Among them, attributes include video perspective, video quality, and frame rate.

19. A system for rearranging and extracting media data in a tile, wherein the system includes a server and a client Account

 The server includes:

 Slice module for slicing media content

 a rearrangement module, configured to physically store the media data in the segment within the segment that satisfies the rearrangement attribute in a process of slicing, to form a sub-segment corresponding to the rearrangement attribute, where the media data is sampled Or subsample composition;

 An adding module, configured to add index information of the sub-segment, where the index information includes re-arrangement attribute information and location information of the sub-segment;

 a sending module, configured to send, to the client, the media data in the sub-segment corresponding to the predetermined re-sorting attribute; the client includes:

 An obtaining module, configured to acquire, by the server, index information of a sub-segment, where the index information includes re-arrangement attribute information and location information of the sub-segment;

 a calculation module, configured to calculate a byte range of the sub-segment corresponding to the predetermined re-arrangement attribute according to the index information; and a requesting module, configured to request, according to the byte range, the obtained re-arrangement attribute from the server Corresponding sub-segment;

 And a receiving module, configured to receive media data in the sub-segment corresponding to the predetermined re-sorting attribute sent by the server.