JP2013532441A - Method and apparatus for encapsulating encoded multi-component video - Google Patents

Abstract

  Describes a method and apparatus for encapsulating a media entity including multiple layers into multiple component files (encapsulating into one component file per layer) along with a method and apparatus for reading the corresponding component file To do. A new box of ISO BMFF and an extension of the extractor data structure of the SVC / MVC file format are proposed. This new box allows access to the referenced component file in parallel with the processing of the current component file. The extractor extension of the present invention allows for referencing NAL units across different component files. The present invention enables adaptive HTTP streaming of media files.

Description

(Cross-reference of related applications)
This patent application is filed on June 14, 2010, entitled “Extension to the Extractor data structure of SVC / MVC file formats” and US Provisional Patent Application No. 61 / 354,422, filed June 14, 2010. It claims the benefit of the priority of US Provisional Patent Application No. 61 / 354,424 entitled “Some extensions for ISO Base Media File Format for HTTP Streaming”.

  This application is related to United States Patent Application No. __ / _ (Attorney No. PU100140), co-owned owner, entitled “Method and Apparatus for Encapsulating Coded Multi-component Video” filed concurrently with this application. It is.

  The present invention generally relates to HTTP streaming. More particularly, the invention relates to the encapsulation of media entities of encoded multi-component video streams, such as scalable video coding (SVC) streams and multi-view coding (MVC) streams of HTTP streaming.

  In the field of HTTP streaming, on the server side, encoded video is often encapsulated and stored as a BMFF-compliant file such as an MP4 file. In addition, to achieve adaptive HTTP streaming, a file is typically divided into a plurality of video fragments that are further grouped into segments that can be addressed by a client URL request. In practice, these segments store various encoded representations of the video content so that the client can dynamically select the desired representation to download and play during the session. It has become.

  Encoded hierarchical video, such as SVC and MVC bitstreams, allows different operating points or representations in terms of temporal / spatial resolution, image quality, views, etc. by decoding different subsets of the bitstream By doing so, natural support for such bit rate adaptation is realized. However, the existing ISO Base Media File Format (BMFF) standard, such as the MP4 file format, does not support individual access to each layer or expression, and therefore cannot be applied to the HTTP streaming field. As shown in FIG. 1, in the MP4 file format, the metadata for all layers or representations of one media file is stored in the moov video box, while the media content data for all layers or representations is , Mdat video box. In HTTP streaming, when a client requests one layer, all layers or representations are mixed and the client does not know where to find the required layer or representation, so the entire file must be sent .

As described below, in the field of adaptive HTTP streaming, it is desirable to be able to reference media data samples, such as network abstraction layer (NAL) units, across video fragment or component file boundaries. In the SVC / MVC context, such a reference can be achieved by using a mechanism such as an “extractor”. "Extractor" is an internal file data structure defined in SVC / MVC correction for the following BMFF AVC file format extension. Information Technology-coding of audio-visual objects-Part 15: Advanced Video Coding (AVC) File format, Amendment 2: File format superV. The extractor is designed to allow NAL units to be extracted from other tracks by reference without copying. Here, a track is a timed sequence of related samples in an ISO base media file. In media data, a track corresponds to a sequence of images or sampled audio. The syntax of the extractor is shown below.
class aligned (8) Extractor () {
NALUnitHeader ();
unsigned int (8) track_ref_index;
signed int (8) sample_offset;
unsigned int ((lengthSizeMinusOne + 1) * 8)
data_offset;
unsigned int ((lengthSizeMinusOne + 1) * 8)
data_length;
}

The semantics of the extractor data structure are as follows:
NALunitHeader: NAL unit structure specified in ISO / IEC 14496-10 Annex G of type 20 NAL unit:
nal_unit_type shall be set to extractor NAL unit type (type 31).
Forbidden_zero_bit, reserved_one_bit, and reserved_three_2 bits shall be set as specified in ISO / IEC 14496-10 Annex G.
Other fields (nal_ref_idc, idr_flag, priority_id, no_inter_layer_pred_flag, dependency_id, quality_id, temporal_id, use_ref_base_pic_flag, discardpable_bladtable_bladg. 4 of Information technology-Coding of audio-visual objects-part 15: Advanced Video Coding (AVC) file format, Amendment 2: File format, Amendment 2: File format 2: 2008, page 17 is set as specified.

  track_ref_index specifies an index of a track reference of type “scal” used to find the track from which data is extracted. The sample in the track from which the data is extracted is closest in time before or ahead of the sample containing the extractor in the media decoding timeline, i.e., using only the time / sample table. Located in position and adjusted by the offset specified by sample_offset. The first track reference has an index value of 1. The value 0 is reserved.

  sample_offset gives a relative indication of the samples in the linked track to be used as an information source. Sample 0 (zero) is the sample that has the same decoding time or the closest preceding decoding time as compared to the decoding time of the sample containing the extractor. Sample 1 (one) is the next sample, and sample-1 (minus 1) is the previous sample. The same goes for the following.

  data_offset: the offset of the first byte in the reference sample to be copied. If the extraction starts from the first byte of data in the sample, the offset takes the value 0. The offset shall refer to the beginning of the NAL unit length field.

  data_length: the number of bytes to be copied. If this field takes the value 0, the entire referenced NAL unit is copied (ie, the length to be copied is taken from the length field referenced by the data offset, and in the case of an aggregator, additional_bytes. Augment in the field).

  Further details are: Information technology-Coding of audio-visual objects-part 15: Advanced Video Coding (AVC) file format, Amendment 2: File format, Amendment 2: File format. 2: 2008 can be seen.

  Currently, the extractor can extract NAL units from other tracks by reference, but only from within the same video box / fragment. In other words, using an extractor cannot extract NAL units from another segment or file. This restriction limits the use of the extractor to the use case described above.

  If the client has already downloaded one or more content components of one media content from the server and is in the process of downloading another content component, the client can complete the component as needed It needs to know if the previously downloaded content component is included in the set of dependent components of the new content component so that other requests to download the set can be made. This use case also requires a mechanism for signaling externally dependent content components and their location information.

  In BMFF there is a box type called “tref” that is used to realize a reference from the containing track being presented to another track. This box can be used to describe dependencies between tracks, but the dependencies are limited to tracks within the same media file.

  One approach is to signal this information using some out-of-band mechanism. For example, in the field of HTTP streaming, the server can send a manifest file to the client before the session starts. The manifest file is a file that includes dependency and location information for each content component of the requested media content. In this case, the client can request all necessary component files. However, this out-of-band technique cannot be applied to local file playback where a manifest file is not available.

  Conventional solutions to the above problems are not yet well established in the art. It would be desirable if multiple layers could be parsed and encapsulated without sacrificing speed and transport efficiency. Such results have not been achieved so far in the art.

  The present invention relates to a method and apparatus for encapsulating and reading a component file from a media entity that includes multiple layers.

  According to one aspect of the invention, a method is provided for encapsulating and creating a component file from a media entity that includes multiple layers. In this method, metadata is extracted from the media entity to each layer, and media data corresponding to the extracted metadata is extracted. The extracted media data and the metadata are associated with each other, and a component file including the extracted metadata and the extracted media data can be created in each layer.

  According to another aspect of the invention, a file encapsulation device is provided. The file encapsulating apparatus extracts metadata from a media entity to each layer and extracts media data corresponding to the extracted metadata; and metadata obtained by extracting the extracted media data; And a correlator that makes it possible to create a component file for each layer.

  The above features of the present invention will become more apparent from the detailed description of exemplary embodiments thereof with reference to the accompanying drawings.

FIG. 3 illustrates an exemplary MP4 file format. FIG. 3 illustrates an embodiment of the present invention that encapsulates a media entity. FIG. 2 illustrates the structure of an encapsulating device used to encapsulate or create a component file from a media entity that includes multiple layers / representations. It is a figure which shows an example which associates additional media data with a component file based on a dependency. It is a figure which shows an example which extracts a NAL unit by reference from the moving image box / fragment different from the moving image box / fragment which an extractor exists. FIG. 6 illustrates the associated encapsulation operation from a SVC / MVC type video bitstream to multiple component files using one of the new extractor data structures of the present invention. FIG. 2 shows the structure of a file reader used to read a component file. FIG. 6 illustrates a process for reading an encapsulated component file of a video decoder that includes an embodiment of the present invention. FIG. 5 illustrates an encapsulation operation from an SVC / MVC type video bitstream into multiple video fragments using another preferred new extractor data structure. FIG. 6 illustrates a process for reading an encapsulated component file of a video decoder that includes another embodiment of the present invention.

  In the present invention, a media entity, such as a media file or media file set or streaming media, is divided or encapsulated into a plurality of video component files that are addressable by a client URL request. Here, component files are used in a broad sense to represent fragments, segments, files, and other equivalent terms.

  In one embodiment of the invention, a media entity that includes multiple representations or components is parsed to extract metadata and media data for each representation / component. Examples of this representation / component include layers such as layers with various temporal / spatial resolutions, layers such as SVC image quality layers, and MVC views. In the following, layers are also used to refer to representations / components, and these terms are interchangeable. The metadata describes, for example, what is contained in the media entity of each representation and how to use the media data contained therein. The media data includes any necessary information regarding the media data samples necessary to perform the purpose of the media data, such as decryption of the content, or how to obtain the required data samples. The extracted metadata and media data for each representation or layer is correlated / correlated and stored together for user access. Storage operations may be performed physically on a hard drive or other storage medium, or other, even when metadata and media data are actually located at different locations on the storage medium When interfacing with other applications or modules, they may be executed virtually via a relationship management mechanism so that they appear to be stored together. FIG. 2 shows an example of this embodiment. In FIG. 2, the media entity includes three layers: a base layer, an enhancement layer 1 and an enhancement layer 2. Parse the media entity to extract metadata and media data for each of the three layers, and separate these data as component files with associated metadata and corresponding media data Remember.

  FIG. 3 shows the structure of a preferred encapsulation device 300 used to encapsulate and create a component file from a media entity that includes multiple layers, such as SVC encoded video. Input media entity 310 is passed to metadata extractor 320 and media data extractor 340. The metadata extractor 320 extracts the metadata 330 of each layer. The media data extractor 340 takes in the metadata 330 and extracts the corresponding media data 350. It should be noted that in another embodiment, metadata extractor 320 and media data extractor 340 are implemented as one extractor. Both metadata 330 and media data 350 are sent to a correlator 380 that associates these two types of data to create an output component file 390. One component file is created for each layer.

  Hierarchical video, such as video encoded with an SVC or MVC AVC extension, includes multiple media components (scalable layers or views). Such an encoded bitstream can provide different operating points or representations or layers with respect to temporal / spatial resolution, image quality, view, etc. by decoding different subsets of the bitstream. Furthermore, there is an encoding dependency between the layers of the bitstream, that is, decoding of one layer may depend on other layers. Thus, when requesting one of such bitstream representations, it may be necessary to retrieve and decode one or more components or media data from the encapsulated video file. To facilitate the various representation extraction processes, encoded hierarchical video is often encapsulated in MP4 files in such a way that each layer is stored separately in a different segment or component file. In this case, due to the decoding dependencies described above or other dependencies by the application, a particular media data sample of the bitstream, such as a NAL unit, is required by multiple segments or component files, or Need to be taken into account.

  In another embodiment of the invention, additional media data required by the segment or component file is extracted and associated with the segment or component file. FIG. 4 shows an example of this embodiment. In this figure, the SVC bitstream has three spatial layers, HD 1080p, SD and QVGA. Three video fragments or component files are formed corresponding to these three operating points, each addressable by a different URL. Within each video fragment or component file, all media data samples (NAL units in this example) required for decoding are copied and stored as media samples stored in the “mdat” box. Thus, when a client requests a specific operating point or representation using an appropriate URL, the server can retrieve the corresponding video fragment or component file and transfer it to the client. In this embodiment, the media data extractor 340 of FIG. 3 further extracts additional media data related to the media data extracted for each layer from the input media entity 310 for each layer. The correlator 380 further associates this additional extracted media data for each layer to create a corresponding component file.

  To save storage space, reference media data samples such as NAL units across multiple video fragment or component file boundaries without actually duplicating the same data in each component file It is desirable to be able to. However, the ISO Base Media File Format (BMFF) and its extensions do not currently support this function. To solve this problem, yet another embodiment of the present invention identifies references for additional media data related to or required by the media data of a video fragment or component file. And build. Instead of these additional media data, this reference is associated with the component file along with its metadata and media data. The reference can be embedded in each layer of extracted media data, and each layer's extracted metadata and extracted media data can then be associated to create a corresponding component file.

  In this embodiment, a reference identifier 360 is added to the structure of the encapsulation device 300. Reference identifier 360 identifies references 370 to those additional media data related to each layer of extracted media data 350 from input media entity 310. Correlator 380 then associates reference 370 with each layer of extracted metadata 330 and extracted media data 350, for example by embedding reference 370 in extracted media data 350, to create a corresponding component file 390. .

  As mentioned above, in the SVC / MVC context, such a reference can be constructed by using a mechanism such as an “extractor”. Currently, the extractor can extract NAL units from other tracks by reference, but only from within the same video box / fragment. In other words, using an extractor cannot extract NAL units from another segment or file. This restriction limits the use of extractors in other cases. In the following, an extension of the extractor data structure is disclosed, which is intended to support efficient encapsulation of the above-mentioned SVC / MVC type hierarchical video content into multiple component files. Is.

  This extension was added to provide an extractor data structure with additional functionality to reference NAL units residing in a video box / fragment or component file that is different from the video box / fragment or component file in which the extractor resides It is what is done.

  The extended extractor is defined as follows:

Syntax:
aligned (8) class DataEntryUrlBox (bit (24) flags)
extends FullBox ('url', version = 0, flags) {
string location;
}
aligned (8) class DataEntryUrnBox (bit (24) flags)
extends FullBox ('urn', version = 0, flags) {
string name;
string location;
}
class aligned (8) Extractor () {
NALUnitHeader ();
DataEntryBox (entry_version, entry_flags) data_entry; // added extension
unsigned int (8) track_ref_index;
signed int (8) sample_offset;
unsigned int ((lengthSizeMinusOne + 1) * 8)
data_offset;
unsigned int ((lengthSizeMinusOne + 1) * 8)
data_length;
}

Semantics:
data_entry is a unified resource locator (URL) or unified resource name (URN) entry. The name is URN and is required in the URN entry. The location is a URL and is required in the URL entry, but is optional in the URN entry, giving the location where a resource with a given name is found. Each is a null-terminated character string using UTF-8 characters. If the independence flag is set, the URL form is used, there is no character string, and the box ends with the entry flag field. The URL type is a service for distributing files. Relative URLs are allowed, but relate to files containing video boxes / fragments containing the track to which the extractor belongs.

  The other fields have the same semantics as the original extractor described above.

  In the extended extractor, by reference, the NAL unit can be extracted from a moving image box / fragment different from the moving image box / fragment in which the extractor exists. FIG. 5 shows such an example, where the SVC bitstream is the same as in FIG. 4 but uses a new extended extractor data structure. As can be seen from this figure, the NAL unit of the QVGA video fragment can be referred to for the SD video fragment. Similarly, HD 1080p video fragments can use an extractor to reference the NAL units of both QVGA and SD video fragments. Compared to FIG. 4, there are no NAL units replicated across these video fragments, thus saving storage space.

  FIG. 6 illustrates the associated encapsulation operation from a SVC / MVC type video bitstream to multiple video fragments or component files using the new extractor data structure of the present invention. The process begins at step 601. In step 610, each NAL unit is read one by one. If the end of the bitstream is reached at step 620, the process stops at 690, otherwise the process proceeds to the next step 630. In decision step 630, it is determined whether the current NAL unit depends on the NAL unit of another track for decoding. If the result of this determination is that the current NAL unit does not depend on the NAL units of other tracks for decoding, control passes to step 640 where a sample is formed using the current NAL unit and Be placed. If the determination in step 630 shows that there is a dependency between the current NAL unit and the NAL unit of another track, the process proceeds to step 650. In decision step 650, it is further determined whether a track of the NAL unit required by the current NAL unit exists within the same video fragment. If the result of this determination is that the track is in the same video fragment, step 670 is used to fill in the expanded extractor and refer to the NAL unit of the other track. If the result of the determination is that the track is in another video fragment, then in step 660 the URL or URN of this video fragment is identified and the identified URL and URN are entered into the extended extractor. The process proceeds to step 670. This expanded extractor is embedded in the current track at step 680 after entry. Thereafter, in step 610, the process for the next NAL unit is started.

  In another embodiment, correlator 380 embeds reference 370 in extracted metadata 330 and adds an index of reference 370 to extracted media data 350. The correlator 380 further associates the metadata of each layer with the media data to create a corresponding component file 390. In the situation of ISO Media Base File Format, a box called an HTTP streaming information box is disclosed. This box contains information that can assist HTTP streaming of ISO files. The HTTP streaming information box is preferably placed in the component file as early as possible, for example at the beginning of the file. This box can also serve as a source when the server creates a manifest file for the client. Another type of box, called a media reference box, included in an HTTP streaming information box is also disclosed. This box contains information about the external dependency file. The extractor structure is further extended to allow multiple media samples to be referenced across different component files. The information contained in the media reference box can be used by the extractor to save signaling overhead.

  Detailed definitions of the proposed HTTP streaming information box, media reference box and further improved extractor are as follows.

HTTP streaming information box Definition:
Box type: 'hsin'
Container: File Required: No Quantity: 0 or 1

  The HTTP streaming information box assists the HTTP streaming operation of the ISO media file. The HTTP streaming information box contains relevant information regarding HTTP streaming delivery of files, including the media reference box defined below, among other possible types of boxes. The HTTP streaming information box is preferably placed in the file as early as possible in order to maximize availability.

Syntax:
aligned (8) class HTTPStreamingInfoBox extends Box ('hsin') {
}

Media reference box Definition:
Box type: 'mref'
Container: 'hsin'
Required: No Quantity: 0 or 1

  The media reference box is included in the HTTP streaming information box and includes a data reference table in the form of a URL that declares the location of an external file on which each track included in the box depends. By reading this box, the file reader can identify the external dependent file sources of the tracks in the file, such as external component files, and the means for retrieving them.

Syntax:
aligned (8) class DataEntryUrlBox (bit (24) flags) extends Box ('url') {
string location;
}
aligned (8) class MediaReferenceBox extends Box ('mref') {
unsigned int (16) entry_count;
for (i = 1; i <= entry_count; i ++) {
unsigned int (32) track_ID;
unsigned int (16) dependent_source_count;
for (j = 1; j <= dependent_source_count; j ++) {
DataEntryUrlBox data_entry;
}
}
}

Semantics:
entry_count is an integer obtained by counting actual entries.
track_ID is an integer that uniquely identifies the track in the file to which the box is applied.
The dependent_source_count is an integer that counts the external media source on which the track in the file with track_ID depends.
data_entry is a URL entry that points to one external media source on which the specified track depends. Each is a null-terminated character string using UTF-8 characters. The URL type is a service for distributing files. Relative URLs are allowed, but relate to the file containing this media reference box.

  The media reference box defined above is designed to facilitate HTTP streaming of media entities containing multiple layers in several ways.

  First, the media reference box can explicitly signal dependencies between component files at the beginning of the component file by means of a lookup table. Thus, once a client has downloaded a small portion of a component file, it can know all the associated external component files for that track or tracks, and if necessary, references included in the table Can make a corresponding request to obtain one or more complete sets for playback.

  Secondly, the in-file information in this box can be easily extracted and included in the manifest file. By including this information in the manifest, the client discovers the relevant service information prior to the actual HTTP streaming and performs the corresponding service initialization, eg requesting all relevant component files and allocating necessary buffer resources. Can help to perform.

  Third, when a client requests some multi-component media content with a different representation that has another representation already delivered as a component file, it checks the corresponding media reference box in the file. , Whether the file contains any dependency components of the new representation that can be reused.

  Finally, this box helps to reduce the signaling overhead of the extended extractor structure defined as follows.

Extractor The extractor has also been proposed to extend the ability to refer to track data in external media files.

Extended syntax:
class aligned (8) Extractor () {
NALUnitHeader ();
unsigned int (16) media_reference_index;
unsigned int (8) track_ref_index;
signed int (8) sample_offset;
unsigned int ((lengthSizeMinusOne + 1) * 8)
data_offset;
unsigned int ((lengthSizeMinusOne + 1) * 8)
data_length;
}

Semantics:
media_reference_index: specifies the index of the reference table entry contained in the media reference box with the same associated track_ID value as the track containing the extractor. If media_reference_index is equal to 0, the extractor refers to data in the same file as the extractor, although it is data of another track. In this case, it is assumed that there is no reference table in the media reference box having the same track_ID value as this track. If media_reference_index is between 1 and the value of dependent_source_count in the reference table associated with the track in the media reference box, the URL referenced by the media_reference_index in the reference table points to the external file, and this external file is The extractor includes a track from which data is extracted.

  The semantics of the other fields remain the same as in the original extractor definition.

  In this further expanded extractor structure, an extractor can be used to link and extract data for tracks belonging to an external component file. This is particularly useful when encapsulating content components from one encoded multi-component media content, such as those encoded with SVC or MVC, into different component files. This extended extractor can perform extraction across file boundaries. This eliminates the need to duplicate the same data in different component files.

  FIG. 9 shows the association of a SVC / MVC type video bitstream to multiple video fragments or component files using the disclosed HTTP streaming information box and media reference box and the above-extended extractor data structure described above. The encapsulating operation is shown. This process is similar to the process shown in FIG. 6 except for some modifications due to further expansion of the box and extractor described above. After the location information URL / URN is identified in step 660, in step 965, the location information is used to fill in the reference table in the mref box (media reference box). In step 970, the index of position information of the reference table is further written in the extractor. The extractor is then embedded in the current track. If the end of the bitstream has been reached in step 620, the mref box and its container hsin box (HTTP streaming information box) are embedded in the metadata of the component file.

  In order to read the component file, a file reader 700 shown in FIG. 7 is used. Parser 710 first parses the component file to obtain metadata and media data, and obtains a reference if available. If the decrypted reference reveals that the media data is related to the media data of other component files, such as by decryption dependencies, the retriever 720 can retrieve the associated media data. Retrieve from the other component file indicated in the reference. In addition, the processor 730 processes the metadata and media data obtained from the component file, and any additional media data if available. Parsing operations by parser 710 include the various operations necessary to obtain metadata and media data prepared for processor 730 and a reference prepared for retriever 720. This parsing operation includes further parsing the metadata and / or media data as needed. In one embodiment, the reference is obtained by parsing the media data as it is embedded in the media data. If the reference is available, the parsing step further includes analyzing the syntax of the reference and decoding the reference. The processor 730 can include a video decoder if the component file includes video content. In another embodiment, a parser and retriever may be incorporated into the processor.

  FIG. 8 illustrates the process of reading the SVC / MVC type video bitstream of the video decoder including the present invention. Step 801 accesses the component video file, and step 805 identifies the metadata and media data for each layer of the component video file. At step 810, the identified metadata and media data are parsed, and at step 815, each NAL unit of media data is read one by one. For the current NAL unit, a determination is first made at step 820 to determine if the end of the bitstream has been reached, and if the result is “yes”, the process ends at step 825. If not, the process proceeds to decision step 830 to determine if the current NAL unit is an extractor. If the current NAL unit is not an extractor, it means that the current NAL unit is a normal NAL unit containing decoding data, so in step 835 this NAL unit is sent to the decoder. If the current NAL unit is an extractor, step 840 determines whether the current NAL unit depends on a NAL unit outside the same component file. If the required NAL unit is in the same component file, the NAL unit is retrieved from the current file at step 845 and sent to the decoder at step 835. If the required NAL unit is from another component file, then in step 850 the reference information Data_entry in the extractor is used to locate the NAL unit, in step 855 it is retrieved from the remote file, then In step 835, the data is transmitted to the decoder.

  In another embodiment, parser 710 identifies references by parsing the media data to obtain embedded reference indices and obtains corresponding references according to these reference indices. The corresponding process for reading the SVC / MVC type video bitstream of the video decoder is shown in FIG. This is similar to the process of FIG. Since the reference is placed at the beginning of the component file in the preferred embodiment, the parsing of the metadata in step 810 allows the analysis of the references contained therein to be performed in parallel with the parsing of the media data. become. When analyzing a reference, step 1014 identifies the other referenced component files. The retrieval of these other component files begins in step 1012 in parallel with the remaining steps of the process. In step 850, after accessing the location information of the component file on which the current NAL unit depends, the local storage device, such as a media buffer, is checked to see if this component file is available. If the required component file is available locally, the local copy NAL unit is retrieved; otherwise, the remote file NAL unit is retrieved. Note that a local copy of this component file may be obtained by parallel fetching at step 1012, or from a previous request for this component file.

  Although the present invention has been described in detail with reference to preferred embodiments thereof, it is to be understood that the invention is not limited to these embodiments and that those skilled in the art will recognize the invention as defined in the appended claims. It should be understood that other modifications and variations can be made without departing from the scope thereof.

Claims (37)

A method of creating a component file from a media entity containing multiple layers,
Extracting metadata for each layer from the media entity;
Extracting from the media entity media data corresponding to the extracted metadata of each layer of the media entity;
Associating the extracted media data with the extracted metadata to enable creation of component files including the extracted metadata and the extracted media data in each of the layers. , Said method.   The method of claim 1, wherein the component file is at least one of a video box, a video fragment, a segment, and a file. Extracting additional media data related to the extracted media data of each layer from the media entity to each layer;
The method of claim 1, further comprising associating the extracted media data and the additional media data for each layer for creation of a corresponding component file. Identifying additional media data references related to the extracted media data for each layer;
The method of claim 1, further comprising associating the reference with the extracted metadata and the extracted media data for each layer to create a corresponding component file.   The method of claim 4, wherein the media data and the additional media data include data samples.   6. The method of claim 5, wherein the data sample includes a network abstraction layer unit.   The method of claim 6, wherein the reference includes at least one of a unified resource locator and a unified resource name of the network abstraction layer unit in the additional media data. Embedding the reference in the extracted metadata of each layer;
The method of claim 4, further comprising: adding the reference indication to the extracted media data.   9. The method of claim 8, wherein the reference is placed at the beginning of the component file for each layer.   9. The method of claim 8, wherein the reference is entered in a media reference box and the indicator is entered in an extractor. A file encapsulation device that creates a component file from a media entity containing multiple layers,
An extractor that extracts metadata of each layer from the media entity and extracts media data corresponding to the extracted metadata of each layer of the media entity from the media entity;
A correlator that associates the extracted media data with the extracted metadata and that enables creation of component files including the extracted metadata and the extracted media data in each of the layers. The file encapsulation device.   12. The file encapsulation device according to claim 11, wherein the component file is at least one of a movie box, a movie fragment, a segment, and a file. The extractor further extracts, for each layer, additional media data related to the extracted media data of each layer from the media entity;
12. The file encapsulation device according to claim 11, wherein the correlator further associates the extracted media data and the additional media data of each layer to create a corresponding component file.   A reference identifier for identifying a reference of additional media data related to the extracted media data of each layer from the media entity, wherein the correlator extracts the reference from the extracted meta data of each layer; 12. The file encapsulation device according to claim 11, wherein a corresponding component file is created in association with data and the extracted media data.   The file encapsulation device of claim 14, wherein the media data and the additional media data include data samples.   16. The file encapsulation device of claim 15, wherein the data sample includes a network abstraction layer unit.   The file encapsulation device of claim 16, wherein the reference includes at least one of a unified resource locator and a unified resource name of the network abstraction layer unit in the additional media data.   15. The file encapsulation device according to claim 14, wherein the correlator further embeds the reference in the extracted metadata of each layer, and adds an index of the reference to the extracted media data.   19. The file encapsulation device according to claim 18, wherein the correlator places the reference at the beginning of the component file for each layer.   20. The file encapsulation device of claim 19, wherein the reference is entered in a media reference box and the indicator is entered in an extractor. A method of reading a component file,
Parsing the component file to obtain metadata, media data and references;
According to the reference, when the media data of the component file is related to media data of another component file, the related media Retrieving the data.   The method of claim 21, wherein the media data of the component file is related to media data of other component files according to encoding dependencies.   The method of claim 21, wherein the media data and the related media data comprise data samples.   24. The method of claim 23, wherein the data sample comprises a network abstraction layer unit.   The method of claim 21, further comprising parsing the metadata to obtain the reference. Parsing the media data to obtain a reference index embedded therein;
26. The method of claim 25, further comprising: obtaining a corresponding reference according to the reference index. Said removing step comprises:
26. The method of claim 25, comprising retrieving the other component file in parallel according to the reference. Said removing step comprises:
Checking the local file storage;
28. The method of claim 27, further comprising retrieving the other component file from the local storage device if the local file storage device includes the other component file. A parser that parses component files to obtain metadata, media data, and references;
A retriever for retrieving media data related to the media data from other component files according to the reference;
A processor for processing media data retrieved from the metadata, the media data, and the other component files.   30. The file reader of claim 29, wherein the media data of the component file is related to media data of other component files in terms of encoding dependency.   30. The file reader of claim 29, wherein the media data and the related media data include data samples.   32. The file reader of claim 31, wherein the data sample includes a network abstraction layer unit.   30. The file reader of claim 29, wherein the processor includes a video decoder.   30. The file reader of claim 29, wherein the parser further comprises means for obtaining the reference.   35. The file reader of claim 34, wherein the parser further parses the media data to obtain a reference index embedded therein and obtains a corresponding reference according to the reference index.   35. The file reader of claim 34, wherein the retriever further retrieves the other component files in parallel according to the obtained reference.   The retriever further checks a local file storage device, and if the local file storage device includes the other component file, retrieves the other component file from the local storage device. Item 37. The file reader according to Item 36.

Images