KR20130088035A - Method and apparatus for encapsulating coded multi-component video - Google Patents

Abstract

A method and apparatus for encapsulating a media entity comprising one or more layers, for each layer, into a plurality of component files is described together with the corresponding method and apparatus for reading a component file. A new box is proposed for extension to the extractor data structure of ISO BMFF and SVC / MVC file formats. This new box allows access to the referenced component files in parallel with the processing of the current component file. The extractor extension of the present invention allows for NAL units references across different component files. The present invention enables adaptive HTTP streaming of media files.

Description

METHOD AND APPARATUS FOR ENCAPSULATING CODED MULTI-COMPONENT VIDEO}

This patent application is entitled, "Extension to the Extractor data structure of SVC / MVC file formats," and was filed on June 14, 2010. Patent application serial number 61 / 354,422, and entitled "Some extensions for ISO Base Media File Format for HTTP streaming", June 2010 14 It claims the benefit of priority from US Provisional Patent Application Serial No. 61 / 354,424, filed May. The teachings of the above identified patent applications are expressly incorporated herein by reference.

The present application, "the coded multi-method and apparatus for rating the capsule component video (Method and Apparatus for Encapsulating Coded Multi - component Video ) "and at the same time, the following pending and generally owned US Patent Application Serial No. __ / ______ (Agent No. PU100140), filed at the same time. The teachings of the patent application are expressly incorporated herein by reference.

The present invention relates generally to HTTP streaming. More specifically, the present invention provides coded multi-component video streams, such as scalable video coding (SVC) streams and multi-view coding (MVC) streams for HTTP streaming. To encapsulating a media entity for.

In HTTP streaming applications, encoded video is often encapsulated and stored on the server side as a BMFF-compliant file, such as an MP4 file. In addition, to realize adaptive HTTP streaming, the file is usually divided into a number of movie fragments, which are further grouped into addressable segments by client URL requests. Indeed, different encoded representations of video content are stored in the segments so that the client can dynamically select the desired representation for downloading and playing back during the session.

Like SVC or MVC bitstreams, encoded and layered video decodes different subsets of the bitstream, thereby rendering representations of different operating points, i.e., temporal / spatial resolution, quality, views, etc. This makes it possible to provide normal support for this bitrate adaptation. However, existing ISO Base Media File Format (BMFF) standards, such as the MP4 file format, do not support individual access to each layer or representation and therefore cannot be applied to HTTP streaming applications. As shown in FIG. 1, in the MP4 file format, metadata for all layers or representations for one media file is represented by a moov movie box ( moov). Movie content, but media content data for all layers or representations is stored in the mdat movie box ( mdat). Movie Box). In HTTP streaming, when a client requests one layer, all the layers or expressions are mixed together, so the entire file must be transferred because the client does not know where to find the required layer or expression.

As will be known later, in adaptive HTTP streaming applications, it may traverse movie fragment or component file boundaries and reference media data samples such as network abstract layer (NAL) units. It is desirable to be able to. In the SVC / MVC context, such a reference can be established by using mechanisms such as an "extractor". Extractor Corrects SVC / MVC for AVC File Format Extensions in BMFF: Information Technology- coding of audio - visual objects - Part 15: Advanced Video Coding (AVC) file format , Amendment 2: File format support for Scalable Internal file data structure as defined in Video Coding, 2008, pages 15-17 . The extractor is designed to enable extraction of NAL units from other tracks by reference, without duplicating. Here, a track is a timed sequence of related samples in an ISO base media file. For media data, a track corresponds to a sequence of images or sampled audio. The syntax of the extractor is shown below:

class aligned (8) Extractor () {

NALUnitHeader ();

unsigned int (8) track_ref_index;

signed int (8) sample_offset;

unsigned int ((lengthSizeMinusOne + 1) * 8)

 data_offset;

unsigned int ((lengthSizeMinusOne + 1) * 8)

data_length;

}

The semantics of the extractor data structure are as follows:

NALUnitHeader: according to ISO / IEC 14496-10 Annex G for NAL units of type 20 NAL unit structure specified:

nal_unit_type will be set to the extractor NAL unit type (type 31).

forbidden_zero_bit, reserved_one_bit, and reserved_three_2bits are in ISO / IEC 14496-10 Annex G Will be set as specified.

The other fields (nal_ref_idc, idr_flag, priority_id, no_inter_layer_pred_flag, dependency_id, quality_id, temporal_id, use_ref_base_pic_flag, discardable_flag, and output_flag) are described by Information Technology- Coding. of audio - visual objects - Part 15: Advanced Video Coding (AVC) file format , Amendment 2: File format support for Scalable As specified in B.4 of Video Coding, ISO / IEC 14496-15: 2004 / Amd . 2: 2008, page 17 Will be set together.

track_ref_index specifies the index of a track reference of type 'scal' used to find the track from which to extract data. The samples in the track from which the data is extracted are arranged in time, or use only a time-to-sample table adjusted by the offset specified by sample_offset with the sample containing the extractor. When preceded most closely on the media decoding timeline. The first track reference has an index value of 1; The value 0 is reserved.

sample_offset gives the associated index of the sample in the linked track to be used as the source of information. Sample 0 (zero) is a sample having the same or closest leading decoding time as compared to the decoding time of the sample comprising the extractor; Sample 1 (one) is the next sample and sample -1 (minus 1) is the previous sample.

data_offset: The offset of the first byte in the reference sample to duplicate. If the extraction starts with the first byte of data in that sample, the offset gets a value of zero. The offset will refer to the beginning of the NAL unit length field.

data_length: The number of bytes to duplicate. If this field gets a value of 0, the entire single referenced NAL unit is replicated (i.e., the length field to be referenced by the data offset is increased by an additional_bytes field in the case of aggregators). Obtained from.

Additional details are provided in the Information Technology - Coding of audio - visual objects- Part 15: Advanced Video Coding ( AVC ) file format , Amendment 2: File format support for Scalable Video Coding , ISO / IEC 14496-15: 2004 / Amd.2: 2008 .

Currently extractors can extract NAL units from other tracks but belonging to the same movie box / fragment only by reference. In other words, it is not possible to use extractors to extract NAL units from different segments or files. This restriction limits the use of extractors in the case of said use.

If the client has already downloaded one or more content components of a piece of media content from the server and is in the process of downloading another content component, the client may request another request that is necessary to download the complete set of components. In order to be able to hear it, it is necessary to know whether previously downloaded content components exist between the set of dependent components of the new content components. The above use case also requires a mechanism for signaling external and dependent content components and their location information.

For BMFF, there is a box type called "tref", which is used to provide a reference from the containing track to another track in the presentation. The box can be used to account for dependencies between tracks, but the dependency is limited to tracks in the same media file.

One approach is to signal this information using some out-of-band mechanism. For example, in an HTTP streaming application, the server may send a manifest file to the client before the session begins. The manifest file is a file containing dependency and location information for each content component of the requested media content. This allows the client to request all necessary component files. However, this out-of-band approach is not applicable for local file playback, in which case no manifest file is available.

Previous solutions to the above mentioned problems have not been properly established in the art. It would be desirable to provide the ability to analyze and encapsulate layers without sacrificing speed and transport efficiency. These results have not been achieved so far in the industry.

To save storage space, it is desirable to be able to reference media data samples, such as NAL units, across movie fragment or component file boundaries, without actually copying the same data within each component file. However, ISO-based media file formats (BMFFs) and extensions do not currently support this feature. Therefore, this is to be solved.

The present invention relates to methods and apparatuses for encapsulating component files from a media entity comprising one or more layers and for reading the component file.

According to one aspect of the present invention, a method is provided for encapsulating and generating component files from a media entity comprising one or more layers. The method extracts metadata and media data corresponding to the extracted metadata for each layer from the media entity. The extracted media data and metadata are combined for each layer to enable creation of a component file that includes the extracted metadata and the extracted media data.

According to another aspect of the present invention, a file encapsulator is provided. The file encapsulator includes an extractor for extracting metadata and media data corresponding to the extracted metadata for each layer from the media entity; And a correlator for combining the extracted media data with the extracted metadata to enable generation of a component file for each layer.

The above features of the present invention will become more apparent by explaining exemplary embodiments in detail, with reference to the following attached drawings.

In order to solve the problems presented above, embodiments of the present invention relate to media data of a movie fragment or component file, whereby references to their additional media data required are identified and established. The reference is coupled to the component file with metadata and media data, but not the additional media data. References to each layer may be inserted into the extracted media data, and then the extracted metadata and the extracted media data may be combined for each layer to generate corresponding component files.

1 is a diagram illustrating an example of an MP4 file format.
2 illustrates one embodiment of the present invention for encapsulating a media entity.
3 illustrates the structure of an encapsulator used to encapsulate or generate component files from a media entity comprising multiple layers / expressions.
4 illustrates an example of combining additional media data with component files based on a dependency relationship.
5 shows an example for extracting, by reference, a NAL unit from a movie box / fragment that is different from a movie box / fragment in which an extractor is included.
FIG. 6 illustrates included encapsulation operations for an SVC / MVC type video bitstream into multiple component files, using one of the new extractor data structures invented. FIG.
7 shows the structure of a file reader used to read component files.
8 illustrates a process for reading an encapsulated component file for a video decoder that includes one embodiment of the present invention.
9 illustrates encapsulation operations for an SVC / MVC type video bitstream into multiple movie fragments using other preferred new extractor data structures.
10 illustrates a process for reading an encapsulated component file for a video decoder incorporating another embodiment of the present invention.

In the present invention, a media entity, such as one media file, a set of media files, or streaming media, is divided or encapsulated into a number of movie component files addressable by client URL requests. Here, component file is used in a broader sense to represent fragments, segments, files, and other equivalent terms.

In one embodiment of the present invention, a media entity comprising multiple representations or components is analyzed to extract metadata and media data for each representation / component. Examples of representations / components include layers such as layers having various temporal / spatial resolutions and quality of SVC, and a view of MVC. Next, layers are also used to refer to representations / components, and these terms are used interchangeably. Metadata describes, for example, what is included in the media entity for each representation and how to use the media data contained therein. The media data includes any essential information about the purpose of the media data, such as the media data samples required to achieve decoding of the content, or how to obtain the required data samples. The extracted metadata and media data for each representation or layer are combined / correlated and stored for user access. The storage operation can be performed physically on a hard drive or other storage media, or when interfacing with other applications or modules, if the metadata and media data are actually located in different places on the storage media. It may be performed virtually through a relationship management mechanism to indicate that data is stored together. 2 shows an example for this embodiment. In FIG. 2, the media entity includes three layers: base layer, enhancement layer 1 and enhancement layer 2. The media entity is analyzed to extract metadata and media data for each of the three layers, which are stored separately as component files containing the metadata and corresponding media data joined together.

3 shows the structure of a preferred encapsulator 300 used to encapsulate and generate component files from a media entity that includes multiple layers, such as SVC encoded videos. The input media entity 310 is communicated to the metadata extractor 320 and the media data extractor 340. The metadata extractor 320 extracts metadata 330 for each layer. The media data extractor 340 accepts the metadata 330 and extracts the media data 350. Note that in another embodiment, the metadata extractor 320 and the media data extractor 340 are implemented as one extractor. Both data, metadata 330 and media data 350, are supplied to the correlator 380, which combines these two types of data and output component files (one component file for each layer). 390).

Layered video, such as video encoded by AVC extension of SVC or MVC, includes a number of media components (scalable layers or views). This encoded bitstream may provide different operating points, i.e., representations or layers, by temporal / spatial resolution, quality, view, etc., by decoding different subsets of the bitstream. In addition, there are coding dependencies between the layers of the bitstream, ie the decoding of the layer may depend on other layers. Thus, requesting one of the representations of such a bitstream may require retrieving and decoding one or more components or media data from the encapsulated video file. To facilitate the extraction process for different representations, encoded and layered video is often encapsulated into an MP4 file in such a way that each layer is stored separately in different segments or component files. In this case, certain media data samples, such as NAL units, of the bitstream are required by multiple segments or component files, due to the decoding dependencies described above or other dependencies based on the application, It is worth considering that they are relevant.

In another embodiment of the present invention, additional media data required by the segment or component file is extracted and combined with the segment or component file. 4 shows an example for this embodiment. In this figure, the SVC bitstream includes three spatial layers, HD1080p, SD and QVGA. Three movie fragments or component files corresponding to three operating points are formed, each of which may be addressed by a different URL. Within each movie fragment or component file, all the media data samples required for decoding, in this example, NAL units are duplicated and stored as media samples contained in the "mdat" box. Thus, when a client requests a particular operating point or expression by using the appropriate URL, the server can retrieve that movie fragment or component file, which can be passed to the client. In this embodiment, the media data extractor 340 of FIG. 3 further extracts, for each layer, additional media data related to the extracted media data for each layer. The correlator 380 further combines additional extracted media data for each layer to generate corresponding component files.

To save storage space, it is desirable to be able to reference media data samples, such as NAL units, across movie fragment or component file boundaries, without actually copying the same data within each component file. However, ISO-based media file formats (BMFFs) and extensions do not currently support this feature. In order to solve this problem, in a further embodiment of the present invention, references are made to and associated with the media data of a movie fragment or component file, as required by them. The reference is coupled to the component file with metadata and media data, but not the additional media data. References to each layer may be inserted into the extracted media data, and then the extracted metadata and the extracted media data may be combined for each layer to generate corresponding component files.

In this embodiment, the reference identifier 360 is added to the structure of the encapsulator 300. Reference identifier 360 identifies from input media entity 310 references to their additional media data related to the extracted media data 350 for each layer. References 370 then extract the extracted metadata 330 and extraction for each layer by, for example, inserting the references into the extracted media data 350 to generate corresponding component files 390. Media data 350 is combined with a correlator 380.

As discussed above, in an SVC / MVC environment, such a reference may be established by using mechanisms such as an “extractor”. Currently extractors can only extract, by reference, NAL units from other tracks, but belonging to the same movie box / fragment. In other words, it is not possible to use extractors to extract NAL units from different segments or files. This constraint limits the use of extractors in other cases. Subsequently, an extension to the extractor data structure is disclosed, where the extension is aimed at supporting efficient encapsulation of SVC / MVC type layered video content into a number of component files described above.

The extension is added to provide the extractor data structure with additional functionality that references NAL units that belong to different movie box / fragment or component files other than the file to which the extractor belongs.

The extended extractor is defined as follows:

construction:

aligned (8) class DataEntryUrlBox (bit (24) flags)

extends FullBox ('url', version = 0, flags) {

string location;

}

aligned (8) class DataEntryUrnBox (bit (24) flags)

extends FullBox ('urn', version = 0, flags) {

string name;

string location;

}

class aligned (8) Extractor () {

NALUnitHeader ();

DataEntryBox (entry_version, entry_flags) data_entry; // added extesion

unsigned int (8) track_ref_index;

signed int (8) sample_offset;

unsigned int ((lengthSizeMinusOne + 1) * 8)

data_offset;

unsigned int ((lengthSizeMinusOne + 1) * 8)

data_length;

}

meaning :

data_entry is a Uniform Resource Locator (URL) or Uniform Resource Name (URN) entry. Name is a URN and is required for a URN entry. Location is a URL, required for a URL entry, selectable from the URL entry, which gives the location to find the resource via the given name. Each is a null-terminated string that uses UTF-8 characters. If the self-contained flag is set, the URL form is used and no string exists; The box ends with an entry-flags field. The URL type will be a service for delivering files. Relative URLs are acceptable and are associated with a file containing a movie box / fragment containing the track to which the extractor belongs.

The other fields have the same meaning as the original extractor described above.

With the extended extractor, it is now possible to extract, by reference, a NAL unit from a movie box / fragment that is different than the extractor belongs to. FIG. 5 shows an example having the same SVC bitstream as FIG. 4 but using a new extended extractor data structure. As can be seen from this figure, the SD movie fragment can now refer to NAL units from QVGA movie fragments. Similarly, the HD1080p movie fragment can use extractors to reference NAL units from both QVGA and SD movie fragments. Compared with FIG. 4, no NAL units are copied across these movie fragments, thus saving storage space.

FIG. 6 illustrates included encapsulation operations for an SVC / MVC type video bitstream into multiple movie fragments or component files, using the inventive extractor data structure. The process begins at step 601. Each NAL unit is read one at step 610. If the end of the bitstream reaches step 620, the process ends at step 690; Otherwise, the process proceeds to the next step 630. Decision step 630 determines whether the current NAL unit is dependent on NAL units from another track for decoding. If the determination is that the current NAL unit is not dependent on NAL units from other tracks for decoding, then control passes to step 640, where a sample using the current NAL unit is formed. , It is located on the current track. If the determination from step 630 is that there is a dependency between the current NAL unit and NAL units from another track, the process proceeds to step 650. The decision step 650 further determines whether the track for which NAL units are required by the current NAL unit belongs to the same movie fragment. If the determination is that the track belongs to the same movie fragment, step 670 is utilized to fill in the extractor extended to reference the NAL unit from another track. If the determination is that the track belongs to a different movie fragment, then the URL or URN of this movie fragment is identified at step 660 and the process proceeds to step 670 via the identified URL and URN to be populated with the extended extractor. Proceed. After this extended extractor is filled, it is inserted in the current track at step 680. The process then begins at step 610 with the next NAL unit.

In another embodiment, the references 370 are inserted into the extracted metadata 330, with indices for the reference 370 added to the extracted media data 350 through the correlator 380, where the correlator 380 further combines metadata and media data for each layer to create corresponding component files 390. In the context of the ISO media based file format, a box called an HTTP streaming information box is disclosed. This box contains information that can help with HTTP streaming of ISO files. It is desirable that the HTTP streaming information box be located as early as possible in the component files, eg at the beginning of the files. The box may also serve as a source by the server when forming the manifest file for the client. Another type of box called a Media Reference Box included in an HTTP streaming information box is also disclosed. This box contains information about external dependent files. The extractor structure is further extended to reference media samples across different component files. The information contained within the media reference box can be utilized by the extractors to avoid signaling overhead.

Detailed definitions of the proposed HTTP streaming information box, media reference box, and further improved extractors are as follows.

● HTTP streaming information box

Definition:

Box type: 'hsin'

Container: File

Mandatory: No

Quantity: 0 or 1

The HTTP Streaming Information box assists in the HTTP streaming operation of ISO media files. This includes, among other possible types of boxes, relevant information about the HTTP streaming delivery of a file containing a media reference box defined below. The HTTP streaming information box is preferably located as early as possible within the files for maximum utility.

Syntax:

aligned (8) class HTTPStreamingInfoBox extends Box ('hsin') {

}

● Media Reference Box

Justice :

Box type: 'mref'

Container: 'hsin'

Force: No

Quantity: 0 or 1

The media reference box is included in the HTTP streaming information box and includes a table of data references in the form of URLs indicating the locations of the external files on which each track included in this box depends. By reading this box, the file reader can identify external dependent file sources, such as external component files, of tracks in the file, and means for retrieving them.

Syntax:

aligned (8) class DataEntryUrlBox (bit (24) flags) extends Box ('url') {

string location;

}

aligned (8) class MediaReferenceBox extends Box ('mref') {

unsigned int (16) entry_count;

for (i = 1; i <= entry_count; i ++) {

unsigned int 32 track_ID;

unsigned int (16) dependent_source_count;

for (j = 1; j <= dependent_source_count; j ++) {

DataEntryUrlBox data_entry;

}

}

}

meaning :

entry_count: An integer that counts actual entries.

track_ID: an integer identifying only the track in the file to which the box applies;

dependent_source_count: An integer that counts external media sources that the track in the file with track_ID depends on.

data_entry: A URL entry indicating one external media source that the specified track is dependent on. Each of these is a null terminated string, using UTF-8 characters. The URL type will be a service for delivering files. Relative URLs are acceptable and related to the file containing this media reference box.

The media reference box defined above is designed to facilitate, in many ways, the HTTP streaming of a media entity comprising one or more layers.

First of all, it is possible to explicitly signal the dependencies between the component files at the beginning of the component file via the reference table. Thus, once the client has downloaded a small portion of the component file, all relevant external component files for its own track (s) can be known and, if necessary, the complete set (s) for playback via the references contained in the table. The requests can be made to obtain.

Second, in-file information from this box can be easily extracted to be included in the manifest file. This information in the manifest allows the service initialization, such as the client to find relevant service information, request all associated component files, and allocate required buffer resources, before actual HTTP streaming. Can help to perform.

Third, when a client requests a different representation of some multi-component media content and another representation has already been delivered as a component file, the client checks the corresponding media reference box in the file so that the file can be reused. You can check whether it includes any dependent components for the new representation.

In turn, this box helps to reduce the signaling overhead of the extended extractor structure as defined below.

● extractors

The extractor was further proposed to extend the ability to reference data from tracks of external media files.

Extended syntax:

class aligned (8) Extractor () {

NALUnitHeader ();

unsigned int (16) media_reference_index;

unsigned int (8) track_ref_index;

signed int (8) sample_offset;

unsigned int ((lengthSizeMinusOne + 1) * 8)

data_offset;

unsigned int ((lengthSizeMinusOne + 1) * 8)

data_length;

}

meaning :

media_reference_index: specifies the index of the entry for the reference table contained within the media reference box with the same combined track_ID value as the track containing the extractor. If media_reference_index is equal to 0, the extractor references data from another track but belonging to the same file as the extractor. In this case, there would not be a lookup table in the media reference box with the same track_ID value as the track. If media_reference_index is between 1 and the value of dependent_source_count from the reference table associated with the track from the media reference box, the URL referenced by the media_reference_index from the reference table indicates an external file, where the extractor extracts data. It includes a track.

The meanings of the other fields are still the same as the original extractor definition.

With an additional extended extractor structure, it is now possible to link to data from tracks belonging to an external component file and to use extractors to extract the data. This is particularly useful when content components from encoded pieces of multi-component media content, such as encoded by SVC or MVC, are encapsulated into different component files. With extended extractors, extraction can occur across file boundaries. This avoids copying the same data in different component files.

FIG. 9 includes an encapsulation operation for an SVC / MVC type video bitstream into multiple movie fragments or component files, using the disclosed HTTP streaming information box, media reference box, and additional extended extractor data structure. Show them. The process is similar to the process shown in FIG. 6, including some modifications due to the boxes described above, and further expansion of the extractor. After the location information URL / URN has been identified at step 660, the location information is used to populate a lookup table in the mref box (Media Reference Box) at step 965. Step 970 further populates the extractor with indices for the location information of the lookup table. The extractor is then inserted into the current track. When the end of the bitstream reaches step 620, the mref box and the container hsin box (HTTP Streaming Information Box) are inserted into the metadata of the component file.

To read the component file, the file reader 700 shown in FIG. 7 is utilized. The analyzer 710 first analyzes the component file to obtain metadata, media data, and, if available, a reference. According to the decoded reference, if the media data is related to the media data of other component files, such as through a decoding dependency, then the searcher 720 retrieves the relevant media data from the other component files indicated by the reference. do. Processor 730 further processes metadata obtained from the component file, media data, and additional media data, if available. Analysis operations by analyzer 710 include various and necessary operations to obtain metadata, media data prepared for processor 730, and references prepared for searcher 720. This may include further analyzing the metadata and / or media data when needed. In one embodiment, the reference is inserted into the media data, so that the reference is obtained by analyzing the media data. If a reference is available, the analyzing step further includes parsing the reference, and decoding the reference. If the component file includes video content, processor 730 may include a video decoder. In one different embodiment, the analyzer and searcher may be incorporated into a processor.

8 shows a process for reading an SVC / MVC type video bitstream for a video decoder incorporating the present invention. Step 801 accesses the component video file, where the metadata and media data of the component video file is identified in step 805 for each layer. The identified metadata and media data are analyzed at step 810 and each NAL unit of media data is read one at step 815. For the current NAL unit, a decision is first made at step 820 to determine whether the end of the bitstream has been reached, and if the answer is yes, then the process ends at step 825. do. Otherwise, the process proceeds to decision step 830 to determine whether the current NAL unit is an extractor. If the current NAL unit is not an extractor, this means that the current NAL unit is a generic NAL unit that includes decoding the data, which is sent to the decoder in step 835. If the current NAL unit is an extractor, it is determined at step 840 whether the current NAL unit is dependent on NAL units outside the same component file. If the required NAL unit belongs to the same component file, it is retrieved from the current file in step 845 and sent to the decoder in step 835. If the NAL unit that is required comes from another component files, NAL units are located in the extractor use the reference information Data _ entry in step 850, it is retrieved from the remote file (remote file), at step 855, since In step 835 it is sent to the decoder.

In another embodiment, the reference is identified at analyzer 710 by obtaining the inserted reference indices and analyzing the media data to obtain a corresponding reference according to the reference indices. The corresponding process of reading an SVC / MVC type video bitstream for a video decoder is shown in FIG. 10, which is similar to the process of FIG. 8. In step 810, according to one preferred embodiment, because the reference is located at the beginning of the component file, the analysis of the metadata in step 810 enables analysis of the included reference in parallel with the analysis of the media data. Let's do it. When resolving the reference, the other component files referenced are identified in step 1014. The search for the other component files begins at step 1012, in parallel with the remaining steps of the process. After accessing the location information of the component file upon which the current NAL unit is dependent in step 850, local storage, such as a media buffer, is checked for availability of such component file. If the required component file is available locally, a NAL unit of local copy is retrieved; Otherwise, the NAL unit from the remote file is retrieved. Note that a local copy of the component file may be obtained by a parallel search in step 1012, or this may be obtained from a previous request of such a component file.

Although preferred embodiments of the present invention have been described in detail herein, the present invention is not limited to the above embodiments, and other modifications and variations are intended from the scope of the invention, which is defined by the appended claims. It will be appreciated that it may be accomplished by one of ordinary skill in the art without departing.

300: encapsulator 310: media entity
320: metadata extractor 330: metadata
340: Media Data Extractor 350: Media Data
360: reference identifier 370: reference
380: Correlator 390: Component Files
700: File Reader 710: Analyzer
720: Finder 730: Processor

Claims (37)

A method for generating component files from a media entity comprising one or more layers, the method comprising:
Extracting metadata for each layer from the media entity;
Extracting media data from the media entity, corresponding to the extracted metadata for each layer of the media entity; And
Associating the extracted media data with the extracted metadata to enable generation of a component file including the extracted metadata and extracted media data for each layer; Included,
A method for generating component files from a media entity that includes one or more layers. The method of claim 1, wherein the component file is at least one of a movie box, a movie fragment, a segment, and a file.
A method for generating component files from a media entity that includes one or more layers. The method of claim 1, further comprising: extracting, from the media entity, for each layer, additional media data related to the extracted media data for each layer; And
Combining, for each layer, the extracted media data and the additional media data to generate corresponding component files;
A method for generating component files from a media entity that includes one or more layers. The method of claim 1,
Identifying references to additional media data related to the extracted media data for each layer; And
Combining the extracted metadata and extracted media data with the references, for each layer, to generate corresponding component files;
A method for generating component files from a media entity that includes one or more layers. The method of claim 4, wherein the media data and additional media data comprise data samples.
A method for generating component files from a media entity that includes one or more layers. The apparatus of claim 5, wherein the data sample comprises a network abstract layer unit.
A method for generating component files from a media entity that includes one or more layers. The apparatus of claim 6, wherein the references include, in the additional media data, at least one of a uniform resource locator and a uniform resource name of the network abstract layer unit.
A method for generating component files from a media entity that includes one or more layers. 5. The method of claim 4,
Inserting the references into the extracted metadata for each layer; And
Adding indices to the references in the extracted media data; further comprising:
A method for generating component files from a media entity that includes one or more layers. The method of claim 8, wherein the references are located at the beginning of the component file for each layer.
A method for generating component files from a media entity that includes one or more layers. The apparatus of claim 8, wherein the references are populated in media reference boxes and the indices are populated in extractors.
A method for generating component files from a media entity that includes one or more layers. A file encapsulator for generating component files from a media entity that includes one or more layers, the encapsulator comprising:
An extractor for extracting metadata for each layer from the media entity and extracting media data from the media entity corresponding to the extracted metadata for each layer of the media entity; And
A correlator for combining the extracted media data with the extracted metadata to enable generation of the respective layer of the extracted metadata and the component file including the extracted media data. Comprising;
File encapsulator for generating component files from a media entity that includes one or more layers. The method of claim 11, wherein the component file is at least one of a movie box, a movie fragment, a segment, and a file.
File encapsulator for generating component files from a media entity that includes one or more layers. 12. The apparatus of claim 11, wherein the extractor further extracts additional media data related to the extracted media data for each layer from the media entity for the respective layer; The correlator further combines the extracted media data and the additional media data for each layer to generate corresponding component files,
File encapsulator for generating component files from a media entity that includes one or more layers. 12. The apparatus of claim 11, further comprising a reference identifier for identifying a reference to additional media data from the media entity associated with the extracted media data for each layer, the reference being: Combined with the extracted metadata and extracted media data for each layer through the correlator to generate component files,
File encapsulator for generating component files from a media entity that includes one or more layers. The method of claim 14, wherein the media data and the additional media data comprise data samples.
File encapsulator for generating component files from a media entity that includes one or more layers. The apparatus of claim 15, wherein the data sample comprises a network abstract layer unit.
File encapsulator for generating component files from a media entity that includes one or more layers. 17. The apparatus of claim 16, wherein the reference comprises, in the additional media data, at least one of a uniform resource locator of the network abstraction layer and a uniform resource name.
File encapsulator for generating component files from a media entity that includes one or more layers. 15. The apparatus of claim 14, wherein the correlator further inserts the reference to the extracted metadata for each layer and adds indices to the references in the extracted media data.
File encapsulator for generating component files from a media entity that includes one or more layers. 19. The system of claim 18, wherein the correlator locates the references at the beginning of the component file for each layer.
File encapsulator for generating component files from a media entity that includes one or more layers. 20. The apparatus of claim 19, wherein the references are populated in media reference boxes and the indices are populated in extractors.
File encapsulator for generating component files from a media entity that includes one or more layers. A method for reading a component file,
Analyzing the component file to obtain metadata, media data, and references; And
In accordance with the references, if the media data of the component file relates to media data of other component files, retrieving the related media data from the other component files using the references; Included,
Method for reading a component file. The method of claim 21, wherein the media data of the component file is related to media data of other component files according to a coding dependency.
Method for reading a component file. The method of claim 21, wherein the media data and the related media data comprise data samples.
Method for reading a component file. The method of claim 23, wherein the data sample comprises a network abstract layer unit.
Method for reading a component file. The method of claim 21, further comprising analyzing the metadata to obtain the references.
Method for reading a component file. 26. The method of claim 25, further comprising: analyzing the media data to obtain embedded reference indices; And
Acquiring corresponding references according to the reference indices;
Method for reading a component file. The method of claim 25, wherein the retrieving comprises simultaneously retrieving the other component files in accordance with the references.
Method for reading a component file. 28. The method of claim 27, wherein said retrieving comprises: checking a local file storage; And
If the local file store is included in the other component files, retrieving the other component files from the local store; further comprising:
Method for reading a component file. As a file reader,
A parser for analyzing component files to obtain metadata, media data, and references;
A searcher for retrieving media data related to the media data from other component files in accordance with the reference; And
A processor for processing the retrieved media data from the metadata, media data, and other component files;
File reader. The method of claim 29, wherein the media data of the component file is related to media data of other component files by a coding dependency.
File reader. 30. The apparatus of claim 29, wherein the media data and the related media data comprise data samples.
File reader. 32. The apparatus of claim 31, wherein the data sample comprises a network abstract layer unit.
File reader. The processor of claim 29, wherein the processor comprises a video decoder.
File reader. 30. The apparatus of claim 29, wherein the analyzer further comprises means for obtaining the references.
File reader. 35. The apparatus of claim 34, wherein the analyzer further analyzes the media data to obtain inserted reference indices, and obtains corresponding references according to the reference indices,
File reader. 35. The system of claim 34, wherein the searcher further searches the other component files simultaneously in accordance with the obtained references.
File reader. 37. The system of claim 36, wherein the searcher further checks a local file store and, if the local file store includes the other component files, retrieves the other component files from the local store,
File reader.

Images