GB2620585A

GB2620585A - Method and apparatus for encapsulating media data in a media file

Info

Publication number: GB2620585A
Application number: GB2210192.7A
Authority: GB
Inventors: Bellessort Romain; Denoual Franck; Maze Frédéric; Tocze Lionel; Le Feuvre Jean
Original assignee: Canon Inc
Current assignee: Canon Inc
Priority date: 2022-07-11
Filing date: 2022-07-11
Publication date: 2024-01-17
Also published as: GB202210192D0

Abstract

Encapsulating in a media file media data comprising samples organized into subsamples, involving generating a subsample description box. The subsample description box comprises a description of each subsample comprising a subsample size. The key feature is the flexibility of the new box (here labelled FlexibleSubSampleInformationBox (subf) in one embodiment) as it associates property values with subsamples and further includes an indication of the property types of the associated property values. This solves issues with existing ISO BMFF SubsampleInformationBox (subs) as it allows more flexible use of a variety of different parameters to be defined. The box structure may refer to another external structure for the definitions of the property types. Preferably the box includes a list of property types used in the box for ease and functionality when parsing.

Description

METHOD AND APPARATUS FOR ENCAPSULATING MEDIA DATA IN A MEDIA

FILE

FIELD OF THE INVENTION

The present disclosure concerns a method and a device for encapsulating media data in a media file. It concerns more particularly the encapsulation of media data samples comprising subsamples.

BACKGROUND OF INVENTION

The ISO base media file format (ISOBMFF, also called file format) is a general format forming the basis for a number of other more specific file formats. ISOBMFF is standardized by the International Standardization Organization as ISO/IEC 14496-12. This format contains characteristics of the media data like the timing, size or media information for timed sequences of media data, such as audio-visual presentations. We call these characteristics metadata (or structure-data), in opposition to media data (or data). In other words, an ISOBMFF based file comprises media data along with metadata describing the media data, their nature, their location in the file, and their organization. An ISO Base media file (that we later call media file or movie file or media presentation) may come as one file containing the whole presentation or as multiple files, each segment containing a temporal portion of the presentation. A segment may contain one or more fragments. An ISO Base media file is structured into "boxes". In the file format, the overall presentation is called a movie ('rnoov' box, as illustrated by Figure 6a). It is logically divided into tracks (see also box 'trak' on Figure 6a); each track represents a timed sequence of media (frames of video, audio, or any other timed media data). Within each track, each timed unit is called a sample. Each track has one or more sample description boxes ('stbl' box on Figure 6a).

All the structure-data or metadata, including that defining the placement and timing of the media, is contained in structured boxes. The media data (frames of video, for example) is referred to by this structure-data or metadata. The overall duration of each track is defined in the metadata. Each sample has a defined duration. The exact decoding timestamp of a sample is defined by summing the durations of the preceding samples. Each box has a type identified by an identifier constituted by a four-character string, called 4CC. We may say that a given box identified by a 4CC, 'meta' or 'trak' for example, is called 'meta' or 'trak', which means that the box has a type identified by the 4CC 'meta' or 'trak'. A box of a given type as identified by its 4CC must comply to a predefined syntax.

ISOBMFF file may also contain general untimed media data, for example still images or sequence of untimed images. These untimed media data are described in meta data through items boxes of type 'item'.

Figure 6a illustrates a media file comprising media data and a subsample description box 'subs'.

For each sample of data stored in media data, a metadata box, called 'subs', is defined to describe subsample organisation and content. Figure 6a provides an example of such a box. In this example, the 'subs' box typically comprises a description of the samples comprised in the corresponding tridat box for the given track. This description is not included on Figure 6a, but it basically indicates that the media file comprises 2 samples, and that the first one contains 3 subsamples, while the second one contains 4 subsamples. For each subsample, its byte range in particular is provided in the subsample description box 'subs'. More details about the description of subsamples according to ISOBMFF are provided in this document.

A subsample is a contiguous range of bytes of a sample. The specific definition of a subsample shall be supplied for a given coding system. Depending on the kind of data a sample represents, a subsample may be a NAL unit (ISO/IEC 14496-15) from AVC, HEVC, WC... or any NAL-unit based video codec, a data chunk, a Type-lengthvalue unit or G-PCC unit (130/IEC 23090-18). In video samples, a subsample may correspond to a picture, to a sub-picture, to a tile, to a slice or to a coding tree unit or to a NAL unit. As another example, for volumetric media data, a subsample may correspond to a Point Cloud frame, to a Point Cloud subframe, to a tile, to a slice or to a TLV unit. Finally, as a last example, in the context of HEIF (High Efficiency Image Format), a subsample may for instance correspond to the files that are contained within one VCL NAL unit. Subsample information for a coded image may be given using an item property, associated with the image item describing the coded image, that is exactly identical to SubSampleInformationBox as defined in ISO/I EC 14496-12 and for the coding format of the associated coded image item.

Subsample description may be available at the beginning of a media file (e.g. in sample description of tracks described under a 'moov' box, as illustrated in Figure 6a) or may come with movie fragments (e.g. in 'ft& boxes). It may also come as an item property when media data are described as data which does not require timed processing, as opposed to sample data (e.g. as information items under a 'meta' box).

Figure 6b illustrates the syntax of the subsample description box 'subs' according to the prior art.

As illustrated by Figure 6b, a subsample has the following attributes: - subsample_size: 16-or 32-bit integer that defines the number of bytes for this subsample; subsample_priority 8-bit integer that defines a level of priority for considered subsample; - discardable: 8-bit integer that defines whether considered subsample may be discarded or not; codec_specific_parameters: 32-bit integer that defines some codec specific parameters for considered subsample (e.g. VVC-related parameters if the considered codec is VVC).

Another important aspect of the box 'subs' syntax is that multiple 'subs' boxes can be present in the same container box provided that they have different flags value. This mechanism allows codec specification to define different types of 'subs' descriptions: for each value of flags, the type of information comprised in codec_specific_parameters may be different. For instance, in G-PCC carriage (130/IEC 23090-18), it is defined that when flags value is equal to 0, codec_specific_parameters 32-bit value represents a payload type. On the other hand, when flags value is equal to 1, codec_specific_parameters represents whether the subsample is associated to a tile, and if so, it provides its tile identifier. Consequently, when there is a need to support a new type of codec specific parameter, a new flags value has to be defined in the corresponding codec specification along with an associated description of said 32 bits (e.g. 1St bit represents whether the subsample is associated with a tile or not, and if so, next 31 bits represent corresponding tile identifier).

In this document, the word "subsample" is used. This word may also be spelled "sub-sample", for instance in ISOBMFF specification.

Media data is typically made of several samples, which are themselves split in a number of subsamples. Subsamples of a sample are described using a "subs" box, which defines 4 values for each subsample: its size, its priority, whether it is discardable, and a codec_specific_parameters made of 32 bits. In case of need, the same data can be described using multiple "subs" boxes, each being associated to a different flags value; the actual content of codec_specific_parameters precisely depends on this value. As a result, when a codec would like to support a new kind of metadata that may be associated with a subsample, a new flags value has to be specified.

The description of each subsample is therefore very constrained. Some items may not be needed (discardable or subsample_priority fields for instance). Using 32 bits for codec_specific_parameters may sometimes be too much and sometimes not enough. Using multiple "subs" boxes with different flags value may lead to describe several times the same piece of data, or byte range. More flexibility in the description of subsamples would be advantageous and may be associated with a more compact description of the subsample organization.

SUMMARY OF THE INVENTION

The present invention has been devised to address one or more of the foregoing concerns. It is proposed to define the notion of property, each property being characterized by an ID (implicit, explicit or predefined) and an encoding format for its associated value (explicit or predefined). Then, each subsample can be mapped to an arbitrary number of properties. In addition, optionally, so that reader can easily determine whether to parse such a box or not, all the types of properties used in a given box are indicated at the beginning of said box.

According to a first aspect of the invention there is provided a method for encapsulating in a media file media data comprising one or more samples, at least one sample being organized into subsamples, the method comprising: generating for the at least one sample organized into subsamples a subsample description box, the subsample description box comprising for each subsample a subsample description comprising a subsample size; wherein the method further comprises: -associating property values with subsamples; and wherein: -the subsample description box comprises an indication of the property types of the associated property values; and -encapsulating the media data and the subsample description box into the media file.

In an embodiment, the indication is provided in the subsample description.

In an embodiment, the indication is provided at the sample level in the subsample description box.

In an embodiment, the indication is provided as a reference on an external data structure comprising the property types.

In an embodiment, all the property values associated with a subsample are

provided in the subsample description.

In an embodiment, at least one default property value is provided for a property and the corresponding property value is omitted in the subsample description.

In an embodiment, at least one default property value is provided for a property and a property value in the subsample description overwrites the default property value.

In an embodiment, the external data structure comprises at least one default property value.

In an embodiment, the subsample description box is provided with a used properties description structure describing the set of properties which are each associated with at least one subsample.

In an embodiment, the used properties description structure is a list of property identifiers.

In an embodiment, the used properties description structure is a bitmask, each bit of the bitmask indicating, when set, a type of property used in the subsample description 25 box.

In an embodiment, the used properties description structure comprises at least one default property value.

In an embodiment, the indication comprises in the subsample description at least a reference to properties in the used properties description structure.

In an embodiment, the indication comprises in a subsample description a bitmask of the properties in the used properties description structure, each bit of the bitmask when set indicates that the corresponding property is used in the subsample description.

In an embodiment, the subsample description comprises for each property value for which a default property value is provided an indication regarding whether the default property value applies to the subsample.

In an embodiment, the sample description does not comprise a property value for a property associated with the subsample, the missing property value corresponds to the property value provided for the same property for a previous subsample.

According to another aspect of the invention there is provided a method for reading a media file comprising media data comprising one or more samples, at least one sample being organized into subsamples, the method comprising: reading for the at least one sample organized into subsamples a subsample description box, the subsample description box comprising for each subsample a subsample description comprising a subsample size; reading in a subsample description an indication of the property types of property values associated with the subsample; obtaining the property values associated with the subsample based on the property types.

According to another aspect of the invention there is provided a computer program product for a programmable apparatus, the computer program product comprising a sequence of instructions for implementing a method according to the invention, when loaded into and executed by the programmable apparatus.

According to another aspect of the invention there is provided a computer-readable storage medium storing instructions of a computer program for implementing a method according to the invention.

According to another aspect of the invention there is provided a computer program which upon execution causes the method of the invention to be performed.

According to another aspect of the invention there is provided a device for encapsulating in a media file media data comprising one or more samples, at least one sample being organized into subsamples, the device comprising a processor configured for: - generating for the at least one sample organized into subsamples a subsample description box, the subsample description box comprising for each subsample a subsample description comprising a subsample size; wherein the processor is further configured for: associating property values with subsamples; and wherein: - the subsample description box comprises an indication of the property types of the associated property values; and encapsulating the media data and the subsample description box into the media file.

According to another aspect of the invention there is provided a device for reading a media file comprising media data comprising one or more samples, at least one sample being organized into subsamples, the device comprising a processor configured for: reading for the at least one sample organized into subsamples a subsample description box, the subsample description box comprising for each subsample a subsample description comprising a subsample size; - reading in a subsample description an indication of the property types of property values associated with the subsample; obtaining the property values associated with the subsample based on the property types.

At least parts of the methods according to the invention may be computer implemented. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit", "module" or "system". Furthermore, the present invention may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium.

Since the present invention can be implemented in software, the present invention can be embodied as computer readable code for provision to a programmable apparatus on any suitable carrier medium. A tangible, non-transitory carrier medium may comprise a storage medium such as a floppy disk, a CD-ROM, a hard disk drive, a magnetic tape device or a solid state memory device and the like. A transient carrier medium may include a signal such as an electrical signal, an electronic signal, an optical signal, an acoustic signal, a magnetic signal or an electromagnetic signal, e.g. a microwave or RF signal.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will now be described, by way of example only, and with reference to the following drawings in which: Figure 1 illustrates the main steps of an example of a method for encapsulating media data according to the invention; Figure 2a and 2b illustrates the main steps of a method for generating the used properties description structure according to two different embodiments.

Figure 3 illustrates the main steps of a method for describing a subsample in the subsample description box according to an embodiment of the invention; Figure 4 illustrates the main steps of a method for describing a subsample in the subsample description box according to another embodiment of the invention; Figure 5 illustrates the main steps of an example of method for reading encapsulated media data according to one of the encapsulation methods herein described; Figure 6a and 6b illustrates the location and syntax of the subsample description box of the prior art; Figure 7 is a schematic block diagram of a computing device for implementation of one or more embodiments of the invention; Figure 8 illustrates an example of syntax for a subsample description box according to an embodiment; Figure 9 provides an example of a SubSampleToGroupBox that may be used as

a subsample description box.

DETAILED DESCRIPTION OF THE INVENTION

Figure 1 illustrates the main steps of an example of a method for encapsulating media data according to the invention.

The process starts at step 100 by obtaining media data comprising one or more samples. Then, at step 110, a data structure, typically a box, for describing the subsamples of said samples is generated. As a remark, please note that we do not describe here the encapsulation process at a higher level (e.g. 'moov', 'trak', 'stbl' or tridat boxes) as this is considered to be well known by one skilled in the art. Said data structure generally has the same role has the 'subs' box, hence it is typically located inside the sample description, for example an 'stbl', a tar or an 'ipco' box. Generated data structure may either be a new type of box box (e.g. with a new 4CC), or a 'subs' box with a new version number, hence associated to a new syntax compared to existing 'subs' box. The data structure generated at step 110 is sometimes called subsample description box in the following.

Optionally, at step 120, a description of the set of the properties which are each associated to at least one of the considered subsamples is inserted into generated structure. A property is defined by an identifier, which may be implicit or explicit, and an encoding format used to encode values corresponding to said property. This identifier actually identifies the type of the property. For instance, a Boolean such as the discardable attribute may be defined as a property whose values are encoded on a single bit, 0 meaning false and 1 meaning true. Alternatively, another property whose values are Boolean values may be encoded on 8 bits, the first bit corresponding to the Boolean value, and the next 7 bits being used just to keep byte alignment. As another example, a property indicating a tile identifier may be defined as a 32-bit integer, with a first bit indicating whether a tile id is indicated, and next 31 bits indicating the value of considered tile identifier. More details about this optional step are provided with regards to Figures 2a and 2b.

At step 130, at least one sample organized into subsamples is obtained. The number of samples can then be inserted into the generated subsample description box of step 110 (in Figure 6b, this number corresponds to the variable "entry_count"). At step 140, it is checked whether there remains an unprocessed sample. If so, step 140 is followed by step 150 where a description of considered sample is inserted in generated structure of step 110. Such a description is typically similar to known sample description in existing 'subs' box or complies with the different embodiments described below. As illustrated by Figure 6b, sample may be characterized using sample_delta and subsample_count. Sample_delta is used to allow skipping some samples for which no subsample description is provided, typically because these samples are not divided into subsamples.

Following step 150, it is checked at step 160 whether there remains any unprocessed subsample. If so, a description of considered subsample is inserted into generated structure at step 170. This description typically comprises subsample size and, for each property, an indication on the property type as well as corresponding property value. The indication may be explicit or implicit. More details about this step are provided with regards to Figure 3 and Figure 4, that each provides a different embodiment of the subsample description method.

Step 170 is followed by step 160. When all subsamples have been processed, step 160 is followed by step 140. When all samples have been processed, step 140 is followed by step 190, where the process ends.

Eventually, the media data comprising the corresponding subsamples are encapsulated in the media file along with the subsample description box.

Figures 2a and 2b provide two different embodiments for the optional description, we may call it the used properties description structure in the following, that may be inserted in the data structure in optional step 120. This optional description describes the set of the properties which are each associated with at least one of the considered subsample in the subsample description box generated at step 110.

A property is defined by an identifier, which may be implicit or explicit, and an encoding format used to encode values corresponding to said property. If such a value is indicated for a given subsample, we say that this property is specified for said subsample. Encoding formats used to encode values are advantageously byte-aligned, so that byte alignment can be preserved when encoding said values. An example of an implicit identifier is the index of a given property in a list of properties. The list of properties may be included in the media file, for example as a Box, or may be pre-defined as code values or code points in a standard specification. An explicit identifier is, for instance, a unique integer or string assigned to a given property. In a particular embodiment, a property identifier may be a 4CC, in which case different file format specifications may refer to the same properties (e.g. "file" property could be defined as "tipC for "tile property", and any codec using tiles, such as HEVC, VVC or G-PCC, could refer to this property in its corresponding file format specification).

A property that is not present in the considered set of properties cannot be specified for any subsample of considered structure. On the other hand, a property present is necessarily used for the description of at least one of the subsamples described in the data structure generated at step 110, but not necessarily specified for each subsample.

The used properties description structure comprising the description of the set of properties is optionally inserted in the subsample description box generated at step 110, where it may allow a reader to determine whether it should parse said structure or not. Indeed, depending on use cases, a reader may need a given type of information to provide a given feature, but it may not be interested in another type of information. For instance, if an application makes no use of tile information, corresponding reader has no need to parse a structure that describes only tile information (i.e. the only property in the list of properties is a tile-related property). Parsing only useful structures allows saving time and resources. Therefore, it is advantageous that this description is inserted at the beginning of the subsample description box, so that reader can quickly determine whether it should continue the parsing or not. If there is no need to continue the parsing, reader can directly move to the next box. This is possible since with ISOBMFF, the size of each box is indicated to reader. When the optional description of the used properties is not inserted in the subsample description box, the parser needs to parse the entire box to find out if it contains relevant information to be used in the parsing depending on the target application for example.

In a given embodiment, this description is inserted at the beginning of the structure of step 110, but only after all the subsamples descriptions have been inserted in said structure, or at least generated and stored prior to their insertion in said structure (i.e. after step 140 and before step 190). Indeed, by doing so, the list of properties that are present in said structure can be built by adding each property not yet added to said list while iterating through subsamples.

Figure 2a illustrates the main steps of a method for generating the used properties description structure according to a first embodiment. This method starts at step 200 by writing the number of properties in the used properties description structure.

These properties are the ones that may be specified for a given subsample of considered obtained data (each of these properties may or may not be specified for each subsample). Then, at step 210, it is checked whether there remains an unprocessed property. If so, step 210 is followed by step 220 where an indication regarding the type of the property is written. As previously mentioned, the indication may for instance be an index in a list, for instance if properties are defined by a specification through a list, or an identifier, for instance if specification defines a unique identifier for each property. This identifier may be a 4CC associated with the property.

Step 220 is followed by step 210, and when all properties have been processed, the process ends at step 230.

Figure 2b illustrates the main steps of a method for generating the used properties description structure according to a first embodiment. The method starts at step 250 by determining a predefined set of properties. This predefined set may for instance be the list of all properties defined in the context of a given specification. We consider the predefined set to be ordered, e.g. by growing order of corresponding identifiers. For each property in this set, 1 bit may be used to indicate whether it is specified for one of the subsamples of considered structure or not. By concatenating these bits, it is therefore possible to create a bitmask indicating which properties actually occur in the subsample description box generated at step 110 once the process of Figure 1 has been completed (step 260). In this embodiment, this bitmask corresponds to the used properties description structure. This bitmask can then be inserted into said structure (at same step 260), which allows determining the set of properties that may be specified for each subsample in considered structure. Following step 260, the process ends at step 270.

If there is a need to work with aligned bytes, the bitmask may be complemented with additional bits so that the number of bits in the bitmask is a multiple of 8. For instance, if 5 different properties may be specified for each subsample, three zeros may be appended to the corresponding bitmask to reach 1 byte (for example the most significant bits, since the predefined list may use identifiers in growing order).

Optionally, in some embodiments, default values may be provided for each property present in the used properties description structure. This may be all the more useful as a given value is frequent for a given property.

In such a case, when a property is associated with a default value, the default values applies unless another value is specified for a given subsample. The default value defined at the used properties description structure level may be overwritten in the subsample description box for a given property.

As a remark, and contrary to the case where no default values are provided, it should be noted that a property for which a default value has been indicated cannot be omitted for a given subsample. This means that, for a given subsample for which the property is not explicitly specified in the subsample description box, the value of the property for said subsample is the default value. In other words, the property provided with a default value applies to all the subsamples with the provided default value if not specified at the subsample level, and with the provided value overwriting the default value if specified. Indeed, the value for a considered subsample is either the default one, either another one which has been explicitly indicated. Even though the encoding format associated to each property may define a specific value corresponding to "undefined", the result is still somehow different from having the property omitted (even though it may practically be identical, since an omitted property may be equivalent to an undefined value for said property).

The description of the set of properties that may be specified for a given subsample may also comprise a flag indicating whether this description applies to all subsamples by default. If so, this means that all properties are expected to be specified for each subsample, unless indicated differently e.g. at subsample level. For instance, even if such a flag is set to true, some means may be provided at subsample level to indicate that this default does not apply to a given subsample (e.g. through a dedicated Boolean in subsample description, or by defining that 1 bit of subsample_size is used to represent this Boolean value), in which case the list of properties specified for considered subsample would be indicated explicitly (on the other hand, if the default applies, no list of properties would be indicated at subsample level, and only the values for each property would be provided).

Figures 2a and 2b describe how to create a description of the set of properties that are used to describe subsamples. In the previously described embodiment, the optional used properties description structure is inserted in the subsample description box. In some embodiments, this description is provided in a different structure, e.g. a specific box dedicated to such kind of description. A reference to said structure may be inserted in the structure of step 110. In other embodiments, the reference is implicit due to the location of the structure. For example, if a single box containing a used properties description structure is comprised in the same container box as the subsample description box, said used properties description may implicitly be associated with said subsample description box.

As an example, descriptions of set of properties may be embedded in a SampleGroupDescriptionBox. This box enables defining a type of grouping (characterized by a unique grouping_type value), said type being associated with a SampleToGroupBox with the same type. The SampleToGroupBox defines groups of samples that can be mapped to different entries (each entry being designated by its index in the list of entries in the SampleGroupDescriptionBox, indexing starting at 1). For instance, a first entry in the SampleGroupDescriptionBox may define a given set of properties, while another entry may define a different set of properties. Once such entries have been described in a SampleGroupDescriptionBox, each entry can be referred to through the group_description_index of the associated SampleToGroupBox (with same grouping_type). For example, a specific SampleGroupEntry may be defined to store list of indexes, each index corresponding to a property index.

Figure 3 illustrates the main steps of a method for describing a subsample in the subsample description box according to an embodiment of the invention. This method describes a possible implementation of the process of step 170 in Figure 1.

First, at step 300, the subsample size is written. Subsample size is typically written on 16 or 32 bits; whether 16 or 32 bits should be used is for instance indicated through the version number or the flags value of the subsample description box. After that, at step 305, the number of properties for considered subsample is written. This number corresponds to the number of property values that will be indicated in subsample description. Then, at step 310, it is checked whether there remains an unprocessed property. If so, at step 320, an indication of the next unprocessed property type is written. As previously mentioned, this indication is typically an identifier, which may be implicit or explicit, such as an index in a list (implicit) or a unique identifier associated to considered property (e.g. an integer or a 4CC). In an embodiment, where the optional used properties description structure is provided, the indication relative to the type of the property may be an index of the property in the used properties description structure.

Step 320 is followed by step 310, and when there is no more unprocessed property, step 310 is followed by step 330 where the iteration on properties is reset (i.e. all properties are considered as unprocessed). Another iteration then starts, and at step 340, it is checked whether there remains an unprocessed property. If so; the value of the next unprocessed property is written at step 350. The way to encode this value is directly determined by the property. As previously indicated, a property is indeed characterized by an identifier, and an encoding format for its values. As a remark, the reason for distinguishing two iterations is that it enables the factorization described in the following as a possible embodiment (i.e. when applicable, the type indications may be factorized for several subsamples, typically at sample level, while the values remain indicated at subsample level). However, if this factorization is not considered useful, a single loop on properties may be used, with indication types and values being interleaved (or alternatively, indication types and values may be written to two distinct variables, which may then be concatenated as a single field of the data structure, so that indication types and values are not interleaved).

Step 350 is followed by step 340, and when there does not remain any unprocessed property, the process ends at step 390.

In a specific embodiment where the optional used properties description structure is provided, the number of properties is not indicated at step 305. Instead, an indication on the presence of a value for each property of the used properties is provided as one bit, the concatenation of these bits forming a bitmask. The size of this bitmask can be directly determined from the number of distinct used properties and is typically equal to the lowest multiple of 8 greater or equal to the number of distinct used properties. In a variant, it may be decided that state-of-the-art subsample_priority and discardable properties should be handled differently from other properties, and that they are therefore not added to the used properties description. Instead, two additional bits can be added to the bitmask, one indicating whether a value is provided for subsample_priority, the other one indicating whether a value is provided for discardable. An example of such an embodiment is provided with regards to Figure 8.

Figure 4 illustrates the main steps of a method for describing a subsample in the subsample description box according to another embodiment of the invention. This method describes another possible implementation of the process of step 170.

First, at step 400, subsample size is written. After that, at step 405, a structure comprising a list of properties to be used to describe considered subsample is determined. As previously described, a list of properties may be defined through an entry in a SampleGroupDescriptionBox. This entry can then be referred to through the groupingtype of considered SampleGroupDescriptionBox and entry's index in said box, indexing starting at 1 for example. Alternatively, a new kind of box that simply aims at describing a list of properties may be defined, for example a SubSampleListBox. In this case, such structure could be referred to through a unique identifier assigned to this box. Optionally, some values may also be specified for some of the properties in said structure. In the case of a subsample list box defined in a SampleGroupDescriptionBox, different entries may be distinct simply due to their different values for some properties, meaning that they describe an identical set of properties, but with some differences in the specified property values.

At step 410, a reference to determined structure is provided in the description of considered subsample in the subsample description box. Based on this reference, it is possible to determine the properties for which a value is going to be specified in the description of the subsample. In particular, if a value has already been specified for a given property in determined structure, the subsample list box, then no value is expected for this property in considered description, and the property is considered as already processed.

Step 410 is followed by step 420, where it is checked whether there remains an unprocessed property. If so, its value is written at step 430 using the encoding format defined for considered property, and the process loops at step 420. When there does not remain any unprocessed property, the process ends at step 490.

For each property for which a default value is provided, an indication regarding whether the default value applies to current subsample may be indicated in subsample description in the subsample description box. This indication is typically a single bit, but that may be converted to a byte in order to preserve byte alignment. Alternatively, all such single bits may be concatenated to form a bitmask and said bitmask may be added to the description. By using such a bitmask, fewer bits are wasted due to byte alignment. Finally, another solution may also be to use, for each property, 1 bit from its subsample_size to indicate whether its default value applies or not.

In other words, this embodiment proposes to describe the types of the properties associated with a given subsample in an external data structure (box or data structure inside a box) that is referred to in the subsample description box. The corresponding property values being still described in the subsample description box. Optionally, the external data structure may comprise default values. In that case, no value is expected at the subsample level. In a variant, the default value can be overwritten in the subsample description box at subsample level. In a variant, the subsample description box may comprise a first reference to default property values in an external data structure and a second reference in sub-sample description for a given subsample to the types of properties that override the default property values with properties values for the given subsample.

When the file is segmented in track fragments, there may be an ambiguity with regards to the references made to the subsample list box describing list of properties. In particular, it may happen that such structures may be defined at two different levels, one global, common to all fragments On a imoov' box), and one specific to each fragment On a 'frac box, inside a 'moof' box). Generally speaking, some data may be defined at these two levels since this allows defining some default values at the file level, that may be used by all fragments, but that can also be overwritten, meaning replaced at fragment level.

Therefore, in such a situation and in the context of the embodiment when references to structures describing the list of properties are made, an indication may be added to each subsample description in order to indicate whether the referenced structure is the one defined at the global level, or at the fragment level. When the structure of step 400 relies on sample group, this is already supported by using

group_description index offseted by Ox10000.

In order to minimize the number of values to be written, it may be decided to define a rule such that, starting from the second subsample, unless a value is specified for a given property and a given subsample, the value of said property for said subsample is the same as the value of said property for previous subsample. In particular, if the property was omitted for previous subsample, the property is also considered as omitted for current subsample.

Since this rule requires to be aware of previous subsamples, it may be defined that the rule applies only within a given scope. For instance, it may be decided to reset the rule for each new fragment or for each subsample description box generated at step 110.

When using such a rule, once a property has been specified for a given subsample, it cannot be omitted. If a given value has been defined as "undefined" for considered property, it is possible to specify this value of the property for a considered subsample, which should be equivalent to omitting it. Yet, if there is a need to enable an omission, a dedicated indication may be included in the description of subsample. For instance, it may be decided that a bitmask is included in each subsample description, the bits in the bitmask corresponding to Boolean values indicating whether the "same if not specified" rule shall apply or not to considered properties (considered properties would typically be the properties that were specified for previous subsample). If the flag is equal to 0 for a given property, this means that the value should be omitted, unless a new value is explicitly specified for this property in subsample description.

If the same properties are specified for all the subsamples of a given sample, it may be decided to factorize corresponding subsamples description at sample level to minimize the number of bytes used for these descriptions. In order to determine whether a factorized description is provided or not, a Boolean attribute may be defined. When true, such a factorized description is provided and applies to all corresponding subsamples; when false, no such factorized description is provided, and instead, a specific description is indicated for each subsample at the subsample level.

Figure 5 illustrates the main steps of an example of method for reading encapsulated media data according to one of the encapsulation methods herein described.

First, at step 500, such encapsulated media data comprising one or more samples is obtained. Then, at step 510, the data is parsed by a reader until obtaining a structure describing at least one subsample of at least one considered sample. This is typically the subsample description box generated at step 110.

If the parsed subsample description box comprises the optional used properties description structure, step 520 is performed to read the description of the set of properties that may be specified for the subsamples described in this subsample description box. By reading said set of properties, the reader can determine whether it would like to parse the whole structure, or if it may skip it.

Step 520, or step 510 if step 520 is not performed, is followed by step 530 where it is checked whether there remains any unread sample. If so, the description of next sample is read at step 540. The reading of this description typically involves reading sample_delta and subsample_count values, as described with reference to Figure 1.

Next, it is checked at step 550 whether there remains any unread subsample for considered sample. If so, the description of next subsample is read at step 560. In particular, its size, at least one property value, and, for each property value, an indication on the type of the property are read. As described with regards to the encapsulating side, some property values may be defined through the usage of default values, and specific mechanisms allowing to reduce the verbosity of the description may be used (e.g. the "same if not specified" rule). More generally, all the mechanisms that have been defined for the encapsulating side also apply to the reader side.

Eventually, the subsamples are read according to the parsed properties associated with them.

While the presented methods allows having properties of arbitrary length, in some embodiments, the properties may have a fixed length. In particular, properties may always comprise 32 bits. A benefit of this approach is that it allows defining a single encoding format for a property that may be encoded as a codec_specific_parameters in a state-of-the-art 'subs' box and in a new box according to the invention. In this case, each distinct property is associated to a given flags value in the case of the 'subs' box (this flags value is used to indicate the format of codec_specific_parameters value). That flags value therefore uniquely identifies a given property: hence, it can be used as an identifier of said property when using a subsample description box according to the invention. By doing so, the current 'subs' may easily coexist with an alternative box according to the presented methods. Adopting the box according to the presented methods may therefore be seamless for file format specifications. On the other hand, if properties of variable length are considered, state-of-the-art 'subs' box is not adapted, and specifications would have to deal with this issue (e.g. by dropping existing 'subs' box, or by defining somehow twice the same properties, once on 32 bits for existing 'subs', and once on an arbitrary number of bytes for the box according to the invention).

Figure 8 illustrates an example of syntax for a subsample description box according to this embodiment. As an example, this box is named FlexibleSubSampleInformationBox and may be referred to as 'subf' or can be named ExtendedSubSampleInformationBox and may be referred to as 'esub'. First, the used properties description structure of step 120 is indicated, with the process described in Figure 2a. The distinct count of properties is first indicated, then identifiers for each property are listed (i.e. each property_id corresponds to an existing flags value for the 'subs' box).

After that, the organisation of samples and subsamples is described using means similar to existing 'subs' box. The description of each subsample is made according to Figure 3 process (and without step 305, which is not required when using a bitmask to provide indication of types of properties). A slight difference with state-of-the-art comes from the fact that the number of bits used to encode the sample size is not determined based on the version value of the box, but based on a dedicated subsample_size_32_bits flag. This flag value is for instance the value of the first bit of box's flags value.

Then, each subsample is further described by first indicating a bitmask (variable subsample_properties_bitmask) indicating which values are provided for current subsample. This bitmask comprises nb_bits, nb_bits being the lowest multiple of 8 that is greater or equal to disfinct_properties_count + 2. Values that may be provided for current subsample include a subsample_priority value and discardable value, both with the same format as in SubSampleInformafionBox, as well as values corresponding to the distinct properties listed at the beginning of current box. A bit equal to 1 indicates that a value is explicitly provided, while a bit equal to 0 indicates that no value is explicitly provided. The first bit indicates whether subsample_priority is indicated, the second bit indicates whether discardable is indicated, and next nb_bits -2 bits indicate whether a value is provided for each of the properties listed in the list of distinct properties present in current box, corresponding to the optional used properties description structure described above. If no value is explicitly specified for a given property, the value of said property for current sub-sample is the last explicitly specified value for said property. For the first described sub-sample, a property must be explicitly specified for each distinct property.

Finally, following the subsample_properties_bitmask, the corresponding values are provided (property_value field). The two first values that may be provided are subsample_priority and discardable, which are encoded on 8 bits. Then, next values (if any) are encoded on 32 bits.

Figure 9 provides an example of a SubSampleToGroupBox that may be used as a subsample description box in conjunction with sample groups defined in a SampleGroupDescriptionBox, as previously described, for instance in the description of Figure 4. This box comprises a grouping_type value that indicates the SampleGroupDescriptionBox it refers to, as well as a grouping_type_parameter that may be defined if the box's version value is equal to 1. The grouping_type is used to associate the SubSampleToGroupBox ('ssgp') to its corresponding SampleGroupDescriptionBox ('sgpd'). A SubSampleToGroupBox can be defined in the SampleTableBox 'stbl' or at movie fragment level, for example in a Etre box. The indication of samples having a sub-sample description is described through parameters from the sub-sample information box, namely entry_count and sample_delta values. The number of subsamples for a sample is then indicated through the value subsample_count, and for each subsample, its size is indicated (here on 32 bits, but size may be variable depending e.g. on box's version value or flags value), as well as the group_description_index corresponding to the entry from corresponding SampleGroupDescriptionBox that is used as a reference for considered subsample. Said entry describes the actual properties specified for considered subsample, including its values. Alternatively, and as previously described, only some of the values may be indicated in said entry, in which case missing values would be added to the subsample description. An entry in the SampleGroupDescriptionBox with a grouping_type indicating sub-sample information can be defined as a specific SampleGroupDescriptionEntry (since applies to any kind of track). It may consist in a parameter on a fixed number of bits, as follows: class SubSampleGroupEntry() extends SampleGroupDescriptionEntry ('ssgp') unsigned int(32) subsample_property; This representation allows backward compatibility with existing sub-samples properties, defined for example in ISO/IEC 14496-15. Moreover the default_length of the SampleGroupDescriptionBox can then be used, since all entries have the same size.

Alternatively, the content of the sub-sample group entry may be provided as an encoding size (preferably byte-aligned) and a value for the subsample_property (for example the property_value parameter): class SubSampleGroupEntry() extends SampleGroupDescriptionEntry (issgp') 1 unsigned int(8) encoding size in bytes; unsigned int(encoding size in bytes) property value; This representation allows efficient representation of a sub-sample property, fitting the number of bytes to the possible range of values for a sub-sample property. The parameter property_value provides the value for the sub-sample property. Alternatively, the content of the sub-sample group entry may be provided as a list of parameters, encoded on fixed or variable size (only variable size illustrated below): class SubSampleGroupEntry() extends SampleGroupDescriptionEntry (IssgP') { unsigned int(8) property count; for (int i=0; i < property count; i++) { unsigned int(8) encoding size in bytes; unsigned int(encoding_size_in_bytes) property_value; This representation allows to associate sub-sample(s) to one or more sub-sample properties, through the group_description_index parameter of the SubSampleToGroupBox.

Yet as another alternative, the content of the sub-sample group entry may consist in a type-length-value, or in a list of type-length-values (as illustrated below), the length being given by the default length or by the description_length parameter of

the SampleGroupDescriptionBox:

class SubSampleGroupEntry() extends SampleGroupDescriptionEntry (issgp') 1 unsigned int(8) property count; for (int i=0; i < property count; i++) ( unsigned int(8) encoding_size_in_bytes unsigned int(8) property type; // property_length is given by SampleGroupDescriptionBox; unsigned int(encoding size _ in _bytes) property value; Where property_type indicates the type of the property (e.g. an index in a pre-defined list, a reserved code, a 4CC, a URN, or any means to uniquely identify a property type). It is to be noted that, depending on the means used for indication of the property_type, 8 bits may not be sufficient and 32 bits may be used instead (for example when using a 4CC). The property_value contains the actual value for the property of the indicated type. It can be encoded on a number of bytes fit to the property_length.

Another variant may describe the encoding_size_in_bytes outside the loop on properties, considering that all values will be encoded with the same number of bytes. This is easier to set the value of the description_length parameter in the SampleGroupDescriptionBox. The SubSampleToGroupBox and its associated SampleGroupDescriptionBox inherit the properties of sample groups like the default sample grouping (unmapped sub-samples are associated to the default group description index when the version 2 of the SampleGroupDescriptionBox is used) or like the static mapping of sub-samples or like the static group description index and their combinations.

Finally, subsample description from Figure 9 also comprises a Boolean value is_traf_group_description_index; this value is useful when fragments are used, as it indicates whether the SampleGroupDescriptionBox referred to through grouping_type is the global one ('moov' box), or the fragment-specific one (traf box).

Figure 7 is a schematic block diagram of a computing device 700 for implementation of one or more embodiments of the invention. The computing device 700 may be a device such as a micro-computer, a workstation or a light portable device. The computing device 700 comprises a communication bus connected to: -a central processing unit 701, such as a microprocessor, denoted CPU; -a random access memory 702, denoted RAM, for storing the executable code of the method of embodiments of the invention as well as the registers adapted to record variables and parameters necessary for implementing the method according to embodiments of the invention, the memory capacity thereof can be expanded by an optional RAM connected to an expansion port for example; -a read only memory 703, denoted ROM, for storing computer programs for implementing embodiments of the invention; -a network interface 704 is typically connected to a communication network over which digital data to be processed are transmitted or received. The network interface 704 can be a single network interface, or composed of a set of different network interfaces (for instance wired and wireless interfaces, or different kinds of wired or wireless interfaces). Data packets are written to the network interface for transmission or are read from the network interface for reception under the control of the software application running in the CPU 701; -a graphical user interface 705 may be used for receiving inputs from a user or to display information to a user; -a hard disk 706 denoted HD may be provided as a mass storage device; -an I/O module 707 may be used for receiving/sending data from/to external devices such as a video source or display.

The executable code may be stored either in read only memory 703, on the hard disk 706 or on a removable digital medium such as for example a disk. According to a variant, the executable code of the programs can be received by means of a communication network, via the network interface 704, in order to be stored in one of the storage means of the communication device 700, such as the hard disk 706, before being executed.

The central processing unit 701 is adapted to control and direct the execution of the instructions or portions of software code of the program or programs according to embodiments of the invention, which instructions are stored in one of the aforementioned storage means. After powering on, the CPU 701 is capable of executing instructions from main RAM memory 702 relating to a software application after those instructions have been loaded from the program ROM 703 or the hard-disc (HD) 706 for example. Such a software application, when executed by the CPU 701, causes the steps of the flowcharts of the invention to be performed.

Any step of the algorithms of the invention may be implemented in software by execution of a set of instructions or program by a programmable computing machine, such as a PC ("Personal Computer), a DSP ("Digital Signal Processor") or a microcontroller; or else implemented in hardware by a machine or a dedicated component, such as an FPGA ("Field-Programmable Gate Array") or an ASIC ("Application-Specific Integrated Circuit').

Although the present invention has been described hereinabove with reference to specific embodiments, the present invention is not limited to the specific embodiments, and modifications will be apparent to a skilled person in the art which lie within the scope of the present invention.

Many further modifications and variations will suggest themselves to those versed in the art upon making reference to the foregoing illustrative embodiments, which are given by way of example only and which are not intended to limit the scope of the invention, that being determined solely by the appended claims. In particular the different features from different embodiments may be interchanged, where appropriate.

Each of the embodiments of the invention described above can be implemented solely or as a combination of a plurality of the embodiments. Also, features from different embodiments can be combined where necessary or where the combination of elements or features from individual embodiments in a single embodiment is beneficial.

In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be advantageously used. 2. 3. 4. 5.

Claims

CLAIMSA method for encapsulating in a media file media data comprising one or more samples, at least one sample being organized into subsamples, the method comprising: generating for the at least one sample organized into subsamples a subsample description box, the subsample description box comprising for each subsample a subsample description comprising a subsample size; wherein the method further comprises: associating property values with subsamples; and wherein: the subsample description box comprises an indication of the property types of the associated property values; and encapsulating the media data and the subsample description box into the media file.
The method of claim 1, wherein the indication is provided in the subsample description.
The method of claim 1, wherein the indication is provided at the sample level in the subsample description box.
The method of claim 1, wherein the indication is provided as a reference on an external data structure comprising the property types.
The method of claim 1, wherein all the property values associated with a subsample are provided in the subsample description.
The method of claim 1, wherein at least one default property value is provided for a property and the corresponding property value is omitted in the subsample description.
The method of claim 1, wherein at least one default property value is provided for a property and a property value in the subsample description overwrites the default property value.
8. The method of claim 4, wherein the external data structure comprises at least one default property value.
9 The method of claim 1, wherein the subsample description box is provided with a used properties description structure describing the set of properties which are each associated with at least one subsample.
10. The method of claim 9, wherein the used properties description structure is a list of property identifiers. 10
11. The method of claim 9, wherein the used properties description structure is a bitmask, each bit of the bitmask indicating, when set, a type of property used in the subsample description box.
12. The method of claim 9, wherein the used properties description structure comprises at least one default property value.
13. The method of claim 9, wherein the indication comprises in the subsample description at least a reference to properties in the used properties description structure.
14. The method of claim 13, wherein the indication comprises in a subsample description a bitmask of the properties in the used properties description structure, each bit of the bitmask when set indicates that the correspondingproperty is used in the subsample description.
15. The method of claim 12, wherein the subsample description comprises for each property value for which a default property value is provided an indication regarding whether the default property value applies to the subsample. 30
16. The method of claim 1, wherein the sample description does not comprise a property value for a property associated with the subsample, the missing property value corresponds to the property value provided for the same property for a previous subsample.
17 A method for reading a media file comprising media data comprising one or more samples, at least one sample being organized into subsamples, the method comprising: reading for the at least one sample organized into subsamples a subsample description box, the subsample description box comprising for each subsample a subsample description comprising a subsample size; reading in a subsample description an indication of the property types of property values associated with the subsample; obtaining the property values associated with the subsample based on the property types.
18. A computer program product for a programmable apparatus, the computer program product comprising a sequence of instructions for implementing a method according to any one of claims 1 to 17, when loaded into and executed by the programmable apparatus.
19. A computer-readable storage medium storing instructions of a computer program for implementing a method according to any one of claims 1 to 17.
20. A computer program which upon execution causes the method of any one of claims 1 to 17 to be performed.
21 A device for encapsulating in a media file media data comprising one or more samples, at least one sample being organized into subsamples, the device comprising a processor configured for: generating for the at least one sample organized into subsamples a subsample description box, the subsample description box comprising for each subsample a subsample description comprising a subsample size; wherein the processor is further configured for: associating property values with subsamples; and wherein: the subsample description box comprises an indication of the property types of the associated property values; and encapsulating the media data and the subsample description box into the media file.
22 A device for reading a media file comprising media data comprising one or more samples, at least one sample being organized into subsamples, the device comprising a processor configured for: reading for the at least one sample organized into subsamples a subsample description box, the subsample description box comprising for each subsample a subsample description comprising a subsample size; reading in a subsample description an indication of the property types of property values associated with the subsample; obtaining the property values associated with the subsample based on the property types.