WO2016204442A1

WO2016204442A1 - Mmt apparatus and mmt method for processing media data

Info

Publication number: WO2016204442A1
Application number: PCT/KR2016/006027
Authority: WO
Inventors: 이진영; 윤국진; 정원식
Original assignee: 한국전자통신연구원
Priority date: 2015-06-17
Filing date: 2016-06-08
Publication date: 2016-12-22

Abstract

An MMT apparatus and an MMT method for processing media data are disclosed. An MMT apparatus according to one embodiment comprises: an asset file generation unit for generating an asset file including media data; and a signaling message generation unit for generating a signaling message for transmission or consumption of the media data, wherein the generated asset file includes asset correlation information which describes the relevance between a current asset and a relevant asset, which is another asset file constituting the same MMT package as the current asset.

Description

MMT apparatus and method for processing media data

The present invention relates to an MPEG Multimedia Transport (MMT) technology, and more particularly, to an MMT apparatus and method for processing media data composed of one or more assets.

MPEG Media Transport (MMT) is a new standard technology developed for the storage and transmission of multimedia contents in MPEG systems sub-working group. The existing standard technology by the MPEG system division defines functions such as packetization, synchronization, and multiplexing required for transmitting multimedia contents in a broadcasting network, and is standardized as MPEG-2 TS (Transport Stream) technology and is now widely used. However, MPEG-2 TS technology was developed under the premise of the existing broadcasting network. The MPEG-2 TS technology is unsuitable for transmitting multimedia contents for IPTV or mobile broadcasting based on IP (Internet Protocol) based network environment. Inefficient Therefore, the MPEG System Division recognized the necessity of a new media transmission standard in consideration of the media transmission environment as well as the new media transmission environment, and developed the MMT standard as a result.

According to the MMT technology, a logical collection of media data is called a package. The MMT package includes one or more assets, presentation information (PI), and transport characteristics (TC). An asset is a collection of one or more Media Processing Units (MPUs) with the same Asset ID and contains encoded media data such as video, audio or web pages. The presentation information defines the spatial and temporal relationships between assets for consumption, and the transport characteristics provide unique information related to the delivery of a particular asset.

The MMT technology supports not only transmitting assets, which are coded media data through encapsulation, using MMT protocols or other transmission protocols, but also storing them as individual files. However, since the aforementioned presentation information is transmitted in a predetermined signaling message, when the content consisting of a plurality of assets is stored in a storage medium, the presentation information cannot be used for consumption in the form of an MMT package.

One problem to be solved by the present invention is an MMT apparatus and method for processing media data that can be consumed in the form of an MMT package even when the media data is transmitted through an MMT protocol or the like. To provide.

Another problem to be solved by the present invention is to provide an MMT apparatus and method for processing media data that can describe the association between a plurality of assets constituting the content without the presentation information (Presentation Information).

An MMT apparatus for processing media data according to an embodiment of the present invention for solving the above problems is an asset file generation unit for generating an asset file including the media data and signaling for delivery or consumption of the media data And a signaling message generator for generating a message, wherein the asset file includes asset correlation information describing a relationship with related assets, which is another asset file constituting the same MMT package as the current asset.

According to an aspect of the embodiment, the asset correlation information is related asset flag indicating whether the related asset exists, related asset number information indicating the number of the related asset, any correlation between the current asset and the related asset It may include correlation type information indicating whether having a related asset ID indicating the identifier of the related asset. In this case, the correlation type information is one of necessity information indicating whether the related asset is needed to process the current asset and independent consumable information indicating whether the current asset can be consumed independently without the related asset. It may contain the above. The correlation type information may be represented by a combination of the necessity information and the independent consumption information. The correlation asset information may be described in a Media Processing Unit (MPU) box defined in the ISO / IEC 23008-11 standard. In addition, the associated asset ID may be described in the same manner as AssetIdentifierBox defined in the ISO / IEC 23008-11 standard.

According to another aspect of the embodiment, the asset correlation information may be included only when the asset file is stored in a predetermined storage medium.

According to an aspect of the present invention, there is provided a method of processing media data in an MMT device, including generating a single asset file including the media data and signaling for delivery or consumption of the media data. Generating a message, and generating the asset file by including asset correlation information describing a relationship with related assets, which is another asset file constituting the same MMT package as the current asset, in the generating of the asset file; .

According to an aspect of the embodiment, the asset correlation information is related asset flag indicating whether the related asset exists, related asset number information indicating the number of the related asset, any correlation between the current asset and the related asset It may include correlation type information indicating whether having a related asset ID indicating the identifier of the related asset. In this case, the correlation type information is one of necessity information indicating whether the related asset is needed to process the current asset and independent consumption information indicating whether the current asset can be consumed independently without the related asset. It may contain the above. The correlation type information may be represented by a combination of the necessity information and the independent consumption information. In the asset file generation step, the correlation asset information may be described in a Media Processing Unit (MPU) box defined in the ISO / IEC 23008-11 standard. In the asset file generation step, the associated asset ID may be described in the same manner as AssetIdentifierBox defined in the ISO / IEC 23008-11 standard.

According to another aspect of the embodiment, the asset file generation step may include the asset correlation information only when the asset file is stored in a predetermined storage medium.

According to the above-described embodiment of the present invention, the asset file includes information related to related assets, that is, asset correlation information, which is another asset constituting the same MMT package as the current asset. Therefore, a coded media file may not be encoded without certain information describing the correlation between the assets, such as when the asset file is stored in a predetermined storage medium, for example, playback information provided by ISO / IEC 23008-11. Can consume in the form of MMT package defined in 1.

1 is a diagram illustrating a hierarchical structure including respective functional areas of an MMT system.

2 illustrates a logical structure of an MMT package.

3 is a block diagram schematically illustrating a configuration of an MMT apparatus for processing media data according to an embodiment of the present invention.

4 is a block diagram illustrating a configuration of asset correlation information according to an embodiment of the present invention.

5 is a diagram illustrating an example of syntax of asset correlation information described in an MPU box.

6 is a flowchart illustrating an example of a method of processing media data in an MMT device according to an embodiment of the present invention.

Hereinafter, exemplary embodiments will be described in more detail with reference to the accompanying drawings. However, these drawings are only examples for easily describing the contents and scope of the technical idea, and thus the technical scope is not limited or changed. And it will be apparent to those skilled in the art that various modifications and changes are possible within the scope of the technical idea based on these examples. In addition, terms and words used herein are terms selected in consideration of functions in the embodiments, and the meaning of the terms may vary depending on the intention or custom of the user or operator. Therefore, the terms used in the embodiments to be described later, when specifically defined herein, will follow the definition, and if there is no specific definition should be interpreted as meaning generally recognized by those skilled in the art.

1 is a diagram illustrating a hierarchical structure including respective functional areas of an MMT system. Referring to FIG. 1, the MMT system 100 includes an encapsulation function layer (Encapsulation Function Layer, Layer E) 110, a delivery function layer (D D) 120, and a signaling function layer. , Layer S) 130.

The multimedia data compressed by the media coding layer 12 is packaged and output in a form similar to a file format via the encapsulation function layer 110. The encapsulation function layer 110 takes coded media data provided from the media encoding layer as input and generates a data fragment or segment, which is a small unit for the MMT service, and uses the data fragment to generate the MMT service. Create Access Units (AUs) for use. In addition, the encapsulation functional layer 110 combines and / or divides access units (AUs) to generate data in a predetermined format for generation and storage and / or transmission of composite content.

The delivery function layer 120 may perform network flow multiplexing, network packetization, QoS control, and the like, of media transmitted through a network. More specifically, the delivery function layer 120 converts one or more data units (DUs) output from the encapsulation function layer 110 into an MMT payload format, and then MMT transport packet header. In addition, it may be configured as an MMT packet or an RTP packet using a real-time protocol (RTP), which is an existing transport protocol. Packets configured at the transport function layer 120 are finally transported as IP packets at the Internet protocol (IP) layer 16 via a transport protocol layer 14 such as User Datagram Protocol (UDP) or Transport Control Protocol (TCP). do.

The signaling function layer 130 generates control information or signaling information necessary for the transmission and consumption of the package in a message of a predetermined format. The generated signaling message may be transmitted in an MMT packetized form along with the media data, or may be transmitted in an MMT packetized form as a separate signaling message and transmitted to a receiver.

FIG. 2 is a diagram illustrating a logical structure of an MMT package, and illustrates a relationship between the components constituting the MMT package and the components.

Referring to FIG. 2, the MMT package is a set of media content data and related meta data, and includes a presentation information (PI), one or more assets, and transport characteristics (TC). It is. PI is information describing the spatiotemporal relationship of assets. Assets are encoded media data such as audio, video or web pages, and each asset has an Asset ID as an identifier. Each asset may include one or more media processing units (MPUs), which may be independently consumed as files in the form of ISO-BMFF (International Standard Organization-Base Media File Format). Two consecutive MPUs within one asset may not contain the same media sample. In addition, the TC provides QoS information necessary for the transmission of an asset. An asset may have only one TC, but one TC may be used for a plurality of assets.

According to the MMT technology, video files, audio files, caption files, etc. constituting multimedia contents are treated as separate assets. The left and right video files that make up the stereoscopic video are also treated as separate assets, and the video file of the base layer and the enhancement layer coded using scalable video coding technology are also separate assets. Are treated. In the MMT technology, content transmission or playback, data processing or service quality management for the content are independently processed on an asset basis. Therefore, in order to consume content composed of multiple assets, that is, an MMT package, the spatial and temporal correlations between the assets are determined. Information is essential.

3 is a block diagram schematically illustrating a configuration of an MMT apparatus for processing media data according to an embodiment of the present invention. Referring to FIG. 3, the MMT device 200 includes an asset file generator 210 and a signaling message generator 220. The MMT device 200 may further include a transmitter 230. The MMT apparatus shown in FIG. 3 may be implemented as a component (or function) of an MMT transmitting entity in which some or all of its components (or function) are defined in the MMT standard.

The asset file generator 210 generates an asset file for each asset based on an access unit (AU). Here, the "assets file" does not necessarily mean a single file, but may be a collection of a plurality of files. In other words, an "assets file" is a collection of one or more files containing encoded media data constituting a single asset as well as related information. As described above, video files, audio files, caption files, and the like, which constitute one MMT package, are treated as separate assets and processed.

One or more asset files generated by the asset file generator 210 of FIG. 3 may be stored in a storage medium. In this case, the generated asset file also includes information describing association with other asset files constituting the same MMT package, that is, asset correlation information.

Alternatively, the asset file generated by the asset file generator 210 may be input to the transmitter 230 and transmitted to the MMT receiver. In this case, asset correlation information is not necessarily included in the asset file generated by the asset file generator 210 and may not be included. This is because the spatiotemporal correlation between assets may be described using presentation information included in the signaling message generated by the signaling message generator 220. Hereinafter, a case in which the asset correlation information is included in the asset file generated by the asset file generator 210 will be described.

The asset file generated by the asset file generator 210 includes media data such as video data, audio data, caption data, and a web page. Such media data may comprise one or more MPUs. As described above, the MPU may be an ISOBMFF-type file encapsulated in a predetermined format, and the encapsulated MPU has a unique sequence number along with the asset ID. MPUs constituting the same asset all have the same asset ID. The MPU contains encoded stereoscopic video data, each MPU containing at most one media track.

According to the embodiment of the present invention, the asset file generated by the asset file generator 210 includes asset correlation information. Asset correlation information is information describing a relationship with another asset file constituting the same MMT package. For example, asset correlation information may include information indicating whether or not a related asset exists, information indicating how many assets are related when the related asset exists, information indicating a specific type of correlation, and information on related assets. ID information may be included. To this end, the asset file generator 210 may include a correlation description unit 212. The correlation description unit 212 performs a function of describing and adding the asset correlation information to the generated asset file.

4 is a block diagram illustrating a configuration of asset correlation information according to an embodiment of the present invention. Referring to FIG. 4, the asset correlation information 300 includes an associated asset flag 310, an associated number of assets information 320, correlation type information 330, and an associated asset ID 340.

The related asset flag 310 is information indicating whether there is a related asset currently associated with the asset. Here, the 'related asset' may be an asset that configures the same MMT package as the current asset. The related asset flag 310 may be represented, for example, with the syntax "is_associated". For example, the related asset flag 310 is set to "1" when there is at least one related asset currently associated with the asset, and the related asset flag 310 is set to "0" when there are no related assets. Can be set.

The related asset number information 320 is information indicating the number of related assets associated with the current asset. The related asset number information 320 may be expressed by, for example, the syntax “num_association”. The related asset number information 320 may be included only when the related asset flag 310 indicates that at least one related asset exists.

Correlation type information 330 is information indicating what relationship each of the current asset and one or more assets associated with it has. Correlation type information 330 may also be included only if the related asset flag 310 also indicates that one or more related assets exist, such as, for example, such correlation type information 330, which may be represented by the syntax "association_type". have.

According to an aspect of the present embodiment, the information that can be indicated using the correlation type information 330 may include one or more of 'necessity information' and 'independent consumption information'. Here, 'necessity information' indicates whether related assets are needed to process the current asset, and 'independent consumption information' indicates whether the current asset can be used or consumed independently without related assets. For example, correlation type information 330 may include only need information or independent consumable information, or both. In the latter case, the necessity information and the independent consumable information may be individually displayed using 1 bit or may be displayed in the form of a combination using 2 bits.

Table 1 shows an example of a value of the correlation type information 330 and a description thereof. In Table 1, a combination of 'necessity information' and 'independent consumption information' is represented by 2 bits as the correlation type information 330. Here, the description that 'necessity information' is supplementary indicates that the related asset is not necessarily required to process the current asset, while the description that is mandatory indicates that the related asset is required to process the current asset. To instruct. In addition, 'independent consumption information' indicates whether the current asset can be consumed independently without the related asset.

The related asset ID 340 is information indicating an identifier of the related asset to which it is associated. If there are two or more related assets, the identifier of the asset is displayed for each asset. It will be apparent to those skilled in the art that the related asset ID 340 may be included only when indicating that there is more than one related asset, similar to the related asset number information 320 or the correlation type information 330. According to an example of this embodiment, the associated asset ID 340 may be described in the same manner as described in "AssetIdentifierBox" defined in the ISO / IEC 23008-1 standard.

As described above, the asset file generated by the asset file generator 210 includes asset correlation information. The asset correlation information may be described by the correlation description unit 212 and included in the asset file. According to one aspect of this embodiment, asset correlation information may be described in an MPU box (mmpu) as defined in the ISO / IEC 23008-1 standard.

5 is a diagram illustrating an example of syntax of asset correlation information described in an MPU box. In FIG. 5, the related asset flag 310, the related asset number information 320, the correlation type information 330, and the related asset ID 340 of FIG. 4 are respectively syntax 'is_associated' 310a, num_association 320a, It is described by an 'association_type' 330a and an 'assetIdentifierBox' 340a.

6 is a flowchart illustrating an example of a method of processing media data in an MMT device according to an embodiment of the present invention. 6 may be a procedure performed in the MMT apparatus described above with reference to FIGS. 3 to 5. In the present embodiment, a media data processing method in the MMT device will be briefly described to avoid unnecessary duplication. Therefore, the matters not described in detail herein may be equally applicable to the description of the MMT apparatus with reference to FIGS. 3 to 5.

Referring to FIG. 6, the MMT device first describes an associated asset flag as '0' or '1' (S410). To this end, the MMT device may determine whether there is a related asset associated with the current asset, and set it to '0' or '1' according to the determination result. Here, '0' and '1' respectively indicate that there is no related asset and that the related asset exists, which is exemplary. If the related asset flag is described as '0' in step S410, steps S420 to S440 described later may be omitted.

When the related asset flag is described as '1' in step S410, that is, when there is a related asset associated with the current asset, the MMT device describes the number of related assets (S420). In addition, the MMT apparatus describes a correlation type of each of one or more related assets (S430). An example of a correlation type is as described above with reference to Table 1 below. In addition, the MMT device describes the ID of each related asset (S440). In FIG. 6, a flowchart is shown as steps S420, S430 and S440 are performed sequentially, which is merely exemplary and the order of steps S420, S430 and S440 may be reversed or two or more steps may be performed simultaneously.

Tables 2 to 4 show examples of describing asset correlation information of media data constituting an MMT package in the MMT device, respectively, according to an embodiment of the present invention. The example shown in Table 2 is the case where the MMT package consists of one video, one audio, and one caption file. The example shown in Table 3 is the case where the MMT package consists of stereoscopic video and audio in a service compatible manner. In the example shown in Table 4, the MMT package includes scalable video coding (SVC) video having three layers. In Tables 2 to 4, the syntax asset_id, is_associated, num_association, association_type, and asset_id (associated) mean an ID of a current asset, an associated asset flag, related asset number information, correlation type information, and an associated asset ID, respectively.

3, the signaling message generator 220 generates a signaling message for delivery and / or consumption of stereoscopic video data. Here, the signaling message is a message of a format defined in the ISO / IEC 23008-1 standard, and may include information in a format such as signaling tables or descriptors. For example, the signaling message generated by the signaling message generator 220 may include the above-described asset correlation information as part of information indicating temporal and spatial correlations between assets. However, the present embodiment is not limited thereto, and the above-described asset correlation information may not be included in the signaling message generated by the signaling message generator 220.

The transmitter 230 receives the asset file generated by the asset file generator 210 and / or the signaling message generated by the signaling message generator 220 and transmits the signal to the MMT receiving entity. Here, the transmitter 230 may convert the received asset file and / or signaling message into an MMT packet according to an MMT protocol (MMTP Protocol, MMTP) according to the ISO / IEC 23008-1 standard, and transmit the same to an MMT receiving apparatus. However, the present embodiment is not limited thereto, and the transmitter 230 may transmit the received asset file and / or signaling message to the MMT receiving apparatus according to another Internet transmission protocol, real time transmission protocol, or 3D broadcast protocol.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable arrays (FPAs), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. Or may be permanently or temporarily embodied in a signal wave to be transmitted. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The foregoing descriptions are merely examples, and should not be construed as being limited thereto. The technical idea of the present invention should be specified only by the invention described in the claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention. Therefore, it will be apparent to those skilled in the art that the above-described embodiments may be implemented in various forms.

The present invention can be usefully used for procedures or apparatuses related to storage or transmission of media data.

Claims

In the MMT apparatus for processing media data,

An asset file generator for generating an asset file including the media data; And

A signaling message generator for generating a signaling message for delivering or consuming the media data;

The asset file comprises asset correlation information describing the association with the related asset, which is another asset file constituting the same MMT package as the current asset.
The method of claim 1, wherein the asset correlation information is

An associated asset flag indicating whether the related asset exists;

Related asset number information indicating the number of related assets;

Correlation type information indicating a correlation between the current asset and the related asset; And

And an associated asset ID indicating an identifier of the related asset.
The method of claim 2,

The correlation type information includes one or more of necessity information indicating whether the related asset is needed to process the current asset and independent consumable information indicating whether the current asset can be consumed independently without the related asset. MMT device, characterized in that.
The method of claim 3,

The correlation type information is represented by a combination of the necessity information and the independent consumption information.
The method of claim 2,

The correlation asset information is described in the Media Processing Unit (MPU) box defined in the ISO / IEC 23008-11 standard.
The method of claim 2,

The related asset ID is described in the same manner as AssetIdentifierBox defined in the ISO / IEC 23008-11 standard.
The method of claim 1,

The asset correlation information is included only when the asset file is stored in a predetermined storage medium.
In the method for processing media data in the MMT device,

Generating a single asset file containing the media data; And

Generating a signaling message for delivery or consumption of the media data,

Generating the asset file by including asset correlation information describing an association with an associated asset, which is another asset file constituting the same MMT package as the current asset, in the generation of the asset file.
The method of claim 8, wherein the asset correlation information is

An associated asset flag indicating whether the related asset exists;

Related asset number information indicating the number of related assets;

Correlation type information indicating a correlation between the current asset and the related asset; And

And a related asset ID indicating an identifier of the related asset.
The method of claim 9,

The correlation type information includes one or more of necessity information indicating whether the related asset is needed to process the current asset and independent consumable information indicating whether the current asset can be consumed independently without the related asset. Characterized in that.
The method of claim 10,

The correlation type information is represented by a combination of the necessity information and the independent consumability information.
The method of claim 9,

In the asset file generating step, the correlation asset information is described in a Media Processing Unit (MPU) box defined in ISO / IEC 23008-11.
The method of claim 9,

In the asset file generation step, the related asset ID is described in the same manner as AssetIdentifierBox defined in the ISO / IEC 23008-11 standard.
The method of claim 8,

The asset file generating step includes the asset correlation information only when the asset file is stored in a predetermined storage medium.