KR20120100799A - Method of providing media data and interface for providing media data - Google Patents

Abstract

PURPOSE: An interface and a media data supplying method for supplying media data are provided to give a standard for distributing a role of a transfer layer and a role of an encapsulation layer. CONSTITUTION: An encapsulation layer encapsulates encoded media data. A transfer layer transfers the encapsulated media data to the other entity from a specific entity. A control layer controls the transmission of the media data. The encapsulation layer includes a depend function of payload or media. The transfer layer includes an independent function of the payload or media. [Reference numerals] (AA) E-Layer; (BB) Media Packetization; (CC) Media Synchronization; (DD,OO) Additional function; (EE) Fragmentation; (FF) Aggregation; (GG) Application-Dependent Error Handling; (HH) Operation dependant to payload(media); (II) D-Layer; (JJ) Multiplexing; (KK) Timing Information for Inter-flow Synchronization; (LL) Timing Information for System Synchronization; (MM) Generic Error Handling; (NN) Real time media flow control; (PP) Operation independent to payload(media); (QQ) UDP/TCP; (RR) IP; (SS) Transfer operation

Description

Interface for providing media data and method for providing media data {METHOD OF PROVIDING MEDIA DATA AND INTERFACE FOR PROVIDING MEDIA DATA}

The present invention relates to an interface for providing media data and a method for providing media data, and more particularly, to a hierarchical packetization method for providing media data.

Since the standardization of MPEG-2, the video compression standard (or audio compression standard) has been steadily developing new standards over the past decade: MPEG-4, H.264 / AVC, Scalable Video Coding (SVC). In addition, each new standard has expanded the scope of use of the MPEG standard by forming a new market, but in the case of transport technologies such as the MPEG-2 Transport System (TS), the digital and mobile broadcasts have remained unchanged for almost 20 years. It is widely used in T-DMB, DVB-H, etc., and even in multimedia transmission over the Internet, that is, IPTV service, which was not considered at the time of enactment of standards.

However, when the MPEG-2 TS is developed, the multimedia transmission environment and today's multimedia transmission environment are undergoing major changes. For example, the MPEG-2 TS standard was developed in consideration of transmitting multimedia data through an ATM network at the time of enactment, but it is hard to find a case which is used for this purpose today. In addition, since the requirements such as multimedia transmission using the Internet were not considered at the time of enactment of the MPEG-2 TS standard, there are elements that are not efficient for multimedia transmission over the recent Internet. Therefore, in MPEG, it is recognized that the establishment of a new multimedia transmission standard, MMT (MPEG Media Transport), which considers multimedia services on the Internet suitable for the changing multimedia environment, is very important.

As such, the important reason for the MMT standardization is that the MPEG2-TS standard, which was created 20 years ago, has not been optimized for IPTV broadcasting services and the Internet environment recently. Therefore, multimedia optimized for multimedia transmission environments in various heterogeneous networks has recently been developed. Due to the urgent need of the international transmission standard, MPEG MMT is being standardized as a new transmission technology standard.

That is, as the integration of broadcast service and communication service has recently progressed, various contents and services have been reached through different networks. However, according to the conventional media transmission method, each media constituting each multimedia content is provided. Independent protocol should be applied and cause inefficiency of overall multimedia content transmission.

In order to solve this drawback, the establishment of the MMT is in progress, but it is a question of what function should be in charge of each layer defined in the MMT.

Korean Patent Publication No. 10-2011-0117033 ("Interface device and method for transmitting and receiving media data", Samsung Electronics Co., Ltd., published on October 26, 2011)

As described above, it is a question of what function should be in charge in each layer defined in the MMT. When designing a wide range of protocols, such as MMT, comparing the different technologies that provide solutions for the same functionality requires that each layer's function must be clearly determined. Conventionally, basic functions have been proposed for each layer of MMT, but more specific role distribution criteria are needed.

Accordingly, an object of the present invention can define the functions of the delivery layer and the encapsulation layer by presenting a criterion for distributing the roles of the delivery layer and the encapsulation layer among the respective layers of the MMT. It is to provide an interface for providing media data.

Another object of the present invention is to provide a method for providing media data that can define the functions of the delivery layer and the encapsulation layer by providing a criterion for distributing the roles of the delivery layer and the encapsulation layer among the layers of the MMT. It is.

In order to achieve the above object of the present invention, an interface for providing media data includes: an encapsulation layer encapsulating encoded media data; A delivery layer for delivering the encapsulated media data from one entity to another entity; And a control layer for controlling the transmission of the media data, wherein the encapsulation layer includes a function dependent on at least one of a payload and media, and the transport layer includes at least one of a payload and media. It may be characterized by including an independent function.

Here, the encapsulation layer may include a media packetization function that constitutes packetization that is optimal for a specific media type. In addition, the encapsulation layer may be characterized as including an intra-media level Media Synchronization function. In addition, the encapsulation layer may include a fragmentation function that divides one media unit into a plurality of packets. Here, the encapsulation layer may include an aggregation function for combining a plurality of media units into one media unit.

In addition, the encapsulation layer may include an application-dependent error handling function. In this case, the transport layer may include a function of multiplexing two or more sub-streams in one port stream. In addition, the transport layer may include timing information for inter-flow synchronization.

The transport layer may include timing information for system synchronization. In addition, the transport layer may include a generic error handling function. Here, the transport layer may include a real-time media friendly flow control function.

According to another aspect of the present invention, there is provided a method of providing media data, the method comprising: encapsulating encoded media data; Passing the encapsulated media data from one entity to another; And controlling the delivery of the media data, wherein the encapsulating comprises an operation dependent on at least one of a payload and media, and the delivering step is an operation independent of at least one of the payload and the media. It may include. Here, the encapsulation step may include media packetization that constitutes packetization that is optimal for a specific media type. The encapsulation step may include media synchronization at an intra-media level.

In addition, the encapsulation step may include fragmentation of dividing one media unit into a plurality of packets. The encapsulation step may include aggregation that combines a plurality of media units into one media unit. The encapsulation step may include application-dependent error handling. In this case, the delivering may include multiplexing two or more substreams in one port stream. In this case, the transmitting may include transmitting timing information for inter-flow synchronization.

The delivering may include delivering timing information for system synchronization. In addition, the delivering may include batch error handling. Herein, the delivering may include real time media friendly flow control.

According to the above-described interface for providing media data and a method for providing media data according to the embodiment of the present invention, by providing a criterion for distributing roles among respective layers of the MMT, in particular, a delivery layer and an encapsulation layer, Functions of the transport layer and encapsulation layer can be defined.

Accordingly, by presenting the functions essential to each layer in more detail, and defining the respective functions, it is possible to provide a basis for understanding the layers and functions.

1 is a conceptual diagram illustrating an MMT hierarchical structure.
FIG. 2 is a block diagram in which functions are arranged according to a role sharing criterion of an interface for providing media data according to an embodiment of the present invention.
3 is a flowchart of a method of providing media data according to an embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail.

It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present disclosure rather unclear.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

In addition, the components shown in the embodiments of the present invention are shown independently to represent different characteristic functions, which does not mean that each component is composed of separate hardware or software constituent units. That is, each constituent unit is included in each constituent unit for convenience of explanation, and at least two constituent units of the constituent units may be combined to form one constituent unit, or one constituent unit may be divided into a plurality of constituent units to perform a function. The integrated embodiments and separate embodiments of the components are also included within the scope of the present invention, unless they depart from the essence of the present invention.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate the understanding of the present invention, the same reference numerals are used for the same constituent elements in the drawings and redundant explanations for the same constituent elements are omitted.

1 is a conceptual diagram illustrating an MMT hierarchical structure.

Referring to FIG. 1, the MMT layer includes an encapsulation layer, a delivery layer, and a functional area of a control layer. The MMT layer operates on a transport layer.

The encapsulation layer (E-layer) may be in charge of, for example, packetization, fragmentation, synchronization, multiplexing, and the like of transmitted media.

Encapsulation layer (E-layer), as shown in Figure 1, MMT E.1 Layer (MMT E.1 Layer), MMT E.2 Layer (MMT E.2 Layer) and MMT E.3 Layer (MMT) E.3 Layer).

The E.3 layer encapsulates a Media Fragment Unit (MFU) provided from the Media Codec (A) layer to create an M-Unit.

The MFU may have a format, independent of any particular codec, that can carry data units that can be consumed independently in the media decoder. The MFU can be, for example, a picture or slice of the video.

The M-unit may consist of one or a plurality of MFUs and may have a format, independent of a particular codec, that may carry one or a plurality of access units.

The E.2 layer encapsulates the M-units created in the E.3 layer to generate an MMT asset.

An MMT asset is a data entity composed of one or a plurality of M-units from a single data source and is a data unit in which composition information and transport characteristics are defined. MMT assets can correspond to packetized elementary streams (PES), for example video, audio, program information, MPEG-U widgets, JPEG images, MPEG 4 file format, M2TS (MPEG transport stream), etc.

The E.1 layer creates an MMT package by encapsulating the MMT asset generated in the E.2 layer.

The MMT package may be composed of one or more MMT assets together with additional information such as composition information and transport characteristics. Composition information includes information about a relationship between MMT assets, and when one content consists of a plurality of MMT packages, it indicates a relationship between a plurality of MMT packages. It may further include information. The transport characteristics may include transmission characteristic information necessary for determining a delivery condition of an MMT asset or an MMT packet, and may include, for example, a traffic description parameter and a QoS descriptor. ) May be included. The MMT package may correspond to a program of MPEG-2 TS.

The delivery layer may perform, for example, network flow multiplexing, network packetization, and QoS control of media transmitted through a network.

The transport layer (D-layer), as shown in Figure 1, MMT D.1 Layer (MMT D.1 Layer), MMT D.2 Layer (MMT D.2 Layer) and MMT D.3 Layer (MMT) D.3 Layer).

The D.1 layer receives the MMT package generated in the E.1 layer and generates an MMT payload format. The MMT payload format is a payload format for carrying MMT assets and for transmitting information for consumption by the MMT application protocol or other existing application transport protocol such as RTP. The MMT payload may include a fragment of the MFU along with information such as AL-FEC.

The D.2 layer receives the MMT payload format generated in the D.1 layer and generates an MMT transport packet or an MMT packet. The MMT transport packet or MMT packet is a data format used in an application transport protocol for MMT.

D.3 layer (D.3-layer) supports QoS by providing the function of exchanging information between layers by cross-layer design. For example, the D.3 layer may perform QoS control using QoS parameters of the MAC / PHY layer.

The control layer is, for example, session initialization / control / management of transmitted media, server-based and / or client-based trick modes, service discovery, synchronization ) Functions can be performed.

As shown in FIG. 1, a control layer (C-layer) may be configured of an MMT C.1 layer and an MMT C.2 layer.

C.1 layer includes service discovery, media session initialization / termination of media, media session presentation / control of media, delivery (D) layer and encapsulation (E). The interface function with the layer can be performed. The C.1 layer may define the format of control messages between applications for media presentation session management.

Layer C.2 provides delivery of flow control, delivery session management, delivery session monitoring, error control, and hybrid network synchronization control. It is possible to define the format of the control message exchanged between delivery end-points of the D-layer.

Layer C.2 supports delivery session establishment and release, delivery session monitoring, flow control, error control, resource scheduling for established delivery sessions, and synchronization in a complex delivery environment to support the behavior of the delivery layer. Signaling for adaptive delivery, and signaling for adaptive delivery. Required signaling may be provided between a sender and a receiver. That is, the C.2 layer may provide signaling required between the sender and the receiver in order to support the operation of the transport layer as described above. In addition, the C.2 layer may be responsible for interfacing with the transport layer and the encapsulation layer.

Interface for Providing Media Data

When designing a wide layer range protocol such as MMT, the function of each layer must be clearly determined in order to compare different technologies that provide a solution for the same function.

The current MMT Context and Objectives (Non-Patent Document 1) documents provide basic information on the functions that must be provided for each Control (C) / Delivery (D) / Encapsulation (E) layer. It is defined. An embodiment of the present invention presents in more detail the functions that a delivery layer (D-layer) and an encapsulation layer (E-layer) must have in addition to the contents defined in the MMT context and objective (Non-Patent Document 1) document. can do. In addition, the interface for providing media data according to an embodiment of the present invention may provide a basis for common understanding of layers and functions by defining the respective functions.

The MMT context and the objective (Non-Patent Document 1) define the roles of the encapsulation and delivery layer as follows.

An Encapsulation (E) layer defines a format for encapsulating encoded media data so that it can be stored in a specific storage device or transmitted as payloads of delivery protocols.

The delivery (D) layer provides the functions required for transferring encapsulated media data from one network entity to another network entity.

The interface for providing media data according to an embodiment of the present invention may further include role sharing according to the following criteria from a functional point of view in addition to the contents described in the context and the objective (Non-Patent Document 1).

Encapsulation (E) layer: Payload or media dependent functions

2. Delivery (D) layer: payload or media independent functions

FIG. 2 is a block diagram in which functions are arranged according to a role sharing criterion of an interface for providing media data according to an embodiment of the present invention. The meaning of each function is explained in detail below.

Encapsulation Layer (E- Layer )

Functions dependent on the payload type may be located in the encapsulation layer (E-Layer). The payload type may be a different media type, or in the case of in-band signaling, may be a different signaling method. In addition, it may be a redundant packet encapsulation that is coupled and operated with a specific media type for packet error management such as forward error correction (FEC) or retransmission.

1. Media Packetization: Media packetization means a method of constructing packetization that is optimal for a specific media type. It may include a header field (for example, a View ID field of multi-view encoding) that is necessary only for a specific media type.

2. Media Synchronization: Media synchronization at the encapsulation layer can address the issue of intra-media level synchronization. For example, in case of media packet streams using the same codec, it may be necessary to guarantee packet ordering according to decoding order, in which case the commonly used decoding order number (DON) is intra- This may correspond to media level synchronization.

3. Fragmentation: Fragmentation, which divides one media unit into several packets, may have different policies (selection of whether to use fragmentation) according to the payload type. In addition, for example, in the case of a video frame having multiple slices, segmentation techniques such as segmentation may be performed according to a media type. Thus, partitioning may be located in the encapsulation layer in an operation that depends on the payload or media type.

4. Aggregation: Aggregation that combines multiple media units into a single media unit, like partitioning, may have different policies on whether to use them or their preferences for optimal usage depending on the payload type or media type. have. Therefore, the operation is dependent on the payload or media type, and may be located in the encapsulation layer.

5. Application-Dependent Error Handling: Error handling can be divided into error handling specific to payload type and error handling that can be applied independently to payload type. For example, in the case of the FEC technique, a different FEC technique may be most preferable, such as using a reed-solomon code for each specific codec or using a turbo code. The policy of whether to use packet-based FEC or frame-based FEC may also vary depending on the payload type.

Delivery layer (D- layer )

Functions independent of payload type may be located in the D-layer. Since the MMT standard clarifies that it is a protocol used over the IP protocol, it may not include network functions already provided in the IP layer, and many payload types among the additional functions required for media delivery are commonly required. May include functionality.

1. Multiplexing: Multiplexing is a function of multiplexing substreams having two or more different characteristics in one port stream. The sub stream may be a video stream or an audio stream, or a global signaling stream containing all the settings for the entire configuration of the multiplexed stream, or a signaling stream dedicated to a corresponding codec coupled with a specific video stream or an audio stream. Can be. In addition, separate sub-streams for ARQ (Auto Repeat Request) or FEC may be multiplexed. Therefore, the transport layer may provide a function of freely multiplexing and demultiplexing the streams regardless of what kind or stream of payload types are coming from an upper encapsulation layer.

2. Timing Information for Inter-flow Synchronization: The difference between most media streams and data streams is that the time actually presented must be tightly managed. Payload streams, such as video streams, audio streams, and time data, that MMT targets for transmission, require presentation time information. Therefore, timing information may be located in the transport layer as information that should always be provided regardless of the media type.

3. Timing information for System Synchronization: For media applications, the system clocks of the sending and receiving systems should be synchronized independently of the payload type. Therefore, the transport layer may include a function for system time synchronization.

Generic Error Handling: Application layer error recovery may take a media-dependent policy, but takes advantage of the additional global FEC that applies to all payload types. Can be. However, this batch error recovery function can be applied redundantly depending on the lower layer interface type (for example, whether it is wireless or wired media, or which wireless media). can do.

5. Real-Time Media-Friendly Flow Control: IP network recommends that the UE avoids network congestion control through proper flow control. TCP-based flow control, which is widely used, adopts a flow control method that is directly coupled with packet loss. If the packet loss is not recovered, the transmission rate becomes very low, which is not suitable for real-time support of media applications. Do. Therefore, in the case of media transmission, existing or new flow control methods that can guarantee real-time can be introduced.

The functions of the above-described delivery / encapsulation layer are exemplified by the functions listed in the MMT Context and Objective (Non-Patent Document 1) document, but are not limited to the functions listed above, and new functions may be freely added as necessary.

How to Provide Media Data

3 is a flowchart of a method of providing media data according to an embodiment of the present invention.

As shown in FIG. 3, the method for providing media data according to an embodiment of the present invention first encapsulates the encoded media data (S310). The encapsulating step S310 may include an operation dependent on at least one of a payload and a media.

Here, the encapsulation step S310 may include media packetization that constitutes packetization that is optimal for a specific media type. In addition, the encapsulation step S310 may include media synchronization at an intra-media level. Here, the encapsulation step S310 may include a fragmentation of dividing one media unit into a plurality of packets. The encapsulation step S310 may include an aggregation that combines a plurality of media units into one media unit. The encapsulation step S310 may include application-dependent error handling.

Then, the media data providing method according to an embodiment of the present invention delivers the encapsulated media data from a specific entity to another entity (S320). The delivering step S320 may include an operation independent of at least one of a payload and a media. The delivering step S320 may include multiplexing two or more substreams in one port stream. In addition, the step of transmitting (S320) may include transmitting timing information for inter-flow synchronization. The transmitting step S320 may include delivering timing information for system synchronization. The delivering step S320 may include a generic error handling. The delivering step S320 may include real time media friendly flow control.

Then, the media data providing method according to an embodiment of the present invention controls the delivery of the media data (S330).

Details of the method for providing media data according to an embodiment of the present invention are the same as the above-described interface for providing media data.

Claims (22)

An encapsulation layer encapsulating the encoded media data;
A delivery layer for delivering the encapsulated media data from one entity to another entity; And
Including a control layer (Control layer) for controlling the transmission of the media data,
The encapsulation layer includes functionality dependent on at least one of a payload and media,
The transport layer includes a function independent of at least one of the payload and the media.
The method of claim 1, wherein the encapsulation layer is
An interface for providing media data, comprising a Media Packetization function for configuring packetization that is optimal for a specific media type.
The method of claim 1, wherein the encapsulation layer is
An interface for providing media data, comprising an intra-media level Media Synchronization function.
The method of claim 1, wherein the encapsulation layer is
An interface for providing media data, comprising: a fragmentation function of dividing one media unit into a plurality of packets.
The method of claim 1, wherein the encapsulation layer is
Interface for providing media data, characterized in that it comprises an aggregation function for combining a plurality of media units into one media unit.
The method of claim 1, wherein the encapsulation layer is
An interface for providing media data, including application-dependent error handling.
The method of claim 1, wherein the transport layer is
And a function of multiplexing two or more sub-streams in one port stream.
The method of claim 1, wherein the transport layer is
Interface for providing media data, characterized in that it comprises timing information for inter-flow synchronization.
The method of claim 1, wherein the transport layer is
Interface for providing media data, characterized in that it comprises timing information for System Synchronization.
The method of claim 1, wherein the transport layer is
Interface for providing media data, characterized by including a generic error handling (Generic Error Handling).
The method of claim 1, wherein the transport layer is
Interface for providing media data, comprising a real-time media-friendly flow control function.
Encapsulating the encoded media data;
Passing the encapsulated media data from one entity to another; And
Controlling delivery of the media data;
The encapsulating comprises an operation dependent on at least one of a payload and media,
And wherein said delivering step comprises an operation independent of at least one of a payload and media.
The method of claim 12, wherein the encapsulation step
Media packetization comprising media packetization that is optimal for a specific media type.
The method of claim 12, wherein the encapsulation step
A method of providing media data, comprising intra-media level media synchronization.
The method of claim 12, wherein the encapsulation step
And a fragmentation of dividing one media unit into a plurality of packets.
The method of claim 12, wherein the encapsulation step
And aggregating a plurality of media units into one media unit.
The method of claim 12, wherein the encapsulation step
Method for providing media data, comprising application-dependent error handling (Application-Dependent Error Handling).
The method of claim 12, wherein said delivering step
And multiplexing two or more sub-streams in one port stream.
The method of claim 12, wherein said delivering step
A method for providing media data, comprising the delivery of timing information for inter-flow synchronization.
The method of claim 12, wherein said delivering step
A method of providing media data, comprising the delivery of timing information for system synchronization.
The method of claim 12, wherein said delivering step
A method for providing media data, comprising batch error handling.
The method of claim 12, wherein said delivering step
A method for providing media data comprising real time media friendly flow control.

Images