KR20130032843A - Media data transmission apparatus and method, and media data reception apparatus and method in mmt system - Google Patents

Abstract

PURPOSE: A media data transmitting device for an MMT system, a method thereof, a media data receiving device, and a method thereof are provided to supply basic timing information required for maintaining synchronization between different media. CONSTITUTION: A receiving unit(510) receives a delivery layer packet. A depacketizing unit(520) generates encapsulation layer data by depacketizing the delivery layer packet. A decapsulating unit(530) generates an encoded media stream by decapsulating the encapsulation layer data and extracts timing information. A decoder(550) decodes the encoded media stream. [Reference numerals] (505) Output device; (510) Receiving unit; (520) Depacketizing unit; (530) Decapsulating unit; (540) Buffer; (550) Decoder; (560) Rendering buffer; (570) Control unit; (AA) D-layer packet;

Description

MEDIA DATA TRANSMISSION APPARATUS AND METHOD, AND MEDIA DATA RECEPTION APPARATUS AND METHOD IN MMT SYSTEM}

The present invention relates to an apparatus and method for transmitting and receiving media data, and more particularly, to an encapsulation layer (E) required for transmission and reception of media data for an MMT (MPEG Media Transport) system. -layer) timing information.

MPEG Media Transport (MMT) is a new standard technology that has begun development in the MPEG systems sub-working group. The existing MPEG-2 system has been widely used since MPEG-2 transport stream (TS) technology has been standardized as a standard for packetization, synchronization, and multiplexing for transmitting A / V content in a broadcasting network. However, MPEG-2 TS is inefficient in a packet transmission environment in which a network is based on IP (Internet Protocol). Therefore, ISO MPEG recognized the necessity of a new media transmission standard in consideration of the new media transmission environment and the expected media transmission environment, and started to standardize MMT.

Conventional techniques similar to MMT's E-layer timing model include the decoding time stamp (DTS), the presentation time stamp (PTS) -based timing model, and the real-time transport protocol (RTP) protocol adopted by MPEG-2 system technology. There is a timing model based on RTP timestamp and network time protocol (NTP) timestamp information.

More specifically, there are two conventionally developed timing models for media transmission, firstly, MPEG-2 system technology, and secondly, a method of combining RTP and RTCP (RTP control protocol). to be. In the MPEG-2 system, PTS and DTS timing information are used as timing information for constructing a timing model for determining a media playback time. In the case of using a combination of RTP and RTCP, the RTP timestamp information recorded in the RTP and the NTP time stamp recorded in the RTCP sender report are simultaneously used.

MPEG-2 system technology proposes a timing model for delivering compressed media through a stable transmission network such as a broadcasting network. Since MPEG-2 system is a standard developed for digital broadcasting service, MPEG-2 TS (transport stream) packets transmitted to the receiver are transmitted to the receiver through a broadcasting network which is a circuit switched network with relatively stable channel quality. do. Accordingly, MPEG-2 TS packets have a relatively short and constant packet delay time experienced in a transport channel, and a timing model for sequentially processing TS packets arriving at a receiver also operates relatively stably. However, in the case of an IP network other than a broadcasting network, the arrival delay time intervals experienced by the TS packets transmitted are very irregular, and therefore, the timing model adopted by the MPEG-2 system technology is difficult to maintain stably.

In the RTP / RTCP-based timing model, the RTP timestamp recorded in the header of the RTP packet indicates the internal temporal ordering relationship of a specific media stream. Accordingly, in order to provide synchronization between different media streams, timing information corresponding to an absolute time (wall-clock) must be delivered. The timing information transmitted to the terminal for this purpose is an NTP timestamp. The NTP timestamp is transmitted in an RTCP sender report (SRTCP) packet. The NTP timestamp is repeatedly transmitted at regular intervals. Since RTCP SR packets are transmitted separately from the RTP stream for transmitting media, the operation of the transmitting and receiving system is complicated by increasing the traffic burden on the network and increasing the number of UDP ports and streams to be managed by the server / terminal. Lose.

Therefore, in order to solve the problems of the above methods, it is necessary to consider a new timing model that enables simple and efficient operation in an E-layer of a newly standardized MMT technology.

Korean Unexamined Patent Publication No. 10-2011-0022664 ("Synchronization of Media Stream Components", published by Koninkli Philips Electronics V., March 7, 2011)

Accordingly, an object of the present invention for solving the above problems is to provide the basic timing information necessary for the timing of the playback of the media and the time synchronization between the media transmitted in the media transmission service based on the MMT system, MPEG- which is a conventional timing model The present invention provides an apparatus and method for transmitting media data that can provide timing information compatible with the timing system of the 2 system and the RTP / RTCP-based timing model.

In addition, another object of the present invention is to provide the basic timing information necessary for the timing of the playback of the media and the time synchronization between the media transmitted in the MMT system-based media transmission service, and the timing model of the conventional MPEG-2 system The present invention provides an apparatus and method for receiving media data that can provide timing information compatible with an RTP / RTCP-based timing model.

An apparatus for transmitting media data according to an embodiment of the present invention for encapsulating an encoded media stream generates an encapsulation layer data (E-layer data) including timing information. (Encapsulator), wherein the timing information may include first sampling time information, second sampling time information, and rendering time information. Here, the media data transmission device includes an encoder for generating media streams by encoding media data; A buffer that stores the encoded media stream; A packetizer for packetizing the E-layer data to generate a delivery layer packet (D-layer packet); And a transmitter configured to transmit the packetized D-layer packet. The first sampling time information may be 1 bit, and the second sampling time information may be 32 bits. Here, the first sampling time information is a value of the 33rd bit of the sampling time based on a sampling clock frequency, and the second sampling time information is a lower 32 bit value of the sampling time. It may be characterized by.

In accordance with another aspect of the present invention, an apparatus for receiving media data generates an encoded media stream by decapsulating encapsulation layer data (E-layer data), and extracts timing information. A decapsulator may be included. The timing information may include first sampling time information, second sampling time information, and rendering time information. Here, the media data receiving apparatus includes a receiving unit for receiving a delivery layer packet (D-layer packet); A depacketizer configured to depacketize the D-layer packet to generate the E-layer data; A buffer for storing the encoded media stream; A decoder for decoding the encoded media stream; And a rendering buffer for rearranging the decoded media data for display. In addition, the rendering time information may represent a time required until the playback time after the media stream is decoded. The method may further include a controller configured to determine a sampling time based on the first sampling time information and the second sampling time information, and to determine a rendering time indicating an accurate playback time of media based on the sampling time. It can be characterized. The controller may generate a decoding time stamp (DTS) and a presentation time stamp (PTS) of the MPEG-2 system based on the first sampling time information, the second sampling time information, and the rendering time information. The controller may generate an RTP time stamp of a real time protocol (RTP) based on the second sampling time information and the rendering time information.

In accordance with another aspect of the present invention, there is provided a media data transmission method for generating encapsulation layer data (E-layer data) including timing information by encapsulating an encoded media stream. In addition, the timing information may include first sampling time information, second sampling time information, and rendering time information. The media data transmission method may include generating a media stream by encoding media data; Storing the encoded media stream; Packetizing the E-layer data to generate a delivery layer packet (D-layer packet); And transmitting the packetized D-layer packet. The first sampling time information may be 1 bit, and the second sampling time information may be 32 bits. Here, the first sampling time information is a value of the 33rd bit of the sampling time based on a sampling clock frequency, and the second sampling time information is a lower 32 bit value of the sampling time. It may be characterized by.

In accordance with another aspect of the present invention, a method for receiving media data includes decapsulating encapsulation layer data (E-layer data) to generate an encoded media stream and extract timing information. The timing information may include first sampling time information, second sampling time information, and rendering time information. The method for receiving media data may include receiving a delivery layer packet (D-layer packet); Depacketizing the D-layer packet to generate the E-layer data; Storing the encoded media stream; Decoding the encoded media stream; And rearranging the decoded media data for display. Here, the rendering time information may represent a time required until a playback time after the media stream is decoded. The method may further include determining a sampling time based on the first sampling time information and the second sampling time information, and determining a rendering time indicating a correct playback time of the media based on the sampling time. It can be characterized. The method may further include generating a decoding time stamp (DTS) and a presentation time stamp (PTS) of the MPEG-2 system based on the first sampling time information, the second sampling time information, and the rendering time information. The method may further include generating an RTP time stamp of a real time protocol (RTP) based on the second sampling time information and the rendering time information.

According to the above-described apparatus for transmitting and receiving media data according to an embodiment of the present invention, the MMT transmission system provides timing information such as Sampling_Time_Ext, Sampling_Time_Base, Rendering_Time_Offset, etc. to the receiving terminal to derive a playback time for media, Basic timing information is provided to maintain lip-synchronization between different media such as audio. In particular, the Sampling Time information is divided into a 1-bit Sampling_Time_Ext field and a 32-bit Sampling_Time_Base field to express the 33-bit DTS / PTS timing information used in the existing MPEG-2 system and the 32-bit RTP used in the RTP. Maintain compatibility with time stamp information.

1 is a conceptual diagram illustrating an MMT hierarchical structure.
2 shows basic timing information to be recorded in an E-layer header of an MMT.
3 shows a temporal correlation between timing information recorded in an E-layer header.
4 is a block diagram illustrating a configuration of an apparatus for transmitting media data according to an embodiment of the present invention.
5 is a block diagram illustrating a configuration of an apparatus for receiving media data according to an embodiment of the present invention.
FIG. 6 illustrates an operation procedure performed by the media data receiving apparatus of FIG. 5 at a receiving terminal of an MMT system for timing synchronization using the timing information shown in FIG. 3.
FIG. 7 illustrates a case where timing information recorded in a header of MMT E-layer data is mapped to DTS and PTS information of an MPEG-2 system.
8 shows a case where the timing information recorded in the header of the MMT E-layer data is mapped to the RTP time stamp information of the RTP packet.
9 is a flowchart illustrating a method of transmitting media data according to an embodiment of the present invention.
10 is a flowchart illustrating a method of receiving media data according to an embodiment of the present invention.
11 is a detailed flowchart of a step of determining the sampling time and rendering time of FIG. 10.

As the inventive concept allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description.

It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. In order to facilitate the understanding of the present invention, the same reference numerals are used for the same constituent elements in the drawings and redundant explanations for the same constituent elements are omitted.

MMT  Hierarchy

1 is a conceptual diagram illustrating an MMT hierarchical structure.

Referring to FIG. 1, the MMT layer includes an encapsulation layer, a delivery layer, and a functional area of a signaling layer. The MMT layer operates on a transport layer.

The encapsulation layer (E-layer) can perform functions such as packetization, fragmentation, synchronization, and multiplexing of the transmitted media, for example.

The encapsulation layer (E-layer) includes an MMT E.1 layer (MMT E.1 Layer), an MMT E.2 layer (MMT E.2 Layer), and an MMT E.3 layer E.3 Layer).

The E.3 layer encapsulates a Media Fragment Unit (MFU) provided from the Media Codec (A) layer to create an M-Unit.

The MFU may have a format, independent of any particular codec, that can carry data units that can be consumed independently in the media decoder. The MFU can be, for example, a picture or slice of the video.

The M-unit may consist of one or a plurality of MFUs and may have a format, independent of a particular codec, that may carry one or a plurality of access units.

The E.2 layer encapsulates the M-units created in the E.3 layer to create an MMT asset.

An MMT asset is a data entity composed of one or a plurality of M-units from a single data source and is a data unit in which composition information and transport characteristics are defined. MMT assets can correspond to packetized elementary streams (PES), for example video, audio, program information, MPEG-U widgets, JPEG images, MPEG 4 file format, M2TS (MPEG transport stream) and the like.

E.1 Layer (E.1 Layer) encapsulates MMT asset created in E.2 layer to create MMT Package (MMT Package).

The MMT package may be composed of one or more MMT assets together with additional information such as composition information and transport characteristics. Composition information includes information about a relationship between MMT assets, and when one content consists of a plurality of MMT packages, it indicates a relationship between a plurality of MMT packages. It may further include information. The transport characteristics may include transmission characteristic information necessary for determining a delivery condition of an MMT asset or an MMT packet, and may include, for example, a traffic description parameter and a QoS descriptor. ) May be included. The MMT package may correspond to a program of MPEG-2 TS.

The delivery layer may perform, for example, network flow multiplexing, network packetization, and QoS control of media transmitted through a network.

1, the D-layer includes an MMT D.1 layer (MMT D.1 Layer), an MMT D.2 layer (MMT D.2 Layer), and an MMT D.3 layer (MMT D.3 Layer).

The D.1 layer receives the MMT package generated in the E.1 layer and generates an MMT payload format. The MMT payload format is a payload format for carrying MMT assets and for transmitting information for consumption by the MMT application protocol or other existing application transport protocol such as RTP. The MMT payload may contain fragments of MFUs with information such as AL-FEC.

The D.2 layer receives the MMT payload format generated in the D.1 layer and generates an MMT transport packet or an MMT packet. The MMT transport packet or MMT packet is a data format used in an application transport protocol for MMT.

D.3 layer (D.3-layer) supports QoS by providing the function of exchanging information between layers by cross-layer design. For example, the D.3 layer may perform QoS control using QoS parameters of the MAC / PHY layer.

The signaling layer performs a signaling function. For example, signaling functions for session initialization / control / management of transmitted media, server-based and / or client-based trick modes, service discovery, synchronization, etc. Can be done.

Signaling layer (Signaling layer), as shown in Figure 1, may be composed of MMT S.1 layer (MMT S.1 Layer) and MMT S.2 layer (MMT S.2 Layer).

The S.1 layer consists of service discovery, media session initialization / termination, media session presentation / control, transport layer (D) and encapsulation (E) And an interface function with the layer. The S.1 layer may define the format of control messages between applications for media presentation session management.

The S.2 layer is responsible for flow control, delivery session management, delivery session monitoring, error control, and hybrid network synchronization control. It is possible to define the format of the control message exchanged between delivery end-points of the D-layer.

The S.2 layer supports delivery session establishment and release, delivery session monitoring, flow control, error control, resource scheduling for established delivery sessions, and synchronization in a complex delivery environment to support the behavior of the delivery layer. Signaling for adaptive delivery, and signaling for adaptive delivery. Required signaling may be provided between a sender and a receiver. That is, the S.2 layer may provide signaling required between the sender and the receiver in order to support the operation of the transport layer as described above. In addition, the S.2 layer may be responsible for interfacing with the transport layer and the encapsulation layer.

The present invention relates to an apparatus and method for transmitting media data, including basic E-layer timing information for acquiring playback time information on media and playing media while maintaining time synchronization between media in an MMT system. It is about. The proposed timing information can be utilized while maintaining compatibility with the existing MPEG-2 system and RTP timing model, so that the data generated in the MMT E-layer is not only D-layer (Delivery layer) of MMT. In addition, it can be smoothly transmitted by the existing RTP. When the data generated by the MMT E-layer is converted into the MPEG-2 system format, the timing information can be easily converted while maintaining compatibility with the representation method of the timing information adopted by the MPEG-2 system.

That is, in the present invention, by devising efficient timing information to be included in the E-layer, the timing of the media transmission and the time synchronization between the media in the MMT system-based media transmission service aiming at efficient media transmission over the IP network are determined. Provide the basic timing information you need. Here, we devise a simple and efficient timing model that can be applied to the E-layer of the MMT system by complementing the shortcomings of the existing timing model of the MPEG-2 system and the RTP / RTCP based timing model. Suggest timing information for operation. The timing information is used to provide accurate playback time information of frames in the media stream to enable media playback at a predetermined time, and further, to provide basic time information required for synchronization between different media.

4 is a block diagram illustrating a configuration of an apparatus for transmitting media data according to an embodiment of the present invention. As shown in FIG. 4, the media data transmission apparatus 400 according to an embodiment of the present invention includes an encoder 410 for generating a media stream by encoding media data, and a buffer that stores the encoded media stream. 420, an encapsulator 430 that encapsulates the encoded media stream to generate encapsulation layer data (E-layer data) including timing information, and transmits the packetized E-layer data. It may include a packetizer 440 for generating a delivery layer packet (D-layer packet) and a transmission unit 450 for transmitting the packetized D-layer packet. Here, the timing information included in the E-layer data may be compatible with the MPEG-2 system and the Real Time Protocol (RTP). More specifically, the timing information may include first sampling time information, second sampling time information, and rendering time information.

2 shows timing information recorded in a header of E-layer data of an MMT. Referring to FIG. 2, first sampling time information, second sampling time information, and rendering time information included in the timing information will be described in detail.

The timing information may be allocated to a sample of media data, such as a picture of a video or a frame of audio, or to E-layer data in units of one or more of the samples of the media data. As shown in FIG. 2, the E-layer timing information includes first sampling time information (hereinafter, 'Sampling_Time_Ext', 210), second sampling time information (hereinafter, 'Sampling_Time_Base', 220), and rendering time information (hereinafter, 'Rendering_Time_Offset' ', 230) may be included. Here, the Sampling_Time_Ext 210 may be 1 bit, the Sampling_Time_Base 220 is 32 bits, and the Rendering_Time_Offset 230 may be 20 bits.

Sampling Time can be obtained from a sampling clock frequency operating at 90 KHz precision commonly used in MPEG-2 systems and RTP transmission systems. If a sampling clock frequency other than 90 KHz precision is adopted, the same principle can be applied. Sampling time is the result of obtaining the sampling time for the pictures inputted in the order of compression to the media encoder of the MMT transmission system from the 90 KHz sampling clock frequency. Sampling time can be a total of 33 bits. Here, the Sampling_Time_Ext 210 may mean the value of the 33rd bit of the Sampling Time, and the Sampling_Time_Base 220 may represent the lower 32 bits of the Sampling Time.

The Rendering_Time_Offset 230 represents a time required until decoding, which is a playback time, after decoding. After the decompressed media data obtained after decoding is stored in a rendering buffer, the decompressed media data may be reproduced by an output device at a rendering time. The rendering buffer is also used for picture reordering of I-pictures, P-pictures, and B-pictures that occur when a B-picture is present in the video sequence structure. . Normally, B-pictures are reproduced immediately after decoding, so they are immediately output without undergoing a rendering buffer process. However, I-pictures or P-pictures must wait for Rendering_Time_Offset until they are decoded and played.

If the correlation between such time information is represented graphically as shown in FIG.

3 shows a temporal correlation between timing information recorded in an E-layer header. As shown in FIG. 3, delivery time and decoding time are obtained through timing information delivered in a header of a D-layer, which is responsible for generating an MMT packet of a transport format.

5 is a block diagram illustrating a configuration of an apparatus for receiving media data according to an embodiment of the present invention.

As shown in FIG. 5, the apparatus 500 for receiving media data according to an embodiment of the present invention includes a receiver 510 for receiving a delivery layer packet (D-layer packet) and the D-layer packet. A depacketizer 520 for generating encapsulation layer data (E-layer data) by depacking the data, and generating an encoded media stream by decapsulating the E-layer data and extracting timing information. A decapsulator 530, a buffer 540 for storing the encoded media stream, a decoder 550 for decoding the encoded media stream, and a rendering buffer for reordering the decoded media data for display. Rendering buffer 560). Here, the timing information may be compatible with the MPEG-2 system and the Real Time Protocol (RTP). In detail, the timing information may include first sampling time information, second sampling time information, and rendering time information. The timing information is the same as the timing information of the aforementioned media data transmission device. That is, the E-layer timing information may include three fields such as first sampling time information (hereinafter referred to as 'Sampling_Time_Ext'), second sampling time information (hereinafter referred to as 'Sampling_Time_Base'), and rendering time information (hereinafter referred to as 'Rendering_Time_Offset'). Can be.

Here, the media data receiving apparatus 500 determines a sampling time based on the first sampling time information and the second sampling time information, and renders time indicating an accurate playback time point of the media based on the sampling time. It may further include a control unit 570 for determining. Equation for the control unit 570 to obtain the Sampling Time and Rendering Time values by the time relationship of FIG. 3 is as follows. First, the sampling time may be calculated by Equation 1 below.

Decoding Time can be calculated by Equation 2 below using Sampling Time information of the E-layer and Sender_Processing_Delay and Receiver_Processing_Delay provided from the D-layer.

 The rendering time representing the exact playback time of the media can be calculated by Equation 3 below.

On the other hand, when the operation of the MMT system is not intended for transmission over the IP network, but only for direct storage to the local device, Sender_Processing_Delay, Transmission_Delay, and Receiver_Processing_Delay do not exist. Therefore, Decoding Time and Rendering Time required for locally reproducing MMT data stored in a local storage device in the terminal may be calculated by Equations 4 and 5, respectively.

In this case, unlike Equation 2, the Sampling_Time value itself is used as the Decoding_Time value.

FIG. 6 illustrates an operation procedure performed by the media data receiving apparatus of FIG. 5 at a receiving terminal of an MMT system for timing synchronization using the timing information shown in FIG. 3. An operation procedure performed at the media data receiving apparatus side of the MMT system using the timing information of the MMT E-layer shown in FIG. 3 is shown in FIG. 6.

As shown in FIG. 2, the reason why the Sampling Time is divided into Sampling_Time_Ext and Sampling_Time_Base fields is displayed so that timing information of the MMT E-layer can be directly converted into timing information and RTP timestamp value of the existing MPEG-2 system. To do this. Data generated by MMT should be easily convertible to a storage format such as the MPEG-2 system and a transport format such as an RTP packet.

Therefore, when the data generated by the E-layer of the MMT system is converted into the MPEG-2 system format, the DTS and PTS timing information included in the MPEG-2 packetized elementary system (PES) packet header is converted to the E-layer timing information of the MMT. Efficiently derived from In addition, even when transmitting data generated in the E-layer of the MMT system using the RTP protocol, the RTP timestamp value included in the RTP protocol header should be efficiently derived from the MMT E-layer timing information. In order to satisfy both cases, in the present invention, a value corresponding to the 33rd bit of the Sampling Time value having a total size of 33 bits is represented by a Sampling_Time_Ext field having a size of 1 bit, and the value corresponding to the remaining 32 bits is represented by a Sampling_Time_Base field. Can be displayed as As described above, when the Sampling Time information is divided into a 1-bit Sampling_Time_Ext field and a 32-bit Sampling_Time_Base field, both 33-bit DTS / PTS values and 32-bit RTP timestamp values of the MPEG-2 system can be directly obtained. .

FIG. 7 illustrates a case where timing information recorded in a header of MMT E-layer data is mapped to DTS and PTS information of an MPEG-2 system. The control unit 570 of the apparatus for receiving media data according to an embodiment of the present invention uses the DTS (MPS) of the MPEG-2 system based on the first sampling time information (Sampling_Time_Ext), the second sampling time information (Sampling_Time_Base), and the rendering time information. Decoding Time Stamp) and PTS (Presentation Time Stamp) can be generated.

More specifically, the timing information recorded in the header of the MMT E-layer data may be mapped to the DTS value of the MPEG-2 system based on Equation 6 below.

In Equation 6, Sender_Processing_to_Decoding_Delay represents an expected value for the sum of all of the Sender Processing Delay, Transmission Delay, and Receiver Processing Delay shown in FIG. 3.

In addition, the timing information recorded in the header of the MMT E-layer data may be mapped to the PTS value of the MPEG-2 system based on Equation 7 below using the result of Equation 6.

On the other hand, if the DTS and PTS information calculated by Equations 6 and 7 are used, the time information of the media data transmitted by the MMT system can be interpreted as a format of the time information used by the MPEG-2 system. Time synchronization can also be achieved with media data transmitted via the -2 system. That is, when one stream is transmitted through the MMT system and another stream is transmitted through the MPEG system, temporal synchronization between the two streams can be achieved.

8 shows a case where the timing information recorded in the header of the MMT E-layer data is mapped to the RTP time stamp information of the RTP packet. The controller 570 of the apparatus for receiving media data according to an embodiment of the present invention may generate an RTP time stamp of a real time protocol (RTP) based on the second sampling time information and the rendering time information. . Since the RTP timestamp has a total length of 32 bits, direct mapping to the RTP timestamp value is possible based on Equation 8 below.

Therefore, the timing information of the MMT E-layer can be directly converted into the timing information and the RTP timestamp value of the conventional MPEG-2 system. That is, the data generated by the MMT can be easily converted into a storage format such as the MPEG-2 system and a transmission format such as an RTP packet.

9 is a flowchart illustrating a method of transmitting media data according to an embodiment of the present invention.

As shown in FIG. 9, the media data transmission method according to an embodiment of the present invention includes generating a media stream by encoding media data (S910), storing the encoded media stream (S920), and Generating encapsulation layer data (E-layer data) including the timing information by encapsulating the encoded media stream (S930); packetizing the E-layer data to deliver a delivery layer packet; Generating a D-layer packet (S940) and transmitting the packetized D-layer packet (S950). Here, the timing information may include first sampling time information, second sampling time information, and rendering time information.

Here, the first sampling time information may be 1 bit, and the second sampling time information may be 32 bits. The first sampling time information may be a sampling time based on a sampling clock frequency. It is the value of the 33rd bit of the second sampling time information may be characterized in that the value of the lower 32 bits of the sampling time (Sampling Time).

10 is a flowchart of a method of receiving media data according to an embodiment of the present invention, and FIG. 11 is a detailed flowchart of steps of determining a sampling time and a rendering time of FIG. 10.

As shown in FIG. 10, the method for receiving media data according to an embodiment of the present invention first receives a delivery layer packet (D-layer packet) (S1010). The E-layer data is generated by depacketizing the D-layer packet (S1020). Thereafter, the encapsulation layer data (Encapsulation layer data, E-layer data) may be decapsulated to generate an encoded media stream and to extract timing information (S1030). Here, the timing information may be compatible with the MPEG-2 system and the Real Time Protocol (RTP). In detail, the timing information may include first sampling time information, second sampling time information, and rendering time information. In addition, the rendering time information may represent a time required until the playback time after the media stream is decoded.

Thereafter, a sampling time may be determined based on the first sampling time information and the second sampling time information, and a rendering time indicating an accurate playback time of the media may be determined based on the sampling time (S1040).

Referring to FIG. 11, the sampling time and rendering time determination step (S1040) will be described in more detail. After receiving the data header of the MMT E-layer (S1041), the first sampling time information (Sampling_Time_Ext) is extracted (S1042). In operation S1043, the second sampling time information Sampling_Time_Base may be extracted. The sampling time may be calculated based on the first sampling time information and the second sampling time information (S1044), and may be used to calculate a decoding time (S1046). In addition, rendering time information (Rendering_Time_Offset) is extracted from the data header of the MMT E-layer (S1047), and a rendering time (Rendering Time) may be calculated based on the calculated decoding time and the rendering time information (S1047). .

Referring back to FIG. 10, the encoded media stream may be stored (S1050), the encoded media stream is decoded (S1060), and the decoded media data may be rearranged (S1070) for display.

Claims (22)

An apparatus for transmitting media data,
An encapsulator for encapsulating the encoded media stream to generate encapsulation layer data (E-layer data) including timing information.
And the timing information is compatible with an MPEG-2 system and a Real Time Protocol (RTP).
The method of claim 1, wherein the timing information is
And first sampling time information, second sampling time information, and rendering time information.
The method of claim 1,
An encoder for encoding the media data to produce a media stream;
A buffer that stores the encoded media stream;
A packetizer for packetizing the E-layer data to generate a delivery layer packet (D-layer packet); And
And a transmitter for transmitting the packetized D-layer packet.
The method of claim 2,
The first sampling time information is 1 bit,
And the second sampling time information is 32 bits.
The method of claim 4, wherein
The first sampling time information is a value of the 33rd bit of the sampling time based on the sampling clock frequency.
And the second sampling time information is a lower 32-bit value of the sampling time.
An apparatus for receiving media data, the apparatus comprising:
A decapsulator for decapsulating encapsulation layer data (E-layer data) to generate an encoded media stream and extracting timing information;
And the timing information is compatible with an MPEG-2 system and a Real Time Protocol (RTP).
The method of claim 6, wherein the timing information is
And first sampling time information, second sampling time information, and rendering time information.
The method according to claim 6,
A receiver for receiving a delivery layer packet (D-layer packet);
A depacketizer configured to depacketize the D-layer packet to generate the E-layer data;
A buffer for storing the encoded media stream;
A decoder for decoding the encoded media stream; And
And a rendering buffer for reordering the decoded media data for display.
8. The method of claim 7,
And a controller configured to determine a sampling time based on the first sampling time information and the second sampling time information, and to determine a rendering time indicating an accurate playback time of media based on the sampling time. Media data receiving device.
8. The apparatus of claim 7, wherein the control unit
And generating a decoding time stamp (DTS) and a presentation time stamp (PTS) of the MPEG-2 system based on the first sampling time information, the second sampling time information, and the rendering time information.
8. The apparatus of claim 7, wherein the control unit
And generating an RTP time stamp of a real time protocol (RTP) based on the second sampling time information and the rendering time information.
In the method for transmitting media data,
Encapsulating the encoded media stream to generate encapsulation layer data (E-layer data) including timing information;
And the timing information is compatible with an MPEG-2 system and a Real Time Protocol (RTP).
The method of claim 12, wherein the timing information is
And first sampling time information, second sampling time information, and rendering time information.
13. The method of claim 12,
Encoding the media data to generate a media stream;
Storing the encoded media stream;
Packetizing the E-layer data to generate a delivery layer packet (D-layer packet); And
And transmitting the packetized D-layer packet.
The method of claim 13,
The first sampling time information is 1 bit,
And the second sampling time information is 32 bits.
The method of claim 15,
The first sampling time information is a value of the 33rd bit of the sampling time based on the sampling clock frequency.
And the second sampling time information is a lower 32-bit value of the sampling time.
In the method for receiving media data,
Decapsulating encapsulation layer data (E-layer data) to generate an encoded media stream and extracting timing information.
And the timing information is compatible with an MPEG-2 system and a Real Time Protocol (RTP).
18. The method of claim 17, wherein the timing information is
And first sampling time information, second sampling time information, and rendering time information.
The method of claim 17,
Receiving a delivery layer packet (D-layer packet);
Depacketizing the D-layer packet to generate the E-layer data;
Storing the encoded media stream;
Decoding the encoded media stream; And
Reordering the decoded media data for display.
19. The method of claim 18,
Determining a sampling time based on the first sampling time information and the second sampling time information, and determining a rendering time indicating an accurate playback time of media based on the sampling time. The method of receiving media data.
19. The method of claim 18,
Generating a decoding time stamp (DTS) and a presentation time stamp (PTS) of the MPEG-2 system based on the first sampling time information, the second sampling time information, and the rendering time information. How to receive data.
19. The method of claim 18,
And generating an RTP time stamp of a real time protocol (RTP) based on the second sampling time information and the rendering time information.

Images