WO2013009131A2

WO2013009131A2 - Apparatus and method for transmitting packet and apparatus and method for receiving packet in mmt system

Info

Publication number: WO2013009131A2
Application number: PCT/KR2012/005584
Authority: WO
Inventors: 이진영; 서광덕; 윤국진; 정원식; 허남호
Original assignee: 한국전자통신연구원
Priority date: 2011-07-13
Filing date: 2012-07-13
Publication date: 2013-01-17
Also published as: WO2013009131A3

Abstract

The present invention discloses an apparatus and a method for transmitting a packet and an apparatus and a method for receiving a packet, based on a timing model of an MPEG media transport (MMT) system, which is integrated by considering the advantages of a timing model in an MPEG-2 system and an RTP timing model. The apparatus for transmitting the packet transmits the packet from the MMT system, and comprises: a video stream that is encoded based on an input picture; and an MMT packetizing portion for generating the MMT packet by packetizing an audio stream that corresponds to the video stream, wherein the MMT packetizing portion generates the MMT packet by including synchronization information, which is generated based on a system time clock (STC) that is a master program clock for a video and audio encoder, and an MMT buffering delay. As a result, synchronization of various media sources, measurement of a packet arrival jitter, as well as STC clock locking of the apparatus for transmitting and the apparatus for receiving the MMT can be achieved.

Description

Packet transmission apparatus and method, and packet reception apparatus and method in MMT system

The present invention relates to a packet transmission apparatus and method, and a packet receiving apparatus and method, and more particularly, to a packet transmission apparatus and method based on a timing model for an MPEG Media Tranport (MMT) system, and a packet receiving apparatus and method. It is about.

In an MPEG-2 system, every picture and audio sample contained in an encoder appears exactly once each after the end-to-end delay, which occurs continuously at the output of the decoder. Research is underway to implement the modeling (timing model). Given that the decoding system correctly delivers and receives a compliant bit stream containing the timing model, it is not too difficult to synchronize the high quality video and audio output at the decoder. In contrast, the Real Time Transport Protocol (RTP) timing model does not have any timing information related to the actual presentation time. The timestamp included in the data packet has relative timing, and the Real Time Control Protocol (RTCP) transmitter report provides additional information about inter-stream synchronization. However, RTP does not provide any information about the capacity of the buffer required at the receiver or the decoding time of the packet. Thus, RTP has a transmitter-oriented flexible timing model (the description of the receiver is not intentionally specifically limited to allow for flexibility). In contrast, the MPEG-2 system provides a clear timing model for the receiver. Therefore, due to the difference between the timing model of the MPEG-2 system and the timing model for the receiver requiring the timing model of the RTP system, it is difficult to match the packet synchronization in the MMT system, which requires a complicated configuration. .

SUMMARY OF THE INVENTION An object of the present invention is to provide an apparatus and method for transmitting a packet based on a timing model of an integrated MMT system in consideration of the advantages of the timing model and the RTP timing model in an MPEG-2 system. It is to provide an apparatus and method.

In addition, another object of the present invention is to transmit a packet stream transmitted through the MPEG-2 system or RTP into a new type of packet including time stamp information used for network jitter compensation, time synchronization, and the like. It is an object of the present invention to provide a packet transmission apparatus and method, and a packet receiving apparatus and method in an MMT system that effectively perform synchronization without countering this.

A packet transmission apparatus of an MMT system for achieving the above object is a device for transmitting a packet in a MPEG Media Transport (MMT) system, the packet is encoded video stream and the audio stream corresponding to the video stream based on the input picture And an MMT packetizer for generating an MMT packet, wherein the MMT packetizer includes synchronization information generated based on a system time clock (STC) and an MMT buffering delay, which are master program clocks of a video and audio encoder. To generate the MMT packet.

The MMT packet transmission apparatus may include a video encoder that encodes the input picture to generate a video stream; A video buffer that stores the encoded video stream; And a transmitter configured to transmit the packetized MMT packet, wherein the video buffer may output the video stream at a constant rate.

The MMT packetizer may packetize the video stream output from the video buffer including MCR clock reference (MCR) time stamp information generated by time stamping the video stream output from the video buffer.

The packetizer may control at least one of a data rate and a packet rate in a channel for transmitting the MMT packet using the MCR time stamp information.

The MMT packetizer may insert the MCR time stamping information into the D.1 layer of the MMT packet.

The MMT packetization unit generates a PTS (Presentation Time Stamp) generated by adding a sum of a buffering delay at the transmitting apparatus, a buffering delay at the receiving apparatus, and a network delay to a time stamp value of the input picture time stamped using the STC. ) May be generated to include the MMT packet.

The MMT packetizer may insert the PTS information into the E.3 layer of the MMT packet.

The STC value varies depending on a program, and the MMT packetizer may multiplex different programs having different STCs.

In the packet transmission method of the MMT system for achieving the above object, in the method for transmitting a packet in the MMT system, MMT packet by packetizing the encoded video stream and the audio stream corresponding to the video stream based on the input picture And generating an MMT packetization step, wherein the MMT packetization step includes the MMT packet including synchronization information generated based on an STC (System Time Clock) value, which is a master program clock of a video encoder, and an MMT buffering delay. It may include the step of generating.

A packet receiving apparatus of an MMT system for achieving the above object is an apparatus for receiving a packet in an MPEG Media Transport (MMT) system, the MMT depacket to depacketize the received MMT packet to generate a video stream and an audio stream The MMT depacketization unit includes a video unit, and estimates network jitter using synchronization information generated based on an STC (System Time Clock) value, which is a master program clock of a video encoder, included in the MMT packet, and an MMT buffering delay. And an MMT packet digitizing unit for digitizing the MMT packet.

The packet receiving apparatus includes a video buffer for storing the video stream; A video decoder for decoding the video stream; And a reorder buffer for reordering the decoded video picture for display.

The MMT packet digitizing unit calculates a difference value between an estimated arrival time of the MMT packet and an actual arrival time using MCR clock reference (MCR) time stamp information among the synchronization information and the STC value; A network jitter estimator for estimating the size of the network jitter based on the difference value; And a digitizer configured to perform synchronization by digitizing the MMT packet using the estimated size of network jitter.

The MMT packet digitizing unit may perform a digitizing operation before the encoded MCR time stamp information is received.

The MMT packet digitizing unit may extract the MCR time stamp information from the header of the D.1 layer of the received MMT packet.

The MMT depacketizer may generate information on whether the video picture is an I picture, a P picture, or a B picture by comparing PTS (Presentation Time Stamp) information included in the MMT packet with the STC value.

When the video picture includes only an I picture or a P picture, the MMT depacketizer may parse the PTS information to generate time information when the video stream is output from the video buffer and decoded.

When the B picture is included in the video picture, the MMT depacketizer may parse the PTS information to generate time information that is output from the reordering buffer to the decoded video picture.

The MMT depacketizer may parse the PTS information from the E.3 layer of the received MMT packet.

A packet receiving method of an MMT system for achieving the above object is a method of receiving a packet in an MPEG Media Transport (MMT) system, the MMT depacket to depacketize the received MMT packet to generate a video stream and an audio stream The MMT depacketizing step may include an MMT packet digitizing step of digitizing the MMT packet by estimating network jitter using MCR information included in the MMT packet.

The MMT depacketizing step includes generating at least one of time information at which the video stream is decoded and time information at which the decoded video picture is displayed using PTS (Presentation Time Stamp) information included in the MMT packet. It may include.

According to the packet transmission apparatus and method, and the packet reception apparatus and method in the MMT system of the present invention, as well as synchronization and packet arrival jitter measurement of various media sources, the MMT transmission apparatus / receiver STC clock locking is achieved. It is effective.

In addition, according to the packet transmission apparatus and method, and the packet reception apparatus and method in the MMT system of the present invention, it is possible to efficiently perform the synchronization of packets transmitted through the MPEG-2 system or RTP without violating the conventional standard. It works.

1A is a conceptual diagram illustrating an MMT hierarchical structure;

1B is a conceptual diagram illustrating a transmission delay of a packet transmitter and a packet receiver in an MMT system;

2 is a block diagram schematically showing the configuration of a packet transmission apparatus in an MMT system according to an embodiment of the present invention;

3 is a conceptual diagram illustrating encapsulation of an MPEG TS packet into a D layer payload of an MMT;

4 illustrates an MMT packet including an MPEG TS packet having MCR information in a TS packet header;

5 is a diagram illustrating MCR time stamp information existing in a D layer header;

6 is a block diagram schematically showing the configuration of a packet receiving apparatus in an MMT system according to an embodiment of the present invention;

7 is a detailed block diagram illustrating an MMT depacketization unit of a packet receiving apparatus in an MMT system according to an embodiment of the present invention;

8 is a detailed block diagram illustrating in detail an MMT packet digitizing unit of a packet receiving apparatus in an MMT system according to an embodiment of the present invention;

9 is a view for explaining an interlocking relationship between an MMT depacketizer and a receiver buffer model for MPEG-2 TS packets in a packet receiving apparatus in an MMT system according to another embodiment of the present invention;

10 is a flowchart illustrating a method of processing synchronization according to a video picture by parsing MCR information in an MMT depacketizer of a packet receiving apparatus in an MMT system according to an embodiment of the present invention.

As the present invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description.

However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes a combination of a plurality of related items or any item of a plurality of related items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. In the following description of the present invention, the same reference numerals are used for the same elements in the drawings and redundant descriptions of the same elements will be omitted.

MMT hierarchy

1A is a conceptual diagram illustrating an MMT hierarchical structure.

Referring to FIG. 1A, the MMT layer includes an encapsulation layer, a delivery layer, and a functional area of the S layer. The MMT layer operates on a transport layer.

The encapsulation layer (E-layer) may be responsible for, for example, packetization, fragmentation, synchronization, multiplexing, and the like of transmitted media.

Encapsulation layer (E-layer), as shown in Figure 1a, MMT E.1 Layer (MMT E.1 Layer), MMT E.2 Layer (MMT E.2 Layer) and MMT E.3 Layer (MMT) E.3 Layer).

The E.3 layer encapsulates a Media Fragment Unit (MFU) provided from the Media Codec (A) layer to create an M-Unit.

The MFU may have a format, independent of any particular codec, that can carry data units that can be consumed independently in the media decoder. The MFU can be, for example, a picture or slice of the video.

The M-unit may consist of one or a plurality of MFUs and may have a format, independent of a particular codec, that may carry one or a plurality of access units.

The E.2 layer encapsulates the M-units created in the E.3 layer to create an MMT asset.

An MMT asset is a data entity composed of one or a plurality of M-units from a single data source and is a data unit in which composition information and transport characteristics are defined. MMT assets can correspond to packetized elementary streams (PES), for example video, audio, program information, MPEG-U widgets, JPEG images, MPEG 4 file format, M2TS (MPEG transport stream), etc.

The E.1 layer creates an MMT package by encapsulating the MMT asset generated in the E.2 layer.

The MMT package may be composed of one or more MMT assets together with additional information such as composition information and transport characteristics. Composition information includes information about a relationship between MMT assets, and when one content consists of a plurality of MMT packages, it indicates a relationship between a plurality of MMT packages. It may further include information. The transport characteristics may include transmission characteristic information necessary for determining a delivery condition of an MMT asset or an MMT packet, and may include, for example, a traffic description parameter and a QoS descriptor. ) May be included. The MMT package may correspond to a program of MPEG-2 TS.

The delivery layer may perform, for example, network flow multiplexing, network packetization, and QoS control of media transmitted through a network.

The transport layer (D-layer) is, as shown in Figure 1a, MMT D.1 Layer (MMT D.1 Layer), MMT D.2 Layer (MMT D.2 Layer) and MMT D.3 Layer (MMT) D.3 Layer).

The D.1 layer receives the MMT package generated in the E.1 layer and generates an MMT payload format. The MMT payload format is a payload format for carrying MMT assets and for transmitting information for consumption by the MMT application protocol or other existing application transport protocol such as RTP. The MMT payload may include a fragment of the MFU along with information such as AL-FEC.

The D.2 layer receives the MMT payload format generated in the D.1 layer and generates an MMT transport packet or an MMT packet. The MMT transport packet or MMT packet is a data format used in an application transport protocol for MMT.

D.3 layer (D.3-layer) supports QoS by providing the ability to exchange information between layers by cross-layer design. For example, the D.3 layer may perform QoS control using QoS parameters of the MAC / PHY layer.

The S layer performs a signaling function. For example, signaling functions for session initialization / control / management of transmitted media, server-based and / or client-based trick modes, service discovery, synchronization, etc. Can be done.

As shown in FIG. 1A, the S layer may be configured of an MMT S.1 layer and an MMT S.2 layer.

S.1 layer includes service discovery, media session initialization / termination of media, media session presentation / control of media, delivery (D) layer and encapsulation (E). The interface function with the layer can be performed. The S.1 layer may define the format of control messages between applications for media presentation session management.

The S.2 layer is responsible for flow control, delivery session management, delivery session monitoring, error control, and hybrid network synchronization control. It is possible to define the format of the control message exchanged between delivery end-points of the D-layer.

The S.2 layer supports delivery session establishment and release, delivery session monitoring, flow control, error control, resource scheduling for established delivery sessions, and synchronization in a complex delivery environment to support the behavior of the delivery layer. Signaling for adaptive delivery, and signaling for adaptive delivery. Required signaling may be provided between a sender and a receiver. That is, the S.2 layer may provide signaling required between the sender and the receiver in order to support the operation of the transport layer as described above. In addition, the S.2 layer may be responsible for interfacing with the transport layer and the encapsulation layer.

Maintaining a timing relationship between packets of a single media stream or different media streams is an essential element in MPEG Media Transport (MMT) systems. The MMT system re-adjusts the timing relationship for ensuring synchronization time between MMT packets, and has synchronization functions and de-jittering algorithms. The timing model for the MMT system is an extended concept from any of the MPEG system standards, which utilizes a network de-jitter estimation function similar to a real time transport protocol (RTP) based streaming system.

Delay in MMT Systems

1B is a conceptual diagram illustrating a transmission delay of a packet transmitter and a packet receiver in an MMT system. As shown in FIG. 1B, the packet transmission apparatus and the packet reception apparatus according to an embodiment of the present invention transmit and receive packets with a constant data delay.

Referring to FIG. 1B, the MMT system may include a transmitter 10, an IP network 20, and a receiver 30. The transmitter 10 generates an MMT packet by packetizing the input picture, and transmits the packet to the receiver 30. IP network 20 may be a wireless network.

At this time, the transmission device 10 buffers the video stream encoded by the video encoder in order to synchronize with the audio stream. Therefore, in the transmission device 10, a delay D _{S, i} occurs due to buffering. In addition, the delay D _{N, i} may occur on the transmission path in the IP network 20. After receiving the MMT packet, the receiving device 30 must also buffer until the received packet is decoded. If the decoded data includes the B pictures, the receiving device 30 needs to reorder the buffers, so the buffering is necessary. D _{R, i} ). Due to these delays, the synchronization between the audio data and the video data may be a problem in a service broadcast in real time.

Therefore, according to an embodiment of the present invention, the delay D _{S, i} in the transmitting apparatus 10, the delay D _{N, i} in the network 20 and the receiving apparatus 30 as described above. The synchronization problem can be solved by integrating the delay (D _{R, i} ) to obtain the total delay (D _{T, i} ) and determining the representation information time at which the received image is displayed in consideration of the obtained delay (D _{T, i} ). have. Therefore, the transmitting device 10 of the present invention transmits the packet in a form including the above synchronization parameter, and when the receiving device 30 receives the packet, it extracts the synchronization parameter to synchronize the decoding to perform decoding. Can be.

MMT packet transmitter

2 is a block diagram schematically illustrating a configuration of a packet transmission apparatus 10 in an MMT system according to an embodiment of the present invention. The packet transmission apparatus 10 according to an embodiment of the present invention includes an audio encoder 110, an audio buffer 120, a video encoder 130, a video buffer 140, an MMT packetizer 150, and an STC 160. System Time Clock).

Referring to FIG. 2, the audio encoder 110 encodes an audio block of content to be transmitted to the receiving device 30 to generate an audio stream. The audio buffer 120 stores the encoded audio stream until it is packetized by the MMT packetizer 150. In the service transmitted in real time, the audio buffer 120 may receive audio data output from the audio encoder 110 invariably as an input and output the same at a constant rate. By doing so, the packetization with the video stream inputted at a constant rate by the MMT packetizer 150 can be induced to be easier.

The video encoder 130 receives a picture as an input and performs encoding to generate a video stream. The video buffer 140 stores the encoded video stream and transmits the encoded video stream to the MMT packetizer 150.

The MMT packetizer 150 generates an MMT packet by packetizing a video stream and an audio stream corresponding to the video stream. At this time, synchronization information may be inserted into the MMT packet.

The STC 160 belongs to a specific program and is the master clock of the video encoder 130 and the audio encoder 110 for that program. That is, at the input of the

encoders

110 and 130, the generation time of the input picture or the audio block is sampled and recorded by the STC 160. Thus, each picture or audio block obtains timing information t _i . The STC 160 may be different depending on the program.

Referring to a process in which an input picture becomes an MMT packet and is transmitted to the receiving device 30, the input picture is input to the video encoder 130 at a constant rate, and the video encoder 130 encodes the input picture. Since the time required may vary depending on the picture, the encoded video stream is output at various rates. Accordingly, the video buffer 140 stores video streams output at various rates and outputs them at a constant rate. The video stream output at a constant rate through the video buffer 140 is transmitted to the MMT packetizer 150. The MMT packetizer 150 generates an MMT packet by packetizing the video stream output at a constant rate and the audio stream output through the audio buffer 120, and then receives the received device 30 through the IP network 20. To send. The reception device 30 may depacketize the MMT packet and then decode the MMT packet to display the picture.

The MMT system model may assume that the delay occurs only in the

buffers

120 and 140 when it is assumed that the delay time at the decoder is 0 and that the delay from the

encoders

110 and 130 is also 0. . Thus, the MMT system model accepts the fact that zero-delay elements, such as transmission path delays, can have a constant delay without disrupting operation. The decoder may be synchronized with the

encoders

110 and 130 by time stamps.

The transmission device 10 may include a master oscillator, a counter, and the STC 160. The transmission device 10 may generate a presentation time stamp (PTS) by adding a predetermined amount of delay equal to the delay of the encoder and the decoder to the STC value for the picture or audio block. The video encoder 130 transmits the MMT buffering delay, that is, the buffering delay in the transmitting apparatus 10 and the buffering delay information in the receiving apparatus 30 to the MMT packetizer 150. The MMT packetizer 150 may generate a PTS by calculating the MMT buffering delay with a time stamp of an input picture based on the STC 160. In some cases, assuming an ideal model, the operation time of the MMT packetization process and the depacket process may be assumed to be zero. The MMT packetizer 150 inserts the PTS information into the E.3 layer of the MMT packet representing the picture or audio block. Decoding Time Stamp (DTS) and PTS are the same except for the case of changing the order of B-pictures. This is because the I picture or the P picture is not changed in order, and therefore the time to be decoded is the time to be displayed. Thus, if it consists only of I or P pictures, the PTS can be used to indicate the time at which the access device is output from the buffer of the receiving device 30, enters the decoder and decodes. If a B picture is present in the video stream, the PTS related information may be used to indicate the time output from the reordering buffer for display by the display device of the related picture.

Moreover, the output from video buffer 140, i.e., the video stream, includes a time stamped time with an STC value, which is referred to as MMT Clock Reference (MCR). The MCR time stamp may be inserted in the D.1 layer of the packetization process of the D.1 MMT packet. The MCR time stamp may be required to be generated at maximum 100 ms interval in MPEG-2 TS. Thus, all video and audio streams included in the program have time stamped values from the common STC 160, to achieve synchronization of the video and audio decoders. An important characteristic of the system of MCR time stamps is that the data rate and packet rate on the channel (at the output of the transmitting device 10) can be completely asynchronous with the STC 160, and still the STC 160 can still be synchronized at the decoder. will be. It may also be sent in the multiplexed MMT packet while different programs with different STCs 160 allow recovery of the STC 160 for each program.

MPEG-2 TS is a standard format for storing and transmitting audio, video and data, and can be used in broadcasting systems such as DVB and ATSC. According to the requirements of the MMT, the MMT may support streaming of pre-stored content. In order to transmit an MPEG-2 TS packet using an IP network-based MMT system, the MMT packet must provide timing information for calculating network jitter. MPEG-2 TS was originally developed to be used for digital broadcasting services, and TS packets can be transmitted based on a constant circuit switching network with a relatively short transmission delay compared to an IP-based packet switched network.

In order to process the delay in the TS transmission system, a transmission delay may be processed by using a timing buffer model of the TS in a T-STD. However, MMT using an IP-based packet-switched network may require a redesign of the buffer model because the T-STD cannot handle reliable transmissions due to variations in arrival time and jitter due to packet delays. have.

FIG. 3 is a conceptual diagram illustrating encapsulation of the MPEG TS packet 300 in the D layer payload of the MMT. As shown in FIG. 3, 16 MPEG TS packets 300 are shown, and 7 MPEG TS packets 300 may be packetized into one

MMT packet

310 or 320, and one MMT. The

header

312 and 322 data may be attached to the

payloads

314 and 324 of the packet and transmitted.

Referring to FIG. 3, the TS packet 300 mapped to the

D layer payloads

314 and 324 may include an MCR in a TS packet header field included in the

D layer payloads

314 and 324 as described above. May contain information. As described above, the MMT packetizer 150 generates MCR information based on the MMT buffering delay, the network delay, and the STC value received from the video encoder 130, and the

packet payloads

314 and 324 of the D layer are generated. It is included in the TS packet header field included in the packetized packet so that the decoder can easily extract synchronization information. The

packet headers

312 and 322 of the D layer may include MCR time stamp information.

4 is a diagram illustrating an

MMT packet

400, 420 including an MPEG TS packet having

MCR information

410, 434, 440 in a TS packet header. As shown in FIG. 4, there are n th MMT packet 400 and (n + 1) th MMT packet 420, and each

MMT packet

400, 420 includes an IP /

UDP header

402, 422 and TS packet including

D layer header

404, 424 and

TS packet header

406, 426, 430, 436 and

TS packet payload

408, 428, 432, 438.

Referring to FIG. 4, as shown in the first MMT packet 400, it is not necessarily guaranteed that the MCR information 410 is located in the packet header 406 of the first TS packet in the D layer payload. That is, the MCR information 410 is located in the packet header 406 of the first TS packet only in the first MMT packet 400, and in the case of the second MMT packet 420, the packet header 426 of the first TS packet 400 is located in the packet header 426 of the first TS packet. There is no MCR information, and the

MCR information

434 and 440 are included in the second packet header 420 to the seventh TS packet header 436. Therefore, only the MCR value present in the TS packet cannot calculate the accurate jitter value in the MMT service scenario. Accordingly, there is a need for a new reference clock shared by both the MMT transmitting device 10 and the receiving device 30, such as MCR time stamp information.

5 is a diagram illustrating MCR time stamp information 502 existing in the D layer header 500. As shown in FIG. 5, MCR time stamp information 502 is included in the header 500 of the D layer of the MMT packet.

Referring to FIG. 5, the MCR time stamp information is included in a sampling instance clock value of the first byte in the D layer payload. According to one embodiment of the invention, this value may be used for the purpose of system clock locking between the transmitting device 10 and the receiving device 30 as well as jitter estimation and round trip time (RTT) calculation.

MMT Packet Receiver

6 is a block diagram schematically illustrating a configuration of a packet receiving apparatus 600 in an MMT system according to an embodiment of the present invention. As shown in FIG. 6, the MMT packet receiving apparatus 600 according to an embodiment of the present invention may include an MMT depacketizer 610, an audio buffer 620, an audio decoder 630, a video buffer 640, Video decoder 650 and reorder buffer 660.

Referring to FIG. 6, the MMT packet receiving apparatus 600 receives an MMT packet transmitted from the transmitting apparatus 690 through the IP network 680, depackets the received MMT packet into an audio block and a video picture. And the synchronization information between the two is extracted and displayed in a synchronized state.

First, the MMT depacketizer 610 depackets an MMT packet to generate a video stream and an audio stream. The MMT packet depacketizer 610 may estimate network jitter using the MCR information included in the D.1 layer of the MMT packet and the STC 607 (S602). Digitization may be performed based on the estimated network jitter to synchronize synchronization between MMT packets. In addition, by comparing the PTS included in the E.3 layer of the MMT packet with the STC 607 (S604), synchronization information necessary for decoding in the video decoder 650 may be provided. Through the comparison (S604), it may be determined whether the video picture decoded is an I picture, a P picture, or a B picture. In addition, when a B picture is included, synchronization information is provided to the reordering buffer 660 to change the order so that pictures may be sequentially output.

The audio buffer 620 stores the audio stream and outputs it to the audio decoder 630 at a constant rate. This is to store the audio stream in front of the decoder 630 in order to keep the input to the decoder 630 in the service provided in real time.

The audio decoder 630 generates an audio block by receiving an audio stream from the audio buffer 620 and performing decoding.

The video buffer 640, like the audio buffer 620, stores a video stream. The stored video stream is output to the audio decoder 630 at a constant rate.

The video decoder 650 receives the video stream from the video buffer 640 and performs decoding to generate a video picture. The video decoder 650 may perform decoding in consideration of the decoding time of the picture by providing synchronization information calculated by comparing the PTS and the STC 607 from the MMT depacketizer 610. In particular, unless the B picture is included, since the PTS is the same as the DTS, decoding may be performed based on the PTS. That is, the PTS is used to indicate the time that should be output from the video buffer 640 and decoded. However, when the B picture exists, the PTS related information may be used to indicate the time displayed by the reordering buffer when the display device of the image is displayed.

The reorder buffer 660 rearranges and displays the video picture generated by decoding from the video decoder 650. For video pictures consisting only of I pictures or B pictures, the reorder buffer 660 does not perform a special function. Because there is no room for change. However, when the B picture is included, since the B picture needs to be changed in order, the B picture needs to be changed in order with the previous picture, and such an operation is performed through the reorder buffer 660. The reordering buffer 660 extracts the display time of the image using the PTS information on the video picture identified as the B picture by comparing the PTS and the STC 607 in the MMT depacketizer 610 (S604). The picture may be output at the display time.

According to the requirements of the MMT system, the MMT system supports elasticity to packet arrival jitter and adjusts end-to-end delay in order to perform continuous decoding and presentation of content under various network conditions. This goal can be achieved in RTP-based systems by supporting a cooperative manner of RTP time stamps and NTP time stamps. With such a time stamp, it is possible to estimate network jitter, packet loss rate, and packet round trip time, which are essential information to protect the system in operation against the network change.

7 is a detailed block diagram illustrating an MMT depacketizer 610 of a packet receiving apparatus in an MMT system according to an embodiment of the present invention. As shown in FIG. 7, the MMT depacketizer 610 according to an embodiment of the present invention may include an MMT packet digitizer 710 and a depacketizer 720.

Referring to FIG. 7, the MMT digitizing unit 710 estimates network jitter occurring during MMT packet transmission using MCR time stamp information. The MMT digitizing unit 710 may calculate the time difference between the expected arrival time and the actual arrival time of the MMT packet with the MCR time stamp information and the local STC clock. This time difference can be used to estimate the actual value of network jitter.

The depacketizer 720 depackets the digitized packet through the MMT packet digitizer 710 to estimate the network jitter, and divides the packet into an audio stream and a video stream. The divided video stream is sent to the video buffer, and the divided audio stream is sent to the audio buffer.

8 is a detailed block diagram illustrating in detail the MMT packet digitizing unit 710 of the packet receiving apparatus in the MMT system according to an embodiment of the present invention. As illustrated in FIG. 8, the MMT packet digitizer 710 may include a digital buffer 810, an extractor 820, a difference value calculator 830, a network jitter estimator 840, and a digital digitizer ( 850).

Referring to FIG. 8, the digit buffer 810 stores the received MMT packet until digitization of the MMT packet is performed to restore the temporal relationship between the MMT packets at the output. The size of the digital buffer 810 may be calculated using MCR information.

The extractor 820 parses the D.1 layer of the MMT packet stored in the digital buffer 810 and extracts MCR time stamp information included therein. As described above, the D.1 layer of the MMT packet packetized in the transmitting apparatus includes the MCR time stamp information of which the video stream is time stamped based on the STC. MCR time stamp information is extracted from the.

The difference calculator 830 calculates a difference value between the estimated arrival time of the MMT packet and the actual arrival time by using the STC value and the MCR time stamp information extracted by the extractor 820. Since the MCR time stamp information is a time stamped value based on the STC value, the MCR time stamp information includes information on the time of packetization, and thus, the estimated time of arrival of the MMT packet may be calculated. By calculating the difference between the calculated estimated arrival time and the actual arrival time of the MMT packet, the time difference value can be calculated.

The network jitter estimator 840 estimates the size of the network jitter based on the time difference calculated by the difference calculator 830. If the time difference is large, the size of network jitter also increases proportionally, and thus the size of jitter can be estimated using a proportional relationship.

The digitizer 850 digitizes the MMT packet using the size information of the network jitter estimated by the network jitter estimator 840. That is, the delay is offset by the amount of network jitter so that synchronization between MMT packets can be achieved. Accordingly, the MMT packet passing through the digitizing unit 850 is a digitized packet and is transmitted to the depacketizer 720 because synchronization between the MMT packets is correct.

Here, according to an embodiment of the present invention, in order to ensure the stable recovery of the STC, the MMT digitizing unit 710 may be preferably performed before the MCR encoded encoding is transferred to the MMT receiving apparatus. .

Through this, the MMT digitizer 710 may remove jitter generated by the IP network, and may forward the digitized MMT packet to the depacketizer 720.

9 is a diagram illustrating an interworking relationship between an MMT depacketizer and a receiver buffer model for MPEG-2 TS packets in a packet receiving apparatus in an MMT system according to another embodiment of the present invention. As shown in FIG. 9, the received MMT packet is digitized through the digitizer 910, depacketized into MPEG-2 TS packets through the depacketizer 920, and MPEG-2 T-STD. The model 930 can complete the delay process.

Referring to FIG. 9, an MMT packet may include a plurality of MPEG-2 TS packets in a payload, and as described above, an MPEG-2 TS packet is developed to be used for a digital broadcasting service and is generally based on IP. Compared to a packet switched network, the transmission delay is relatively short and can be transmitted based on a constant circuit switched network.

Accordingly, by converting the MMT packet into the MPEG-2 TS packet, the MPEG-2 T-STD model 930 which is in charge of delay processing of the MPEG-2 TS packet can be utilized. This improves backward compatibility.

The digitizing unit 910 calculates network jitter based on MCR time stamp information in the same manner as the MMT digitizing unit 710 described above, calculates an appropriate size of the digital buffer and generates a digital MMT packet through the same. .

The depacketizer 920 simply converts an MMT packet into an MPEG-2 TS packet. The packet passing through the depacketizer 920 becomes a digitized MPEG-2 TS packet.

The MPEG-2 T-STD model 930 basically has a structure for processing a transmission delay using a TS timing buffer model and may perform delay processing on a TS packet.

Referring to FIG. 10, the MMT depacketizer 610 depackets an MMT packet and extracts a PTS included in the E.3 layer of the MMT packet (S1010). Then, by comparing the extracted PTS information and the STC value (S1020), it is possible to provide synchronization information necessary for decoding in the video decoder. On the basis of the comparison result, it is determined whether the B picture exists in the decoded video picture (S1030). That is, it is possible to determine whether the picture is an I picture, a P picture, or a B picture.

If the B picture is included, the display time is determined by parsing the PTS information (S1040). Then, the display time is transmitted to the reordering buffer (S1050), and the pictures are sequentially output. The reordering buffer enables the corresponding picture to be output at the display time based on the display time received for the video picture identified as the B picture.

When the B picture is not included, that is, when the video picture is composed of only I and P pictures, the decoding time is determined based on the PTS information (S1060). Then, the time information is transmitted to the video decoder (S1070). Since the PTS information is the same as the DTS, the video decoder receiving the time information may perform decoding based on the PTS. That is, the PTS is used to indicate the time that should be output from the video buffer and decoded through the video decoder. The video decoder may perform decoding in consideration of the decoding time of the picture based on the synchronization information received from the MMT depacketizer.

Although described with reference to the embodiments above, those skilled in the art will understand that the present invention can be variously modified and changed without departing from the spirit and scope of the invention as set forth in the claims below. Could be.

Claims

In the apparatus for transmitting a packet in a MPEG Media Transport (MMT) system,

An MMT packetizer for generating an MMT packet by packetizing an encoded video stream based on an input picture and an audio stream corresponding to the video stream,

In the MMT system, the MMT packetizer generates the MMT packet including synchronization information generated based on an STC (System Time Clock) and an MMT buffering delay, which are master program clocks of a video and audio encoder. Packet transmission device.
The method of claim 1,

A video encoder for encoding the input picture to produce a video stream;

A video buffer that stores the encoded video stream; And

Further comprising a transmitter for transmitting the packetized MMT packet,

The video buffer is a packet transmission apparatus in the MMT system, characterized in that for outputting the video stream at a constant rate (rate).
The method of claim 1,

The MMT packetizing unit packetizes the video stream output from the video buffer including MCR clock reference (MCR) time stamp information generated by time stamping based on the STC.
The method of claim 3, wherein

The packetization unit controls at least one of a data rate and a packet rate in a channel for transmitting the MMT packet using the MCR time stamp information.
The method of claim 3, wherein

The MMT packetizer inserts the MCR time stamping information into the D.1 layer of the MMT packet.
The method of claim 1,

The MMT packetization unit generates a PTS (Presentation Time Stamp) generated by adding a sum of a buffering delay at the transmitting apparatus, a buffering delay at the receiving apparatus, and a network delay to a time stamp value of the input picture time stamped using the STC Packet transmission apparatus according to claim 1, wherein the MMT packet is generated including information.
The method of claim 6,

The MMT packetizer inserts the PTS information into the E.3 layer of the MMT packet.
The method of claim 1,

The STC value is program dependent,

The MMT packetization unit is a packet transmission apparatus in the MMT system, characterized in that for multiplexing different programs having different STC.
In the method for transmitting a packet in the MMT system,

MMT packetization step of packetizing the encoded video stream based on the input picture and the audio stream corresponding to the video stream to generate an MMT packet,

The MMT packetization step includes generating the MMT packet including synchronization information generated based on an STC (System Time Clock) value, which is a master program clock of a video encoder, and an MMT buffering delay. Packet transmission method in the MMT system.
An apparatus for receiving a packet in an MPEG Media Transport (MMT) system,

An MMT depacketizer for generating a video stream and an audio stream by depacketizing the received MMT packet,

The MMT depacketization unit estimates network jitter using synchronization information generated based on an STC (System Time Clock) value, which is a master program clock of a video encoder, and an MMT buffering delay, included in the MMT packet, to decode the MMT packet. Packet receiving apparatus in the MMT system, characterized in that it comprises a MMT packet digitizing unit for digitizing (dejittering).
The method of claim 10,

A video buffer for storing the video stream;

A video decoder for decoding the video stream; And

And a reorder buffer for reordering the decoded video picture for display.
The method of claim 10, wherein the MMT packet digitizing unit

A difference value calculator configured to calculate a difference value between an estimated arrival time of the MMT packet and an actual arrival time by using MCR clock reference (MCR) time stamp information among the synchronization information and the STC value;

A network jitter estimator for estimating the size of the network jitter based on the difference value; And

And a digitizer configured to perform synchronization by digitizing the MMT packet using the estimated size of the network jitter.
The method of claim 10,

The MMT packet digitizing unit performs a digitizing operation before the encoded MCR time stamp information is received.
The method of claim 10,

The MMT packet digitizing unit extracts the MCR time stamp information from the header of the D.1 layer of the received MMT packet.
The method of claim 11,

The MMT depacketizer generates information on whether the video picture is an I picture, a P picture, or a B picture by comparing PTS (Presentation Time Stamp) information included in the MMT packet with the STC value. Packet receiving apparatus in MMT system.
The method of claim 15,

In the MMT system, when the video picture is composed only of I pictures or P pictures, the MMT depacketizer parses the PTS information to generate time information from which the video stream is output from the video buffer and decoded. Packet receiving device.
The method of claim 15,

In the MMT system, when the B picture is included in the video picture, the MMT depacketizer parses the PTS information to generate time information that is output from the reordering buffer to the decoded video picture. Packet receiving device.
The method of claim 15,

The MMT depacketization unit parses the PTS information from the E.3 layer of the received MMT packet.
In the method for receiving a packet in an MPEG Media Transport (MMT) system,

And depacketizing the received MMT packet to generate a video stream and an audio stream.

The MMT depacketizing step includes an MMT packet digitizing step of digitizing the MMT packet by estimating network jitter using MCR information included in the MMT packet. Receiving method.
The method of claim 19,

The MMT depacketizing step includes generating at least one of time information at which the video stream is decoded and time information at which the decoded video picture is displayed using PTS (Presentation Time Stamp) information included in the MMT packet. Packet receiving method in the MMT system, characterized in that it comprises a.