KR20130007993A - Method of transmitting inter-dependent frames based on allocating additional time informaion - Google Patents

Abstract

PURPOSE: A transmission method of a media stream including an interdependence frame is provided to suitably schedule transmission without having advance information for a dependency structure by additionally providing screen display time information. CONSTITUTION: Frames including PTS(Presentation Time Stamp) of a current frame are allocated through an earliest dependency PTS in order to decode the frames(S410). The allocated DPTS is transmitted to a receiver by binding the frames(S420). [Reference numerals] (AA) Start; (BB) End; (S410) Allocating PTS which requires a current frame for decoding frames including earlier time than the current frame to DPTS of the current frame; (S420) Transmitting DPTS to a reception end by binding frames with the DPTS

Description

A method of transmitting a media stream having interdependent frames using allocation of transmission time information for each frame {METHOD OF TRANSMITTING INTER-DEPENDENT FRAMES BASED ON ALLOCATING ADDITIONAL TIME INFORMAION}

The present invention relates to a method for transmitting a media stream, and more particularly, to a method for transmitting a media stream having an interdependent frame using allocation of transmission time information for each frame.

Recent video codecs may have B-frames, so that the order of frames displayed on the screen and the order of transmitted frames may be designed differently. In particular, the hierarchical B-frame structure employed after H.264 has a data dependency relationship between complex frames and can determine the order of transmission according to the interframe dependency. Therefore, if the streaming system does not fully grasp the inter-dependency structure between each frame, it is very difficult for the receiver to know the transmission timing of each frame to be meaningful.

An object of the present invention for solving the above problems is to provide the presentation time information of additional media data for each frame during media streaming, so that transmission scheduling can be appropriately performed without prior information on the codec or inter-frame dependency structure. The present invention provides a method for transmitting a media stream having an interdependent frame using allocation of transmission time information for each frame.

In order to achieve the above object of the present invention, a method of transmitting a media stream having an interdependent frame using frame-by-frame transmission time information allocation according to an embodiment of the present invention has an earlier time than a PTS (Presentation Time Stamp) of the current frame. Allocating a PTS of the earliest frame that requires the current frame to decode among frames having a PTS as an Earliest Dependency PTS (DPTS) of the current frame; And transmitting the same DPTS to the receiving end in a bundle together with the same frames.

According to the transmission method of the media stream having the interdependence frame using the above-mentioned allocation of transmission time information for each frame, the information on the codec or inter-frame dependency structure is provided by providing presentation time information of additional media data for each frame. Transmission scheduling can be properly performed without prior information. Therefore, more robust transmission to network jitter may be performed, and transmission scheduling may be performed so that even if the packet loss of the network is experienced, the ripple effect in actual image quality is relatively small.

1 is a conceptual diagram illustrating an MMT hierarchical structure.
2 shows the structure of a typical B-frame.
3 illustrates a structure of a hierarchical B-frame used in an H.264 SVC Extension or a High Efficiency Video Codec (HEVC).
4 is a flowchart illustrating a method of transmitting a media stream having an interdependent frame using allocation of transmission time information for each frame according to an embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail.

It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate the understanding of the present invention, the same reference numerals are used for the same constituent elements in the drawings and redundant explanations for the same constituent elements are omitted.

MMT  Hierarchy

1 is a conceptual diagram illustrating an MMT hierarchical structure.

Referring to FIG. 1, the MMT layer includes an encapsulation layer, a delivery layer, and a functional area of an S layer. The MMT layer operates on a transport layer.

The encapsulation layer (E-layer) can perform functions such as packetization, fragmentation, synchronization, and multiplexing of the transmitted media, for example.

The encapsulation layer (E-layer) includes an MMT E.1 layer (MMT E.1 Layer), an MMT E.2 layer (MMT E.2 Layer), and an MMT E.3 layer E.3 Layer).

The E.3 layer encapsulates a Media Fragment Unit (MFU) provided from the Media Codec (A) layer to create an M-Unit.

The MFU may have a format, independent of any particular codec, that can carry data units that can be consumed independently in the media decoder. The MFU can be, for example, a picture or slice of the video.

The M-unit may consist of one or a plurality of MFUs and may have a format, independent of a particular codec, that may carry one or a plurality of access units.

The E.2 layer encapsulates the M-units created in the E.3 layer to create an MMT asset.

An MMT asset is a data entity composed of one or a plurality of M-units from a single data source and is a data unit in which composition information and transport characteristics are defined. MMT assets can correspond to packetized elementary streams (PES), for example video, audio, program information, MPEG-U widgets, JPEG images, MPEG 4 file format, M2TS (MPEG transport stream), etc.

E.1 Layer (E.1 Layer) encapsulates MMT asset created in E.2 layer to create MMT Package (MMT Package).

The MMT package may be composed of one or more MMT assets together with additional information such as composition information and transport characteristics. Composition information includes information about a relationship between MMT assets, and when one content consists of a plurality of MMT packages, it indicates a relationship between a plurality of MMT packages. It may further include information. The transport characteristics may include transmission characteristic information necessary for determining a delivery condition of an MMT asset or an MMT packet, and may include, for example, a traffic description parameter and a QoS descriptor. ) May be included. The MMT package may correspond to a program of MPEG-2 TS.

The delivery layer may perform, for example, network flow multiplexing, network packetization, and QoS control of media transmitted through a network.

The transport layer (D-layer) is, as shown in Figure 1a, MMT D.1 Layer (MMT D.1 Layer), MMT D.2 Layer (MMT D.2 Layer) and MMT D.3 Layer (MMT) D.3 Layer).

The D.1 layer receives the MMT package generated in the E.1 layer and generates an MMT payload format. The MMT payload format is a payload format for carrying MMT assets and for transmitting information for consumption by the MMT application protocol or other existing application transport protocol such as RTP. The MMT payload may contain fragments of MFUs with information such as AL-FEC.

The D.2 layer receives the MMT payload format generated in the D.1 layer and generates an MMT transport packet or an MMT packet. The MMT transport packet or MMT packet is a data format used in an application transport protocol for MMT.

D.3 layer (D.3-layer) supports QoS by providing the function of exchanging information between layers by cross-layer design. For example, the D.3 layer may perform QoS control using QoS parameters of the MAC / PHY layer.

The S layer performs a signaling function. For example, signaling functions for session initialization / control / management of transmitted media, server-based and / or client-based trick modes, service discovery, synchronization, etc. Can be done.

As shown in FIG. 1A, the S layer may be configured of an MMT S.1 layer and an MMT S.2 layer.

The S.1 layer consists of service discovery, media session initialization / termination, media session presentation / control, transport layer (D) and encapsulation (E) And an interface function with the layer. The S.1 layer may define the format of control messages between applications for media presentation session management.

The S.2 layer is responsible for flow control, delivery session management, delivery session monitoring, error control, and hybrid network synchronization control. It is possible to define the format of the control message exchanged between delivery end-points of the D-layer.

The S.2 layer supports delivery session establishment and release, delivery session monitoring, flow control, error control, resource scheduling for established delivery sessions, and synchronization in a complex delivery environment to support the behavior of the delivery layer. Signaling for adaptive delivery, and signaling for adaptive delivery. Required signaling may be provided between a sender and a receiver. That is, the S.2 layer may provide signaling required between the sender and the receiver in order to support the operation of the transport layer as described above. In addition, the S.2 layer may be responsible for interfacing with the transport layer and the encapsulation layer.

Media with interdependent frames Of stream  Transmission way

When the video stream is transmitted so that it can be input to the decoder in the frame order output from the encoder, intermediate buffering can be minimized because there is no need for additional reordering and reorganizing processes. End-to-end delay is minimized. Therefore, in case of streaming, the scenario of transmitting in the decoding order (or the output order of the encoder) as described above is an optimal method in terms of transmission delay, and is generally used most.

As described above, when reordered frames are transmitted in decoding order, a video stream having B-frames has a very complicated frame reordering structure. Only the attached PTS (Presentation Time Stamp) can be used to determine the dependency between the frames, and frame-by-frame transmission scheduling is inevitable.

The IP transport channel is not a channel that guarantees strict CBR, is error-prone, and has relatively high delay-jitter, making it difficult to predict the reception time according to transmission. Channel. Therefore, when transmission scheduling is performed on a frame-by-frame basis, there is a high probability that a frame required in time is not received.

The transmission time information allocation method for each frame according to an embodiment of the present invention may further include an Earliest Dependency PTS (DPTS) for use as a reference from a transmission point of view in addition to the existing PTS. Through the best DPTS, the streaming system can use a more efficient frame transmission scheduling policy by receiving meaningful time information on the decoder side for each frame without prior information on codec or inter-dependency between frames.

Best dependency PTS ( earliest dependency PTS , DPTS )

The earliest dependency PTS (DPTS) refers to the PTS of the earliest frame that requires the current frame for decoding among the frames having the PTS at a time earlier than the PTS of the current frame.

The following examples are the structure of a video sequence having a typical B-frame. FIG. 2 illustrates a structure of a general B-frame, and FIG. 3 illustrates a hierarchical-B frame structure used in an H.264 SVC extension, HEVC, and the like.

As shown in FIG. 2, the frames of the presentation order are output in the actual decoding order and transmitted in the decoding order. The DPTS may write down the PTS of the earliest frame based on the PTS reference that requires each frame, as indicated by each arrow in the stream of the decoding order. For example, in case of frame 2, its own PTS is 2 (x3000), but since the best earliest frame that requires itself is frame 1, the DPTS of frame 2 is 1 (x3000), which is the PTS of frame 1. Write down.

3 shows an example in a hierarchical B-frame structure. As in the example of a normal B-frame, the stream is transmitted in a decoding order below. DPTS writes the PTS of the earliest frame in terms of the PTS that requires each frame, as indicated by each arrow in the stream in decoding order. In this case, frames 8 to 2 have their own PTSs of 8 (x3000) to 2 (x3000), but since the best earliest frame that needs them is frame 1, the DPTS for each frame is 1 One (x3000), which is the PTS of the frame, may be written.

In the example of the dependent frames, the DPTS always has a monotonically increasing value even when reordered as shown in FIGS. 2 to 3, and frames having the same DPTS from the system point of view have the same transmission time limit. It is a set of frames with a transmission time constraint.

Therefore, DPTS indicated by AU can be utilized as follows in terms of a transmission system in an IP network, thereby improving performance of transport streaming. The following examples can be presented by way of example through actual invention.

1. Batch sending: Instead of sending each frame at fixed time intervals, if frames with the same DPTS are sent in advance, they are placed at the deepest part of the dependency tree and have no transmission time. For example, the frame 1 of FIG. 3 may be allowed to be transmitted to perform a more robust transmission to network jitter (generally, such a “deepest frame” may be transmitted to a receiver). If the time difference between the arrival time and the time required by the decoder after arrival is short, the transmission delay is long.

2. Priority sending: A frame having a long dependency (e.g., frame 8 of FIG. 3) has a much larger ripple effect on loss than other frames. Therefore, the frame that has a large difference from the original PTS on the basis of DPTS is regarded as a frame having a long dependency, and a sending policy is set to send such a frame more importantly, so that the actual picture quality may be Transmission scheduling can be performed so that the ripple effect in R is relatively small.

4 is a flowchart of a method for batch transmission of a media stream having interdependence frames according to an embodiment of the present invention.

As shown in FIG. 4, in the method for transmitting a media stream having an interdependent frame according to an embodiment of the present invention, DPTS may be allocated to each frame (S410). In more detail, in the media stream having the interdependent frames, the PTS of the frame having the best PTS among the frames requiring the first frame may be allocated as the DPTS of the first frame. That is, the PTS of the earliest frame that requires the current frame to decode among the frames having the PTS earlier than the current PTS of the current frame is allocated as the earliest dependency PTS (DPTS) of the current frame. Can be. As described above with reference to FIG. 2, for example, in the case of frame 2 of FIG. 2, its own PTS is 2 (x3000), but since the earliest frame that requires itself is frame 1, frame 1 is A PTS of 1 (x3000) may be allocated as a DPTS of frame 2.

When the allocation of the DPTS for each frame is completed, the allocated DPTS may transmit the same frames together in a bundle (S420). Therefore, compared to the case of transmitting at fixed time intervals for each frame, a time when a frame (for example, frame 1 of FIG. 3) that is located at the deepest part of the dependency tree and has no transmission time margin can be transmitted. This allows for more robust transmission of network jitter.

Claims (1)

In the method of transmitting a media stream having an interdependent frame,
Among the frames having a PTS earlier than the PTS (Presentation Time Stamp) of the current frame, the earliest dependency PTS (DPTS) of the earliest dependent PTS of the current frame is used as the PTS of the earliest frame requiring the current frame for decoding. Assigning to; And
And transmitting the same DPTS to the receiving end in a bundle together with the allocated DPTS.

Images