KR20130119885A

KR20130119885A - Methods of transporting mmt packet for in-network adaptation of layered video

Info

Publication number: KR20130119885A
Application number: KR1020130045496A
Authority: KR
Inventors: 김창기; 김태정; 유정주; 홍진우; 서광덕
Original assignee: 한국전자통신연구원; 연세대학교 원주산학협력단
Priority date: 2012-04-24
Filing date: 2013-04-24
Publication date: 2013-11-01

Abstract

The present invention provides a structure and extension information of an MMT packet having extension information necessary for adaptively transmitting a multi-layer video such as SVC to a network environment in a media transport service based on an MPEG Media Transport (MPEG Media Transport) system. The present invention provides a transmission method and apparatus for transmitting an MMT packet.

Description

METHODS OF TRANSPORTING MMT PACKET FOR IN-NETWORK ADAPTATION OF LAYERED VIDEO}

The present invention relates to a method of transmitting and receiving video content, and more particularly, to a method of transmitting an MMT packet.

As various terminals consume contents of various resolutions for each terminal and mobile services are available, a contents consumption environment in which a network environment changes in real time is being created. In order to provide high quality content suitable for such a changing real-time consumption environment, there is a limit to the existing video encoding method that supports a single format.

In this regard, studies on scalable video coding (SVC) and multiview video coding (MVC) have been actively conducted. SVC is designed to be adaptable to various transmission environments and terminal performance in real time. That is, it is a video encoding method capable of real-time adaptation that supports a format suitable for a variable heterogeneous consumption environment, such as various resolutions and network conditions required by the terminal.

In addition, H.264 Advanced Video Coding (AVC) / Multi-view Video Coding (MVC) is a coding method for recording a stream capable of displaying a 3D image. In the H.264 AVC / MVC profile standard, an image stream called a base view video and an image stream called a dependent view video are defined. In the following, the H.264 AVC / MVC profile specification is simply referred to as MVC as appropriate.

In the transmission of such SVC or MVC video, it is required to transmit the video data adaptively in consideration of the environment in the network.

An object of the present invention for solving the above problems is to provide a method and apparatus for transmitting an MMT packet to determine whether to deliver the MMT packet without fully parsing the MMT packet.

An MMT packet generation method for achieving the object of the present invention is a method for generating an MMT packet by the MMT packet transmission device for delivering video data, the extension generated using the information of at least one NAL unit included in the MMT packet Generates an MMT packet containing information.

The extension information may be included in a packet header of an MMT packet or a payload header of an MMT payload included in an MMT packet.

The extension information may include a TID (Temporal_ID) parameter generated using a temporal level of the at least one NAL unit, a DID (Dependency_ID) parameter generated using a spatial level of the at least one NAL unit, and the at least one NAL unit. At least one NAL included in the MMT packet or including at least one of a QID (Quality_ID) parameter generated by using a quality level of the PSI and a Priority_ID parameter generated by using priority information of the at least one NAL unit It may include a VID parameter generated using the view information of the unit.

The extension information includes a TID_low parameter indicating the smallest value among the temporal levels of the NAL unit included in the MMT packet, a DID_low parameter indicating the smallest value among the spatial levels of the NAL unit included in the MMT packet and the MMT packet. QID_low parameter representing the smallest value among the quality levels of the NAL unit, PID_low parameter representing the smallest value among the priority information of the NAL unit included in the MMT packet, and the largest value of the temporal level of the NAL unit included in the MMT packet. TID_high parameter indicating a, DID_high parameter indicating the largest value of the spatial level of the NAL unit included in the MMT packet, QID_high parameter indicating the highest value of the quality level of the NAL unit included in the MMT packet and included in the MMT packet Of PID_high parameters representing the largest value among the priority information of the NAL unit A VID_low parameter indicating at least one of view information of NAL units included in at least one or included in the MMT packet and a VID_high parameter indicating a largest value among view information of NAL units included in the MMT packet. It may include at least one of.

The extension information may further include an ML_Type parameter indicating an encoding method of video data included in the MMT packet.

In addition, the MMT packet transmission method for achieving the object of the present invention in the method for transmitting the MMT packet by the video data transmission device, the step of determining whether or not to forward the MMT packet using the extended information included in the MMT packet It includes.

The MMT packet may include an MMT packet header, an MMT payload header, and an MMT payload, and the extension information may be located in the MMT packet header or the MMT payload header.

In the determining of whether the transmission is performed, the transmission device sets a reference value TID_opt of a temporal level greater than or equal to the value of TID_high, a reference spatial value DID_opt of a set spatial level is greater than or equal to a value of DID_high, and a reference value of a set quality level. It may be performed by delivering the MMT packet when QID_opt is greater than or equal to the value of QID_high and the reference value PID_opt of the set priority information is greater than or equal to the value of PID_low.

In the determining of whether the transmission is performed, the transmission apparatus sets a reference value TID_opt of a temporal level smaller than the value of TID_low, sets a reference value DID_opt of a spatial level smaller than a value of DID_low, and sets a reference value QID_opt of a set quality level of QID_low. It may be performed by not transmitting the MMT packet when the reference value PID_opt smaller than the value and the reference value PID_opt of the set priority information is smaller than the value of PID_low.

The extended information further includes an ML_type parameter indicating an encoding method of video data included in the MMT packet, and the transmitting method includes the SVC transmitting video data included in the MMT packet using a value of the ML_type parameter. (Scalable Video Coding) encoding method comprising the step of confirming that the encoding method, and the step of determining whether the delivery device, the reference value TID_opt of the set temporal level is less than the value of the TID_high and greater than the value of the TID_low When the reference value DID_opt of the set spatial level is less than the value of the DID_high and greater than or equal to the value of the DID_low, or the reference value QID_opt of the set quality level is less than the value of the QID_high and greater than or equal to the value of the QID_low Whether to deliver the MMT packet by examining the NAL unit of the MMT packet Determination can be performed by.

The transmitting device is a Media Aware Network Element (MANE), and determining whether to transmit the MMT packet using the data of the MMT packet comprises: arriving at the MMT packet; Parsing the MMT packet header; Obtaining extension information from the MMT packet header; And determining whether to transmit the MMT packet from the extension information.

In addition, MMT packet transmission apparatus for achieving the object of the present invention, MMT packet transmission apparatus for transmitting video data, MMT packet generation unit; And an MMT packet transmitter, wherein the MMT packet generator generates an MMT packet including extension information generated using information of an MMT payload and at least one NAL unit included in the MMT payload.

The MMT packet generator may generate the MMT packet by including the extension information in a packet header of the MMT packet or a payload header of the MMT payload included in the MMT packet.

The extension information may include a TID parameter generated using the temporal level of the at least one NAL unit, a DID parameter generated using the spatial level of the at least one NAL unit, and a quality level of the at least one NAL unit. Include at least one of a generated QID parameter and a PID parameter generated using priority information of the at least one NAL unit, or include a VID parameter generated using view information of the at least one NAL unit. Can be.

In addition, MMT packet transmission apparatus for achieving the object of the present invention, MMT packet transmission apparatus for transmitting video data, including the MMT packet transmission unit for determining whether or not to forward the MMT packet using the extension information of the MMT packet. do.

The MMT packet may include an MMT packet header, an MMT payload header, and an MMT payload, and the extension information may be located in an MMT packet header or a payload header of the MMT packet.

The transmitting device has a reference value TID_opt of a set temporal level greater than or equal to the value of TID_high, a reference value DID_opt of a set spatial level is greater than or equal to a value of DID_high, and a reference value QID_opt of a set quality level is greater than or equal to a value of QID_high. When the reference value PID_opt of the set priority information is greater than or equal to the value of PID_high, the MMT packet may be delivered.

The transmitting device has a reference value TID_opt of a set temporal level smaller than a value of TID_low, a reference value DID_opt of a set spatial level smaller than a value of DID_low, a reference value QID_opt of a set quality level smaller than a value of QID_low, and sets priority information. When the reference value PID_opt is smaller than the value of PID_low, the MMT packet may not be delivered.

The extension information further includes an ML_type parameter indicating an encoding method of video data included in the MMT packet, and the transmitting device uses the value of the ML_type parameter to determine that the video data of the MMT packet is multi-view video. coding) When it is determined that the encoding is generated, the MMT packet may be delivered when the reference value VID_opt of the set view information is greater than or equal to the VID_high.

The transmitting device is a media aware network element (MANE), and the transmitting device further includes a receiving unit for receiving the MMT packet, and the MMT packet transmitting unit is generated by parsing an MMT packet header included in the MMT packet. It may be determined whether the MMT packet is delivered from the information.

In addition, the MMT packet structure for achieving the object of the present invention includes an MMT packet header; And a MMT packet including an MMT payload, wherein the MMT payload includes a payload header, and the MMT packet header or the payload header includes extension information generated using at least one NAL unit. do.

A method and apparatus for transmitting an MMT packet according to an embodiment of the present invention described above are adapted to adapt a multi-layer video such as SVC and MVC to a network environment in a media transport service based on an MPEG Media Transport (MPT) system. Using the structure and extension information of the MMT packet having the extension information necessary for transmission to the MANE, the MANE can quickly determine the MMT packet to be selected or discarded for delivery to the target terminal among the MMT packets input to the MANE. Has

1 is a conceptual diagram illustrating an MMT hierarchical structure according to an embodiment of the present invention.
FIG. 2 shows a format of unit information (or data or packet) used for each layer of the MMT hierarchical structure of FIG.
3 shows an MMT package configuration according to an embodiment of the present invention.
4 illustrates the structure of a MANE-based network adaptive SVC and MVC video transmission system according to an embodiment of the present invention.
5 is an internal structure of an MMT packet according to an embodiment of the present invention.
6 is a structure of payload scalability information according to an embodiment of the present invention.
FIG. 7 is an algorithm for determining whether to forward, discard, and inspect an inner packet of an MMT packet made in a media aware network element (MANE) according to an embodiment of the present invention.
8 is a flowchart illustrating a delivery order of MMT packets of a MANE according to an embodiment of the present invention.
FIG. 9 illustrates an embodiment of an MMT packet header format including extension information for adaptation in a NAME based network according to another embodiment of the present invention.
10 illustrates an ML_type field indicating a state of a type represented by multilayer information.
FIG. 11 illustrates a state in which an MMT packet header format has a format for multilayer information in SVC according to an embodiment of the present invention.
FIG. 12 is an algorithm for determining whether to forward, discard, and inspect an internal packet of an MMT packet including SVC video performed in a MANE according to another embodiment of the present invention.
FIG. 13 illustrates a work flow applied in a MANE including SVC video in another embodiment of the present invention.
14 illustrates a state in which an MMT packet header format has a format for multilayer information in MVC according to another embodiment of the present invention.
FIG. 15 is an algorithm for determining whether to forward, discard, and inspect an inner packet of an MMT packet including MVC video that is performed in a MANE according to another embodiment of the present invention.
FIG. 16 illustrates a work flow applied in a MANE including MVC video in another embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail.

It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate the understanding of the present invention, the same reference numerals are used for the same constituent elements in the drawings and redundant explanations for the same constituent elements are omitted.

Hereinafter, the meaning of the term is defined as follows.

Non-timed data defines all data elements that are consumed without specifying time. Non timed data is data that does not have unique synchronization information for decoding and / or representation of its media unit.

Timed data is data that carries unique synchronization information for decoding and / or representation of its media unit and defines data elements associated with a particular time to be decoded and presented.

An access unit (AU) is the smallest data entity that can have time information.

An MMT asset is a logical data entity consisting of at least one MPU with the same MMT asset ID or a specific chunk of data with a format defined by other standards. An MMT asset is a data entity that contains data with the same transport characteristics.

The MMT Asset Delivery Characteristics (MMT-ADC) is a description of the quality of service (QoS) requirements for transmitting MMT assets. The MMT-ADC is represented by parameters that are independent of the particular delivery environment.

MMT Composition Information (MMT CI) describes the spatial and temporal relationship between MMT assets.

A Media Fragment Unit (MFU) is a general container independent of any particular codec and accommodates encoded media data that can be consumed independently by the media decoder. The AU includes one file for non-timed media data or media data that can be independently decoded by a media decoder, such as all or part of the AU for timed media data. It has a size smaller than or equal to the access unit (AU) and accommodates information that can be used in the transport layer.

The Media Processing Unit is a generic container that is independent of any particular media codec and is a generic container for timed or non-timed data that can be decoded independently. It includes at least one AU or part of non-timed data in timed data along with information related to further delivery and consumption. Processing of the MPU means encapsulation into a package or packetization for delivery. MPU is encoded media data that can be processed completely and independently. However, for scalable video coding (SVC) and multiview video coding (MVC), in some cases the MPU may not be consumed independently and completely in the media codec server.

An MMT entity is an implementation of software or hardware that conforms to an MMT profile.

An MMT packet is a formatted unit of data generated or consumed according to the MMT protocol.

An MMT package is a collection of logically structured data, consisting of at least one MMT asset, MMT-composition information, MMT-asset transfer characteristics, and descriptive information.

An MMT payload is a formatted unit of data that carries a package or signals a message using the MMT protocol or the Internet Application Layer Transport Protocol (for example, RTP).

The MMT protocol is an application layer transport protocol for delivering MMT payloads over an IP network.

The MMT payload format is a format for payload of an MMT package or an MMT signaling message to be delivered by an MMT protocol or an internet application layer protocol (eg, RTP).

A presentation is defined as an operation performed by one or more devices to allow a user to experience one content component or one service (eg, watch a movie).

A service is defined as one or more content components that are transmitted for presentation or storage.

Service information is defined as metadata describing one service, characteristics and components of the service.

The content component or media component is defined as a media of a single type or a subset of the media of a single type. , Video tracks, movie subtitles, or a video enhancement layer of video.

Content is defined as a set of content components, and may be, for example, a movie or a song.

Hybrid delivery is defined as one or more content components being simultaneously transmitted through one or more physically different types of networks, or simultaneously transmitted through one or more physically identical or different networks.

Hereinafter, the first network or the second network may include various networks including a broadcast network, a broadband network, a cable network, or a satellite communication network. It includes.

Hereinafter, when hybrid transmission may be transmitted in MMT asset unit, substream unit, MFU unit, MPU unit, MMT package unit, or MMT packet unit, and the video content is composed of a plurality of layers such as a first layer and a second layer. Hybrid transmission may also be performed on a layer basis.

1 is a conceptual diagram illustrating an MMT hierarchical structure according to an embodiment of the present invention.

Referring to FIG. 1, an MMT layer includes an encapsulation layer, a delivery layer, and a functional area of a signaling layer. The MMT layer operates on a transport layer.

The encapsulation layer (E-layer) is the logical structure of the format of the media content, the MMT package, and the data unit to be processed by the MMT entity and the ISO base media file format. , ISOBMFF). To provide the necessary information for adaptive delivery, the MMT package specifies the components that contain the media content and the relationships between them. The format of the data units is defined to encapsulate the encoded media for storage or delivery and to easily convert between the two formats.

The encapsulation layer (E-layer) may be responsible for, for example, packetization, fragmentation, synchronization, multiplexing, and the like of transmitted media. have.

Various kinds of multimedia components may be encapsulated and combined with each other for transmission and consumption by the functions provided in the encapsulation layer (E-layer). Encapsulated media components and configuration information of the media components are provided in the functional area of the encapsulation layer (E-layer).

The main information about encapsulated media components includes aggregation, priority, media fragment dependence, MPU timing and structure information, MMT asset 150 identification information, initialization information, and codec information. It may include. The configuration information of the media components may include identification information of the MMT package 160 and the MMT asset 150, configuration information having a list of the MMT asset 150, and composition information of the MMT assets 150 in the MMT package 160. (composition information) 162 and Asset Delivery Characteristics (ADC) 164.

The encapsulation layer (E-layer), as shown in Figure 1, MMT E.1 Layer (MMT E.1 Layer), MMT E.2 Layer (MMT E.2 Layer) and MMT E.3 Layer (MMT E.3 Layer).

The E.3 layer encapsulates a Media Fragment Unit (MFU) provided from the Media Codec (A) layer to generate a Media Processing Unit (MPU).

Encoded media data from the upper layer is encapsulated into MFU. The type and value of the encoded media is abstracted so that the MFU can be generally used for a particular codec technology. This allows the lower layer to process the MFU without access to the encapsulated encoded media and the lower layer retrieves the required encoded media data from the network or storage buffer and sends it to the media decoder. The MFU has enough information media subunits to perform the above operation.

A plurality of MFUs in one or one group that can be independently transmitted and decoded generates an MPU. Independently transmittable and executable non-temporal media also generates the MPU. The MPU describes the internal structure, such as the arrangement and pattern of the MFUs, to enable quick access and partial consumption to the MFU.

The MFU may have a format that is independent of any particular codec and can carry data units that can be consumed independently in the media decoder. The MFU can be, for example, a picture or slice of the video.

The E.2 layer encapsulates the MPU created in the E.3 layer to generate an MMT asset.

The sequence of MPUs from the same source component creates an MMT asset. The MMT asset is packaged by the MMT package and is configured differently by composition information (CI) and transport characteristics (TC), multiplexed with another by the MMT payload format, transmitted by the MMT protocol do.

An MMT asset is a data entity composed of one or a plurality of MPUs from a single data source, and is a data unit in which composition information (CI) and asset delivery characteristics (ADC) are defined. MMT assets can correspond to packetized elementary streams (PES), for example video, audio, program information, MPEG-U widgets, JPEG images, MPEG 4 file format, M2TS (MPEG transport stream), etc.

The E.1 layer generates an MMT package by encapsulating the MMT asset generated in the E.2 layer.

The MMT asset is packaged with MMT composition information (MMT-CI) for later response of the same user experience together or separately with other functional areas—transport area and signal area. MMT assets are also packaged with Asset Delivery Characteristics (ADCs) that select an appropriate delivery method for each MMT asset to meet the haptic quality of the MMT asset.

Referring to FIG. 3, the MMT package 160 includes one composition information (CI) 162, at least one MMT asset 150, and asset delivery characteristics (ADC) 164 associated with each asset. ). The MMT package may correspond to a program of the MPEG-2 TS.

In addition, processing of the package is applied on an MPU basis, and an asset is a set of at least one MPU having the same asset ID, and one package includes one composition information, at least one MPU, and an asset transmission characteristic associated with each asset. It can be said that it is configured.

An asset can be an element of a package that encapsulates encoded media data, such as audio, video, or webpage data with timed or non-timed properties.

Composition information includes information about a relationship between MMT assets, and when one content consists of a plurality of MMT packages, the composition information indicates a relationship between a plurality of MMT packages. It may further include information.

The Asset Delivery Characteristics (ADC) 162 represents the QoS requirements and statistics for delivery of the asset. A plurality of assets may be associated with one ADC. The ADC can be used to set the parameters of the MMT payload and the MMT protocol by the entity packetizing the package for effective delivery of the asset.

Asset Delivery Characteristics (ADCs) may include delivery characteristic information necessary to determine delivery conditions of MMT assets or MMT packets, for example, traffic description parameters and QoS. It may include a descriptor (QoS descriptor).

A delivery layer (D-layer) defines a payload format and an application layer transport protocol. Payload format is defined to carry coded media data regardless of media type or encoding method. The application layer transport protocol provides enhanced features for delivery of packages including multiplexing and cross-layer communications.

The delivery layer (D-layer) can perform, for example, network flow multiplexing, network packetization, QoS control, etc. of media transmitted through a network. The transport layer (D-layer) is a packet between the transport layer and the encapsulation layer (E-layer), multiplexing of media such as video, audio, etc. transmitted over a network. Levels of aggregation and / or fragmentation, network packetization, QoS control, synchronization capabilities, transport layers such as traditional RTP, existing UDP, TCP and It is responsible for the interface with the same transport layer, an encapsulation layer, and a signaling layer.

The transport layer (D-layer) identifies different types of payloads from the encapsulation layer (E-layer) to handle payloads from the encapsulation layer (E-layer). The transport layer (D-layer) can handle temporal relations between packets transmitted over different networks and different channels. The synchronization function may include a hybrid network synchronization using a time stamp or the like.

The transport layer (D-layer) can handle timing constraints of MMT delivery packets for real-time media transmission. The transport layer (D-layer) can perform error control of MMT media packets such as forward error correction and retransmission. The transport layer (D-layer) may perform flow control of the MMT media packet. In order to maintain a certain level of QoS for delivery of MMT media packets, the transport layer (D-layer) interacts with other MMT layers as well as the lower layer (MAC, PHY) through a cross-layer design interaction can be performed. In addition, the D-layer may provide a function for performing group communication.

1, the D-layer includes an MMT D.1 layer (MMT D.1 Layer), an MMT D.2 layer (MMT D.2 Layer), and an MMT D.3 layer (MMT D.3 Layer). The D.1 layer (D.1-layer) receives the MMT package generated from the E.1 layer and creates the MMT payload. The MMT payload consists of the MMT payload format. The MMT payload format is a payload format for transmitting MMT assets and for transmitting information for consumption by MMT application protocols or other existing application transport protocols such as RTP. The MMT payload may contain fragments of MFUs with information such as AL-FEC.

The MMT payload format is defined as a general payload format for packetization of content components of a package. The MMT payload format is defined irrespective of a particular media codec so that any type of encapsulated media, such as an MPU, can be packetized into a payload for an application layer transport protocol that supports streaming delivery of media content. The MMT payload can be used as a payload format for RTP, MMT, and other packet transport protocols. The MMT payload may be used to packetize the signaling message.

The D.2 layer receives the MMT payload generated in the D.1 layer and generates an MMT packet. The MMT packet is a data format used in the application transmission protocol for the MMT.

D.3 D.3-layer supports QoS by providing the ability to exchange information between layers by cross-layer design. For example, the D.3 layer may perform QoS control using QoS parameters of the MAC / PHY layer. The QoS parameters of the MAC / PHY may be, for example, a bitrate, a packet loss ratio, an expected delay, an available buffer size, and the like.

A signaling layer (S layer) performs a signaling function. For example, session initialization / control / management of transmitted media, server-based and / or client-based trick modes, service discovery, synchronization, and other layers, i.e., delivery A signaling function for interfacing with a D-layer and an encapsulation layer (E-layer) may be performed. The synchronization may include synchronization control in a hybrid network.

The signaling layer defines the format of messages that govern the delivery and consumption of MMT packages. The message for consumption management is used to announce the structure of the MMT package, and the message for the delivery management is used to inform the structure of the payload format and the configuration of the protocol.

The signaling layer (S layer) may be composed of an MMT S.1 layer (MMT S.1 Layer) and an MMT S.2 layer (MMT S.2 Layer), as shown in FIG.

The S.1 layer may define the format of control messages between applications for media presentation session management. The presentation session management may define the format of control messages exchanged between applications for providing information required for media presentation, session management, and media consumption. The S.1 layer includes service discovery, media session initialization / termination of media, media session presentation / control of media, delivery (D) layer and encapsulation ( E) The interface function with the layer can be performed.

The S.2 layer may perform delivery session management. The delivery session management may be related to flow control, delivery session management, delivery session monitoring, error control, error control, and delivery regarding hybrid network synchronization control. It is possible to define the format of the control message exchanged between delivery end-points of the D-layer.

S.2 layer supports delivery session establishment and release, delivery session management (delivery session monitoring, flow control, error control, etc.), resource reservation for established delivery session, It may include signaling for synchronization in a complex delivery environment and signaling for adaptive delivery. Required signaling may be provided between a sender and a receiver. That is, the S.2 layer may provide signaling required between the sender and the receiver in order to support the operation of the transport layer as described above. In addition, the S.2 layer may be responsible for interfacing with the transport layer and the encapsulation layer.

A control message (or control information) may be generated at the signaling layer (S layer) and transmitted through the broadcast network and / or the broadband network.

When transmitted through both the broadcast network and the broadband network, the function of the control message transmitted through the broadcast network may be the same as the function of the control message transmitted through the broadband network. The syntax and format of the control message may vary depending on the type of application and delivery. For example, in the case of a hybrid transmission, a common format identical to the common control information may be used for control messages transmitted to the broadcasting network and the broadband network, respectively. Alternatively, in the case of the hybrid transmission, the same common control information may be transmitted in a different format for each of the broadcasting network and the broadband network. Alternatively, in the case of the hybrid transmission, different control information and different format may be transmitted for each of the broadcasting network and the broadband network.

FIG. 2 shows a format of unit information (or data or packet) used for each layer of the MMT hierarchical structure of FIG.

The Media Fragment Unit (MFU) defines a format for encapsulating a part of the AU in the transport layer to perform adaptive transmission in the range of the MFU. The MFU may be used to transmit certain types of encoded media so that portions of the AU can be independently decoded or discarded.

The MFU has an identifier for distinguishing one MFU from other MFUs, and has general relationship information between MFUs in a single AU. The dependencies between the MFUs in a single AU are described, and the relevant priorities of the MFUs are described as part of such information. The information may be used to handle transmissions at the lower transport layer. For example, the transport layer may skip transmission of disposable MFUs to support QoS transmission in insufficient bandwidth.

A Media Fragment Unit (MFU) 130 comprises coded media fragment data 132 and a Media Fragment Unit Header (MFUH) 134. The media fragment unit 130 has a general container format independent of a specific codec and carries the smallest data unit that can be consumed independently in the media decoder. The MFUH 134 may include additional information such as media characteristics-for example, loss-tolerance. MFU) 130 may be, for example, a picture or slice of a video.

The MPU is a collection of media fragment units including a plurality of media fragment units (130). The MPU may have a timed data unit or a non-timed data unit. The MPU may include a media processing unit header (MPUH) having additional information such as media fragment unit data and a time stamp for synchronization. The MPU has a general container format independent of a specific codec and includes media data equivalent to an access unit. MPU is data that is independently and completely processed by an entity following the MMT, and processing includes encapsulation and packetization. The MPU may comprise at least one MFU or may have portions of data having a format defined by another standard.

The MPU can be uniquely identified in the MMT package by its sequence number and associated asset ID, which distinguishes it from other MPUs. A single MPU can accept integral or non-time data of at least one AU. For time data, an AU may be delivered from at least one MFU, but one AU may not be divided into multiple MPUs. In non-time data, one MPU receives a portion of non-time data that has been independently and completely processed by an entity following the MMT.

The MPU has at least one random access point. The first byte of the MPU payload can always start with a random access point. In time data, this fact means that the decoding order of the first MFU in the MPU payload is always zero. In the time data, the presentation period and decoding order of each AU can be sent to inform the presentation time. The MPU does not have its own initial presentation time, and the presentation time of the first AU of one MPU may be described in the composition information. The composition information can specify the MPU's first presentation time.

The MPU may include an MMT hint track. For packetized delivery of the MPU, an MMT hint track may provide information for converting the encapsulated MPU into an MMT payload and an MMT packet.

The MMT hint track implies the fragmentation of the MPU to the transport. Whereby at least one MFU can be used to generate the MMT payload. The media data can be generated and transmitted as MMT payload at the transmission time by the transmission unit. Therefore, the format to be stored may be different from the format at the time of transmission. In this case, a dynamic transmission unit capable of extracting media data and generating an MMT payload during transmission is required.

The MMT hint track suggests the extraction and generation of MFU for encapsulation using the MMT payload format. The MMT payload may include MPU metadata or at least one MFU. The MMT hint track suggests to the transfer part how to extract the MFU data. If fragments are not used, the hint track can be omitted.

The MMT hint track can represent the entry format of the sample. Each media sample is assigned to at least one MFU, and samples of the MMT hint track will generate at least one MFU.

The MMT hint track contains a number of parameters. For example, a value of 1 may include a multilayer_flag indicating that multilayer information is provided. The dependency_id is an ID indicating the dependency of the MFU. If not a value of zero, the video is enhanced by at least one scalability level in terms of at least one temporal, quality, or spatial resolution. The depth_id indicates whether the corresponding MFU delivers the depth data of the video. quality_id is the quality ID of the MFU. If it is not zero, the video is enhanced by at least one scalability level in at least one of temporal, quality, or spatial resolution. temporal_id is the temporal ID of the MFU. If it is not zero, the video is enhanced by at least one scalability level in at least one of temporal, quality, or spatial resolution. view_id is the view ID of the MFU. If it is not zero, the video is enhanced by at least one scalability level in at least one of temporal, quality, or spatial resolution. In addition, the hint track may have a layer_id parameter indicating an ID of a scalable layer in which scalability dimensions information is provided in the initial information.

The MMT asset 150 is a collection of MPUs composed of a plurality of MPUs. The MMT asset 150 is a data entity composed of multiple MPUs (timed or non-timed data) from a single data source, and the MMT asset information 152 is an asset packaging metadata (Asset). additional information such as packaging metadata) and data type. The MMT asset 150 may include, for example, video, audio, program information, an MPEG-U widget, a JPEG image, an MPEG 4 FF, a packetized elementary streams (PES) stream, and the like.

The MMT asset may also be a logical data entity having encoded media data. The MMT asset has an MMT asset header and encoded media data. The encoded media data may be a group of MPUs collectively referred to by the same MMT asset ID. The type of data consumed by each entity directly associated with the MMT client may be a separate MMT asset. Examples of such data types are MPEG-2 TS, PES, MP4 file, MPEG-U Widget Package, and JPEG files.

The encoded media of the MMT asset may be time data or non-time data. Temporal data is audiovisual media data that requires synchronized decoding and presentation of specific data at specified times. Non-timed data is data of a data type that can be decoded and provided at any time in accordance with the provision of a service or user interaction.

A service provider may aggregate MMT assets to create MMT assets on a space-time axis.

The MMT package 160 is a collection of MMT assets including one or more MMT assets 150. The MMT assets in the MMT package may be multiplexed or concatenated in a chain.

The MMT package is a container format for MMT asset and configuration information. The MMT package provides a repository of MMT asset and configuration information for the MMT program.

The MMT program provider generates configuration information by encapsulating the encoded data into MMT assets and describing the temporal and spatial layout of the MMT assets and their transmission characteristics. The MMT asset can be sent directly to the D.1 payload format. Configuration information may be sent by the S.1 Presentation Session Management message. However, MMT program providers and clients that allow relays or future reuse of MMT programs store them in MMT package format.

In parsing the MMT package, the MMT program provider determines what transmission path (e.g., broadcast or broadband) the MMT asset will be served to the client. The configuration information in the MMT package is transmitted in the S.1 Presentation Session Management message along with the transmission related information.

The client receives the S.1 Presentation Session Management message to know which MMT program is available and how to receive the MMT asset for that MMT program.

The MMT package may also be transmitted by the D.1 payload format. The MMT package is packetized and delivered in D.1 payload format. The client receives the packetized MMT package and constructs all or part of it, where it consumes the MMT program.

The package information 165 of the MMT package 160 may include configuration information. Configuration Information includes additional information such as a list of MMT assets, package identification information, composition information 162 and Asset Delivery Characteristics (ADC) 164. can do. Composition information 162 includes information about a relationship between MMT assets 150.

The composition information 162 may further include information for indicating a relationship between a plurality of MMT packages when one content includes a plurality of MMT packages. Composition information 162 may include information about temporal, spatial and adaptive relations in an MMT package. Composition information provides information for package delivery optimization and presentation in a multi-screen environment. In a multi-screen environment, this can represent information that maps an asset to a specific screen. Detailed description will be described later.

Like information to assist in the transmission and presentation of the MMT package, Composition Information in the MMT provides information about the spatial and temporal relationships between MMT assets in the MMT package.

MMT-CI is an explanatory language that extends HTML5 to provide such information. If HTML5 is designed to describe a page-based presentation of text-based content, the MMT-CI primarily represents spatial relationships between sources. In order to support the representation of the temporal relationship between MMT assets, information related to MMT assets in the MMT package, such as presentation resources, time information to determine the transmission and consumption order of MMT assets, and various MMT assets in HTML5 Lt; RTI ID = 0.0 > media elements. &Lt; / RTI >

Asset Delivery characteristics information 164 includes information about delivery characteristics and may provide information needed to determine the delivery conditions of each MMT asset (or MMT packet). The asset delivery characteristic may include a traffic description parameter and a QoS descriptor.

The traffic description parameter may include bit rate information, priority information, and the like for the media fragment unit (MFU) 130 or the MPU. The bit rate information may include, for example, information about whether the MMT asset is a Variable BitRate (VBR) or a Constant BitRate (CBR), a guaranteed bitrate for a Media Fragment Unit (MFU) (or MPU) ), And a media fragment unit (MFU) (or MPU). The traffic description parameter may be used for resource reservation between servers, clients, and other components on the delivery path and may include, for example, the maximum size information of the media fragment unit (MFU) (or MPU) in the MMT asset . &Lt; / RTI > The traffic description parameter may be updated periodically or aperiodically.

The QoS descriptor includes information for QoS control, and may include, for example, delay information and loss information. The loss information may include, for example, a loss indicator of whether the delivery loss of the MMT asset is allowed or not. For example, if the loss indicator is '1', it indicates 'lossless', and if it is '0', it indicates 'lossy'. The delay information may include a delay indicator used to classify the sensitivity of the transmission delay of the MMT asset. The delay indicator may indicate whether the type of MMT asset is conversation, interactive, real time, and non-realtime.

One content may consist of one MMT package. Or one content may consist of a plurality of MMT packages.

When one content is composed of a plurality of MMT packages, composition information or configuration information indicating temporal, spatial, and adaptive relations between the plurality of MMT packages. configuration information) may exist inside one MMT package or outside the MMT package.

For example, in the case of hybrid delivery, some of the content components are transmitted through a broadcast network and the remaining parts of the content components are transmitted through a broadband network. Can be. For example, in the case of a plurality of AV streams constituting a multi-view service, one stream may be transmitted to the broadcasting network and the other streams may be transmitted to the broadband network. Each AV stream is multiplexed and transmitted to the client terminal Can be individually received and stored. Alternatively, there may be scenarios in which application software such as a widget is transmitted to a broadband network and an AV stream (AV program) is transmitted to an existing broadcast network.

In the multi-view service scenario and / or widget scenario as described above, the entire plurality of AV streams may be a single MMT package, and in this case, one of the plurality of streams may be stored in only one client terminal. The storage content becomes part of the MMT package, and the client terminal must rewrite the composition information or configuration information, and the rewritten content becomes a new MMT package independent of the server. .

In the multi-view service scenario and / or widget scenario as described above, each AV stream may be one MMT package, and in this case, a plurality of MMT packages constitute one content, and storage Storage is recorded in MMT package units and requires composition information or configuration information indicating a relationship between MMT packages.

Configuration information or configuration information included in one MMT package may refer to an MMT asset in another MMT package, and may also refer to an outside of an MMT package that refers to the MMT package in an out-band situation. I can express it.

There are SVC (Scalable Video Coding) coding technology and MANE (Media Aware Network Element) technology for network adaptive video transmission over IP network. SVC encoding is a hierarchical video compression technique that compresses the input video into multiple layers so that only the video layer suitable for a given network state can be transmitted. MANE can extract only a portion of video data suitable for transmission from an input video bitstream. A router or gateway with intelligent features.

In the MMT packet transmission method according to an embodiment of the present invention, when transmitting multi-layer video such as SVC (Scalable Video Coding) through an IP-network, an MMT generated in the D.1 layer to perform rate adaptation in the network. Provides the header format field of the payload. The delivery layer (D-layer) of the MMT is an application layer protocol including an MMT payload format for transmitting encapsulated media data from one network entity to another network entity. protocol). The MMT payload format defines a logical structure of information containing an MMT package to be carried by the MMT transport protocol.

The content targeted for transmission through the MMT scheme may be SVC video content including a base layer and at least one enhanced layer. One SVC video content may be composed of one MMT package, or one SVC video content may be composed of a plurality of MMT packages. That is, a plurality of layers included in the SVC video content may be included in one MMT package or may be separately included in a plurality of MMT packages.

4 illustrates a structure of a MANE-based network adaptive SVC or MVC video transmission system according to an embodiment of the present invention. FIG. 4 is a diagram illustrating a MANE (Media Aware) for selectively retransmitting an MMT packet received from a multilayer video server 410 and an MMT packet received from a multilayer video server 410 according to a network state. A plurality of MMT packet receiving devices 430 for receiving MMT packets retransmitted from the Network Element 420 and the MANE 430 are shown.

4 illustrates a structure of a network adaptive SVC video transmission system based on MANE, in which an SVC or MVC encoding technique and a MANE technique are combined. Since transmission of SVC video and transmission of MVC video using MANE according to an embodiment of the present invention have the same structure, transmission of SVC video will be described below. The following description may be applied in a similar manner in the case of transmission of MVC video.

The multilayer video server 410 stores SVC video having a total of four layers from L0 to L3. The SVC video layers are delivered to the MANE 420 by a transport protocol, and the MANE 420 is suitable for taking into consideration the quality of the network, the codec capability of the connected target terminal, the display resolution, and the like. Only the layer is extracted and transmitted to each terminal 430. In this case, when the MMT protocol is applied as a protocol for transmitting the SVC video layers to the MANE 420 in the server 410, information necessary for selecting a network adaptive video layer in the MANE needs to be provided. Therefore, it is necessary to record SVC video layer related information in MMT payload header format and deliver it to MANE. In this case, it is necessary to specify what information is to be recorded in MMT payload header format.

The Media Aware Network Element (MANE) 430 is a router or switch of an IP network capable of performing rate adaptation in the network as shown in FIG. According to an embodiment of the present invention, the MANE 430 may obtain signaling from a signaling procedure and a packet payload header to determine whether any packet is dropped or forwarded so that it is no longer transmitted. Decisions and judgments are made based on signaling information.

In a packet-switched network where it is difficult to provide a constant bandwidth, the MANE 430 may be used to reduce the congestion problem of the network. In one embodiment, hierarchical video transmission may be used to drop a portion of the video layer in coarse grain adaptation. In fine grain adaptation, MANE can drop some of the video data of the upper video layer and consequently reduce the amount of video data transmitted to fit the available bandwidth. In general, layered multicast is only considered for the core network of the streaming pass, but MANE connected to various terminals close to the edge network will reassemble the hierarchical multicast into a packet stream containing only the appropriate video layer. Can be. In this embodiment, the MANE must obtain extension information from the MMT packet to determine which packet to drop or pass without inspecting the packet. If scalability information is represented in the MMT payload header, the MANE can efficiently determine whether or not it is worth going forward in the MMT packet.

5 is an internal structure of an MMT packet according to an embodiment of the present invention. The MMT packet includes an MMT packet header generated in the D.2 layer, an MMT payload header generated in the D.1 layer, and an MMT payload. The MMT payload includes an SVC NAL unit generated at the E layer.

The MMT payload header according to an embodiment of the present invention receives an MMT packet including a plurality of aggregated NAL units from a different layer of MANE, and does not completely parse the received MMT packet. It is possible to determine whether to continue delivery or to discard MMT packets without continuing delivery of MMT packets. This approach can be used when a certain video layer is not needed for the current streaming session, and for this operation of the MANE packets arriving at the MANE need to be examined at the MANE.

In order for the MANE to simply determine the progress of the MMT packet and to perform the processing thereof, an informative summary of payloads, which are hierarchical video data collected in the MMT packet, must be provided to the MANE. In the NAL unit header of the SVC bit stream, specific extension information of the NAL packet is basically embedded.

The MMT payload header according to an embodiment of the present invention is configured to include additional extension information in the information of the MMT payload header corresponding to the D.1 layer of the MMT. The appropriate extension information can be obtained by examining the extension information of the NAL unit included in the MMT payload header, without having MANE directly look inside the NAL unit included in the payload of the MMT packet aggregated according to the SVC video coding specification. In this case, it is possible to determine whether to forward the MMT packet, forward it without discard, or examine the inside of the MMT packet.

6 is a structure of extension information according to an embodiment of the present invention. FIG. 6 shows payload scalability information included in an MMT payload header. Payload extension information includes TID_low, DID_low, QID_low, TID_high, DID_high and QID_high parameters. A structure of syntax constituting extended information will be described with reference to FIG. 6. Payload scalability information is composed of six parameters such as TID_low, DID_low, QID_low, TID_high, DID_high, and QID_high. The individual semantics for these six parameters are shown in FIG. 6.

The TID_low parameter represents the smallest TID value among TID values of NAL units aggregated in an MMT packet. The DID_low parameter represents the smallest DID value among the DID values of NAL units aggregated in an MMT packet. The QID_low parameter represents the smallest QID value among QID values of NAL units aggregated in an MMT packet. The TID_high parameter indicates the largest TID value among TID values of NAL units aggregated in an MMT packet. The DID_high parameter represents the largest DID value among the DID values of NAL units aggregated in an MMT packet. The QID_high parameter indicates the largest QID value among QID values of NAL units aggregated in an MMT packet.

Each parameter is explained in more detail. In SVC video, three types of layers (ID) are used to represent a spatio-temporal relationship between NAL (Network Abstraction Layer) units. In SVC video, the temporal level of each NAL unit is represented by a TID (temporal_ID), the spatial level is represented by a DID (Dependency_ID), and the quality level is represented by a QID (quality_ID). That is, TID (Temporal_ID) represents a temporal level of a NAL unit, Dependency_ID represents a spatial level, and QID (Qualtiy_ID) represents a quality level. This extended layer information (TID, DID, QID) value is recorded for each header of each NAL unit.

The extended information according to an embodiment of the present invention includes TID_low, DID_low, QID_low, TID_high, DID_high, and QID_high parameters, and may include only one of six parameters. The extension information according to an embodiment of the present invention may be composed of a set of (TID_low, DID_low, QID_low) and a set of (TID_high, DID_high, QID_high). In addition, the payload extension information according to an embodiment of the present invention may be composed of a set of (TID_low, TID_high), a set of DID_low, a DID_high, and a set of (QID_low, QID_high) or may include only one set. . In addition, it may include a set of various parameters.

7 is an algorithm for determining whether to forward, discard, and inspect an inner packet of an MMT packet made in a MANE according to an embodiment of the present invention. Referring to the algorithm of FIG. 7, by using the TID_low, DID_low, QID_low, TID_high, DID_high and QID_high parameters, the MANE forwards or discards an MMT packet including a plurality of aggregated NAL units from different video layers. (discard) or simply to inspect the inside of the MMT packet (inspect).

Referring to payload_scalability_information () of FIG. 6, the MANE uses the algorithm of FIG. 7 to determine which MMT packet to drop, forward, or for each (TID) present inside the NAL unit header. , DID, QID) can be determined simply. The algorithm of FIG. 7 compares the (TID_low, DID_low, QID_low) and (TID_high, DID_high, QID_high) values given by NAME with the TID_opt, DID_opt and QID_opt values determined as optimal extraction points for layer adaptation.

In the algorithm of FIG. 7, TID_opt, DID_opt, and QID_opt indicate (TID, DID, QID) values corresponding to optimal extraction points determined in MANE in order to adapt to a given network environment. When all values of TID_opt, DID_opt and QID_opt are greater than or equal to each of TID_high, DID_high and QID_high recorded in the payload header of the MMT packet, the MANE transmits the corresponding MMT packet to the target terminal. MANE discards the MMT packet without transmitting it if all values of TID_opt, DID_opt and QID_opt are smaller than TID_low, DID_low and QID_low respectively recorded in the payload header of the MMT packet. If both of the above conditions are not satisfied, MANE analyzes the TID, DID and QID values of all NAL units included in the MMT packet individually to determine NAL units to be delivered or discarded.

8 shows a work flow in which the present invention is applied in a MANE. First, the MMT packet reaches MANE (S100). MANE parses the payload header of the MMT packet (S110). MANE obtains (TID_low, DID_low, QID_low) and (TID_high, DID_high, QID_high) values from the MMT payload header (S120). Next, MANE determines the processing of the MMT packet using the algorithm of FIG. 7 described above.

As described above, the MMT packet transmission method according to an embodiment of the present invention provides a D.1 layer payload header format field that enables adaptation in a network when transmitting multi-layer video such as SVC over an IP network. . With payload extension information included in the MMT payload header format according to an embodiment of the present invention, the MANE drops and forwards any MMT packet, or each TID present inside the NAL unit header. You can easily decide whether to examine the DID, QID values.

In general, when an MMT packet includes only one SVC NAL unit, since the NAL unit header itself serves as an MMT payload header, the (TID, DID, QID) value recorded in the NAL unit header is automatically payloaded. It is recorded in the header and can be referenced by MANE. MANE itself is basically a protocol capable of processing up to the D-layer in FIG. 1, so that the necessary adaptive processing is performed by referring to the TID, DID, and QID values recorded in the MMT payload header. In this case, the adaptive processing means that the NAL unit to be delivered to the target terminal and the NAL unit not to be delivered are classified based on the TID, DID, and QID values. That is, only NAL units having spatial resolution, frame rate, and image quality suitable for delivery to a target terminal are filtered based on TID, DID, and QID values.

Meanwhile, the MMT packet may be aggregated by multiple SVC NAL units into one MMT packet and transmitted at once. When the size of an individual NAL unit is very small, when a large number of NAL units are aggregated into one MMT packet, the overhead of generating multiple MMT packets can be greatly reduced. In this case, as shown in FIG. 6, payload scalability information, which may represent a plurality of NAL units, may be separately recorded in the payload header of the MMT packet. MANE does not need to analyze the TID, DID and QID values recorded in all NAL unit headers when such payload scalability information is provided in the payload header. That is, according to an embodiment of the present invention, the MANE may quickly determine the MMT packet to be forwarded and the discarded MMT packet to the target terminal by simply referring to only the payload extension information recorded in the payload header. .

While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. It will be apparent to those skilled in the art that the present invention can be carried out without departing from the spirit and scope of the invention.

That is, although the above-described embodiments of the present invention describe that extension information is generated in the MMT payload header, in another embodiment, the extension information is present in the MMT packet header to provide the above extension information at the MMT packet header level. It may be. In another embodiment, the extension information may be included in the MMT payload. In another embodiment, the extension information may be present in a separate MMT packet.

FIG. 9 illustrates an embodiment of an MMT header format including extension information for adaptation in a NAME based network according to another embodiment of the present invention. In FIG. 9, the extended information is represented by multilayer information (ML-Info). The MMT packet header according to another embodiment of the present invention is an ML_type (2 bits) to classify the type of multilayer information carried by the current MMT packet and corresponding multilayer information related to the current SVC and MVC video data. It further includes a flag. The bit length of the multilayer information for SVC is 32 bits, and the bit length of the multilayer information for MVC (Multiview Video Coding) is 20 bits. The multilayer information further includes values of PID_low, PID_high, VID_low, and VID_high related to the PID and the VID. The definition and use of PID_low, PID_high, VID_low and VID_high may be used in a manner similar to the definition and use of the maximum and minimum values of the TID, DID and QID parameters described above.

A syntax and semantics of an MMT packet header format according to another embodiment of the present invention for reflecting multilayer information will be described with reference to FIG. 9.

packet_id (16 bits) is an integer value assigned to each MMT asset to distinguish a packet of one MMT asset from another packet. Different values are assigned to signaling messages and FEC parity flows. packet_sequence_number (32 bits) is an integer value assigned to each MMT packet, incremented by one from an arbitrary value. The packet_sequence_number starts again from 0 when the maximum value is reached. timestamp (32 bits) represents a time instance of MMT packet delivery. flags (8bits) are header_extension_flag indicating whether the header extension field is provided in the MMT packet header, service_classifier_flag indicating whether service classification information is provided in the MMT packet header, QoS_classifier_flag indicating whether QoS classification information is provided in the MMT packet header, flow confirmation information Flow_identifier_flag indicating whether is provided in the MMT packet header, and private_user_data_flag indicating whether personal user data information is provided in the MMT packet header. The flags also include Multilayerinfo_flag indicating whether multilayer information is provided in the MMT packet header. Each flag of Flags may be provided in 1 bit.

ML_Info is a variable that depends on the type of multilayer information. The ML_Info field represents multilayer information required for in-network adaptation in MANE. According to another embodiment of the present invention, the ML_Info field of the MMT packet is different from the MMT packet including the SVC video data and the MMT packet including the MVC video data. Therefore, the structure of the ML_Info varies according to the type of video data. First, an ML_type field indicating a state of a type represented by multilayer information will be described with reference to FIG. 10.

Referring to FIG. 10, if the ML_type field has a value of 00, it is multilayer information related to the SVC after the ML_type field. It can be seen that the values of (TID_low, DID_low, QID_low, PID_low) and (TID_high, DID_high, QID_high, PID_high) will be provided subsequently. Similarly, multilayer information related to MVC after ML_type field when ML_type field has value of 01. It will be appreciated that the VID_low and VID_high values will be provided subsequently. When the ML_type field has a value of 10, 3D video related multilayer information may be applied to a value appearing after the ML_type field. That is, the ML_type field indicates in which encoding method the video data included in the MMT packet is encoded. Although the meaning of the value after the ML_Info field has been described according to the value of the ML_type field, the meaning may be different and may be changed to another meaning according to use.

FIG. 11 illustrates a state in which an MMT packet header format has a format for multilayer information in SVC according to an embodiment of the present invention. In the MMT packet header format containing SVC video data, as shown in FIG. 11, the ML_Info field includes ML-type (2 bit), TID_low (T_low, 3bit), DID_low (D_low, 3bit), QID_low (Q_low, 4bit), and PID_low (P_low, 10bit), TID_high (T_high, 3bit), DID_high (D_high, 3bit), QID_high (Q_high, 4bit) and PID_high (P_high, 6bit) fields, and may be configured by arranging the above fields in the order mentioned. According to another embodiment, the arrangement order of the fields may be changed, and some fields may be omitted.

In FIG. 11, the fields related to the values of TID, DID and QID have the same meaning as the TID_low, DID_low and QID_low parameters and the TID_high, DID_high and QID_high parameters. PID (priority identification) indicates the priority of the NAL unit, and the smaller the value, the higher the priority. The PID_low value represents the smallest PID value among the PID values of the NAL units collected in the MMT packet. Therefore, PID_low is the PID value of the highest priority NAL unit. The PID_high value represents the highest PID value among PID values of NAL units aggregated in an MMT packet. Therefore, PID_high is the PID value of the NAL unit with the lowest priority. PID_low and PID_high can be selectively used as needed.

FIG. 12 is an algorithm for determining whether to forward, discard, and inspect an internal packet of an MMT packet including SVC video performed in a MANE according to another embodiment of the present invention. In the algorithm of FIG. 12, TID_opt, DID_opt, QID_opt, and PID_opt represent (TID, DID, QID, PID) values corresponding to optimal extraction points determined in MANE in order to adapt to a given network environment. When all values of TID_opt, DID_opt, QID_opt and PID_opt are greater than or equal to each of TID_high, DID_high, QID_high and PID_high recorded in the MMT packet header, the MANE forwards the corresponding MMT packet to the target terminal. MANE discards the corresponding MMT packet without transmitting it if all values of TID_opt, DID_opt, QID_opt and PID_opt are smaller than TID_low, DID_low, QID_low and PID_low recorded in the MMT packet header, respectively. If both of the above conditions are not satisfied, MANE analyzes the TID, DID, QID and PID values of all NAL units included in the MMT packet individually to determine NAL units to be delivered or discarded.

In another embodiment of the present invention, as described above, the use of the PID is optional, and thus the determination of the PID value may be selectively performed. In addition, the determination of the values of the TID, the DID, and the QID may be selectively performed as necessary. In addition, only one of the maximum value and the minimum value may be compared with the optimal extraction point to determine whether or not to deliver the MMT packet as needed.

FIG. 13 illustrates a work flow applied in a MANE including SVC video in another embodiment of the present invention. First, the MMT packet reaches the MANE (S200). MANE parses the header of the MMT packet (S210). MANE obtains (TID_low, DID_low, QID_low, PID_low) and (TID_high, DID_high, QID_high, PID_high) values from the MMT packet header (S220). Next, MANE determines the processing of the MMT packet using the algorithm of FIG. 12 described above.

That is, MANE determines whether the value of TID_opt is greater than or equal to TID_high, the value of DID_opt is greater than or equal to DID_high, the value of QID_opt is greater than or equal to QID_high, and the value of PID_opt is greater than or equal to PID_high (S230). If the determination result is true, the corresponding MMT packet is transmitted to the target terminal (S240).

If the determination result is false, MANE determines whether all values of TID_opt, DID_opt, QID_opt, and PID_opt are smaller than TID_low, DID_low, QID_low, and PID_low recorded in the MMT packet header, respectively (S250). If the determination result is true, the MANE discards the corresponding MMT packet without transmitting (S260). However, if the determination result is false, that is, if the condition for transmitting the MMT packet is not satisfied and the condition for discarding the MMT packet is not satisfied, the MANE examines the NAL unit inside the MMT packet (S270).

14 illustrates a state in which an MMT packet header format has a format for multilayer information in MVC according to another embodiment of the present invention. In the MMT packet feather format containing MVC video data as shown in FIG. 14, the ML_Info field may be arranged in the order of ML-type (2 bits), VID_low (V_low, 10bit), and VID_high (V_high, 10bit). In another embodiment, the order of VID_low and VID_high may be changed, and either may be omitted. View identification (VID) indicates the ID of a view in MVC. Therefore, VID_low has the lowest value among the VIDs of the NAL units belonging to the MMT packet, and VID_high has the highest value among the VIDs of the NAL units belonging to the MMT packet.

If you do not need to send video from all views, you can filter in MANE using the VID value. When encoding by H.264 AVC / MVC, a VID is set in an access unit constituting a stream of encoded results. VID allows to identify which view component each Access Unit is in MVC. In MVC, an Access Unit (Dependent Unit, in the case of a Dependent View) is, for example, a unit in which data of one picture is integrated so as to be accessible by picture units. When encoding according to the H.264 AVC / MVC profile specification, an MVC header is added to each view component. MVC header includes view_id.

FIG. 15 is an algorithm for determining whether to forward, discard, and inspect an inner packet of an MMT packet including MVC video that is performed in a MANE according to another embodiment of the present invention. In the algorithm of FIG. 15, VID_opt represents a VID value corresponding to an optimal extraction point determined in MANE in order to adapt to a given network environment. MANE transfers the corresponding MMT packet to the target terminal when the VID_opt value is greater than or equal to VID_high recorded in the MMT packet header. MANE discards the MMT packet without transmitting it if the VID_opt value is smaller than VID_low recorded in the MMT packet header. If both of the above conditions are not satisfied, MANE analyzes the VID values of all NAL units included in the MMT packet individually to determine NAL units to be delivered or discarded.

Unlike the algorithm shown in FIG. 15, in an algorithm for determining whether to deliver an MMT packet including MVC video according to another embodiment of the present invention, MANE examines only if the VID_opt value is greater than or equal to VID_high recorded in the MMT packet header. In this case, the MMT packet may be delivered only to the target terminal. Otherwise, the MMT packet may be discarded without transmitting the MMT packet. Similarly, in the algorithm according to another embodiment of the present invention, MANE examines only if the VID_opt value is smaller than VID_low recorded in the MMT packet header and discards the corresponding MMT packet only when it is applicable. MMT packet can be transmitted without discarding.

FIG. 16 illustrates a work flow applied in a MANE including MVC video in another embodiment of the present invention. First, the MMT packet reaches MANE (S300). MANE parses the header of the MMT packet (S310). MANE obtains VID_low and VID_high values from the MMT packet header (S320). Next, MANE determines the processing of the MMT packet using the algorithm of FIG. 15 described above.

That is, MANE determines whether the value of VID_opt is greater than or equal to VID_high (S330). If the determination result is true, the corresponding MMT packet is transmitted to the target terminal (S340).

If the determination result is false, MANE determines whether the value of VID_opt is smaller than the value of VID_low recorded in the MMT packet header in a similar manner to S330 (S350). If the determination result is true, the MANE discards the MMT packet without transmitting (S360). However, if the determination result is false, that is, if the condition for transmitting the MMT packet is not satisfied and the condition for discarding the MMT packet is not satisfied, MANE examines the inside of the MMT packet (S370).

In another embodiment, step S320 may be performed by the MANE obtaining only the VID_high value from the MMT packet header. In this case, step S330 may be performed by examining only the VID_opt value that is greater than or equal to VID_high recorded in the MMT packet header, and only if this is the case, delivers the corresponding MMT packet to the target terminal (S340). It may be discarded without transmitting the MMT packet (S360). That is, the step of S350 may be omitted in the flowchart of FIG. 16.

Similarly, in another embodiment of the present invention, step S320 may be performed by MANE obtaining only the VID_low value from the MMT packet header. In this case, step S330 is performed by examining only the VID_opt value smaller than VID_low recorded in the MMT packet header. If this is not the case, the MMT packet is transmitted without discarding (S340), and only when this is the case, the MMT packet is transmitted without transmitting (S360). That is, one determination step may be omitted in the flowchart of FIG. 16.

The combination of the above-described embodiments is not limited to the above-described embodiment, and various combinations of types as well as the above-described embodiments may be provided according to implementation and / or necessity.

That is, as in the above-described embodiments of the present invention, the multi-layer video server 410 controls the transmission of the MMT packet by using the extension information as the MANE controls the transmission of the MMT packet by using the above-described extension information. It may be.

In the above-described embodiments, the methods are described on the basis of a flowchart as a series of steps or blocks, but the present invention is not limited to the order of steps, and some steps may occur in different orders or in a different order than the steps described above have. It will also be understood by those skilled in the art that the steps depicted in the flowchart illustrations are not exclusive, that other steps may be included, or that one or more steps in the flowchart may be deleted without affecting the scope of the present invention. You will understand.

The foregoing embodiments include examples of various aspects. While it is not possible to describe every possible combination for expressing various aspects, one of ordinary skill in the art will recognize that other combinations are possible. Accordingly, it is intended that the invention include all alternatives, modifications and variations that fall within the scope of the following claims.

Claims

In the method for generating an MMT packet by the MMT packet transmission device for delivering video data,
Generating an MMT packet including extension information generated using information of at least one NAL unit included in the MMT packet.

The method of claim 1,
The extension information is MMT packet generation method, characterized in that included in the packet header of the MMT packet or the payload header of the MMT payload included in the MMT packet.

The method of claim 1,
The extension information may include a TID (Temporal_ID) parameter generated using a temporal level of the at least one NAL unit, a DID (Dependency_ID) parameter generated using a spatial level of the at least one NAL unit, and the at least one NAL unit. At least one NAL included in the MMT packet or including at least one of a QID (Quality_ID) parameter generated by using a quality level of the PSI and a Priority_ID parameter generated by using priority information of the at least one NAL unit MMT packet generation method comprising a VID parameter generated using the view information of the unit.

The method of claim 1,
The extension information includes a TID_low parameter indicating the smallest value among the temporal levels of the NAL unit included in the MMT packet, a DID_low parameter indicating the smallest value among the spatial levels of the NAL unit included in the MMT packet and the MMT packet. QID_low parameter representing the smallest value among the quality levels of the NAL unit, PID_low parameter representing the smallest value among the priority information of the NAL unit included in the MMT packet, and the largest value of the temporal level of the NAL unit included in the MMT packet. TID_high parameter indicating a, DID_high parameter indicating the largest value of the spatial level of the NAL unit included in the MMT packet, QID_high parameter indicating the highest value of the quality level of the NAL unit included in the MMT packet and included in the MMT packet Of PID_high parameters representing the largest value among the priority information of the NAL unit A VID_low parameter indicating at least one of view information of NAL units included in at least one or included in the MMT packet and a VID_high parameter indicating a largest value among view information of NAL units included in the MMT packet. MMT packet generation method comprising at least one of.

The method according to claim 3 or 4,
The extension information MMT packet generation method further comprises an ML_Type parameter indicating the encoding method of the video data included in the MMT packet.

In the method for transmitting a video data transmission apparatus MMT packet,
Determining whether or not to deliver the MMT packet by using the extension information included in the MMT packet.

The method according to claim 6,
The MMT packet includes an MMT packet header, an MMT payload header, and an MMT payload.
The extension information MMT packet transmission method, characterized in that located in the MMT packet header or the MMT payload header.

The method according to claim 6,
The extension information includes a TID_low parameter indicating the smallest value among the temporal levels of the NAL unit included in the MMT packet, a DID_low parameter indicating the smallest value among the spatial levels of the NAL unit included in the MMT packet and the MMT packet. QID_low parameter representing the smallest value among the quality levels of the NAL unit, PID_low parameter representing the smallest value among the priority information of the NAL unit included in the MMT packet, and the largest value of the temporal level of the NAL unit included in the MMT packet. TID_high parameter indicating a, DID_high parameter indicating the largest value of the spatial level of the NAL unit included in the MMT packet, QID_high parameter indicating the highest value of the quality level of the NAL unit included in the MMT packet and included in the MMT packet Of PID_high parameters representing the largest value among the priority information of the NAL unit A VID_low parameter indicating at least one of view information of NAL units included in at least one or included in the MMT packet and a VID_high parameter indicating a largest value among view information of NAL units included in the MMT packet. MMT packet transmission method comprising at least one of.

The method of claim 8, wherein the determining of the delivery step,
The reference apparatus TID_opt of the temporal level set by the transmitting device is greater than or equal to the value of the TID_high, the reference value DID_opt of the set spatial level is greater than or equal to the value of DID_high, and the reference value QID_opt of the set quality level is greater than or equal to the value of QID_high. MMT packet transmission method, characterized in that carried out by forwarding the MMT packet.

The method of claim 8, wherein the determining of the delivery step,
The reference apparatus TID_opt of the temporal level set by the transmitting device is smaller than the value of the TID_low, the reference value DID_opt of the set spatial level is smaller than the value of DID_low, the reference value QID_opt of the set quality level is smaller than the value of QID_low, and set priority information. And when the reference value PID_opt is smaller than the value of PID_low, the MMT packet transmission method is performed by not transmitting the MMT packet.

The method of claim 8,
The extension information further includes an ML_type parameter indicating an encoding method of video data included in the MMT packet.
The transmitting method further includes the step of confirming, by the transmitting apparatus, that the video data included in the MMT packet is encoded by the scalable video coding (SVC) encoding method using the value of the ML_type parameter.
Determining whether the delivery is,
The reference device TID_opt of the set temporal level is smaller than the value of the TID_high and greater than or equal to the value of the TID_low, the reference value DID_opt of the set spatial level is smaller than the value of the DID_high and greater than or equal to the value of the DID_low or When the reference value QID_opt of the set quality level is smaller than the value of the QID_high and greater than or equal to the value of the QID_low, the MMT packet transmission method is performed by determining whether the MMT packet is delivered by examining the NAL unit of the MMT packet. .

The method according to claim 6,
The transmission device is a Media Aware Network Element (MANE),
Determining whether or not to deliver the MMT packet using the data of the MMT packet,
Arriving at the MANE by an MMT packet;
Parsing the MMT packet header;
Obtaining extension information from the MMT packet header; And
And determining whether or not to forward the MMT packet from the extension information.

In the MMT packet transmission apparatus for delivering video data,
MMT packet generation unit; And
MMT packet transmission unit,
The MMT packet generator generates an MMT packet including an MMT payload and extension information generated using information of at least one NAL unit included in the MMT payload.

The method of claim 13,
The MMT packet generating unit generates the MMT packet by including the extension information in the packet header of the MMT packet or the payload header of the MMT payload included in the MMT packet.

The method of claim 13,
The extension information may include a TID parameter generated using the temporal level of the at least one NAL unit, a DID parameter generated using the spatial level of the at least one NAL unit, and a quality level of the at least one NAL unit. At least one of a generated QID parameter and a PID parameter generated using priority information of the at least one NAL unit or a VID parameter generated using view information of the at least one NAL unit. MMT payload transmission apparatus, characterized in that.

In the MMT packet transmission apparatus for delivering video data,
MMT packet transmission apparatus comprising an MMT packet transmission unit for determining whether or not to deliver the MMT packet using the extension information of the MMT packet.

17. The method of claim 16,
The MMT packet includes an MMT packet header, an MMT payload header, and an MMT payload.
And the extension information is located in an MMT packet header or a payload header of the MMT packet.

17. The method of claim 16,
The extension information includes a TID_low parameter indicating the smallest value among the temporal levels of the NAL unit included in the MMT packet, a DID_low parameter indicating the smallest value among the spatial levels of the NAL unit included in the MMT packet and the MMT packet. QID_low parameter representing the smallest value among the quality levels of the NAL unit, PID_low parameter representing the smallest value among the priority information of the NAL unit included in the MMT packet, and the largest value of the temporal level of the NAL unit included in the MMT packet. TID_high parameter indicating a, DID_high parameter indicating the largest value of the spatial level of the NAL unit included in the MMT packet, QID_high parameter indicating the highest value of the quality level of the NAL unit included in the MMT packet and included in the MMT packet Of PID_high parameters representing the largest value among the priority information of the NAL unit A VID_low parameter indicating at least one of view information of NAL units included in at least one or included in the MMT packet and a VID_high parameter indicating a largest value among view information of NAL units included in the MMT packet. MMT packet transmission apparatus comprising at least one of.

The method of claim 18,
The transmission device has a reference value TID_opt of a set temporal level greater than or equal to the value of TID_high, a reference value DID_opt of a set spatial level is greater than or equal to a value of DID_high, and a reference value QID_opt of a set quality level is greater than or equal to a value of QID_high. MMT packet transmission apparatus, characterized in that for transmitting the MMT packet.

The method of claim 18,
The transmitting device has a reference value TID_opt of a set temporal level smaller than a value of TID_low, a reference value DID_opt of a set spatial level smaller than a value of DID_low, a reference value QID_opt of a set quality level smaller than a value of QID_low, and sets priority information. MMT packet transmission apparatus characterized in that the MMT packet is not delivered when the reference value PID_opt is smaller than the value of PID_low.

The method of claim 18,
The extension information further includes an ML_type parameter indicating an encoding method of video data included in the MMT packet.
When the transmission device determines that the video data of the MMT packet is generated by MVC encoding using the value of the ML_type parameter, when the reference value VID_opt of the set view information is greater than or equal to the VID_high. MMT packet transmission apparatus, characterized in that for transmitting the MMT packet.

17. The method of claim 16,
The transmission device is a Media Aware Network Element (MANE),
The transmission device,
Further comprising a receiving unit for receiving the MMT packet,
The MMT packet transmission unit determines whether or not to deliver the MMT packet from the extended information generated by parsing the MMT packet header included in the MMT packet.

MMT packet header; In the structure of the MMT packet including an MMT payload,
The MMT payload includes a payload header,
The MMT packet header or the payload header includes extension information generated using at least one NAL unit.