KR102196504B1 - Apparatus and method for providing contents - Google Patents

Abstract

A method of receiving content from a client is provided. The content receiving method according to an embodiment includes receiving a spatial set identifier for a group of at least one spatial object from a server, and requesting a first content corresponding to the spatial set identifier from the server. And receiving the first content corresponding to the request from the server.

Description

Content providing device and method {APPARATUS AND METHOD FOR PROVIDING CONTENTS}

The technical field of the present invention relates to a technology for providing streaming content, and in particular, to an apparatus and method for providing media content using adaptive streaming.

Streaming is one of methods for transmitting and playing multimedia contents such as sound and video. The client can play the content while receiving the content through streaming.

The adaptive streaming service means providing a streaming service through a communication method consisting of a request from a client and a response from a server to the request.

The client can request a media sequence suitable for its environment (e.g., its transmission channel) through the adaptive streaming service, and the server selects a media sequence that meets the client's request among various quality media sequences that it has. Can provide.

The adaptive streaming service can be provided based on various protocols.

The HTTP adaptive streaming service refers to an adaptive streaming service provided based on the HTTP protocol. The client of the HTTP adaptive streaming service can receive content from the server using the HTTP protocol and can transmit a request related to the streaming service to the server.

Currently, a method of downloading and streaming video simultaneously using TCP or HTTP protocol is widely used. Meanwhile, MPEG, a video-related standardization organization, is also standardizing a method for streaming while downloading videos using the HTTP protocol through a project called DASH (Dynamic Adaptive Streaming over HTTP). In the existing TCP or HTTP-based video transmission method and MPEG-DASH, one program is divided into segments divided by time units, and then sequentially downloaded and played. In this process, you can select an appropriate bandwidth or video type according to the network or terminal environment.

The present disclosure may provide an apparatus and method for transmitting spatially divided video content in units of spatial segments.

The content receiving method of a client according to an embodiment includes receiving a spatial set identifier for a group of at least one spatial object from a server, and requesting a first content corresponding to the spatial set identifier to the server And receiving the first content corresponding to the request from the server.

The spatial set identifier may identify a tile group including at least one tile in which a full-frame video is spatially divided.

The first content may be a video corresponding to the tile group.

The spatial set identifier may group a set of spatial objects.

In the receiving of the spatial set identifier, metadata including the spatial set identifier may be received.

The metadata may be media presentation description (MPD) of the content.

The media information may include a spatial relationship description (SRD).

The spatial relationship description may include at least one of the spatial set identifier, x-axis position, y-axis position, width, height, reference space width, and reference space height of the spatial object.

In the step of requesting the first content from the server, a request including a URL corresponding to the first content may be transmitted to the server.

The URL may include a URL corresponding to a tile in which a full-frame video is spatially divided.

According to another embodiment, a method for transmitting content by a server includes transmitting a spatial set identifier for a group of at least one spatial object to a client, and requesting a first content corresponding to the spatial set identifier from the client. It may include receiving and transmitting the first content corresponding to the request to the client.

The spatial set identifier may include at least one tile in which a full-frame video is spatially divided.

The first content may be a video corresponding to the tile group.

The spatial set identifier may group a set of spatial objects.

In the transmitting of the spatial set identifier, metadata including at least one of the spatial set identifier, the x-axis position, y-axis position, width, height, reference space width, and reference space height of the spatial object may be transmitted. have.

The metadata may be media presentation description (MPD) of the content.

The media information may include a spatial relationship description (SRD).

The spatial relationship description may include the spatial set identifier.

In the step of receiving the request for the first content, a request including a URL corresponding to the first content may be received from the client.

The URL may include a URL corresponding to a tile in which a full-frame video is spatially divided.

According to various embodiments, an apparatus and a method for transmitting spatially divided video content in units of spatial segments may be provided.

When streaming using a spatial segment divided by one content, the location of each spatial segment can be displayed simply, a subset of the spatial segment can be provided, and separate index information can be provided. have.

1 is a conceptual diagram of spatially divided video content according to an exemplary embodiment.
2 is a flowchart illustrating a signal processing process according to an exemplary embodiment.
3 is a hierarchical diagram of a method for displaying a spatial segment according to an exemplary embodiment.
4 is a hierarchical diagram of a method for displaying a spatial segment according to another exemplary embodiment.
5 is a structural diagram of a client according to an embodiment of the present invention.
6 is a structural diagram of a client according to an embodiment of the present invention.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited or limited by the embodiments. The same reference numerals in each drawing indicate the same members.

Dynamic Adaptive Streaming over HTTP (DASH) enables 1) the delivery of media content from an HTTP server to an HTTP client, and 2) the content by standard HTTP caches. Specifies the formats that enable caching.

A media component is, for example, audio, video, or timed text with certain attributes, such as bandwidth, language or resolution. It is an encoded version of individual media types.

Media content is a collection of media components with a common timeline, such as video, audio and subtitles, for example. In addition, media components describe the relationships between how media components will be presented as a program or movie (e.g., individually, jointly or mutually exclusive). Have.

Media content and content may be used interchangeably.

A media presentation is a structured collection of data that establishes a bounded or unbounded presentation of media content composed of contiguous media components.

That is, media presentation is a structured collection of data accessible to DASH clients in order to provide streaming services to users.

The media presentation description (MPD) may be a formalized description of a media presentation.

Media presentation may be described by the MPD, including possible updates of the MPD.

The content may be content on demand or live content.

The content may be divided into one or more intervals. That is, the content may include one or more intervals.

Interval and period may be used interchangeably. The period may be a term of 3GPP (Generation Partnership Project) adaptive HTTP streaming.

The period may be an interval of media presentation. The successive sequence of all periods constitutes a media presentation.

That is, the media presentation may include one or more periods.

One or more intervals may be a basic unit. One or more intervals may be described by signaling metadata. That is, the metadata may describe each of one or more intervals.

The metadata may be MPD.

MPD may define a format for informing resource identifiers for segments. The MPD may provide context for identified resources within a media presentation. Resource identifiers may be HTTP-URL. URLs can be limited by the byte range attribute.

Each interval can be divided into fragments.

Fragment and segment may have the same meaning. Segment may be a term of 3GPP (Generation Partnership Project) adaptive HTTP streaming.

A segment is an object of a response to an HTTP/1.1 GET request for an HTTP-URL (or a GET request for a portion indicated by a byte range), for example defined in RFC 2616 ( entity) may mean a body. The metadata may include a URL attribute and a scope attribute. The client may receive bytes specified by the range attribute from the URL indicated by the URL attribute. That is, the bytes may be received by a partial HTTP GET command for the URL, indicated by the byte range. Here, the bytes designated by the range attribute may represent the aforementioned segment.

The range attribute can point to multiple byte ranges.

The client can use the received bytes (ie, segment) to play the media content.

The sub-segment may mean the smallest unit in segments that can be indexed by a segment index at the segment level.

There may be sets of two or more fragments corresponding to one interval. Each of the sets of fragments is called an alternative.

Alternatives and representations (or representations) may be used with the same meaning.

Each cycle may contain one or more groups.

Each group may contain one or more representations of the same media content.

A representation is a structured collection of one or more media components within a period.

The MPD (or MPD element) provides descriptive information that enables the client to select more than one representation.

Random Access Point (RAP) is a specific location within a media segment. The RAP is identified as being a location where playback can continue from the location of the RAP using only the information contained within the media segment.

Each representation may consist of one or more segments. That is, the representation may include one or more segments.

MPD is a document containing metadata required by a DASH client to 1) access segments, and 2) construct appropriate HTTP-URLs to provide streaming services to users. HTTP-URLs can be absolute or relative.

MPD can be an XML-document.

MPD may include an MPD element. MPD can contain only one MPD element.

The client can be a DASH client.

DASH clients can be compatible with clients specified in RFC 2616.

A DASH client may use the HTTP GET method or HTTP partial GET method in general, as specified in RFC 2616, to access segments or portions of segments.

1 is a conceptual diagram of spatially divided video content according to an exemplary embodiment.

As shown in FIG. 1, one content may be spatially divided into 24 segments, for example. Here, the segment may be referred to as a tile or a piece. Each segment may be assigned a segment identification (segment ID).

Meanwhile, a spatial relationship may be defined. The spatial relationship may be related to a video representing a spatial portion of a full-frame video. The spatial portion may be a region of interest or a tile described in terms of position and size. This technique may provide additional options for the client in terms of adaptation.

In addition, a spatial relationship description (SRD) may be defined. Spatial relationship description allows media presentation authors to express relationships between spatial objects. The spatial object may be represented by either an adaptation set or a sub-representation. The spatial relationship can represent a video that represents a spatial portion of a full-frame video. In FIG. 1, a video representing a spatial part of a full-frame video may be referred to as a spatial object. Alternatively, a group of videos representing a spatial part may be referred to as a spatial object. In order to ensure compatibility with legacy clients, MPD authors may use SupplementalProperty and EssentialProperty to be interpreted by legacy clients even after at least one representation discards the element containing EssentialProperty.

A DASH client that has grasped the spatial relationship description (SRD) can select a full-frame presentation or a spatial part of a full-frame presentation to suit the needs using SRD annotations. This can save bandwidth and client-side computation, resulting from avoiding full-frame fetching, decoding and cropping.

According to an embodiment, when the size occupied by each spatial segment is the same, a method of arranging the horizontal number, the vertical number and the spatial segment of the spatial segment is described, and in each spatial segment description unit, a simple number or an identifier that can know the order Can describe

For example, a state divided into 24 spatial segments as shown in FIGS. 1A and 1B may be arranged in a horizontal raster scan method or a vertical raster scan method. In this case, the information to be transmitted may include the number of horizontal segments, the number of vertical segments, and a direction in which segments are arranged.

In addition, the coordinates (x,y) of the upper left vertex of the “19” tile of FIG. 1A, the width (w) and height (h) of the tile, and the width (W) and height (H) of the full-frame video ) May also be included in the SRD and may be named as SRD parameters. Alternatively, the SRD may include an identifier of a source including a full-frame video or tile.

The source identifier (source_id) parameter may provide a unique identifier for the source of the content within the period. The source identifier can implicitly define a coordinate system related to the source; The coordinate system includes an arbitrary origin (0; 0), and the x-axis extends left and right, and the y-axis extends up and down. All SRDs sharing the same source identifier contain the same origin and coordinate axis. Spatial relationships for spatial objects using SRDs of different source identifier values are not defined.

Meanwhile, x and y parameters represent a two-dimensional position related to a spatial object in a coordinate system related to a source. The W and h parameters represent the two-dimensional size associated with the spatial object within the coordinate system associated with the source. The values of the X, y, w and h parameters may be relative to the values of the W and H parameters. The location (x,y) and size (w,h) of SRDs that share the same source identifier value can be compared after first considering the size of the reference space, that is, the x and w values are divided by the W value ( divided), y and h values are divided by H values.

Here, different W and H values may be used by different descriptors to provide location and size information within different units.

A group identifier can be defined. The group identifier may be an identifier indicating a group including tiles included in the basic view at the beginning of content playback or according to a user's request, and will be described in more detail later. Meanwhile, the group identifier may be referred to as a spatial set identifier (spatial_set_id). Here, the spatial set identifier has the same meaning as the group identifier, but may otherwise be defined as an identifier for a group of spatial objects.

2 is a flowchart illustrating a signal processing process according to an exemplary embodiment.

As shown in FIG. 2, the client 100 and the server 110 may communicate with each other. The client 100 may be a DASH client. DASH clients can be compatible with clients specified in RFC 2616.

A DASH client may use the HTTP GET method or HTTP partial GET method in general, as specified in RFC 2616, to access segments or portions of segments.

The server 110 may host DASH segments. The server 110 may be compatible with the server specified in RFC 2616.

In step 120, the client 100 receives 1) metadata for the content or 2) metadata for a specific interval of the content from the server 110. That is, the server 110 transmits 1) metadata about the content or metadata about a specific interval of the content to the client 100. The specific interval represents an interval currently processed among one or more intervals constituting the content.

Here, the metadata may be media presentation description (MPD) of media content. In addition, the media information may include an SRD, and the SRD may include at least one of a group identification and a tile identifier. The group identifier may be an identifier indicating a group including tiles included in a basic view when content playback starts or according to a user's request. In other words, the group identifier may be an identifier for grouping a set of spatial representations. For example, in the SRD as shown in FIG. 1(a), a tile group having a group identifier of “0” may be set as a group including tiles of 15, 16, 21, and 22. In addition, a tile group having a group identifier of "1" may be set as a group including 11, 12, 17, and 18. The group identifier may be 0 or a natural number.

Meanwhile, the content expressed by one of an adaptation set or a sub-representation may be defined as a spatial object. The group identifier may be defined as an identifier for a group of spatial objects.

For example, in step 120, it is assumed that the client 100 receives metadata including a group identifier of "1" from the server 110.

In step 130, the client 100 processes metadata for a specific interval. For example, the client 100 may check the group identifier and check the content to be received.

In step 140, the client 100 may request content or fragments from the server 110. The client 100 may request the server 110 for appropriate content for a specific interval based on the above metadata processing. That is, the requested content is selected based on the metadata. For example, the client 100 may request content including a tile group corresponding to the group identifier. In an embodiment, the client 100 may transmit a request including a URL corresponding to the tile group to the server 110. The request here may include multiple URLs. For example, the client 100 may transmit a request including a URL for each of the tiles including the tile identifiers 11, 12, 17, and 18 to the server 110.

In step 150, the server 110 transmits the appropriate content to the client 100 for each request from the client 100. The client 100 receives content from the server. For example, the client 100 may receive tiles including tile identifiers of 11, 12, 17, and 18, that is, a tile group.

Next, the whole process is executed from scratch. That is, step 120 is repeated for the next interval of a specific interval.

The client 100 may reproduce the content by using the received fragment as steps 120 to 150 are repeated.

As described above, the client 100 may receive content including partial tiles other than full-frame video. Accordingly, the bandwidth and the amount of computation can be significantly reduced.

Meanwhile, the above-described embodiment is merely exemplary, and the same may be applied to a tile identifier other than a group identifier. In more detail, the client 100 may receive metadata including a tile identifier. The client 100 may transmit a request including a URL corresponding to the tile identifier to the server 110. The server 110 may transmit the corresponding content to the client 100.

Hereinafter, a method of displaying a spatial segment according to various embodiments will be described.

3 is a hierarchical diagram of a method for displaying a spatial segment according to an exemplary embodiment.

As shown in Fig. 3, in order to display spatial segments in the period element of the MPD defined in DASH, the method of arranging the horizontal number, the vertical number and the spatial segment of the spatial segment is described, and adaptation, which is each spatial segment description unit. In the Set (AddaptationSet) element, a simple number or an identifier that can know the order can be described. That is, as follows, numOfColumns for the horizontal number of spatial segments, numOfRows for the vertical number, and typeOfNumbering are added to the period element in the order of listing, and a tile identifier (tileId) is added to each adaptation set to Be aware of the enemy location. In the embodiment of FIG. 3, it may be described that the horizontal number is 6, the vertical number is 4, and the method of arranging spatial segments is a horizontal raster scan method. The following may be semantics and XML for describing this.

numOfColumns: number of tile columns

numOfRows: number of tile rows

typeOfNumbering: numbering or scanning type of tiles

tileId: Sequential number to indicate a tile and its position

<xs:complexType name="PeriodType">

<xs:sequence>

<xs:element name="BaseURL" type="BaseURLType" minOccurs="0" maxOccurs="unbounded"/>

<xs:element name="SegmentBase" type="SegmentBaseType" minOccurs="0"/>

<xs:element name="SegmentList" type="SegmentListType" minOccurs="0"/>

<xs:element name="SegmentTemplate" type="SegmentTemplateType" minOccurs="0"/>

<xs:element name="AdaptationSet" type="AdaptationSetType" minOccurs="0" maxOccurs="unbounded"/>

<xs:element name="Subset" type="SubsetType" minOccurs="0" maxOccurs="unbounded"/>

<xs:any namespace="##other" processContents="lax" minOccurs="0" maxOccurs="unbounded"/>

</xs:sequence>

<xs:attribute ref="xlink:href"/>

<xs:attribute ref="xlink:actuate" default="onRequest"/>

<xs:attribute name="id" type="xs:string" />

<xs:attribute name="start" type="xs:duration"/>

<xs:attribute name="duration" type="xs:duration"/>

<xs:attribute name="bitstreamSwitching" type="xs:boolean" default="false"/>

<xs:attribute name="numOfColumns" type="xs:unsignedInt" use="optional" />

<xs:attribute name="numOfRows" type="xs:unsignedInt" use="optional" />

<xs:attribute name="typeOfNumbering" type="xs:string" use="optional" />

<xs:anyAttribute namespace="##other" processContents="lax"/>

</xs:complexType>

In another embodiment, a method of expressing a subset of spatial segments while individually describing and delivering contents of a plurality of spatial segments enables various applications. For example, a partial set of spatial segments may be separately provided in order to specify a spatial area that is the center of the screen, or an area in which an important event such as a decisive scene in sports occurs may be displayed. In addition, an area that becomes a basic view in the entire space can be displayed through a subset of spatial segments. To this end, partial set information including a plurality of identifiers of spatial segments may be separately provided, which may be a group identifier.

For example, in the case of DASH, a subset of tiles described as SRD in the MPD period can be displayed at the beginning of content playback through a new element called DefaultView or the tiles included in the basic view according to the user's request. have.

Alternatively, a separate element such as a tile group, which is a subset of tiles, may be added to the period to list the spatial segment identifier, or as another example, a subset of the elements currently included in MPD::Period It is also possible to expand. For example, an extended subset that extends the existing subset without invading the semantic of the existing subset (ExtendedSubSet) is defined, and an identifier not included in the subset is added to the extended subset, and the existing content attribute ( contains attribute) to express the spatial segment as shown below. The following may be semantics and XML for describing this.

TileGroup id = 0, contains = “15,16,21,22”

DefaultView id = 1, contains = “11,12,17,18”

ExtendedSubSet id = 0, contains = “15,16,21,22”

ExtendedSubSet id = 1, contains = “11,12,17,18”

<xs:complexType name="ExtendedSubsetType">

<xs:complexContent>

<xs:extension base="SubsetType">

<xs:attribute name="id" type="ID" use="optional" />

</xs:extension>

</xs:complexContent>

</xs:complexType>

4 is a hierarchical diagram of a method for displaying a spatial segment according to another exemplary embodiment.

The method according to the present embodiment is a method of allocating an index of each spatial segment. In other words, the current spatial segment description method has only identifier information, which is intended to indicate each spatial segment, but is not suitable for transmitting additional information of each spatial segment. For example, if additional additional information is not provided, only location information is provided for each spatial segment, and priority information or the like cannot be displayed. Accordingly, there is a need for a method for delivering additional additional information such as priority information of spatial segments and subset information of the same type. In this method, the index attribute is transmitted separately from the identifier information of each spatial segment, and the receiving terminal interprets the index information according to a predetermined method. For example, in the case of MPEG-DASH, an index attribute is added to each spatial segment information in order to deliver additional additional information of each spatial segment. The meaning and utilization method of the index is promised in advance between the author and the terminal. The meaning of the index can be used as the priority information, and the same index can be used as a subset. The following may be semantics and XML for describing this.

Role tileId = “ts1” index = “0”

Role tileId = “ts2” index = “0”

…

Role tileId = “ts24” index = “9”

5 is a structural diagram of a client 100 according to an embodiment of the present invention.

The controller 510 performs steps 130 and 140. That is, the controller 510 may process metadata for the interval and select the content suitable for the interval based on the processing.

The transceiver 520 performs steps 120, 140, and 150. That is, the transmission/reception unit 520 receives metadata about an interval of content from the server 110, requests a content suitable for the interval from the server, and receives the content from the server.

6 is a structural diagram of a client 100 according to an embodiment of the present invention.

The client 100 includes an access engine 610 and a media engine 620.

The access engine 610 may be a DASH access engine.

The access engine 610 receives metadata (eg, MPD) from the server 110.

The access engine 610 constructs the requests and issues the configured requests to the server 110.

The access engine 610 receives content (eg, segments, or portions of segments) from the server 110.

The access engine 610 provides the content to the media engine 620.

The output of the access engine 610 is the media (or media) of an MPEG container (eg, ISO/IEC 14492-12 ISO base media file format or ISO/IEC 13818-2 MPEG-2 transport stream). May include parts of). Further, the output of the access engine 610 may include timing information for mapping the internal timing of the media to a timeline of the media presentation.

The media engine 610 plays the provided content. That is, the media engine 610 may output media using media and timing information output from the access engine 610.

A method according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of the program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The above-described hardware device may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

As described above, although the present invention has been described by the limited embodiments and drawings, the present invention is not limited to the above embodiments, and various modifications and variations from these descriptions are those of ordinary skill in the field to which the present invention belongs. This is possible.

Therefore, the scope of the present invention is limited to the described embodiments and should not be defined, but should be defined by the claims to be described later as well as equivalents to the claims.

Claims (20)

In the media content providing method performed by a client,
Transmitting a request for a segment of media content to a server based on the metadata;
Receiving media content from the server
Including,
The metadata includes information on a spatial relationship description (SRD) for expressing a spatial relationship between spatial objects,
Media in which at least one of SupplementalProperty and EssentialProperty is used so that at least one representation can be interpreted by legacy clients after the element including EssentialProperty is discarded to ensure compatibility with legacy clients How to provide content. The method of claim 1,
The spatial relationship is a method of providing media content related to a video representing a spatial part of a full-frame video. The method of claim 1,
The method of providing media content in which the spatial object is represented by an adaptation set. The method of claim 1,
The spatial relationship description,
Including x parameter, y parameter, w parameter and h parameter,
The x parameter and y parameter represent a two-dimensional position related to a spatial object in a coordinate system related to a source,
The w parameter and the h parameter represent a two-dimensional position related to a spatial object in a coordinate system related to a source. The method of claim 1,
The metadata is,
Media content providing method including a spatial set identifier (spatial_set_id). The method of claim 1,
The spatial relationship description,
A method of providing media content including a source identifier (source_id) that is an identifier for a source of the media content within a period. The method of claim 6,
A spatial relationship description that shares the same source identifier is a method of providing media content having the same origin and axes orientation. delete delete delete In the media content providing method performed by the server,
Receiving a request for a segment of media content based on metadata from a client.
Transmitting media content to the client
Including,
The metadata includes information on a spatial relationship description (SRD) for expressing a spatial relationship between spatial objects,
Media in which at least one of SupplementalProperty and EssentialProperty is used so that at least one representation can be interpreted by legacy clients after the element including EssentialProperty is discarded to ensure compatibility with legacy clients How to provide content. The method of claim 11,
The spatial relationship is a method of providing media content related to a video representing a spatial part of a full-frame video. The method of claim 11,
The method of providing media content in which the spatial object is represented by an adaptation set. The method of claim 11,
The spatial relationship description,
Including x parameter, y parameter, w parameter and h parameter,
The x parameter and y parameter represent a two-dimensional position related to a spatial object in a coordinate system related to a source,
The w parameter and the h parameter represent a two-dimensional position related to a spatial object in a coordinate system related to a source. The method of claim 11,
The metadata is,
Media content providing method including a spatial set identifier (spatial_set_id). The method of claim 11,
The spatial relationship description,
A method of providing media content, including a source identifier (source_id) that is an identifier for a source of the media content within a period. The method of claim 16,
A spatial relationship description that shares the same source identifier is a method of providing media content having the same origin and axes orientation. delete delete delete

Images