KR20120035831A - Random access support method for streaming service based on scalabe video coding - Google Patents

Abstract

PURPOSE: A random access supporting method for a scalable video coding based streaming service is provided to support a scalable random access function by including an enhancement layer and a base layer in a DASH(Dynamic Adaptive Streaming over HTTP) service. CONSTITUTION: A segment index box includes an RAP(Random Access Point) index and an RAP(Random Access Point) offset. An RAP synchronization box provides synchronization information existed between SVC(Scalable Video Coding) video layer and an RAP by using the segment index box. A scalable random access function is provided by using the information of the RAP synchronization box and the segment index box.

Description

Random access for streaming services based on scalable video coding {RANDOM ACCESS SUPPORT METHOD FOR STREAMING SERVICE BASED ON SCALABE VIDEO CODING}

Embodiments of the present invention relate to a random access support method for scalable video coding based streaming service.

The Hyper Text Transfer Protocol (HTTP) streaming service, which is the basis of DASH (Dynamic Adaptive Streaming over HTTP), uses a terminal for HTTP to request a download of a content file, and the server transmits the requested file in response. TCP (Transmission Control Protocol) is used.

1 shows a basic service overview of an HTTP-based progressive streaming service.

The terminal transmits a GET Request of the HTTP protocol to inform the server of the service request for the desired content file, and the server uses TCP as a transmission protocol for transmitting the content file. The content file arriving at the terminal by TCP undergoes a buffering process for some data that arrives initially, and then plays out video and audio through a parsing process of the content file. Therefore, the HTTP streaming service is differentiated from the download method in which the playback process starts only after all data of the content file arrives at the terminal, and can significantly reduce service delay time, which is a problem of the existing download method.

The segment index box included in the DASH standard includes RAP (RAP) to support random access to video tracks included in movie fragments constituting the media segment as well as decoding time information about the media segment. Presence of Random Access Point) and the temporal location of RAPs. Since each media segment including compressed information of H.264 / Advanced Video Coding (AVC) typically serves only one representation, only one video track exists in the movie fragment constituting the media segment. . Accordingly, the segment index box of the DASH standard provides information on whether there is a RAP for a video track existing one by one for each movie fragment constituting a media segment and the time position of the RAPs.

The DASH standard allows the segment index box to be included for each media segment when constructing the media segment, which is the unit of transmission in the HTTP streaming service. The segment index box is located at the front of the media segment containing several movie fragments.

2 shows a structure of a media segment including six movie fragments F1 to F6 and a segment index box.

The segment index box provides information related to random access point (RAP) information and decoding time information for F1, F3, and F5 of the video track included in the movie fragment.

The following shows the syntax included in the segment index box of the DASH standard. The segment index box is for the case of including the H.264 / AVC bitstream in the movie fragment. Since the segment index box includes only one video layer, only the information of one video track is displayed.

aligned (8) class SegmentIndexBox extends FullBox ('sidx' version, 0)

{

unsigned int (32) reference_track_ID;

if (version == 0)

{

          unsigned int (32) earliest_composition_time;

}

else

{

 unsigned int (64) earliest_composition_time

}

unsigned int (16) reference_count;

for (i = 1; i <= reference_count; i ++)

{

 bit (1) reference_type;

unsigned int (31) reference_offset;

unsigned int (32) ment subsegment_duration;

bit (1) tain contains_RAP;

unsigned int (31) RAP_delta_time;

}

}

An embodiment of the present invention proposes a syntax and semantics for supporting scalable random access in consideration of the layered structure of scalable video coding (SVC). A method of supporting random access for a streaming service based on scalable video coding is provided.

In addition, an embodiment of the present invention provides a scalable random access function for enabling random access in a scalable video coding-based DASH service including not only a base layer of an SVC but also an enhancement layer. A random access support method for a scalable video coding based streaming service is provided.

In addition, an embodiment of the present invention provides a random access support method for a scalable video coding-based streaming service that enables a scalable random access function in a scalable video coding-based HTTP streaming service.

According to another aspect of the present invention, there is provided a random access support method for a scalable video coding-based streaming service, including: defining a segment index box including a random access point (RAP) index and a RAP offset; Defining a RAP sync box providing sync information between RAPs present in a scalable video coding (SVC) video layer using the segment index box; And providing a scalable random access function using information defined in the segment index box and the RAP sink box.

According to an embodiment of the present invention, a scalable video coding-based DASH service supports a scalable random access function for enabling random access including not only the base layer of the SVC but also an enhancement layer, and is included in the DASH standard. By extending the segment index box and expanding the segment index box, video tracks including various enhancement layers of the SVC stored in the SVC file based on the ISO file format can be utilized for random access, thereby enabling the performance of the terminal and the available network bandwidth. It enables an environment adaptive scalable random access service considering the

1 shows a basic service overview of an HTTP-based progressive streaming service.
2 shows a structure of a media segment including six movie fragments F1 to F6 and a segment index box.
3 illustrates a structure of a media segment capable of enhancing the quality of a base quality video in four steps by using enhancement layers of the SVC video according to an embodiment of the present invention.
FIG. 4 is an enlarged view of an internal structure of movie fragments including a base layer and an enhancement layer of an SVC in a media segment shown in FIG. 3 in units of a group of pictures (GoP).
FIG. 5 is a diagram illustrating media segment parts transmitted by an HTTP streaming server when a client requests a service for video quality from a RAP # (n + 1) position to an enhanced quality 3 according to an embodiment of the present invention. The structure is shown.
6 shows a structure of a media segment including a RAP Synchronization Box ('raps') according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited or limited by the embodiments. Like reference numerals in the drawings denote like elements.

One embodiment of the present invention proposes a technique for effectively representing RAP information required when providing an HTTP streaming service using a media segment including SVC video as well as H.264 / AVC.

When the HTTP streaming server utilizes a media segment including SVC video to provide a DASH service, the HTTP streaming server includes a sink between the SVC base layer included in the multiple movie fragment constituting the media segment and the RAPs included in the enhancement layer. In terms of synchronization, they need to be related. In this case, the HTTP streaming server provides enhanced quality by additionally servicing the enhancement layer as a complementary representation of the base quality as well as the base quality that the base layer can provide for random access requests from clients. Can be provided. In other words, the HTTP streaming server enables random access to the enhancement layers that are synchronized with the SVC base layer simultaneously, thus enabling scalable adaptive random access service considering the performance of the client terminal and the available network bandwidth.

3 shows a structure of a media segment in which an HTTP streaming server can enhance the quality of a base quality video in four steps by utilizing enhancement layers of SVC video.

The movie fragment F_E1 includes an enhancement layer that can be used as a complementarity expression for F_B, and the movie fragment F_E2 includes an enhancement layer that can be used as a complementarity expression for F_E1. F_E3 to F_E4 also show similar correlations.

F_B and F_E1 to F_E4 respectively include compressed data generated in a corresponding layer from SVC layers belonging to the same time frame. There is a reference offset field (reference_offset field) in the segment index box. The information indicates how far from the segment index box the movie fragments included in the media segment are in bytes.

Therefore, the type of movie fragments required for transmission is determined according to whether the video quality required for the DASH service is base quality or enhanced quality, and the HTTP streaming server extracts from the media segment through byte position information on the fragments. Can be easily determined the data location of the fragments to be transmitted. In the SVC-based DASH service utilizing the media segment having the structure shown in FIG. 3, an HTTP byte range request is sent to the server to request a video quality desired by the client, and the server requests the quality required by the client. Complementary representations suitable for satisfying s are extracted from the media segment by the virtual segment and transmitted.

In the SVC-based DASH service, the HTTP streaming server may provide a quality adaptive HTTP streaming service by utilizing multiple complementarity expressions by the base layer and the enhancement layer. Therefore, HTTP streaming service that can provide environment-adaptive video quality by comprehensively considering the performance of the client terminal and the available downlink transmission rate of the network experienced by the client becomes possible. The DASH standard includes a segment index box for each media segment to indicate the presence and time of RAP, so that ISO files containing H.264 / AVC video can be used for DASH services. . If the random access function is also applied to the SVC-based DASH service, a scalable random access service that is adaptive to a given service environment becomes possible. In other words, random access is possible not only to the SVC base layer but also to enhancement layers in which the base layer and inter-layer synchronization are established, so that the environment-adaptive scalable random considering the performance of the client terminal and the available network bandwidth is possible. Access service is available.

FIG. 4 is an enlarged representation of an internal structure of movie fragments including a base layer and an enhancement layer of an SVC in the media segment shown in FIG. 3 in GoP units.

As shown in FIG. 4, the movie fragments F_B and F_E1 to F_E4 included in one media segment each include compression information of a video layer belonging to the same time frame period. Each movie fragment includes compression information of a video layer belonging to a time frame section consisting of GoP #n to GoP # (n + 2) corresponding to a total of three GoPs. That is, RAP #n exists at the beginning of GoP #n, RAP # (n + 1) exists at the beginning of GoP # (n + 1), and RAP at the beginning of GoP # (n + 2). # (n + 2) is present. If the client requests random access for the GoP # (n + 1) interval, the HTTP streaming server transmits the base layer video from the RAP # (n + 1) point of F_B to provide the random access service for the base quality. It can be provided. However, if RAP information in which interlayer sync is established with GoP # (n + 1) of F_B is provided for F_E1 to F_E4, not only the base quality random access function by the base layer transmission of F_B, but also the GoP of F_E1 to F_E4 Enhanced quality random access is possible by additionally transmitting data corresponding to # (n + 1). RAPs corresponding to the same random access time may exist in F_B and F_E1 to F_E4. Such a set of RAPs is called scalable RAPs. In the embodiment of FIG. 4, since there are five SVC video layers in total, a set of scalable RAPs including up to five RAPs for each of RAP #n, RAP # (n + 1), and RAP # (n + 2) may be provided. It is possible to configure scalable RAPs #n, scalable RAPs # (n + 1), and scalable RAPs # (n + 2). If the HTTP streaming server uses RAPs belonging to the same Scalable RAPs and belonging to different SVC layers, scalable random access service is possible.

In the exemplary embodiment of FIG. 4, a total of three GoPs are included in each movie fragment, and there are three RAPs per movie fragment assuming that random access is supported in an IDR picture, which is the beginning of each GoP. That is, RAP #n exists at the beginning of GoP #n, RAP # (n + 1) exists at the beginning of GoP # (n + 1), and RAP at the beginning of GoP # (n + 2). # (n + 2) is present. In addition, there may be one RAP at a specific random access location in F_B and F_E1 to F_E4, respectively, and such a set of RAPs is called scalable RAPs. The HTTP streaming server can support scalable random access, which can provide various video qualities for the random access request of the client by utilizing a set of scalable RAPs. In the embodiment of FIG. 4, since there are five SVC video layers in total, a scalable RAPs set including up to five RAPs for each of RAP #n, RAP # (n + 1), and RAP # (n + 2) may be provided. It is possible to configure scalable RAPs #n, scalable RAPs # (n + 1), and scalable RAPs # (n + 2).

FIG. 5 illustrates a structure of a media segment part transmitted by an HTTP streaming server when a client requests a service for video quality from a RAP # (n + 1) position to enhanced quality 3. FIG.

Since the client requested random access from RAP # (n + 1), the RAPs corresponding to the set of scalable RAPs # (n + 1) become targets of the random access point, so from GoP # (n + 1) of each movie fragment It is the object of transmission. However, since the client requests the quality up to enhanced quality 3, transmission starts from GoP # (n + 1) only for movie fragments F_B to F_E3. In other words, F_E4 necessary to provide enhanced quality 4 or higher is excluded from transmission.

In one embodiment of the present invention, an existing segment index box is extended to efficiently provide a scalable random access function. In addition, the HTTP streaming server newly proposes RAP sink boxes (raps) to provide RAP sync information between RAPs existing in SVC video layers by using an extended segment index box. The RAP Synchronization box is located after the extended segment index box, and the structure of the media segment including the RAP sync box is shown in FIG. 6.

The structures of the extended segment index box and RAP synchronization box (raps) proposed to support scalable random access applicable to an SVC-based DASH service are as follows.

The extended segment index box sidx simplifies the structure of the RAP sink box by adding a RAP index field and a RAP offset field to the existing segment index box.

aligned (8) class SegmentIndexBox extends FullBox ('sidx' version, 0)

{

unsigned int (32) reference_track_ID;

unsigned int (16) track_count;

unsigned int (16) reference_count;

for (i = 1; i <= track_count; i ++)

{

unsigned int 32 track_ID;

if (version == 0)

{

unsigned int 32 decoding_time;

} else

{

unsigned int 64 decoding_time;

}

}

for (i = 1; i <= reference_count; i ++)

{

bit (1) reference_type;

unsigned int (31) reference_offset;

unsigned int (32) subsegment_duration;

bit (1) contains_RAP;

unsigned int (31) RAP_count;

for (j = 1; j <= RAP_count; j ++)

{

unsigned int (32) RAP_index; // newly added

unsigned int (32) RAP_delta_time;

    unsigned int (32) RAP_offset; // newly added

}

}

The RAP index field RAP_index field and RAP offset field RAP_offset field added to the extended segment index box sidx are as follows.

RAP index (RAP_index) is an index for the RAP points existing in each movie fragment,

The RAP offset RAP_offset is a distance in bytes from the reference offset offset_offset to the position where the RAP point is located.

In the RAP sync box, RAPs of a lower SVC layer synchronized with RAPs of an SVC layer included in a specific movie fragment are associated with each other. To this end, the RAP sync relationships existing between SVC video layers can be described simply and efficiently by utilizing the RAP_sequence_number added to the extended segment index box.

aligned (8) class RAPSynchronizationBox extends FullBox ('raps' version, 0)

{

unsigned int (8) svc_layer_count;

for (i = 1; i <= svc_layer_count-1; i ++) {

                 unsigned (8) current_svc_layer_ID;

           unsigned (8) RAP_count_of_current_svc_layer;

           for (j = 1; j <= RAP_count_of_current_svc_layer; j ++) {

                  unsigned (8) RAP_index_of_current_svc_layer;

                 unsigned (8) RAP_synchronized_lower_svc_layer_ID;

                  unsigned (8) RAP_index_of_lower_svc_layer;

                 }

       }

}

The fields included in the RAP sync box are as follows.

svc_layer_count is the sum of the number of SVC base layers representing the base quality and the SVC enhancement layers representing the enhanced quality,

current_svc_layer_ID is the ID of the current SVC layer to which the RAP sink information is set.

RAP_count_of_current_svc_layer can be known from the segment index box as the number of RAPs present in the current SVC layer.

RAP_index _of_current_svc_layer is the index of RAPs existing in the current SVC layer,

RAP_synchronized_lower_svc_layer_ID is the ID of the lower SVC layer that is RAP synced with the RAP of the current SVC layer.

RAP_index_of_lower_svc_layer is a RAP_index in the lower SVC layer in which the RAP sink is established with the RAP_index_of_current_svc_layer.

In one embodiment of the present invention, the scalable random access function is enabled by efficiently providing RAP sync information existing between SVC video layers for an SVC-based DASH service. To this end, RAP_index and RAP_offset are added to the segment index box included in the existing DASH standard. In addition, a RAP sync box is newly proposed by using RAP_index to associate RAPs in a lower SVC layer synchronized with RAPs of an SVC layer included in a specific movie fragment.

Further, embodiments of the present invention include a computer readable medium having program instructions for performing various computer implemented operations. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

As described above, the present invention has been described by specific embodiments such as specific components and the like. For those skilled in the art to which the present invention pertains, various modifications and variations are possible. It is therefore to be understood that within the scope of the appended claims, the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. .

F_B: a movie fragment containing base layer
F_En: a movie fragment containing enhancement layers to be used as a complementary representation to the base quality or the lower enhanced quality

Claims (1)

Defining a segment index box including a random access point (RAP) index and a RAP offset;
Defining a RAP sync box providing sync information between RAPs present in a scalable video coding (SVC) video layer using the segment index box; And
Providing a scalable random access function using information defined in the segment index box and the RAP sink box
The random access support method for a scalable video coding-based streaming service comprising a.

Images