KR101803082B1 - Container Generation Method for Ultra High Definition Scalable Video Streaming Services - Google Patents

Abstract

A method for generating a container for an ultra high definition scalable video streaming service is provided. A video streaming method according to an embodiment of the present invention includes the steps of: generating a scalable encoded video stream; separating the stream according to a layer; constructing the containers by storing encoding information for each intra frame according to the layer; and transmitting the constructed containers. Accordingly, the streaming service can be provided for an ultra high definition (4,8K) video which is scalably encoded.

Description

[0001] Container Generation Method for Ultra High Definition Scalable Video Streaming [

The present invention relates to a video streaming service technology, and more particularly, to a container creation method for an ultra-high-quality scalable video streaming service.

With the development of displays, various handsets capable of supporting full-HD or higher resolution are being produced. Many consumers, therefore, want to watch high-definition video at the time they want on a variety of terminals (high-definition content on mobile handsets, laptops, desktops, TVs, etc.). Therefore, there is a high demand for an ultra-high-definition video streaming service capable of providing service in various displays.

Ultra-high-quality images are so large that they require a lot of time and resources to encode. Therefore, it is very inefficient to encode the same image to fit different resolutions. Scalable encoding (SVC, SHVC, etc.) hierarchically encodes super-high-resolution images to satisfy terminals with various resolutions in a single encoding.

For video streaming services, encapsulation must be done in a format suitable for transmitting the encoded video. However, there is no definition and method of encapsulation format for streaming scalable encoded ultra high definition (4,8K) video.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a container generation method of a format capable of streaming scalable encoded super high definition (4, 8K) video.

According to an aspect of the present invention, there is provided a video streaming method including: generating a scalable encoded video stream; Separating streams into layers and constructing containers by storing encoding information for each intra frame in each layer; And transmitting the configured containers.

The encoding information contained in the container of the first layer includes a first parameter set for video, a second parameter set for the first layer, a second parameter set for the second layer, and a third parameter for the first layer And the encoding information contained in the container of the second layer may include a third set of parameters for the second layer.

The encoding information contained in the container of the first layer may include a first parameter set for the video, a second parameter set for the first layer, a second parameter set for the second layer, a second parameter for the third layer, And a third parameter set for the first layer, the encoding information included in the container of the second layer includes a third parameter set for the second layer, and encoding information May include a third set of parameters for the third layer.

The first parameter set may be a VPS (Video Parameter Set), the second parameter set may be an SPS (Sequence Parameter Set), and the third parameter set may be a PPS (Picture Parameter Set).

According to another aspect of the present invention, there is provided a broadcast transmission system including: a content providing server for generating broadcast content; And a broadcasting server which separates the scalable encoded video streams generated from the broadcasting contents into layers and constructs containers by storing encoding information for each intra frame in each layer and transmits the configured containers.

Meanwhile, according to another embodiment of the present invention, a video receiving method includes: receiving a scalable encoded video; And reproducing the received video, wherein the video includes containers configured by separating the scalable encoded video stream into layers and storing encoding information for each intra frame in each layer.

The encoding information contained in the container of the first layer includes a first parameter set for video, a second parameter set for the first layer, a second parameter set for the second layer, and a third parameter for the first layer And the encoding information contained in the container of the second layer may include a third set of parameters for the second layer.

The encoding information contained in the container of the first layer may include a first parameter set for the video, a second parameter set for the first layer, a second parameter set for the second layer, a second parameter for the third layer, And a third parameter set for the first layer, the encoding information included in the container of the second layer includes a third parameter set for the second layer, and encoding information May include a third set of parameters for the third layer.

The first parameter set may be a VPS (Video Parameter Set), the second parameter set may be an SPS (Sequence Parameter Set), and the third parameter set may be a PPS (Picture Parameter Set).

Meanwhile, in a video receiving terminal for receiving and reproducing scalably encoded video according to another exemplary embodiment of the present invention, the video may include scalable encoded video streams separated by layers, encoding information for each intra frame, Containers are also included.

As described above, according to the embodiments of the present invention, it is possible to provide a streaming service even for a super-high-definition (4,8K) video which is scalably encoded, thereby increasing the satisfaction of the user.

1 is a flow chart provided in the description of a video streaming service procedure,
2 shows a structure of a NAL stream,
FIG. 3 is a diagram for comparing three-layer scalable image sizes,
FIG. 4 is a diagram illustrating a scalable encoded NAL stream in three layers as shown in FIG. 3,
Figure 5 shows the init file structure of MP4,
Figure 7 is a diagram provided in the description of a first layer container,
8 is a diagram provided in the description of a second layer container,
9 is a diagram provided in the description of a third layer container,
10 is a configuration diagram of a broadcasting system to which an embodiment of the present invention is applicable.

Hereinafter, the present invention will be described in detail with reference to the drawings.

1. Video streaming service

Figure 1 is a flow chart provided in the description of a video streaming service procedure. As shown in FIG. 1, first, video data is acquired through a camera (S10), and scalable encoding is performed (S20).

The number of layers of video encoded according to the encoding information can be changed. For the next step, the scalable encoded video is separated into encoding information (VPS, SPS, PPS) and layers (S30, S40).

Then, the encoding information is encapsulated in a container format at a necessary position so that the user can arbitrarily access (S50).

2. Scalable encoded video stream

2 is a diagram showing a structure of a NAL stream. The Network Abstract Layer (NAL) is a layer for transmitting encoded video data over a network, and is composed of a plurality of NALU (NALU) sets, and is generated in a stream format.

NALU consists of a 2-byte NALU header and a NALU payload (RBSP: raw byte sequence payload). One unit separated in Fig. 2 is one NALU. NALU is classified as VCL, VPS, SPS and PPS.

The VCL (Video Coding Layer) is a NALU in which compressed data is compressed to h.265. VPS (Video Parameter Set) is a NALU in which encoding information for the entire video is recorded, and this information is generated only in scalable encoding.

An SPS (Sequence Parameter Set) is a NALU in which encoding information for each layer is stored, and is generated by the number of layers generated at the time of scalable encoding. A PPS (Picture Parameter Set) has encoding information for one picture.

FIG. 3 is a diagram for comparing 3-layer scalable image sizes, and FIG. 4 is a diagram illustrating NAL stream_1 scalar-encoded in three layers as shown in FIG.

As shown in Fig. 4, only one VPS is generated per content, and the SPS is sequentially generated in the number of scalable layers. Like the SPS, PPS is generated as many as the number of layers, and its position is generated one by one before the IDR (intra frame) of each layer. That is, the first layer PPS is generated before the first layer IDR NALUs, the second layer PPS is generated before the second layer IDR NALUs, and the third layer PPS is generated before the third layer IDR NALUs.

3. Box structure

MP4 codecs based on ISOBMFF support streaming mode. To do this, the encoded video must be separated into encoding information and actual data for each layer.

For streaming, the MP4 codec separates the codec information (content information) and the content into a logical structure and generates an init file including a trak box for loading the corresponding media information. Its structure is shown in Fig. When the video is scalably encoded into three layers, each layer is separated into traks, and SPS and PPS having encoding information of each layer are recorded in the trak box.

4. How to create a container for streaming

The streaming service must support the user's random access to the service. To this end, the streaming server must encapsulate the scalable encoded video data and the decode information together in a container format. Through the generated transfer file, the user can use the service at any time.

6 is a three-layer scalably encoded video stream _2. In FIG. 6, I 'denotes an intra frame, which is an important frame which is a key frame and is a reference frame in decoding another frame. IDR, BLA, and CRA frames. The type field in the NALU header tells what kind of frame each NALU is.

In the order of NALU, VPS, SPS, and PPS are repeatedly inserted every time an intra frame occurs in the first layer. In the case of the second layer and the third layer, every time an intra frame comes out, the PPS for that layer is repeated for each intra frame. That is, the second layer repeatedly inserts the PPS for the second layer every time an intra frame comes out, and the third layer repeatedly inserts the PPS for the third layer every time an intra frame comes out. The VPS is inserted only in the first layer.

The streams are divided into a first layer, a second layer and a third layer, respectively, and are configured according to a container format. At this time, each layer is segmented at a constant time interval (1 second, 2 seconds, etc.). The separated segments are encapsulated according to the container structure. If the intra frame is included in the segment, encapsulation is performed by inserting encoding information together.

4.1 Creating the first tier container

The styp box contains the segments that are split. Segments are created as styp boxes. The actual video data is inserted behind the mdat box. Depending on the time interval and encoding options, the segments may contain intra frames. At this time, information (VPS, SPS, PPS) necessary for decoding is inserted in front of the intra frame as shown in FIG. Specifically, 1) VPS for content / video, 2) SPS for the first layer, the second layer and the third layer, and 3) the PPS for the first layer are inserted.

4.2 Creating the first tier container

The basic content is the same as the first layer. However, the encoding information to be inserted in front of the intra frame only inserts the PPS for the second layer except for the VPS.

4.3 Creating a third-tier container

The encoding information to be inserted in front of the intra frame is the PPS for the third layer. The third layer container generation method is the same as that of the second layer, and a detailed description thereof will be omitted.

5. Broadcasting system

10 is a configuration diagram of a broadcasting system to which an embodiment of the present invention is applicable. 10, the broadcasting system includes a content provider 110, an MMT-based service generator 120, a broadcasting network 130, an MMT-based server 140, and an MMT-based terminal 150.

The MMT service generator 120 receives the video and the like supplied from the content provider 110 and adds the encoded media data and the synchronization information to each other to encapsulate And MMT packets. The content provider 110 and the MMT service generator 120 may be one broadcasting station or may be located at physically separated locations to perform respective roles.

The MMT-based service generator 120 encodes the UHD image and separates the UHD image into a base layer and an additional layer so as to have spatial scalability, and transmits the base layer to the broadcasting network 130 and the additional layer to the MMT-based server 140 .

In this case, the MMT-based terminal 150 can receive the base layer through the broadcasting network 130 to display the HD image, and additionally receive the additional layer through the network to provide the UHD image to the user .

In addition, the MMT-based service generator 120 provides the streaming service for the UHD image based on the container creation method described in "4."

6. Variations

It goes without saying that the technical idea of the present invention can also be applied to a computer-readable recording medium having a computer program for performing the functions of the apparatus and method according to the present embodiment. In addition, the technical idea according to various embodiments of the present invention may be embodied in computer-readable code form recorded on a computer-readable recording medium. The computer-readable recording medium is any data storage device that can be read by a computer and can store data. For example, the computer-readable recording medium may be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, or the like. In addition, the computer readable code or program stored in the computer readable recording medium may be transmitted through a network connected between the computers.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present invention.

110: Content provider
120: MMT based service generator
130: Broadcast network
140: MMT based server
150: MMT based terminal

Claims (12)

Generating a scalable encoded video stream;
Separating the streams into layers;
Encoding information including a VPS (Video Parameter Set) for a video, a SPS (Sequence Parameter Set) for a first layer, an SPS for a second layer, and a PPS (Picture Parameter Set) for a first layer is recorded for each intra frame A first configuration step of configuring a container of a first hierarchy;
A second constructing step of constructing a second layer container by storing encoding information including a PPS for the second layer for each intra frame; And
And transmitting the configured containers,
In the transmitting step,
Wherein a container of the first layer is transmitted through the first network and a container of the second layer is transmitted through the second network.
The method according to claim 1,
The encoding information,
Wherein the video stream is preceded by an intra frame.
delete The method according to claim 1,
The first configuration step comprises:
A first layer container is constructed by storing encoding information including an SPS for the third layer for each intra frame,
In the video streaming method,
And a third configuration step of constructing a third layer container by storing encoding information including a PPS for the third layer for each intra frame,
In the transmitting step,
And the third layer of containers is transmitted through the second network.
delete A content providing server for generating broadcast content;
And a broadcasting server for separating the scalable encoded video streams generated from the broadcast content into layers and configuring containers by storing encoding information for each intra frame in each layer,
The broadcast server,
Encoding information including a VPS (Video Parameter Set) for a video, a SPS (Sequence Parameter Set) for a first layer, an SPS for a second layer, and a PPS (Picture Parameter Set) for a first layer is recorded for each intra frame Thereby constituting a container of the first hierarchy,
A second layer container is constructed by storing encoding information including a PPS for the second layer for each intra frame,
Wherein the first layer container is transmitted through the first network and the second layer container is transmitted through the second network.
Receiving scalably encoded video;
And playing the received video,
In the video,
Containers configured by separating scalable encoded video streams by hierarchy and encoding information for each intra frame,
In the container of the first layer,
Encoding information including a VPS (Video Parameter Set) for a video, a SPS (Sequence Parameter Set) for a first layer, an SPS for a second layer, and a PPS (Picture Parameter Set) for a first layer is recorded for each intra frame And,
In the container of the second layer,
Encoding information including PPS for the second layer is recorded for each intra frame,
In the receiving step,
Wherein a container of a first layer is received via a first network and a container of a second layer is received via a second network.

The method of claim 7,
The encoding information,
Wherein the video frame is preceded by an intra frame.
delete The method of claim 7,
In the container of the first layer,
The SPS for the third layer is further included,
In the container of the third hierarchy,
Encoding information including PPS for the third layer is recorded for each intra frame,
In the receiving step,
And a third layer of containers is received over the second network.
delete A video receiving terminal for receiving and reproducing scalably encoded video,
The video includes containers configured by separating scalable encoded video streams by hierarchy and encoding information for each intra frame,
In the container of the first layer,
Encoding information including a VPS (Video Parameter Set) for a video, a SPS (Sequence Parameter Set) for a first layer, an SPS for a second layer, and a PPS (Picture Parameter Set) for a first layer is recorded for each intra frame And,
In the container of the second layer,
Encoding information including PPS for the second layer is recorded for each intra frame,
The video receiving terminal,
Wherein a container of the first hierarchy is received through the first network and a container of the second hierarchy is received through the second network.

Images