KR20110123644A - File format structure for multimedia streaming service and method and apparatus for servicing multimedia streamming using the same - Google Patents

Abstract

PURPOSE: A multimedia streaming file format structure and a method and apparatus for a multimedia streaming service using the same are provided to transmit multimedia data before all multimedia data corresponding to one fragment is received. CONSTITUTION: A file type box(110) describes the type of a content file. A movie metadata box(120) describes meta data related to the content file. A media data box(131,132,141,142) stores multimedia data. A movie fragment metadata box(136,146) records meta data related to the multimedia data. One fragment(130,140) includes a plurality of media data boxes and one movie metadata box. The plurality of media data boxes are located in front of the movie meta data box.

Description

File format structure for multimedia streaming service and method and apparatus for servicing multimedia streamming using the same}

The present invention relates to a multimedia streaming service system, and more particularly, to a multimedia streaming file format structure capable of minimizing latency in a live multimedia streaming service based on HTTP (hypertext transfer protocol), and a multimedia streaming service method using the same. Relates to a device.

Recently, a streaming service for transmitting a video through the Internet or a mobile communication network is widely used.

In general, a streaming service is a multimedia service that removes multimedia data after playing without storing the multimedia data in a user's terminal. The streaming service is widely used in a wired communication network, but is very useful for a mobile communication terminal such as a mobile phone that lacks a separate storage space.

In particular, streaming services based on hypertext transfer protocol (HTTP), which can utilize most existing web infrastructures, are gradually increasing.

Typically, the file format for live HTTP streaming uses fragmented mp4.

Therefore, the existing live HTTP streaming service transmits and receives data in units of fragments between the server and the client.

In other words, since the size of the entire multimedia data cannot be known in advance, the server transmits the fragments by fragments every few seconds.

However, if you use the existing fragment mp4 file format to service live HTTP streaming, the server receives the fragment request and sends data in units of fragments. At this time, the server generates latency as much as the fragment time unit (f seconds). The client requires more than a certain amount of buffering time (b seconds) for jitter compensation and audio / video synchronization. Therefore, the server capture time and the client render time differ by at least (f + b) seconds.

An object of the present invention is to provide a multimedia streaming file format structure capable of minimizing latency between a server and a client in an HTTP-based live multimedia streaming service, and a multimedia streaming service method and apparatus using the same.

 In order to solve the above problems, in the multimedia streaming file format structure according to an embodiment of the present invention,

An M-dot box for storing multimedia data;

A muff box for recording meta data related to the multimedia data,

One fragment box is composed of a plurality of m-dot boxes for dividing and storing the multimedia data and one mof box for recording metadata of the multimedia data stored in the plurality of m-dot boxes,

The plurality of m-dot boxes are positioned in front of the muffle box.

The streaming file may be formatted by adding an F-tip box describing the type of the content file and a move box describing the multimedia metadata to the at least one fragment.

The minimum unit of the m-dot box is characterized in that the frame unit.

The minimum unit of the m-dot box is characterized in that the slice unit.

The maximum unit of the M-dot box is characterized in that the entire audio / video data in the fragment.

The muffle box is characterized in that it comprises a fragment size box.

In order to solve the other problems described above, in the Tmedia streaming service method according to an embodiment of the present invention,

Receiving a request for multimedia data in units of fragments;

Transmitting the requested multimedia data by dividing it into units of a plurality of M-dot boxes;

When the transmission of the multimedia data in units of the M dot box is completed, the method includes transmitting a muff box that records the meta data of the multimedia data to the client.

The fragment is

A plurality of m-dot boxes for dividing and storing multimedia data and one muff box associated with the plurality of m-dot boxes,

The plurality of m-dot boxes are characterized in that located in front of the muffle box.

The minimum unit of the M-dot box is characterized in that the frame unit or slice unit.

The maximum unit of the M-dot box is characterized in that the entire image / audio data in the fragment.

The frame size box is further added to the muffle box for transmission.

The M-dot box is first identified by the frame size box added in the muff box.

The client may be configured to play the multimedia data received in units of fragments and the multimedia data in units of fragments received after the multimedia transmission is completed.

In order to solve the above other problem, in the multimedia streaming service apparatus according to an embodiment of the present invention,

A file mux section constituting one fragment with a plurality of m-dot boxes and one muft box for dividing and storing multimedia data;

Receiving a fragment file request from a client and transmits the multimedia data divided into a plurality of M box unit, and the web server unit for transmitting the muffle box recording the meter data when the M box transmission is completed.

1 is a structural diagram of a multimedia streaming file format according to an embodiment of the present invention.
2 is a block diagram of a multimedia streaming service system according to an embodiment of the present invention.
3 is a flowchart of signals and data in a multimedia streaming service system according to an embodiment of the present invention.
4 is a flowchart of signals and data in the multimedia streaming service system of the existing technology for explaining the difference from the present invention.
5 is a flowchart illustrating a multimedia streaming service method according to an embodiment of the present invention.
6 is a flowchart illustrating a multimedia streaming processing method in a client according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings. In addition, in the present specification and drawings, components having substantially the same functional configuration will be omitted by the same reference numerals.

1 is a file format structure diagram of multimedia streaming according to an embodiment of the present invention.

The file format of the multimedia streaming of FIG. 1 is a ftyp (file type) 110, a move box (moov: movie metadata) 120, a plurality of fragments (130, 140), mpra box (mfra: movie fragment random access) 150.

Ftip box 110 describes the type of content file. The move box 120 describes the metadata related to the content file, for example, image data location, image data size, and the like.

The mfra box 150 stores location information of each fragment.

The fragment 130 or 140 includes a plurality of m-dot boxes 131, 132 or 141, 142 and a mof box 136 or 146. One fragment 130 or 140 contains a few seconds of multimedia data.

M-dot boxes (mdat: 131, 132 or 141, 142) stores the multimedia data divided into a plurality of units.

Mof box 136 or 146 records metadata related to the multimedia data stored in the M-dot boxes.

Here, the fttip box 110, the move box moov 120, and the mfra box 150 are identical to the existing fragmented mp4 file format.

However, the streaming file format format of the present invention and the existing fragmented mp4 file format differ in the elements constituting the fragment 230 or 240.

The existing fragmented mp4 file format composes a fragment into one muff box and one mdot box. And the Mute Box is located in front of the M Dot Box.

However, the improved streaming file format of the present invention is divided into a plurality of M. boxes for dividing and storing multimedia data, and one fragment box for recording metadata of multimedia data stored in the plurality of M. boxes. 130 or 140, and a plurality of m-dot boxes (131, 132 or 141, 142) are placed in front of the muffle box (136 or 146).

In addition, the minimum unit of one M. dot box is a frame unit or a slice unit, and the maximum unit of one M. dot box is all audio / video data in a fragment.

The Mute Box also adds a fragment size box for easy access to the first MD Box. Thus, the M-Dot box can be identified the first time by the frame size box added in the Mute box.

2 is a block diagram of a multimedia streaming service system according to an embodiment of the present invention.

The multimedia streaming service system of FIG. 2 includes a sender 210 that provides multimedia data, a server 220 that formats and services the multimedia data into fragments, and a client 240 that reproduces the multimedia data.

First, the sender 210 will be described.

The capture unit 212 captures video and audio data using a camera.

The encoder 214 compresses the image and audio data captured by the capture unit 112 using a preset compression algorithm such as MPEG.

The communication unit 216 converts the video and audio data compressed by the encoder unit 214 into a predetermined protocol, and transmits the converted video and audio data to the server 220 using the protocol.

Next, referring to the server 220, the communication unit 222 receives the compressed video and audio data from the sender 210.

A file muxer 224 muxes the compressed video and audio data into the improved streaming file format as shown in FIG. That is, the file muxer 224 configures one fragment with a plurality of M-dot boxes and one muff box, and stores at least one fragment a f-tip box and multimedia metadata describing the type of the content file. A moving box for recording is added to form a streaming file format.

The web server 226 transmits the video and audio data to the client 240 in the streaming file format shown in FIG. 1 in the file muxer 224 according to the manifest file request from the client 240. . In one embodiment, when the web server 226 receives the fragment file request from the client 240, the web server 226 divides the multimedia data into a plurality of M-dot boxes and transmits the multimedia data. When the M-box transmission is completed, the web server 226 transmits a muff box in which metadata is recorded.

Next, the client 240 will be described.

The communication unit 242 receives the video and audio data of the improved streaming file format from the server 220 through the Internet 230.

A file parser 244 parses video and audio data from a streaming file format received through the communication unit 242.

The decoder 146 decodes the video and audio data parsed by the file parser 244 into a video and audio signal.

The renderer 248 performs rendering to display the video signal decoded by the decoder 246 on the screen.

3 is a flowchart of signals and data in a multimedia streaming service system according to an embodiment of the present invention.

First, the client 240 requests a manifest file from the server 220 (312). In this case, the manifest file includes content information such as a total contents duration, a stream type, a codec, a number of fragments, and a fragment duration.

Subsequently, the server 220 transmits the manifest file (mf.xml) to the client 240 (314).

Subsequently, the client 240 requests the fragment file from the server 220 (316).

Subsequently, the server 220 requests the sender 210 the multimedia data desired by the client 240, and performs an improved fragment response on the multimedia data received from the sender 210 (318).

That is, the server 220 divides the multimedia data into M-dot box units mdat1, mdat2, mdat3, ... in real time and transmits the multimedia data to the client 240, and then the server 220 transmits all the multimedia data. After that, send the Mute Box that recorded the metadata.

At this time, in the conventional technology, a fragment period for configuring the multimedia data in the unit of a fragment consisting of one muffle box and one M-dot box in the server 220 takes.

However, in the present invention, since the server 220 transmits the multimedia data to the client 240 without dividing the multimedia data into fragments, the server 220 takes the frame period 320.

Finally, the client 240 composes a fragment of the m-boxes and the muffle box received from the server 220 to play the fragment data multimedia data. At this time, the client 240 requires a buffering time 340 or more for a predetermined time for jitter compensation and audio / video synchronization. In addition, the client 240 has a multimedia data transmission delay 330 of the server 220.

Therefore, according to an embodiment of the present invention, the multimedia data can be transmitted even before all the multimedia data corresponding to one fragment are received, thereby minimizing the latency.

4 is a signal and data flow diagram of an existing multimedia streaming service system for explaining a difference from the present invention.

Manifest file request 412, manifest file transfer 414, and fragment request 416 of FIG. 4 are identical to FIG. 3 of the present invention. Also, the transmission delay 440 and the buffering time 440 at the client 240 are the same as in FIG. 3 of the present invention.

However, the existing fragment response 418 is different from the fragment response 318 of the present invention. That is, the fragment response 418 of the conventional technology transmits the multimedia data in the fragment unit, the present invention transmits the fragment response 318 in the unit of m dot.

Referring to the difference between the multimedia streaming service of the present invention and the technology with reference to the present invention of Figure 3 and the existing technology of Figure 4,

Since the conventional technology transmits multimedia data in units of fragments including a muffle box and an M. dot box as illustrated in FIG. 4, the fragment period 420 for configuring the multimedia data in a fragment unit consisting of one muffle box and one M. box. Takes)

However, the present invention does not send a fragment including a mumbox and an Mdot box, as shown in FIG. At this time, the present invention takes only the frame period 320 for storing the multimedia data in the MD box. In other words, the present invention performs a fragment unit request from the client 240 to the server 220, but performs a frame unit response of the M. dot box from the server 220 to the client 240.

As a result, the present invention shown in Fig. 3 requires a reversal of only the frame period of the M-dot box. However, the existing technique of FIG. 4 requires a latency of a fragment period of a fragment including a mute box and an M-dot box. Therefore, when comparing the minimum latency (350, 450) between the server 220 and the client 240 of the present invention and the existing technology, the present invention can minimize the latency more than the existing technology.

In addition, since the present invention does not need to wait for the fragment period for constructing the fragment in the server 220, the buffering time 340 for jitter compensation and video / audio synchronization becomes the minimum latency in the client 240.

5 is a flowchart illustrating a multimedia streaming service method according to an embodiment of the present invention.

First, a manifest file request corresponding to content information is received from the client 240 (step 510).

Subsequently, the manifest file is read (step 520).

In operation 530, the read manifest file is transmitted to the client 240.

In operation 540, the client 240 receives a fragment file request corresponding to the manifest file.

Subsequently, the multimedia data corresponding to the fragment file is received by the sender 210, and the multimedia data is divided and transmitted in units of M-dot boxes (step 550). That is, as soon as the multimedia data is received from the sender 210, the multimedia data in the unit of the m box is transmitted to the client 240.

Subsequently, after completing the transmission of the plurality of M-dot boxes, the muffle box is finally transmitted to the client 240 (operation 560).

Therefore, since the server 220 may transmit the multimedia data to the client 240 even before all the multimedia data corresponding to one fragment is received, the latency may be minimized.

6 is a flowchart illustrating a multimedia streaming processing method in a client according to an embodiment of the present invention.

First, when the device is turned on (step 612), it is checked whether content is played (step 614).

Subsequently, if the content is played, the server 220 requests a manifest file corresponding to the content information (step 616).

In operation 618, the manifest file is received from the server 220.

Subsequently, the play block is initialized using various pieces of content information included in the manifest file (step 622).

Subsequently, the server 220 requests a fragment file corresponding to the manifest file (step 624).

Subsequently, the multimedia data is received from the server 220 in units of M-dot boxes (step 626).

Subsequently, when all the multimedia data is received from the server 220, a muff box corresponding to the metadata is received from the server 220 (step 628).

Subsequently, parsing is performed from a fragment file including a plurality of M-dot boxes and a muff box (step 632).

Next, the parsed video and audio data are decoded (step 634).

In operation 636, a rendering operation is performed to display an image on a screen.

The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, hard disk, floppy disk, flash memory, optical data storage, and the like. The computer readable recording medium may also be distributed over a networked computer system and stored and executed as computer readable code in a distributed manner.

The above description is only one embodiment of the present invention, and those skilled in the art may implement the present invention in a modified form without departing from the essential characteristics of the present invention. Therefore, the scope of the present invention should be construed to include various embodiments which are not limited to the above-described examples but are within the scope equivalent to those described in the claims.

Claims (15)

In the multimedia streaming file format structure,
An M-dot box for storing multimedia data;
A muff box for recording meta data related to the multimedia data,
One fragment box is composed of a plurality of m-dot boxes for dividing and storing the multimedia data and one mof box for recording metadata of the multimedia data stored in the plurality of m-dot boxes,
And the plurality of M-dot boxes are located in front of the mute box.
2. The multimedia streaming file format structure as claimed in claim 1, wherein the streaming file is formatted by adding a ftip box describing the type of the content file and a move box describing the multimedia metadata to at least one fragment. The multimedia streaming file format structure according to claim 1, wherein the minimum unit of the m-dot box is a frame unit. The multimedia streaming file format structure according to claim 1, wherein the minimum unit of the m-dot box is a slice unit. The multimedia streaming file format structure according to claim 1, wherein the maximum unit of the m-dot box is all audio / video data in a fragment. 2. The multimedia streaming file format structure as recited in claim 1, wherein the mute box comprises a fragment size box. In the multimedia streaming service method,
Receiving a request for multimedia data in units of fragments;
Transmitting the requested multimedia data by dividing it in units of M-dot boxes;
When the transmission of the multimedia data in the unit of the m dot box is completed, the multimedia streaming service method comprising the step of transmitting to the client a muff box recording the meta data of the multimedia data. The method of claim 7, wherein the fragment
A plurality of m-dot boxes for dividing and storing multimedia data and one muff box for recording meta data related to the plurality of m-dot boxes,
The plurality of m-dot boxes are located in front of the muffle box multimedia streaming service method. The method of claim 8, wherein the minimum unit of the m-dot box is a frame unit or a slice unit. The method of claim 8, wherein the maximum unit of the M-dot box is the entire video / audio data in the fragment. The multimedia streaming service method of claim 8, wherein a frame size box is further added and transmitted in the muff box. 9. The method of claim 8, wherein the M-Dot box is first identified by the frame size box added in the Mute box. The multimedia streaming service method of claim 7, wherein the client plays the multimedia data received in units of M-dots and the fragment data received in a fragment box received after completion of the multimedia transmission. In the multimedia streaming service apparatus,
A file mux section constituting one fragment with a plurality of m-dot boxes and one muft box for dividing and storing multimedia data;
And a web server unit configured to transmit the multimedia data in units of a plurality of M-dots when receiving a fragment file request from a client, and to transmit a muff box recording the meter data when the M-box transmission is completed. A computer-readable recording medium having recorded thereon a program for executing the method of any one of claims 1 to 13.

Images