KR20180086113A - Media playback apparatus and method for synchronously reproducing video and audio on a web browser - Google Patents

Abstract

The present invention relates to a media playback apparatus for synchronously reproducing video and audio on a web browser, and a method for the same. According to the present invention, audio is buffered to be outputted at a time point at which the video is outputted, thereby synchronizing outputs of video and audio. In addition, according to an embodiment of the present invention, video is synchronized with audio at a time point at which the audio is outputted. According to the present invention, video is synchronized with audio even when the video and the audio are decoded by different decoders, thereby providing an environment capable of separating and processing the video and the audio. Therefore, it is possible to implement a decoder separately from a decoder embedded in a web browser in a non-plug-in environment, thereby reducing dependency on a codec format of media according to a decoder.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a media playback apparatus and method for synchronously reproducing video and audio on a web browser,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and a method for reproducing media on a web browser, and more particularly, to an apparatus and a method for reproducing media by synchronizing video and audio when a video or audio is decoded in units of a container .

There is known a method of using a plug-in in which a codec decoder and a renderer are written in native code in order for a user to play media data on a web browser through the Internet. ActiveX and NPAPI are used for these web browser plugins.

ActivieX is a combination of two technologies developed by MicroSoft, Component Object Model (COM) and Object Linking and Embedding (OLE). However, it is generally used in addtion form in internet explorer which is a narrow web browser. ActiveX is used to play media from Internet Explorer web browser.

NPAPI is an API developed in Netscape, similar in functionality to Internet Explorer's ActivieX. It is an API provided for dragging an external application into a plug-in format to enhance the function of the web browser when the initial web environment is poor. In other words, it was developed for playing music and moving pictures on the initial web page, such as Applet of Java, Flash of Adobe, and Real Player.

But plug-ins are used by hackers as a way to distribute malicious code, and Web browser vendors are no longer supporting it. NPAPI is no longer supported by NPAPI since Chrome version 45 on Google, which makes and distributes Chrome. In the case of ActiveX, from the Edge version, which is used by Microsoft as the default browser for Windows 10, It is no longer supported by ActivieX.

There is a method of performing decoding and rendering using a video tag of HTML5 as a method for playing media in a web browser without a plug-in. HTML5 is HTML's fifth markup language developed by W3C and web browser developers, providing an environment that allows media playback without plugins via video tags. It has full support for HTML5 video tags since Chrome 4.0, Explorer 9.0, Firefox 3.5, Safari 4.0, and Opera 10.5.

When decoding using video tags, high-resolution video and high-fps video can be processed with excellent performance. However, there are limitations on the video formats supported by video tags, and they currently support only three formats: MP4, WebM, and Ogg. More details include codec formats such as H.264, ACC, MP3, WebM VP8, VP9, Vorbis, Opus and Ogg Theora and Vorbis of MP4, and video codec format H.265 or audio codec format G .711, G.726, etc. are not supported by video tags.

The limitations of the codec format limitations of video tags are complementary when using a decoder implemented in JavaScript. The JavaScript decoder can decode video in any format, so it is possible to decode media in codec format not supported by the video tag.

But there is still a limit to this complement. Since the video tag basically performs decoding by receiving video in units of a container packaging a plurality of frames, when video and audio are decoded by using different decoders of the video tag decoder and the JavaScript decoder, It can be. For example, if video in H.264 format is decoded by a video tag in container units and audio in G.711 format is decoded by a JavaScript decoder, the number of frames used in the container is 30 and fps of video is 1 , Video can be output up to 30 seconds after the audio is played back first.

Accordingly, the present invention proposes a method capable of media playback without a problem of synchronization between video and audio.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a media playback method in which synchronization of video and audio is not a problem when video is decoded in a container unit.

The problems of the present invention are not limited to the above-mentioned problems, and other problems not mentioned can be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided a media playback apparatus for synchronously reproducing video and audio on a web browser, the media playback apparatus comprising: A receiving unit; A data separator for separating the received media data into video data and audio data; A container unit for packaging the frames constituting the video data into units and converting the frames into chunk data; A decoder for decoding the converted chunk data by a decoder embedded in a web browser to reconstruct the video and outputting the reconstructed video, A media restoring unit for providing the media; And an audio sync unit for outputting the audio data in synchronization with the restored video based on time information provided in the chunk data unit.

According to an aspect of the present invention, there is provided a media service apparatus for transmitting media data to a media playback apparatus, the media service apparatus comprising: a script for playing the media data on a web browser of the media playback apparatus; A module storage unit for storing a module; A module transfer unit for transferring the script module to the media player in response to connection of the media player; A packetizing unit for packetizing the media data to generate a transmission packet; And a web server that forms a communication session with the playback apparatus and transmits the transport packet to the media playback apparatus in response to a request from the media playback apparatus, A process of packaging the video frames included in the communication packet into chunks by packaging the chunks of video frames in units of a unit number; and a process of converting the chunks of the video packets into chunks by decoding the converted chunks by a media restoring unit And outputting the audio data included in the communication packet in synchronization with the chunk data based on the time information on the time point.

According to another aspect of the present invention, there is provided a media playback apparatus for synchronizing video and audio on a web browser, the media playback apparatus comprising: ; A data separator for separating the received media data into first media type data and second media type data; A container unit for packaging the frames constituting the first media data into unit numbers and converting the frames into chunk data; The method comprising: decoding the converted chunk data by a decoder embedded in a web browser to restore a first medium; and outputting the restored first media, To the chunk data unit; And a sync unit for outputting the second media data in synchronization with the restored first media based on time information provided in units of the chunk data.

 Other specific details of the invention are included in the description and drawings.

The embodiments of the present invention have at least the following effects.

According to the solution of the above problem, it is possible to reproduce the synchronized media by restoring video and audio in accordance with the container specification.

In addition, even if video and audio are decoded by different decoders by the solution of the above-mentioned problems, since synchronization is not a problem, an environment in which video and audio are separated can be provided.

Also, the dependency on the codec format is reduced because the codec format audio that is not supported by the video tag can be used without synchronization problems.

The effects according to the present invention are not limited by the contents exemplified above, and more various effects are included in the specification.

Figure 1 shows an embodiment of an overall system for media playback.
FIG. 2 is a diagram illustrating a TCP / IP 4-layer model that is hierarchically defined for communication between devices.
FIG. 3 shows a process of connecting a web socket between the media service apparatus 110 and the media playback apparatus 120.
FIG. 4 shows an example of a process of transmitting and receiving data through a web socket connection.
5 is a diagram showing a structure of a communication packet transmitted through the network interface 21. As shown in FIG.
6 shows an embodiment of a media service apparatus 110. [
FIG. 7 shows another embodiment of the media service apparatus 110. FIG.
Figure 8 illustrates one embodiment of the script module.
Figure 9 shows another embodiment of a script module.
Figure 10 shows another embodiment of a script module.
11 shows another embodiment of the script module.
12 shows an embodiment of the media playback apparatus 120. In Fig.
Fig. 13 shows another embodiment of the media playback apparatus 120. Fig.
Fig. 14 shows another embodiment of the media playback apparatus 120. Fig.
15 shows a process of generating chunk data.
16 shows an audio chunk generation process.
17 is an exemplary diagram for explaining a process of generating a script module implemented with JavaScript in accordance with an embodiment of the present invention.
18 is an exemplary diagram of a computing device 400 for implementing the media playback device 120. As shown in FIG.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. To fully disclose the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

Unless defined otherwise, all terms (including technical and scientific terms) used herein may be used in a sense commonly understood by one of ordinary skill in the art to which this invention belongs. Also, commonly used predefined terms are not ideally or excessively interpreted unless explicitly defined otherwise.

The terminology used herein is for the purpose of illustrating embodiments and is not intended to be limiting of the present invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. The terms " comprises " and / or " comprising " used in the specification do not exclude the presence or addition of one or more other elements in addition to the stated element.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Figure 1 shows an embodiment of an overall system for media playback. 1 comprises a media service apparatus 110, a media playback apparatus 120, and a network 430 for connecting the two apparatuses.

The media service device 110 includes a computing or processing device suitable for providing computing services for one or more video playback devices. For example, the media service apparatus 110 includes a device capable of generating or storing a video stream, such as a network camera, a network video recorder (NVR), a digital video recorder (DVR)

Media playback apparatus 120 includes a computing or processing device suitable for interacting with media service device 110 or other computing user devices over a network. For example, media playback device 120 may include a desktop computer, a mobile phone or smart phone, a personal digital assistant (PDA), a laptop computer, and a tablet computer.

The media shot or stored in real time by the media service apparatus 110 is transmitted via the network 430 at the request of the media playback apparatus 120. [ The user is able to reproduce the media transmitted through the user interface implemented on the web browser 210 of the media player 120. [ At this time, the web browser 210 not only includes a commonly known browser installed on a desktop computer or a mobile phone such as Google Chrome, Microsoft Explorer, Mozilla Firefox, Apple Safari, And software applications that are created separately.

An RTSP / RTP protocol transmitted over a web socket, which is a network communication method between the media service apparatus 110 and the media playback apparatus 120, will be described with reference to FIGS.

FIG. 2 is a diagram illustrating a TCP / IP 4-layer model that is hierarchically defined for communication between devices.

The fourth layer comprises a network interface layer 21, an Internet layer 22, a transport layer 23 and an application layer 24. In order to use a web socket connection, a TCP transmission connection is established between the media service apparatus 110 and the media playback apparatus 120 (RTSP / RTP protocol) in order to use the web socket connection in the RTSP / ) Should be established first. Once the web socket connection is established between the media service device 110 and the media playback device 120, for example, via a three-way handshake process, the web socket communication is initiated by sending web socket packets . The web socket connection and the web socket packet will be described in detail with reference to FIGS. 3 to 5 below.

FIG. 3 shows a process of connecting a web socket between the media service apparatus 110 and the media playback apparatus 120.

The media playback apparatus 120 requests the media service apparatus 110 to initiate a web socket connection using the web socket URI. Web socket URIs can be obtained using the GetServiceCapabilities command. The web socket URI is expressed in the same manner as, for example, " ws: //192.168.0.5/webSocketServer " (S1000).

The media playback apparatus 120 requests the media service apparatus 110 to upgrade the web socket. The media service apparatus 110 responds with a 101 code which is a status code to approve the protocol change request (S1100).

After the web socket connection is established between the media service apparatus 110 and the media playback apparatus 120, data is exchanged through the RTSP / RTP protocol transmitted over the web socket instead of the HTTP / 1.1 protocol. 3, DESCRIBE, SETUP, PLAY, PAUSE, TEARDOWN are RTSP commands. A DESCRIBE request contains a URL. The response message to DESCRIBE also includes a description of the request. A SETUP request specifies that a single media stream should be transmitted. A PLAY request is a request to play one or all media streams and multiple requests are possible. A PAUSE request commands a pause for one or all media streams. The PLAY request can be restarted again. A TEARDOWN request is a command to terminate a session. The reproduction of all the media streams is stopped by the TEARDOWN request and all sessions related to the related data are also released (S1200).

Examples of a request message sent from the media playback apparatus 120 and a response message sent from the media service apparatus 110 in the Web socket connection process shown in FIG. 3 are shown in Table 1 below.

The media playback apparatus 120 -> the media service apparatus 110 -
GET / webSocketServer HTTP / 1.1
Host: 192.168.0.1
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ ==
Origin: http://example.com
Sec-WebSocket-Protocol: rtsp.onvif.org
Sec-WebSocket-Version: 13.

The media service apparatus 110, the media playback apparatus 120,
HTTP / 1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK + xOo =
Sec-WebSocket-Protocol: rtsp.onvif.org

These Web socket connections are made according to the Web Socket protocol, the HTML5 standard. In particular, since the web socket connection continuously supports bidirectional communication, it is possible to continuously transmit and receive data without disconnecting the connection between the media service apparatus 110 and the media playback apparatus 120.

FIG. 4 shows an example of a process of transmitting and receiving data through a web socket connection.

Referring to FIG. 4, the media player 120 transmits a TCP / IP connection request message to the media service apparatus 110, and the media service apparatus 110 accepts the TCP / When the message (SYN-ACK) is transmitted to the media playback apparatus 120, a TCP / IP connection is established. A TCP transport connection can be formed by a pair of local TCP sockets and remote TCP sockets, each of which is defined by an identifier, such as at least a port number and an IP address. Of course, instead of these TCP / IP-based connections, you can also establish a UDP / IP-based connection between them.

Thereafter, when a web socket connection is established between the media playback apparatus 120 and the media service apparatus 110 through a handshake process, continuous data transmission / reception between the media playback apparatus 120 and the media playback apparatus 120 can be performed thereafter. That is, the media playback apparatus 120 transmits a media streaming request in the form of a transmission web socket packet (socket.send) to the media service apparatus 110, and the media service apparatus 110 transmits the media streaming request to the media playback apparatus 120 Sends the stream in the form of a response web socket packet (socket.onMessage). This process can be done continuously between media streams until transmission is stopped or completed.

5 is a diagram showing a structure of a communication packet transmitted through the network interface 21. As shown in FIG.

When the RTP header 44 is added to the RTP payload corresponding to the data 45, it becomes an RTP packet. The RTP packet is the same as the Web socket payload, and a Web socket header (43) is added to it to become a Web socket packet. The Web socket packet is the same as the TCP payload, and the TCP header 42 is added to the packet to become a TCP packet. Finally, the TCP packet is the same as the IP payload, and when the IP header 41 is added thereto, a communication packet, that is, an IP packet is finally completed. The processes of completing the IP packet and removing the respective headers are performed in the media service apparatus 110 and the media playback apparatus 120.

Since the communication between the media service apparatus 110 and the media playback apparatus 120 is performed through the HTML5-based web socket protocol described above with reference to FIGS. 2 to 5, the module responsible for RTSP / RTP transmission / This can be implemented by any possible script code. For example, JavaScript can be implemented as a script that can be parsed in HTML5. Therefore, media playback using the RTSP / RTP protocol can be implemented in the web browser of the HTML5 environment without installing the plug-in separately as in the prior art.

The network communication method between the media service apparatus 110 and the media playback apparatus 120 has been described. Hereinafter, the configuration and operation of the media service apparatus 110 and the media playback apparatus 120 will be described with reference to FIGS. 6 to 18. FIG.

6 shows an embodiment of a media service apparatus 110. [ In one embodiment, the media service apparatus 110 includes a real-time video camera 111, an encoder 112, a packetizer 113, a web server 114, a module storage 115, a module transmitter 116, And a control unit 117.

The real-time video camera 111 is a means for photographing media in real time, and the photographing includes a method of performing both video capturing and audio recording, and a method of performing only video capturing.

The encoder 112 compresses and encodes the media photographed by the real-time video camera 111. The encoding by the encoder 112 is not necessarily made up of a specific codec format supported by the decoder embedded in the web browser, but can be encoded in any codec format.

The packetizing unit 113 packetizes the encoded media data to generate a transmission packet. Packetizing means dividing the media data into suitable lengths for facilitating transmission through the network 430, or giving control information such as an address to be received to each of the media data in an appropriate length when the media data is short. At this time, the control information is located in the header of the transport packet. The transport packet is in the form of the Web socket packet described above.

The packetizing unit 113 may packetize media data according to a method requested by the media playback apparatus 120. [ For example, when the media playback apparatus 120 requests a frame-by-frame video, the packetizer 113 generates a transport packet in a frame format. In the media playback apparatus 120, a container unit The packetizing unit 113 can generate a transport packet in a container format.

The web server 114 performs a function of establishing a communication session with the media playback apparatus 120. That is, a web socket connection is established between the web server 114 of the media service apparatus 110 and the media playback apparatus 120 through a handshake process. And then transmits the transport packet generated by the packetizing unit 113 through the web server 114 at the request of the media playback apparatus 120. [

The module storage unit 115 is a module for storing a script module necessary for playing media on the media playback apparatus 120. [ The script module is a module that allows a media player to play media on a web browser in an HTML5 environment without installing a plug-in or a separate application program in the media player 120 with a script written in a script that can be parsed by a web browser. The script module may be, for example, code written in JavaScript. The script module will be described later with reference to FIG. 8 to FIG.

The module transfer unit 116 is a module for transferring a script module stored in the module storage unit 115 to the media playback apparatus 120. The module transfer unit 116 transmits the script module in response to the media playback apparatus 120 accessing the media service apparatus 110 via the network 430. [

The control unit 117 is a module for controlling other configuration modules of the media service apparatus 110. For example, when the media playback apparatus 120 is connected to the web server 114 via the network 430, the script module stored in the module storage unit 115 is transmitted to the media playback apparatus 120 The control unit 117 transmits and receives signals from the respective modules and controls the smooth operation of the corresponding operation.

The operation method will be described based on the description of the configuration modules of the media service apparatus 110 of FIG. When the media playback apparatus 120 is connected to the web server 114 via the network 430, the module transfer unit 116 transmits the script module stored in the module storage unit 115 to the media playback apparatus 120 . When a script module is installed in the media player 120, the user requests media playback through the user interface. The media service apparatus 110 encodes the real-time live media photographed by the real-time video camera 111 in the encoder 112 and packetizes the media data into transport packets in accordance with the frame or container format in the packetizer 113, And transmits it to the media playback apparatus 120 via the server 114. [

FIG. 7 shows another embodiment of the media service apparatus 110. FIG. The media service apparatus 110 of FIG. 6 is an embodiment for transmitting real-time live media by the real-time video camera 111, and the media service apparatus 110 of FIG. 7 transmits media data stored in the media storage unit 118 Fig.

The media storage unit 118 includes a network video recorder (NVR) or a personal video recorder (PVR), but will be described as an example of a network video recorder in the embodiment of FIG.

The media storage unit 118 receives media data from a camera or a server and compresses and stores the media data. When there is a media data transmission request from the media playback apparatus 120, the media service apparatus 110 packetizes the media data stored in the media storage unit 118 in the packetizing unit 113 and transmits the packetized data through the web server 114 .

The packetizing unit 113, the web server 114, the module storage unit 115, the module transfer unit 116, and the control unit 117 of the media service apparatus 110 configuration module shown in FIG. Duplicate descriptions of modules are omitted.

8 to 11 illustrate embodiments of a script module that is transmitted from the media service apparatus 110 to the media playback apparatus 120. Fig. The script module can be implemented as a script that can be parsed in a web browser, and a case where the script module is implemented as JavaScript in the embodiment of Figs. 8 to 11 will be described.

Figure 8 illustrates one embodiment of the script module. 8 includes a RTSP / RTP client module 121, a depacketization module 122, a JS decoder module 124, and a JS renderer module 125. The RTSP / do.

The RTSP / RTP client module 121 performs a function of supporting RTSP / RTP communication with the media service apparatus 110. And can receive a transmission packet from the web server 114 of the media service apparatus 110 via the RTSP / RTP client module 121. [ Currently, it is not possible to process media according to RTSP / RTP on a web browser without a plug-in. Using the RTSP / RTP client module 121, the data transmitted in the RTSP / RTP protocol Lt; / RTI >

The decapsulation module 122 is a module for depacketizing a packet received from the RTSP / RTP client module 121. If the packetization means forming the packet by dividing the media data into an appropriate length, it means to restore the media data to the state prior to the packetization by rejoining it.

The JS decoder module 124 is a module for releasing compression and encoding of the encoded video, that is, decoding. The JS decoder module 124 may be implemented in JavaScript as in the other modules of the script module. Since the JS decoder module 124 is implemented in JavaScript, unlike a decoder embedded in a web browser, it is possible to decode an arbitrary codec format without limitation to the codec format. It is also possible to decode on a frame-by-frame basis.

For example, the JS decoder module 124 may be represented by a code shown in Table 2 and Table 3, as shown in Table 3, according to an embodiment of FIG.

function HevcDecoder () {

var _name = "HevcDecoder";
var self = this;

this._decoderParameters = null;
this._isRequestActive = false;
this._player = null;
this._requestContext = null;
this._requestContextQueue = [];

this.pushRequestContext = function (requestContext) {
self._requestContextQueue.push (requestContext);
};
this.decode = function () {
if (self._isRequestActive) {
return;
}

if (self._requestContextQueue.length) {
self._isRequestActive = true;
self._requestContext = self._requestContextQueue.pop ();
self._playStream (self._requestContext.stream);
}
};
this._createDecodeCanvas = function (parentElement) {
self.canvas = document.createElement ("canvas");
self.canvas.style.display = "none";
self.canvas.id = "decode-canvas";
parentElement.appendChild (self.canvas);
self.ctx = self.canvas.getContext ("2d");
};
this._playStream = function (buffer) {
this._reset ();
this._handleOnLoad (buffer);
}
this._onImageDecoded = function (image) {
var width = image.get_width ();
var height = image.get_height ();
this.canvas.width = width;
this.canvas.height = height;
this._imageData = this.ctx.createImageData (width, height);

image.display (this._imageData, function (displayImageData) {
var itemId = self._requestContext.itemIds [self._requestContext.currentFrameIndex];
var payload = self._requestContext.payload;
if (height> payload.displayHeight) {
payload.displayHeight = height;
}

if (! (itemId in self._requestContext.dependencies)) {
if (width> payload.displayWidth) {
payload.displayWidth = width;
}
payload.frames.push ({
canvasFrameData: displayImageData.data,
itemId: itemId,
width: width,
height: height
});
}
self._requestContext.currentFrameIndex ++;
if (self._requestContext.currentFrameIndex> = self._requestContext.itemIds.length) {
self._requestContext.callback (payload);
self._isRequestActive = false;
self.decode (); // Decode next queued request
}
});
};

...

this._createDecodeCanvas (document.documentElement);
this._reset ();
}

The JS renderer module 125 performs a function of rendering the decoded video in the JS decoder module 124 and displaying it on an output device such as a monitor or the like. The JS Renderer module 125 uses WebGL to convert YUV format video to RGB format. WebGL is a web-based graphical library that allows you to create 3D graphics interfaces that are available through JavaScript.

Figure 9 shows another embodiment of a script module. The script module of FIG. 9 is a module used for restoring video in a container format, and includes an RTSP / RTP client module 121, a depacketization module 122, and a container creation module 127. The RTSP / RTP client module 121 and the depacketization module 122 are omitted from the overlapping description of the modules described in FIG. 8, respectively.

9, it can be seen that the container generation module 127 is included, unlike the case of FIG. The container generation module 127 may perform a function of forming a container by packaging the frames in units of units when video is not packaged in container units. In addition, video frames and audio can be packaged in units. At this time, the container generation module 127 can variably adjust the number of units according to fps (frames per second) of the video. It is assumed that the container generation module 127 packages video frames in a unit number or packages video frames and audio in a unit number as chunk data.

A container is a concept that includes a container supported by a video tag, such as an MPEG-DASH container. The container creation module 127 can configure the media in a container format compatible with the HTML5 video tag so that the use of the video tag in the media playback apparatus 120 without compatibility problems even if the media is not transmitted in the container format in the image capturing apparatus . That is, the present invention provides an environment in which video tags can be used without modification of the existing image capturing apparatus.

Figure 10 shows another embodiment of a script module. The script module of FIG. 10 includes a RTSP / RTP client module 121, a decapsulation module 122, and an audio transcoder 123 as a module for audio reproduction. The RTSP / RTP client module 121 and the depacketization module 122 are omitted from the overlapping description of the modules described in FIG. 8, respectively.

The audio transcoder 123 is a module for performing transcoding when audio data is configured in a codec format not supported by a decoder embedded in a web browser. The transcoding converts the codec format, and the audio transcoder 123 decompresses the audio data and compresses the audio data again in a different codec format. For example, audio data in the G.711 codec format, which is not supported by the video tag, which is a decoder embedded in a web browser, is converted into the codec format of ACC that is supported by the video tag.

The audio transcoder 123 converts the audio data into a codec format supported by a decoder embedded in a web browser, so that the audio data can be restored using the decoder embedded in the web browser together with the video data, Environment.

11 shows another embodiment of the script module. The script module of FIG. 11 differs from the script module of FIG. 10 in modules and configurations for audio reproduction. The script module of FIG. 11 includes an RTSP / RTP client module 121, a decapsulation module 122, an audio decoder 126, an audio chunk part 128 and a buffer controller 129, 121 and the depacketizing module 122 are the modules described in Fig.

The audio decoder 126 is a module that performs decoding of audio data, and can be implemented by JavaScript, which is a script that can be parsed by a web browser as with other modules.

The audio data decoded in the audio decoder 126 is packaged as audio chunks in the audio chunk unit 128. [ At this time, the audio chunk is generated by packaging the audio data in units of frames constituting the chunk data generated by the container generation module 127. That is, an audio chunk is generated depending on the chunk data. 12 shows a media playback apparatus 120 including both a container generation module 127 and an audio chunk unit 128. The generation of the audio chunks in FIG. 12 will be described again.

The buffer controller 129 receives the audio chunk from the audio chunk unit 128 and buffers the audio chunk in the audio buffer and provides the audio chunk to the audio renderer 136. The audio renderer 136 is shown in Figs. 12 and 13 as a configuration module of the media playback apparatus 120. Fig.

The buffer controller 129 receives time information from another module for restoring video and performs a function of synchronizing audio to video. The time information is information related to the time at which the chunk data is decoded and output by the media restoration unit 143 installed in the media playback apparatus 120. The start time of the chunk data is set by the renderer in the media playback apparatus 120 Indicates when to render. The buffer controller 129 buffers or transmits the audio chunks to the audio renderer 136 so that audio can be outputted according to the time when the video is displayed through the time information, thereby synchronizing the video and the audio.

The script module of FIG. 11 can be configured in a different order, moving the audio decoder 126 to the rear end of the buffer controller 129. In this case, the buffer controller 129 synchronizes the chunk data and the audio data first, and the audio decoder 126 decodes the buffered audio data.

8 to 11 are transmitted from the media service apparatus 110 to the media playback apparatus 120 as the media playback apparatus 120 is connected to the media service apparatus 110 so that the media playback apparatus 120 Provides an environment in which the media playback can be performed without a plug-in in the web browser 210 of the mobile terminal. That is, the script module is installed in the media playback apparatus 120 to configure a system for media playback. An embodiment of the media playback apparatus 120 in which the script module is installed will be described with reference to FIG. 12 through FIG.

Figs. 12 to 14 show embodiments of the media playback apparatus 120. Fig. The main modules for media playback in the media playback apparatus 120 are configured by a script module. Since the functions of the script module have been described with reference to FIGS. 8 to 11, the media playback apparatus 120 will be described with reference to FIGS. 12 to 14. FIG.

12 shows an embodiment of the media playback apparatus 120. In Fig. 12 includes a receiving unit 141, a data separating unit, a container unit 142, a media restoring unit 143, and an audio sink unit 144. [ At this time, the receiving unit 141, the data separating unit, the container unit 142, and the audio sink unit 144 may be implemented as JavaScript. The media playback apparatus 120 of FIG. 12 can be configured by receiving the script modules of FIG. 9 and FIG.

The receiving unit 141 receives the media data generated by the media service apparatus 110 using a communication protocol supporting the web service. At this time, the communication protocol supporting the web service may be the RTSP / RTP protocol transmitted over the web socket. The receiving unit 141 includes a Web socket client 131 and an RTSP / RTP client module 121.

The Web socket client 131 is a module for establishing a Web socket connection with the web server 114 of the media service apparatus 110. The media playback apparatus 120 and the media service apparatus 110 send and receive a transmission packet through a handshake between the Web socket client 131 and the Web server 114, respectively.

The RTSP / RTP client module 121 performs a function of supporting RTSP / RTP communication to the user's web browser 210 as described in the embodiment of FIG. Accordingly, the user can play the media through the web browser 210 of the HTML5 environment using the RTSP / RTP protocol without installing a separate plug-in.

The media data that has passed through the receiving unit 141 is separated into video data and audio data by the data separating unit. The video data is transmitted from the RTSP / RTP client module 121 along the lower left arrow, and the audio data is transmitted to the depacketizing module 122a, 122b along the lower right arrow respectively. The decapsulation modules 122a and 122b decapsulate the video data and the audio data. The decoded video data and the audio data are transmitted to the container unit 142 and the audio sync unit 144, respectively.

The container unit 142 includes a container generation module 127. When the video data is not in a container format by the container generation module 127, the container unit 142 packages the video frames into unit numbers and converts them into chunk data.

Reference is made to Fig. 15 to describe a process of generating chunk data in the container creation module 127. Fig. Video frames 311 in FIG. 15 represent video data configured in units of frames, obtained by depacketing in the depacketizing module 122a. At this time, the video frames 311 are data compressed by various video codecs such as MPEG-2, H.264, and H.265.

When the depacketization module 122a transfers the video frames 311 to the container creation module 127, the container creation module 127 determines whether the received data is in the container format using the header information. Since the video frames 311 are data in frame format, the container generation module 127 converts the video frames 311 into a container format in order to be processed in the MSE 134 and the video tag 135.

The conversion from the frame format to the container format is performed by packaging a plurality of frames in a unit number. At this time, the number of units can be variably selected according to fps (frames per second) of the video data. FIG. 15 shows a process of generating chunk data when the number of units is three. The container generation module 127 parses each frame header information (frame size, intra or inter frame type, inter frame type (I, B or P), etc.) of the video frames 311 Container header, and encapsulates the data of the video frames 311 and the container header into the container format chunk data 312. The chunk data 312 thus converted into the container format can be processed in the media restoration section 143 including the MSE 134 and the video tag 135 without any compatibility problem.

Referring again to FIG. 12, an arrow with a dotted line is drawn from the container creation module 127 to the audio chunk unit 128, which means that information for packaging audio data is delivered. Information to be transmitted will be described in detail through the audio chunk unit 128 below.

The media restoring unit 143 decodes the chunk data by the decoder embedded in the web browser, restores the video, and outputs the restored video. At this time, the decoder embedded in the web browser may be a video tag. In the embodiment of FIG. 12, the media recovery unit 143 includes the MSE 134 and the video tag 135.

MSE 134 is a JavaScript API for HTML5 created for video streaming playback using HTTP download. Standardized by the W3C, this technology enables streaming playback on console gaming devices such as Xbox and Playstation 4 (PS4), as well as on chrome cast browsers.

The video tag 135 performs decoding and rendering so that video is displayed on a web browser. Using the decoder of the video tag 135, decoding can be performed with better performance than the JS decoder module 124 having the limitation due to the dynamic language characteristic of JavaScript. That is, high resolution images and high fps decoding are possible.

The video data is separated from the data separating unit and transferred to the container unit 142. When the video data is not contained in a container unit, Packaged into unit numbers and converted into chunk data. The video data composed of chunk data is output through a decoding and rendering process in the media restoring unit 143. Next, the process of outputting the audio data will be described.

The audio data demultiplexed by the data demultiplexing unit is demultiplexed by the demultiplexing module 122b and is output in synchronization with the video data through the audio sync unit 144. [ The audio sync section 144 may include an audio decoder 126, an audio chunk section 128, a buffer controller 129, and an audio renderer 136.

The audio decoder 126 decodes the separated audio data by a script that can be parsed by a web browser. At this time, the script that can be parsed by the web browser may be JavaScript.

The audio chunk unit 128 packages the decoded audio data in the audio decoder 126 to generate audio chunks. The audio chunk unit 128 must package the audio data in a range corresponding to the chunk data generated by the container generation module 127 so that the audio chunk unit 128 transmits information about the chunk data from the container generation module 127 Receive. An arrow with a dotted line connected to the audio chunk unit 128 in the container generation module 127 means that the information is transmitted.

Please refer to FIG. 16 to describe the process of generating audio chunks. The audio data 321 shown in FIG. 16 shows audio data before being rendered in the audio renderer 136 after being decoded by the audio decoder 126 and decompressed. The audio data 321 shown in FIG. And may be a. Wav file. Since the audio signal is defined in the form of an amplitude according to time in terms of its attribute, an arbitrary section is cut out, and an audio signal in a range corresponding thereto can be extracted. For example, if the frame rate of the video data is 30 fps and the unit number of the frames constituting the chunk data in the container generation module 127 is 3, the container generation module 127 generates information of 30 fps and unit number 3 And transmits it to the audio chunk unit 128. The time corresponding to one video frame is 1/30 second and the number of units is 3, the audio chunk unit 128 packages the audio data 321 in 1/10 second corresponding to three video frames, Chunk 322 is generated.

As described above, when the audio chunks, which are audio data synchronized with the chunk data, are compared with the chunk data generated by the container generation module 127, the chunk data is packaged in units of video frames or packaged in units of video frames and audio And the audio chunk is the audio data packaged in accordance with the unit number. The chunk data is compressed data before being decoded by the decoder 135a and the audio chunk is decoded by the audio decoder 126 and decompressed.

12, the buffer controller 129 receives an audio chunk from the audio chunk unit 128, buffers the audio chunk in the audio buffer, and provides the buffered audio chunk to the audio renderer 136. [

The buffer controller 129 is a module that enables audio data to be output in synchronization with video data through buffering. To be more specific, when the media restoring unit 143 transmits the time information about the time when the video is output to the renderer 135b to the buffer controller 129 in units of chunk data, the buffer controller 129 buffers the audio chunk And provides audio chunks to the audio renderer 136 according to the timing at which video is output through the time information. An arrow with a dotted line from the renderer 135b to the buffer controller 129 in FIG. 12 indicates that the time information is transferred from the media restoring unit 143 to the buffer controller 129. FIG. In this case, the time information indicates the time at which the chunk data corresponding to the time information is rendered in the renderer 135b, and the reference at the time when the chunk data is rendered is the chunk data corresponding to the time information, It may be the time when it is rendered.

The audio renderer 136 performs rendering to output audio chunks received from the buffer controller 129. The audio renderer 136 may be implemented with a Web Audio API supported by a web browser.

To summarize the process of outputting audio through the above-described modules, the audio data is separated from the data separator and transferred to the audio decoder 126. The audio data is decoded and restored in the audio decoder 126, and the restored audio is packaged into an audio chunk in the audio chunk unit 128. The packaging is performed according to the number of frames constituting the chunk data. The audio data packaged in the audio chunks is buffered in the buffer controller 129 and outputted through the audio renderer 136 according to the time when the video is output. At this time, the time point at which the video is outputted is contained in the time information transmitted from the media restoring unit 143 to the buffer controller 129.

Since the media playback apparatus 120 of FIG. 12 synchronizes the video data with the audio data without transcoding audio data, there is no fear of damaging the audio data due to the transcoding. In addition, according to the media playback apparatus 120 of FIG. 12, even if the video data and the audio data are decoded by different decoders, synchronization is not a problem, so that an environment is provided in which video data and audio data can be separately processed. That is, even if the audio data of the codec format not supported by the video tag is restored by a separate decoder, there is no problem in synchronization, so the dependency on the codec format supported by the video tag is reduced.

Fig. 13 shows another embodiment of the media playback apparatus 120. Fig. In the embodiment of FIG. 12, if the audio data is restored and then synchronized with the video data, the audio data is decoded after being synchronized with the video data in the embodiment of FIG. Compared with the embodiment of FIG. 12, the position of the audio decoder 126 in the embodiment of FIG. 13 is changed to the rear end of the buffer controller 129. The other configurations are the same as those of the embodiment of FIG. 12, so duplicate descriptions will be omitted and the following modified audio sync section 144 will be described.

According to the embodiment of FIG. 13, the audio chunk unit 128 packages the audio data before being decoded and restored to generate audio chunks. The audio chunk unit 128 receives information on chunk data from the container creation module 127 to package the audio data in a range corresponding to the chunk data generated by the container generation module 127.

The buffer controller 129 synchronizes the audio chunks with the video data through the time information received from the renderer 135b and transfers the audio chunks to the audio decoder 126. [ In the embodiment of FIG. 13, the audio data synchronized with the chunk data is data before it is decoded by the audio decoder 126, which corresponds to a distinction point from the embodiment of FIG.

The audio decoder 126 restores the synchronized audio data to the audio renderer 136, and the audio renderer 136 performs rendering to output the restored audio data.

In the embodiments of FIGS. 12 and 13, the audio data is synchronized based on the video data. Alternatively, the audio data may be synchronized based on the audio data.

The media data is referred to as first media type data (audio) and second media type data (video), and is applied to the embodiments of FIGS. 12 and 13.

The receiving unit 141 receives the media data and transmits the received media data to the data separating unit. The data separation unit separates the media data into the first media type data (audio) and the second media type data (video).

If the first media type data (audio) is not a container unit, the container unit 142 packages the frames constituting the first media type data (audio) into unit numbers and converts them into chunk data.

The media restoring unit 143 restores the first media type data (audio) by decoding the chunk data by the decoder embedded in the web browser, and when outputting the first media type data (audio) And transfers the information to the sync unit (corresponding to the audio sync unit in Figs. 12 and 13) in units of chunk data. At this time, the decoder embedded in the web browser may be a video tag, and audio data can be decoded and rendered by a video tag, so that it is not a problem that the media restoration unit 143 outputs the first media type data (audio).

The sync unit synchronizes the second media type data (video) with the restored first media type data (audio) based on the received time information.

As described above, the synchronized media can be reproduced according to the embodiment of FIGS. 12 and 13 even when the first media type data is audio and the second media type data is video.

At this time, the first media type data is audio and the second media type data is video, which is a setting for showing various embodiments, and does not limit the types of the first media type data and the second media type data.

Next, an embodiment of another structure of the media player 120 will be described. Fig. 14 shows another embodiment of the media playback apparatus 120. Fig. The media playback apparatus 120 of FIG. 14 includes a receiver, a data separator, a container unit, a media restoration unit, and an audio restoration unit. At this time, the receiving unit, the data separating unit, the container unit, and the audio restoring unit may be implemented by JavaScript. The media playback apparatus 120 of FIG. 14 can be configured by receiving the script modules of FIGS. 9 and 10. FIG.

The receiving unit and the data separating unit are configured in the same manner as in Fig. Receives the media data generated by the media service apparatus 110 through the receiver, and separates the media data from the video data and the audio data in the data separator. The separated video data is passed from the depacketizing module 122a to the container section via decatization.

The container unit includes a container creation module 127. The container creation module 127 packages the frames constituting the video data into unit numbers and converts them into chunk data. At this time, the unit number of frames can be variably controlled according to fps (frames per second) of the video data.

The media restoration unit includes the MSE 134 and the video tag 135, and decodes the chunk data transmitted from the container unit to restore the video and output the restored video.

If the media restoration unit restores and outputs the video data, the audio restoration unit decodes and restores the audio data separated by the data separation unit, and outputs the restored audio data in synchronization with the video data output from the media restoration unit.

The audio restoration section may include a transcode section. The transcode section includes an audio transcoder 123 as a module for transcoding the audio data into another codec format. The audio transcoder 123 transcodes and outputs audio data in a codec format supported by the video tag when the input audio data is in a format not supported by the media restoration unit, that is, in a codec format not supported by the video tag have.

The audio transcoded in the audio decompression unit is delivered to the container creation module 127. When the video data received from the depacketization module 122a is not in units of containers, the container generation module 127 packages audio data received from the audio decompression unit into the unit Chunk data can be generated by packaging them in the number. The generated chunk data can be passed to the MSE 134 without incompatibility problems.

The process of decoding, rendering, and outputting the audio data in the audio decompression unit may be included in the media decompression unit because the container unit generates the chunk data with the video data and the audio data. Accordingly, video data and audio data can be output by decoding and rendering the chunk data through the MSE 134 and the video tag 135 of the media restoring unit.

14, it is possible to decode and render audio data in a codec format not supported by the video tag to a video tag through transcoding of audio data and conversion into chunk data. Therefore, since the audio data is restored together with the video data, it is possible to reproduce the media data without a synchronization problem.

17 is an exemplary diagram for explaining a process of generating a script module implemented with JavaScript in accordance with an embodiment of the present invention.

Referring to FIG. 17, a script module implemented as a JavaScript can convert a source written in native C and C ++ native code into a converter such as Emscripten to obtain a JavaScript code that can be used in a browser.

When a converter such as Emscripten is used, a decoder or a container implemented with JavaScript can be obtained from a conventional native code, so that there is an advantage that codec dependency can be lowered.

You do not have to worry about abandoning support from web browser makers because you use JavaScript code instead of plug-ins. Also, depending on the web browser, there is no need to worry about whether to use ActiveX interface or NPAPI interface. That is, there is an advantage that the dependency of the web browser can be lowered.

The media playback apparatus 120 shown in FIG. 1 may be implemented, for example, in the computing device 400 shown in FIG. 18 below. The computing device 400 may be, but is not limited to, mobile handheld devices (smartphones, tablet computers, etc.), laptop or notebook computers, distributed computer systems, computing grids or servers. The computing device 400 may include a processor 401 and memory 403 and storage 408 that communicate with each other or other elements via a bus 440. [ The bus 440 may be coupled to a display 432, one or more input devices 433, and one or more output devices 434.

All of these elements may be connected to the bus 400 directly or through one or more interfaces or adapters. The bus 440 is coupled to a wide variety of subsystems. Bus 440 may include a memory bus, a memory controller, a peripheral bus, a local bus, and combinations thereof.

The memory 403 may comprise a random access memory (RAM) 404, a read-only component (ROM) 405, and combinations thereof. Also, a BIOS (or firmware) having the basic routines necessary for booting in the computing device 400 may be included in the memory 403.

The storage 408 is used to store an operating system 409, executable files (EXEC) 410, data 411, API applications 412, and the like. The storage 408 may include a hard disk drive, an optical disk drive, a solid-state memory device (SSD), and the like.

The computing device 400 may include an input device 433. A user may enter commands and / or information into the computer device 400 via the input device 433. [ Examples of the input device 433 include a keyboard, a mouse, a touch pad, a joystick, a game pad, a microphone, an optical scanner, and a camera. The input device 433 may be connected to the bus 440 via an input interface 423 including a serial port, a parallel port, a game port, a USB, and the like.

In a particular embodiment, the computing device 400 is connected to a network 430. The computing device 400 is connected to other devices through a network 430. [ At this time, the network interface 420 receives communication data in the form of one or more packets from the network 430, and the computing device 400 stores the received communication data for processing by the processor 401. Similarly, the computing device 400 stores the transmitted communication data in the form of one or more packets on the memory 403, and the network interface 420 transmits the communication data to the network 430. [

The network interface 420 may include a network interface card, a modem, and the like. Examples of the network 430 include the Internet, a wide area network (WAN), a local area network (LAN), a telephone network, a direct connection communication, and the like.

The execution result of the software module by the processor 401 may be displayed via the display 432. [ Examples of the display 432 include a liquid crystal display (LCD), an organic liquid crystal display (OLED), a cathode ray tube (CRT), and a plasma display panel (PDP). The display 432 is connected to the bus 440 via the video interface 422 and the data transfer between the display 432 and the bus 440 can be controlled by the graphics controller 421.

In addition to the display 432, the computing device 400 may include one or more output devices 434, such as audio speakers, printers, and the like. Output device 434 is coupled to bus 440 via output interface 424. The output interface 424 may be, for example, a serial port, a parallel port, a game port, a USB, or the like.

It will be understood by those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

110: Media service device 111: Camera
112: Encoder 113: Packetizer
114: Web server 115: module storage unit
116: module transfer unit 117: control unit
120: Media playback device 121: RTSP / RTP client module
122: depacketization module 123: audio transcoder
124: JS decoder module 125: JS renderer module
126: Audio decoder 127: Container creation module
128: audio chunk unit 129: buffer controller
141: Receiver 142: Container
143: Media restoration unit 144: Audio sink unit
210: Web browser

Claims (20)

A receiving unit for receiving the media data generated by the media service apparatus using a communication protocol supporting the web service;
A data separator for separating the received media data into video data and audio data;
A container unit for packaging the frames constituting the video data into units and converting the frames into chunk data;
A decoder for decoding the converted chunk data by a decoder embedded in a web browser to reconstruct the video and outputting the reconstructed video, A media restoring unit for providing the media; And
And an audio sync unit for outputting the audio data in synchronization with the restored video based on time information provided in units of the chunk data, wherein the audio sync unit reproduces video and audio synchronously on a web browser. The container according to claim 1, wherein the container portion
Wherein the number of units is variably controlled according to an FPS (frame per second) of the video data. The method according to claim 1,
Wherein the media restoration unit includes a renderer that renders the restored video before outputting the time information, wherein the time information indicates a time at which the chunk data corresponding to the time information is rendered in the renderer, And reproducing the media. The method of claim 3,
Wherein the time information indicates a time at which a start portion of the chunk data corresponding to the time information is rendered by the renderer, in synchronization with video and audio on a web browser. The audio signal processing apparatus according to claim 1,
The audio data is decoded to decode the separated audio data, and then the restored audio data is synchronized with the time information provided in units of the chunk data, and the synchronized video and audio data are synchronized and reproduced on a web browser. 6. The apparatus of claim 5, wherein the audio sink
An audio decoder for decoding the separated audio data by a script that can be parsed by the web browser;
A buffer controller for synchronizing the decoded audio data with chunk data corresponding to the time information and providing the decoded audio data to an audio renderer; And
And the audio renderer that renders the decoded audio data, in synchronization with video and audio on a web browser. The method according to claim 6,
Wherein the chunk data is compressed data before being decoded and the audio data synchronized with the chunk data is decoded and decompressed data to reproduce video and audio synchronously on a web browser. 7. The apparatus of claim 6, wherein the audio renderer
And reproducing the video and audio synchronously on a web browser, which is implemented by an audio application program interface (API) supported by the web browser. The audio signal processing apparatus according to claim 1,
Wherein the audio data is synchronized with the time information provided in units of the chunk data, and the audio data is decoded by decoding the buffered audio data to output the audio data. 10. The apparatus of claim 9, wherein the audio sink
A buffer controller for buffering the separated audio data by synchronizing with the chunk data corresponding to the time information;
An audio decoder for decoding the buffered audio data by a script that can be parsed by the web browser; And
And the audio renderer that renders the decoded audio data, in synchronization with video and audio on a web browser. 11. The method of claim 10,
Wherein the chunk data is compressed data before being decoded and the audio data synchronized with the chunk data is data before being decoded by the audio decoder. The method according to claim 1,
Wherein the decoder embedded in the web browser is a video tag supported by HTML5, and the receiver, the container unit, and the audio sync unit are implemented as JavaScript, and reproduce the video and audio synchronously on the web browser. 13. The method of claim 12,
Wherein the JavaScript is downloaded from the media service apparatus to the media playback apparatus, and reproduces the video and audio in synchronization with the web browser. A media service apparatus for transmitting media data to the media playback apparatus,
A module storage unit for storing a script module necessary for reproducing the media data on a web browser of the media player;
A module transfer unit for transferring the script module to the media player in response to connection of the media player;
A packetizing unit for packetizing the media data to generate a transmission packet; And
And a web server that forms a communication session with the playback apparatus and transmits the transport packet to the media playback apparatus in response to a request from the media playback apparatus,
Wherein the script module comprises: a process of receiving the communication packet through the communication session; a process of packaging the video frames included in the communication packet into a unit number and converting the packet into chunk data; And outputting the audio data included in the communication packet in synchronization with the chunk data based on the time information about the time when the audio data is decoded and outputted by the media restoring unit installed in the apparatus. 15. The media service apparatus according to claim 14, wherein the script module is a code written in Javascript that can be parsed in the web browser. 15. The method of claim 14,
Wherein the time information indicates a time at which a start portion of the chunk data corresponding to the time information is rendered by the renderer in the media playback apparatus. 15. The method of claim 14, wherein the process of synchronizing and outputting the audio data comprises:
And decoding the audio data included in the communication packet to restore audio, and outputting the restored audio in synchronization with the time information provided in units of the chunk data. 15. The method of claim 14, wherein the process of synchronizing and outputting audio comprises:
Buffering the audio data by synchronizing with the chunk data corresponding to the time information;
Decoding the buffered audio data by a script that can be parsed by the web browser; And
And rendering the decoded audio data. 15. The method of claim 14,
Wherein the chunk data is compressed data before being decoded and the audio data synchronized with the chunk data is decoded by the media reproduction apparatus and decompressed. A receiving unit for receiving the media data generated by the media service apparatus using a communication protocol supporting the web service;
A data separator for separating the received media data into first media type data and second media type data;
A container unit for packaging the frames constituting the first media data into unit numbers and converting the frames into chunk data;
The method comprising: decoding the converted chunk data by a decoder embedded in a web browser to restore a first medium; and outputting the restored first media, To the chunk data unit; And
And a sync unit for outputting the second media data in synchronization with the restored first media based on time information provided in the chunk data unit, in synchronization with video and audio on a web browser.

Images