KR100796550B1 - System for realtime multimedia transcoding and direct streaming in p2p network and method thereof - Google Patents

Abstract

A system and a method for streaming multimedia directly in real-time based on P2P(Peer-to-Peer) are provided to enable a PC to convert multimedia contents stored in the PC into a serviceable format, thereby enabling the PC to offer a streaming service to other terminals directly through the P2P network. A service terminal(100) searches, converts, and streams a file in real-time when multimedia contents streaming is requested from a user terminal(200) connected through the P2P network. The user terminal includes a player for playing a streaming file received from the service terminal. A linking server(300) downloads a service application to the service terminal, provides an interface for searching the multimedia contents stored in the service terminal to the user terminal, and connects the user and service terminal through the P2P network. The service terminal includes a file system storing the multimedia contents, a P2P platform, a service terminal state keeper, a request processor, a codec converter(120) and a streaming part(110).

Description

System for realtime multimedia transcoding and direct streaming in P2P network and method

1 is a schematic structural diagram of a conventional content streaming service system.

2 is a schematic structural diagram of another conventional content streaming service system.

3 is a schematic diagram of a P2P based real-time multimedia direct streaming system according to the present invention.

4 is a detailed block diagram of a P2P based real-time multimedia direct streaming system according to the present invention.

5 is a flowchart of a P2P-based real-time multimedia direct streaming system according to the present invention.

6A and 6B illustrate another embodiment of a P2P-based real-time multimedia direct streaming process according to the present invention.

`` Explanation of symbols for main parts of drawings ''

10, 100: (user) service terminal 20, 200: user terminal

110: streaming unit 120: codec conversion unit

130: request processing unit 140: P2P platform

150: service terminal state maintenance unit 160: file system

300: linked server unit 310: P2P server

311: user authentication performing section 312: service terminal status management section

       313: connection relay unit 314: relay unit

320: search server 321: database

322: content search interface 330: user directory server

331: Database 332: membership / management interface

The present invention relates to a P2P-based real-time multimedia direct streaming system and method, and more particularly, P2P-based real-time multimedia that can be converted into a serviceable form in a PC that stores video multimedia to play the multimedia video directly to the client through P2P A direct streaming system and method.

Unlike in the past, current netizens watch movies and dramas on the Internet, share photos and videos taken with cameras / camcorders on blogs or video UCC sites, share them with friends, convert newly purchased CDs to music files, and listen to them through MP3 players. Convert / save animation to terminal such as PMP, watch on subway or bus, and watch TV on mobile phone / DMB terminal.

With the advent of the multimedia era, all kinds of UCCs, or MP3s, DMBs, cameras, camcorders, PMPs, Wibros, HSDPAs, etc. are constantly pouring in to meet users' needs. Multimedia video service is simply to deliver video files, which is converted into one format that can be serviced by a PC and serviced by a server, or transmitted to a server to be converted / serviced by a server.

1 is a schematic structural diagram of a conventional content streaming service system.

When uploading the multimedia content from the user terminal 10 of FIG. 1 to the upload server 15, the conversion server 40 is stored while being stored in the storage 30 to convert the content into a format capable of streaming playback. The content is delivered back to the streaming server 25 from the storage 30, the streaming service is transmitted from the streaming server 25 to the user terminal 20, and the streaming is reproduced in the user terminal 20.

2 is a schematic structural diagram of another conventional content streaming service system.

In FIG. 2, the user terminal 10 first converts the multimedia content into a format capable of streaming playback, uploads the uploaded content to the upload server 150, stores the converted content file in the storage 30, and requests the user terminal 20. In accordance with the streaming server 25 to the streaming service to the user terminal 20.

However, in the conventional content streaming service system, the burden of storage for storing a huge amount of multimedia content and the burden of traffic for service of content streaming become heavy on the Internet service provider.

In addition, additional CPU power is required to convert to a serviceable format, and each newly released terminal requires a player and client implementation for playback.

On the other hand, the user PC takes most of the idle traffic for a time other than upload / download.

There are now thousands of related codecs for creating / playing multimedia.

When using a certain terminal, the multimedia is generated only in the format of a specific codec. On the other hand, in order to play the multimedia, the codec applied when generating the multimedia must be installed in the terminal. In addition, there is a need to install or convert the format into a format that the terminal can play.

In addition, most portable multimedia terminals except PC terminals cannot store a large amount of multimedia because their storage space is not so large. In addition, since the terminal is not directly connected to the Internet, there is a hassle of downloading from a PC terminal and copying it to the portable terminal.

The present invention has been made to solve the conventional problems as described above, P2P-based to convert the multimedia contents of the personal PC stored video multimedia into a serviceable form on the PC to play services directly from the PC to another personal terminal through the P2P network To provide a multimedia streaming real-time transmission and optimization system and method.

It is also an object of the present invention to provide a real-time transcoding method for converting a myriad of multimedia codecs related to multimedia contents into a unified format in real time.

P2P-based real-time multimedia direct streaming system according to the present invention for achieving the above object, when receiving a multimedia content streaming request from the user terminal connected to the P2P network, retrieves the requested file codec conversion and real-time to the user terminal A user terminal having a service terminal for streaming transmission, a playback player for reproducing a streaming file received from the service terminal, and downloading a service application to the service terminal, and transmitting the multimedia content search interface stored in the service terminal to the user terminal. It includes and provides an interworking server unit for P2P connection between the user terminal and the service terminal.

In addition, in the P2P-based real-time multimedia direct streaming system according to the present invention, the service terminal is a file system for storing multimedia content, P2P platform for performing a P2P connection to the user terminal through the interworking server unit, and the user terminal A service terminal state maintaining unit for maintaining a P2P connection between the service terminals, a request processing unit for receiving and processing a multimedia content streaming request requested by the user terminal, and searching the requested multimedia contents in a file system in real time in a single file format. And a streaming unit for transmitting a codec conversion unit for outputting the multimedia content coded by the codec conversion unit to the user terminal.

In addition, in the P2P-based real-time multimedia direct streaming system according to the present invention, the codec converter is characterized in that the conversion of the real-time codec to the WMV format file using the ASF writer filter. .

In addition, in the P2P-based real-time multimedia direct streaming system according to the present invention, the multimedia content file stored in the file system is characterized in that it comprises FLV, AVI, MPEG, MOV, MP3, MP4 format.

In addition, in the P2P-based real-time multimedia direct streaming system according to the present invention, the companion server unit is a user authentication performing unit for authenticating a user of a user terminal or a service terminal, confirms the state of the service terminal and access information A service terminal state management unit which maintains and manages and provides connection information corresponding to the P2P connection request, a connection relay unit which receives a P2P connection request transmitted from the user terminal or transmits a response using an HTTP protocol, and the When the service terminal is unable to access the P2P by the firewall or the NAT, it is characterized in that it comprises a relay unit for establishing a connection path with the service terminal and relays the P2P connection request transmission and response.

In addition, in the P2P-based real-time multimedia direct streaming system according to the present invention, the companion server unit provides a user interface for the P2P user terminal, a user directory server for storing user ID, password information, and the user terminal for the It further comprises a search server for providing a search interface for the list of multimedia content stored in the service terminal.

In addition, in the P2P-based real-time multimedia direct streaming system according to the present invention, the playback player of the user terminal is characterized in that the window media player.

On the other hand, P2P-based real-time multimedia direct streaming method according to the present invention, the playback player of the user terminal requesting the streaming of the multimedia content to the service terminal connected to the user terminal P2P, the service terminal receiving the streaming request Requesting unit for real-time codec conversion for encoding the requested multimedia content to a codec conversion unit, performing the real-time codec conversion by searching the requested multimedia content in the codec conversion unit, the request of the service terminal The processor may include streaming the codec-converted multimedia content to a streaming unit, and requesting the streaming unit to transmit the codec-converted multimedia content to a user terminal to play the stream in a playback player of the user terminal. All.

 In addition, in the P2P-based real-time multimedia direct streaming method according to the present invention, the request processing unit receiving the streaming request further comprises the step of transmitting a change request in the streaming mode to the playback player of the user terminal. will be.

In addition, in the P2P-based real-time multimedia direct streaming method according to the present invention, the playback player of the user terminal further comprises the step of requesting and receiving the header portion before starting the streaming.

In addition, in the P2P-based real-time multimedia direct streaming method according to the present invention, when the playback stop request from the playback player of the user terminal occurs, the playback player forcibly closing the socket connection to which the content data is transmitted; The streaming unit of the service terminal is characterized in that it further comprises the step of releasing resources to release and terminating the process of the codec conversion unit.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

3 is a schematic diagram of a P2P based real-time multimedia direct streaming system according to the present invention.

In FIG. 3, the P2P-based real-time multimedia direct streaming system according to the present invention shows that the user's personal terminals are in an office, home, remote or mobile environment and are placed in a wired or wireless Internet environment.

Each of the personal terminals has a storage device, and various contents such as a video file and a document and a music file are respectively stored.

The present invention converts multimedia contents stored in the service terminal 100 into a format capable of streaming playback among the various personal terminals, and directly streams the contents to various wired / wireless users connected to the service terminal 100 by a P2P network. In the terminal 200, the multimedia content is played by streaming.

Here, the service terminal 100 or the user terminal 200 connected to the P2P network through the wired or wireless Internet according to the present invention includes a PC, a laptop, a PDA, a mobile phone, a smartphone, a PMP, and various other personal digital terminals. However, the system according to the present invention is not limited to the "personal" terminal, and the scope of the present invention may be expanded to various terminals other than the "personal" terminal.

Detailed configuration of the P2P based real-time multimedia direct streaming system according to the present invention is as shown in FIG.

Referring to Figure 4, P2P-based real-time multimedia direct streaming system according to the present invention, when receiving a multimedia content streaming request, the service terminal 100 and the service terminal 100 to search for the codec conversion and to transmit the real-time streaming It includes a user terminal 200 having a playback player for playing the streaming file received from the) and the interworking server unit 300 for P2P connection between the user terminal 200 and the service terminal 100.

The service terminal 100 may download and install a service application for performing a streaming service from the companion server unit 300 to perform a streaming service function.

In addition, the companion server unit 300 provides the user terminal 200 with an interface for searching for multimedia content stored in the service terminal 100.

Referring to FIG. 4, the companion server unit 300 is a P2P server 310 for performing P2P connection between the service terminal 100 and the user terminal 200, and searches for multimedia content in the user terminal 200. It includes a search server 320 and a user directory server 330 for P2P user management to use the streaming service of the multimedia content via the interlocking server unit 300.

Here, the P2P server 310 includes a user authentication unit 311, a service terminal state management unit 312, a connection relay unit 313, and a relay unit 314.

The user authentication unit 311 performs authentication of a user of the user terminal 200 or the service terminal 100 that has been issued a user ID as a P2P member using a content service according to a multimedia content streaming request.

The service terminal state management unit 312 checks the connection state of the service terminal 100 and maintains and manages connection information such as IP and port information required when transmitting a service request to the service terminal 100 and the P2P connection request. Provides connection information corresponding to

The access relay unit 313 receives an access request transmitted from the P2P user terminal 200 using an HTTP protocol or transmits a response.

If the service terminal 100 is located inside a firewall or NAT and fails to deliver the streaming request to the service terminal 100, a connection path is established with the service terminal 100 and the service terminal 100 is connected to the service terminal 100. It may further comprise a relay unit 314 for relaying the delivery and response of the streaming request.

The relay unit 314 relays the HTTP request / response in the middle through a connection with the service terminal 100 prepared in advance when the service terminal 100 cannot directly access the terminal by a firewall / NAT. It will play a role similar to.

In addition, the search server 320 provides the user terminal with a database 321 for storing a list of multimedia contents stored in the service terminal and a content search interface 322 therefor.

In addition, the user directory server 330 stores a member registration / management interface 332 which provides a user interface for P2P membership, member information input, and change for a P2P user, and stores information such as a user ID and a password. 410.

The service terminal 100 registers a service application for streaming multimedia content of various formats directly to the user terminal 200 through a user directory server 330 to a P2P site or a site for providing content and downloads the terminal application. After installation, it is driven by the service terminal 100 by being executed.

Referring to FIG. 4, a codec conversion unit 120 installed and configured in the service terminal 100 by the service application and converting the multimedia contents of various formats stored in the file system 160 into a single unitary format codec converting unit 120 and The P2P platform 140 and the service terminal state maintaining unit 150 for P2P connection and connection maintenance with the request processing unit 130 for receiving and processing the streaming processing unit 110 and the streaming processing request for direct streaming service. Include.

Here, the P2P platform 140 performs a P2P connection with the user terminal 200 through the interworking server unit 300.

The service terminal state maintaining unit 150 performs user authentication of the service terminal 100 and continuously transmits network state information of the service terminal to the server interlocking unit 300.

The service terminal state maintaining unit 150 is initially connected to the server interlocking unit 300 to perform user authentication, and transmits access information to the server interlocking unit 300, and is continuously connected online every predetermined time. By transmitting a message notifying that there is a change in network status or situation, such as network connection termination, IP change, etc. by sending a message notifying the server interlocking unit 300 always maintains the connection state according to the request.

Now, the codec converter 120 and the streaming unit 110, which are core components for processing the streaming service, will be described.

The real-time codec converter 120 may be produced by using DirectShow technology when the operating system of the service terminal is MS Windows series, uses a ffdshow video decoder filter for subtitle processing, and streams the converted multimedia content in a WMV format. The ASF Writer filter will be used to pass to 110.

The ffdshow video decoder assumes that an integrated codec pack is installed on a user's computer.

ASF Writer filter is installed by default when user PC (server PC) is WinXP or higher. Separate installation file is required for Win2000 users.

When using the ffdshow video decoder filter for subtitle processing, some additional registry value setting is required, and a function for performing this function is always executed when the real-time codec converter 120 is executed.

The codec converter 120 itself may be executed regardless of the existence of the streaming unit 110, and may be directly connected to a Windows media player or the like to watch a video.

The codec converter 120 may execute a target file, a standby port, and a transcoding start time of the file system 160 in which multimedia contents are stored.

In addition, the streaming unit 110 plays a role of reproducing a media service function of Windows Server 2003 as a playback player of the user terminal 200, for example, a streaming request of a Windows media player.

Streaming-only protocols such as http, mms and rtsp can be used for real-time media transmission between the streaming and playback clients.

The streaming protocol is mainly dependent on the OS environment of the playback client and the service terminal. For example, it is preferable to use http or mms in an environment using an MS Windows-based OS and a Windows media player.

In addition, the format of the file system 160 retrieved and transcoded into the WMV by the codec converter 120 may be varied, including FLV, AVI, MPEG, MOV, MP3, and MP4 formats.

5 is a flowchart of a P2P-based real-time multimedia direct streaming system according to the present invention.

Referring to FIG. 5, in the P2P-based real-time multimedia direct streaming transmission process according to the present invention, the playback player of the user terminal 200 requests streaming of multimedia content to the P2P-connected service terminal 100 (step ①). .

The request processing unit 130 of the service terminal 100 receiving the streaming request requests the real-time codec conversion for the WMV encoding to the codec converter 120 for the requested multimedia content (step ②).

Next, the codec converter 120 searches for the requested multimedia content to perform real-time codec conversion (step ③).

The request processing unit 130 of the service terminal 100 requests the streaming unit 110 to process the streaming of the codec-converted multimedia content (step ④).

Finally, the streaming unit 110 transmits the codec-converted multimedia content to the user terminal 200 to play the streaming in the player of the user terminal 200 (step ⑤).

6A and 6B illustrate another embodiment of a P2P based real-time multimedia direct streaming system transmission processing process according to the present invention.

In operation S10, a window media player, which is a playback player of the user terminal 200, requests a video stored in a remote PC to the streaming unit 110 of the service terminal 100 using an HTTP protocol that is advantageous for passing through a firewall.

At this time, the requested streaming unit 110 checks whether there is a requested file, and if there is no file, sends a 404 File Not Found response, and if there is a file, sends a request to change the streaming mode to the window media player of the user terminal 200. .

The streaming unit 110 is executed while maintaining the connection with the user terminal 200. Execution arguments are

<Original video file absolute path> <Profile file absolute path> <Start time (ms)> -p <port> <Disconnection wait time (s)>

The original image path obtained at the request is converted to an absolute path and designated at the time of execution of the codec converter 120. The port number designates a random free port every time the codec converter 120 starts, so that even if several streaming units 110 are executed at the same time, a port collision does not occur (S20).

The streaming unit 110 accesses the codec conversion unit 120 executed above and sends a request (S30). This request mimics a request sent when a direct connection is made using various user terminals.

The codec converter 120 responds to the request of the streaming unit 110, but when it is normal, 200 OK comes (S40). If the response is not 200 OK, the streaming unit 110 no longer provides a proper response to the user terminal 200, and thus sends a "500 Server Internal Error" to the user terminal 200 and disconnects.

The codec converter 120 responds to the request of the streaming unit 110, and at this time, transmits the ASF header part (S50).

After making a first GET request, a response message is made and sent to the window media player of the user terminal 200 which is in a standby state (S60).

Here, before starting the streaming, the Windows Media Player requests only the header portion except for the image data for the purpose of testing whether the file currently requested for streaming is a file of a format that can be played by the user or whether the file can be streamed. S70). This process may be repeated 2-3 times.

In this case, as shown in FIG. 6A, when the file currently requested for streaming by the Windows Media Player cannot play by itself, the codec converter 120 terminates the process of the streaming unit 110 ( S80).

Windows Media Player usually disconnects the socket at this stage (S90).

As illustrated in FIG. 6B, when the file currently requested for streaming by the window media player is capable of being played by itself, the codec converter 120 transcodes the corresponding multimedia content file. The streaming unit 110 reads the transcoded data to the WMV from the codec converter 120 and relays the data to the window media player of the waiting user terminal (S85 and S95).

In addition, in order to handle the pause function and the end of normal video playback, the "current playback time" of the currently transmitted data is read and stored as binary data.

After the desired content or video is played on the window media player of the user terminal and the image is played, the user terminal may request a subtitle file, search, pause and play, and stop playback.

When a play stop request is generated by pressing a play cleanup button on a window media player of the user terminal, the window media player forcibly closes the socket connection to which data has been transmitted.

The streaming unit 110 detects this, releases the resources to be released by itself, and terminates the related codec conversion unit 120 process.

Although the present invention has been described in detail only with respect to the specific embodiments described, it will be apparent to those skilled in the art that various modifications and variations are possible within the technical scope of the present invention, and such modifications and modifications belong to the appended claims. .

According to the P2P based real time multimedia direct streaming system of the present invention, the following positive effects are expected.

The internet service provider can directly stream to the terminal for various multimedia contents without storage for storing the contents.

In addition, since most of the traffic required for streaming playback of multimedia content is generated between terminals, it is possible to reduce costs of servers and Internet data centers.

On the other hand, individual users can watch multimedia video contents stored in their PCs in real time on any device under various wired and wireless network environments.

Claims (11)

A service terminal for retrieving the requested file, converting the codec, and transmitting the real-time streaming to the user terminal when the multimedia content streaming is requested from the user terminal connected to the P2P network; A user terminal having a player for playing the streaming file received from the service terminal; And The service terminal may include downloading a service application, providing a multimedia content search interface stored in the service terminal to the user terminal, and including an interworking server unit for P2P connection between the user terminal and the service terminal. A file system for storing multimedia content; A P2P platform for performing P2P access to the user terminal through the interworking server unit; A service terminal state maintaining unit maintaining a P2P connection between the user terminal and the service terminal; A request processor for receiving and processing a multimedia content streaming request requested by the user terminal; A codec converter for retrieving the requested multimedia content from a file system and converting the codec into a single file format in real time; And P2P-based real-time multimedia direct streaming system characterized in that it comprises a streaming unit for streaming the output of the codec-converted multimedia content directly to the user terminal at the same time as the codec conversion. delete The method of claim 1, The codec converter, And converting the requested multimedia content into a WMV format file using a ASF writer filter. The method of claim 1, The multimedia content file stored in the file system includes a FLV, AVI, MPEG, MOV, MP3, MP4 format P2P-based real-time multimedia direct streaming system. The method of claim 1, The linked server unit, A user authentication performing unit for authenticating a user of a user terminal or a service terminal; A service terminal state manager to check a state of the service terminal, maintain and manage access information, and provide access information corresponding to the P2P access request; An access relay unit configured to receive a P2P connection request transmitted from the user terminal or transmit a response using an HTTP protocol; And If the service terminal is unable to access the P2P by a firewall or NAT, P2P-based real-time multimedia direct streaming comprising a relay unit for establishing a connection path with the service terminal and relaying the delivery and response of the P2P connection request. system. The method of claim 1, The linked server unit, A user directory server providing a user interface to a P2P user terminal and storing user ID and password information; And And a search server for providing a search interface for the multimedia content list stored in the service terminal to the user terminal. The method of claim 1, P2P based real-time multimedia direct streaming system, characterized in that the playback player of the user terminal is a window media player. Requesting a player for playing multimedia content to a service terminal connected to the user terminal by P2P; Requesting, by the request processing unit of the service terminal receiving the streaming request, the real-time codec conversion for encoding to the codec converter for the requested multimedia content; Retrieving the requested multimedia content from the codec converter and performing real-time codec conversion into a single file format; Requesting a processing unit of a service terminal to request a streaming service to simultaneously process the codec converted multimedia content to a streaming unit; And P2P-based real-time multimedia direct streaming method comprising the step of streaming the codec-converted multimedia content in real time to the user terminal by the streaming unit in the playback player of the user terminal. The method of claim 8, The request processing unit receiving the streaming request further comprises the step of transmitting a change request in the streaming mode to the playback player of the user terminal P2P-based real-time multimedia direct streaming method. The method of claim 8, The P2P-based real-time multimedia direct streaming method of the user terminal further comprises the step of requesting and receiving the header portion before starting the streaming. The method of claim 8, If the playback stop request is generated from the playback player of the user terminal, the playback player forcibly closing the socket connection to which the content data was transmitted; And The P2P-based real-time multimedia direct streaming method further comprises the step of releasing the resources to release the service terminal and terminate the process of the codec conversion unit.

Images