KR101242478B1 - Real time personal broadcasting system using media jockey based on multi-angle - Google Patents

Abstract

PURPOSE: A real time individual broadcasting system using a multi angle based media jockey is provided to reduce difference due to change of a channel by inserting a low-resolution image in change of an angle. CONSTITUTION: A decoding module(13) decodes a received video from a buffering module(11). A user interface module(19) displays an individual broadcasting interface to a user. The user interface module transmits a control message of a user to the decoding module and a client manager module(17). A streaming module(15) streams the received video to a client terminal. The client manager module provides client terminal information to the user interface module and the streaming module. [Reference numerals] (11) Buffering module; (13) Decoding module; (15) Streaming module; (17) Client manager module; (19) User interface module; (AA,BB,EE,JJ) Streaming media data; (CC,GG,HH,LL) Control message; (DD) Decoded media data; (FF,II) Client information; (KK) Client connection request

Description

Real Time Personal Broadcasting System using Media Jockey based on Multi-Angle}

The present invention relates to a real-time personal broadcasting system using a multi-angle-based media jockey, and more particularly, by providing images of the requested angles to a plurality of client terminals, and inserting and transmitting a low-resolution video when converting angles. The present invention relates to a personal broadcasting system capable of reducing the heterogeneity felt by a user during channel switching.

Recently, social network services such as social TV, personal multimedia services, and personal broadcast services have been developed. Social TV can provide an experience sharing experience using VOD or real-time video with social network services. However, personalized content is not supported on social TV. The personal broadcast service may generate personal content and provide a personal broadcast channel. However, they do not support traditional multimedia displays and real-time generated content.

Thus, personal broadcast services do not support multiple content or multi-angle media but only pre-generated content or traditional media such as TV shows and movies.

In the case of Korean Patent Publication No. 10-2006-0037721, only a broadcast service system for producing personal content and providing the produced content is disclosed in response to a request from a client computer. I'm not doing it.

Korean Patent Publication No. 10-2006-0037721

The present invention has been made in view of the above-described problems, and an object thereof is to provide a multi-angle-based customized personal broadcasting system based on images received from various media sources.

To this end, a personal broadcast system according to an embodiment of the present invention includes a media jockey module for processing images received from a plurality of media sources with respect to the same event, and one or more clients for receiving and playing back the images processed by the media jockey module. And a terminal, wherein the media jockey module processes or edits images received from the plurality of media sources and provides them to the client terminal, or selects one image among images received from the plurality of media sources, and selects the client terminal. Can be provided to

The plurality of media sources may include one or more of an internet medium, a broadcast medium, a broadcaster camera, and a storage server.

An image received from the plurality of media sources may include a multi angle image.

The media jockey module includes a buffering module for storing images received from the one or more media sources, a decoding module for decoding the images received from the buffering module, and a video decoded by the decoding module to receive a personal broadcast. A user interface module for displaying an interface for providing to a user, receiving a user's control command, a streaming module for streaming an image received from the buffering module to the client terminal, and managing a connection request of the client terminal; It may include a client manager module for providing information of the client terminal to the user interface module and the streaming module.

The client terminal may include: a buffering module for storing an image received from the media jockey module, a decoding module for decoding the image received from the buffering module, and a video decoded by the decoding module to be played back to the client terminal. It may include a user interface module for displaying an interface for the user, and receives a control command of the user.

The media jockey module, when converting an image provided to the client from an angle different from the current angle, has a lower resolution than the image of the converted angle prepared in advance during a delay time before the image of the other angle is reproduced. The video may be inserted into and provided to the client terminal.

The media jockey module may adjust the multi-angle video at the same frame rate, and assign a sequence number to each frame to synchronize the reproduction time of the multi-angle video.

According to the personal broadcasting system according to the present invention, the media jockey module is used to perform personalized broadcasting by using internet broadcast images, TV broadcast images, broadcaster camera images, videos stored in computers, and various videos found on the Internet. Can provide.

In addition, when providing a multi-angle image, by providing a low-resolution image of the angle to be converted during the delay time during the angle conversion, it is possible to minimize the heterogeneity caused by the delayed channel channeling.

1 is a schematic structural diagram of a personal broadcasting system according to an embodiment of the present invention.
2 is a schematic structural diagram of a media jockey of a personal broadcasting system according to an embodiment of the present invention.
3 is a schematic structural diagram of a client terminal of a personal broadcasting system according to an embodiment of the present invention.
4 is a diagram illustrating a channel switching method of an existing personal broadcasting system.
5 is a diagram illustrating a channel switching method of a personal broadcasting system according to an embodiment of the present invention.
6 is a diagram illustrating a multi-step segmentation technique of the present invention.
7 is a diagram illustrating a structure of an existing NAL.
8 is a diagram illustrating a structure of a segmented NAL according to the segmentation method of the present invention.
9 is a diagram illustrating an example of channel conversion of a personal broadcast system according to an embodiment of the present invention.
10 is a diagram showing the structure of an interface of a media jockey module of a personal broadcasting system according to an embodiment of the present invention.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. However, the accompanying drawings and the following description are only possible embodiments of the personal broadcasting system according to the present invention, and the technical idea of the present invention is not limited to the following.

1 is a schematic structural diagram of a personal broadcasting system according to an embodiment of the present invention.

Referring to FIG. 1, a personal broadcasting system 1 according to an embodiment of the present invention includes a media jockey module 10 and a plurality of client terminals 20: 20a, 20b, 20c, 20d, 20e, and 20f. Is done.

The media jockey module 10 receives streaming data from a plurality of media sources, processes the data, and transmits the stream data to the plurality of client terminals 20. Here, the term “media jockey” refers to a role that a specific terminal constituting the personal broadcasting system 1 receives and processes media data received from various client terminals 20 and provides the same to the client terminal 20. Point to the terminal to be.

As shown in FIG. 1, the client terminal 20 may be a storage device 20a, a specific camera source 20b, an internet source 20c, a smart device 20d, an IPTV 20e, a notebook 20f, or the like. Can be. In this case, the client terminal 20 is a device that stores data obtained externally, such as the storage device 20a, or a device that obtains data in real time, such as a camera source 20b, or real-time from the outside, such as a smart device 20d. It can be a device that can get data or store it at the same time.

In this case, the camera source 20b may include a panoramic camera that supports multiple angles. When the camera source 20b is a multi-angle panoramic camera or a camera of various angles for the same channel, the personal broadcasting system 1 may provide images of various angles to the client terminal 20 as described below. have.

Here, the "channel" refers to an image provided by various media sources for one event. That is, in the case of the baseball game relay image, it includes an air wave relay image, an image of Internet personal broadcasting, an image obtained from a camera of a broadcaster arranged in a stadium, and the like for the same baseball game. That is, the present specification will be described based on the case where various multi-angle images of the same channel, such as one baseball game, are obtained.

Hereinafter, a case in which there is a media jockey module 10 capable of acquiring a multi-angle image and a plurality of client terminals 20 connected to the media jockey module 10 will be described.

2 is a schematic structural diagram of a media jockey module of a personal broadcasting system according to an embodiment of the present invention.

2, a media jockey module 10 according to an embodiment of the present invention may include a buffering module 11, a decoding module 13, a streaming module 15, a client manager module 17, It comprises a user interface module 19.

The buffering module 11 temporarily stores an image received from a media source and transmits the image to the decoding module 13 or the streaming module 15.

The decoding module 13 decodes an image received from the buffering module 11 and receives a control message from the user interface module 19 and transmits the control message to the buffering module 11.

The user interface module 19 receives an image decoded by the decoding module 13, displays an interface for providing a personal broadcast to a user, receives a user's control message (command), and the decoding module 13 and It transmits the data to the client manager module 17.

The streaming module 15 plays a role of streaming the image received from the buffering module 11 to the client terminal 20.

The client manager module 17 manages a connection request of the client terminal 20 and provides information of the client terminal 20 to the user interface module 19 and the streaming module 15.

3 is a schematic structural diagram of a client terminal of a personal broadcasting system according to an embodiment of the present invention.

Referring to FIG. 3, the client terminal 20 according to an embodiment of the present invention includes a buffering module 21, a user interface module 23, and a decoding module 25.

The buffering module 21 temporarily stores an image received from the media jockey module 10.

The decoding module 25 decodes an image received from the buffering module 21. At this time, the decoding module 25 converts the JAVA format data received from the buffering module 21 into the FFMPEG (C code) format.

The user interface module 23 receives an image decoded by the decoding module 25, displays an interface for playback to the client terminal 20, and receives a user's control command.

With such a configuration, the personal broadcasting system according to the present invention may provide images of different angles in response to requests from the plurality of client terminals 20 connected to the media jockey module 10.

However, in the case of zapping a channel in order to provide an image of another angle according to the user's request, there is a delay time until a high resolution image of a new channel arrives, and the user feels a great heterogeneity. This heterogeneity causes a great inconvenience for users who are used to the conventional air broadcasting. Therefore, there is a need to remove such channel zapping delay. The following briefly describes the existing channel chipping delays and describes a method for removing channel channeling delays adopted by the present invention.

4 is a diagram illustrating a channel switching method of an existing personal broadcasting system.

Referring to FIG. 4, it can be seen that the media jockey module 43 provides a user with a stream stored in the stored streaming database 41 through the media jockey interface 45. Stream 1 is currently selected and played. That is, the largest baseball playing screen on the left on the media jockey interface 45 is the screen currently being played.

In this case, when a screen of one other angle of the right thumbnail image of the media jockey interface 45 is requested by the user's request, the media jockey module 43 may newly start the playback service of the stream 2. However, as shown in FIG. 4, there is a delay time until a new stream 2 video arrives.

5 is a diagram illustrating a channel switching method of a personal broadcasting system according to an embodiment of the present invention.

Referring to FIG. 5, the media jockey module 53 provides the streaming data in the streaming database 51 to the user through the media jockey interface 55.

Likewise, when stream 2 is played during stream 1 playback at the request of a user, unlike the conventional method, a previously stored video having a lower resolution than that of stream 2 (high resolution) requested is played for a delay time.

At this time, the streaming database 51 is pre-stored in the low-resolution data, down-sampled separately from the high-resolution streaming data generally provided to the user. 5 is stored at a resolution of 320p, which is half of the 720p resolution. This will be described in more detail with reference to FIG. 6.

6 is a diagram illustrating a multi-step segmentation technique of the present invention.

Referring to FIG. 6, when the segments exist in each channel, as shown in FIG. 5, the playback stage is changed.

That is, when stream 1 of channel 1 is being played while stream 1 of resolution 720p is being played, streams of resolution 720p are played while segments of resolution 320p are played for a delay time.

As can be seen in Figure 7, the H.264 standard stream is composed of a Network Abstraction Layer (NAL) unit. At this time, one unit is independently decoded into an Instantaneous Decoding Refresh (IDR) picture. This IDR picture may be one option of H.264 encoding. If the IDR filter interval of the NAL unit is one, the shopping delay is zero. However, since each NAL unit becomes an IPR picture, very large network bandwidth is required.

Thus, the IDR picture is not considered correlation coding, and the IDR picture is larger than a typical NAL unit. Therefore, the optimal spacing between IDR pictures should be considered for the system environment.

In FIG. 7, RBSP stands for Raw Byte Sequence Paylode, stands for Sequence Parameter Set (SPS), stands for Picture Parameter Set (PSS), and NAL means sum of NAL units and payload.

8 is a diagram illustrating a structure of a segmented NAL according to the segmentation method of the present invention.

As shown in FIG. 8 of the present invention, stream data having a structure of each segmented NAL is used as respective channel data, and data reproduced during delay cyan is composed of low resolution stream data.

In this manner, the present invention adjusts each multi-angle video by the same frame rate, and assigns a sequence number to each frame (predefining unit 81, segment 1 (83), segment 2 (85), segment) in FIG. 8). 3 (87)), a playback time synchronization technique of matching the starting points of all the multi-angle images with the same starting point of the base point (a game starting point in the case of a baseball game) is used. In this way, all multi-angle playback start synchronization is performed, and when the angle is converted, heterogeneity due to the delay time can be solved by reproducing a low resolution image using the segmented NAL structure during the delay time.

9 is a diagram illustrating an example of angle conversion of a personal broadcast system according to an embodiment of the present invention.

As shown in FIG. 10, the personal broadcasting system 1 according to the present invention has a channel from the angle 1 101 to the angle 2 103 and the angle 2 103 to the angle 3 105 at the request of the user. Channels can be switched without a chipping delay.

10 is a diagram showing the structure of an interface of a media jockey module of a personal broadcasting system according to an embodiment of the present invention.

10, the full screen 111, the screen editing 112, the group start response 113, the mobile angle request confirmation 114, and the thumbnail image source 115 may be configured.

The full screen 111 corresponds to a screen currently provided to all users as a default.

The screen edit 112 is an interface unit that allows the media jockey to edit the screen to be provided to the user.

Group response start 113 corresponds to a support form specifically set in the interface of the media jockey module. In the case of relaying a baseball game as shown in FIG. 11, when activating the group response start 113, an interface for experiencing group response may be provided to each client terminal 20.

The mobile angle request confirmation 114 is an interface unit that can confirm the request of the client terminal 20. When there are a plurality of participating client terminals 20, a request of each participant may be confirmed. In the case of FIG. 11, it can be seen that each participant requests a video of a specific angle.

The thumbnail image sources 115: 115a, 115b, 115c, 115d, and 115e are interface units that display thumbnail images of game screens of various angles currently collected. As shown in FIG. 11, when selecting each thumbnail image, selections such as “send to full screen”, “edit screen”, “send to mobile”, and the like are provided, and an interface is provided that can be applied only to specific participants.

The personal broadcasting system according to the present invention may provide customized image information to various client terminals based on the multi-angle.

1: personal broadcast system 10: media jockey module
11: buffering module 13: decoding module
15: streaming module 17: client manager module
19: user interface module 20: client terminal
21: Buffering Module 23: User Interface Module
25: decoding module

Claims (7)

A media jockey module for processing images received from a plurality of media sources with respect to the same event; And
At least one client terminal for receiving and playing the image processed by the media jockey module,
The media jockey module processes or edits images received from the plurality of media sources and provides them to the client terminal, or selects one image among images received from the plurality of media sources and provides the image to the client terminal.
The plurality of media sources includes one or more of an internet medium, a broadcast medium, a broadcaster camera, and a storage server;
The images received from the plurality of media sources include multi-angle images,
The media jockey module,
A buffering module that stores an image received from the one or more media sources;
A decoding module for decoding an image received from the buffering module;
A user interface module which receives an image decoded by the decoding module, displays an interface for providing a personal broadcast to a user, and receives a user's control command;
A streaming module for streaming the image received from the buffering module to the client terminal; And
And a client manager module for managing a connection request of the client terminal and providing information of the client terminal to the user interface module and the streaming module. delete delete delete The method of claim 1,
The client terminal,
A buffering module that stores an image received from the media jockey module;
A decoding module for decoding an image received from the buffering module; And
And a user interface module for receiving an image decoded by the decoding module, displaying an interface for playback to the client terminal, and receiving a user's control command. A media jockey module for processing images received from a plurality of media sources with respect to the same event; And
At least one client terminal for receiving and playing the image processed by the media jockey module,
The media jockey module processes or edits images received from the plurality of media sources and provides them to the client terminal, or selects one image among images received from the plurality of media sources and provides the image to the client terminal.
The plurality of media sources includes one or more of an internet medium, a broadcast medium, a broadcaster camera, and a storage server;
The images received from the plurality of media sources include multi-angle images,
The media jockey module,
When converting an image provided to the client terminal into an image of an angle different from the current angle, a lower resolution image is inserted by inserting a lower resolution image than the image of the other angle to be converted in advance during a delay time before the image of the other angle is reproduced. Personal broadcasting system, characterized in that provided to the client terminal. The method according to claim 6,
The media jockey module,
And adjusting the multi-angle video at the same frame rate, and assigning a sequence number to each frame to synchronize the reproduction time of the multi-angle video.

Images