WO2015034122A1

WO2015034122A1 - Method and terminal for providing synchronization of multiple action cam images

Info

Publication number: WO2015034122A1
Application number: PCT/KR2013/008290
Authority: WO
Inventors: 이동채; 유수근
Original assignee: 주식회사 모브릭
Priority date: 2013-09-04
Filing date: 2013-09-13
Publication date: 2015-03-12
Also published as: KR20150027532A

Abstract

The present invention synchronizes multiple action cam images through a process of comprising the steps of: transmitting information on each terminal to a server by a plurality of terminals which shoot images from different angles in different locations; determining, by the server, on the basis of the transmitted terminal information, whether each of the plurality of terminals is a terminal included in a pre-configured shooting group; transmitting, by the server, synchronization information and a control signal to terminals which are determined to be included in the shooting group; encoding, by the terminal, the shot images into digital images; generating, by the terminal, marking information by performing marking on the shot images on the basis of the received synchronization information; and transmitting, by the terminal, the encoded images and the marking information to the server.

Description

Method and provision terminal of multiple action cam video synchronization

The present invention relates to a digital broadcasting service, and more particularly, to an apparatus and method for synchronizing video signals in providing a plurality of action cam images.

Recently, with the spread of high-speed communication networks and convergence of broadcasting and communication technologies, IPTV, which allows a user to selectively receive desired contents through a high-speed communication network, has become a major field of the broadcasting communication industry.

In addition to sports broadcasts, viewers often want to focus on the players they want when broadcasting programs that involve multiple people, such as game broadcasts, but existing television broadcast services provide the broadcast screen of the broadcaster's choice through a single channel. do.

In order to remedy this, recently, several action cam broadcasting services have been deployed, in which a plurality of cameras are arranged in real-time events (e.g., sports events and concert halls), and images recorded by cameras of different angles are transmitted through different channels. It is implemented. By changing the channel, the user can watch the image photographed from the desired angle. According to an embodiment, the action cam of the present invention is a name that refers to a camcorder (cam coder) fixedly mounted on a moving object to capture an image.

However, the existing multi-angle service is difficult to watch images outside the TV screen because the broadcast station can only watch the video provided by editing to the TV screen. In addition, when several people are watching TV together, since the angles they want to see are different from each other, when one viewer selects a specific camera angle, there is a problem that prevents others from watching TV.

Furthermore, even when using a multi-angle service using a terminal other than the TV, since the video provided to the TV and the other terminal are transmitted independently, delays may occur between the two images depending on the IPTV service system and network conditions. If a user watches in the same place, a problem may arise.

In particular, when multiple action cams are broadcasted with multiple terminals (e.g. mobile phones, mobile broadcasting equipment, etc.) with built-in mobile cameras, the event images at the same point in time are the same due to the difference in the performance of the recording terminal and the upload communication speed of the image. It is very difficult to watch on. This requires synchronization of multiple action cam cameras, either artificial or mechanical.

In order to solve the problems of the prior art, by providing a user with an optimal viewing environment by synchronizing the respective images in the case of shooting and broadcasting the same event in each of the plurality of terminals, For that purpose.

In accordance with an embodiment of the present invention, a method of providing a plurality of action cam images comprises: transmitting, by a plurality of terminals photographing images of different angles, respective terminal information to a server, wherein the server receives the received terminal information. Determining whether the terminals are included in a preset shooting group, transmitting, by the server, synchronization information and a control signal to terminals determined to be included in the shooting group; The terminal encoding the photographed image into a digital image, the terminal performing marking on the photographed image based on the received synchronization information to generate marking information, and the terminal Transmitting the encoded image and marking information to the server.

In addition, the terminal for providing a plurality of action cam images according to another embodiment of the present invention, the camera unit for photographing the external image, the encoding unit for encoding the image taken by the camera unit, the terminal information of the terminal server And a communication unit for receiving the synchronization information and the control signal, which are determined by the terminal as a terminal included in the preset shooting group, and performing marking on the server based on the received synchronization information. A synchronization unit for generating and transmitting the encoded image and marking information to the server through the communication unit, wherein the synchronization information is provided at predetermined intervals with other terminals photographing images of different angles, respectively. It is characterized in that the information to enable the synchronization through the marking (Marking) according.

Through multiple action cam video providing methods and multiple action cam video providing terminals according to an embodiment of the present invention, even when a user views multiple action cam videos, each action cam screen is synchronized with each other at the same time. The screen of can be provided to the user.

Therefore, it is possible to stably provide a plurality of action cam screens to the user. Accordingly, the interest and convenience of the user can be improved, and the use of multiple action cam services can be promoted.

1 is a diagram illustrating a system for providing multiple action cam screens according to an exemplary embodiment of the present invention.

2 is a flowchart illustrating steps of a method of providing a plurality of action cam images according to an exemplary embodiment of the present invention.

3 is a block diagram showing the configuration of multiple action cam video providing terminals according to an embodiment of the present invention.

4 is a diagram illustrating a communication process between a plurality of action cam video providing terminals and a server according to an exemplary embodiment of the present invention.

5 is a diagram illustrating a communication process between a plurality of action cam video providing terminals and a server according to another embodiment of the present invention.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings. Furthermore, the suffixes "unit" and "device" for components used in the following description are merely given in consideration of ease of preparation of the present specification, and the "unit" and "device" may be used interchangeably with each other. It can be designed in hardware or software.

Furthermore, although an embodiment of the present invention will be described in detail with reference to the accompanying drawings and the contents described in the accompanying drawings, the present invention is not limited or restricted by the embodiments.

According to an embodiment, the plurality of image capturing apparatuses 100 may be connected to the server 200 to perform an operation for providing a plurality of action cam screens. That is, when a plurality of video recording apparatuses captures images of the same event at different locations, and transmits the captured images to the server, the server provides the subscribers with multiple action cam screens including the plurality of images. In addition, several action cam screen providing services may be performed. The server 200 described in this drawing and the following drawings is a sync server and may be distinguished from a streaming server for broadcasting.

That is, for example, a plurality of video recording apparatuses 100 are attached to each helmet of football players, the server 200 receives each image from the plurality of attached video recording apparatuses 100 By generating the action cam screen, the user can watch the game from each player's point of view.

An image capturing apparatus 100 and a method of providing a plurality of action cam images using the same will be described in detail below.

According to an embodiment, multiple action cam images may be provided through the steps as shown in the drawing. In addition, these steps may be performed using a terminal as shown in FIG. 3.

First, a plurality of terminals respectively photographing images of different angles transmit respective terminal information to a server (S101).

According to an embodiment, a plurality of shooting terminals may be used to acquire a shooting screen of different angles, and in order to generate a plurality of action cam screens through the plurality of shooting terminals, Information can be sent to the server.

In addition, according to an embodiment, a mobile communication network may be used as the transmission means. That is, a 3G communication network or an LTE communication network may be used, and for this purpose, the terminal should be subscribed to the mobile communication network. According to another exemplary embodiment, when the Wi-Fi communication is possible, the information may be transmitted to the server through Wi-Fi communication. In other words, various communication formats capable of transmitting and receiving data between the server and the terminal are not limited to the communication means.

Next, the server determines whether the plurality of terminals are terminals included in a preset photographing group based on the received terminal information (S102).

According to an embodiment, upon receiving terminal information, the server may determine whether a terminal is a terminal included in a preset shooting group. The shooting group may be, for example, a team A group, and in this case, all the shooting terminals attached to the helmets of the players of the team A may be included in the shooting group.

According to an embodiment of the present disclosure, the method may include determining whether the plurality of terminals are terminals included in a preset shooting group, and when information on a terminal is pre-registered with the same group name (eg, group A) in the server. The terminal may be determined to be a terminal included in the preset shooting group. That is, for example, the server may store a group information table, and the table may include information items of target terminals included in the table.

The server may update the pre-stored terminal information when there is a difference between the received terminal information and the pre-stored terminal information. That is, the server can always keep the terminal information up to date.

On the other hand, according to an embodiment, when the information on the terminal is not pre-registered with the same group name in the server, it may be determined that the terminal is not included in the preset shooting group. In this case, for a terminal determined not to be included in the photographing group, a new new group can be created based on the terminal information of the terminal, and the terminal can be selected as a master terminal. That is, a new photographing group may be configured around the terminals not included in the photographing group. The server may generate a group table for the created group, and store information on a synchronization period among the information on the master terminal in the group table. Therefore, a new group is created, and information about the new group can be stored and utilized in the server.

According to an embodiment, when a group information request signal is transmitted from another terminal not included in a photographing group to a server, the server may transmit information about the generated groups to the terminal that transmitted the request signal. . That is, the terminal without a preset photographing group may request all group information or pre-generated group information corresponding to a preset interest of interest to the server, and transmit a subscription request signal to the server with reference to this. In addition, when the subscription request signal is received, transmits a recording group join permission signal of the another terminal to the master terminal. In response, the master terminal may transmit a subscription permission signal or a subscription permission signal to the server after determining whether to subscribe based on a predetermined criterion.

According to an embodiment of the present disclosure, when the subscription request signal is transmitted, the master terminal displays a related menu on a display unit and receives a subscription permission signal or a subscription denied signal by receiving a signal indicating whether to permit subscription from a user through a user interface unit. Can be created and sent.

In addition, according to another embodiment, any one of the plurality of terminals, the Master mode is On (On), the other terminal is the Master Mode (Master Mode) is Off (Off) In this case, when the subscription request signal is transmitted, the subscription permission signal is automatically generated and transmitted.

In addition, when the subscription permission signal of the terminal is received from the master terminal, the server includes a terminal that transmits the subscription request signal to the generated group as a slave terminal to configure and operate a shooting group. Can be. Accordingly, the digital image photographed by the master terminal and the slave terminal may be marked in the master terminal and the slave terminal according to the marking period reference of the generated group. Therefore, synchronization can be performed based on this marking information.

In addition, according to an embodiment, when the synchronization end signal is received from the master terminal to the server, the server may control to stop the synchronization operation by transmitting the synchronization end signal to each slave terminal.

Next, when it is determined that the server is included in the photographing group, the server transmits synchronization information and a control signal to the terminals determined to be included (S103).

According to an embodiment, in transmitting the synchronization information and the control signal, the terminal is determined to be included in the shooting group, the synchronization period information (eg, marking every 5 seconds) and the synchronization information transmission method information (marking) Transmission of a control signal including at least one of synchronization information including at least one of the presence / absence determination and shooting instruction information (eg, recording start / stop / end signal) and shooting information (frames per second, color, size information); have. According to an embodiment, the synchronization information delivery method information may include information on whether to mark directly on the image photographed by the terminal or to store as separate marking information.

In addition, according to an embodiment, the process of transmitting the synchronization information and the control signal is performed in the server, transmitting the synchronization period information to the terminals determined to be included in the shooting group, and transmits the synchronization period information It may be determined whether the photographing preparation of the terminals is completed, and when the photographing preparation is completed, the photographing start signal may be transmitted to the terminals, and a response (ACK) signal may be received from the terminal transmitting the photographing start signal. That is, the server can control the start of shooting.

Next, an external image is photographed through the terminal (S104). In addition, the terminal encodes the photographed image into a digital image and generates marking information by performing marking on the photographed image based on the received synchronization information (S105). The encoded image and marking information are transmitted to the server (S106).

According to an embodiment, the marking may be performed on the encoded image and may be encoded after the marking is performed. However, the information on the order is included in the image so that it can be grasped by the decoder.

In addition, according to an embodiment, in performing the marking, the terminal performs marking on the photographed image at a predetermined period based on synchronization information received from the server. In addition, according to an embodiment, the terminal may transmit the mark information to the server according to a predetermined period based on the information received from the server.

According to an embodiment of the present disclosure, in performing the marking, the terminal stores the marking information in the form of dummy data, that is, information on the number of synchronizations, information on time, and information on the image image itself. It can be inserted into an image and stored in the terminal and transmitted to the server.

According to another embodiment, in performing the marking, the terminal may store the mark information in a storage unit of the terminal according to a predetermined period based on the information received from the server, and in the terminal When encoding is completed, the completed encoded image and the mark information stored up to the completion point may be transmitted to the server.

That is, according to an embodiment, when marking is performed, marking information may be immediately transmitted to a server. According to another exemplary embodiment, after some marking information is accumulated, the subscriber terminal capable of playing the accumulated marking information on a streaming server and a video It can transmit to at least one of.

In addition, according to another embodiment of the present invention, the separately existing streaming server according to the synchronization information generated by the marking, the streaming server waits until the synchronization time for a plurality of broadcast video, and then the marking information Remove the to transmit the synchronized video. This operation may be performed by the video player in the subscriber terminal when the network situation is poor. In addition, in this case, since the image may remain as a still image in the terminal for a long time, a specific image (eg, an advertisement image) may be exposed during the waiting period.

Through the above steps, the server may receive the encoded image and marking information from each of the terminals, it may provide a number of action cam screen services by providing the received information to the user. That is, since the playback time of the captured images can be synchronized using the encoded image and the marking information, the user can smoothly receive several action cam screens through the streaming server or the video player in the user terminal.

According to an embodiment, the plurality of action cam video providing terminals 100 may include a camera unit 110, a synchronization unit 120, a communication unit 130, an encoding unit 140, a transmission unit 150, and a storage unit ( 160). The configurations may be configured in hardware / modules. However, one module / unit may be configured to perform a function of another module / unit. That is, it can be integrated or detailed according to needs or purposes.

The camera unit 110 may capture an external image. According to an embodiment, the aperture speed may be automatically adjusted according to the external brightness or the operation speed of the object.

The synchronizer 120 may generate marking information by performing marking based on the synchronization information received from the server. Embodiments related to marking have been described in detail with reference to FIG. 2.

The synchronization information corresponds to information for performing synchronization with other terminals photographing images of different angles at different positions through marking according to a predetermined period.

That is, according to an embodiment, the synchronization unit 120 may perform marking on the photographed image at a predetermined period based on the synchronization information received by the terminal from the server.

The communication unit 130 may transmit terminal information of the terminal to a server, and receive synchronization information and a control signal transmitted from the server as determined by the terminal as a terminal included in a preset shooting group. According to an embodiment, a mobile communication network may be used as a transmission means using a communication unit. That is, a 3G communication network or an LTE communication network may be used, and for this purpose, the terminal should be subscribed to the mobile communication network. According to another exemplary embodiment, when the Wi-Fi communication is possible, the information may be transmitted to the server through Wi-Fi communication. In other words, various communication formats capable of transmitting and receiving data between the server and the terminal are not limited to the communication means.

The synchronization information may include at least one of synchronization period information and synchronization information delivery method information, and the control signal may include at least one of photographing instruction information and photographing information.

The encoder 140 may encode an image photographed by the camera unit. Various encoding formats according to the MPEG standard may be used, and image formats other than the MPEG standard may also be used according to the terminal setting. In addition, encoding may be performed by selecting a specific encoding format from a user.

The transmission unit 150 may control to transmit the encoded image and the marking information generated by the terminal to at least one of a streaming server and a subscriber terminal capable of playing a video through the communication unit.

The storage unit 160 may store the encoded image and marking information. In some embodiments, the communication unit 130 may transmit the encoded image and marking information stored in the storage unit to the server. The storage unit may be configured with various storage means such as RAM, SSD, HDD, and the like.

According to an embodiment, as a result of determining the terminal by the server, when information on the terminal is pre-registered with the same group name in the server, mutual communication as shown in this figure may be performed.

First, the terminal 100 transmits an inclusion request command and group name information to be included in a shooting group to the server 200 (JOIN Group_Name and Member_Num).

Next, the server 200 examines the information of the terminal 100 included in the pre-registered shooting group, and if there is a difference, updates and stores the information of the terminal stored in the server. , To transmit the synchronization period (Sync_Period) information to the terminal 100.

Next, when it is determined that each of the terminals is ready for shooting, the server 200 transmits a recording start (Sync_Start) command to the terminal when an artificial judgment or completion of member configuration is detected. In addition, an Ack command is received from the terminal in response to the recording start signal.

Accordingly, the terminal may start shooting and perform marking at a period based on the synchronization period information (eg, 5 seconds). In this case, if the value of the synchronization period is '0', the recording starts simultaneously without the marking information at the start of shooting or generates the marking information only once at the start of the shooting.

Next, when the server 200 determines that it is time to end the shooting synchronization based on artificial or systemic determination, the server 200 transmits a recording stop (Sync_END) command to the terminal 100 to terminate the recording and end the marking. To indicate.

In addition, according to an exemplary embodiment, if an inclusion request command of a new terminal is received even during shooting, synchronization is performed through the above steps.

According to an embodiment, as a result of determining the terminal by the server, when the information on the terminal is not pre-registered with the same group name in the server, mutual communication as shown in this figure may be performed.

First, the terminal 100 may transmit a command (Master Group_Name and Sync_Info) to the server 200 to create a photographing group in which the terminal is a master terminal to the server 200. Synchronization information including information on the master terminal 100 and synchronization period information may be included.

Next, upon receiving the command, the server 200 may newly create a photographing group.

In addition, the group information request signal Get Group_Info for selecting a group for photographing synchronization may be transmitted from the other terminal 101 to the server 200.

Accordingly, the server 200 may correspondingly transmit information including the group name to the terminal 101.

In addition, the terminal 101 may transmit a join request signal JOIN Group_Name to a server in order to be included in a specific photographing group.

In response, the server 200 may transmit a photographing group join permission signal JOIND Member_Info to the other terminal 101 to determine whether to allow the photographing group subscription to the master terminal 100. .

Next, the master terminal 101 can determine whether or not the terminal 101 allows the shooting group. That is, the master terminal may transmit a subscription permission signal or a subscription permission signal to the server 200 after determining whether to subscribe based on a preset criterion. If the terminal 101 is subscribed, the terminal 101 is included in the photographing group as a slave terminal.

In addition, when the master terminal determines that the formation of the photographing group is completed at a specific time point, it may notify the other terminal 101 via the server 200 of the start recording control signal, and the master terminal 100 photographs. When the synchronization is to be ended, the recording synchronization may be terminated by transmitting a recording end (Sync_End) command to the other terminal 101 through the server 200. On the other hand, when a new terminal is connected to the synchronization group during the synchronization shooting, the shooting group can be configured through the above steps.

In addition, although the preferred embodiment of the present invention has been shown and described above, the present invention is not limited to the above-described specific embodiment, the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

Claims

Transmitting, by a plurality of terminals photographing images of different angles at different positions, respective terminal information to a server;

Determining, by the server, whether the plurality of terminals are terminals included in a preset photographing group based on the received terminal information;

Transmitting, at the server, synchronization information and a control signal to terminals determined to be included in the photographing group;

Photographing an external image in the terminal;

The terminal encoding the photographed image into a digital image;

Generating marking information by the terminal performing marking on the photographed image based on the received synchronization information; And

And transmitting, by the terminal, the encoded video and marking information to at least one of a streaming server and a subscriber station capable of playing a video.
The method of claim 1,

The transmitting of the synchronization information and the control signal,

The server determines that the server is included in the shooting group, and transmits a synchronization signal including at least one of synchronization period information and synchronization information transmission method information, and a control signal including at least one of shooting instruction information and shooting information. Multiple action cam video providing method comprising the step of.
The method of claim 2,

The synchronization information delivery method information,

Method of providing a plurality of action cam video, characterized in that it includes information on whether or not to mark the image taken by the terminal.
The method of claim 1,

Generating the marking information,

And displaying, by the terminal, marking the photographed image according to a preset period based on the synchronization information received from the server.
The method of claim 1,

The transmitting of the encoded image and marking information to the server,

Storing the encoded image and marking information in a storage unit of the terminal; And

And transmitting the encoded video and marking information to the server.
The method of claim 1,

Determining whether the plurality of terminals are terminals included in a preset shooting group,

If the information on the terminal is pre-registered with the same group name in the server, determining the terminal as a terminal included in a preset shooting group; And

And updating the pre-stored terminal information when there is a difference between the information on the terminal and the pre-stored terminal information in the server.
The method of claim 1,

The transmitting of the synchronization information and the control signal,

Transmitting synchronization period information to terminals determined to be included in the photographing group;

Determining whether photographing preparation of the terminals transmitting the synchronization period information is completed;

When the preparation for photographing is completed, transmitting a photographing start signal to the terminals; And

And receiving an acknowledgment (ACK) signal from the terminal which has transmitted the photographing start signal.
The method of claim 1,

Generating marking information by performing the marking may include:

And transmitting, by the terminal, mark information to the server according to a predetermined period based on the information received from the server.
The method of claim 1,

Generating marking information by performing the marking may include:

Storing, by the terminal, mark information in a storage unit of the terminal according to a predetermined period based on the information received from the server; And

And transmitting the completed encoded image and the mark information stored until the completion point to the server when encoding is completed in the terminal.
The method of claim 1,

Determining whether the plurality of terminals are terminals included in a preset shooting group,

Determining that the terminal is not included in a preset photographing group when information on the terminal is not registered in the server with the same group name;

Creating a group for a terminal not included in the photographing group based on the information on the terminal and selecting the terminal as a master terminal;

Generating a group table for the created group; And

And storing information on a synchronization period among the information on the master terminal in the group table.
The method of claim 10,

When the group information request signal is transmitted from the terminal not included in the photographing group to the server, the server transmitting the information about the generated groups to the terminal that transmitted the request signal;

Receiving a subscription request signal for the generated group from a terminal which has transmitted the information on the group;

Receiving a subscription permission signal of the terminal from the master terminal;

If the subscription permission signal is received, including a terminal transmitting the subscription request signal to the generated group as a slave terminal; And

And synchronizing the digital image photographed by the master terminal and the slave terminal based on the marking information generated according to a specific period corresponding to the generated group. .
The method of claim 11,

And receiving a synchronization end signal from the master terminal to the server, wherein the server further comprises transmitting the synchronization end signal to each slave terminal.
In the terminal for providing multiple action cam video,

A camera unit for photographing an external image;

An encoding unit for encoding an image photographed by the camera unit;

A communication unit which transmits terminal information of the terminal to a server, and receives synchronization information and a control signal which are determined and transmitted by the server to a terminal included in a preset shooting group;

A synchronization unit generating marking information by performing marking based on the received synchronization information; And

And a transmission unit which controls the encoded image and marking information to be transmitted to at least one of a streaming server and a subscriber station capable of playing a video through the communication unit.

The synchronization information is a plurality of action cams, characterized in that the synchronization can be performed through the marking (Marking) according to a certain period with the other terminals for taking pictures of different angles at different locations, respectively. Video provision terminal.
The method of claim 13,

The synchronization information includes at least one of synchronization period information and synchronization information delivery method information.

The control signal, multiple action cam video providing terminal, characterized in that it comprises at least one of the shooting instruction information and the shooting information.
The method of claim 13,

The synchronization unit,

And a plurality of action cam image providing terminals, wherein the terminal performs marking on the captured image based on a preset period based on the synchronization information received from the server.
The method of claim 13,

Further comprising a storage for storing the encoded image and marking information,

The communication unit,

Multiple action cam video providing terminal, characterized in that for transmitting the encoded video and marking information stored in the storage to the server.