KR20220023606A - Multi-View Live Casting System using smartphone - Google Patents

Abstract

A multi-view live casting system using a smartphone according to an embodiment is provided. According to an embodiment of the present invention, a multi-view live casting system using a smartphone includes: a shooting app; a live casting server; a viewing app or program. It is possible to prevent the infringement of the portrait rights of broadcasters.

Description

Multi-View Live Casting System using smartphone}

Embodiments relate to a multi-view relay system using a smartphone.

Recently, the public prefers to play videos using mobile devices. In line with this preference, companies are providing broadcasting platforms such as V-app, AfreecaTV, and Youtube Live. The public viewing this platform is watching the edited video from one point of view, even if it is a screen shot through multiple cameras. However, recent viewers want to be able to directly watch the scene as it is, rather than a video composed of screens selected through the editing process.

Currently, an image service that provides a multi-view image to a user by geometrically correcting and synthesizing a plurality of images obtained by photographing a subject in various channels with a plurality of cameras has been disclosed. This multi-view video provides realistic images that go beyond the concept of high-definition, and through this, users feel more immersed in the media and can greatly increase the effect of video information delivery in fields such as advertising, education, medical care, national defense, and entertainment. there is.

Embodiments aim to provide a multi-view relay system using a smartphone.

The embodiment provides a multi-view relay system using a smartphone.

A multi-view relay system using a smartphone according to an embodiment is provided.

1 and 2 are exemplary views of a multi-view relay system using a smartphone according to an embodiment.
3 to 6 are exemplary diagrams of a multi-view implementation relay system according to the prior art.

The terms used in the present embodiments are selected as currently widely used general terms as possible while considering the functions in the present embodiments, but may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, etc. . In addition, in a specific case, there are also arbitrarily selected terms, and in this case, the meaning will be described in detail in the description of the embodiment. Therefore, the terms used in the present embodiments should be defined based on the meaning of the term and the overall contents of the present embodiments, rather than the simple name of the term.

In the description of the embodiments, when it is said that a certain part is connected to another part, this includes not only a case in which it is directly connected, but also a case in which it is electrically connected with another component interposed therebetween. In addition, when it is said that a part includes a certain component, this means that other components may be further included, rather than excluding other components, unless otherwise stated. In addition, the term “…unit” described in the embodiments refers to a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

Terms such as “consisting” or “comprising” used in the present embodiments should not be construed as necessarily including all of the various components or various steps described in the specification, and some components or some of them It should be construed that steps may not be included, or may further include additional components or steps.

The description of the following embodiments should not be construed as limiting the scope of rights, and what can be easily inferred by those skilled in the art should be construed as belonging to the scope of the embodiments. Hereinafter, embodiments for purposes of illustration will be described in detail with reference to the accompanying drawings.

System overview

Multi-video channel real-time relay (streaming) system using smartphone app

Configuration 1 Shooting app

Real-time transmission of video recorded by inserting the time of each mobile phone as time code from multiple smartphones connected to one main smartphone to the server designated by the main smartphone.

Configuration 2 relay server

The server bundles each video stream transmitted from multiple smartphones into one channel and transmits it to the viewer's device.

Bundling channels means transmitting multiple streams in a format that indicates that they belong to one channel, not one file.

Component 3 Watch Apps or Programs

Viewers can watch multiple video streams bundled into one channel and simultaneously play on one screen and select the screen they want to watch.

The video stream arriving at the viewer's device is delayed depending on the transmission path and state.

The sync of each screen is adjusted based on the time code inserted at the time of shooting.

Viewing is possible on a variety of devices, including mobile, PC, and dedicated hardware.

The whole system is implemented by organic interworking of the above three systems.

As shown in Figure 1, several smartphones connected to one main smartphone are placed in several places on the baseball field and taken.

The videos shot by each phone are bundled with one channel name and sent to the server.

The transmission path varies depending on the connection status of each phone.

The main phone can use the mobile communication network, and the sub phone can use the WIFI communication network.

Referring to Figure 2,

1) Channel Integration

Integrate so that images transmitted from multiple smartphones can be recognized as one channel

2) Relay and Transmission

To the user who wants to watch one channel, all the videos included in the channel are transmitted to the viewer through the Internet. Each video stream is transmitted separately.

3) City Hall

Simultaneous playback of multiple transmitted video channels on one screen.

At this time, multiple screens are synchronized using the time code inserted during shooting.

In the playback device, the screen is divided and transformed according to the user's selection and played at the same time.

Viewers watch the screen in the form they want regardless of the server.

The scope of use of the service can be applied in various ways as follows.

amateur sports broadcast

In the existing broadcasting system, amateur sports are rarely broadcast.

If an individual or the event organizer directly broadcasts it, it is the existing private broadcasting format.

Because the current personal real-time broadcasting cannot properly watch the game on a single screen

Streaming sites, such as YouTube, broadcast most of the matches recorded and edited.

When using the mobile multi-view relay service,

As in the case of the baseball field above, the entire game can be relayed easily, and viewers can enjoy the game in the form of selecting each scene according to the situation.

It lacks editing and special effects compared to professional broadcasting, but it has the advantage of being able to relay amateur matches similar to watching them in the field.

Live streaming of performances

In the existing system, performances are broadcast by a professional broadcasting team.

Camera settings to capture various scenes, linkage and adjustment of individual cameras,

It is relayed on a single screen through the process of special effects and mixing.

Broadcasting amateur performances through YouTube, etc. Mostly, one or two cameras are linked to the relay PC SW to be mixed and adjusted before relaying.

Even for this, in most cases, professional staff to support and manage the relay are required.

When using the mobile multi-view relay service,

Performers can relay the entire performance from various views and angles by placing several smartphones in appropriate positions.

There is no professional camera work, but viewers can enjoy the performance while choosing each view as if they were in a concert hall.

Taking the performance of a small band as an example, if you film the entire band, each performer, and each instrument with a smartphone and transmit it, viewers can enjoy the performance while selecting the performers they are interested in.

The screen shown in FIG. 3 is a service screen provided by KT, SKT, etc. using specialized broadcasting equipment.

In the existing service, multi-view broadcasting is possible only with professional broadcasting staff and equipment.

In the current system, it is impossible for an individual to relay on a screen like the one above.

* online lecture

Existing online lectures require editing.

The instructor's appearance, reference screens, video footage, and writings are appropriately combined to form a single screen that is immersive and not boring.

Real-time lecture broadcasting requires staff such as broadcast producers and cameramen.

With the help of the staff, the broadcast proceeds by switching reference screens and transmitting data screens.

It takes almost a lot of effort to conduct a personal lecture in real time as above.

When using the mobile multi-view relay service,

Lectures can be conducted in the same format as usual while recording the lecturer, blackboard, reference screen, and material video with each smartphone.

By adding means of communication such as a separate two-way text message or voice chat, real-time lectures that are almost similar to lectures in offline classrooms are possible.

Participants select and watch the necessary parts such as the lecturer, the blackboard, and reference materials.

Listening to lectures in a form very similar to participating in actual classroom classes.

If there is a two-way communication method, online lectures similar to real classrooms can be taken.

Ceremonies and other events

After the corona virus, ceremonial occasions and various events are being converted to non-face-to-face.

It is regrettable to broadcast ceremonial occasions or events in real time using existing technology.

If you use the mobile multi-view relay service, you can easily conduct ceremonial occasions and various events online.

ex) wedding

If the officiating ceremony, the bride and groom, the parents of both families, guests, and the view of the ceremonial hall are simultaneously transmitted, the online audience can watch the ceremony as if they were at the wedding hall.

It is possible to deliver congratulatory money in real time by linking payment in the form of Super Chat.

ex) Kindergarten presentation

The event is carried out while simultaneously transmitting the front, rear, left, right, side view, and foreground of the stage.

The audience can enjoy the foreground when my child is not present and the stage screen when my child is present. Similar to viewing from an audience.

Technology subject to patent application (claimed item)

In order to realize the above screen with existing technology, several broadcasting cameras, dedicated relay equipment, and connection equipment are required.

This technology can implement the above screen without specialized equipment.

The components of the technology and the claims

1) A technology that transmits images captured by multiple mobile devices (cell phones) through one channel.

In the conventional video transmission, the video captured in the field is integrated in the field relay device and then transmitted through one channel. (Synchronous transmission when sending multiple screens together)

In this technology, a unique code indicating that it is the same channel and a time code indicating the recording time are inserted into the video captured by each smartphone, and transmitted along each communication path. (Each shot image is transmitted asynchronously)

2) Multi-view transmission from server

The file transmitted through different communication paths is transmitted to the receiving device in an asynchronous transmission method.

At this time, a unique code indicating that each video and audio is one channel is inserted.

3) Multi-view playback on the receiving device

The receiving device receives each file and plays it on one screen.

At this time, according to the user's selection, the display contents and order of the screens on the playback device may be changed.

In the existing technology, the sender changes it and transmits it.

4) Simultaneous playback on the receiving device

1) In the video shooting step, the time of the smartphone is inserted into the video as Time Code.

The receiving device plays each video according to the time based on the time code.

In this way, if the video arrival time is different, the screen may become blank, but the time sync during video playback is accurately maintained.

Since the existing technology is based on synchronous playback, if the arrival time of the video is different, the time sync of each screen may be out of sync.

1. shooting

Shooting while the dedicated shooting equipment is connected to a single relay system through a dedicated communication line

2. Editing/mixing

Consolidated into one screen via editing/mixing equipment

3. Relay and Transmission

One edited screen is transmitted to the viewer through various paths through the transmission device.

Transmission via Internet, wireless, wired, etc.

4. City Hall

If you send a single screen, it is a format to select from among them.

The process of making a relay screen that makes one screen must be included

cf. Real-time broadcasts such as YouTube are also transmitted after creating a single screen using a PC as an editing device.

Capture of personal broadcasting using existing SKT 5G technology

It is a method of transmitting the video taken with the phone like a photo through the Internet after editing (video division, etc.) on the PC.

1. shooting

Shooting via smartphone

When shooting, the time of the smartphone (absolute time synchronized with the base station) is inserted into the time code and transmitted to the server as a set unique value

2. Channel Integration

The video transmitted from multiple smartphones is integrated into a unique code that can be identified as one channel.

3. Relay and Transmission

Asynchronous transmission of all images included in a channel to viewers over the Internet to users who want to watch one channel

4. City Hall

Simultaneous playback of multiple transmitted video channels on one screen

At this time, multiple screens are synchronized using the time code inserted during shooting.

Split and transform the screen according to the user's selection on the playback device

Viewers can see the screen they want regardless of the server

Provided are a photographing device equipped with a real-time image modulation device for protecting portrait rights, and an image delivery and playback device.

In a concert hall or sports stadium, it is common to be photographed by a camera without knowing it. Usually this is acceptable, but there are some who are reluctant to expose their face.

Attach a specific mark around a specific part of the face or body that can be identified as the person using a method such as a sticker, scarf, hat, or one-time tattoo.

Marks are made in a size and shape that can be recognized at a certain resolution or higher, and include methods of refusal such as prohibition of video recording or use of alternative images.

When the camera or video relay and playback device recognizes the above mark, it modulates an alternative image (representing a human face as a cat, etc.) according to the given information, or modulates the screen by a specified method such as mosaic, blur, or cross section.

Image modulation can be applied in real time in the shooting stage, and when applied in the transmission and playback stages, it is possible to prevent infringement of the portrait rights of broadcasters (relayers).

If this process is applied to the filming equipment, it is possible to block the infringement of the portrait right from the filming stage.

Even in the case of data captured without image tampering, if a mark is recognized during transmission and playback, the image is automatically modulated to prevent infringement of the right to portrait. This technology can be used to protect not only portrait rights, but also works of art, documents, trademarks, etc.

An embodiment may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism, and includes any information delivery media.

The above description is for illustration, and those of ordinary skill in the art to which the present invention pertains will understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

Claims (1)

Multi-view relay system using a smartphone.

Images