KR102465403B1 - Method and device for providing video contents that combine 2d video and 360 video - Google Patents

Abstract

According to one embodiment of the present invention, a method for providing a video content combined with a 2D video and 360-degree video, performed by a server, comprises the following steps of: (a) receiving a 2D video and a panorama video combining 2D images taken from a plurality of angles from a video production terminal; and (b) providing a video in which the 2D video and the panoramic video are combined in order of playback time to a user terminal. In the step (b), when the server provides the panoramic video to the user terminal, 360-degree video format information is transmitted to the user terminal through a communication channel distinct from the video, and the user terminal plays the 2D video or a 360-degree video according to the order of playback time, while the panoramic video is played by being converted into the 360-degree video with reference to the 360-degree video format information. According to the present invention, realism and vitality of 360-degree video and concentration and visibility of 2D video are provided in one content, thereby contributing to creation of high-quality content.

Description

Method and apparatus for providing video content combining 2D video and 360-degree video

The present invention relates to a method and apparatus for providing video content in which a 2D image and a 360-degree image are combined.

As the demand for media content providing a three-dimensional three-dimensional image increases, the three-dimensional image transmission and playback technology is being actively developed.

Recently, it is not limited to a 3D TV or HMD (Head Mounted Display) device, but has expanded to a 360-degree video market supported on a general smart phone.

A 360-degree video is an image that is produced or streamed so that it can be viewed from a viewpoint in a direction desired by a user, away from a fixed viewpoint of a two-dimensional image.

The 360-degree video is generated from cameras for at least one field of view based on a virtual graphic model.

Specifically, images or videos for various viewpoints are captured through at least one camera to collect raw data composed of a two-dimensional plane.

Thereafter, a stitching operation is performed on the raw data to generate projection data in which a plurality of viewpoints are connected, which is transmitted to a terminal requesting an image service through a predetermined communication protocol and is converted into a 360-degree video.

However, despite the fact that high-quality, large-capacity media files can be transmitted due to the development of standardized networks, 360-degree images can only be transmitted by themselves and cannot be combined with 2D images.

That is, in the prior art, as information on a panoramic image for generating a 360-degree image is transmitted in the form of a packet through one communication channel, it is difficult to implement moving picture content in which 2D and 3D images are alternately reproduced.

As such, there is a limit to meeting the recent trend in the media service field, which includes 360-degree images and wants more diverse content.

The present invention is to solve the problems of the prior art described above, and discloses an embodiment capable of transmitting and playing a video in which a 360-degree image and a 2D image are combined, and through this, various types of media content production and provision aims to promote

However, the technical problems to be achieved by the present embodiment are not limited to the technical problems described above, and other technical problems may exist.

According to an embodiment of the present invention, a method for providing video content in which a 2D image and a 360-degree image are combined, performed by a server, is (a) combining a 2D image from a video production terminal and 2D images taken at a plurality of angles. receiving a panoramic image; and (b) providing, to a user terminal, a video in which the 2D image and the panoramic image are combined according to the playback time order; and, in step (b), the server transmits the panoramic image to the user terminal. When providing, 360-degree image format information is transmitted to the user terminal through a communication channel that is distinct from the video, and the user terminal reproduces a 2D image or a 360-degree image according to the playback time order, but the 360-degree image format Referring to the information, the panoramic image is converted into a 360-degree image and reproduced.

According to an embodiment of the present invention, the 360 degree image format information is information about a virtual cube or sphere on which the panoramic image is projected.

According to an embodiment of the present invention, the time at which the 2D image or the 360-degree image is reproduced is preset or determined by a signal transmitted by the image production terminal in real time.

According to an embodiment of the present invention, in step (b), when the video production terminal transmits the 2D video to the server in real time, the server streams the 2D video to the user terminal, and the video production terminal When the panoramic image is transmitted to the server, the server streams the panoramic image and the 360-degree image format information to the user terminal following the previously provided 2D image.

According to an embodiment of the present invention, when a 360-degree image is provided after a 2D image is provided, the user terminal displays an identification mark indicating that the 360-degree image is being provided on an area of the screen of the user terminal.

According to an embodiment of the present invention, a method for providing video content in which a 2D image and a 360-degree image are combined is the 360-degree image from the 2D image when a preset time arrives or a signal transmitted by the image production terminal in real time A combination of 2D image and 360-degree image, wherein the 2D image preset by the image production terminal is provided at the center of the screen of the user terminal among a plurality of 2D images constituting the panoramic image when converted to and provided How to provide video content.

According to an embodiment of the present invention, in the method for providing video content in which a 2D image and a 360-degree image are combined, after step (b), the user terminal determines whether the currently playing video mode is a 2D image or a 360-degree image After checking whether the video mode transmitted by the server is a 2D video or a 360 degree video, if the video mode currently being played and the video mode transmitted by the server are different, the video is played according to the video mode transmitted by the server will do

According to an embodiment of the present invention, the server receives the identification information of the video playback player installed in the user terminal, and when the video playback player does not support 360 video playback, the panoramic image instead of the panoramic image A 2D image of any one angle among the 2D images taken from a plurality of angles constituting the image is transmitted to the user terminal.

According to an embodiment of the present invention, the video provided to the user terminal is a video in which a plurality of 2D images and a plurality of 360-degree images are alternately reproduced.

According to an embodiment of the present invention, a video content providing server in which a 2D image and a 360-degree image are combined includes a memory storing a program for performing a method for providing video content in which a 2D image and a 360-degree image are combined; and a processor executing the program; the method comprising: (a) receiving a panoramic image combining a 2D image and 2D images photographed at a plurality of angles from an image production terminal; and (b) providing, to a user terminal, a video in which the 2D image and the panoramic image are combined according to the playback time order; and, in step (b), the server transmits the panoramic image to the user terminal. When providing, 360-degree image format information is transmitted to the user terminal through a communication channel that is distinct from the video, and the user terminal reproduces a 2D image or a 360-degree image according to the playback time order, but the 360-degree image format Referring to the information, the panoramic image is converted into a 360-degree image and reproduced.

An embodiment of the present invention discloses a computer-readable storage medium in which a program is recorded for performing the method for providing video content in which a 2D image and a 360-degree image are combined according to an embodiment of the present invention.

According to an embodiment of the present invention, a method for providing video content in which a 2D image and a 360-degree image are combined, performed by a user terminal, according to an embodiment of the present invention, (a) a panorama combining a 2D image from a server and 2D images taken from a plurality of angles (panorama) receiving a video in which the video is combined according to the playback time order; and (b) playing a 2D image or a 360-degree image according to the playback time sequence, converting the panoramic image into a 360-degree image and playing it with reference to the 360-degree image format information received from the server; In step (a), when the server provides the panoramic image to the user terminal, the user terminal receives the 360-degree image format information through a communication channel differentiated from the video.

According to an embodiment of the present invention, a video content providing terminal in which a 2D image and a 360-degree image are combined includes a memory storing a program for performing a method for providing video content in which a 2D image and a 360-degree image are combined; and a processor executing the program; the method includes: (a) receiving a video in which a 2D image and a panoramic image combining 2D images taken from a plurality of angles are combined according to the playback time order from the server to do; and (b) playing a 2D image or a 360-degree image according to the playback time sequence, converting the panoramic image into a 360-degree image and playing it with reference to the 360-degree image format information received from the server; In step (a), when the server provides the panoramic image to the user terminal, the user terminal receives the 360-degree image format information through a communication channel differentiated from the video.

An embodiment of the present invention may provide a user with a video produced by combining a 2D image and a 360-degree image.

An embodiment of the present invention may provide a user with real-time streaming in which a 2D image and a 360-degree image are combined.

As described above, according to an embodiment of the present invention, it is possible to expand the existing media content limited to 360-degree images or 2D images to more various services.

For example, a content creator or broadcaster may incorporate an additional service suitable for a 360-degree image or a 2D image while providing a main service through a basic image mode of a moving image.

That is, it contributes to the creation of high quality of content by appropriately combining the effect of providing the realism and vitality of a 360-degree image and the effect of providing the concentration and visibility of the 2D image into one content.

1 is a structural diagram of a video content providing system in which a 2D image and a 360-degree image are combined according to an embodiment of the present invention.
2 is a block diagram showing the structure of a video content providing server in which a 2D image and a 360-degree image are combined according to an embodiment of the present invention.
3 is a flowchart of a method for providing video content in which a 2D image and a 360-degree image are combined according to an embodiment of the present invention.
4 is an exemplary view of a panoramic image according to an embodiment of the present invention.
5 is an operation flowchart for a method of providing a streaming service in which a 2D image and a 360-degree image are combined according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . Also, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

The "terminal" referred to below may be implemented as a computer or portable terminal that can access a server or other terminal through a network. Here, the computer is, for example, a laptop, desktop, laptop, VR HMD (eg, HTC VIVE, Oculus Rift, GearVR, DayDream, PSVR, etc.) equipped with a web browser (WEB Browser), etc. may include Here, VR HMD is for PC (e.g., HTC VIVE, Oculus Rift, FOVE, Deepon, etc.), mobile (e.g., GearVR, DayDream, Storm Horse, Google Cardboard, etc.) Independently implemented Stand Alone models (eg Deepon, PICO, etc.) are included. A portable terminal is, for example, a wireless communication device that guarantees portability and mobility, and includes not only a smart phone, a tablet PC, a wearable device, but also Bluetooth (BLE, Bluetooth Low Energy), NFC, RFID, and ultrasound (Ultrasonic). , infrared, Wi-Fi, Li-Fi, etc. may include various devices equipped with a communication module. In addition, "network" refers to a connection structure capable of exchanging information between each node, such as terminals and servers, and includes a local area network (LAN), a wide area network (WAN), and the Internet. (WWW: World Wide Web), wired and wireless data networks, telephone networks, wired and wireless television networks, and the like. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasound communication, Visible Light Communication (VLC), LiFi, and the like, but are not limited thereto.

Hereinafter, an embodiment of the present invention will be described in detail using the accompanying drawings.

1 is a structural diagram of a video content providing system in which a 2D image and a 360-degree image are combined according to an embodiment of the present invention.

A system according to an embodiment may include a user terminal 10 , an image production terminal 20 , and a server 100 .

The user terminal 10 refers to a terminal in which users who request and consume an online video content service through a video content providing site, platform, application, etc. supplied by the server 100 are the main subjects.

According to the disclosure of the present invention, the user terminal 10 may define a term as a video content providing terminal in which a 2D image and a 360-degree image are combined.

The video production terminal 20 means a terminal in which a video producer, a video transmitter, a broadcaster, etc. provide an online video content service through a video content providing site, platform, application, etc. supplied by the server 100 as the main body do.

Here, the video content may include a VOD that is produced and distributed in advance. Specifically, a video may be included in which time codes are synchronized in units of frames and reproduced in the order of playback time. That is, it is a moving picture such as TS, mp4, avi, mkv, in which a frame-by-frame static image is matched with a time code and continuously displayed in a preset order, and the file format is not limited thereto.

Also, the moving picture content may include a moving picture that is streamed in real time. That is, it may include a moving picture that is displayed on the screen of the user terminal 10 in the order of the frames transmitted by the image production terminal 20 and constitutes the playback time.

According to an embodiment, the video content may include media content in which a 2D image and a 360-degree image are reproduced at a preset playback time.

Here, the “360-degree image” may include 360-degree video data, which is visual data, and 360-degree audio data, which is creative data. 360-degree video data may refer to image or video content captured or reproduced in some or all directions in a 360-degree direction. 360-degree video data may be displayed in various types of stereoscopic space according to a preset virtual 3D model. The 360-degree audio data may mean spatial stereo-type audio content recognized as a sound generating source located in a three-dimensional preset position according to a preset 3D model. Such 360-degree video data and 360-degree audio data may be generated from a specific terminal including a camera and transmitted to terminals of users who have requested them through a server, through which media such as VR (virtual reality) and MR (Mixed Reality) A content providing service may be implemented.

According to an embodiment, the user terminal 10 includes a memory (not shown), a processor (not shown), and a communication module (not shown), wherein the communication module is controlled by the server 100 and data carry out communication

According to an embodiment, a program for performing a method for providing video content in which a 2D image and a 360-degree image are combined is stored in the memory.

The processor executes a program stored in the memory to perform a series of operations for providing the user with moving picture content in which a 2D image and a 360-degree image are combined.

As an example of the overall operation, the processor receives a video in which a 2D image and a panoramic image are combined in the order of playback time from the server 100 . After that, the processor plays the 2D image according to the playback time sequence, or converts the panoramic image into a 360-degree image and plays it.

According to an embodiment, the image production terminal 20 includes a memory (not shown), a processor (not shown) and a communication module (not shown), wherein the communication module is controlled by the server 100 and Perform data communication.

According to an embodiment, a program for performing a method for producing and transmitting moving image content in which a 2D image and a 360-degree image are combined is stored in the memory.

The processor executes a program stored in the memory to perform a series of operations for producing or transmitting moving image content in which a 2D image and a 360-degree image are combined.

As an example of the overall operation, the image production terminal 20 may include a plurality of cameras for photographing a preset direction, and the processor controls each camera to display a 2D image photographed from one viewpoint or a plurality of angles. You can create a panoramic image taken from the In addition, the processor may set a playback time for each of the 2D image and the panoramic image and combine them to generate one or more moving images. Alternatively, based on the content plan, the 2D image and the panoramic image may be transmitted to the server 100 at a preset time. That is, the processor transmits the 2D image and the panoramic image to the server 100 so that the VOD and real-time streaming that are alternately provided with the 2D image and the 360-degree image are provided to the user terminal 10 .

The server 100 may refer to a server operated by an online video sharing and broadcasting service provider that provides video content transmitted by the video production terminal 20 to the user terminal 10 .

Accordingly, the server 100 may define the term as a video content providing server in which a 2D image and a 360-degree image are combined.

2 is a block diagram showing the structure of a video content providing server in which a 2D image and a 360-degree image are combined according to an embodiment of the present invention.

According to an embodiment, the server 100 may include a communication module 110 , a memory 120 , a processor 130 , and a database 140 .

The communication module 110 processes data communication with the user terminal 10 and the image production terminal 20, respectively. For example, it is possible to implement a data stream to communicate with the user terminal 10 and the image production terminal 20 and socket (socket) in real time.

According to an embodiment, the communication channel to which the communication module 110 is connected may include a first channel 210 and a second channel 220 .

Specifically, the communication module 110 may receive a 2D image and a panoramic image from the image production terminal 20 through the first channel 210 , and transmit a combined video to the user terminal 10 . In addition, the communication module 110, when receiving or transmitting the panoramic image through the first channel 210, receives the 360-degree image format information from the image production terminal 20 through the second channel 220, or the user terminal (10) can be transmitted.

Here, the 360-degree image format information is attribute information for generating a 360-degree image, and may include information about a type of virtual stereoscopic space to project a panoramic image. The type may include, for example, information about a cube or a sphere.

In this way, information about a moving picture including a 2D image and a panoramic image and information about a 360-degree image format are transmitted through different communication channels. Therefore, unlike the conventional method in which image data and its metadata are transmitted and received together through one channel, the coexistence of 2D images and 360-degree images within one video and the change of the playback mode are possible, so that Video content can be produced and consumed by users.

In the memory 120 , a program for performing a method for providing video content in which a 2D image and a 360-degree image are combined is stored. For example, the program may be linked with a video content providing site, platform, application, and the like.

The processor 130 processes various processes within the program for executing a program stored in the memory 120 to perform a method for providing video content in which a 2D image and a 360-degree image are combined.

As a brief example of the process, the processor 130 provides a user interface for a video content providing service to the user terminal 10 and the video production terminal 20 . Thereafter, a 2D image and a panoramic image with a set playback time are received from the image production terminal 20 , and a moving picture combining them in the order of playback time is provided to the user terminal 10 . As described above, the information about the image and the 360-degree image format information may be separately transmitted and received through different communication channels.

According to an embodiment, in the database 140 , information on users (including video producers or broadcasters) registered as members in the server 100 may be stored. In addition, the video contents received from the video production terminal 20 may be matched with the video production terminal 20 and stored together with information identifying it. Thereafter, in the database 140 , information generated in various processing processes performed by the server 100 according to an embodiment of the present invention may be stored.

The database 140 may be formed in the form of a cloud so that users can request and use a service anytime, anywhere, but the form does not limit the present invention.

Hereinafter, embodiments of a method for providing video content in which a 2D image and a 360-degree image are combined according to the present invention will be described with reference to FIGS. 3 to 5 .

3 is a flowchart of a method for providing video content in which a 2D image and a 360-degree image are combined according to an embodiment of the present invention.

In step S310 , the image production terminal 20 transmits the 2D image and the panoramic image for which the playback time is set according to the content composition planned in advance to the server 100 . As will be described later, the panoramic image may be changed to a 360-degree image on the user terminal 10 later.

According to an embodiment, the video production terminal 20 sets a time code section in which a 2D image is reproduced and a time code section in which a 360-degree image is reproduced, and based on this, a VOD combining a 2D image and a panoramic image is provided to the server 100 ) can be transmitted.

According to an embodiment, in the case of a real-time streaming service, the image production terminal 20 may determine an image mode to be transmitted among a 2D image and a 360-degree image during streaming, and whenever each image is transmitted, a corresponding transmission signal is transmitted in real time. can be transmitted to the server 100 . That is, the image production terminal 20 may implement a streaming service in which the 2D image or the 360-degree image is converted in real time by transmitting the 2D image or the panoramic image to the server 100 according to the broadcasting plan.

In this way, one VOD or streaming service may include a plurality of 2D images and panoramic images, and the video provided to the user terminal 10 includes a plurality of 2D images and a plurality of videos according to the transmission of the image production terminal 20 . The 360-degree video may be a video in which the video is played alternately.

According to an embodiment, the image production terminal 20 may transmit the 360-degree image format information to the server 100 by separating the 360-degree image format information through another communication channel when transmitting the panoramic image to the server 100 . For example, VOD or real-time panoramic image may be transmitted to the server 100 through the first channel 210 , and 360 degree image format information may be transmitted through the second channel 220 .

According to an embodiment, the 360-degree image format information is attribute information for generating a 360-degree image using the panoramic image, and may include type information on a virtual stereoscopic space in which the panoramic image is projected.

According to an embodiment, the type may include equiangular, cube, equiranctangular, and the like, and accordingly, information on a virtual three-dimensional space may include information on a hexahedron or a sphere. However, it is not limited to this type of configuration, and a projection type in the form of a cylinder or a pyramid may be adopted.

According to an embodiment, the image production terminal 20 may generate a panoramic image by combining 2D images photographed from a plurality of angles. For example, the image production terminal 20 may be equipped with a plurality of cameras for photographing a preset direction. The image production terminal 20 may collect 2D images captured from a plurality of cameras and attribute information for each 2D image. Thereafter, by arranging and connecting the 2D images collected based on the attribute information and the input 360-degree image format information, the panoramic image 400 as shown in the example of FIG. 4 can be generated.

According to an embodiment, the image production terminal 20 may perform a stitching operation on a plurality of 2D images based on attribute information of the 2D images including a viewpoint. For example, the 2D images may be stitched together by extracting edge information of each 2D image and correcting parallax, brightness, object movement, etc. according to each edge information.

According to an embodiment, the image production terminal 20 may perform a projection operation of projecting a stitched image on a plane according to the input 360-degree image format information. For example, when the format information is set to the cube type, the stitched image may be projected in the form of a development view divided into 2D images of preset units to form a cube (hexahedron) by combining them.

According to an embodiment, the producer or the transmitter may set the front (center) image of the 360-degree image through the image production terminal 20 . For example, it is possible to select one of a plurality of 2D images and set it as an initial viewpoint of a 360-degree image, which is included in attribute information of the corresponding 2D image and stored, so that it can be utilized in stitching and projection work.

Referring to FIG. 4 , an example of the panoramic image 400 generated according to the above-described embodiment and the 2D image 401 disposed in front of which an initial viewpoint is set can be confirmed. In this way, the 2D image 401 in which the initial viewpoint is set may be provided at the center of the initial screen when the 360-degree image is later reproduced in the user terminal 10 .

According to a further embodiment, the image production terminal 20 may determine the viewpoints of other 2D images based on the 2D image set as the initial viewpoint. For example, in the case of a cube type, each 2D image is projected on a plane in the form of a development view. top), bottom, left, right, and back views can be set automatically. As such, the determined viewpoint information is included in the attribute information of each 2D image and stored, thereby contributing to the improvement of the stitching and projection speed.

According to a further embodiment, in the case of VOD in which video content is produced and supplied in advance, the video production terminal 20 may perform a 2D video mode for at least one or more panoramic images matched for each time code (playback time) for a 360-degree video section. can be set to For example, assuming that the time code of the 360-degree video section is from 10:00 to 15:00, and the panoramic image matched with the time code of 13:00 to 13:10 is important only from the front view, video production The terminal may extract only the 2D image of the front view from the panoramic image from 13:00 to 13:10 and remove the remaining 2D image. While a 360-degree image is being played, a 2D image may be required for a short time or its efficiency may be high. and rapid provision of content.

In step S320, the server 100 provides a video in which the 2D image and the panoramic image received from the image production terminal 20 are combined according to the playback time order, to the user terminal 10 requesting it.

According to one embodiment, when the video content is VOD, the server 100 provides the video produced in advance by the video production terminal 20 according to the playback time to the user terminal 10, or the video production terminal 20 ) can be provided to the user terminal 10 by generating the 2D image and the panoramic image received from the video as a moving picture that forms a playback time according to the time code matched to each image.

According to one embodiment, when the video content is real-time streaming, the server 100 receives a signal related to the playback mode transmitted by the video production terminal 20 in real time, and displays the 2D image or the panoramic image at the corresponding playback time. It can be provided to the terminal 10 . That is, a streaming service in which a 2D image or a 360-degree image is converted according to a signal transmitted from the image production terminal 20 may be provided to the user.

Specifically, when the image production terminal 20 transmits the 2D image to the server 100 in real time, the server 100 streams the 2D image to the user terminal 10 . After that, when the image production terminal 20 transmits the panoramic image to the server 100 in real time, the server 100 transmits the panoramic image and 360-degree image format information to the user terminal ( 10) to stream. Of course, the panoramic image may be transmitted first and then the 2D image may be transmitted, and when a plurality of 2D images and 360-degree images are included in the streaming service, these operations are repeatedly performed.

According to an embodiment, the video playback player pre-installed in the user terminal 10 and set as a default setting may be a program that does not support 360-degree video playback. In this regard, when the server 100 provides a video to the user terminal 10 , in response to this, the user terminal 10 may transmit identification information of a video playback player installed therein to the server 100 . When the server 100 determines that the video playback player does not support the 360-degree image playback based on the identification information, the server 100 is a 2D image of any one of the 2D images taken from a plurality of angles constituting the panoramic image instead of the panoramic image. The image may be transmitted to the user terminal 10 .

According to an embodiment, the 2D image of any one angle may be a 2D image that is set as an initial viewpoint from the image production terminal 20 and is disposed in front of the screen of the user terminal 10 .

According to a further embodiment, the image production terminal 20 may set ROI (Region of Interest) information for a 360-degree image. Here, the ROI information relates to information on a specific point in time determined to be of high interest to users among 360-degree images. For example, the image production terminal 20 may select a 2D image corresponding to an important viewpoint among a plurality of 2D images constituting the panoramic image, and include ROI information in attribute information of the 2D image. Accordingly, if the video playback player of the user terminal 10 does not support the 360-degree image playback, the server 100 may transmit the 2D image including the ROI information in the attribute information to the user terminal 10 instead of the panoramic image. have.

According to a further embodiment, when the video playback player of the user terminal 10 does not support 360-degree video playback, the server 100 guides the download of the video playback player that the server 100 supports 360-degree video playback. message to the user terminal 10 may be transmitted. For example, the message may include a uniform resource locator (URL) connected to a web page including a download function or an app page.

In step S330, the user terminal 10 receives the video from the server 100 and plays a 2D video or a 360-degree video according to the playback time order.

According to an embodiment, when the user terminal 10 receives a panoramic image from the server 100 through the first channel 210 , the panorama image is referred to the 360-degree image format information received through the second channel 220 . You can convert a video into a 360 degree video.

According to an embodiment, the user terminal 10 may perform a rendering operation including a process of generating a virtual three-dimensional space according to the type information included in the 360-degree image format information and overlaying the panoramic image thereon. have. For example, in the case of the cube type, a three-dimensional space of a hexahedron may be created and a panoramic image formed in the form of a development view may be arranged along the outer circumferential surface of the hexahedron.

According to an embodiment, when the 2D image for the initial viewpoint is set by the image production terminal 20, the user terminal 10 arranges the 2D image in front and determines the viewpoint of the remaining 2D image based on this. It can be placed in three-dimensional space.

According to an embodiment, when the user terminal 10 generates and reproduces a 360-degree image, an identification mark indicating that the 360-degree image is being provided may be displayed on one area of the screen of the user terminal 10 . For example, the user terminal 10 may display a preset phrase or figure guiding the conversion from a 2D image to a 360-degree image on one area of the screen.

According to a further embodiment, the user terminal 10 may include a preset mark indicating a viewpoint in one region of the 360-degree image while generating the 360-degree image. That is, while performing a rendering operation, information on a viewpoint for each 2D image may be additionally projected onto a stereoscopic space. Through this, the user can easily identify the point in time of the currently playing 360-degree video.

According to an additional embodiment, when switching from a 2D image to a 360-degree image, the user terminal 10 may generate a 360-degree image in consideration of attribute information of the previously played 2D image. For example, the size of the virtual stereoscopic space may be determined corresponding to the frame size of the 2D image. Alternatively, if the 2D image is a real-time captured image, the roll, pitch and You can determine the range of angles for the yaw.

According to an embodiment, when the video received by the user terminal 10 is a pre-produced VOD, the user terminal 10 may check the playback time (time code) of the video in real time. That is, it is possible to determine whether the image set at each playback time is a 2D image or a 360-degree image, and display a 2D image or a 360-degree image on the screen of the user terminal 10 according to the result.

According to a further embodiment, when the video is VOD, the user terminal 10 may receive the video from the server 100 and simultaneously extract the panoramic image and convert it into a 360-degree image at the same time. In this case, since the conversion process of the panoramic image is not required every time the 2D image is converted to a 360-degree image and played back, it can contribute to the improvement of processing speed.

According to an embodiment, when the video received by the user terminal 10 is a real-time streaming service, the user terminal 10 may check in real time whether the currently played video mode is a 2D video or a 360-degree video. That is, it can be checked in real time whether the currently played video mode matches the video mode received from the server 100 .

According to an embodiment, when the video mode is different as a result of the check, the user terminal 10 may play the video according to the video mode transmitted by the server 100 . For example, the video production terminal 20 can transmit a 2D image and a 360-degree image while switching during streaming, and transmits a transmission signal for this to the server 100 every time it is switched, and the server 100 transmits the corresponding transmission signal may be transmitted to the user terminal 10 . The user terminal 10 may reproduce an image in a video mode corresponding to the received transmission signal.

In the present invention, when panoramic image and 360-degree image format information are transmitted and received through different communication channels, the state of one communication channel changes depending on the distance between the server 100 and the user terminal 10 or the connection state of the communication network , delay or data omission may occur. That is, although the image production terminal 20 has already transmitted a 360-degree image while transmitting the 2D image, the user terminal 10 may still display the 2D image due to factors such as delay or data omission. In this case, while the current video playback mode is 2D, if the transmitted signal relates to a 360-degree video, the video can be played by converting the playback mode to a 360-degree video. Alternatively, when a new user terminal accesses the server 100 at the time when a 2D image is transmitted and converted into a 360-degree image, a 2D image may be provided due to factors such as delay, but the current video playback mode is 2D. On the other hand, if the transmitted signal is about a 360-degree video, the video can be played by converting the playback mode to a 360-degree video. However, according to the above-described embodiment, since the user terminal 10 checks the playback mode of the video in real time and corrects it according to the transmitted signal, when the above phenomenon occurs, it can be automatically solved.

Hereinafter, with reference to FIG. 5 , an embodiment of a case in which a video is a streaming service will be introduced. Duplicate contents are replaced by the above contents.

On the other hand, in the following embodiment, it is described as streaming in which a 360-degree image is transmitted while a 2D image is transmitted, but it is easily understood by those of ordinary skill in the art to which the present invention pertains that the same principle is the case in the opposite case or repeated case. will be able

5 is an operation flowchart for a method of providing a streaming service in which a 2D image and a 360-degree image are combined according to an embodiment of the present invention.

First, when broadcasting starts, the broadcast transmitter performs shooting using one of the cameras installed in the image production terminal 20 and transmits the 2D image generated through this to the server 100 . The server 100 receives the 2D image together with the transmitted signal and streams it to the user terminal 10 through the first channel 210 . Accordingly, the 2D image is reproduced on the screen of the user terminal 10 . Meanwhile, when the user terminal 10 receives the first video from the server 100 , the user terminal 10 may transmit video playback player identification information to the server 100 .

The broadcaster performs shooting for a plurality of angles in the field using a plurality of cameras installed in the video production terminal 20 at a time when it is determined that 360-degree support broadcasting is necessary according to the broadcast plan. The image production terminal 20 combines the generated 2D images to generate a panoramic image, and then transmits it to the server 100 together with 360-degree image format information and a transmission signal.

The server 100 streams the received panoramic image and 360-degree image format information to the user terminal 10 . In this case, the panoramic image may be transmitted to the first channel 210 , and 360-degree image format information may be transmitted to the second channel 220 through different communication channels.

The user terminal 10 checks whether the currently playing video mode matches the video mode received from the server 100 . If the 2D image is reproduced, the 360-degree image playback mode may be changed according to the 360-degree image transmission signal. On the other hand, the location of the playback mode confirmation block in the operation flowchart of FIG. 5 is for explanation, and the user terminal 10 performs a playback mode check operation in real time from the moment it receives the first video from the server 100 until the end of streaming do.

Thereafter, the user terminal 10 converts the panoramic image into a 360-degree image based on the 360-degree image format information. Accordingly, a 360-degree image is reproduced following the 2D image on the screen of the user terminal 10 .

An embodiment of the present invention may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer-readable media may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

Although the methods and systems of the present invention have been described with reference to specific embodiments, some or all of their components or operations may be implemented using a computer system having a general purpose hardware architecture.

The foregoing description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and likewise components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

10: user terminal
20: video production terminal
100: server

Claims (13)

In the method for providing video content in which a 2D image and a 360-degree image are combined, performed by a server,
(a) receiving a panorama (panorama) image combining the 2D image and the 2D images taken from a plurality of angles from the image production terminal; and
(b) providing a video in which the 2D image and the panoramic image are combined according to the playback time order to a user terminal;
Step (b) is,
The server provides the 2D image to the user terminal through one communication channel during the time period for providing the 2D image,
During a time period in which the server provides the panoramic image to the user terminal, the 360-degree image format information and the panoramic image are transmitted to the user terminal, and the 360-degree image format information is transmitted through a communication channel that is distinct from the video. transmitted to the user terminal,
The user terminal reproduces a 2D image or a 360-degree image according to the playback time sequence, but converts the panoramic image into a 360-degree image and plays it with reference to the 360-degree image format information,
The 360-degree image format information is information about a virtual hexahedron or sphere on which the panoramic image is projected. delete The method of claim 1,
The time at which the 2D image or the 360-degree image is reproduced is preset or determined by a signal transmitted in real time by the image production terminal. The method of claim 1,
Step (b) is,
When the image production terminal transmits the 2D image to the server in real time, the server streams the 2D image to the user terminal,
When the image production terminal transmits the panoramic image to the server, the server streams the panoramic image and the 360-degree image format information to the user terminal following the previously provided 2D image, 2D image and a method of providing video content in which 360-degree images are combined. 5. The method of claim 4,
The user terminal is
When a 360-degree image is provided after a 2D image is provided, an identification mark indicating that a 360-degree image is being provided is displayed on one area of the screen of the user terminal, providing video content combining a 2D image and a 360-degree image Way. 5. The method of claim 4,
When a preset time arrives or when the 2D image is converted to the 360-degree image by a signal transmitted in real time by the image production terminal and provided, the image production terminal is preset among a plurality of 2D images constituting the panoramic image One 2D image is provided in the center of the screen of the user terminal, a 2D image and a 360-degree image combined video content providing method. The method of claim 1,
After step (b), the user terminal checks whether the video mode currently being played is a 2D video or a 360 degree video, and confirms whether the video mode transmitted by the server is a 2D video or a 360 degree video, and is currently played When the video mode in the video mode and the video mode transmitted by the server are different, the video is played according to the video mode transmitted by the server, a method for providing video content combining a 2D video and a 360 degree video. The method of claim 1,
The server receives the identification information of the video playback player installed in the user terminal, and when the video playback player does not support 360-degree image playback, the panoramic image is taken from a plurality of angles constituting the panoramic image instead of the panoramic image. A method for providing video content in which a 2D image and a 360-degree image are combined by transmitting a 2D image of any one angle among 2D images to the user terminal. The method of claim 1,
The video provided to the user terminal is a video that alternately reproduces a plurality of 2D images and a plurality of 360-degree images, a method for providing video content combining a 2D image and a 360-degree image. In the video content providing server in which 2D image and 360-degree image are combined,
a memory in which a program for performing a method for providing video content combining a 2D image and a 360-degree image is stored; and
a processor executing the program; and
The method is
(a) receiving a panorama (panorama) image combining the 2D image and the 2D images taken from a plurality of angles from the image production terminal; and
(b) providing a video in which the 2D image and the panoramic image are combined according to the playback time order to a user terminal;
Step (b) is,
The server provides the 2D image to the user terminal through one communication channel during the time period for providing the 2D image,
During a time period in which the server provides the panoramic image to the user terminal, the 360-degree image format information and the panoramic image are transmitted to the user terminal, and the 360-degree image format information is transmitted through a communication channel that is distinct from the video. transmitted to the user terminal,
The user terminal reproduces a 2D image or a 360-degree image according to the playback time sequence, but converts the panoramic image into a 360-degree image and plays it with reference to the 360-degree image format information,
The 360-degree image format information is information about a virtual hexahedron or sphere on which to project the panoramic image, a video content providing server in which a 2D image and a 360-degree image are combined. A computer-readable storage medium in which a program is recorded for performing the method for providing video content in which a 2D image and a 360-degree image are combined according to claim 1 . In the method for providing video content in which a 2D image and a 360-degree image are combined, performed by a user terminal,
(a) receiving, from a server, a video in which a 2D image and a panorama image combining 2D images taken from a plurality of angles are combined in the order of playback time; and
(b) playing a 2D image or a 360-degree image according to the playback time sequence, and converting the panoramic image into a 360-degree image by referring to the 360-degree image format information received from the server and playing;
The step (a) is,
During a time period in which the server provides the 2D image, the 2D image is received through one communication channel,
During a time period in which the server provides the panoramic image to the user terminal, the 360-degree image format information and the panoramic image are received, wherein the user terminal transmits the 360-degree image format information to a communication channel distinguished from the video. is received through
The 360-degree image format information is information about a virtual hexahedron or sphere on which the panoramic image is projected. In the video content providing terminal in which a 2D image and a 360-degree image are combined,
a memory in which a program for performing a method for providing video content in which a 2D image and a 360-degree image are combined; and
a processor executing the program; and
The method is
(a) receiving, from a server, a video in which a 2D image and a panorama image combining 2D images taken from a plurality of angles are combined in the order of playback time; and
(b) playing a 2D image or a 360-degree image according to the playback time sequence, and converting the panoramic image into a 360-degree image by referring to the 360-degree image format information received from the server and playing;
The step (a) is,
During a time period in which the server provides the 2D image, the 2D image is received through one communication channel,
During a time period in which the server provides the panoramic image to the user terminal, the 360-degree image format information and the panoramic image are received, wherein the user terminal transmits the 360-degree image format information to a communication channel distinguished from the video. is received through
The 360-degree image format information is information about a virtual hexahedron or sphere on which to project the panoramic image, a video content providing terminal in which a 2D image and a 360-degree image are combined.

Images