KR20190126518A - Virtual reality viewing method and virtual reality viewing system - Google Patents

Abstract

The present invention relates to a VR viewing method and a VR viewing system. The VR viewing method comprises the steps of: receiving a selection for one of a plurality of viewing positions by a VR providing server; receiving one or more VR images captured at each viewing position by the VR providing server in real time; and transmitting one or more real time VR images corresponding to viewing positions selected among the viewing positions to a VR terminal by the VR providing server.

Description

VIRTUAL REALITY VIEWING METHOD AND VIRTUAL REALITY VIEWING SYSTEM}

The present invention relates to a virtual reality viewing method and a virtual reality viewing system, and more specifically, to enable dynamic viewing of performances, sports, etc., which are performed in real time, through virtual reality technology according to the setting and operation of a remote viewer. It relates to a reality viewing method and a virtual reality viewing system.

Virtual Reality (VR) refers to a system that allows a user to experience a space that does not exist or is not in a specific place by creating a virtual space on a computer.

A representative VR application product is a virtual reality headset device. The VR headset device outputs various images of the virtual reality space to the user. For example, the VR headset device may output a stereoscopic three-dimensional image.

Modern VR headset devices can be combined with smartphones to output stereoscopic three-dimensional images. For example, VR headset devices output a three-dimensional image output from the smartphone to the left and right eyes of the wearer, including a holder to mount the smartphone and a lens on the holder, and accordingly, the wearer images the left and right eyes. Three-dimensional stereoscopic images can be recognized according to the difference.

In addition, a camera for obtaining a three-dimensional stereoscopic image is known. The 3D stereo camera may acquire a stereoscopic image using two camera lenses. The two acquired images can be used to generate a 3D image. For example, two series of images are output to a VR headset device so that the wearer can recognize a three-dimensional stereoscopic image or obtain a natural stereoscopic image (e.g., stereoscopic operation) to output the VR headset device or three-dimensional output. The device is output as a 3D stereoscopic image.

As the application of VR technology spreads, it can be used in various fields. For example, VR technology can be used for 3D games, driving training for cars, ships, airplanes, and the like. Even as communication technology advances, VR technology is available for real-time performance relay.

For example, the VR relay system may receive a sports relay image and sound in real time through a 3D camera or a microphone, and deliver the received real-time 3D image and sound to a VR headset wearer connected through a network. Accordingly, the wearer can remotely recognize 3D video such as sports relay and feel an atmosphere similar to that in the field.

Known VR relay system has a limitation that does not reflect the real-time viewing environment, there is a limit to feel the same or similar atmosphere as seen in the actual venue, sports stadium, etc.

For example, it is necessary to relay VR at various locations depending on the location of a concert hall or a grandstand, but a VR image captured at a specific location is unilaterally transmitted to the viewer.

In addition, it is necessary to acquire various VR images (3D images) even at a specific position. For example, even at a location for watching sports, if the front view, the right side of the front side, and the VR image of the left side of the front side can be obtained, the VR service linked to the wearer's movement can be provided.

In addition, in order to feel the atmosphere in the grandstand, there is a need to provide a VR image linked to the wearer's movement.

As such, there is a need for a virtual reality viewing method and a virtual reality viewing system that can solve various problems caused or recognized by the existing virtual reality viewing technique.

10-2015-0105058 (A), September 16, 2015

The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a virtual reality viewing method and a virtual reality viewing system capable of real-time remote viewing using a VR image for an ongoing real-time event.

In addition, the present invention is a virtual reality viewing method for capturing a plurality of VR images each at a plurality of locations that can be viewed for the real-time event in progress and to provide a corresponding VR image according to the movement of the viewer at a location corresponding to the viewer's selection and The purpose is to provide a virtual reality viewing system.

In addition, the present invention provides a virtual reality viewing method and a virtual reality viewing system that can provide a real-time event viewing environment by dynamically providing a friend video or a relay video according to the movement of the viewer along with the VR video viewing for the ongoing real-time event Its purpose is to.

In addition, the present invention enables the viewer to view through the VR image at the desired location through the link provided and the selection of the viewable location associated with the provided VR image, the VR service provider can create a variety of added value from the VR service provision The purpose of the present invention is to provide a virtual reality viewing method and a virtual reality viewing system.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned above will be clearly understood by those skilled in the art from the following description. Could be.

Virtual reality viewing method according to an aspect of the present invention comprises the steps of (a) the VR providing server receives a selection for one viewing position of the plurality of viewing positions, (b) the VR providing server to capture at each of the plurality of viewing positions Receiving at least one VR image to be ringed in real time, and (c) the VR providing server transmitting one or more real-time VR images corresponding to the selected viewing position among the plurality of viewing positions to the VR terminal.

In the virtual reality viewing method, step (b) receives a plurality of VR images captured at different angles of view of each viewing position, and step (c) is a VR of the plurality of VR images captured in real time. The current VR image and the adjacent VR image determined in association with the terminal are transmitted to the VR terminal.

In the virtual reality viewing method, after step (c), (d) the VR providing server to receive the control information from the VR terminal, (e) the VR providing server to be output to the VR terminal according to the control information And (f) transmitting, by the VR providing server, an image configured corresponding to the control information to the VR terminal.

In the virtual reality viewing method, the control information includes motion data of the VR terminal, step (b) receives a plurality of VR images captured at different angles of view of each viewing position, step (e) When the motion data indicates the movement in the left and right direction, the VR image of the angle of view corresponding to the movement in the left and right direction among the plurality of VR images is configured as an image to be output.

In the virtual reality viewing method, step (e) selects two VR images corresponding to the motion data among the plurality of VR images and configures the VR image to be output by synthesizing the selected two VR images.

In the above-described virtual reality viewing method, step (e) comprises an image to be outputted and an image representing another user set when the movement data indicates a movement in the left and right direction and is a movement equal to or greater than a first angle specified. In the case where the movement is shown in the up and down direction and the movement is greater than or equal to the designated second angle, the relay image generated by the broadcaster is configured as the image to be output.

In the above-described virtual reality viewing method, the control information includes selection data indicating the selection of the link included in the VR image transmitted to the VR terminal, step (e) is an image corresponding to the link in accordance with the reception of the selection data Is configured as an image to be output.

The virtual reality management system according to an aspect of the present invention includes a VR providing server constituting a VR image to be transmitted to the viewer from a plurality of connected cameras, the VR providing server receives a plurality of VR images from the plurality of cameras Communication unit for receiving a selection of one viewing position of the viewing position of, the VR image at each viewing position is classified and stored temporarily according to the angle of view of the corresponding camera and set by the viewer and the selection for the viewing position A storage unit for storing the indicating setting information and a control unit corresponding to the selected viewing position from among the plurality of viewing positions and transmitting the temporarily stored VR image to the VR terminal through the communication unit.

In the virtual reality management system, the control unit receives control information including motion data from the VR terminal through the communication unit, and when the motion data indicates movement in the left and right directions, the left and right of the plurality of VR images at the set viewing position. Two temporarily stored VR images of the angle of view corresponding to the movement of the direction are selected, and the two selected VR images are synthesized and the synthesized VR images are transmitted to the VR terminal through the communication unit.

In the virtual reality management system, the VR terminal further comprises a VR terminal for receiving a real-time VR image from the VR providing server, and generates control information according to the event generated by the user to transmit to the VR providing server, the VR terminal is a motion sensor Alternatively, the control information including motion data sensed by the motion sensor, including an input device connected through a wireless network, is transmitted to the VR providing server, or the control information including input data through the input device is transmitted to the VR providing server, and VR The providing server selects a VR image among the plurality of VR images at the viewing position set according to the data of the control information, and transmits the VR image configured based on the selected VR image to the VR terminal.

As described above, the virtual reality viewing method and the virtual reality viewing system according to the present invention have the effect of enabling remote viewing with a sense of reality using a VR image for a real-time event in progress.

In addition, the virtual reality viewing method and the virtual reality viewing system according to the present invention as described above captures a plurality of VR images at various locations that can be viewed for the real-time event in progress and according to the movement of the viewer at the position corresponding to the viewer's selection There is an effect that can provide a corresponding VR image.

In addition, the virtual reality viewing method and the virtual reality viewing system according to the present invention as described above provides a real-time event viewing environment by dynamically providing a friend image or a relay video according to the movement of the viewer along with the VR image viewing for the real-time event in progress. It can work.

In addition, the virtual reality viewing method and the virtual reality viewing system according to the present invention as described above through the link provided and the selection of the viewing location associated with the provided VR video, the viewer can view through the VR video at the desired location The VR service provider has an effect of generating various added values from the VR service provision.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned above may be clearly understood by those skilled in the art from the following description. will be.

1 is a diagram illustrating an exemplary virtual reality viewing system according to the present invention.
FIG. 2 is a diagram illustrating an example of arrangement of cameras installed in an arbitrary performance hall or sports stadium.
3 is a diagram illustrating an exemplary block diagram of a VR providing server.
4 is a diagram illustrating an exemplary viewing reservation control flow.
5 is a diagram illustrating an exemplary flow of providing a user with a real-time VR image according to a viewing reservation.

The above objects, features, and advantages will become more apparent from the detailed description given hereinafter with reference to the accompanying drawings, and accordingly, those skilled in the art to which the present invention pertains may share the technical idea of the present invention. It will be easy to implement. In addition, in describing the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an exemplary virtual reality viewing system according to the present invention.

According to FIG. 1, the virtual reality viewing system includes a VR providing server 100, one or more VR terminals 200, one or more reservation terminals 300, and a plurality of cameras 400.

The virtual reality viewing system designates and reserves a performance or a sport to be watched through the reservation terminal 300, and captures in real time through the camera 400 provided with the reserved performance or sport, and the VR image of the captured camera 400 ( And sound) is configured to provide to the VR terminal 200.

Referring to FIG. 1, the virtual reality viewing system is briefly described. The VR providing server 100 receives a VR image from a plurality of cameras 400 of each of a performance hall or a sports stadium connected thereto, and constructs a VR image to be transmitted to the viewer. It corresponds to or transmits to the VR terminal 200 worn by the viewer. The transmitted VR image may further include audio captured by a microphone or the like. Hereinafter, in order to simplify the description, descriptions related to sound will be omitted.

The VR providing server 100 receives VR images captured from the plurality of cameras 400 at each viewing position in real time connected to a plurality of performance halls or sports stadiums, and the VR at the viewing position according to the selection of the viewers. Among the images, the corresponding or configured VR image may be transmitted to the VR terminal 200 of the viewer.

The VR providing server 100 may be connected to the VR terminal 200 to receive control information from the VR terminal 200. The VR providing server 100 selects a VR image (one, two, or three) corresponding to the data of the control information from among various VR images of the viewing position set by the viewer (user) according to the received control information and selects the selected one. A VR image to be transmitted is configured based on the VR image, and the configured VR image is transmitted to the VR terminal 200.

The VR providing server 100 will be described in detail with reference to FIG. 3.

The VR terminal 200 is a device that can watch and listen to a VR image and further received sound. The VR terminal 200 may include, for example, a VR headset capable of visually recognizing a VR image.

The VR headset is configured to be worn on the viewer's head and is configured to allow the viewer to visually recognize the image displayed through the display, including a left eye lens, a right eye lens and one or more displays.

The display can be a single display or two separate displays. When the display is configured as a single display, the display may be divided into two regions, and the left eye image of the VR image may be displayed on the left display area, and the right eye image of the VR image may be displayed on the right display area.

The VR terminal 200 may receive a real-time VR image from the VR providing server 100, display it on a display, and the viewer may recognize a stereoscopic VR image (3D image). The VR terminal 200 may be a dedicated device for outputting a VR image or a device combined with another terminal. For example, the VR terminal 200 may include a space for accommodating a smart phone, mount the smart phone in the space, and output the VR image received from the VR providing server 100 through the display.

The VR terminal 200 (smartphone) may further include an input device having a motion sensor and further connected to a wireless network (eg Bluetooth). The motion sensor is a sensor capable of recognizing the movement of the VR terminal 200, for example, a gyro sensor or a geomagnetic sensor or an arbitrary direction (for example, three axes of X, Y, Z axes, etc.) of the VR terminal 200. It may be a sensor that can know the movement (rotation) of. The motion sensor may be configured to sense movement of the VR terminal 200 (viewer) and to sense at least the direction of the movement of the VR terminal 200 (viewer) (the rotation direction of the VR terminal 200).

The input device of the VR terminal 200 may transmit an input signal to a smartphone or an internal control unit of the VR terminal 200 through a wireless network. The input signal may be, for example, data indicating a selected button among a plurality of buttons included in the input device or indicating pressing of a specific button.

The VR terminal 200 generates control information according to an event generated by a user (observer) such as an input signal event or a motion event of an input device and transmits the control information to the VR providing server 100.

For example, the VR terminal 200 transmits control information including motion data sensed through the motion sensor to the VR providing server 100. The motion data may be, for example, a pitch, yaw, roll value, etc. sensed by a three-axis (or more) gyro sensor or the like and filtered if necessary. The pitch, yaw, and roll values may indicate the directionality with respect to the X axis, the Y axis, and the Z axis with respect to the center point of the VR terminal 200 (motion sensor). The VR terminal 200 assigns control information including motion data (direction data, pitch, yaw and roll data of the X, Y, and / or Z axes) determined through the motion sensor to a specified period (for example, 1 second). In the case of transmitting according to the < RTI ID = 0.0 >) or when the difference between the previous motion data and the threshold value is recognized, the control information including the sensed motion data is transmitted to the VR providing server 100.

In addition, the VR terminal 200 receives the input signal from the input device, configures input data corresponding to the received input signal, and provides the control information including the configured input data through the broadband network through the VR providing server 100. To send. The input data may indicate the selection of adjacent (left or right) VR images, the selection of a relay image, or the selection of a link image in the VR image.

The reservation terminal 300 is a terminal that reserves real-time VR viewing available in the VR providing server 100. The reservation terminal 300 may be, for example, a smartphone, a tablet PC, a notebook computer, a personal computer, or the like. The reservation terminal 300 may be the same device as the smartphone included in the VR terminal 200.

The reservation terminal 300 may log in to the VR providing server 100 through a provided web browser and reserve one or more VR viewings among various real time viewings (relays) provided by the VR providing server 100.

The reservation terminal 300 may log in to the VR providing server 100 through a web browser, select one or more of the plurality of reserved viewing performances and sports viewing, and select one of several locations that can be viewed in the selected viewing. have. The reservation terminal 300 may designate the VR terminal 200 to receive the VR image. For example, the reservation terminal 300 may set an identifier (for example, an IP address or a mobile phone number) of the VR terminal 200.

In addition, the reservation terminal 300 may select another spectator (for example, a user allowed access by the user of the reservation terminal 300) reserved at the same viewing position, and the VR terminal 200 may be a VR terminal ( According to the movement of 200, an image from the viewer may be received.

The camera 400 is installed in a performance hall or sports stadium, captures a VR image (3D image), and transmits the captured VR image to the VR providing server 100 in real time.

Each venue has a plurality of 3D cameras 400 at each viewing position, and each of the plurality of cameras 400 at each viewing position is centered on a stage (stage, sports ground, playground, etc.) where a performance or sport is performed. Capture 3D images of different angles of view.

2 is a diagram showing an arrangement example of a camera 400 installed in an arbitrary performance hall or sports stadium.

2 may be a performance venue such as a play, an opera, a musical, or a sports field such as soccer, baseball, basketball, volleyball, or tennis.

Cameras 400 for capturing 3D images (VR images) are fixedly installed at a designated location in a performance hall or a stadium. The cameras 400 are fixedly installed at any specific position around a stage or a playground in which performances or sports are performed.

The cameras 400 are installed at a plurality of viewing positions, and a plurality of cameras 400 are installed at each viewing position. Each camera 400 has an angle of view for capturing a 3D image. Each camera 400 installed at each viewing position has a different field of view. The plurality of cameras 400 at each viewing position may have an angle of view according to a lens, and the at least one camera 400 may be a wide angle camera.

The plurality of cameras 400 at each viewing position have different angles of view with respect to the stage. For example, the camera 400 of the center (reference) of the plurality of cameras 400 has an angle of view toward the center of the stage, and the camera 400 on the right side has an angle of view toward the right side of the stage than the center camera 400. The camera 400 on the left has an angle of view toward the left side of the stage than the center camera 400.

As such, the cameras 400 installed on the right side from the center camera 400 have an angle of view toward the right side more than the camera 400 in the adjacent left direction. In addition, the cameras 400 installed on the left side from the central camera 400 have an angle of view toward the left side more than the camera 400 in the adjacent right direction.

Adjacent cameras 400 have angles of view facing different angles (eg, by a set 10 degrees), and the two cameras 400 capture overlapping VR image images and the ends of the VR images according to the difference in the angles of view. To capture different video images. For example, the center camera 400 has a camera orientation angle (0 degrees) toward the center of the stage, and the adjacent right camera 400 has an orientation angle of 10 degrees to the right direction than the center camera 400 and has an adjacent left camera ( 400 has a directing angle of 10 degrees to the left of the central camera 400. As such, adjacent cameras 400 have different camera orientation angles.

The specific viewing position may be located closer to the stage (stage) or in a grandstand cheering for a specific team (home team). Other viewing locations may be located in the bleachers away from the stage or cheering for other teams. Different merit for each viewing position can be provided to the viewer.

There are a plurality of viewing positions that can be viewed in each performance hall or stadium, and one or more (preferably a plurality of) cameras 400 are installed in each viewing position. The plurality of cameras 400 may capture a VR image having different orientation angles overlapping with the adjacent camera 400 and transmit the captured VR images to the VR providing server 100. The transmitted VR image may be transmitted as a compressed image according to a designated compression format, and the compressed image includes a time stamp.

For example, the plurality of cameras 400 at the same location may transmit a compressed image including the same time stamp to the VR providing server 100. In this case, the plurality of cameras 400 transmits a captured image signal to an image transmitting apparatus (not shown), and the image transmitting apparatus compresses each image signal into a compressed image and at the same time from the plurality of cameras 400. The same time stamp may be recorded in the recognized or compressed image frame and transmitted to the VR providing server 100.

VR images captured at different angles of view may include a time stamp for each image frame, and the time stamp may be used for a later VR image change (angle view change).

A broadband network (not shown) is a network for transmitting and receiving various data with the VR terminal 200, the reservation terminal 300, the camera 400, and / or the VR providing server 100. A broadband network may consist of a combination of wired and wireless networks and may, for example, constitute an internet network.

Various control data and VR images may be transmitted and received through the broadband network.

3 is a diagram illustrating an exemplary block diagram of the VR providing server 100.

According to FIG. 3, the VR providing server 100 includes a communication unit 101, a storage unit 103, a connection unit 105, and a control unit 107. 3 preferably shows a functional block diagram and each functional block has a corresponding hardware block. The VR providing server 100 of FIG. 3 may be configured using various hardware. For example, the VR providing server 100 may be configured using one or more PCs, workstations and / or server frames.

The VR providing server 100 may be divided into a plurality of physical or logical servers according to the function provided to the viewer. For example, the VR providing server 100 may be divided into a server that performs a viewing reservation and a server that is responsible for relaying a real-time VR image after the reservation.

Referring to the VR providing server 100 through Figure 3, the communication unit 101 is an interface for transmitting and receiving data with the broadband network. The communication unit 101 is configured to transmit / receive data with the plurality of cameras 400, the reservation terminal 300, and / or the VR terminal 200. The communication unit 101 includes, for example, a communication chipset for connecting to a wired LAN, and can be connected to a broadband network through a wired LAN.

For example, the communication unit 101 receives from the reservation terminal 300 information or data related to various viewing reservation settings such as selection of reservation viewing, selection of viewing position, selection of another viewing audience, and the like, and the control unit 107. Or store in the storage unit 103. In addition, the communication unit 101 may receive a plurality of VR images from the plurality of cameras 400 and store the received plurality of VR images in the storage unit 103 (via the control unit 107).

The storage unit 103 stores various data and programs, including mass storage media such as volatile memory, nonvolatile memory, and / or hard disk. The storage unit 103 may provide various services to viewers, including a reservation program for registration of a viewing reservation, a VR providing program for providing a corresponding VR terminal 200 of a viewer who has reserved a real-time VR image, and the like.

The storage unit 103 also stores various data. For example, the storage unit 103 temporarily stores a VR image captured at a performance hall or a sports ground that is currently being performed in real time. For example, the storage unit 103 may temporarily store a VR image for each viewing position in a corresponding subdirectory, including a directory corresponding to a performance hall or a stadium currently being performed in real time, and a subdirectory divided by viewing positions in the directory. have. The VR image corresponding to the camera 400 stored in the sub directory is classified and stored to be recognizable by the control unit 107 according to the angle of view of the camera 400. For example, the sub-directory for each viewing position stores the VR image for each camera 400 by separating the VR image for each camera 400 (with a different file name or a different directory name).

In addition, the storage unit 103 stores setting information set by the viewer and at least representing a selection of a specific viewing position within the specific stage and further including other information. To this end, the storage unit 103 includes a plurality of DBs as storage structures.

For example, the storage unit 103 includes a user DB, a viewing DB and a reservation DB.

The user DB is a DB for specifying a user who can view real-time VR for a specific performance or sport through the VR providing server 100. The user DB consists of one or more entries, each entry including a user identifier and a password, and an identifier of the VR terminal 200 and an identifier list of other viewers.

The user identifier represents, for example, a user ID (composed of a combination of numbers, letters and / or special characters) and the password is set by the user and used to log in to the VR provision server 100, numbers, letters and / or special characters. It is composed of a combination. The identifier of the VR terminal 200 is information for identifying the VR terminal 200 used by a user (viewer), and may be, for example, an IP address, a MAC address and / or a mobile phone number of the VR terminal 200. The identifier list may include identifiers of other users. The identifier list may include one or more identifiers of other users registered by the present user and permitted by other users, for example. The identifier list allows the user to acquire a video or image of a friend (who is watching the same performance or sport in VR).

The viewing DB is a DB for specifying a viewing event capable of providing a VR image in real time through the VR providing server 100. The viewing DB consists of one or more entries. Each entry includes a viewing event identifier, a viewing date and time, start time, end time, viewing event description information, and selectable viewing position information on the stage of the viewing event.

The viewing event identifier is composed of a combination of numbers, letters and / or special characters to distinguish it from other viewing events. The viewing event identifier may be set by the VR providing server 100 or the like. The viewing date, time, start time and end time specify the time at which the viewing event occurs. The viewing event description information is information provided to explain the viewing event to the user and includes, for example, information such as a title of the viewing event, a feature point of the viewing event, and notes.

Viewing location information includes information on one or more locations that can be viewed within the viewing event, and information about each viewing location relative to the stage, costs for applying for VR viewing at each viewing location, and feature points of each viewing location. Save the data.

The viewing position information stores identifiers of the plurality of cameras 400 installed at each viewing position, (direction) angle of view information, and corresponding plurality of movement information. The identifier of each of the cameras 400 is data for distinguishing it from other cameras 400 and includes, for example, an IP address, a MAC address or an index composed of numbers of the cameras 400. Each angle of view information may be expressed as, for example, a relative left and right angle of the angle of view directed by each camera 400 in a line from the reference (center) camera 400 to the center of the stage among the plurality of cameras 400. .

For example, the camera 400 of the center may be set to an angle of view of 0 degrees, the camera 400 of the adjacent left side may be set to +10 degrees, and the camera 400 of the adjacent right side may be set to −10 degrees. Each camera 400 captures a VR image through a different view angle.

The motion information may represent or extract motion data of the VR terminal 200 that is matched with each angle of view. The motion data of the VR terminal 200 may represent the viewer's movement matching the angle of view of the camera 400. The motion information may further include a range of other motion data.

The reservation DB stores viewing reserved information by registered users. The reservation DB is composed of one or more entries and each entry includes a user identifier, a viewing event identifier, and reservation setting information. The reservation setting information includes selection data of a viewing position set by a user (viewer), and an identifier (hereinafter referred to as a 'friend identifier') of another user (friend) set by the user. There may be two friend identifiers, and each friend identifier is stored in the reservation setting information along with the mapped direction information (for example, left or right).

The connection unit 105 transmits and receives data between blocks in the VR provision server 100. The connection unit 105 is configured using a local area network, a parallel bus, a serial bus, or the like.

The control unit 107 is provided with one or more execution units to load the program stored in the storage unit 103 and execute the program instruction code through the execution unit to control the VR providing server 100 and real-time It provides a VR image providing service to a user (a viewer).

For example, the control unit 107 can load a reservation program of the storage unit 103 to receive reservation requests from various users and add new entries to the reservation DB to perform various processes related to reservation.

In addition, the control unit 107 loads the VR providing program of the storage unit 103 to receive a plurality of VR images of each of the plurality of viewing positions of the viewing event currently in progress in real time, and temporarily stores them in the storage unit 103. The VR unit 200 which stores and reserves the corresponding viewing event corresponds to the selected viewing position among the plurality of viewing positions according to the selection data of the reservation setting information and temporarily stores the VR image stored in the storage unit 103 in the communication unit 101. To send).

In addition, the control unit 107 receives control information through the communication unit 101 from the VR terminal 200 viewing the VR image. The control information includes motion data or input data. The control unit 107 may select one or more VR images corresponding to the motion data or the input data of the control information and transmit the selected VR images to the VR terminal 200 through the communication unit 101.

For example, the control unit 107 selects the VR image corresponding to the movement in the left and right directions when the motion data indicates the movement in the left and right directions. The control unit 107 selects the camera 400 closest to the movement data among the plurality of cameras 400 at the set viewing position, and transmits the temporarily stored VR image of the selected camera 400 through the communication unit 101 to the VR terminal. Send to 200. Alternatively, the control unit 107 selects two cameras 400 that are closest to the movement data among the plurality of cameras 400 at the set viewing position. The control unit 107 synthesizes the temporarily stored VR images of the two selected cameras 400 into one image by reflecting the distance between the received motion data and each of the adjacent cameras 400 and converts the synthesized VR image into a communication unit ( 101 through the VR terminal 200.

The main control flow in the control unit 107 will be described in detail in FIGS. 4 and 5.

4 is a diagram illustrating an exemplary viewing reservation control flow.

First, the reservation terminal 300 accesses the VR providing server 100 (S101). For example, the reservation terminal 300 may access the VR provision server 100 through a provided web browser and log in to the VR provision server 100 through input of a user ID and a password.

Thereafter, the reservation terminal 300 browses the reservation list that can be reserved for viewing through the web browser (S103). The control unit 107 of the VR providing server 100 (communication unit 101) constructs a reservation page including a reservation list using a viewing DB according to a request for transmission of a reservation page through a web browser, and communicates with the reservation terminal 300 by the communication unit 101. To send). The reservation list is composed of one or more reservation items, each reservation item including a viewing event identifier and displayed on a web browser or the like including information on a viewing event's title, viewing date, start time, end time, and the like.

The user of the reservation terminal 300 may browse the reservation list and select a specific reservation item to check additional information. For example, upon receipt of a user input from the reservation terminal 300, the reservation terminal 300 may output viewing event description information of the reservation item selected in cooperation with the VR provision server 100 on a display.

The reservation terminal 300 selects a specific reservation item from a plurality of reservationable items for viewing reservation through browsing the reservation list (S105). For example, the reservation terminal 300 may select a specific reservation item according to a user input.

As a specific reserved item is selected, the control unit 107 of the VR providing server 100 (the control unit 107 of the reservation terminal 300) of the entry retrieved from the corresponding viewing event identifier of the selected reserved item from the reservation terminal 300 as the viewing event identifier in the viewing DB The viewing position information is extracted and the extracted viewing position information is transmitted to the reservation terminal 300 through the communication unit 101.

The reservation terminal 300 displays the received viewing position information, and the reservation terminal 300 selects a specific viewing position according to a user's input (S107). The VR providing server 100 then receives the selection data for the selected viewing position.

In addition, the reservation terminal 300 may select another spectator or user to be located adjacent to the logged-in user when viewing the real-time VR image from the viewing position (S109). For example, the reservation terminal 300 receives from the VR providing server 100 a list of other spectator identifiers included in the entry of the logged-in user (entry of the user DB) and displays it and specifies one (two) spectator identifiers. The selection for can be received via user input. The selected viewer identifier (friend identifier) may be received from the user with direction selection (eg, left direction, right direction, etc.).

Thereafter, the reservation terminal 300 may complete the reservation through the confirmation process or the payment process (S111). As the reservation is completed, the reservation terminal 300 transmits a reservation completion signal (request) to the VR provision server 100 and the control unit 107 of the VR provision server 100 generates a new entry in the reservation DB. And the identifier of the user logged into the entry, the viewing event identifier of the reserved viewing, and the reservation setting information. The reservation setting information includes the selected viewing position data and one or more friend identifiers. The friend identifier may be mapped and stored together with the direction information.

The control unit 107 of the VR providing server 100 (the control unit 107) may obtain a user identifier through a login process, and the viewing event identifier and reservation setting information may be reserved according to a user selection linked to the reservation terminal 300. The terminal 300 may be received and set.

As the reservation is completed, the control unit 107 of the VR provision server 100 (the control unit 107) may then provide the VR image at the designated viewing position in real time to each user who requested the reservation.

5 is a diagram illustrating an exemplary flow of providing a user with a real-time VR image according to a viewing reservation.

As the viewing event of a specific entry of the viewing DB is started (through recognition of the viewing date and time of entry of the viewing DB), the VR providing server 100 (control unit 107 of) enters a plurality of viewing of the viewing event. The VR image captured in real time from each of the plurality of cameras 400 of each position is received through the communication unit 101 (S201).

The control unit 107 of the VR providing server 100 temporarily stores the VR image of the received camera for each viewing position in the storage unit 103 (S203). For example, the control unit 107 of the VR providing server 100 (the control unit 107 of the storage unit 103) stores a subdirectory corresponding to each camera 400 of each viewing position directory of the viewing event in the storage unit 103 (for example, a camera ( Directory of the identifier name 400) to receive and temporarily store the VR image from each camera 400. The plurality of cameras 400 received from each of the plurality of viewing positions are installed with different angles of view at the viewing positions, and the VR provision server 100 (control unit 107 of) is captured at different angles of view for each viewing position. VR images are received from the cameras 400.

Processes S201 and S203 are repeatedly performed from the start of the viewing event to the end of the viewing event. The received VR image may be received in a compressed format and each frame of the VR image may have a time stamp. The frame of each VR image having the same time stamp is an image captured within the same time or at least frame time difference.

In addition, the control unit 107 of the VR providing server 100 searches for a reservation entry for viewing a viewing event that is currently receiving a VR image from the reservation DB of the storage unit 103. For example, the control unit 107 of the VR providing server 100 may extract all reservation entries having an identifier of a viewing event. Thereafter, the VR providing server 100 (the control unit 107) may transmit the real-time VR image of the viewing position selected for each viewer among the various viewing positions to all the VR terminals 200 of the reservation entry.

Looking at the process in detail, the VR providing server 100 (control unit 107 of) (S205) sets the initial VR image setting for each spectator who reserved the viewing event that the real-time VR image is generated.

For example, the control unit 107 of the VR providing server 100 (at least one communication channel for transmitting and receiving VR images and control information through the broadband network with each VR terminal 200 of the searched viewer-specific reservation entry) is provided. Setting information is received from each VR terminal 200, including setting data including initial motion data, from the set communication channel via the communication unit 101.

The VR terminal 200 establishes a communication channel with the VR providing server 100 and uses three axes (X, Y, X, Y) from the data sensed through a motion sensor provided according to a user's input (for example, a button input of an input device). Z-axis (or pitch, yaw, roll) generates the motion data and transmits the setting information including the generated motion data to the VR providing server 100.

The control unit 107 of the VR providing server 100 (movement data, motion data, etc.) of the reference (central) camera 400 of the viewing position selected by the viewer for the initial movement data received from the VR terminal 200 for each viewer. Viewing position information of the viewing entry). According to the matching, a relative difference between the initial motion data and the motion data of the reference camera 400 may be determined.

Accordingly, the initial motion data is matched to the motion information of the central camera 400 having the angle of view toward the center of the stage of the viewing position selected by the viewer, and then the camera 400 of the different angle of view is changed according to the relative difference of the received motion data. You can choose.

The control unit 107 of the VR providing server 100 may store the initial VR image setting for each viewer as data in the storage unit 103. For example, the control unit 107 of the VR providing server 100 (the control unit 107) may include a viewer's selected viewing position identifier, current movement data from the VR terminal 200, and an identifier of the currently selected image (currently selected camera 400). Stores various setting information for VR image control, such as an identifier of an adjacent camera 400 of the currently selected camera 400, a difference between initial received motion data, and motion data of the reference camera 400.

According to the initially received setting information, the current motion data is set as the initial motion data, and the identifier of the image currently selected according to the reception of the initial motion data may be, for example, an identifier of the reference (central) camera 400. The identifier of the adjacent camera 400 may be a camera 400 immediately adjacent to one direction (left or right) of the reference (center) camera 400.

Prior to the reception of the control information according to the movement or the like from the VR terminal 200, the control unit 107 of the VR providing server 100 configures the VR image according to the VR image setting at the selected viewing position for each viewer (S207). )do.

For example, the control unit 107 of the VR providing server 100 (the control unit 107 of the VR image setting) uses the identifier of the current image (currently selected camera 400) of the VR image setting (data) to select a subdirectory (currently selected) A VR video stream to be transmitted may be configured by extracting a series of compressed VR images according to a time stamp temporarily stored in a subdirectory corresponding to the camera 400. Also, the control unit 107 of the VR providing server 100 may further include a link in the VR video stream. This link may be a URL for linking to a specific site or a URL for linking to a specific advertisement.

The control unit 107 of the VR providing server 100 transmits the VR image configured for each viewer to the corresponding VR terminal 200 of the viewer through the communication unit 101 (S209).

Accordingly, the VR terminal 200 may display a VR image at a location selected by a viewer and received in real time on one or two displays provided.

The VR terminal 200 may generate control information according to various events occurring in response to the received VR image.

For example, the VR terminal 200 includes a motion sensor and configures the motion data according to the data from the motion sensor according to the left and right rotation or the up and down rotation of the VR terminal 200 (the head of the VR headset). Control information may be generated.

Alternatively, the VR terminal 200 may receive a user input for selecting an adjacent VR image from an input device through a wireless network (for example, Bluetooth) and generate control information including input data representing the user input. Can be.

Alternatively, the VR terminal 200 may receive a user input for selecting a link displayed in a VR image from an input device through a wireless network and generate control information including selection data indicating user selection.

The VR terminal 200 transmits the generated control information to the VR providing server 100 via the broadband network and the control unit 107 of the VR providing server 100 receives the control information through the communication unit 101. (S211). If control information is not received from the corresponding VR terminal 200 for each viewer, the VR image is configured according to the existing VR image setting and continues to be transmitted (S207, S209).

When control information is received for each viewer, the control unit 107 of the VR providing server 100 changes the VR image setting according to the received viewer-specific control information (S213).

For example, when the control information includes input data, the control unit 107 of the VR providing server 100 (the left camera (for example, the left camera (for example)) is selected according to the input data in the VR image setting. 400 or the camera 400 corresponding to the right camera 400 is set as the identifier of the currently selected image, and the identifier of the adjacent camera 400 is the identifier of one camera 400 adjacent to the identifier of the current image. Set it. Alternatively, the control unit 107 of the VR providing server 100 sets the identifier of the relay video selected according to the input data to the VR video setting as the identifier of the currently selected video.

For example, if the control information includes selection data indicating a selection for a link included in the VR image, the control unit 107 of the VR providing server 100 (link) includes the link included in the VR image in the VR image setting. Set the identifier of to the identifier of the currently selected image.

For example, if the control information includes motion data, the control unit 107 of the VR provision server 100 (the control unit 107) stores the received motion data as the current motion data in the VR image setting and various types of motion data. Select the image to be output.

Specifically, the control unit 107 of the VR providing server 100 (the control unit 107 of the received control information) is the left and right directions of the VR terminal 200 (for example, the vector direction of the X-axis and Y-axis of the horizontal plane or Pitch, yaw, and the recognized left and right directions from the roll, etc.), the VR provision server 100 (control unit 107 of the) provides one or more cameras 400 corresponding to the received movement data at the viewing position. Decide

For example, the control unit 107 of the VR providing server 100 (the control unit 107 of the) provides a camera 400 closest to the camera 400 closest to the received motion data and the distance (for example, the vector distance, etc.) to the second. The determination is made using the viewing position information of the viewing entry. The control unit 107 of the VR providing server 100 sets the identifier of the camera 400 closest to the VR image setting as the identifier of the currently selected image and sets the identifier of the second closest camera 400 to the adjacent camera ( 400).

When the received motion data is a motion equal to or greater than an angle set to the left and right directions (eg, +30 degrees, -30 degrees, etc.), the identifier of another user preset at the time of reservation is set as the identifier of the currently selected image.

When the motion data is a vertical motion of the VR terminal 200 and the vertical motion indicates a motion of a specified angle or more (for example, -20 degrees, etc.), the control unit of the VR providing server 100 ( 107) sets the identifier of the relay image (relayed VR broadcast) generated by the broadcaster as the identifier of the image currently selected in the VR image setting for the viewing event that the viewer is watching.

As the VR image setting is changed, according to the changed VR image setting, a subsequent VR image is configured (S207) and transmitted to the VR terminal 200 (S209).

For example, upon reception of the selection data, when the identifier of the link is set to the identifier of the currently selected video, the control unit 107 of the VR providing server 100 is connected to the link identifier stored in the storage unit 103. The corresponding content VR image is configured as an image to be output and transmitted to the VR terminal 200.

For example, when the motion data of the control information indicates a left-right movement and a motion within a specified angle, the currently selected image corresponding to the movement among the plurality of VR images at the viewing position (camera ( The VR image corresponding to the identifier of 400) may be configured and transmitted as an image to be extracted and output from the temporarily stored storage unit 103. As such, the control unit 107 of the VR providing server 100 may configure the VR image of the angle of view corresponding to the movement in the left and right directions as the image to be output.

There may be one or a plurality of images transmitted to the VR terminal 200. When a single image is transmitted, the control unit 107 of the VR providing server 100 may transmit the currently selected VR image to the VR terminal 200. Alternatively, the VR providing server 100 (the control unit 107 of) configures and synthesizes an image to be output by combining the VR image corresponding to the currently selected camera 400 and the VR image corresponding to the adjacent camera 400. The VR image may be transmitted to the VR terminal 200. In the process of synthesizing the images, the VR providing server 100 (control unit 107 of) preferably combines one VR image with respect to two VR images having the same time stamp.

For example, the control unit 107 of the VR providing server 100 (the control unit 107 of the VR image setting) is currently selected using the current motion data of the VR image setting, the motion information of the currently selected camera 400, and the motion information of the adjacent camera 400. The regions of the VR image to be used in the camera 400 may be determined and the regions of the VR image of the adjacent camera 400 may be determined to synthesize the two regions determined. The VR image of the adjacent camera 400 may have a relative similarity between the adjacent direction of the adjacent camera 400 and the current motion data (for example, the similarity between the currently selected camera 400 and the current motion data and the adjacent camera 400). The size of the region to be synthesized in the VR image region at the end of the adjacent direction is determined according to the relative similarity between similarities between the current motion data. Frames of two VR images to be synthesized have the same time stamp.

As such, the control unit 107 of the VR providing server 100 selects and transmits a single VR image of an angle of view corresponding to the left and right motion data received from the VR terminal 200 or transmits two VR images. A VR image to be selected and output by combining two selected VR images may be configured and transmitted to the VR terminal 200.

There may be a plurality of images to be transmitted, for example, the VR providing server 100 (control unit 107 of the) is a VR image of the currently selected image (camera 400) of the VR image setting and the adjacent camera 400 The VR image may be configured as a VR image to be transmitted and transmitted to the VR terminal 200. In this case, the VR terminal 200 may synthesize the two VR images into a single VR image by synthesizing similar to the synthesis process described above, and output them to the display.

According to the changed VR image setting when the motion data of the control information indicates the movement in the left and right direction and the movement above the designated angle, the control unit 107 of the VR providing server 100 outputs an image of another user. It is configured as and transmits to the VR terminal 200.

The control unit 107 of the VR providing server 100 may store an image of another user (friend) in advance in the storage unit 103 or may receive in real time from the VR terminal 200 of the other user. The image of another user may be an image captured in real time by a camera or a pre-installed camera of the VR terminal 200. Alternatively, the image of another user may be an AR (Augmented Reality) emoji configured according to image capturing. AR emoji can be extracted directly from the real-time image or pre-configured.

Through this, the user of the VR terminal 200 may view a friend image (viewing) together according to his movement.

According to the changed VR image setting when the motion data of the control information indicates the movement in the up-down direction and the movement above the designated angle, the control unit 107 of the VR providing server 100 can output the image from the broadcaster or the like. It consists of an image and transmits it to the VR terminal 200.

Specifically, when the identifier of the currently selected image is the identifier of the relay image transmitted by the relay broadcasting station or the like, according to the changed VR image setting, the control unit 107 of the VR providing server 100 relays the relay generated by the relay broadcaster. The VR image is configured as an image to be output and transmitted to the VR terminal 200. The relay VR image may also be temporarily stored in the storage unit 103 and output to the VR terminal 200.

As such, the control flow of FIG. 5 dynamically constructs a corresponding VR image in response to control information received from the VR terminal 200 of the viewer, and transmits the corresponding VR image to the VR terminal 200. Accordingly, the virtual reality viewing system according to the present invention may provide a viewer with an environment such as actual viewing by providing various real-time VR video contents corresponding to various types of movements of the viewer.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited by the drawings.

100: VR provision server
101: communication unit
103: storage unit
105: connection unit
107: control unit
200: VR terminal
300: reservation terminal
400: camera

Claims (10)

(a) the VR providing server receiving a selection of one viewing position among the plurality of viewing positions;
(b) the VR providing server receiving in real time one or more VR images captured at each of the plurality of viewing positions; And
(c) transmitting, by the VR providing server, one or more real-time VR images corresponding to the selected viewing position among the plurality of viewing positions to the VR terminal;
How to watch virtual reality. The method of claim 1,
Step (b) receives a plurality of VR images captured at different viewing angles of each viewing position,
Step (c) is to transmit the current VR image and the adjacent VR image determined in conjunction with the VR terminal of the plurality of VR images captured in real time to the VR terminal,
How to watch virtual reality. The method of claim 1,
After step (c),
(d) the VR providing server receiving control information from the VR terminal;
(e) the VR providing server constructing an image to be output to the VR terminal according to the control information; And
(f) transmitting, by the VR providing server, an image configured corresponding to the control information to the VR terminal; further comprising:
How to watch virtual reality. The method of claim 3,
The control information includes motion data of the VR terminal,
Step (b) receives a plurality of VR images captured at different viewing angles of each viewing position,
In the step (e), when the motion data indicates movement in the left and right directions, a VR image having an angle of view corresponding to the movement in the left and right directions among a plurality of VR images is configured as an image to be output.
How to watch virtual reality. The method of claim 4, wherein
In step (e), selecting two VR images corresponding to the motion data among a plurality of VR images and configuring a VR image to be output by combining two selected VR images,
How to watch virtual reality. The method of claim 4, wherein
In the step (e), when the motion data indicates the movement in the left and right directions and the motion data is a motion more than the first angle specified, the image to be output is configured as an image to be output, and the motion data indicates the movement in the vertical direction. If the motion is more than the specified second angle to configure the relay image generated by the broadcaster as an image to be output,
How to watch virtual reality. The method of claim 3,
The control information includes selection data indicating a selection for a link included in the VR image transmitted to the VR terminal,
In step (e), the image corresponding to the link is configured as an image to be output according to the reception of the selection data.
How to watch virtual reality. As a virtual reality viewing system,
And a VR providing server constituting a VR image to be transmitted to a viewer from a plurality of connected cameras.
The VR providing server,
A communication unit for receiving a plurality of VR images from the plurality of cameras and receiving a selection for one viewing position among the plurality of viewing positions;
A storage unit for classifying and temporarily storing VR images at each viewing position according to an angle of view of a corresponding camera, and storing setting information set by the viewer and indicating selection of the viewing position; And
And a control unit corresponding to a selected viewing position among a plurality of viewing positions according to the selection data of the setting information, and transmitting a temporarily stored VR image to the VR terminal through the communication unit.
Virtual reality viewing system. The method of claim 8,
When the control unit receives control information including motion data from the VR terminal through a communication unit and the motion data indicates movement in the left and right directions, the control unit corresponds to the movement in the left and right directions among a plurality of VR images at a set viewing position. Selecting two temporarily stored VR images of an angle of view and synthesizing the selected two VR images to transmit the synthesized VR images to the VR terminal through the communication unit,
Virtual reality viewing system. The method of claim 8,
And a VR terminal for receiving a real-time VR image from the VR providing server and generating control information according to an event generated by a user and transmitting the generated control information to the VR providing server.
The VR terminal transmits control information including motion data sensed through the motion sensor, including an input device connected through a motion sensor or a wireless network, to the VR providing server or control information including input data through the input device. Send to the VR providing server,
The VR providing server selects a VR image among a plurality of VR images at a viewing position set according to the data of the control information, and transmits a VR image configured based on the selected VR image to the VR terminal.
Virtual reality viewing system.

Images