KR20180054377A - Interactive virtual reality system - Google Patents

Abstract

In a virtual reality system including a client system and a server system, the client system includes: a UI navigator unit for controlling the movement of a cursor according to a gaze of a viewer and determining whether to select according to a set time interval; and a video manager and switch unit for managing and playing a plurality of 2D images according to an instruction of the UI navigator unit and switching a played video source. According to an interactive virtual reality system, VR real-time broadcasting can be viewed by a better environment in an HMD terminal.

Description

INTERACTIVE VIRTUAL REALITY SYSTEM [0002]

The present invention relates to an interactive virtual reality system.

In the Virtual Reality (VR) contents, which are provided in Korea, live contents excluding game and VOD are mainly relayed video images using a 360 degree camera. However, while 360-degree images have the advantage of presence, they have the disadvantage of interfering with the immersion, and it is urgent to expand the universal contents. However, there has yet to be a case in which the VR image, the HD image, the additional information, and the optional viewing technology providing the virtual space through the head mount display (HMD) have not been provided yet.

Various efforts are being made to produce VR contents for various video contents such as 2D, 3D, and 360 degree video mainly in USA, which is an IT powerhouse. In particular, live streaming technology of sports and performances is being developed recently. However, according to the current live streaming technology, sports and performances (concerts, musicals, etc.) enjoyed in the stadium and / or theater are excellent in the real sense, but the distance is inconvenient and inconvenient. . In addition, the present live streaming technology provides a sense of presence of the VR, but there is also a problem of a disconnection effect which occurs alone when the VR terminal is used.

An object of the present invention is to provide an interactive virtual reality system capable of viewing VR real-time broadcasting in a better environment in an HMD terminal.

The interactive virtual reality system according to an embodiment of the present invention is a virtual reality system including a client system and a server system, wherein the client system controls movement of the cursor according to a line of sight of the viewer, An infant navigator unit for determining whether to select according to the selection; And a video manager and a switch unit for managing and playing a plurality of two-dimensional images according to an instruction of the infant navigator unit, and switching a video source to be played.

In addition, the client system may include a virtual three-dimensional room unit that can draw and maintain a virtual three-dimensional space; And a social meta bus unit for displaying icon and status information of a friend on the social network and allowing the friend to call or invite the social meta bus unit, And the invited friend can be displayed in an avatar format.

In addition, it is preferable that the virtual three-dimensional room part can create a virtual sky box and fill the space of the sky box.

In addition, the client system may include a dashboard unit that draws an area for displaying metadata, receives information on each area from the server system, and displays information corresponding to the area; And a virtual spatial audio unit that can output virtual audio by adding spatial information of an image currently being played.

Advantageously, the client system further comprises: a video player for displaying video information received from the server system; And an HMD renderer unit for displaying video, metadata and CG objects on the head mount display, wherein the HMD renderer unit is capable of simultaneously displaying a 180-degree image and a 360-degree image on the head mount display do.

Specifically, when the cursor stays at a preset position for a predetermined time or longer, the HMD renderer may switch the 360-degree image to the 180-degree image or switch the 180-degree image to the 360-degree image.

In addition, the server system may include an audio meta server unit for managing virtual audio information according to spatial information; A room management server unit for managing virtual room information; An AD metadata server unit for managing advertisement related metadata; And a camera meta server unit for managing camera information for acquiring video information. Preferably, the camera information includes a camera model, a lens, resolution, and photographing position information.

According to the interactive virtual reality system of the present invention, the VR real time broadcast can be viewed by the HMD terminal in a better environment.

1 is an explanatory diagram of a virtual reality to be implemented by an interactive virtual reality system according to a preferred embodiment of the present invention;
2 is a configuration diagram of an interactive virtual reality system according to an embodiment of the present invention;

Hereinafter, an interactive virtual reality system according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

It should be understood that the following embodiments of the present invention are only for embodying the present invention and do not limit or limit the scope of the present invention. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

1 is an explanatory diagram of a virtual reality to be implemented by an interactive virtual reality system 1000 according to a preferred embodiment of the present invention.

1, according to an interactive virtual reality system 1000 according to a preferred embodiment of the present invention, when a viewer wears a head-mounted display (HMD), he or she enters a virtual lobby space It is possible to receive program information currently being screened, and to move to a virtual sky box.

The viewer can be provided with the program information in two steps of genre selection and program selection.

Moving to the virtual skybox, the viewer can view the 180 degree image and / or the 360 degree image, and each image can be provided as two dimensional and / or three dimensional images.

When watching the game on the stadium, most of the time watching forward, rarely watching the back, 180 degree image is used because it is bigger than the viewing angle so you can feel the presence feeling. In addition, the resolution of the 180-degree image is higher than that of the 360-degree image. Utilizing the 180 degree image will enable you to take two big advantages that can increase the realism and resolution, and it will be able to get much better viewing experience by accepting the disadvantage that you can not see the rear.

In addition, VR real-time relay broadcasting for real-time broadcasting of professional baseball is 360 degree mono video. So I can not feel the three-dimensional feeling. Therefore, you should be satisfied with feeling a little sympathetic feeling. This is because it is difficult to capture a 360-degree image in stereo-scopes and the amount of image data to be transmitted is too large. However, 180-degree images can be shot with stereoscopic images, and the amount of data to be transmitted is not more than 360-degree monochrome images.

However, even if the resolution of the 180-degree image is high, it is not easy to know the number of players, the content of the electric signboard. In addition, since the virtual reality system 1000 of the present invention can attract a viewer's attention by having more special information than a direct view to the stadium, the virtual two-dimensional relay image that viewers can easily enjoy using the dashboard function Take it and place it like a billboard. In addition, since the virtual reality system 1000 of the present invention utilizes a 180-degree image, the remaining 180-degree space becomes void in the HMD, so a virtual sky box such as a sky box of the stadium is created to fill the empty space, It also provides an experience to sit and watch the game in a VIP-like mood.

Accordingly, viewers can watch the baseball game relay in the HMD terminal, enjoy the lively movements of the arena with the 180-degree image, and enjoy the familiar relay broadcast in the two-dimensional image at the same time. In addition, the virtual reality system 1000 of the present invention develops a function capable of selectively viewing a plurality of two-dimensional images, thereby enabling a user to interactively and selectively watch a plurality of two-dimensional sports relay images.

A virtual advertisement is placed in the sky box so that the viewer can view the advertisement as well as allow the viewer to order the advertisement product.

In addition, it is also possible to invite a friend on a social network by the social metaverse function, and to watch the program together in the virtual space.

Detailed detailed configurations for implementing an interactive virtual reality system 1000 according to a preferred embodiment of the present invention will be described below with reference to FIG.

2 is a block diagram of an interactive virtual reality system 1000 according to an embodiment of the present invention.

2, the interactive virtual reality system 1000 includes a client system 100 and a server system 200 according to an embodiment of the present invention.

The client system 100 includes an HMD renderer unit 101, a video manager and switch unit 102, a video player unit 103, a virtual three-dimensional room unit 104, a baby navigator unit 105, a meta handler unit 106 A dashboard unit 107, a video navigator unit 108, a virtual space audio unit 109, an AD client unit 110, an AR client unit 111, a social meta bus unit 112, a voice communication unit 113 And a middleware unit 114. [ The client system 100 may be installed in the form of an application program on the viewer's terminal and operated by the processor.

The HMD renderer unit 101 plays a role of drawing video, metadata, and CG (Computer Graphoc) objects on a head mount display. Specifically, the HMD renderer unit 101 can simultaneously display the 180-degree stereoscopic image and the 360-degree image on the head mount display. When the reticle cursor controlled by the infant navigator 105 stays at a predetermined position or longer than the predetermined time, the HMD renderer unit 101 switches from the 360-degree image to the 180-degree image , Or a 180-degree image to a 360-degree image. That is, when the viewer is looking at the 360-degree image, if the user gazes at the first point, he or she switches to the 180-degree image, or when the viewer looks at the 180-degree image, .

The video manager and switch unit 102 manages and plays a plurality of two-dimensional images in accordance with an instruction from the infant navigator unit 105 so as to switch the video source to be played. That is, the video manager and switch unit 102 functions to change the video source in accordance with the instruction of the infant navigator unit 105. [ By the video manager and switch unit 102, the viewer can selectively view a plurality of 2D sports relay images selectively by developing a function of selectively viewing a plurality of 2D videos on the developed player and framework basis do.

The video player unit 103 serves to display video information received from the server system 200. [ Specifically, the video player unit 103 decompresses the transmitted video into a codec and sends it to the display. The codec varies depending on the type of image such as HD, UHD, 180 degree image and 360 degree image.

The virtual three-dimensional room unit 104 can draw and maintain a virtual three-dimensional space. By virtue of the virtual three-dimensional room 104, a virtual sky box can be created to fill an empty space.

The infant's navigator unit 105 controls movement of the reticle cursor, determines whether to select according to a set time interval (length of time for gazing), and transmits the selected coordinates and object information to the related component. The reticle cursor may or may not be visible depending on the UI (User Interface) situation.

The meta handler unit 106 functions to request, fetch, and send metadata to the HMD renderer unit 101.

The dashboard unit 107 draws a region for displaying metadata (additional information), receives information about each region from the server system 200, and plays a role of displaying information corresponding to the region do.

The video navigator unit 108 searches the EPG server unit 213 or the multi-video meta server unit 215 for a corresponding video source when a request is made from the infant navigator, and obtains information (such as a link or a channel number) To the video manager and the switch unit 102 through the middleware unit 114.

The virtual space audio unit 109 can output virtual audio by adding spatial information of an image currently being played. That is, the virtual space audio unit 109 can be referred to as an audio codec and a player playing when the spatial audio source of the program comes in. By placing sounds that can be heard in the actual arena properly in the space, it is possible to provide a sense of being in a virtual space, thereby enhancing the sense of presence and immersion.

The AD client unit 110 receives the virtual advertisement related request and information from the middleware unit 114 and receives the related information from the AD meta server unit 214 to display the virtual advertisement. Here, the virtual advertisement related request and information include a program identification number (ID), a time stamp (Time Stamp), and an advertisement exposure condition.

The AR client unit 111 takes a separate CG object from the server system 200 in response to a request from the middleware unit 114, and displays the CG object in a virtual space.

The social meta bus unit 112 displays icons and status information of a friend on the social network, and allows the friend to call or invite. Specifically, the social meta bus unit 112 displays icons and status information (on / off line) of the social friends registered and registered in the user management server unit 223 of the server system 200 on the display, If you choose, you will be able to call or invite the friend. The social meta bus unit 112 can add the avatar system and the mVoIP function to the voice communication unit 113 and can also provide a multi-party conversation function so that a friend in another place can be summoned and talked while watching the same video .

Accordingly, the virtual three-dimensional room unit 104 can display the friends invited by the social metaverse unit 112 in the virtual three-dimensional space in the avatar format.

The voice communication unit 113 provides an mVoIP function for supporting communication between viewers.

The middleware unit 114 can perform functions to activate / deactivate other components in accordance with an input signal received from the infant navigator unit 105 and to perform communication (metadata delivery, action triggering, etc.) And controls hardware resources such as CPU / memory and priority control.

Hereinafter, the server system 200 of the present invention will be described in detail.

The server system 200 can be largely divided into a first part 210 for meta information and a second part 220 for a social meta bus and each part can be implemented by at least one server have.

The first part 210 for meta information includes an event server unit 211, a camera meta server unit 212, an EPG server unit 213, an AD meta server unit 214, a multi-video meta server unit 215, An AR server unit 216, and an audio meta server unit 217. The second part 220 for the social metaverse includes a lobby management server section 221, a room management server section 222, a user management server section 223 and a communication management server section 224. [

The event server unit 211 divides an input signal received from a viewer's terminal, requests a server call for processing an action according to a signal, transmits required information, and controls processing priority.

The camera meta server unit 212 serves to manage camera information for acquiring video information. The camera information includes a camera type, a lens, resolution, photographing position information, and activation / deactivation information. The photographing position information may include an identification number and coordinate information of the facility.

The EPG server unit 213 manages and provides information on the real-time channel, program information of the channel, update information management, program and transmission time information management, channel / program resolution, activation / deactivation information, and the like .

The AD meta server unit 214 manages and updates the advertisement-related metadata in the form of images or texts, and provides the requested information.

The multi-video meta-server unit 215 manages, updates, and provides information on stored image files and information on the advertisement image.

The AR server unit 216 plays a role of managing a CG object.

The audio meta server unit 217 manages and updates virtual audio information according to the spatial information.

The lobby management server unit 221 plays a role of controlling and lobbying the lobby. Specifically, the lobby management server unit 221 provides an API for interlocking a billing (ticket box) system, and manages and updates the arena / venue information.

The room management server unit 222 manages and updates virtual room information for each stadium / venue. The room management server unit 222 performs room management for each user in cooperation with the AD meta server unit 214 and the AR server unit 216. [

The user management server unit 223 performs user registration, login, activation / deactivation management, user state management, and relationship management between users.

The communication management server unit 224 preferably supports communication management and functions between users, and is preferably interlocked with the mVoIP softswitch of the communication service provider. In addition, the communication management server unit 224 performs call control and the like in cooperation with the user management server.

As described above, according to the interactive virtual reality system 1000 of the present invention, it can be seen that the HMD terminal can view the VR real time broadcasting in a better environment.

1000: Virtual Reality System
100: client system 200: server system
101: HMD Renderer Part 102: Video manager and switch section
103: Video player part 104: Virtual 3D room part
105: infant navigator unit 106: meta handler unit
107: Dashboard section 108: Video navigator section
109: virtual space audio unit 110: AD client unit
111: AR client part 112: Social meta bus part
113: Voice communication part 114: Middleware part
210: first part 220: second part
211: event server unit 212: camera meta server unit
213: EPG server unit 214: AD meta server unit
215: Multi-video meta-server unit 216: AR server unit
217: audio meta server unit 221: lobby management server unit
222: Room management server unit 223: User management server unit
224: communication management server unit

Claims (10)

In a virtual reality system including a client system and a server system,
The client system comprises:
An infant navigator unit for controlling the movement of the cursor according to a line of sight of a viewer and determining whether to select according to a set time interval; And
And a video manager and a switch unit for managing and playing a plurality of two-dimensional images according to an instruction of the infant's navigator unit, and switching a video source to be played. The method according to claim 1,
The client system comprises:
A virtual three-dimensional room unit capable of drawing and maintaining a virtual three-dimensional space; And
A social meta bus unit for displaying an icon and status information of a friend on the social network and allowing the friend to call or invite,
And the friend invited by the social metaverse unit can be displayed in an avatar format in the virtual three-dimensional space. The method according to claim 1,
The virtual three-
And a virtual sky box can be created and the space of the sky box can be filled up. The method according to claim 1,
The client system comprises:
Further comprising a dashboard unit that draws an area for displaying the metadata, receives information on each area from the server system, and displays information corresponding to the area, system. The method according to claim 1,
The client system comprises:
Further comprising: a virtual space audio unit capable of outputting virtual audio by adding spatial information of an image currently being played. The method according to claim 1,
The client system comprises:
A video player for displaying video information received from the server system; And
An HMD renderer portion for rendering video, metadata and CG objects on the head mount display,
Wherein the HMD renderer unit is capable of simultaneously displaying a 180 degree image and a 360 degree image on the head mount display. The method according to claim 6,
The HMD renderer unit,
When the cursor stays at a predetermined position for a predetermined time or longer,
A 360 degree image can be switched to a 180 degree image, or a 180 degree image can be switched to a 360 degree image. The method according to claim 1,
The server system comprises:
And an audio meta server unit for managing virtual audio information according to spatial information. The method according to claim 1,
The server system comprises:
A room management server unit for managing virtual room information; And
And an AD metadata server unit for managing metadata related to the advertisement. The method according to claim 1,
The server system comprises:
And a camera meta server unit for managing camera information for acquiring video information,
The camera information includes:
A camera model, a lens, a resolution, and photographing position information.

Images