KR102082670B1 - Method for providing virtual reality userinterface for viewing broadcastand apparatus using the same - Google Patents

Abstract

According to an embodiment of the present invention, provided are a method for providing a user interface for watching a virtual reality broadcast and an apparatus thereof. The method for providing a user interface for watching a virtual reality broadcast comprises the steps of: receiving first virtual reality image data representing a virtual personal space implemented in virtual reality to watch a real-time streaming broadcast of a broadcast provider from a broadcast management server; displaying the received first virtual reality image data through a display; and displaying a virtual reality effect object corresponding to a generated event with a predetermined condition in the virtual personal space when the event occurs in relation to the real-time streaming broadcast of the broadcast provider.

Description

METHOD FOR PROVIDING VIRTUAL REALITY USERINTERFACE FOR VIEWING BROADCASTAND APPARATUS USING THE SAME}

The present invention relates to a method and apparatus for providing a virtual reality user interface for broadcasting viewing.

In general, virtual reality (VR) refers to a specific virtual environment or situation that allows a user to experience a spatial and temporal experience similar to reality. The user may be immersed in the virtual reality using the device providing the virtual reality, or interact with the object implemented in the virtual reality through the operation or command of the device.

Recently, as the functions of various information communication devices including the Internet of Things have developed, virtual reality related technologies have been commercialized, and various popular HMD (Head Mounted Display) equipments have been distributed for this purpose. .

Furthermore, as the spread of the Internet spreads rapidly, the number of broadcast viewers who watch Internet broadcasts to transmit broadcast image data through the Internet by combining communication and broadcasting is rapidly increasing, thus combining virtual reality and Internet broadcasting. The supply and demand of content is also increasing.

Accordingly, there is a need for a method of providing a user with a variety of broadcast viewing experiences by providing virtual reality broadcast contents in which various types of virtual reality and internet broadcasting are combined.

An object of the present invention is to provide a method and apparatus for providing a virtual reality user interface for broadcasting.

Specifically, the present invention is to provide a method and apparatus for providing a virtual reality user interface implemented to allow a user to experience more diverse for watching a broadcast.

The objects of the present invention are not limited to the above-mentioned objects, and other objects that are not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above problems, a method and apparatus for providing a user interface for viewing a virtual reality broadcast according to an embodiment of the present invention are provided. A method for providing a user interface for viewing a virtual reality broadcast includes: receiving first virtual reality image data representing a virtual private space implemented as a virtual reality to watch a live streaming broadcast of a broadcast provider from a broadcast management server; Displaying the received first virtual reality image data on a display; And displaying a virtual reality effect object corresponding to the generated event in the virtual personal space when an event having a predetermined condition occurs in relation to the live streaming of the broadcast provider.

An apparatus for providing a user interface for viewing a virtual reality broadcast includes a communication interface, a display, and a processor connected to the communication interface and the display, wherein the processor is configured to broadcast live streaming of a broadcast provider from a broadcast management server through the communication interface. Receiving first virtual reality image data representing a virtual private space implemented in virtual reality to watch the video, and displays the received first virtual reality image data on the display, and related to the live streaming broadcast of the broadcast provider When an event of a preset condition is generated, the virtual reality effect object corresponding to the generated event is displayed in the virtual personal space.

Details of other embodiments are included in the detailed description and drawings.

The present invention enables a user to experience a variety of spatial and temporal experiences while watching a broadcast by watching a real time streaming broadcast image provided by a broadcast provider through a user's virtual personal space implemented in virtual reality.

The effects according to the present invention are not limited by the contents exemplified above, and more various effects are included in the present specification.

1 and 2 are schematic diagrams for describing a virtual reality broadcasting system according to an embodiment of the present invention.
3 and 4 are schematic views for explaining a user terminal according to an embodiment of the present invention.
5 is a schematic diagram illustrating a broadcast providing apparatus according to an embodiment of the present invention.
6 is a schematic diagram illustrating a broadcast management server according to an embodiment of the present invention.
7 is a schematic flowchart illustrating a method of providing a user interface for watching a virtual reality broadcast in a user terminal according to an embodiment of the present invention.
8 is an exemplary diagram for describing first virtual reality image data representing a virtual personal space of a user according to an embodiment of the present invention.
9A and 9B are exemplary diagrams for describing graphic screens of a main interface area according to an exemplary embodiment of the present invention.
10A, 10B, 11A, 11B, 11C, 11D, 11E, 11F, 11G, 12, 13, and 14 illustrate interface screens of a sub interface area according to an embodiment of the present invention. Illustrative diagrams for explanation.
15 to 17 are exemplary diagrams for describing second virtual reality image data representing a virtual private space of a BJ according to an embodiment of the present invention.
18 is an exemplary diagram for describing first virtual reality image data representing a virtual personal space of a user according to an embodiment of the present invention.
19 is an exemplary diagram for describing an interface screen related to a friend finder icon in a sub-interface area according to an embodiment of the present invention.
20 is an exemplary diagram for describing first virtual reality image data representing a virtual personal space of a user according to an embodiment of the present invention.

Advantages and features of the present invention, and methods for achieving them will be apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, only the present embodiments to make the disclosure of the present invention complete, and common knowledge in the art to which the present invention pertains. It is provided to fully inform the person having the scope of the invention, which is defined only by the scope of the claims.

Although the first, second, etc. are used to describe various components, these components are of course not limited by these terms. These terms are only used to distinguish one component from another. Therefore, of course, the first component mentioned below may be a second component within the technical spirit of the present invention.

Like reference numerals refer to like elements throughout.

Each of the features of the various embodiments of the present invention may be combined or combined with each other, partly or wholly, and various technically interlocking and driving are possible as one skilled in the art can fully understand, and each of the embodiments may be independently implemented with respect to each other. It may be possible to implement the association together.

In the present specification, a broadcasting jockey (BJ) may be a broadcast content provider including all users who perform Internet broadcasting. For example, the BJ may be a broadcast streamer.

In this specification, the virtual reality device may include all devices configured to provide a virtual reality image. For example, the virtual reality device may be a head mounted display (HMD) device that contacts / wears on a part of a user's body and outputs a virtual reality image. For example, the virtual reality device may include an image output device that outputs a virtual reality image when the display is not provided.

In the present specification, an operation of displaying at least one of various graphic screens, interface screens, and graphic objects in a virtual reality space (for example, a virtual private space) may be performed by the HMD device in various virtual screens in the virtual reality space and the virtual reality space. For example, at least one of the interface screens and the graphic objects may be displayed on the display.

In this specification, the operation of the user visiting the virtual personal space of another user may be an operation in which the HMD device displays the virtual reality image represented or implemented as a user moves to the virtual personal space of another user through a display.

In the present specification, the virtual reality image data may be an image obtained by rendering horizontal and vertical 360 degree display line directions with respect to a user in 3D graphics. In addition, the virtual reality data renders only a partial area around the user's gaze, and when the user's gaze changes, the screen area corresponding to the changed gaze is predicted through analysis of the surrounding area, and the changed gaze of the user is based on the predicted information. It may be an image obtained by rendering a screen area corresponding to. Such virtual reality image data may be implemented in various manners as described above. In various embodiments, the virtual reality image data may be an image of a first person view or a third person view.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 and 2 are schematic diagrams for describing a virtual reality broadcasting system according to an embodiment of the present invention.

Referring to FIG. 1, a virtual reality broadcasting system is connected to a first HMD device 100 and a first HMD device 100 wired / wireless that a broadcast viewer can mount for viewing a broadcast and transmit streaming broadcast data to the first HMD. The electronic device 200 transmitting to the device 100, the broadcast providing device 300 of the BJ providing streaming broadcast, the photographing device 400, the second HMD device 410 on which the BJ is mounted, and the broadcast image data of the BJ. It may include a broadcast management server 500 to manage the. Here, the virtual reality broadcasting system provides a user's virtual personal space implemented in virtual reality to watch BJ's real-time streaming broadcast, so that the user can use BJ's live streaming broadcast or video on demand (VOD) through the virtual personal space. ) Or a virtual space for viewing a personal broadcast such as arranging a virtual reality object (VR object) related to broadcast content in a user's virtual personal space and decorating the same.

The first HMD device 100 may be a virtual experience device that is mounted on the user's head and may provide a virtual reality image to the user so that the user may have a spatial and temporal experience similar to the reality.

In detail, the first HMD device 100 is formed to be worn on a user's head, and is implemented in a form in which a virtual reality image is processed inside the first HMD device 100 or as shown in FIG. 1B. An image output apparatus 200 for providing a virtual reality image may be mounted on a portion of the apparatus 100, and the image output apparatus 200 may be implemented to perform image processing for the virtual reality in the mounted image output apparatus 200.

When the first HMD device 100 includes a display, one surface of the display may be disposed to face the user's face so that the user may check the virtual reality image when the user wears the first HMD device 100.

One side of the first HMD device 100 may be formed with at least one sensor (not shown) for acquiring the gaze data of the user. For example, the at least one sensor may include a gaze tracking sensor that tracks the gaze of the user. In various embodiments, at least a gaze tracking sensor is formed at a position capable of capturing the user's eyes or face or a position in contact with the user's skin, and captures the user's eyes or face when the user wears the first HMD device 100. In addition, the gaze data of the user may be acquired by analyzing the photographed image.

The first HMD device 100 may display the virtual reality image corresponding to the viewpoint of the user through the display based on the obtained gaze data. For example, when the user wearing the first HMD device 100 moves, the first HMD device 100 detects a change in gaze according to the movement of the user through the gaze tracking sensor, and the gaze of the user is changed according to the gaze change. The virtual reality image corresponding to the viewing direction may be displayed on the display. In various embodiments, the user's viewpoint may be a first person view or a third person view.

The first HMD device 100 may be connected to an external input device (eg, a controller) that can be operated, and the user may control the user interface in the virtual reality image by manipulating the input device connected to the first HMD device 100. have.

In various embodiments, the first HMD device 100 may include a camera module (not shown) for capturing an image according to the user's gaze direction on the other side. In detail, the camera module may capture a front image of the user according to the user's gaze direction. For example, the captured image may be an image of an input device operable in front of the user and a user's hand operating the same. The first HMD device 100 renders the user's hand in the captured front image and displays the user's hand on the virtual reality image or converts a portion of the virtual reality image screen to a see-through area so that the user's hand is displayed. You may. In this way, the user may easily use an input device such as a keyboard or a mouse located in front of the user even when the first HMD device 100 is worn.

The first HMD device 100 may display, via a display, a first virtual reality image representing a virtual personal space of a user implemented as a virtual reality for viewing a broadcast. The virtual private space may include a main interface area for broadcasting a specific BJ and a sub interface area related to various functions related to broadcasting viewing. For example, the first virtual reality image may be an image obtained by combining a first layer on which a background area of a virtual personal space is displayed, a second layer on which a main interface area is displayed, and a third layer on which a sub interface area is displayed.

The main interface area of the user's virtual personal space may include a first graphic screen for playing a live streaming broadcast of BJ and a second graphic screen showing a live chat between broadcast viewers simultaneously watching the live streaming broadcast. In various embodiments, the main interface area may further include a third graphic screen. In various embodiments, a plurality of first graphic screens may be implemented. Specifically, the first HMD device 100 is connected to the electronic device 200 by wireless or wired to receive chat data between broadcast viewers who are watching the live streaming video of BJ and the live streaming broadcast from the electronic device 200. The received live streaming broadcast video may be displayed on the first graphic screen, and the received chat data may be displayed on the second graphic screen. The second graphic screen may include a chat display area indicating a chat content of broadcast viewers and a chat input area for a user's chat input.

The sub-interface region of the user's virtual private space may include a first interface screen representing a video list or a VOD video list related to real-time streaming broadcasting of at least one BJ, and a first view related to at least one function for broadcasting viewing or a personal virtual space. 2 may include an interface screen. The first interface screen may include a graphic object such as a thumbnail image or a thumbnail video related to at least one BJ's real-time streaming broadcast image selected by a user or registered as a favorite. The second interface screen may include a graphic object such as a thumbnail image or a thumbnail video related to the VOD image of at least one BJ selected by a user or registered as a favorite. The third interface screen may include a function for setting a graphic screen for playing broadcast video, a function for setting a real-time broadcast list, a function for setting a VOD video list, and a sponsoring function related to a specific BJ (eg, a fee for a BJ). Items, etc.) or menu objects related to various functions for setting up a user's virtual personal space. For example, various functions for setting a virtual personal space may include setting a number, size, skin, type or resolution of a first graphic screen, and size or on / off of the second graphic screen. It may include a function for setting a main interface area, such as a function for setting. The various functions include a function for configuring a first interface screen, a function for selecting a BJ for watching a live streaming broadcast, and a function for configuring a live streaming video list, a function for selecting a BJ for watching a VOD video, and a VOD. The apparatus may further include a function for configuring a second interface screen, such as a function of configuring a video list. The various functions may include purchasing or presenting a paid item for viewing a broadcast, purchasing or presenting an avatar or a paid item for decorating a virtual personal space, and placing a purchased or received paid item in a virtual personal space, decorating an avatar, etc. Similarly, the third interface screen may further include a function.

In various embodiments, the sub-interface area may include a fourth interface screen representing a community space (eg, a bulletin board, etc.) related to watching a broadcast, a fifth interface screen representing a purchase or gift of paid items, or at least one favorite BJ. The display device may further include a sixth interface screen indicating a list of the information about the list. For example, the fourth interface screen may display an announcement or a bulletin board of the user's virtual personal space when the user is located in his or her virtual personal space, and the user may visit the virtual personal space of the BJ. It can indicate the announcement of the space or bulletin board. In various embodiments, the fourth interface screen may indicate a community space selected by the user.

The first HMD device 100 may display a virtual reality effect object corresponding to the generated event in a virtual personal space when an event having a predetermined condition occurs in relation to the real time streaming broadcast. The virtual reality effect object is a three-dimensional space rendering effect, may further include a voice effect, such as background music or sound effects. For example, an event with a predetermined condition may include a user purchasing a paid item for sponsoring a BJ, a user with a high contribution contribution indicating a degree of using the BJ support function, or an advertisement content set by the BJ. It may include an event using. If the first HMD device 100 selects a menu icon corresponding to a sponsoring function for purchasing a paid item for BJ through a third interface screen, the first HMD device 100 displays an item purchase interface screen for purchasing a paid item, and purchases an item. When a particular paid item is purchased through the interface screen, the virtual reality effect object implemented in virtual reality may be displayed in the virtual personal space in response to the purchased paid item. According to various embodiments, even when another user purchases a paid item, the first HMD device 100 may display a virtual reality effect object corresponding to the paid item in the user's virtual personal space. Such virtual reality effect objects may be displayed not only in the user's virtual personal space but also in the virtual personal space of viewers watching the broadcast of BJ.

According to various embodiments of the present disclosure, the first HMD device 100 may provide a user with a visitor as a visitor in response to a request to visit a virtual personal space (for example, a virtual personal space) of another user (for example, BJ) in addition to the user's personal virtual space. Second virtual reality image data implemented to experience a virtual personal space may be received from the broadcast management server 400, and the received second virtual reality image data may be displayed on a display. For example, the second virtual reality image data may be stored in the first layer in which the background area of the virtual private space set by the BJ is displayed, in the second layer in which the main interface area of the virtual private space of the BJ is displayed, and in the virtual private space of the BJ. The third layer displaying the related sub-interface area may be synthesized image data. The sub interface area may include interface screens for various functions that may be provided to the user in relation to the virtual personal space of the BJ.

The second virtual reality image data may include a main interface region for reproducing the broadcast image set by the BJ and a sub interface region related to various functions of the personal broadcasting space of the BJ visited by the user. The main interface area of the virtual private space of the BJ visited by the user is for playing the broadcast video set by the BJ (for example, BJ's real-time streaming broadcast, broadcast replay video, highlight video, or VOD video produced by the viewer). It may include a first graphic screen and a second graphic screen indicating a chat associated with the broadcast video. If the broadcast video is real-time streaming broadcast video data, real-time chat data is displayed through the second graphic screen, and if the broadcast video is VOD video data, the chat data stored corresponding to the VOD video may be displayed through the second graphic screen. have. In various embodiments, the main interface area may include a third graphic screen representing a community space (eg, an announcement, a bulletin board, etc.) related to the broadcast of BJ, a broadcast replay image of BJ, a highlight image, a VOD image produced by the viewer, and the like. The apparatus may further include a fourth graphic screen representing a list of images. For example, the community space of the third graphic screen may be a bulletin board of the virtual personal space of the user selected by the user or a bulletin board of the virtual personal space of the BJ favorite by the user.

The sub-interface area of the virtual private space of the BJ visited by the user may include a first interface screen representing a video list related to the BJ's real-time streaming broadcast, a second interface screen representing a VOD video list of the BJ set by the user, and It may include a third interface screen related to various functions of the virtual private space of the BJ. For example, various functions of BJ's virtual personal space may include BJ-related sponsorship functions (such as purchasing paid items on BJ's broadcasts and presenting paid items to decorate BJ's avatar or virtual personal space). have. According to various embodiments, the second virtual reality image data may represent a virtual personal space set for each user class by the BJ according to the degree (eg, sponsorship contribution) of the user using a sponsoring function such as a paid item purchase or a gift related to the BJ. . When the user visits the virtual personal space of another user, the sub interface area is moved together in response to the user, and some functions of the sub interface area are deactivated in the virtual personal space of the other user, or some interface screen in the sub interface area is It may be deactivated or not displayed.

The electronic device 200 may transmit broadcast streaming data received from the broadcast providing device 300 or the broadcast management server 500 to the first HMD device 100 connected by wire / wireless. The electronic device 200 transmits various requests from the first HMD device 100 to the broadcast providing device 300 or the broadcast management server 500, and sends a response to the various requests to the broadcast providing device 300 or the broadcast. Received from the management server 500 may be delivered to the first HMD device 100.

The electronic device 200 receives from the broadcast providing device 300 or the broadcast management server 500 first virtual reality image data indicating a virtual personal space of a user implemented as a virtual reality for viewing a broadcast, and receives the received first agent. One virtual reality image data may be transmitted to the first HMD device 100.

According to various embodiments of the present disclosure, the electronic device 200 transmits a request to the broadcast management server 500 according to a request for visiting another user's virtual personal space, and stores the other user's virtual personal space from the broadcast management server 500. After receiving the second virtual reality image data implemented to be experienced as a visitor, the second virtual reality image data may be transferred to the first HMD device 100.

According to various embodiments of the present disclosure, the electronic device 200 may include a display to display broadcast data (eg, streaming data or VOD data) received from the broadcast providing device 300 or the broadcast management server 500 on the display. As such, when a request for viewing a virtual reality image is received from the first HMD device 100 while displaying broadcast data on the screen, the electronic device 200 transmits the request to the broadcast management server 500, and the request for In response, the broadcast management server 500 may receive the first virtual reality image data. The electronic device 200 may transmit the first virtual reality image data received from the broadcast management server 500 to the first HMD device 100. For example, the request for viewing the virtual reality image may be a detection signal indicating that the user has mounted the first HMD device 100. When the detection signal is received from the first HMD device 100, the electronic device 200 transmits a request signal to the broadcast management server 500, and receives the first virtual reality image data from the broadcast management server 500. The first virtual reality image data may be transferred to the first HMD device 100. In this case, the electronic device 200 may stop displaying the broadcast streaming data through the display.

According to various embodiments of the present disclosure, the electronic device 200 stores a program or an application capable of generating virtual reality image data (eg, first or second virtual reality image data) for viewing a broadcast, and stores the first HMD device 100. In response to a request from), the program or the application may be executed to generate virtual reality image data, and then the generated virtual reality image data may be transmitted to the first HMD device 100.

The broadcast providing device 300 may transmit broadcast data (eg, real-time streaming data) to at least one user terminal (eg, the first HMD device 100 or the electronic device) connected to the broadcast providing device 300 at the request of a BJ. 200) or a broadcast management server 500 may be transmitted. For example, the broadcast providing device 300 records an image signal input through a photographing device (eg, an image camera) 400 using a broadcast platform (eg, a broadcast program or an application) for broadcasting, and records the recorded signal. The video signal may be transmitted to at least one user terminal or the broadcast management server 500 in real time streaming.

In various embodiments, the broadcast providing device 300 is connected to the second HMD device 410 or an image output device (not shown) included in the second HMD device 410, and the BJ connects the second HMD device 410. When worn, the third virtual reality image data indicating the virtual personal space of the BJ, which is implemented as virtual reality, is received from the broadcast management server 500 and is transmitted to the second HMD device 410 or the second HMD device 410 worn by the BJ. It can be delivered to the provided image output device.

The second HMD device 410 worn by the BJ or the image output device included in the second HMD device 410 may display the third virtual reality image data received from the broadcast providing device 300 on the display. The virtual private space of the BJ may include a main interface area for playing the broadcast video set by the BJ and a sub interface area related to various functions for setting the virtual private space of the BJ. For example, the third virtual reality image data includes a first layer in which a background area of the virtual private space of BJ is displayed, a second layer in which a main interface area of the virtual private space of BJ is displayed, and a virtual private space of BJ. The third layer displaying the sub interface area related to various functions may be synthesized image data.

The main interface area of the virtual private space of the BJ may include a first graphic screen for reproducing a broadcast image such as a broadcast replay image, a highlight image, and a VOD image produced by the viewer, and chat data corresponding to each of the corresponding images. It may include a second graphic screen indicating the. In various embodiments, the main interface area of the virtual private space of the BJ may be a third graphic screen representing a community space (eg, an announcement, a bulletin board, etc.) related to the broadcasting of the BJ, or a broadcast replay image, a highlight image, or a viewer of the BJ. The display apparatus may further include a fourth graphic screen indicating a list of images such as a produced VOD image. In various embodiments, a plurality of first graphic screens may be implemented such that at least two or more of a real-time streaming broadcast image, a broadcast replay image, a broadcast highlight image, and a VOD produced by a viewer may be reproduced.

The sub interface area of the virtual private space of the BJ includes a first interface screen for setting the playback of a live streaming broadcast image, a broadcast replay image, a broadcast highlight image, a viewer VOD image, or a live streaming broadcast region, and a virtual private space of the BJ. It may include a second interface screen related to various functions for setting. For example, various functions for setting the virtual private space of the BJ may include a function for setting the number, size of the first graphic screen, the type or resolution of the playback image, and the size or on / off of the second graphic screen. Function to set the size, layout, structure, etc. of the third graphic screen, the size of the fourth graphic screen, the number of video lists, the function of setting the list display method, and the avatar or personal broadcasting space. It may include a function such as a purchase or a gift of a paid item for decorating and a function of arranging a purchased or received paid item in a virtual personal space.

The broadcast management server 500 receives broadcast data (eg, real-time streaming data) of BJ from the broadcast providing device 300, and transmits the received broadcast data to at least one user terminal (eg, connected to the broadcast management server 500). : May be transmitted to the first HMD device 100 or the electronic device 200.

The broadcast management server 500 represents a first virtual reality representing a virtual personal space of a user implemented as a virtual reality to watch a live streaming broadcast received from the broadcast providing apparatus 300 in response to a request from at least one user terminal. The image data may be generated, and the generated first virtual reality image data may be delivered to at least one user terminal. For example, the broadcast management server 500 may include a first layer on which a background area of a user's virtual personal space is displayed, a second layer on which a main interface area for broadcasting is displayed, and sub interfaces related to various functions related to broadcasting viewing. The first virtual reality image data may be generated by synthesizing the third layer displaying the region.

According to various embodiments, the broadcast management server 500 may transmit second virtual reality image data indicating the virtual private space of the BJ to at least one user terminal according to a request from at least one user terminal. The second virtual reality image data may include various interface regions configured to allow a user to experience the virtual personal space of BJ as a visitor. For example, the second virtual reality image data may be related to the first layer in which the background area of the virtual private space of BJ is displayed, the second layer in which the main interface area of the virtual private space of BJ is displayed, and the virtual private space of BJ. The third layer displaying the sub interface area related to various functions that may be provided to the user may be synthesized image data.

In various embodiments, the virtual personal space of the BJ may be represented differently for each user class. In more detail, the broadcast management server 500 may classify the users in a descending order (eg, the third level) from the highest degree of sponsorship (eg, the first level) according to the contribution contribution of the users. The broadcast management server 500 may provide a setting interface screen for the BJ to set different virtual personal spaces for each user class. For example, when the first user visits the BJ's virtual personal space, the broadcast management server 500 checks the sponsorship contribution level of the first user, and when the first user is classified as '1st grade', the broadcast management server 500 determines the 'BJ' by the BJ. The second virtual reality image data indicating the virtual personal space set by the BJ may be provided to the users classified as 'first grade'. The number of BJ's virtual personal spaces provided to the first class of users is the number, size and resolution of BJ's virtual personal spaces and graphic screens, the structure of the virtual personal spaces, background skins, and virtual reality. Object placement, type, etc. can be different.

In various embodiments, the virtual private space of the BJ may include a virtual private space corresponding to a first chat room (eg, a main room) in which a first number of broadcast viewers are participating, and a second chat room (eg, a relay) in which a second number of broadcast viewers are participating. A virtual private space corresponding to the room) may be provided. For example, the broadcast management server 500 sets the number of people who can visit the virtual private space of BJ as the first number of people, and when the viewers of the first number of people visit the virtual private space of BJ, the broadcasting of the first number of people is performed. Provide the viewer with the second virtual reality video data representing the first virtual private space of BJ set by BJ and other broadcast viewers with the second virtual reality video data representing the second virtual private space of BJ set by BJ. Can provide. The first virtual private space of the BJ and the second virtual private space of the BJ may be set identical to each other by the BJ, but may be set differently.

According to various embodiments of the present disclosure, the broadcast management server 500 generates third virtual reality image data indicating a virtual private space of BJ implemented as a virtual reality according to a request from the broadcast providing device 300, and generates the generated third virtual reality. The image data may be transmitted to the broadcast providing device 300. For example, the broadcast management server 500 relates to a first layer in which a background area of a virtual private space of BJ is displayed, a second layer in which a main interface area of the virtual private space of BJ is displayed, and a virtual private space of BJ. Accordingly, the third virtual reality image data may be generated by synthesizing a third layer in which sub interface areas related to various functions that may be provided to the BJ are displayed. In various embodiments, the virtual private space of the BJ provided to the user and the virtual private space provided to the BJ may be different.

In various embodiments, when the virtual reality image data is the first-person view, the user's appearance in the virtual reality may be implemented in the form of an avatar seen in the first-person view, or may be provided in a form in which the actual shape of the user is rendered.

According to various embodiments of the present disclosure, when the virtual reality image data is the third person view, the user's appearance in the virtual reality may be implemented as a 3D graphic object in the form of an avatar. In this case, the avatar of the user may be provided to the user as a predetermined avatar, or any one of various types of avatars may be selected by the user. In various embodiments, the avatar of the user may be provided in the form of an avatar desired by the user such as selecting a shape of a face, eyes, nose, mouth, ear, etc., selecting a hair shape, a hair color, or the like, or selecting a shape of a body part. Can be. In various embodiments, the avatar of the user may be provided in the form of rendering the actual appearance of the user. For example, it may be provided in the form of rendering the user's face or the entire body of the user.

Next, referring to FIG. 2, the virtual reality broadcasting system is mounted on the first HMD device 100, the broadcast providing device 300, the broadcast management device 500, and the first HMD device 100, and the broadcast management device ( And an image output device 600 for outputting the virtual reality image data received through the 500.

The image output device 600 is a device that provides a user interface for a broadcast viewer (ie, a user) to watch a broadcast of BJ online. The image output device 600 may be a smartphone, a tablet PC, or the like in which an application or a program for viewing a broadcast is installed. . The application or the program may connect the broadcast management server 500 and at least one user terminal (eg, the first HMD device 100 or the image output device 600) to each other. Broadcast viewers can communicate with each other through a communication area such as a chat window or a bulletin board while watching a broadcast with other broadcast viewers online using an application.

The image output apparatus 600 may display, via a display of the image output apparatus 600, first virtual reality image data indicating a virtual private space of a broadcast viewer implemented as a virtual reality for broadcast viewing at the request of the broadcast viewer. Can be. In detail, the image output device 600 transmits a request to the broadcast management server 500 when a request for viewing a virtual reality broadcast is received from a broadcast viewer through an application, and the first virtual reality image from the broadcast management server 500. Data can be received. The image output device 600 may display the received first virtual reality image data through a display.

According to various embodiments of the present disclosure, the image output apparatus 600 may display second virtual reality data including various interface regions implemented to experience the virtual private space of BJ according to a request of a broadcast viewer. Can be displayed through In detail, the video output device 600 transmits a request to the broadcast management server 500 when a request for a visit to the BJ's virtual personal space is received from a broadcast viewer through an application, and the second virtual image is transmitted from the broadcast management server 500. Real image data can be received. For example, the second virtual reality image data may represent a virtual personal space set by the BJ according to the sponsorship contribution grade of the broadcast viewer. The image output device 600 may display the received second virtual reality image data through a display.

The image output device 600 receives the gaze data of the user from the first HMD device 100 and based on the received gaze data, virtual reality image data corresponding to the viewpoint of the user (eg, the first or second virtual). Real image data) can be displayed on the display.

In various embodiments, the image output apparatus 600 may include a camera module for capturing an image according to a user's gaze direction. For example, the camera module may be provided on the rear side of the image output apparatus 600 (for example, the opposite side of the front side on which the display is disposed) to capture the front image of the user. The captured front image may be an image of a user's hand operating an input device and an input device located in front of the user. The image output apparatus 600 may display the captured front image in a pop-up form on the virtual reality image screen or convert a partial region of the virtual reality image screen to a see-through region.

3 and 4 are schematic views for explaining a user terminal according to an embodiment of the present invention.

Referring to FIG. 3, the user terminal may be a first HMD device 100 having a display. The first HMD device 100 includes a communication interface 110, a display 120, a memory 130, a sensor 140, and a processor 150. In various embodiments, the first HMD device 100 may be connected to the input device 550 by wire / wireless communication.

The communication interface 110 connects the first HMD device 100 to communicate with an external device. The communication interface 110 may connect with the electronic device 200 using wired / wireless communication. The communication interface 110 may transmit a request for viewing the real time broadcast of the BJ to the electronic device 200 and receive the real time streaming broadcast image data of the BJ from the electronic device 200. The communication interface 110 transmits a request for first virtual reality image data indicating a virtual personal space implemented in virtual reality to the electronic device 200, and from the electronic device 200, the first virtual reality image for viewing a broadcast. Data can be received. The communication interface 110 may transmit a visit request for another user's virtual personal space to the electronic device 200 and receive second virtual reality image data indicating the virtual personal space of the BJ from the electronic device 200. have. In various embodiments, the communication interface 110 may transfer the gaze data of the user acquired through the sensor 140 to the electronic device 200.

The display 120 may display various contents (eg, text, images, videos, icons, banners, or symbols) to the user. In detail, the display 120 may display broadcast data (eg, a real time streaming image or a VOD image), first virtual reality image data, or second virtual reality image data received through the communication interface 110.

The memory 130 may store various data used to display virtual reality image data. In detail, the memory 130 may store the first or second virtual reality image data and may store an application, a program, a widget, and the like used to display the first or second virtual reality image data. In addition, the memory 130 may store the gaze data acquired through the sensor 140.

The sensor 140 may include a gaze tracking sensor for tracking the gaze of the user, and may detect a gaze of the user through the gaze tracking sensor and output a detection signal. For example, the gaze tracking sensor may include a camera sensor for capturing an eye or a face of a user, and may output gaze data indicating a position coordinate of the gaze of the user by analyzing an image captured by the camera sensor. In various embodiments, a variety of methods may be used to track the user's eyes, and in addition to image analysis, a contact lens method (reflected light of a mirror built-in contact lens or a gaze tracking method using a magnetic field of a coil built-in contact lens) or a sensor may be attached. The eye of the user may be tracked using a method (a line of sight tracking method using an electric field according to eye movement by attaching a sensor around the eye).

The processor 150 is operatively connected with the communication interface 110, the display 120, the memory 130, and the sensor 140, and the first virtual reality image data representing the user's virtual personal space or the virtual individual of the BJ. Various commands for displaying second virtual reality image data representing a space may be performed.

In detail, the processor 150 executes an application for viewing a broadcast according to an application execution request for viewing the broadcast of the user, and receives a request for viewing a virtual reality broadcast, and receives the electronic device 200 through the communication interface 110. It may transmit a request for viewing the virtual reality broadcast. The processor 150 may receive first virtual reality image data representing a virtual personal space of a user implemented as virtual reality from the electronic device 200 and view it on the display 120.

The user's virtual personal space may include a main interface area for broadcasting a specific BJ that the user wants to watch and a sub interface area related to various functions related to broadcasting viewing. The main interface area may include a first graphic screen for reproducing a live streaming broadcast of BJ and a second graphic screen for real time chat between broadcast viewers simultaneously watching the live streaming broadcast. According to various embodiments of the present disclosure, the main interface area may further include a third graphic screen indicating a community space (for example, a bulletin board) related to watching a broadcast. For example, the third graphic screen may be a bulletin board of the BJ's virtual private space related to BJ's viewing of the broadcast. In various embodiments, the first graphic screen may include a plurality of playback screens. For example, the plurality of playback screens may include one main playback screen and one or more sub playback screens. The processor 150 may play first real time streaming broadcast data provided by a specific BJ through a main playback screen, and play second real time streaming broadcast data provided by a specific BJ through at least one sub playback screen. The sub interface area may include a first interface screen indicating a video list related to the live streaming broadcast of at least one BJ, a second interface screen indicating a VOD video list of a specific BJ, and at least one function for broadcasting or a personal virtual space. It may include a third interface screen related to.

The processor 150 may display a virtual reality effect object corresponding to the generated event in a virtual personal space when an event having a predetermined condition is generated in relation to the real time streaming broadcast. The event of the preset condition is an event in which the user purchases a paid item related to BJ, an event in which a user with high contribution to BJ participates in a chat, or after the user uses the advertising content set by the BJ or uses the advertising content. May include an event participating in a chat of a live streaming broadcast. For example, when a pay item is purchased by the user, the processor 150 may display a virtual reality effect object corresponding to the pay item in the virtual personal space. The virtual reality effect object may include a 3D graphic object and voice data. In addition, when the user participates in the chat, if the user's sponsorship contribution to the BJ corresponds to a specific sponsorship contribution, the virtual reality effect object corresponding to the specific sponsorship contribution may be displayed in the virtual personal space. In addition, when a user uses advertising content related to BJ's broadcast while watching a live stream (eg, selects an advertisement banner or an image or participates in an advertisement related event), the processor 150 virtualizes a corresponding virtual reality effect object. Can be displayed in private space. The above-described virtual reality effect objects may be simultaneously displayed in the virtual personal space of the plurality of broadcast viewers watching the broadcast of the BJ or the virtual private space of the BJ visited by the plurality of broadcast viewers.

In various embodiments, the processor 150 may detect a change in the line of sight according to the movement of the user through the sensor 140 and display at least a portion of the virtual reality image data corresponding to the direction of the line of sight of the user through the display. have. According to various embodiments of the present disclosure, the processor 150 receives virtual reality image data corresponding to various gaze directions of the user from the electronic device 200 or receives some virtual reality image data corresponding to the gaze of the user. If changed, other virtual reality image data corresponding to the changed gaze may be received from the electronic device 200.

According to various embodiments of the present disclosure, when a request for chatting input is made through the input device 550, the processor 150 may display a virtual keyboard area for chatting input at a specific position of the virtual reality video screen. The virtual keyboard area may include a plurality of key areas for inputting one or more letters and numbers. When the processor 150 detects a selection related to a specific key area among the virtual keyboard areas through the input device 550, the processor 150 inputs a character or number corresponding to the selected key area through the chat input area as an input value. Alternatively, numbers and the like can be displayed in the chat display area.

In various embodiments, when the HMD device 110 includes a camera module (not shown), when the input device 550 for chatting input is located in front of the user such as a keyboard, the processor 150 may be photographed through the camera module. The user's hand may be rendered on the virtual reality image by rendering the user's hand among the front images of the user, or the user's hand may be displayed by switching a part of the virtual reality image screen.

In various embodiments, the virtual personal space of the user may be implemented to allow not only the user but also other users (or other broadcast viewers) invited by the user to participate together. For example, when a user wants to watch a live streaming broadcast of a specific BJ with other users, the processor 150 may provide an interface screen for the user to transmit an invitation message to other users. When the user delivers an invitation message to other users through the interface screen, and a response message is received from the other users to approve the invitation, the processor 150 receives the user from the broadcast providing server 500 in the virtual personal space of the user. In addition, the first virtual reality image data implemented to involve other users invited by the user may be received.

In various embodiments, when there is a request for visiting the virtual private space of the BJ, the processor 150 may transmit a request for visiting the BJ's virtual private space to the electronic device 200 through the communication interface 110. The processor 150 may receive second virtual reality image data configured to allow a user to experience the virtual personal space of BJ as a visitor from the electronic device 200 and display it on the display 120. The second virtual reality image data includes a main interface region and a virtual image of the BJ for playing a broadcast image set by the BJ (for example, a live streaming image of BJ, a replay image, a highlight image, or a VOD image produced by a viewer). It may include sub-interface areas related to various functions for the personal space. The main interface region may include a first graphic screen for reproducing the broadcast video set by the BJ and a second graphic screen indicating chat data related to the broadcast video. In various embodiments, the main interface area may include a third graphic screen representing a community space (eg, an announcement, a bulletin board, etc.) related to the broadcast of BJ, a broadcast replay image of BJ, a highlight image, a VOD image produced by the viewer, and the like. The apparatus may further include a fourth graphic screen representing a list of images. The sub-interface area may include a first interface screen for setting playback of a broadcast review image, a broadcast highlight image, a viewer VOD image, or a live streaming broadcast region, and various functions provided to a user in relation to a virtual private space of the BJ. 2 may include an interface screen. For example, various functions may include a function of purchasing or presenting a paid item for decorating an avatar or a personal broadcasting space, and a function of placing a paid or received paid item in a virtual personal space.

According to various embodiments of the present disclosure, when the user requests a visit to the virtual private space of the BJ through the input device 550, the processor 150 may be different from the user class by the BJ according to the contribution contribution of the user from the electronic device 200. Second virtual reality image data representing the set virtual private space may be received. In detail, the processor 150 transmits a request to visit the virtual personal space of the BJ to the electronic device 200, and indicates a virtual personal space set by the BJ corresponding to the sponsored contribution of the user from the electronic device 200. The second virtual reality image data may be received and displayed on the display 120. For example, a virtual personal space that is set up for users with high sponsorship contributions may include the number, size, and resolution of virtual personal spaces and graphics screens that are set up for users with low sponsorship contributions, the structure of the virtual personal space, background skins, The arrangement, type, etc. of the virtual reality objects may be different.

According to various embodiments of the present disclosure, the processor 150 displays a third interface screen for purchasing at least one paid item related to broadcasting of BJ according to a user's request, and pays for the selected paid item when a specific paid item is selected by the user. You can perform the operation. When the payment for the paid item is completed, the processor 150 may display a virtual reality effect object corresponding to the paid item in the virtual personal space of the BJ that the user is visiting.

In various embodiments, when the user visits the virtual private space of the BJ and participates in a chat for watching the live streaming broadcast of the BJ, the processor 150 may respond to the specific sponsorship contribution if the user contributes to the BJ. The virtual reality effect object can be displayed in the virtual private space of the BJ that the user is visiting.

According to various embodiments of the present disclosure, the processor 150 may visit a virtual private space of BJ to watch a live streaming broadcast of BJ and participate in an advertisement event related to the broadcast of BJ. Can be displayed in the virtual private space.

The above-described virtual reality effect objects may be simultaneously displayed in the virtual personal space of the plurality of broadcast viewers watching the broadcast of the BJ or the virtual private space of the BJ visited by the plurality of broadcast viewers.

In various embodiments, the input device 550 may be connected to the first HMD device 100 through wired / wireless communication, and may include a sensor (not shown) for detecting a user's grip or detecting a user's movement. . The input device 550 may further include a button to transmit a button input signal according to a button input to the first HMD device 100. The first HMD device 100 may display a pointer corresponding to the movement of the input device 550 through the display 120.

The first HMD device 100 may move and display a pointer in a virtual personal space displayed through the display 120 according to a detection signal of a sensor received from the input device 550. In addition, the first HMD device 100 may select a specific object (eg, graphic screen, interface screen, virtual reality object, etc.) in the virtual reality space according to the location of the pointer and the button input signal received from the input device 550. have.

Next, referring to FIG. 4, when the first HMD device 100 does not have a display, the user terminal may further include an image output device 600 mounted to the first HMD device 100. The first HMD device 100 includes a communication interface 110, a memory 130, a sensor 140, and a processor 150. The image output device 600 includes a communication interface 610 and a display 620. ), A memory 630, and a processor 640.

First, in relation to the first HMD device 100, the communication interface 110 may connect with the electronic device 200 and the image output device 600 using wired / wireless communication. In addition, the communication interface 110 may transmit the gaze data of the user acquired through the sensor 140 to the image output device 600. In various embodiments, the communication interface 110 may be connected to the input device 550 using wired / wireless communication.

The memory 130 may store the gaze data acquired through the sensor 140.

As described with reference to FIG. 3, the sensor 140 may include a gaze tracking sensor that tracks the gaze of a user. For example, the gaze tracking sensor may include a camera sensor (not shown) for capturing a user's eyes or face, and may analyze gaze data captured by the camera sensor to output gaze data indicating a position coordinate of the gaze of the user. Can be.

As shown in FIG. 3, the processor 150 is operatively connected to the communication interface 110, the display 120, the memory 130, and the sensor 140, and displays the gaze data acquired through the sensor 140. It may be delivered to the output device 200. In detail, the processor 150 may obtain gaze data according to a change in the line of sight of the user through the sensor 140, and may transfer the obtained gaze data to the image output apparatus 200.

In various embodiments, when a request for input is received through the image output apparatus 600, the processor 150 connects to the input apparatus 550 using wired / wireless communication, and a specific key is input from the input apparatus 550. An input signal corresponding to an area may be received and the received input signal may be transmitted to the image output device 600.

Next, in relation to the image output device 600, the communication interface 610 may connect with the first HMD device 100 using wired or wireless communication. The communication interface 610 may be connected to the broadcast providing device 300 or the broadcast management device 500 by using wireless communication. The communication interface 610 transmits a request for the first virtual reality image data indicating the virtual personal space implemented in virtual reality to the broadcast providing device 300 or the broadcast management server 500, and the broadcast providing device. The first virtual reality image data may be received from the 300 or the broadcast management server 500. The communication interface 610 transmits a visit request for another user's virtual personal space to the broadcast providing device 300 or the broadcast management server 500, and receives another request from the broadcast providing device 300 or the broadcast management server 500. Second virtual reality image data representing the virtual private space of the user may be received. In various embodiments, the communication interface 610 may receive the user's gaze data or an input signal from the input device 550 from the first HMD device 100. In various embodiments, the communication interface 610 may receive a signal indicating that the image output device 600 is mounted on the first HMD device 100 from the first HMD device 100.

The display 620 may display various contents (eg, text, images, videos, icons, banners, or symbols) to the user. In detail, the display 520 may display the first virtual reality image data or the second virtual reality image data received through the communication interface 610.

The memory 630 may store various data used to display virtual reality image data. In detail, the memory 630 may store the first virtual reality image data or the second virtual reality image data, and may store an application, a program, a widget, and the like used to display the first or second virtual reality image data. . In addition, the memory 630 may store the gaze data received through the first HMD device 100.

The processor 640 is operatively connected with the communication interface 610, the display 620, and the memory 630, and may include first virtual reality image data representing a virtual private space of a user or a virtual private space of another user. 2 Various commands for displaying virtual reality image data may be performed.

In detail, the processor 640 executes an application for viewing a broadcast according to an application execution request according to an application execution request of a user, and receives a broadcast management device through the communication interface 610 when a request for viewing a virtual reality broadcast is received. 500, the request for viewing the virtual reality broadcast may be delivered. For example, the request for viewing the virtual reality broadcast may include a case where a signal indicating that the image output device 600 is mounted on the first HMD device 100 is received from the first HMD device 100. The processor 640 receives first virtual reality image data representing a virtual personal space of a user implemented as virtual reality from the broadcast management apparatus 500, and receives the received first virtual reality image data. It may be displayed through the display 620. The first virtual reality image data may be virtual reality image data corresponding to various gaze directions of a user or some virtual reality image data corresponding to gaze directions of a user.

According to various embodiments, the processor 640 may display an effect object corresponding to the generated event in the user's virtual personal space when an event having a predetermined condition occurs in relation to the real time streaming broadcast. The event under the preset condition may include an event in which a user watching a live streaming broadcast purchases a paid item related to a BJ, an event in which a user with high contribution to BJ participates in a chat of a BJ's live streaming broadcast, and a live streaming broadcast. The viewing user may include an event for participating in a chat of a real time streaming broadcast after using the advertisement content set by the BJ or using the advertisement content. For example, when a user who is watching a live broadcast of BJ purchases a paid item related to the live broadcast of BJ, the processor 640 may display a virtual reality effect object corresponding to the purchased pay item in the user's virtual personal space. Can be. In addition, when a user who is watching a live broadcast of BJ participates in a live broadcast chat, if the sponsor contribution of the user corresponds to a specific sponsor contribution, the processor 640 may generate a virtual reality effect object corresponding to the specific sponsor contribution. You can mark it in space. In addition, when the user who is watching the live broadcast of the BJ participates in the chat of the real time streaming broadcast after using the advertising content set by the BJ or using the advertising content, the processor 640 may detect a virtual reality effect object corresponding to the use of the advertising content. It can be displayed in the user's virtual personal space.

According to various embodiments of the present disclosure, if there is a user's request to visit the virtual private space of the BJ through the input device 550, the processor 640 may transmit the request to the broadcast management device 500 through the communication interface 610. . When the second virtual reality image data is received from the broadcast management device 500 so that the user may experience the virtual personal space of BJ as a visitor, the processor 640 may display the received second virtual reality image data on the display 620. Can be displayed through For example, the second virtual reality image data may represent a virtual personal space set differently for each user class by the BJ according to the user's contribution contribution. The second virtual reality image data includes a main interface region and a virtual image of the BJ for playing a broadcast image set by the BJ (for example, a live streaming image of BJ, a replay image, a highlight image, or a VOD image produced by a viewer). It may include sub-interface areas related to various functions that may be provided to the user in relation to the personal space. The main interface area may include a first graphic screen for playing a live streaming broadcast set by the BJ and a second graphic screen representing a chat content area and a chat input area of the live streaming broadcast when the BJ performs a live streaming broadcast. have.

According to various embodiments of the present disclosure, when an event having a predetermined condition is generated in relation to a real time streaming broadcast, the processor 640 may display a virtual reality effect object corresponding to the generated event in a personal broadcast space of a BJ visited by a user. The event of the preset condition may be an event in which a user watching a live streaming broadcast purchases a paid item about BJ, and an event in which a user whose sponsorship contribution corresponds to a specific sponsorship contribution participates in a chat of BJ's live streaming broadcast. The user watching the live streaming broadcast may use an advertisement content set by the BJ or an event of participating in a chat of the live streaming broadcast after using the advertisement content. For example, when a user who is watching a live broadcast of BJ purchases a paid item related to the live broadcast of BJ, the processor 640 may request a virtual individual of the BJ who is visiting the virtual reality effect object corresponding to the purchased paid item. You can mark it in space. In addition, when a user who is watching a live broadcast of BJ participates in a live broadcast chat, when the sponsored contribution of the user corresponds to a specific sponsored contribution, the processor 640 visits a virtual reality effect object corresponding to the specific sponsored contribution. It can be displayed in the virtual private space of the BJ being used. In addition, when the user who is watching the live broadcast of the BJ participates in the chat of the real time streaming broadcast after using the advertising content set by the BJ or using the advertising content, the processor 640 may detect a virtual reality effect object corresponding to the participation in the advertising event. It can be displayed in the virtual private space of the BJ the user is visiting. For example, the operation using the advertisement content may include selecting an advertisement banner or an advertisement image related to the broadcast of the BJ or participating in an advertisement event.

The above-described virtual reality effect objects are the virtual private space of another broadcast viewer who is also watching BJ's live streaming broadcast or the personal broadcast space when another broadcast viewer is visiting BJ's personal broadcast space to watch BJ's live streaming broadcast. Can be displayed at the same time.

5 is a schematic diagram illustrating a broadcast providing apparatus according to an embodiment of the present invention. Referring to FIG. 5, the broadcast providing device 300 may include a communication interface 310, a display 320, a memory 330, and a processor 340. The broadcast providing device 300 may be connected to a recording device 400 for BJ broadcasting or a second HMD device 410 provided with a display.

The communication interface 310 connects the broadcast providing device 300 to communicate with an external device. The communication interface 310 may connect with the electronic device 200, the broadcast management device 500, the photographing device 400, or the second HMD device 410 using wired / wireless communication. The communication interface 310 may include at least one user terminal (eg, the first HMD device 100 or the electronic device) connected to the broadcast providing device 300 to broadcast data of the BJ (eg, a live streaming video or a VOD video). 200) or a broadcast management server 500 may be transmitted. The communication interface 310 may receive an image signal from the photographing apparatus 400 or transmit third virtual reality image data representing the virtual private space of the BJ to the second HMD device 410. Here, the third virtual reality image data may be a virtual reality image in which the BJ virtual personal space is implemented as virtual reality to provide to the BJ. The virtual private space of the BJ provided to the BJ may be different from the virtual private space of the BJ provided to the user.

The display 320 may display various contents (eg, text, images, videos, icons, banners, or symbols) to the user (eg, BJ). In detail, the display 320 may display an execution screen for a platform (eg, a broadcast program or an application) for viewing a broadcast.

The memory 330 may store various data for broadcasting. In detail, the memory 330 may store a platform (eg, a broadcast program or an application) for broadcasting and store an image signal received from the photographing apparatus 400.

The processor 340 is operatively connected to the communication interface 310, the display 320, and the memory 330, and may perform various commands for broadcasting. In detail, the processor 340 records and records an image signal input through the photographing apparatus 400 using a platform (eg, a broadcast program or an application) for broadcasting transmission in response to an application execution request for broadcasting by BJ. The video signal may be transmitted to at least one user terminal (for example, the electronic device 200 or the image output device 600) or the broadcast management server 500 in real time streaming.

In various embodiments, the processor 340 receives a request for displaying the third virtual reality image data representing the virtual private space of the BJ implemented in virtual reality from the second HMD device 410 through the communication interface 310. The request may be transmitted to the broadcast management server 500. When the third virtual reality image data is received from the broadcast management server 500, the processor 340 may transfer the received third virtual reality image data to the second HMD device 410. For example, when a signal indicating that the BJ mounts the second HMD device 410 is received from the second HMD device 410, the processor 340 sends the broadcast management server 500 to the third virtual reality image data. You can forward the request.

The second HMD device 410 may include a display (not shown) and may display third virtual reality image data received from the broadcast providing device 300 through the display. The virtual private space of the BJ may include a main interface area for playing the broadcast video set by the BJ and a sub interface area related to various functions for setting the virtual private space of the BJ. For example, the third virtual reality image data may be related to a first layer in which a background area of a virtual private space of BJ is displayed, a second layer in which a main interface area of the virtual private space of BJ is displayed, and a virtual private space of BJ. Thus, the third layer displaying the sub-interface area related to various functions that may be provided to the BJ may be synthesized image data.

The main interface area of the virtual private space of the BJ represents a first graphic screen for playing a broadcast image such as a broadcast replay image, a highlight image, a VOD image produced by the viewer, and chat data corresponding to the image. It may include a second graphic screen. In various embodiments, the main interface area may be a third graphic screen representing a community space (eg, announcement, bulletin board, etc.) related to the broadcast of BJ, a broadcast replay image of BJ, a highlight image, a VOD image produced by the viewer, and the like. The apparatus may further include a fourth graphic screen indicating an image list of the video. According to various embodiments of the present disclosure, a plurality of first graphic screens may be implemented to reproduce at least two of a broadcast replay image selected by the BJ, a broadcast highlight image, and VODs produced by the viewer.

The sub-interface area of the virtual private space of the BJ may be provided to the BJ to set a virtual personal space of the BJ and a first interface screen for setting the playback of the broadcast replay image, the broadcast highlight image, or the viewer VOD image. It may include a second interface screen related to the function. For example, various functions for setting the virtual private space of the BJ may include a function for setting the number, size or resolution of the first graphic screen, a function for setting the size or on / off of the second graphic screen, Function to set the size, layout, structure, etc. of the third graphic screen representing the community space, Function to set the size of the fourth graphic screen, the number of the video list, the list display method, and decorate the avatar or personal broadcasting space It may include a function such as the purchase or gift of a paid item for and the function to place the purchased or received paid items in a virtual personal space. Through this, the second HMD device 410 sets the number of the first graphic screen, the type or resolution of the playback image, or the size of the second graphic screen through the second interface screen at the user's request. Set the off, etc., set the size, layout, structure, etc. of the third graphic screen, set the size of the fourth graphic screen, the number of the image list, the display method of the list, or purchase a paid item or a gift. Paid, purchased or gifted paid items can be placed in a virtual personal space.

According to various embodiments of the present disclosure, the image providing apparatus 300 further stores an application or a program for the memory 330 to generate virtual reality image data, and uses the corresponding application or program to first, second, and third virtual images. Real image data may also be generated. The first and second virtual reality image data generated as described above may be transferred to the electronic device 200 or the image output device 600 through the broadcast management server 500.

6 is a schematic diagram illustrating a broadcast management server according to an embodiment of the present invention. Referring to FIG. 6, the broadcast management server 500 may include a communication interface 510, a virtual reality image processor 520, a memory 530, and a processor 540.

The communication interface 510 connects the broadcast management server 500 to communicate with an external device. The communication interface 510 is connected to the first HMD device 100, the electronic device 200, the broadcast providing device 300, the second HMD device 410, or the image output device 600 using wired / wireless communication. Can transmit and receive various information. In detail, the communication interface 510 receives the request for the first virtual reality image data or the request for the second virtual reality image data from the electronic device 200 or the image output device 600, and the electronic device 200 or the image. The first virtual reality image data or the second virtual reality image data may be delivered to the output device 600. The communication interface 510 receives a request for the third virtual reality image data from the image output device 600 or the broadcast providing device 300 mounted on the second HMD device 410, and the second HMD device 410. The third virtual reality image data may be transmitted to the image output apparatus 600 or the broadcast providing apparatus 300 installed in the apparatus. In various embodiments, the communication interface 510 may receive various setting information about the user's virtual personal space from the first HMD device 100 or the electronic device 200. In various embodiments, the communication interface 510 may receive various setting information about the virtual private space or the virtual private space of the BJ from the image providing device 300 or the second HMD device 410. In various embodiments, the BJ's virtual private space provided to the BJ and the BJ's virtual private space provided to the user may be the same, but the BJ provided to the user when the BJ sets different types of virtual private spaces for each user The virtual private space of may be different for each user, and the BJ's virtual private space provided to the BJ and the BJ's virtual private space provided to the user may be different according to the setting of the BJ.

The virtual reality image processor 520 may include first virtual reality image data representing a virtual personal space implemented as a virtual reality to enable a broadcast viewer to watch a broadcast, and a virtual reality so that the broadcast viewer may experience the virtual personal space of BJ as a visitor. The second virtual reality image data representing the virtual private space of the BJ implemented as and the third virtual reality image data representing the virtual private space of the BJ implemented as virtual reality may be generated to be provided to the BJ. In detail, the virtual reality image processor 520 allocates a background region of the user's virtual personal space to the first layer, allocates a main interface region to the second layer, and generates a sub-interface to generate the first virtual reality image data. An area may be allocated to the third layer. For example, the background area of the virtual personal space may include various virtual reality objects such as wallpaper, floors, floorboards, sofas, chairs, tables, picture frames, and interior props such as bookshelves for decorating the virtual personal space set by the user. Can be. The virtual reality image processor 520 may allocate real-time streaming broadcast data received from the broadcast providing device 300 to a second layer to be displayed on the first graphic screen of the main interface region. In various embodiments, when there are two or more first graphic screens, the virtual reality image processor 520 may display two or more real-time streaming broadcast data received from the broadcast providing device 300 on the second layer to be displayed on each of the two or more first graphic screens. Can be assigned. The virtual reality image processor 520 may generate first virtual reality image data obtained by combining the first layer, the second layer, and the third layer.

The virtual reality image processor 520 may generate the second virtual reality image data and the third virtual reality image data, similarly to the above description about the first virtual reality image data. In detail, the virtual reality image processor 520 allocates a background area of the virtual personal space of the BJ to the first layer, allocates a main interface area to the second layer, and generates a sub-interface to generate the second virtual reality image data. An area may be allocated to the third layer. The virtual reality image processor 520 may generate second virtual reality image data obtained by combining the first layer, the second layer, and the third layer. In addition, the virtual reality image processor 520 allocates the background area of the virtual private space of the BJ to the first layer, the main interface area to the second layer, and the sub interface area to generate the third virtual reality image data. Can be assigned to the third layer. The virtual reality image processor 520 may generate third virtual reality image data obtained by combining the first layer, the second layer, and the third layer.

According to various embodiments of the present disclosure, the virtual reality image processor 520 may invite specific users through a user terminal, and the invited specific users may be invited by the user and other users invited by the user in response to a request to join the user's virtual personal space. The first virtual reality image data implemented to participate in the virtual personal space of the user may be generated. The first virtual reality image data may be implemented such that the avatar in the virtual reality of the user and the avatars of each of the invited users may experience together in the virtual private space of the user. The virtual reality image processor 520 may change the number, size, type, or resolution of the first graphic screen or the second graphic screen based on various setting information regarding the virtual personal space of the user received from the user terminal. Change the size of the screen, turn on / off the second graphic screen, set or change the video list, or decorate the virtual private space according to the purchase of a paid item or a gift. The first virtual reality image data may be generated to apply a changed setting such as a change. In various embodiments, the virtual reality image processor 520 may include the second virtual reality image data or the second virtual reality image data such that the changed setting is applied based on various setting information about the virtual personal space or the virtual personal space of the BJ received from the image providing apparatus 300. Third virtual reality image data may be generated.

The memory 530 may store various data used to provide real-time streaming broadcast image data or VOD image data or to generate virtual reality image data. In detail, the memory 530 may store first, second, and third virtual reality image data generated by the virtual reality image processor 520. The memory 530 may store various setting information received from the user terminal or the image providing device 300. In addition, the memory 530 may store real-time streaming video data, VOD video data, replay video data, highlight video data, viewer-produced video, etc. for each BJ identifier.

The processor 540 is operatively connected with the communication interface 510, the virtual reality image processor 520, and the memory 530, and provides a real-time streaming broadcast image or a VOD image, or a first, second or third virtual reality. Various commands for generating image data may be performed. In detail, the processor 540 receives a request for the first, second or third virtual reality image data through the communication interface 510, and generates the first or second through the virtual reality image processor 520. The virtual reality image data may be delivered to the first HMD device 100 or the image output device 600, or the generated third virtual reality image data may be delivered to the second HMD device 410 or the image providing device 300.

In various embodiments, the processor 540 may perform the operation of the virtual reality image processor 520 without providing a separate virtual reality image processor.

7 is a schematic flowchart illustrating a method of providing a user interface for watching a virtual reality broadcast in a user terminal according to an embodiment of the present invention.

Referring to FIG. 7, a user terminal (eg, the first HMD device 100 or the image output device 600) may virtually watch a live streaming broadcast of a broadcast provider (eg, BJ) from the broadcast management server 300. First virtual reality data representing the virtual personal space of the user implemented in reality may be received (S700), and the received first virtual reality data may be displayed on the display of the user terminal (S710). When an event having a predetermined condition is generated in relation to the live streaming broadcast of the broadcast provider, the user terminal may display a virtual reality effect object corresponding to the generated event in a virtual personal space (S720).

Hereinafter, the first virtual reality image data displayed through the display 120 of the first HMD device 100 and the display 620 of the image output device 600 will be described with reference to FIGS. 8, 9A, 9B, 10A, and 10A. 10B, 11A, 11B, 11C, 11D, 11E, 11F, 12, 13, and 14 to be described in detail. Operations described below may be performed by the processor 150 of the HMD device 100 or the processor 640 of the image output device 600.

8 is an exemplary diagram for describing first virtual reality image data representing a virtual personal space of a user according to an embodiment of the present invention.

Referring to FIG. 8, the virtual private space 800 of the first virtual reality image data may include three first graphic screens 802, 804, and 806 for reproducing a live streaming broadcast image, and a live chat of broadcast viewers. The main interface area 810 including two graphic screens 808 may be included. In various embodiments, the main interface area 810 is a third graphic screen 812 for playing the most popular (eg, high number of views or high likes) of BJ's favorite VOD images. It may further include. The graphic screens may be resized or repositioned by an input device 550 (eg, a controller) or a distance perceived by a user in a virtual personal space. For example, if the pointer of the controller is located on the top bar of the graphic screen by the user's controller operation, the pointer is moved to a specific position while the button of the controller is input, and the button input is stopped, the graphic screen is moved to the specific position. Can be. In addition, when the pointer of the controller is positioned at the corner of the graphic screen by the user's controller operation and the controller is moved while the button of the controller is clicked, the size of the graphic screen may be adjusted according to the movement of the controller. In addition, graphic screens may be rotated or minimized by a user's controller operation. In the case of minimization, the graphic screen may be converted to a thumbnail video of a preset size and displayed.

The virtual private space 800 may include an interface screen 814 representing a video list or a VOD video list related to a live streaming broadcast, and interface screens 816, 818, 820, and 822 related to various functions related to broadcasting viewing or virtual private space. Sub-interface area 824 may be included. The interface screens 816, 818, 820, and 822 may be converted to a minimized state as shown in FIG. 8 and displayed as icons in a bar shape.

The main interface area 810 of the virtual private space 800 will be described in more detail below with reference to FIGS. 9A and 9B, and the sub interface area 824 is described with reference to FIGS. 10A, 10B, 11A, and FIG. With reference to 11b, 11c, 11d, 11e, 11f, 12, 13 and 14 will be described in more detail below.

9A and 9B are exemplary diagrams for describing graphic screens of a main interface area according to an embodiment of the present invention.

8 and 9A, the main interface area 810 is a first graphic screen 802 for playing back real-time streaming broadcast video data of BJ and a second data representing chat data of broadcast viewers watching the live streaming broadcast. It may include a graphic screen 808. The first graphic screen 802 may include a display area 900 in which real-time streaming broadcast image data is displayed and a non-display area 910 including icons for display control and volume control. The non-display area 910 may include a first icon 912 for playing, pausing and stopping an image, a second icon 914 for adjusting a volume, and a third icon 916 for turning on and off a chat. have. The second graphic screen 808 may include a chat display area 920 where chat contents are displayed, setting icons 922 for setting a chat, and a chat input icon 924 for chatting input. The setting icons 922 may display a list of broadcast viewers participating in the chat, various functions related to the chat (eg, displaying / hiding emoticons, receiving / not receiving whisper, changing nickname, deleting chat, etc.), and chat window. It may correspond to a delete function. When the chat input icon 924 is selected by a user's controller operation, the chat input area 926 and the virtual keyboard area 928 for input may be displayed as shown in FIG. 9B. For example, when a pointer of the controller is located in a specific key area of the virtual keyboard area and a button of the controller is clicked, a character corresponding to the key area may be input to the chat input area 926. In various embodiments, the chat input area 926 and the virtual keyboard area 928 allow the user to perceive a distance closer than the distance to the first graphic screen 802 and the second graphic screen 808 within the virtual personal space. Can be displayed.

In various embodiments, when the third icon 916 for chatting on / off is selected due to the controller operation of the user as described with reference to FIG. 9A, the display of the second graphic screen 808 is terminated as shown in FIG. 9C, and the display area ( The first icon 930 for minimizing the first graphic screen 802 and the second icon 932 for terminating the display of the first graphic screen 802 may be displayed at a specific position of the 900. For example, when the first icon 930 is selected by a user's controller operation, the first graphic screen 802 may be converted into a specific size that can play a thumbnail video.

10A, 10B, 11A, 11B, 11C, 11D, 11E, 11F, 12, 13, and 14 illustrate interfaces related to icons in a sub interface area according to an embodiment of the present invention. Illustrated diagrams for describing screens.

8 and 10A, the sub interface area 824 may include a first interface screen 814 representing a video list or a VOD video list related to a live streaming broadcast, and a second interface screen related to various functions related to a virtual private space. 816, a third interface screen 818 representing a bulletin board which is a community space related to viewing a broadcast, a fourth interface screen 820 representing a history of purchasing or receiving a paid item, and a broadcast list for a favorite BJ. The fifth interface screen 822 may be included.

The first interface screen 814 may include a first icon 1000 for displaying a live streaming broadcast video list of at least one favorite BJ, a second icon 1002 for displaying a VOD video list, and a broadcast category. The image list screen 1006 may include a third icon 1004 for selecting an image and a thumbnail image (or thumbnail video) 1008 related to at least one image included in the image list. When the first icon 1000 is selected by a user's controller operation, thumbnail images of at least one BJ broadcast that is currently live or favorite related to a specific category (eg, a game) are displayed on the video list screen 1006. Can be displayed. In addition, when the second icon 1002 is selected, thumbnail images of at least one favorite VJ image of the BJ may be displayed on the image list screen 1006. When the third icon 1004 is selected, a selection screen 1010 for selecting a broadcast-related category is displayed as shown in FIG. 10B, and when a menu 1012 corresponding to a game is selected among broadcast-related categories, selecting a game-related category. The selection area 1014 may be displayed.

8 and 11A, the second interface screen 816 may include a first icon (i.e., a first icon for setting, adding, resizing, repositioning, rotating, etc. with respect to the first graphic screens 802, 804, 806). Example: screen setting) 1100, a second icon for setting a live streaming broadcast video list or a VOD video list (eg, video list setting) 1102, a third icon for decorating a virtual private space (eg, interior setting) 1104 and a fourth icon (eg, item purchase) 1106 for purchasing a paid item for broadcasting or watching a BJ. When the first icon 1100 is selected by a user's controller operation, a setting screen 1107 for setting a screen may be displayed as shown in FIG. 11B. The setting screen 1107 may include a first icon (eg, screen addition) 1108 for adding a graphic screen for image reproduction to the main interface area 810, and a size of the graphic screen located in the main interface area 810. A second icon (eg, screen size change) 1110 for changing and a third icon (eg, resolution change) 1112 for changing the resolution of the graphic screen may be included. According to various embodiments of the present disclosure, an icon (not shown) for changing a frame of the graphic screen may be added. For example, a frame of a graphic screen may be converted into a frame of various virtual reality objects having a display area for displaying an image such as a frame, a TV, a picture, etc., which may be disposed in a user's virtual personal space. For example, a graphic screen may be implemented such that a broadcast image is reproduced in an area corresponding to a CRT of a TV when it is changed to a TV, and a broadcast image is reproduced in an area except a frame frame when a frame is changed.

When the first icon 1108 is selected by a user's controller operation, the graphic screen may be added at a specific size in a specific position of the main interface area 810 where the graphic screen is located except the area where the graphic screen is located. Here, the added graphic screen may be displayed in an appropriate size at an appropriate position so as not to overlap the existing graphic screen.

When the second icon 1110 is selected by a user's controller operation, other objects except the graphic screen may be deactivated in order to change the size of the graphic screen located in the main interface area 810. Subsequently, the pointer is positioned at the edge of the graphic screen by the user's controller operation, and the size of the graphic screen may be changed by the user's controller movement while the button of the controller is clicked. In various embodiments, the graphic screen may also be changed in location by a user's controller operation. When the pointer is positioned in an inactivated area other than the graphic screen by the controller operation of the user, the graphic screen of the changed size is displayed and the inactivated area may be activated again.

When the third icon 1112 is selected by a user's controller operation, a selection screen 1114 for selecting a resolution may be displayed. In various embodiments, when there is more than one graphic screen, when the pointer is positioned in a specific area of the graphic screen by the user's controller operation and the edge of the graphic screen is highlighted, the graphic screen is selected, and for changing the resolution of the selected graphic screen. The selection screen may be displayed.

Returning to FIG. 11A again, when the second icon 1102 is selected, the setting screen 1115 for setting the image list as shown in FIG. 11C or 11D may be displayed.

11C is an exemplary diagram for setting a real-time streaming broadcast video list for a BJ selected by a user according to an embodiment of the present invention.

Referring to FIG. 11C, the setting screen 1115 may include a first area 1116 representing a broadcast category and a second broadcast station representing at least one BJ for a corresponding broadcast category when a specific broadcast category (eg, a game) 1118 is selected. Area 1120, selection key areas 1122 for selecting BJ, third area 1124 representing the selected BJ, and key areas (e.g., confirmation key, cancel key) 1126 for confirming or canceling the setting. ) May be included. The second area 1120 indicating the BJ list may include BJ identification information (eg, ID) related to a specific category. When the pointer is positioned at a specific ID and the edge of the specific ID is highlighted by the user's controller operation, and the selection key 1122 is selected, the specific ID may be selected and displayed on the third area 1124. Thereafter, when the confirmation key is selected, the setting information may be stored to display a list of live streaming broadcast images for BJ Africa 1, BJ Africa 3, and BJ Africa 8. As such, when at least one BJ is selected, a real-time streaming broadcast video list of at least one selected BJ (eg, BJ Africa 1, BJ Africa 3, and BJ Africa 8) may be displayed as shown in FIG. 10A.

11D is an exemplary diagram for setting a VOD video list of BJs selected by a user according to an embodiment of the present invention.

Referring to FIG. 11D, the setting screen 1115 may include a first area 1128 indicating a broadcast category and a second area indicating at least one BJ related to the category when a specific category (eg, a game) 1130 is selected. 1132, a third region 1134 indicating a VOD image list of the selected BJ and key regions (eg, confirmation key, cancel key) 1136 for confirming or canceling the setting. For example, the third region 1134 may display a thumbnail image of at least one VOD image of the selected 'BJ Africa 3'. By the controller operation of the user, the pointer is positioned on the specific thumbnail image so that the edge of the specific thumbnail image is highlighted, and when the confirmation key is selected, the specific thumbnail image may be selected. As such, when at least one VOD image of 'BJ Africa 3' is selected, a list of at least one selected VOD image (eg, image 1, image 4, image 5) may be displayed as shown in FIG. 10A.

In various embodiments, the live streaming broadcast video list or the VOD video list may include a live streaming broadcast video for a BJ watched by a user or another BJ in a broadcast category similar to the corresponding BJ, even if not set by the user, or the user. Other VOD images of a broadcast category similar to the VOD images watched by many may be included as recommended images.

11A and when the third icon 1104 is selected, a setting screen 1137 for decorating the virtual personal space may be displayed as shown in FIG. 11E. The setting screen 1137 may include a first area 1138 indicating a category for a wall of a virtual private space, a wallpaper for decorating a floor, a floor material and furniture for placing in a virtual private space, and interior props, and a specific category (eg, (Furniture) 1140 is selected, the second region 1142 representing at least one virtual reality object for the particular category selected and key regions (e.g., confirmation key, cancel key) 1144 for confirming or canceling the setting. ) May be included. The at least one virtual reality object may be a 3D graphic object modeled such that a sofa, table, bed, closet, etc. are optimized for virtual reality. The second area 1142 may include at least one thumbnail image of furniture. By the user's controller operation, the pointer is positioned on the thumbnail image 1146 corresponding to the sofa so that the edge of the thumbnail image 1146 is highlighted, and when the confirmation key is selected, the virtual reality object corresponding to the sofa may be selected. When the virtual reality object corresponding to the sofa is selected, the selected virtual reality object may be displayed in the virtual private space. The pointer of the user's controller is positioned on the displayed virtual reality object, and the virtual reality object in the virtual eye opening space can be moved by moving the controller while the user's button is clicked. For example, when the virtual reality object is located in a specific area of the virtual personal space by moving the controller of the user, and the button input of the controller is stopped, the virtual reality object 1148 corresponding to the sofa becomes the virtual personal space 800 as illustrated in FIG. 11F. ) Can be placed.

11A, when the fourth icon 1106 is selected, a setting screen 1149 for purchasing a paid item for watching a broadcast or sponsoring BJ may be displayed as shown in FIG. 11G. The setting screen 1149 may include a first area 1150 representing a category (eg, a star balloon, a sticker, a chocolate, a spray, an interior accessory, etc.) related to various paid items, and a specific category (eg, a star balloon) related to a paid item. When 1115 is selected, it may include a second area 1154 indicating at least one paid item for the selected category and key areas (eg, confirmation key, cancellation key) 1156 for confirming or canceling the setting. have. For example, various paid items may include items that implement stickers, icons, sprays, billboards, flags, hooligans, and the like related to sports or e-sports as virtual reality objects. In various embodiments, the paid item may be a star balloon, a sticker, chocolate, a spray, interior props, etc. related to BJ or specific content (eg, muckbang content, game content, sports content, etc.). The second area 1158 may display graphic objects including an image of at least one paid item and a button for purchasing the paid item. For example, when a button 1158 for purchasing a specific paid item is selected by a user's controller operation, a payment operation for purchasing the selected paid item may be performed. Since a payment operation for purchasing a paid item is a general technology, a detailed description thereof will be omitted. When the paid item is purchased, the virtual reality effect object 826 corresponding to the purchased pay item as shown in FIG. 8 may be displayed at a specific location in the virtual personal space. For example, a sound effect of popping firecrackers may be output along with a visual effect of star balloons flying in a virtual private space.

In various embodiments, the paid item is not limited to the presented embodiment, and various decorating items for decorating a user's avatar may be further included. For example, a decorating item to decorate a user's avatar may include items such as clothes, accessories, shoes, hats, bags, other miscellaneous goods, or items to change a part of the avatar (e.g., change hairstyle, change hair color, Eyes, nose, mouth, etc.). In this way, items for decorating the avatar may also be provided in relation to the BJ or specific content (eg, Mukbang content, game content, sports content, etc.).

8 and 12, the third interface screen 818 may be an interface screen converted to a default state as shown in FIG. 12 from a state minimized to a bar-shaped icon as shown in FIG. 8. For example, the third interface screen 818 may further include an icon for switching to the minimized state or the changed from the minimized state to the default state at the top.

The third interface screen 818 is selected when a first area 1200 representing a bulletin board category (eg, announcement, broadcast schedule, hangout, and recommendation bulletin board, etc.) and a specific bulletin board category (eg, hangout) 1202 are selected. It may include a second area 1204 indicating a post regarding the bulletin board category. Posts included in the second area may be displayed as a graphic object indicating a title, author ID, author date, number of views, number of recommendations, and comments.

8 and 13, the fourth interface screen 820 may be an interface screen converted to a default state as shown in FIG. 13 while being minimized to a bar-shaped icon as shown in FIG. 8. For example, the fourth interface screen 820 may further include an icon for switching to the minimized state or the changed from the minimized state to the default state at the top.

The fourth interface screen 820 may include a first area 1302 indicating a pay item list 1300 held by the user and a second area 1306 indicating a pay item list 1304 that the user has received from another user. It may include. The first region 1302 may include a graphic object representing a paid item for watching a broadcast or decorating a virtual personal space.

Referring to FIGS. 8 and 14, the fifth interface screen 822 may be an interface screen converted to a default state as shown in FIG. 14 while being minimized to a bar-shaped icon as shown in FIG. 8. For example, the fifth interface screen 822 may further include an icon for switching to the minimized state or the default state from the minimized state.

The fifth interface screen 822 may include graphic objects 1400, 1402, and 1404 representing a broadcast list of at least one favorite BJ. Each of the graphic objects 1400, 1402, 1404 includes an image associated with the BJ, text representing the ID of the BJ, and a button (eg, a move button) 1406, 1408, 1410 for moving to the virtual private space of the BJ. can do. For example, when a specific movement button 1406 is selected by a user's controller operation, a virtual reality image indicating a virtual personal space of the corresponding BJ may be displayed. In addition, when the BJ performs a live streaming broadcast, icons (eg, a LIVE watching icon) 1412 and 1414 for watching the live streaming broadcast of the BJ may be further included. For example, when the live viewing icon 1414 of 'BJ Africa 8' is selected by a user's controller operation, the live streaming broadcast image of 'BJ Africa 8' is displayed on the first graphic screen 802 of the main interface area 810. FIG. ), And a live chat related to the corresponding live streaming broadcast may be displayed on the second graphic screen 808.

The above-described interface screens may further include a button for minimization, and when the minimize button is selected by a user's controller operation, the remaining areas except the upper bar may not be displayed. If the minimize button is selected again, the existing interface screens may be displayed again.

15 to 17 are exemplary diagrams for describing second virtual reality image data representing a virtual private space of a BJ according to an embodiment of the present invention. Hereinafter, a description will be given of an embodiment in which a plurality of broadcast viewers simultaneously (or together) watch a live streaming broadcast such as sports relay or e-sports relay.

Referring to FIG. 15, the virtual private space 1500 of BJ for sports relay broadcasting or e-sports relay broadcasting may include a first graphic screen 1502 for playing a live streaming relay broadcast and broadcast viewers watching the relay broadcast. A main interface area 1506 can be included that includes a second graphical display 1504 representing a live chat. The main interface area 1506 further includes a plurality of graphic screens 1508 and 1510 for reproducing various broadcast images related to sports relay or e-sports relay in addition to the first graphic screen 1502 for reproducing the relay broadcast image. May be included. For example, the plurality of graphic screens may display broadcast images photographed through various cameras for broadcasting, or broadcast images photographing athletes participating in sports. In various embodiments, the BJ's virtual private space 1500 may be implemented like a sports bleacher 1512 to enable a broadcast viewer to actually sit in a sports event. In various embodiments, the BJ's virtual personal space 1500 may be implemented to enable an experience in which an avatar corresponding to each of the plurality of broadcast viewers who are watching the broadcast is sitting in the sports auditorium 1512. This allows the user to have a virtual experience as if they were actually watching a sport or e-sport with other broadcast viewers.

When an event such as a purchase of a paid item or a user who has a high contribution contribution to the virtual private space participates in a chat occurs by the controller operation of the user, the virtual reality effect object 1514 corresponding to the event is displayed in the virtual personal space of the BJ. And may be displayed within 1500. The virtual reality effect object 1514 may be implemented to be common to all broadcast viewers visiting the virtual personal space 1500 of the BJ as well as the user.

Referring to FIG. 16, the virtual reality effect object 1514 displayed in the virtual private space 1500 of the BJ may move to a portion of the virtual private space 150 and be displayed continuously after a predetermined time. For example, when a preset event occurs, a virtual reality effect object 1514 composed of a plurality of balloons is displayed at a specific location of the virtual personal space 1500, and the plurality of balloons are displayed at the top of the virtual personal space 1500. The display may move to regions 1600 and 1610 and may be continuously displayed. According to various embodiments of the present disclosure, another virtual reality effect that causes the selected balloon to pop when a pointer in the virtual personal space is moved by a user's controller operation to select one of the balloons may appear. Through this, the user can experience various virtual experiences through interaction with the virtual reality objects arranged in the virtual reality.

According to various embodiments of the present disclosure, a dedicated icon that can be used only by a user with high sponsored contribution is input through the second graphic screen 1504 by a controller operation of a user with a high sponsored contribution to BJ, or a chat-only sticker for chatting by the user. When the same paid item is purchased and the purchased paid item is input, as illustrated in FIG. 16, a dedicated icon or a speech bubble 1620 and 1630 including the paid item input on the avatar of the user may be displayed. Such speech balloons may be set to be displayed or not displayed by the user. In addition, the speech balloons may be set to display a certain number of times for a predetermined time.

In various embodiments, a voice chat may be performed between at least one of the users who participated in the virtual private space 1500 of the BJ. For example, the second graphic screen 1504 may further include an icon for voice chat. If the user's first HMD device 100 further includes a microphone, and an icon is selected by the user's controller operation, the voice signal acquired through the microphone is transmitted to the HMD device of at least one other viewer participating in the voice chat. Can be. In this case, an icon for indicating that the user is participating in the voice chat may be displayed in the virtual personal space 1500 of the BJ. For example, the icon for indicating that the user is participating in the voice chat may be a microphone image icon, and the microphone image icon may be displayed at a specific position in the virtual personal space 1500 of the BJ. In various embodiments, a three-dimensional graphic microphone image may be displayed on the BJ's virtual personal space 1500 to represent the user's voice output such that the user may actually experience a voice chat using the microphone.

In various embodiments, when a voice signal is received from another viewer's HMD device, the voice signal received through the speaker of the first HMD device 100 worn by the user is output, and the voice chat is performed on the avatar of another user as shown in FIG. 16. A speech bubble 1640 including a speaker icon for indicating the display may be displayed. The speech bubble 1640 may be simultaneously displayed through the HMD device of at least one user participating in the voice chat.

In various embodiments, the voice chat may be performed only by at least one broadcast viewer selected by the BJ or at least one broadcast viewer according to the contribution contribution of the BJ. For example, an icon for voice chat may be displayed on the second graphic screen 1504 only for the selected broadcast viewer or broadcast viewer corresponding to the sponsored contribution.

In various embodiments, voice chat may turn on or off the voice output of a particular user. For example, a speaker icon indicating a voice output is displayed around an avatar of a specific user participating in a voice chat, and when the speaker icon is selected by a user's controller operation, the speaker icon is converted to an icon for turning off the corresponding voice output. The audio output of may be cut off. When the icon is selected again by the user's controller operation, the icon may be converted into an icon for turning on the corresponding voice output and the voice of a specific user may be output.

Referring to FIG. 17, the paid items 1700, 1710, and 1720 are purchased through the setting screen 1149 as illustrated in FIG. 11G by a user's controller operation, and the purchased paid items 1700, 1710, and 1720 are purchased from the BJ. It may be displayed in a specific area within the personal virtual space 1500. For example, a paid item (for example, a cheer sticker or a spray, etc.) for cheering a specific player or team related to a sport or e-sport is purchased through the setting screen 1149 by a user's controller operation. Paid items may be placed in specific locations. In various embodiments, a virtual reality object 1730, such as a cheering rod, may be held in a hand of an avatar corresponding to each of the broadcast viewers for cheering on sports or e-sports. Through this, a broadcast viewer watching a sports relay broadcast or an e-sports relay broadcast may experience a virtual experience as if they are holding a cheering rod and cheering together with other broadcast viewers.

18 is an exemplary diagram for describing first virtual reality image data representing a virtual personal space of a user according to an embodiment of the present invention. Hereinafter, an embodiment in which a user invites a friend to a virtual personal space to watch a broadcast together with the invited friend will be described.

Referring to FIG. 18, the virtual personal space 800 of the first virtual reality image data may include three first graphic screens 802, 804, and 806 for playing back a real-time streaming broadcast image as described above with reference to FIG. 8. And a main interface area 810 including a second graphic screen 808 representing a live chat of broadcast viewers. In addition, the virtual private space 800 may include an interface screen 814 representing a video list or a VOD video list related to a live streaming broadcast, and interface screens 816, 818, 820, and the like related to various functions related to broadcasting viewing or virtual private space. A sub interface area 824 including 822 may be included, and a sixth interface screen 1800 related to a function for inviting another user, who is set as a friend of the user, to the user's virtual personal space may be included. The sixth interface screen 1800 may be converted into a minimized state as described above with reference to FIG. 8 and displayed as a bar icon. The added sixth interface screen 1800 will be described in more detail below with reference to FIG. 19.

19 is an exemplary diagram for describing an interface screen related to a friend finder icon in a sub-interface area according to an embodiment of the present invention.

Referring to FIGS. 18 and 19, the sixth interface screen 1800 may be an interface screen converted to a default state as shown in FIG. 19 from a state minimized to a bar-shaped icon as shown in FIG. 18. For example, the sixth interface screen 1800 may further include an icon for switching to a minimized state or a default state from the minimized state.

The sixth interface screen 1800 includes a first area 1802 indicating a friend list of the user and a second area 1804 indicating a message composition area for requesting an invitation to a friend to invite to the user's virtual personal space. can do. The first area 1802 includes identification information (eg, ID) of another user set as a friend by the user, and a friend corresponding to a function for adding a friend when the friend to be invited does not exist in the friend list. It may further include additional icons. The sixth interface screen 1800 may include a first icon (for example, inviting) 1806 for transmitting a friend invitation message created through the message composition area of the second area 1804, and a second icon for canceling the same. Example: Cancel) 1808 may be further included. For example, a specific ID (eg, ID1) of a friend list is selected by a user's controller operation, a text or the like is input through the message composition area of the second area 1804, and then the first icon 1806 is displayed. If selected, an invitation message may be transmitted to the HMD device of another user corresponding to the selected 'ID1'. The invitation message may be displayed as a graphic object at a specific location of another user's virtual personal space. The graphic object representing the invitation message may further include icons corresponding to a function for accepting or rejecting the invitation.

In various embodiments, the interface screen for inviting friends is not limited as described above. For example, the interface screen for inviting a friend may include an icon corresponding to an invitation function together with profile images of at least one other user set as a friend by the user. When the icon corresponding to the invitation function is selected together with the profile image of the specific user by the controller operation of the user, the invitation request message may be transmitted to the HMD device of the specific user. If the invitation is approved by another user, the avatar of another user who has approved the invitation may be displayed in the user's virtual personal space 800. This will be described below with reference to FIG. 20.

20 is an exemplary diagram for describing first virtual reality image data representing a virtual personal space of a user according to an embodiment of the present invention. Hereinafter, an embodiment in which a user invites another user set as a friend of the user to the user's virtual personal space to watch streaming broadcast or VOD broadcast together.

18 to 20, when an invitation is accepted by another user as described above with reference to FIG. 19, the avatar 1900 of another user who accepts the invitation as shown in FIG. 20 is displayed in the virtual personal space 800 of the user. It can be displayed at a specific location. For example, when a user wants to watch a sports or e-sports broadcast with a friend in the user's virtual personal space, an invitation message may be sent to another user through the sixth interface screen 1800 corresponding to the friend invitation function. have. The invitation message may be displayed as a pop-up interface screen at a specific location of another user's virtual personal space. The interface screen may include an icon for accepting the invitation and an icon for rejecting the invitation together with the content of the invitation message written by the user. When an icon for accepting an invitation is selected by another user, an avatar of another user who accepts the invitation may be displayed in a specific location (eg, sofa 1148) in the user's virtual personal space 800. In this case, the other user who accepts the invitation may have a virtual experience as if they are sitting on the sofa 1148 in the user's virtual personal space 800 through the display of his HMD device. The avatar of the user may also be displayed in the virtual personal space 800 of the user. In this case, as shown in FIG. 20, the display 120 of the first HMD device 100 worn by the user may have a first virtual reality in which another user's avatar is sitting on the sofa 1148 disposed in the user's virtual personal space 800. Image data can be displayed. This allows the user and other users to have a virtual experience as if they are watching a broadcast together in the user's virtual personal space 800.

As described above, the present invention enables a user to experience various spatial and temporal experiences when watching a broadcast by watching a real time streaming broadcast image provided by a broadcast provider through the user's virtual personal space implemented in virtual reality.

Apparatus and method according to an embodiment of the present invention can be implemented in the form of program instructions that can be executed by various computer means may be recorded on a computer readable medium. Computer-readable media may include, alone or in combination with the program instructions, data files, data structures, and the like.

The program instructions recorded on the computer readable medium may be those specially designed and constructed for the present invention, or may be known and available to those skilled in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Hardware devices specially configured to store and execute program instructions such as magneto-optical media and ROM, RAM, flash memory and the like. In addition, the above-described medium may be a transmission medium such as an optical or metal wire, a waveguide, or the like including a carrier wave for transmitting a signal specifying a program command, a data structure, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

Although the embodiments of the present invention have been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention. . Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

100: first HMD device
200: electronic device
300: broadcast providing device
400: shooting device
410: second HMD device
500: broadcast management server
550: input device
600: video output device

Claims (14)

Receiving first virtual reality image data representing a virtual private space implemented in virtual reality to watch a live streaming broadcast of a broadcast provider from a broadcast management server; And
Displaying the received first virtual reality image data through a display of a head mounted display (HMD) device;
The virtual private space includes a main interface area for viewing the live streaming broadcast and a sub interface area related to a plurality of functions related to viewing the live streaming broadcast,
The main interface area includes a first graphic screen for playing a live streaming broadcast image and a second graphic screen for showing a real time chat between broadcast viewers simultaneously watching the live streaming broadcast,
The sub interface area includes a first interface screen representing a list of at least one broadcast provider's real-time streaming broadcast video or a video on demand (VOD) video, and a second interface screen for setting the virtual private space,
And the first and second graphic screens and the first and second interface screens are embodied as virtual reality objects (VR objects) in the virtual private space. The method of claim 1,
And displaying a virtual reality effect object corresponding to the generated event in the virtual personal space when an event having a predetermined condition is generated in relation to the live streaming broadcast of the broadcast provider.
The event of the preset condition,
When a broadcast viewer watching the live streaming broadcast purchases a paid item for the broadcast provider or a broadcast contribution having a sponsored contribution indicating a degree to which the paid item is purchased, the broadcast viewer participates in the chat of the live streaming broadcast. A method for providing a virtual reality user interface for viewing a broadcast, comprising an event. The method of claim 1,
Receiving second virtual reality image data representing a virtual private space of the broadcast provider according to a user's request; And
The method may further include displaying the received second virtual reality image data on the display. The virtual personal space of the broadcast provider is according to claim 3,
A method for providing a virtual reality user interface for broadcasting viewing provided by the broadcast provider in different forms for each user. The method of claim 1, wherein the first graphic screen,
Composed of a plurality of screens,
And each of the plurality of screens is configured to play the real-time streaming broadcast image. Communication interface;
display; And
A processor coupled with the communication interface and the display,
The processor,
Receiving first virtual reality image data representing a virtual personal space implemented in virtual reality to watch a live streaming broadcast of a broadcast provider from a broadcast management server through the communication interface,
And display the received first virtual reality image data through the display,
The virtual private space includes a main interface area for viewing the live streaming broadcast and a sub interface area related to a plurality of functions related to viewing the live streaming broadcast,
The main interface area includes a first graphic screen for playing a live streaming broadcast image and a second graphic screen for showing a real time chat between broadcast viewers simultaneously watching the live streaming broadcast,
The sub interface area includes a first interface screen representing a list of at least one broadcast provider's real-time streaming broadcast video or a video on demand (VOD) video, and a second interface screen for setting the virtual private space,
The first and second graphic screens and the first and second interface screens are implemented as virtual reality objects (VR objects) in the virtual private space, and a head mounted display (HMD) providing a virtual reality user interface for broadcasting viewing. ) Device. The method of claim 6,
The processor is further configured to display a virtual reality effect object corresponding to the generated event in the virtual personal space when an event of a predetermined condition is generated in relation to the live streaming broadcast of the broadcast provider,
The event of the preset condition,
When a broadcast viewer watching the live streaming broadcast purchases a paid item for the broadcast provider or a broadcast contribution having a sponsored contribution indicating a degree to which the paid item is purchased, the broadcast viewer participates in the chat of the live streaming broadcast. An HMD device providing a virtual reality user interface for viewing a broadcast, comprising an event. The method of claim 7, wherein the processor,
Receive second virtual reality image data representing a virtual private space of the broadcast provider according to a user's request,
And displaying the received second virtual reality image data on the display, providing a virtual reality user interface for broadcasting viewing. The method of claim 8, wherein the virtual private space of the broadcast provider,
An HMD device for providing a virtual reality user interface for broadcasting viewing provided by the broadcasting provider in different forms for each user. The method of claim 6, wherein the first graphic screen,
Composed of a plurality of screens,
And each of the plurality of screens is configured to play the real-time streaming broadcast image. delete delete delete delete

Images