KR20200041839A - Method for providing virtual reality userinterface for viewing broadcastand apparatus using the same - Google Patents

Abstract

According to one embodiment of the present invention, provided are a method and an apparatus for providing a user interface for viewing a virtual reality broadcast. The method for providing the user interface for viewing the virtual reality broadcast comprises the steps of: receiving first virtual reality image data representing a virtual personal space implemented in virtual reality to view a real-time streaming broadcast of a broadcast provider from a broadcast management server; displaying the received first virtual reality image data through a display; and displaying a virtual reality effect object corresponding to the generated event in the virtual personal space when an event of a preset condition occurs in relation to the real-time streaming broadcast of the broadcast provider.

Description

METHOD FOR PROVIDING VIRTUAL REALITY USER INTERFACE FOR VIEWING BROADCAST AND METHOD FOR METHOD FOR PROVIDING VIRTUAL REALITY USERINTERFACE FOR VIEWING BROADCASTAND APPARATUS USING THE SAME

The present invention relates to a method and apparatus for providing a virtual reality user interface for broadcast viewing.

Generally, virtual reality (VR) refers to a specific environment or situation in a virtual environment that is created to enable users to experience spatial and temporal experiences similar to the real world. A user may immerse in virtual reality using a device providing virtual reality or interact with an object implemented in virtual reality through manipulation or command of the device.

Recently, as functions of various information and communication devices including the Internet of Things have been developed, virtual reality-related technologies have been commercialized, and supply and demand for virtual reality-related content has also increased due to the spread of various popular HMD (Head Mounted Display) equipment. .

Furthermore, as the spread of the Internet spreads rapidly, the number of broadcast viewers watching Internet broadcasts that combine communication and broadcasting to transmit broadcast image data through the Internet is rapidly increasing, and accordingly, virtual reality and Internet broadcasting are combined. Content supply and demand is also increasing.

Accordingly, there is a need for a method for providing a user with a variety of broadcast viewing experiences by providing virtual reality broadcasting contents in which virtual reality and internet broadcasting are combined in various forms.

The problem to be solved by the present invention is to provide a method and apparatus for providing a virtual reality user interface for viewing a broadcast.

Specifically, a problem to be solved by the present invention is to provide a method and apparatus for providing a virtual reality user interface implemented so that a user can have a wider variety of experiences for broadcasting and viewing.

The problems of the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the problems as described above, a method and apparatus for providing a user interface for viewing a virtual reality broadcast according to an embodiment of the present invention are provided. A method of providing a user interface for viewing a virtual reality broadcast includes: receiving first virtual reality image data representing a virtual personal space implemented in virtual reality to view a real-time streaming broadcast of a broadcast provider from a broadcast management server; Displaying the received first virtual reality image data through a display; And displaying a virtual reality effect object corresponding to the generated event in the virtual personal space when an event of a preset condition occurs in relation to the real-time streaming broadcast of the broadcast provider.

The apparatus for providing a user interface for viewing a virtual reality broadcast includes a communication interface, a display, and a processor connected to the communication interface and the display, wherein the processor is capable of real-time streaming broadcast of a broadcast provider from a broadcast management server through the communication interface To receive the first virtual reality image data representing a virtual personal space implemented in virtual reality to watch, display the received first virtual reality image data through the display, and related to the real-time streaming broadcast of the broadcast provider When an event with a preset condition occurs, a virtual reality effect object corresponding to the generated event is displayed on the virtual personal space.

Specific details of other embodiments are included in the detailed description and drawings.

The present invention enables a user to experience a variety of spatial and temporal experiences when watching a broadcast by watching a real-time streaming broadcast image provided by a broadcast provider through the virtual personal space of the user implemented in virtual reality.

The effects according to the present invention are not limited by the contents exemplified above, and more various effects are included in the present specification.

1 and 2 are schematic diagrams for explaining a virtual reality broadcasting system according to an embodiment of the present invention.
3 and 4 are schematic diagrams for describing a user terminal according to an embodiment of the present invention.
5 is a schematic diagram illustrating a broadcast providing apparatus according to an embodiment of the present invention.
6 is a schematic diagram for explaining a broadcast management server according to an embodiment of the present invention.
7 is a schematic flowchart illustrating a method of providing a user interface for viewing a virtual reality broadcast in a user terminal according to an embodiment of the present invention.
8 is an exemplary diagram for describing first virtual reality image data representing a virtual personal space of a user according to an embodiment of the present invention.
9A and 9B are exemplary views for describing graphic screens of a main interface area according to an embodiment of the present invention.
10A, 10B, 11A, 11B, 11C, 11D, 11E, 11F, 11G, 12, 13, and 14 show interface screens of a sub-interface area according to an embodiment of the present invention. These are example diagrams for explanation.
15 to 17 are exemplary views for describing second virtual reality image data representing a virtual personal space of a BJ according to an embodiment of the present invention.
18 is an exemplary diagram for describing first virtual reality image data representing a virtual personal space of a user according to an embodiment of the present invention.
19 is an exemplary diagram for explaining an interface screen related to a friend search icon in a sub-interface area according to an embodiment of the present invention.
20 is an exemplary diagram for describing first virtual reality image data representing a virtual personal space of a user according to an embodiment of the present invention.

Advantages and features of the present invention, and methods for achieving them will be clarified with reference to embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains. It is provided to completely inform the person having the scope of the invention, and the present invention is only defined by the scope of the claims.

Although the first, second, etc. are used to describe various components, it goes without saying that these components are not limited by these terms. These terms are only used to distinguish one component from another component. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical spirit of the present invention.

The same reference numerals refer to the same components throughout the specification.

Each of the features of the various embodiments of the present invention may be partially or totally combined with or combined with each other, and technically various interlocking and driving may be possible as those skilled in the art can fully understand, and each of the embodiments may be independently implemented with respect to each other. It can also be implemented together in an association relationship.

In this specification, BJ (Broadcasting Jockey) may be a broadcast content provider including all users who broadcast on the Internet. For example, BJ may be a broadcast streamer.

In this specification, the virtual reality device may include any device configured to provide a virtual reality image. For example, the virtual reality device may be a head mounted display (HMD) device that is contacted / weared on a user's body part and outputs a virtual reality image. For example, the virtual reality device may include an image output device that outputs a virtual reality image when a display is not provided.

In this specification, an operation in which at least one of various graphic screens, interface screens, and graphic objects is displayed in a virtual reality space (eg, a virtual personal space) is performed by the HMD device in a virtual reality space and various graphic screens in the virtual reality space. It may be an operation of displaying a virtual reality image implemented to display at least one of the fields, interface screens, and graphic objects through a display.

In the present specification, the operation of the user visiting another user's virtual personal space may be an operation in which the HMD device displays a virtual reality image expressed or implemented as a user moves to another user's virtual personal space through a display.

In this specification, the virtual reality image data may be an image obtained by rendering a horizontal and vertical 360-degree display line direction as a 3D graphic centering on a user. In addition, the virtual reality data renders only a partial area centering on the user's gaze, and when the user's gaze changes, the screen area corresponding to the changed gaze is predicted through analysis of the surrounding area, and the user's changed gaze based on the predicted information. It may be an image that renders a screen area corresponding to. The virtual reality image data may be implemented in various ways in addition to the above. In various embodiments, the virtual reality image data may be an image of a first person view or a third person view.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 and 2 are schematic diagrams for explaining a virtual reality broadcasting system according to an embodiment of the present invention.

Referring to FIG. 1, the virtual reality broadcasting system is connected to a first HMD device 100 and a first HMD device 100 that a broadcast viewer can mount for broadcast viewing. The electronic device 200 delivered to the device 100, the broadcast providing device 300 of the BJ providing streaming broadcasting, the imaging device 400, the second HMD device 410 on which the BJ can be mounted, and the broadcast image data of the BJ It may include a broadcast management server 500 to manage the. Here, the virtual reality broadcasting system provides a virtual personal space of the user implemented in virtual reality to watch the real-time streaming broadcast of the BJ, so that the user can watch the real-time streaming broadcast of the BJ or video on demand (VOD) through the virtual personal space. ) Is a system that provides a virtual space for viewing personal broadcasts, such as viewing or decorating a virtual object related to broadcast content by placing it in a user's virtual personal space.

The first HMD device 100 may be a virtual experience device mounted on the user's head to provide a virtual reality image to the user so that the user can experience a spatial and temporal experience similar to that of the real person.

Specifically, the first HMD device 100 is formed in a structure that can be worn on the user's head, and is implemented in a form that performs processing of the virtual reality image inside the first HMD device 100 or the first HMD as shown in FIG. 1B. The image output device 200 for providing a virtual reality image is mounted on a part of the device 100, and may be implemented in a form of performing image processing for virtual reality inside the mounted image output device 200.

When the first HMD device 100 is provided with a display, one surface of the display may be disposed to face the user's face so that the user can check the virtual reality image when wearing the first HMD device 100.

At least one sensor (not shown) for acquiring the user's gaze data may be formed on one side of the first HMD device 100. For example, the at least one sensor may include a gaze tracking sensor that tracks a user's gaze. In various embodiments, at least the gaze tracking sensor is formed at a position where a user's eye or face can be photographed or a position where the user's skin is contactable, and when the user wears the first HMD device 100, the user's eye or face And, by analyzing the captured image, it is possible to obtain the user's gaze data.

The first HMD device 100 may display a virtual reality image corresponding to a user's viewpoint through a display based on the obtained gaze data. For example, when the user wearing the first HMD device 100 moves, the first HMD device 100 detects a change in gaze according to the user's movement through the gaze tracking sensor, and the user's gaze changes according to the change in gaze The virtual reality image corresponding to the viewing direction may be displayed on the display. In various embodiments, the user's viewpoint may be a first person viewpoint or a third person viewpoint.

The first HMD device 100 is connected to an external input device (eg, a controller, etc.) that can be operated, and a user can control the user interface in the virtual reality image by operating the input device connected to the first HMD device 100. have.

In various embodiments, the first HMD device 100 may include a camera module (not shown) for capturing an image according to a user's gaze direction on the other side. Specifically, the camera module may photograph a user's front image according to a user's gaze direction. For example, the photographed image may be an image of an input device that can be operated in front of the user and a user's hand operating the input device. The first HMD device 100 renders a user's hand among the captured front images and displays it on a virtual reality image, or converts a portion of the virtual reality image screen to a see-through area so that the user's hand is displayed. You may. Through this, the user can easily use an input device such as a keyboard or mouse located in the front even when the first HMD device 100 is worn.

The first HMD device 100 may display a first virtual reality image representing a user's virtual personal space implemented as a virtual reality through a display for broadcast viewing. The virtual personal space may include a main interface area for broadcasting viewing of a specific BJ and a sub-interface area related to various functions related to broadcasting viewing. For example, the first virtual reality image may be a composite image of a first layer displaying a background area of a virtual personal space, a second layer displaying a main interface region, and a third layer displaying a sub-interface region.

The main interface area of the user's virtual personal space may include a first graphic screen for reproducing BJ's real-time streaming broadcast and a second graphic screen representing real-time chat between broadcast viewers watching the real-time streaming broadcast simultaneously. In various embodiments, the main interface area may further include a third graphic screen. In various embodiments, a plurality of first graphic screens may be implemented. Specifically, the first HMD device 100 is wirelessly or wiredly connected to the electronic device 200 to receive chat data between broadcast viewers watching live streaming video and live streaming broadcast of BJ from the electronic device 200 The received real-time streaming broadcast image may be displayed on the first graphic screen, and the received chat data may be displayed on the second graphic screen. The second graphic screen may include a chat display area indicating chat content of broadcast viewers and a chat input area for a user to input a chat.

The sub-interface area of the user's virtual personal space includes a first interface screen indicating a video list or a VOD video list related to real-time streaming broadcast of at least one BJ, and a product related to at least one function for viewing a broadcast or a personal virtual space. 2 may include an interface screen. The first interface screen may include a graphic object such as a thumbnail image or a thumbnail video related to a real-time streaming broadcast image of at least one BJ selected by a user or registered as a favorite. The second interface screen may include a graphic object such as a thumbnail image or a thumbnail video related to a VOD image of at least one BJ selected by a user or registered as a favorite. The third interface screen includes a function for setting a graphic screen for playing a broadcast video, a function for setting a real-time broadcast list, a function for setting a VOD video list, and a sponsoring function related to a specific BJ (eg, a fee for BJ) Graphic items, such as menu items related to various functions for setting an item's virtual personal space, or the like. For example, various functions for setting the virtual personal space include a function for setting the number, size, skin, type or resolution of the first graphic screen, and the size or on / off of the second graphic screen. It may include a function for setting a main interface area, such as a function for setting. The various functions include a function for configuring a first interface screen such as a function for selecting a BJ for viewing a real-time streaming broadcast and a function for configuring a real-time streaming video list, a function for selecting a BJ for viewing a VOD video, and a VOD A function for configuring a second interface screen may be further included, such as a function for configuring a video list. The various functions include a purchase or gift of a paid item for viewing a broadcast, a purchase or gift of a paid item for decorating an avatar or a virtual personal space, and a function for arranging an avatar or placing a paid item for purchase or gift in a virtual personal space. Likewise, a function related to the third interface screen may be further included.

In various embodiments, the sub-interface area may be displayed on a fourth interface screen indicating a community space (eg, a bulletin board, etc.) related to broadcasting and viewing, a fifth interface screen indicating a history of purchasing or receiving paid items, or at least one favorite BJ. It may further include a sixth interface screen showing a list for the. For example, the fourth interface screen may display announcements or bulletin boards of the user's virtual personal space when the user is located in his or her virtual personal space, and BJ's virtual personal when the user visits the BJ's virtual personal space It may indicate a space announcement or a bulletin board. In various embodiments, the fourth interface screen may represent a community space selected by the user.

The first HMD device 100 may display a virtual reality effect object corresponding to the generated event in a virtual personal space when an event of a preset condition occurs in relation to real-time streaming broadcast. The virtual reality effect object is a three-dimensional space rendering effect, and may further include a voice effect such as background music or sound effects. For example, in the event of a preset condition, a user purchases a paid item for sponsoring BJ, or a user with high sponsorship contribution indicating the degree of using the sponsoring function related to BJ participates in a chat, or advertising content set by BJ It may include an event using the. When there is a selection of a menu icon corresponding to a sponsoring function for purchasing a paid item related to BJ through the third interface screen, the first HMD device 100 displays an item purchase interface screen for purchasing a paid item, and purchases the item When a specific paid item is purchased through the interface screen, a virtual reality effect object implemented in virtual reality may be displayed in the virtual personal space in response to the purchased paid item. In various embodiments, the first HMD device 100 may display a virtual reality effect object corresponding to the paid item in the user's virtual personal space even when another user purchases the paid item. The virtual reality effect object may be displayed in the virtual personal space of the user as well as in the virtual personal space of viewers watching the broadcast of BJ.

In various embodiments, the first HMD device 100 may be configured as a user as a visitor in response to a request to visit a virtual personal space (eg, virtual personal space) of another user (eg, BJ) in addition to the user's personal virtual space. The second virtual reality image data implemented to experience the virtual personal space may be received from the broadcast management server 400 and the received second virtual reality image data may be displayed through a display. For example, the second virtual reality image data is in the first layer in which the background area of the virtual private space set by the BJ is displayed, in the second layer in which the main interface area of the virtual private space of the BJ is displayed, and in the virtual private space of the BJ The third layer in which the related sub-interface region is displayed may be synthesized image data. The sub-interface area may include an interface screen for various functions that can be provided to a user in relation to the virtual personal space of BJ.

The second virtual reality image data may include a main interface region for reproducing the broadcast image set by the BJ and a sub-interface region related to various functions of the BJ's personal broadcast space visited by the user. The main interface area of the virtual personal space of the BJ visited by the user is for playing a broadcast video set by the BJ (for example, a real-time streaming broadcast of the BJ, a broadcast replay video, a highlight video, or a VOD video produced by a viewer). A first graphic screen and a second graphic screen representing a chat related to the corresponding broadcast video may be included. When the broadcast video is real-time streaming broadcast video data, real-time chat data is displayed through the second graphic screen, and when the broadcast video is VOD video data, stored chat data corresponding to the corresponding VOD video may be displayed through the second graphic screen. have. In various embodiments, the main interface area may include a third graphic screen representing a community space (eg, announcements, bulletin boards, etc.) related to BJ's broadcast, or a broadcast replay video of BJ, a highlight video, or a VOD video produced by a viewer. A fourth graphic screen indicating a list of images may be further included. For example, the community space of the third graphic screen may be a bulletin board of the virtual personal space of the user selected by the user, or a bulletin board of the virtual personal space of the user's favorite BJ.

The sub-interface area of the virtual personal space of the BJ visited by the user includes a first interface screen showing a video list related to real-time streaming broadcast of the BJ set by the user, a second interface screen showing a VOD video list of the BJ set by the user, and A third interface screen related to various functions of the virtual personal space of the BJ may be included. For example, various functions of BJ's virtual personal space may include BJ-related sponsoring functions (for example, purchase of paid items related to BJ's broadcast and gift of paid items for decorating BJ's avatar or virtual personal space). have. In various embodiments, the second virtual reality image data may represent a virtual personal space set for each user class by the BJ according to the degree to which the user has used a sponsorship function such as a paid item purchase or gift related to the BJ (eg, sponsorship contribution). . When the user visits another user's virtual personal space, the sub-interface area is moved together in response to the user, and some functions of the sub-interface area are deactivated in another user's virtual personal space, or some interface screens in the sub-interface area are It can be disabled or hidden.

The electronic device 200 may transmit broadcast streaming data received from the broadcast providing device 300 or the broadcast management server 500 to the first HMD device 100 connected by wire / wireless. The electronic device 200 transmits various requests from the first HMD device 100 to the broadcast providing device 300 or the broadcast management server 500, and responds to various requests to the broadcast providing device 300 or broadcast It can be received from the management server 500 and delivered to the first HMD device 100.

The electronic device 200 receives first virtual reality image data representing a virtual personal space of a user implemented in virtual reality for broadcast viewing from the broadcast providing device 300 or the broadcast management server 500, and receives the received 1 The virtual reality image data may be transmitted to the first HMD device 100.

In various embodiments, the electronic device 200 transmits the request to the broadcast management server 500 according to a request to visit another user's virtual private space, and transmits the virtual private space of another user from the broadcast management server 500. After receiving the second virtual reality image data implemented to be experienced as a visitor, the received second virtual reality image data may be transmitted to the first HMD device 100.

In various embodiments, the electronic device 200 may include a display to display broadcast data (eg, streaming data or VOD data) received from the broadcast providing device 300 or the broadcast management server 500 through a display. When a request for viewing a virtual reality image is received from the first HMD device 100 while displaying broadcast data on the screen as described above, the electronic device 200 transmits the request to the broadcast management server 500 and responds to the request. In response, the first virtual reality image data may be received from the broadcast management server 500. The electronic device 200 may transmit the first virtual reality image data received from the broadcast management server 500 to the first HMD device 100. For example, the request for viewing the virtual reality image may be a detection signal indicating that the user is equipped with the first HMD device 100. When the detection signal is received from the first HMD device 100, the electronic device 200 transmits the request signal to the broadcast management server 500, and receives the first virtual reality image data from the broadcast management server 500. The first virtual reality image data may be transferred to the first HMD device 100. At this time, the electronic device 200 may stop displaying broadcast streaming data through the display.

In various embodiments, the electronic device 200 stores a program or an application capable of generating virtual reality image data (eg, first or second virtual reality image data) for broadcast viewing, and the first HMD device 100 After generating the virtual reality image data by executing the program or application according to a request from), the generated virtual reality image data may be transmitted to the first HMD device 100.

The broadcast providing device 300 receives at least one user terminal (for example, the first HMD device 100 or the electronic device) connected to the broadcast providing device 300 to broadcast data (eg, real-time streaming data) at the request of the BJ. 200)) (eg, broadcasting) or may be delivered to the broadcast management server 500. For example, the broadcast providing device 300 records a video signal input through the photographing device (for example, an image camera) 400 using a broadcast platform (for example, a broadcast program or application) for broadcast, and records The video signal may be transmitted to at least one user terminal or the broadcast management server 500 through real-time streaming.

In various embodiments, the broadcast providing device 300 is connected to the second HMD device 410 or the video output device (not shown) provided in the second HMD device 410, and the BJ connects the second HMD device 410. When worn, the third virtual reality image data representing the virtual personal space of the BJ embodied in virtual reality is received from the broadcast management server 500 to the second HMD device 410 or the second HMD device 410 worn by the BJ. It can be delivered to the provided video output device.

The video output device provided in the second HMD device 410 or the second HMD device 410 worn by the BJ may display third virtual reality video data received from the broadcast providing device 300 through a display. The virtual personal space of the BJ may include a main interface area for reproducing the broadcast image set by the BJ and a sub-interface area related to various functions for setting the virtual personal space of the BJ. For example, the third virtual reality image data includes a first layer in which the background area of the BJ's virtual private space is displayed, a second layer in which the main interface area for the BJ's virtual private space is displayed, and a virtual private space in the BJ. The third layer displaying sub-interface areas related to various functions may be synthesized image data.

The main interface area of the virtual personal space of the BJ includes a first graphic screen for reproducing a broadcast video such as a BJ broadcast replay video, a highlight video, or a VOD video produced by a viewer, and chat data corresponding to each of the video It may include a second graphic screen indicating. In various embodiments, the main interface area of the virtual personal space of the BJ is a third graphic screen representing a community space (eg, announcements, bulletin boards, etc.) related to the broadcast of the BJ, or a broadcast replay video of the BJ, a highlight video, and a viewer A fourth graphic screen indicating a list of images such as the produced VOD image may be further included. In various embodiments, at least two of the first graphic screen may be reproduced so that at least two of a real-time streaming broadcast video selected by BJ, a broadcast replay video, a broadcast highlight video, and a VOD produced by a viewer may be played.

The sub-interface area of the virtual personal space of the BJ includes a first interface screen for setting playback of a real-time streaming broadcast video, a broadcast replay video, a broadcast highlight video, a viewer VOD video, or a real-time streaming broadcast area and a virtual personal space of the BJ. A second interface screen related to various functions for setting may be included. For example, various functions for setting the virtual personal space of the BJ include a function for setting the number, size, type or resolution of the first graphic screen, and the size or on / off of the second graphic screen. A function to set, a function to set the size, arrangement, structure, etc. of the third graphic screen, a function to set the size of the fourth graphic screen, the number of video lists, a list display method, an avatar or a personal broadcasting space It may include a function such as purchase or gift of a paid item for decorating, and a function for placing a purchased or gifted paid item in a virtual personal space.

The broadcast management server 500 receives broadcast data (eg, real-time streaming data) of BJ from the broadcast providing device 300, and the received broadcast data is at least one user terminal connected to the broadcast management server 500 (eg It can be delivered to the first HMD device 100 or the electronic device 200.

The broadcast management server 500 is a first virtual reality representing a virtual personal space of a user implemented in virtual reality to watch a real-time streaming broadcast received from the broadcast providing device 300 according to a request from at least one user terminal The image data may be generated and the generated first virtual reality image data may be delivered to at least one user terminal. For example, the broadcast management server 500 may include a first layer in which a background area of a user's virtual personal space is displayed, a second layer in which a main interface area for broadcast viewing is displayed, and sub-interfaces related to various functions related to broadcast viewing. The first virtual reality image data may be generated by synthesizing the third layer in which the region is displayed.

In various embodiments, the broadcast management server 500 may transmit second virtual reality image data representing a virtual personal space of the BJ to at least one user terminal according to a request from at least one user terminal. The second virtual reality image data may include various interface areas configured to allow a user to experience the virtual personal space of BJ as a visitor. For example, the second virtual reality image data is related to the first layer in which the background region of the virtual personal space of BJ is displayed, the second layer in which the main interface region of the virtual personal space of BJ is displayed, and the virtual personal space of BJ. The third layer displaying sub-interface areas related to various functions that may be provided to the user may be synthesized image data.

In various embodiments, the virtual personal space of BJ may be differently displayed for each user class. Specifically, the broadcast management server 500 may classify users in order from a high rating (eg, first rating) to a low rating (eg, third rating) according to the user's contribution contribution. The broadcast management server 500 may provide a setting interface screen for the BJ to set different virtual personal spaces for each user class. For example, when the first user visits the virtual personal space of the BJ, the broadcast management server 500 checks the patronage contribution level of the first user, and when the first user is classified as the '1st grade', the BJ ' The second virtual reality image data representing the virtual personal space set by the BJ may be provided to a user classified as 'Class 1'. The virtual personal space of BJ provided to the user of the first class is the number, size and resolution of the virtual personal space and the graphic screen of the BJ provided to the user of the third class, the structure of the virtual personal space, the background skin, and the virtual reality The arrangement and type of objects may be different.

In various embodiments, the virtual private space of the BJ includes a virtual private space corresponding to a first chat room (for example, a main room) in which a first number of broadcast viewers participates and a second chat room in which a second number of broadcast viewers participate (for example, a relay) Room) may be provided. For example, the broadcast management server 500 sets the number of people who can visit the virtual personal space of the BJ as the first number of people, and when the broadcast viewer of the first number of people visits the virtual personal space of the BJ, the broadcast of the first number of people Provide second virtual reality image data representing the first virtual personal space of the BJ set by the BJ to the viewer, and provide second virtual reality image data representing the second virtual personal space of the BJ set by the BJ to other broadcast viewers. Can provide. The first virtual personal space of the BJ and the second virtual personal space of the BJ may be set to be the same as each other by the BJ, but may be set differently.

In various embodiments, the broadcast management server 500 generates third virtual reality image data representing the virtual personal space of the BJ implemented in virtual reality according to a request from the broadcast providing device 300, and the generated third virtual reality The image data may be transmitted to the broadcast providing device 300. For example, the broadcast management server 500 is related to the first layer in which the background region of the virtual personal space of BJ is displayed, the second layer in which the main interface region for the virtual personal space of BJ is displayed, and the virtual personal space of BJ. The third virtual reality image data may be generated by synthesizing the third layer displaying the sub-interface areas related to various functions that can be provided to the BJ. In various embodiments, the virtual personal space of the BJ provided to the user and the virtual personal space provided to the BJ may be different.

In various embodiments, when the virtual reality image data is in the first person view, the user's appearance implemented in the virtual reality may be implemented in the form of an avatar seen from the first person view, or may be provided in a form of rendering the user's actual appearance.

In various embodiments, when the virtual reality image data is in the third person view, the user's appearance implemented in the virtual reality may be implemented as an avatar-shaped 3D graphic object. In this case, the user's avatar may be provided to the user as a predetermined avatar, or one of various types of avatars may be selected by the user. In various embodiments, the user's avatar is provided in the form of an avatar desired by the user, such as selecting a shape of the face, eyes, nose, mouth, ears, selecting a head shape, a hair color, or selecting a shape of a body part. Can be. In various embodiments, the user's avatar may be provided in a form of rendering the user's actual appearance. For example, the user's face may be rendered or the entire body of the user may be rendered.

Next, referring to FIG. 2, the virtual reality broadcasting system is mounted on the first HMD device 100, the broadcast providing device 300, the broadcast management device 500, and the first HMD device 100, and the broadcast management device ( It may include an image output device 600 for outputting the virtual reality image data received through (500).

The video output device 600 is a device that provides a user interface to allow a broadcast viewer (ie, a user) to watch BJ's broadcast online, and may be a smart phone, a tablet PC, or the like having an application or program for viewing the broadcast installed. . The application or program may connect the broadcast management server 500 and at least one user terminal (eg, the first HMD device 100 or the video output device 600) to each other. Broadcast viewers can communicate with each other through a communication area such as a chat window or a bulletin board while watching a broadcast together with other broadcast viewers using an application online.

The video output device 600 displays first virtual reality video data representing a virtual personal space of a broadcast viewer implemented in virtual reality for broadcast viewing according to a request of the broadcast viewer through a display of the video output device 600 You can. Specifically, the video output device 600 transmits a request to the broadcast management server 500 when there is a request to watch a virtual reality broadcast from a broadcast viewer through an application, and the first virtual reality video from the broadcast management server 500 Data can be received. The image output device 600 may display the received first virtual reality image data through a display.

In various embodiments, the video output device 600 displays the second virtual reality data including the various interface areas implemented to experience the virtual personal space of the BJ according to the request of the broadcast viewer, to display the video output device 600. It can be displayed through. Specifically, the video output device 600 transmits a request to the broadcast management server 500 when there is a request from the broadcast viewer to visit the virtual personal space of the BJ through the application, and the second virtual from the broadcast management server 500 Real image data can be received. For example, the second virtual reality image data may represent a virtual personal space set by the BJ according to the sponsored contribution level of the broadcast viewer. The image output device 600 may display the received second virtual reality image data through a display.

The image output device 600 receives the user's gaze data from the first HMD device 100, and based on the received gaze data, the virtual reality image data corresponding to the user's viewpoint (eg, the first or second virtual Reality image data).

In various embodiments, the image output device 600 may include a camera module for photographing an image according to a user's gaze direction. For example, the camera module is provided on the rear side of the image output device 600 (eg, on the opposite side of the front side where the display is disposed) to capture a user's front image. The photographed front image may be an image of an operable input device located in front of the user and a user's hand operating the input device. The image output device 600 may display the photographed front image in a pop-up form on the virtual reality image screen, or may convert and display some areas of the virtual reality image screen as a see-through area.

3 and 4 are schematic diagrams for describing a user terminal according to an embodiment of the present invention.

Referring to FIG. 3, the user terminal may be a first HMD device 100 having a display. The first HMD device 100 includes a communication interface 110, a display 120, a memory 130, a sensor 140, and a processor 150. In various embodiments, the first HMD device 100 may be connected to the input device 550 through wired / wireless communication.

The communication interface 110 connects the first HMD device 100 to communicate with an external device. The communication interface 110 may connect to the electronic device 200 using wired / wireless communication. The communication interface 110 may transmit a request for real-time broadcast viewing of the BJ to the electronic device 200 and receive real-time streaming broadcast video data of the BJ from the electronic device 200. The communication interface 110 transmits a request for first virtual reality image data representing a virtual personal space implemented in virtual reality to the electronic device 200 for broadcast viewing, and transmits a first virtual reality image from the electronic device 200. Data can be received. The communication interface 110 may transmit a request for a visit of another user's virtual personal space to the electronic device 200 and receive second virtual reality image data representing the virtual personal space of the BJ from the electronic device 200. have. In various embodiments, the communication interface 110 may transmit the user's gaze data obtained through the sensor 140 to the electronic device 200.

The display 120 may display various contents (eg, text, images, videos, icons, banners or symbols) to the user. Specifically, the display 120 may display broadcast data (eg, real-time streaming video or VOD video), first virtual reality video data, or second virtual reality video data received through the communication interface 110.

The memory 130 may store various data used to display virtual reality image data. Specifically, the memory 130 may store the first or second virtual reality image data, and may store an application, program, or widget used to display the first or second virtual reality image data. Also, the memory 130 may store gaze data acquired through the sensor 140.

The sensor 140 may include a gaze tracking sensor that tracks a user's gaze, and may output a detection signal by detecting a user's gaze through the gaze tracking sensor. For example, the gaze tracking sensor may include a camera sensor that photographs the user's eyes or face, and analyze the image captured through the camera sensor to output gaze data indicating position coordinates of the user's gaze. In various embodiments, various methods may be used to track a user's gaze, and in addition to image analysis, a contact lens method (a gaze tracking method using reflected magnetic light of a mirror built-in contact lens or a magnetic field of a coil built-in contact lens) or sensor attachment The user's gaze may be tracked using a method (a gaze tracking method using an electric field according to eye movement by attaching a sensor around the eye).

The processor 150 is operatively connected to the communication interface 110, the display 120, the memory 130, and the sensor 140, the first virtual reality image data representing the user's virtual personal space or the virtual person of the BJ Various commands for displaying the second virtual reality image data representing the space may be performed.

Specifically, the processor 150 executes an application for viewing a broadcast according to an application execution request for a user's broadcast viewing, and when a request for viewing a virtual reality broadcast is received, the electronic device 200 through the communication interface 110 A request for viewing a virtual reality broadcast can be delivered. The processor 150 may receive first virtual reality image data representing a virtual personal space of a user implemented in virtual reality for viewing virtual reality broadcasts from the electronic device 200 and display it through the display 120.

The virtual personal space of the user may include a main interface area for broadcasting viewing of a specific BJ that the user wants to watch, and a sub-interface area related to various functions related to broadcasting viewing. The main interface area may include a first graphic screen for reproducing BJ's real-time streaming broadcast and a second graphic screen representing real-time chat between broadcast viewers watching the real-time streaming broadcast simultaneously. In various embodiments, the main interface area may further include a third graphic screen indicating a community space (eg, a bulletin board, etc.) related to broadcast viewing. For example, the third graphic screen may be a bulletin board of BJ's virtual personal space related to BJ's broadcast viewing. In various embodiments, the first graphic screen may include a plurality of playback screens. For example, the plurality of playback screens may include one main playback screen and one or more sub playback screens. The processor 150 may play the first real-time streaming broadcast data provided by a specific BJ through the main playback screen, and play the second real-time streaming broadcast data provided by a specific BJ through one or more sub-play screens. The sub-interface area includes a first interface screen showing a video list related to at least one BJ real-time streaming broadcast, a second interface screen showing a VOD video list of a specific BJ, and at least one function for viewing a broadcast or a personal virtual space It may include a third interface screen related to.

The processor 150 may display a virtual reality effect object corresponding to the generated event in the virtual personal space when an event of a preset condition occurs in relation to the real-time streaming broadcast. The event of the preset condition is an event in which a user purchases a paid item related to BJ, an event in which a user with high sponsorship contribution to BJ participates in a chat, or a user uses advertising content set by BJ or uses advertising content It may include an event participating in the chat of the real-time streaming broadcast. For example, when the paid item is purchased by the user, the processor 150 may display the virtual reality effect object corresponding to the paid item in the virtual personal space. The virtual reality effect object may include a 3D graphic object and voice data. In addition, when the user participates in the chat, if the user's contribution to the BJ corresponds to a specific sponsored contribution, the virtual reality effect object corresponding to the specific sponsored contribution may be displayed in the virtual personal space. In addition, when a user uses advertisement content related to broadcasting of BJ while watching real-time streaming (for example, selecting an advertisement banner or an image, participating in an advertisement-related event, etc.), the processor 150 virtualizes a corresponding virtual reality effect object. You can mark it in your personal space. The above-described virtual reality effect objects may be simultaneously displayed in a virtual personal space of a plurality of broadcast viewers watching a broadcast of BJ or a virtual personal space of a BJ visited by a plurality of broadcast viewers.

In various embodiments, the processor 150 may detect a change in gaze according to a user's movement through the sensor 140 and display at least a portion of virtual reality image data corresponding to a direction in which the user's gaze is viewed through a display. have. In various embodiments, the processor 150 receives virtual reality image data corresponding to various gaze directions of the user from the electronic device 200 or receives some virtual reality image data corresponding to the user's gaze and the user's gaze When changed, the other part of the virtual reality image data corresponding to the changed gaze may be received from the electronic device 200.

In various embodiments, the processor 150 may display a virtual keyboard area for chatting input at a specific location on the virtual reality video screen when a request for chatting input is performed through the input device 550. The virtual keyboard area may include a plurality of key areas for inputting one or more letters and numbers. When the selection related to a specific key area of the virtual keyboard area is detected through the input device 550, the processor 150 inputs a letter or number corresponding to the selected key area as an input value through the chat input area Alternatively, a number or the like can be displayed in the chat display area.

In various embodiments, when the HMD device 110 includes a camera module (not shown), when the input device 550 for chatting input is located in front of the user, such as a keyboard, the processor 150 is photographed through the camera module The user's hand may be rendered from the front image of the user and displayed on the virtual reality image, or a portion of the virtual reality image screen may be switched to display the user's hand.

In various embodiments, the user's virtual personal space may be implemented so that not only the user but also other users (or other broadcast viewers) invited by the user can participate together. For example, when a user wants to watch a real-time streaming broadcast of a specific BJ together with other users, the processor 150 may provide an interface screen for the user to send an invitation message to other users. Through the interface screen, when a user delivers an invitation message to other users, and when a response message acknowledging the invitation is received from the other users, the processor 150 receives only the user from the broadcast providing server 500 within the user's virtual personal space. In addition, it is possible to receive first virtual reality image data implemented to participate by other users invited by the user.

In various embodiments, the processor 150 may transmit a request to visit the virtual personal space of the BJ to the electronic device 200 through the communication interface 110 when there is a request to visit the virtual personal space of the BJ. The processor 150 may receive second virtual reality image data configured to allow a user to experience a virtual personal space of BJ as a visitor from the electronic device 200 and display it through the display 120. The second virtual reality image data is a virtual interface of the BJ and a main interface area for playing a broadcast image set by the BJ (for example, a real-time streaming image of the BJ, a replay image, a high-light image, or a VOD image produced by a viewer). It may include a sub-interface area related to various functions for the personal space. The main interface area may include a first graphic screen for reproducing a broadcast video set by BJ and a second graphic screen representing chat data related to the broadcast video. In various embodiments, the main interface area may include a third graphic screen representing a community space (eg, announcements, bulletin boards, etc.) related to the broadcasting of BJ or a broadcasting replay image of BJ, a high-light image, a VOD image produced by a viewer, and the like. A fourth graphic screen indicating a list of images may be further included. The sub-interface area is a first interface screen for setting playback of a broadcast replay video, a broadcast highlight video, a viewer VOD video, or a real-time streaming broadcast area, and various functions related to various functions provided to a user in connection with the virtual personal space of the BJ. 2 may include an interface screen. For example, various functions may include a function of purchasing or gifting a paid item for decorating an avatar or a personal broadcasting space, and a function for placing a purchased or gifted paid item in a virtual personal space.

In various embodiments, when a user requests to visit the virtual personal space of the BJ through the input device 550, the processor 150 may be different according to the user level by the BJ according to the user's contribution contribution from the electronic device 200 The second virtual reality image data representing the set virtual personal space may be received. Specifically, the processor 150 transmits a request to visit the virtual personal space of the BJ to the electronic device 200, and represents the virtual personal space set by the BJ corresponding to the user's contribution contribution from the electronic device 200 The second virtual reality image data may be received and displayed through the display 120. For example, a virtual personal space set to be provided to a user with high sponsorship contribution includes a virtual personal space set to be provided to a user with low sponsorship contribution and the number, size and resolution of the graphic screens, the structure of the virtual personal space, the background skin, Arrangement and types of virtual reality objects may be different.

In various embodiments, the processor 150 displays a third interface screen for purchasing at least one paid item related to broadcasting of the BJ according to a user request, and when a specific paid item is selected by the user, payment for the selected paid item You can perform the operation. When payment for the paid item is completed, the processor 150 may display the virtual reality effect object corresponding to the paid item in the virtual personal space of the BJ the user is visiting.

In various embodiments, the processor 150 responds to a specific sponsored contribution when the user contributes to the sponsored contribution to the BJ when the user visits the virtual personal space of the BJ to participate in the chat to watch the live streaming broadcast of the BJ The virtual reality effect object can be displayed in the virtual personal space of the BJ the user is visiting.

In various embodiments, the processor 150, when a user visits a virtual personal space of the BJ and participates in an advertisement event related to the broadcast of the BJ to watch the real-time streaming broadcast of the BJ, the user visits the virtual reality effect object corresponding to the BJ It can be displayed in a virtual private space.

The above-described virtual reality effect objects may be simultaneously displayed in a virtual personal space of a plurality of broadcast viewers watching a broadcast of BJ or a virtual personal space of a BJ visited by a plurality of broadcast viewers.

In various embodiments, the input device 550 is connected to the first HMD device 100 through wired / wireless communication, and may include a sensor (not shown) for sensing a user's grip or sensing a user's movement. . The input device 550 may further include a button to transmit a button input signal according to button input to the first HMD device 100. The first HMD device 100 may display a pointer corresponding to the movement of the input device 550 through the display 120.

The first HMD device 100 may move and display the pointer in the virtual personal space displayed through the display 120 according to the detection signal of the sensor received from the input device 550. Also, the first HMD device 100 may select a specific object (eg, a graphic screen, an interface screen, a virtual reality object, etc.) in the virtual reality space according to the position of the pointer and a button input signal received from the input device 550. have.

Next, referring to FIG. 4, the user terminal may further include an image output device 600 mounted on the first HMD device 100 when the first HMD device 100 does not have a display. The first HMD device 100 includes a communication interface 110, a memory 130, a sensor 140, and a processor 150, and the image output device 600 includes a communication interface 610, a display 620 ), A memory 630 and a processor 640.

First, in relation to the first HMD device 100, the communication interface 110 may connect to the electronic device 200 and the video output device 600 using wired / wireless communication. In addition, the communication interface 110 may transmit the user's gaze data acquired through the sensor 140 to the image output device 600. In various embodiments, the communication interface 110 may connect to the input device 550 using wired / wireless communication.

The memory 130 may store gaze data obtained through the sensor 140.

As illustrated in FIG. 3, the sensor 140 may include a gaze tracking sensor that tracks a user's gaze. For example, the gaze tracking sensor includes a camera sensor (not shown) that photographs a user's eye or face, and analyzes an image captured through the camera sensor to output gaze data indicating position coordinates of the user's gaze. You can.

The processor 150 is operably connected to the communication interface 110, the display 120, the memory 130, and the sensor 140, as shown in FIG. 3, and visualizes the gaze data obtained through the sensor 140. It can be delivered to the output device 200. Specifically, the processor 150 may acquire gaze data according to a user's gaze change through the sensor 140 and transmit the obtained gaze data to the image output device 200.

In various embodiments, when a request for input is received through the image output device 600, the processor 150 performs connection with the input device 550 using wired / wireless communication, and a specific key from the input device 550. The input signal corresponding to the region may be received, and the received input signal may be transmitted to the image output device 600.

Next, in relation to the video output device 600, the communication interface 610 may connect to the first HMD device 100 using wired or wireless communication. The communication interface 610 may connect to the broadcast providing device 300 or the broadcast management device 500 using wireless communication. The communication interface 610 transmits a request for first virtual reality image data representing a virtual personal space implemented in virtual reality for broadcasting viewing to the broadcasting providing device 300 or the broadcasting management server 500, and the broadcasting providing device The first virtual reality image data may be received from the 300 or the broadcast management server 500. The communication interface 610 transmits a visit request for another user's virtual personal space to the broadcast providing device 300 or the broadcast management server 500, and the broadcast interface 610 is different from the broadcast providing device 300 or the broadcast management server 500. The second virtual reality image data representing the virtual personal space of the user may be received. In various embodiments, the communication interface 610 may receive the user's gaze data from the first HMD device 100 or an input signal from the input device 550. In various embodiments, the communication interface 610 may receive a signal indicating that the image output device 600 is mounted on the first HMD device 100 from the first HMD device 100.

The display 620 may display various contents (eg, text, images, videos, icons, banners or symbols) to the user. Specifically, the display 520 may display first virtual reality image data or second virtual reality image data received through the communication interface 610.

The memory 630 may store various data used to display virtual reality image data. Specifically, the memory 630 may store the first virtual reality image data or the second virtual reality image data, and may store an application, program, widget, or the like used to display the first or second virtual reality image data. . Also, the memory 630 may store gaze data received through the first HMD device 100.

The processor 640 is operably connected to the communication interface 610, the display 620, and the memory 630, and the first virtual reality image data representing the user's virtual personal space or the virtual user's virtual user space 2 Various commands for displaying virtual reality image data can be executed.

Specifically, the processor 640 executes an application for viewing a broadcast according to an application execution request according to a user's application execution request, and when a request for viewing a virtual reality broadcast is received, the broadcast management device through the communication interface 610 ( 500), a request for viewing a virtual reality broadcast may be transmitted. For example, the request for virtual reality broadcast viewing may include a case in which a signal indicating that the video output device 600 is mounted on the first HMD device 100 is received from the first HMD device 100. The processor 640 receives first virtual reality image data representing a virtual personal space of a user implemented in virtual reality for viewing virtual reality broadcasts from the broadcast management device 500 and receives the received first virtual reality image data. It can be displayed through the display 620. The first virtual reality image data may be virtual reality image data corresponding to various gaze directions of the user or some virtual reality image data corresponding to the user's gaze direction.

In various embodiments, the processor 640 may display an effect object corresponding to the generated event in the user's virtual personal space when an event of a preset condition occurs in relation to the real-time streaming broadcast. The events of the preset conditions include an event in which a user watching a live streaming broadcast purchases a paid item related to BJ, an event in which a user with a high contribution to BJ participates in a chat in BJ's live streaming broadcast, and a live streaming broadcast. It may include an event in which a viewing user uses advertisement content set by BJ or participates in a live streaming broadcast chat after using advertisement content. For example, when a user watching a real-time broadcast of BJ purchases a paid item related to the real-time broadcast of BJ, the processor 640 displays a virtual reality effect object corresponding to the purchased paid item in the user's virtual personal space. You can. In addition, when a user watching a real-time broadcast of BJ participates in a real-time broadcast chat, if the sponsored contribution of the user corresponds to a specific sponsored contribution, the processor 640 displays the virtual reality effect object corresponding to the specific sponsored contribution by the user's virtual individual. Can be marked in space. In addition, when the user watching the real-time broadcast of the BJ uses the advertising content set by the BJ or participates in the chat of the real-time streaming broadcast after using the advertising content, the processor 640 displays a virtual reality effect object corresponding to the use of the advertising content. It can be displayed in the user's virtual personal space.

In various embodiments, the processor 640 may transmit the request to the broadcast management device 500 through the communication interface 610 when there is a user request to visit the virtual personal space of the BJ through the input device 550. . When the second virtual reality image data set to allow the user to experience the virtual personal space of BJ as a visitor is received from the broadcast management device 500, the processor 640 displays the received second virtual reality image data 620. It can be displayed through. For example, the second virtual reality image data may represent a virtual personal space set differently for each user class by the BJ according to the user's contribution contribution. The second virtual reality image data is a virtual interface of the BJ and a main interface area for playing a broadcast image set by the BJ (for example, a real-time streaming image of the BJ, a replay image, a high-light image, or a VOD image produced by a viewer). It may include a sub-interface area related to various functions that can be provided to a user in relation to the personal space. The main interface area may include a first graphic screen for playing a real-time streaming broadcast set by the BJ and a second graphic screen representing a chat content area and a chat input area of the real-time streaming broadcast when the BJ broadcasts the real-time streaming. have.

In various embodiments, the processor 640 may display a virtual reality effect object corresponding to the generated event in the personal broadcast space of the BJ visited by the user when an event of a preset condition occurs in relation to the real-time streaming broadcast. The event of the preset condition is an event in which a user watching a live streaming broadcast purchases a paid item related to BJ, and an event in which a user with a sponsorship contribution in relation to a BJ participates in a chat in BJ's live streaming broadcast It may include, for example, an event that a user watching a real-time streaming broadcast uses advertisement content set by BJ or participates in a chat of the real-time streaming broadcast after using the advertising content. For example, when a user watching a real-time broadcast of the BJ purchases a paid item related to the real-time broadcast of the BJ, the processor 640 is a virtual person of the BJ who is visiting the virtual reality effect object corresponding to the purchased paid item. Can be marked in space. In addition, when a user watching a real-time broadcast of BJ participates in a real-time broadcast chat, when the sponsored contribution of the user corresponds to a specific sponsored contribution, the processor 640 visits the virtual reality effect object corresponding to the specific sponsored contribution by the user BJ can be displayed in the virtual private space. In addition, when the user watching the real-time broadcast of the BJ uses the advertising content set by the BJ or participates in the chat of the real-time streaming broadcast after using the advertising content, the processor 640 displays a virtual reality effect object corresponding to the participation in the advertising event. It can be displayed in the virtual personal space of the BJ the user is visiting. For example, the operation using the advertisement content may include an operation of selecting an advertisement banner or advertisement image related to broadcasting of BJ, participating in an advertisement event, and the like.

The above-described virtual reality effect objects are a corresponding personal broadcasting space when another broadcast viewer is visiting the BJ's personal broadcasting space to watch the BJ's real-time streaming broadcast or another virtual viewer's virtual streaming space watching the BJ's real-time streaming broadcast together. The same can be displayed at the same time.

5 is a schematic diagram illustrating a broadcast providing apparatus according to an embodiment of the present invention. Referring to FIG. 5, the broadcast providing device 300 may include a communication interface 310, a display 320, a memory 330, and a processor 340. The broadcast providing device 300 may be connected to a second HMD device 410 equipped with a display 400 or a display device for BJ broadcasting.

The communication interface 310 connects the broadcast providing device 300 to communicate with an external device. The communication interface 310 may connect to the electronic device 200, the broadcast management device 500, the imaging device 400, or the second HMD device 410 using wired / wireless communication. The communication interface 310 includes at least one user terminal (eg, the first HMD device 100 or the electronic device) connected to the broadcast providing device 300 for broadcasting data (eg, real-time streaming video or VOD video) of the BJ. 200)) (eg, broadcasting) or may be delivered to the broadcast management server 500. The communication interface 310 may receive an image signal from the imaging device 400 or transmit third virtual reality image data representing the virtual personal space of the BJ to the second HMD device 410. Here, the third virtual reality image data may be a virtual reality image in which the BJ virtual personal space is implemented as a virtual reality to provide to the BJ. The virtual personal space of the BJ provided to the BJ may be different from the virtual personal space of the BJ provided to the user.

The display 320 may display various contents (eg, text, images, videos, icons, banners or symbols) to the user (eg, BJ). Specifically, the display 320 may display an execution screen for a platform (eg, a broadcast program or application) for broadcasting.

The memory 330 may store various data for broadcasting. Specifically, the memory 330 may store a platform (for example, a broadcast program or application) for broadcast transmission, and store a video signal received from the imaging device 400.

The processor 340 is operably connected to the communication interface 310, the display 320, and the memory 330, and can perform various commands for broadcasting. Specifically, the processor 340 records a video signal input through the imaging device 400 using a platform (for example, a broadcast program or application) for broadcast transmission according to an application execution request for broadcast of BJ, and records The video signal may be transmitted to at least one user terminal (eg, the electronic device 200 or the video output device 600) or the broadcast management server 500 through real-time streaming.

In various embodiments, the processor 340 receives a request for the display of the third virtual reality image data representing the virtual personal space of the BJ implemented in virtual reality from the second HMD device 410 through the communication interface 310 and , The request may be delivered to the broadcast management server 500. When the third virtual reality image data is received from the broadcast management server 500, the processor 340 may transmit the received third virtual reality image data to the second HMD device 410. For example, when the signal indicating that the BJ is equipped with the second HMD device 410 is received from the second HMD device 410, the processor 340 may transmit the third virtual reality image data to the broadcast management server 500. You can forward the request.

The second HMD device 410 may include a display (not shown) and display third virtual reality image data received from the broadcast providing device 300 through the display. The virtual personal space of the BJ may include a main interface area for reproducing the broadcast image set by the BJ and a sub-interface area related to various functions for setting the virtual personal space of the BJ. For example, the third virtual reality image data is related to the first layer in which the background region of the virtual personal space of BJ is displayed, the second layer in which the main interface region for the virtual personal space of BJ is displayed, and the virtual personal space of BJ. Thus, the third layer displaying the sub-interface area related to various functions that can be provided to the BJ may be synthesized image data.

The main interface area of the virtual personal space of the BJ represents a first graphic screen for playing a broadcast video such as a BJ broadcast replay video, a high-light video, or a VOD video produced by a viewer, and chat data corresponding to the video It may include a second graphic screen. In various embodiments, the main interface area is a third graphic screen representing a community space (eg, announcements, bulletin boards, etc.) related to BJ's broadcast, or BJ's broadcast replay video, highlight video, viewer-generated VOD video, etc. A fourth graphic screen indicating a list of images may be further included. In various embodiments, a plurality of first graphic screens may be implemented so that at least two or more of a broadcast replay image, a broadcast highlight image, and a VOD produced by a viewer may be reproduced.

The sub-interface area of the virtual personal space of the BJ may be provided to the BJ to set the virtual interface of the BJ and the first interface screen for setting the playback of the broadcast replay video, the broadcast highlight video, or the viewer VOD video. A second interface screen related to the function may be included. For example, various functions for setting the virtual personal space of the BJ include a function for setting the number, size or resolution of the first graphic screen, a function for setting the size or on / off of the second graphic screen, and the like. A function to set the size, arrangement, structure, etc. of a third graphic screen representing a community space, a function to set the size of a fourth graphic screen, the number of video lists, a method of displaying a list, and the like, to decorate an avatar or personal broadcasting space It may include a function such as purchase or gift of a paid item for purchase, and a function for placing a purchased or received paid item in a virtual personal space. Through this, the second HMD device 410 sets the number or size of the first graphic screen through the second interface screen at the user's request, sets the type or resolution of the reproduced video, or sizes or turns on / off the second graphic screen. Set off, etc., set the size, arrangement, structure, etc. of the third graphic screen, set the size of the fourth graphic screen, the number of video lists, list display method, etc., purchase or purchase paid items, etc. Or, you can place paid items that you have purchased or gifted in a virtual personal space.

In various embodiments, the image providing device 300 further stores an application or program for generating the virtual reality image data in the memory 330, and uses the application or program to generate the first, second, and third virtual images. It is also possible to generate reality image data. The generated first and second virtual reality image data may be transmitted to the electronic device 200 or the image output device 600 through the broadcast management server 500.

6 is a schematic diagram for explaining a broadcast management server according to an embodiment of the present invention. Referring to FIG. 6, the broadcast management server 500 may include a communication interface 510, a virtual reality image processing unit 520, a memory 530, and a processor 540.

The communication interface 510 connects the broadcast management server 500 to communicate with an external device. The communication interface 510 is connected to the first HMD device 100, the electronic device 200, the broadcast providing device 300, the second HMD device 410, or the video output device 600 using wired / wireless communication. It can transmit and receive various information. Specifically, the communication interface 510 receives the request of the first virtual reality image data or the request of the second virtual reality image data from the electronic device 200 or the image output device 600, and the electronic device 200 or the image The first virtual reality image data or the second virtual reality image data may be transmitted to the output device 600. The communication interface 510 receives a request for third virtual reality image data from the video output device 600 or the broadcast providing device 300 mounted on the second HMD device 410, and the second HMD device 410 The third virtual reality image data may be transmitted to the image output device 600 or the broadcast providing device 300 mounted on the device. In various embodiments, the communication interface 510 may receive various setting information regarding a user's virtual personal space from the first HMD device 100 or the electronic device 200. In various embodiments, the communication interface 510 may receive various setting information regarding the virtual personal space or virtual personal space of the BJ from the video providing device 300 or the second HMD device 410. In various embodiments, the virtual personal space of the BJ provided to the BJ and the virtual personal space of the BJ provided to the user may be identical to each other, but when the BJ sets different types of virtual personal space for each user, the BJ provided to the user The virtual personal space of may be different for each user, and according to the BJ setting, the virtual personal space of BJ provided to the BJ and the virtual personal space of BJ provided to the user may be different.

The virtual reality image processing unit 520 is a first virtual reality image data representing a virtual personal space implemented in virtual reality so that broadcast viewers can watch the broadcast, and virtual reality so that broadcast viewers can experience the virtual personal space of BJ as a visitor. The second virtual reality image data representing the virtual personal space of the BJ implemented as and third virtual reality image data representing the virtual personal space of the BJ implemented as the virtual reality so as to be provided to the BJ may be generated. Specifically, the virtual reality image processing unit 520 allocates the background area of the user's virtual personal space to the first layer, the main interface area to the second layer, and the sub-interface to generate the first virtual reality image data. The region can be assigned to the third layer. For example, the background area of the virtual personal space may include various virtual reality objects such as wallpaper, floor, floorboard, sofa, chair, table, picture frame, and interior related props, such as a bookcase, to decorate the virtual personal space set by the user. You can. The virtual reality image processing unit 520 may allocate real-time streaming broadcast data received from the broadcast providing device 300 to the second layer to be displayed on the first graphic screen of the main interface area. In various embodiments, when there are two or more first graphic screens, the virtual reality image processing unit 520 may display two or more real-time streaming broadcast data received from the broadcast providing device 300 on the second layer so as to be displayed on each of the two or more first graphic screens. Can be assigned. The virtual reality image processing unit 520 may generate first virtual reality image data obtained by combining the first layer, the second layer, and the third layer.

The virtual reality image processing unit 520 may generate the second virtual reality image data and the third virtual reality image data similarly to those described above with respect to the first virtual reality image data. Specifically, the virtual reality image processing unit 520 allocates the background region of the virtual personal space of the BJ to the first layer, the main interface region to the second layer, and the sub-interface to generate the second virtual reality image data. The region can be assigned to the third layer. The virtual reality image processing unit 520 may generate second virtual reality image data obtained by combining the first layer, the second layer, and the third layer. In addition, the virtual reality image processing unit 520 allocates the background region of the virtual personal space of the BJ to the first layer, the main interface region to the second layer, and the sub-interface region to generate the third virtual reality image data. Can be assigned to the third layer. The virtual reality image processing unit 520 may generate third virtual reality image data obtained by combining the first layer, the second layer, and the third layer.

In various embodiments, the virtual reality image processing unit 520 allows the user and other users invited by the user according to a request for the user to invite specific users through the user terminal, and the invited specific users to participate in the user's virtual personal space. The first virtual reality image data implemented to participate in the user's virtual personal space may be generated. The first virtual reality image data may be embodied so that an avatar in the virtual reality of the user and an avatar in each virtual reality of other invited users together can experience the user as if they are in the virtual personal space of the user. The virtual reality image processing unit 520 changes the number, size, type or resolution of the first graphic screen, or the second graphic screen based on various setting information regarding the user's virtual personal space received from the user terminal or the second graphic screen By changing the size of the screen, turning on / off the second graphic screen, setting or changing the video list, decorating the virtual personal space wall, floor, interior props, etc. The first virtual reality image data may be generated so that the changed setting such as changing is applied. In various embodiments, the virtual reality image processing unit 520 may apply the second virtual reality image data or the second virtual reality image data so that the changed setting is applied based on various setting information regarding the virtual personal space or virtual personal space of the BJ received from the image providing device 300. The third virtual reality image data may be generated.

The memory 530 may store real-time streaming broadcast image data, VOD image data, or store various data used to generate virtual reality image data. Specifically, the memory 530 may store first, second, and third virtual reality image data generated through the virtual reality image processing unit 520. The memory 530 may store various setting information received from the user terminal or the image providing device 300. In addition, the memory 530 may store real-time streaming video data, VOD video data, replay video data, highlight video data, and viewer-produced video for each BJ identifier.

The processor 540 is operably connected to the communication interface 510, the virtual reality image processing unit 520, and the memory 530, and provides a real-time streaming broadcast image or VOD image, or a first, second, or third virtual reality Various commands for generating image data can be performed. Specifically, the processor 540 receives the request for the first, second, or third virtual reality image data through the communication interface 510, and the first or second generated through the virtual reality image processing unit 520 The virtual reality image data may be transmitted to the first HMD device 100 or the image output device 600 or the generated third virtual reality image data may be transmitted to the second HMD device 410 or the image providing device 300.

In various embodiments, without a separate virtual reality image processing unit, the processor 540 may perform the operation of the virtual reality image processing unit 520.

7 is a schematic flowchart illustrating a method of providing a user interface for viewing a virtual reality broadcast in a user terminal according to an embodiment of the present invention.

Referring to FIG. 7, a user terminal (eg, the first HMD device 100 or the video output device 600) is virtual to watch a real-time streaming broadcast of a broadcast provider (eg, BJ) from the broadcast management server 300. The first virtual reality data representing the virtual personal space of the user implemented in reality may be received (S700), and the received first virtual reality data may be displayed through the display of the user terminal (S710). When an event of a preset condition occurs in relation to the real-time streaming broadcast of the broadcast provider, the user terminal may display a virtual reality effect object corresponding to the generated event in the virtual personal space (S720).

8, 9A, 9B, and 10A of the first virtual reality image data displayed through the display 120 of the first HMD device 100 and the display 620 of the image output device 600, It will be described in detail with reference to FIGS. 10B, 11A, 11B, 11C, 11D, 11E, 11F, 12, 13, and 14. The operations described below may be performed by the processor 150 of the HMD device 100 or the processor 640 of the image output device 600.

8 is an exemplary diagram for describing first virtual reality image data representing a virtual personal space of a user according to an embodiment of the present invention.

Referring to FIG. 8, the virtual personal space 800 of the first virtual reality image data includes three first graphic screens 802, 804, and 806 for reproducing real-time streaming broadcast images, and real-time chat of broadcast viewers. 2 The main interface area 810 including the graphic screen 808 may be included. In various embodiments, the main interface area 810 is a third graphic screen 812 for reproducing the most popular (eg, a high number of views or a lot of likes) video among the VOD images of the user's favorite BJ. It may further include. The graphic screens may be sized or changed in position by the input device 550 (eg, a controller), or a distance perceived by a user in a virtual personal space may be adjusted. For example, if the pointer of the controller is located on the upper bar of the graphic screen by the user's controller operation, and the button is stopped after the pointer is moved to a specific position while the button of the controller is input, the graphic screen is moved to the specific position. You can. In addition, when the controller pointer is located at the corner of the graphic screen by the user's controller operation, and the controller is moved while the button of the controller is clicked, the size of the graphic screen may be adjusted according to the movement of the controller. Also, graphic screens may be rotated or minimized by a user's controller operation. In the case of minimization, the graphic screen may be converted into a thumbnail video of a preset size and displayed.

The virtual personal space 800 includes an interface screen 814 indicating a video list or a VOD video list related to real-time streaming broadcasts, and interface screens 816, 818, 820, and 822 related to various functions related to broadcast viewing or virtual personal space. ) May include a sub-interface area 824. The interface screens 816, 818, 820, and 822 may be converted into a minimized state as shown in FIG. 8 and displayed as bar-shaped icons.

The main interface area 810 of the virtual personal space 800 will be described in more detail below with reference to FIGS. 9A and 9B, and with respect to the sub-interface area 824, FIGS. 10A, 10B, 11A, and 11B, 11C, 11D, 11E, 11F, 12, 13, and 14 will be described in more detail below.

9A and 9B are exemplary views for describing graphic screens of a main interface area according to an embodiment of the present invention.

8 and 9A, the main interface area 810 is a first graphic screen 802 for reproducing BJ's real-time streaming broadcast image data, and a second representing chat data of broadcast viewers watching the real-time streaming broadcast. It may include a graphic screen (808). The first graphic screen 802 may include a display area 900 in which real-time streaming broadcast image data is displayed and a non-display area 910 in which icons for display control and volume control are included. The non-display area 910 may include a first icon 912 for video playback, pause and stop, a second icon 914 for volume control, and a third icon 916 for chat on / off. have. The second graphic screen 808 may include a chat display area 920 in which chat contents are displayed, setting icons 922 for chatting settings, and a chat input icon 924 for chatting input. The setting icons 922 indicate a list of broadcast viewers participating in the chat, various functions related to chatting (eg, displaying / not displaying emoticons, receiving / non-whispering whispers, changing nickname, clearing chat, etc.) and chat window It can correspond to the delete function. When the chat input icon 924 is selected by a user's controller operation, a chat input area 926 and a virtual keyboard area 928 for input may be displayed as shown in FIG. 9B. For example, when the pointer of the controller is located in a specific key area of the virtual keyboard area, and a button of the controller is clicked, a character corresponding to the key area may be input to the chat input area 926. In various embodiments, the chat input area 926 and the virtual keyboard area 928 allow a user to recognize a distance closer to the first graphic screen 802 and the second graphic screen 808 within the virtual personal space. Can be displayed.

As described in FIG. 9A in various embodiments, when the third icon 916 for chat on / off is selected due to the user's controller operation, the display of the second graphic screen 808 ends as shown in FIG. 9C, and the display area ( A first icon 930 for minimizing the first graphic screen 802 and a second icon 932 for ending the display of the first graphic screen 802 may be displayed at a specific location of 900). For example, when the first icon 930 is selected by the user's controller operation, the first graphic screen 802 may be converted into a specific size in which a thumbnail video can be played.

10A, 10B, 11A, 11B, 11C, 11D, 11E, 11F, 12, 13, and 14 are interfaces related to icons in a sub-interface area according to an embodiment of the present invention. These are example diagrams for explaining the screens.

8 and 10A, the sub-interface area 824 is a first interface screen 814 indicating a video list or VOD video list related to real-time streaming broadcast, and a second interface screen related to various functions related to virtual personal space. 816, a third interface screen 818 representing a bulletin board that is a community space related to viewing a broadcast, a fourth interface screen 820 representing a history of purchasing or receiving paid items, and a broadcasting list of favorite BJs A fifth interface screen 822 may be included.

The first interface screen 814 includes a first icon 1000 for displaying a real-time streaming broadcast video list for at least one favorite BJ, a second icon 1002 for displaying a VOD video list, and a broadcasting category. It may include an image list screen 1006 including a third icon 1004 for selecting and a thumbnail image (or thumbnail video) 1008 of at least one image included in the image list. When the first icon 1000 is selected by the user's controller operation, thumbnail images related to broadcasts of at least one BJ that is currently being streamed in real-time or related to a specific category (eg, a game) are displayed in the image list screen 1006 Can be displayed on. In addition, when the second icon 1002 is selected, thumbnail images of VOD images of at least one favorite BJ may be displayed on the image list screen 1006. When the third icon 1004 is selected, a selection screen 1010 for selecting a broadcasting related category is displayed as shown in FIG. 10B, and when a menu 1012 corresponding to a game is selected among broadcasting related categories, selecting a game related category A selection area 1014 for display may be displayed.

Referring to FIGS. 8 and 11A, the second interface screen 816 is a first icon for setting, such as screen addition, size change, position change, rotation, etc. in relation to the first graphic screen 802, 804, 806. Example: screen settings) (1100), a second icon for setting a real-time streaming broadcast video list or a VOD video list (for example, video list setting) 1102, a third icon for decorating a virtual personal space (for example, interior setting) ) 1104 and a fourth icon (eg, item purchase) 1106 for purchasing a paid item for broadcasting or BJ sponsorship. When the first icon 1100 is selected by the user's controller operation, a setting screen 1107 for screen setting may be displayed as shown in FIG. 11B. The setting screen 1107 is a first icon (eg, adding a screen) 1108 for adding a graphic screen for video playback to the main interface area 810, and the size of the graphic screen located in the main interface area 810. A second icon (eg, screen size change) 1110 for changing and a third icon (eg, resolution change) 1112 for changing the resolution of the graphic screen may be included. In various embodiments, an icon (not shown) for changing a frame of the graphic screen may also be added to the setting screen. For example, the frame of the graphic screen may be converted into a frame of various virtual reality objects equipped with a display area capable of displaying an image, such as a picture frame, TV, picture, etc., which can be placed in the user's virtual personal space. For example, when changing to a TV, a broadcast screen may be played in an area corresponding to a CRT of a TV, and when changing to a picture frame, a graphic screen may be implemented so that the broadcast video is played in an area other than the frame.

When the first icon 1108 is selected by a user's controller operation, a graphic screen may be added at a specific size to a specific location of an area other than the area where the graphic screen is located among the main interface areas 810 where the graphic screen is located. Here, the added graphic screen may be displayed in an appropriate size at an appropriate position so as not to overlap with the existing graphic screen.

When the second icon 1110 is selected by the user's controller operation, other objects except the graphic screen may be deactivated to change the size of the graphic screen located in the main interface area 810. Thereafter, the pointer is positioned at the corner of the graphic screen by the user's controller operation, and the size of the graphic screen may be changed by the user's controller movement while the button of the controller is clicked. In various embodiments, the position of the graphic screen may be changed by the user's controller operation. If the pointer is located in an inactive area other than the graphic screen by the user's controller operation, the changed size graphic screen is displayed, and the inactivated area may be activated again.

When the third icon 1112 is selected by the user's controller operation, a selection screen 1114 for selecting a resolution may be displayed. In various embodiments, when there are two or more graphic screens, a corresponding graphic screen is selected when a pointer is positioned in a specific area of the graphic screen by a user's controller operation and the edge of the graphic screen is highlighted, and for changing the resolution of the selected graphic screen. A selection screen may be displayed.

Returning to FIG. 11A again, when the second icon 1102 is selected, a setting screen 1115 for setting an image list as shown in FIG. 11C or 11D may be displayed.

11C is an exemplary diagram for setting a real-time streaming broadcast video list for a BJ selected by a user according to an embodiment of the present invention.

Referring to FIG. 11C, the setting screen 1115 is a first area 1116 indicating a broadcast category, and when a specific broadcast category (eg, game) 1118 is selected, a second area indicating at least one BJ related to the corresponding broadcast category Area 1120, selection key areas 1122 for selecting BJ, third area 1124 representing the selected BJ, and key areas for confirming or canceling settings (e.g., confirmation key, cancel key) 1126 ). The second area 1120 indicating the BJ list may include BJ identification information (eg, ID) related to a specific category. When a pointer is positioned on a specific ID by the user's controller operation, the edge of the specific ID is highlighted, and when the selection key 1122 is selected, the specific ID may be selected and displayed on the third area 1124. After that, when the confirmation key is selected, setting information may be stored so that a real-time streaming broadcast video list for BJ Africa 1, BJ Africa 3, and BJ Africa 8 is displayed. As described above, when at least one BJ is selected, a real-time streaming broadcast video list for at least one selected BJ (eg, BJ African 1, BJ African 3, BJ African 8) may be displayed as shown in FIG. 10A.

11D is an exemplary diagram for setting a list of VOD images of a BJ selected by a user according to an embodiment of the present invention.

Referring to FIG. 11D, the setting screen 1115 includes a first area 1128 indicating a broadcast category, and a second area indicating at least one BJ related to a corresponding category when a specific category (eg, game) 1130 is selected ( 1132), a third area 1134 representing a list of VOD images of the selected BJ, and key areas (eg, confirmation key, cancel key) 1136 for confirming or canceling the setting. For example, the third region 1134 may display a thumbnail image of at least one VOD image of the selected 'BJ Africa 3'. When a pointer is positioned on a specific thumbnail image by the user's controller operation, an edge of the specific thumbnail image is highlighted, and when a confirmation key is selected, a specific thumbnail image may be selected. As described above, when at least one VOD image of 'BJ Africa 3' is selected, a list of at least one selected VOD image (eg, image 1, image 4, image 5) may be displayed as shown in FIG. 10A.

In various embodiments, a real-time streaming broadcast video list or a VOD video list includes a real-time streaming broadcast video for a BJ viewed by a user or another BJ in a broadcasting category similar to the BJ as a recommended video, even if not set by the user, or a user Another VOD video of a broadcasting category similar to the VOD video viewed a lot may be included as a recommended video.

Returning to FIG. 11A again, when the third icon 1104 is selected, a setting screen 1137 for decorating a virtual personal space as shown in FIG. 11E may be displayed. The setting screen 1137 is a first area 1138 showing a category for a wall, a floor material for decorating a virtual personal space, furniture for placing it in a virtual personal space, and interior accessories, and a specific category (eg : Furniture) 1140 is selected, the second area 1142 representing at least one virtual reality object for the selected specific category and key areas (eg, confirmation key, cancel key) for confirming or canceling the setting (1144) ). The at least one virtual reality object may be a 3D graphic object modeled such that a sofa, table, bed, wardrobe, etc. are optimized for virtual reality. The second region 1142 may include at least one thumbnail image of furniture. When the pointer is positioned on the thumbnail image 1146 corresponding to the sofa by the user's controller operation, the edge of the thumbnail image 1146 is highlighted, and when the confirmation key is selected, a virtual reality object corresponding to the sofa may be selected. When the virtual reality object corresponding to the sofa is selected, the selected virtual reality object may be displayed in the virtual personal space. The virtual reality object in the virtual eye space may be moved by moving the controller of the user while the pointer of the user's controller is displayed and the button of the controller is clicked. For example, when the virtual reality object is located in a specific area of the virtual personal space by the user's controller movement, and the button input of the controller is stopped, the virtual reality object 1148 corresponding to the sofa as shown in FIG. 11F is the virtual personal space 800 ).

Returning to FIG. 11A again, when the fourth icon 1106 is selected, as shown in FIG. 11G, a setting screen 1149 for purchasing a paid item for viewing a broadcast or sponsoring a BJ may be displayed. The setting screen 1149 is a first area 1150 indicating a category (eg, star balloon, sticker, chocolate, spray, interior accessory, etc.) related to various paid items, and a specific category (eg star balloon) related to a paid item. When 1152 is selected, it may include a second area 1154 representing at least one paid item for the selected category and key areas (eg, confirmation key, cancel key) 1156 for confirming or canceling the setting. have. For example, various paid items may include items in which stickers related to sports or e-sports, icons, sprays, billboards, flags, hooligans, etc. are implemented as virtual reality objects. In various embodiments, the paid item may be a balloon, sticker, chocolate, spray, interior accessory, etc. related to BJ or specific content (for example, food content, game content, sports content, etc.). In the second area 1158, graphic objects including an image of at least one paid item and a button for purchasing the paid item may be displayed. For example, when a button 1158 for purchasing a specific paid item is selected by a controller operation of the user, a payment operation for purchasing the selected paid item may be performed. Since the payment operation for purchasing a paid item is a general technique, a detailed description will be omitted. When a paid item is purchased, a virtual reality effect object 826 corresponding to the purchased paid item may be displayed at a specific location in the virtual personal space as shown in FIG. 8. For example, the sound effect of popping firecrackers can be output along with the visual effect that the star balloons fly through the virtual personal space.

In various embodiments, the paid item is not limited to the presented embodiment, and various decoration items for decorating the avatar of the user may be further included. For example, decorating items for decorating a user's avatar include items such as clothes, accessories, shoes, hats, bags, and other miscellaneous items, or items for changing part of the avatar (for example, changing hairstyle, changing hair color, Eye, nose, mouth, etc.). As such, items for decorating the avatar may also be provided in connection with BJ or specific content (eg, food content, game content, sports content, etc.).

8 and 12, the third interface screen 818 may be an interface screen converted from a minimized state to a bar-shaped icon as shown in FIG. 8 to a default state as shown in FIG. 12. For example, the third interface screen 818 may further include an icon at the top for switching to a minimized state or switching from a minimized state to a default state.

The third interface screen 818 is selected when a first area 1200 and a specific bulletin board category (eg, chat room) 1202 indicating a bulletin board category (eg, announcements, broadcast schedules, chat rooms, and recommended bulletin boards) are selected. A second area 1204 indicating posts related to the bulletin board category may be included. Posts included in the second area may be displayed as graphic objects representing a title, an author ID, an author's date, number of views, recommendations, and comments.

8 and 13, the fourth interface screen 820 may be an interface screen converted from a minimized state to a bar-shaped icon as shown in FIG. 8 to a default state as shown in FIG. 13. For example, the fourth interface screen 820 may further include an icon at the top for switching to a minimized state or switching from a minimized state to a default state.

The fourth interface screen 820 displays a first area 1302 indicating a list of paid items 1300 held by a user and a second area 1306 indicating a list of paid items 1304 received by a user from another user. It can contain. The first area 1302 may include a graphic object representing a paid item for viewing a broadcast or decorating a virtual personal space.

8 and 14, the fifth interface screen 822 may be an interface screen converted from a minimized state to a bar-shaped icon as shown in FIG. 8 to a default state as shown in FIG. 14. For example, the fifth interface screen 822 may further include an icon at the top for switching to a minimized state or switching from a minimized state to a default state.

The fifth interface screen 822 may include graphic objects 1400, 1402, and 1404 representing a broadcast list of at least one favorite BJ. Each graphic object (1400, 1402, 1404) includes an image related to the BJ, text indicating the ID of the BJ, and a button (eg, a move button) 1406, 1408, 1410 to move to the virtual personal space of the BJ can do. For example, when a specific movement button 1406 is selected by a user's controller operation, a virtual reality image representing a virtual personal space of the corresponding BJ may be displayed. In addition, when the BJ is performing a real-time streaming broadcast, an icon for viewing a real-time streaming broadcast of the corresponding BJ (for example, a LIVE viewing icon) 1412, 1414 may be further included. For example, when the LIVE viewing icon 1414 of 'BJ Africa 8' is selected by the user's controller operation, the real-time streaming broadcast image of 'BJ Africa 8' is displayed on the first graphic screen 802 of the main interface area 810. ), And a real-time chat related to the real-time streaming broadcast may be displayed on the second graphic screen 808.

The interface screens described above are further provided with buttons for minimization, and when the minimize button is selected by the user's controller operation, the remaining areas except the upper bar may not be displayed. When the minimize button is selected again, the existing interface screens may be displayed again.

15 to 17 are exemplary views for describing second virtual reality image data representing a virtual personal space of a BJ according to an embodiment of the present invention. Hereinafter, an embodiment for simultaneously (or together) watching a plurality of broadcast viewers to watch a real-time streaming relay broadcast such as a sports relay or an e-sports relay will be described.

Referring to FIG. 15, the virtual personal space 1500 of BJ for a sports broadcast or e-sports broadcast includes a first graphic screen 1502 for playing a real-time streaming broadcast, and broadcast viewers watching the broadcast. A main interface area 1506 including a second graphic screen 1504 indicating real-time chat may be included. The main interface area 1506 includes a plurality of graphic screens 1508 and 1510 for playing various broadcast images related to a sports relay or e-sports relay, in addition to the first graphic screen 1502 that reproduces the relay broadcast image. Can be included. For example, a plurality of graphic screens may display broadcast images photographed through various cameras for sports relaying, or broadcast images photographing each of athletes participating in sports. In various embodiments, the virtual personal space 1500 of BJ may be embodied as a sports spectator seat 1512 so that a broadcast viewer can actually experience sitting in a sports game bleachers. In various embodiments, the virtual personal space 1500 of BJ may be embodied to enable an avatar corresponding to each of a plurality of broadcast viewers watching a broadcast to be seated in the sports audience seat 1512. Through this, the user can have a virtual experience as if actually watching sports or e-sports with other broadcast viewers.

When an event such as a purchase of a paid item by the user's controller operation or a user with a high sponsorship contribution participates in a chat in the virtual personal space of the BJ, the virtual reality effect object 1514 corresponding to the event is generated in the virtual personal space of the BJ (1500). The virtual reality effect object 1514 may be embodied to be common to all broadcast viewers who are visiting the virtual personal space 1500 of BJ as well as the user.

Referring to FIG. 16, the virtual reality effect object 1514 displayed in the virtual personal space 1500 of the BJ may be continuously displayed by moving to a portion of the virtual personal space 150 after a certain time. For example, when a preset event occurs, a virtual reality effect object 1514 made up of a plurality of balloons is displayed at a specific location in the virtual personal space 1500, and the plurality of balloons are on top of the virtual personal space 1500 It may be displayed continuously by moving to the areas 1600 and 1610. In various embodiments, when one of the balloons is selected by moving the pointer in the virtual personal space by the user's controller manipulation, another virtual reality effect may be caused to cause the selected balloon to burst. Through this, the user can experience various virtual experiences through interaction with virtual reality objects arranged in the virtual reality.

In various embodiments, through a controller operation of a user with high sponsorship contribution to BJ, a dedicated icon that can only be used by a user with high sponsorship contribution is input through the second graphic screen 1504, or a chat-only sticker for a user to chat. When purchasing the same paid item and inputting the purchased paid item, as illustrated in FIG. 16, a dedicated icon or a speech bubble 1620 or 1630 including a paid item may be displayed on the avatar in the virtual reality of the user. Such speech bubbles may be set to be displayed or not displayed by the user. In addition, these speech balloons may be set to be displayed for a predetermined number of times.

In various embodiments, voice chat between at least one user among users participating in the virtual personal space 1500 of the BJ may be performed. For example, the second graphic screen 1504 may further include an icon for voice chat. When the user's first HMD device 100 further includes a microphone, and the corresponding icon is selected by the user's controller operation, the voice signal obtained through the microphone is transmitted to the HMD device of at least one other viewer participating in the voice chat. Can be. In this case, an icon for indicating that the user is participating in the voice chat may be displayed in the virtual personal space 1500 of the BJ. For example, an icon for indicating that the user is participating in a voice chat may be a microphone image icon, and the corresponding microphone image icon may be displayed at a specific location in the virtual personal space 1500 of the BJ. In various embodiments, a 3D graphic microphone image may be displayed on the BJ's virtual personal space 1500 to indicate the user's voice output so that the user can experience using a microphone as if participating in a voice chat.

In various embodiments, when a voice signal is received from the HMD device of another viewer, the voice signal received through the speaker of the first HMD device 100 that the user is wearing is output, and voice chat on the avatar of another user as shown in FIG. 16. A speech bubble 1640 including a speaker icon for indicating may be displayed. The speech balloon 1640 may be simultaneously displayed through the HMD device of at least one user participating in the voice chat.

In various embodiments, the voice chat may be performed by at least one broadcast viewer selected by the BJ or at least one broadcast viewer according to the sponsored contribution of the BJ. For example, an icon for voice chat may be displayed on the second graphic screen 1504 only for the selected broadcast viewer or broadcast viewer corresponding to the sponsored contribution.

In various embodiments, a voice chat may turn on or off voice output of a specific user. For example, a speaker icon indicating a voice output is displayed around the avatar of a specific user participating in a voice chat, and when a corresponding speaker icon is selected by a user's controller operation, the icon is converted to an icon for turning off the corresponding voice output, and a specific user The audio output of can be blocked. If the icon is selected again by the user's controller operation, it is converted to an icon for turning on the corresponding voice output, so that a specific user's voice can be output.

Referring to FIG. 17, paid items 1700, 1710, and 1720 are purchased through a setting screen 1149 as shown in FIG. 11G by a user's controller operation, and purchased paid items 1700, 1710, and 1720 are BJ's It may be displayed in a specific area in the personal virtual space 1500. For example, a paid item (eg, a cheer sticker or spray) for supporting a specific player or team related to sports or e-sports is purchased through the setting screen 1149 by the user's controller operation, and purchased Paid items can be placed at specific locations. In various embodiments, a virtual reality object 1730 such as a cheering rod may be implemented in the hand of an avatar corresponding to each broadcast viewer for cheering on sports or e-sports. Through this, a broadcast viewer watching a sports broadcast or an e-sports broadcast can experience a virtual experience as if holding a cheering rod and cheering with other broadcast viewers.

18 is an exemplary diagram for describing first virtual reality image data representing a virtual personal space of a user according to an embodiment of the present invention. Hereinafter, an embodiment in which a user invites a friend to a virtual personal space to watch a broadcast together with the invited friend will be described.

Referring to FIG. 18, the virtual personal space 800 of the first virtual reality image data includes three first graphic screens 802, 804, and 806 for playing a real-time streaming broadcast image as described above with reference to FIG. And a main interface area 810 including a second graphic screen 808 representing real-time chat of broadcast viewers. In addition, the virtual personal space 800 includes an interface screen 814 indicating a video list or a VOD video list related to real-time streaming broadcasts, and interface screens 816, 818, 820 related to various functions related to watching a broadcast or virtual personal space. A sub-interface area 824 including 822 is included, and a sixth interface screen 1800 related to a function for inviting another user set as a user's friend to the user's virtual personal space may be included. The sixth interface screen 1800 may be converted into a minimized state as described above with reference to FIG. 8 and displayed as a bar-shaped icon. The added sixth interface screen 1800 will be described in more detail below with reference to FIG. 19.

19 is an exemplary diagram for explaining an interface screen related to a friend search icon in a sub-interface area according to an embodiment of the present invention.

Referring to FIGS. 18 and 19, the sixth interface screen 1800 may be an interface screen converted from a minimized state to a bar-shaped icon as shown in FIG. 18 to a default state as shown in FIG. 19. For example, the sixth interface screen 1800 may further include an icon at the top for switching to a minimized state or switching from a minimized state to a default state.

The sixth interface screen 1800 includes a first area 1802 indicating a user's friend list and a second area 1804 indicating a message composition area for requesting an invitation to a friend who wants to invite to a user's virtual personal space. can do. The first area 1802 includes identification information (eg, ID) of another user set as a friend by the user, and a friend corresponding to a function for adding a friend when the friend to be invited does not exist in the friend list It may further include additional icons. The sixth interface screen 1800 includes a first icon (eg, inviting) 1806 for sending a friend invitation message created through the message creation area of the second area 1804 and a second icon for canceling it ( Example: cancel) (1808). For example, a specific ID (eg, ID1) of a friend list is selected by a user's controller operation, text is input through the message writing area of the second area 1804, and then the first icon 1806 is displayed. When selected, an invitation message may be transmitted to another user's HMD device corresponding to the selected 'ID1'. The invitation message may be displayed as a graphic object at a specific location of another user's virtual personal space. The graphic object representing the invitation message may further include icons corresponding to a function for accepting or rejecting the invitation.

In various embodiments, the interface screen for inviting friends is not limited as described above. For example, the interface screen for inviting a friend may include an icon corresponding to the invitation function together with a profile image of at least one other user set by the user as a friend. When the icon corresponding to the invitation function is selected along with the profile image of a specific user by the user's controller operation, an invitation request message may be transmitted to the HMD device of the specific user. When the invitation is approved by another user, an avatar of another user who has approved the invitation may be displayed on the user's virtual personal space 800. This will be described below in detail with reference to FIG. 20.

20 is an exemplary diagram for describing first virtual reality image data representing a virtual personal space of a user according to an embodiment of the present invention. Hereinafter, an embodiment for watching a streaming broadcast or a VOD broadcast together by inviting another user set as a user's friend to the user's virtual personal space will be described.

18 to 20, when the invitation is accepted by another user as described above in FIG. 19, the avatar 1900 of the other user who accepted the invitation is shown in FIG. 20 of the user's virtual personal space 800. It can be marked at a specific location. For example, when a user wants to watch a sports or e-sports broadcast in a virtual personal space of the user with friends, an invitation message may be transmitted to another user through the sixth interface screen 1800 corresponding to the friend invitation function. have. The invitation message may be displayed as a pop-up interface screen at a specific location of another user's virtual personal space. The interface screen may include an icon for accepting the invitation and an icon for rejecting the invitation along with the content of the invitation message created by the user. When an icon for accepting an invitation is selected by another user, an avatar of another user who accepts the invitation in the user's virtual personal space 800 may be displayed to be located at a specific location (eg, sofa 1148). In this case, another user who accepts the invitation can have a virtual experience as if sitting on the sofa 1148 in the user's virtual personal space 800 through the display of his HMD device. A user's avatar may also be displayed on the user's virtual personal space 800. In this case, the display 120 of the first HMD device 100 that the user is wearing is the first virtual reality in which the avatar of another user sits on the sofa 1148 disposed in the user's virtual personal space 800 as shown in FIG. 20. Video data can be displayed. Through this, the user and other users can have a virtual experience as if watching a broadcast together in the user's virtual personal space 800.

As described above, the present invention enables a user to experience a variety of spatial and temporal experiences when watching a broadcast by watching a real-time streaming broadcast video provided by a broadcast provider through the virtual personal space of the user implemented in virtual reality.

The apparatus and method according to an embodiment of the present invention may be implemented in a form of program instructions that can be executed through various computer means and recorded in a computer readable medium. Computer-readable media may include program instructions, data files, data structures, or the like alone or in combination.

The program instructions recorded on the computer-readable medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks. Includes hardware devices specifically configured to store and execute program instructions such as magneto-optical media and ROM, RAM, flash memory, and the like. In addition, the above-described medium may be a transmission medium such as an optical or metal wire or a waveguide including a carrier wave that transmits a signal specifying a program command, data structure, or the like. Examples of program instructions include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language codes produced by a compiler.

The hardware device described above may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

The embodiments of the present invention have been described in more detail with reference to the accompanying drawings, but the present invention is not necessarily limited to these embodiments, and can be variously modified without departing from the technical spirit of the present invention. . Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The scope of protection of the present invention should be interpreted by the claims below, and all technical spirits within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

100: first HMD device
200: electronic device
300: broadcast providing device
400: photographing device
410: second HMD device
500: broadcast management server
550: input device
600: video output device

Claims (14)

Receiving first virtual reality image data representing a virtual personal space implemented in virtual reality to view a real-time streaming broadcast of a broadcast provider from a broadcast management server;
Displaying the received first virtual reality image data through a display; And
And displaying a virtual reality effect object corresponding to the generated event in the virtual personal space when an event of a preset condition occurs in relation to the real-time streaming broadcast of the broadcast provider. How to provide. According to claim 1, The event of the predetermined condition,
Broadcast viewers watching the real-time streaming broadcasts, events that purchase paid items for the broadcast provider, or broadcast viewers who have a sponsored contribution level indicating a degree of purchasing the paid items participate in chatting of the real-time streaming broadcasts A method of providing a virtual reality user interface for viewing a broadcast, including an event. The method of claim 1, wherein the first virtual reality image data,
And a main interface area for watching the real-time streaming broadcast and a sub-interface area related to a plurality of functions related to watching the real-time streaming broadcast. The method of claim 3, wherein the main interface area,
And a first graphic screen reproducing the real-time streaming broadcast image of the broadcast provider and a second graphic screen representing real-time chat between broadcast viewers watching the real-time streaming broadcast at the same time. The method of claim 3, wherein the sub-interface area,
A virtual reality for broadcast viewing, comprising a first interface screen representing a list of real-time streaming broadcast video or video on demand (VOD) video of at least one broadcast provider and a second interface screen for setting the virtual personal space How to provide a user interface. According to claim 1,
Receiving second virtual reality image data representing a virtual personal space of the broadcast provider according to a user's request; And
And displaying the received second virtual reality image data through the display. The virtual private space of the broadcast provider,
A method of providing a virtual reality user interface for viewing a broadcast, provided in different forms for each user by the broadcast provider. Communication interface;
display; And
A processor connected to the communication interface and the display,
The processor,
Receiving first virtual reality image data representing a virtual personal space implemented in virtual reality to view a real-time streaming broadcast of a broadcast provider from a broadcast management server through the communication interface,
Displaying the received first virtual reality image data through the display,
A virtual reality user interface providing apparatus for viewing a broadcast, when an event of a preset condition occurs in relation to the real-time streaming broadcast of the broadcast provider, displaying a virtual reality effect object corresponding to the generated event in the virtual personal space. The method of claim 8, wherein the event of the preset condition,
Broadcast viewers watching the real-time streaming broadcasts, events that purchase paid items for the broadcast provider, or broadcast viewers who have a sponsored contribution level indicating a degree of purchasing the paid items participate in chatting of the real-time streaming broadcasts Apparatus for providing a virtual reality user interface for viewing a broadcast, including an event. The method of claim 8, wherein the first virtual reality image data,
And a main interface area for watching the real-time streaming broadcast and a sub-interface area related to a plurality of functions related to watching the real-time streaming broadcast. The method of claim 10, wherein the main interface area,
And a first graphic screen displaying a real-time streaming broadcast video of the broadcast provider and a second graphic screen representing a real-time chat between broadcast viewers watching the real-time streaming broadcast at the same time. The method of claim 10, wherein the sub-interface area,
A virtual reality for broadcast viewing, comprising a first interface screen representing a list of real-time streaming broadcast video or video on demand (VOD) video of at least one broadcast provider and a second interface screen for setting the virtual personal space User interface device. The method of claim 8, wherein the processor,
Receiving second virtual reality image data representing a virtual personal space of the broadcast provider according to a user's request,
An apparatus for providing a virtual reality user interface for broadcasting viewing, which displays the received second virtual reality image data through the display. The virtual private space of the broadcast provider,
A device for providing a virtual reality user interface for viewing a broadcast, provided in different forms for each user by the broadcast provider.

Images