KR102481528B1 - Method for broadcasting service of virtual reality game, apparatus and system for executing the method - Google Patents

Abstract

Disclosed are a virtual reality game broadcasting service method and an apparatus and system for performing the same. A virtual reality game broadcasting service method according to an embodiment disclosed herein is a method performed in a computing device having one or more processors and a memory storing one or more programs executed by the one or more processors, comprising: Operation of acquiring sensing data by sensing the motion of the game user according to the progress of the game, operation of acquiring a screen from the viewpoint of the game user viewed from the viewpoint of the game user based on the sensing data, operation of obtaining a screen viewed from the viewpoint of the virtual camera provided in the virtual reality game An operation of acquiring a virtual camera viewpoint screen, and an operation of transmitting the virtual camera viewpoint screen to a broadcasting screen instead of the game user viewpoint screen.

Description

Virtual reality game broadcasting service method and apparatus and system for performing the same

The disclosed embodiment relates to virtual reality game broadcasting service technology.

Recently, games are evolving in a variety of ways, from local games that are executed only within a gaming device to online games that are executed through linkage with other gaming devices or servers through a network. In addition, with the improvement of hardware performance and network technology, games are evolving to be run not only on personal computers and game consoles, but also on mobile devices such as smart phones and tablet PCs.

On the other hand, game broadcasting that broadcasts a game that a game user is playing so that viewers can watch it is also increasing. Here, in the case of VR (Virtual Reality) game broadcasting, since the game user proceeds to play while wearing the head-mounted display, the game screen moves or shakes according to the game user's gaze change, resulting in broadcasting. There is a problem of feeling confused and dizzy when watching.

Disclosed embodiments are to provide a virtual reality game broadcasting service method capable of reducing shaking of a game screen according to a change in a game user's gaze movement, and an apparatus and system for performing the same.

A virtual reality game broadcasting service method according to an embodiment disclosed herein is a method performed in a computing device having one or more processors and a memory for storing one or more programs executed by the one or more processors. obtaining sensing data by sensing motions of game users as the game progresses; obtaining a game user's viewpoint screen viewed from the game user's viewpoint based on the sensing data; obtaining a virtual camera viewpoint screen viewed from a viewpoint of a virtual camera provided in the virtual reality game; and transmitting the virtual camera viewpoint screen to a broadcasting screen instead of the game user viewpoint screen.

The virtual camera may be provided to view the game from the position of the game user in the virtual reality game.

The operation of transmitting to the broadcasting screen may include an operation of determining whether to move the virtual camera viewpoint screen according to a movement displacement of the game user viewpoint screen.

The virtual camera viewpoint screen may be larger than the game user viewpoint screen.

The transmitting of the broadcasting screen may include transmitting without moving the virtual camera viewpoint screen when the movement displacement of the game user viewpoint screen is less than a preset threshold value.

When the movement displacement of the game user viewpoint screen is made within the virtual camera viewpoint screen, it may be determined that it is less than the preset threshold value.

The transmitting to the broadcasting screen may include moving and transmitting the virtual camera viewpoint screen in response to the movement displacement of the game user viewpoint screen when the movement displacement of the game user viewpoint screen is equal to or greater than a preset threshold. .

When the movement displacement of the game user viewpoint screen deviates from the virtual camera viewpoint screen, it may be determined that the displacement is equal to or greater than the predetermined threshold value.

The operation of determining whether to move the screen from the viewpoint of the virtual camera may include: extracting feature points for each frame of the screen from the viewpoint of the game user; checking a feature point movement displacement between each frame of the screen from the game user's point of view; and determining whether to move the virtual camera viewpoint screen according to the movement displacement of the feature point.

The operation of determining whether to move the virtual camera viewpoint screen may include an operation of transmitting the virtual camera viewpoint screen without moving it when the movement displacement of the feature point is less than a preset threshold value.

The operation of determining whether to move the virtual camera viewpoint screen may include moving the virtual camera viewpoint screen in a direction opposite to the movement direction of the feature point corresponding to the movement displacement of the feature point when the movement displacement of the feature point is equal to or greater than a preset threshold value, and transmitting the motion. action may be included.

The virtual reality game broadcasting service method may include: generating a gaze indicator by checking a gaze direction of the game user in the virtual reality game; and displaying the gaze indicator on the virtual camera viewpoint screen.

A computing device according to an embodiment disclosed herein is a computing device having one or more processors and a memory for storing one or more programs executed by the one or more processors, and provides a game user's experience as a virtual reality game progresses. a sensor unit that senses movement and generates sensing data; a first screen providing unit generating a screen from the game user's point of view based on the sensing data; a second screen providing unit generating a virtual camera viewpoint screen viewed from a viewpoint of a virtual camera provided in the virtual reality game; and a screen transmission unit that transmits the virtual camera viewpoint screen as a broadcasting screen instead of the game user viewpoint screen.

The virtual camera may be provided to view the game from the position of the game user in the virtual reality game.

The screen transmission unit may determine whether or not to move the virtual camera viewpoint screen according to the movement displacement of the game user viewpoint screen, and transmit the result.

The virtual camera viewpoint screen may be larger than the game user viewpoint screen.

The screen transmitting unit may transmit the screen without moving the virtual camera viewpoint screen when the movement and displacement of the game user viewpoint screen is made within the virtual camera viewpoint screen.

The screen transmitting unit may move and transmit the virtual camera viewpoint screen corresponding to the movement displacement of the game user viewpoint screen when the movement displacement of the game user viewpoint screen deviates from the virtual camera viewpoint screen.

The screen transmission unit extracts feature points for each frame of the screen from the game user's point of view, checks the movement displacement of the feature point between each frame of the screen from the game user's point of view, and determines whether the screen from the virtual camera's point of view moves according to the movement displacement of the feature point. can decide

The screen transmission unit may transmit the virtual camera viewpoint screen without moving it when the feature point movement displacement is less than a preset threshold value.

The screen transmission unit may move the virtual camera view screen in a direction opposite to the movement direction of the feature point in response to the movement displacement of the feature point and transmit the screen when the feature point movement displacement is equal to or greater than a predetermined threshold value.

The computing device may further include a gaze indicator generating unit that checks a gaze direction of the game user in the virtual reality game, generates a gaze indicator, and displays the gaze indicator on the virtual camera viewpoint screen.

A virtual reality game broadcasting service system according to an embodiment disclosed herein includes a head-mounted display worn on a user's head; A virtual reality game is provided through the head-mounted display, motion of the game user according to progress of the virtual reality game is sensed, and a game user's point of view screen viewed from the game user's point of view is provided to the game user, and the virtual reality game is provided. a gaming device that generates a virtual camera viewpoint screen viewed from a viewpoint of a virtual camera provided in a game and transmits the virtual camera viewpoint screen to a broadcasting screen instead of the game user viewpoint screen; and a broadcasting server that receives the virtual camera viewpoint screen from the gaming device and transmits it to a viewer terminal.

According to the disclosed embodiment, a virtual camera viewpoint screen separate from the game user viewpoint screen is generated, the virtual camera viewpoint screen is transmitted as it is when the movement displacement of the game user viewpoint screen is less than a preset threshold value, and the game user viewpoint screen If the movement displacement is greater than a preset threshold, the virtual camera viewpoint screen is moved by a corresponding amount and transmitted, thereby reducing confusion or dizziness felt by viewers watching the game broadcast due to the shaking of the game screen, and at the same time keeping the eyes of the game user. As you follow along, you will be able to see the game screen.

In addition, by using a game screen separate from the game user viewpoint screen called the virtual camera viewpoint screen, even if a small shake occurs on the game user viewpoint screen, the virtual camera viewpoint screen is sent to the viewer as it is, so the game screen seen by viewers ( That is, it is also possible to reduce the occurrence of small shaking in the virtual camera viewpoint screen).

In addition, by displaying the gaze indicator on the virtual camera viewpoint screen, viewers can check the direction of the gaze of the game user within the game, thereby increasing the sense of immersion in virtual reality game broadcasting.

1 is a diagram showing the configuration of a virtual reality game broadcasting service system according to an exemplary embodiment.
2 is a block diagram showing the configuration of a gaming device according to an embodiment disclosed herein;
3 is a diagram comparing a game user viewpoint screen and a virtual camera viewpoint screen in an embodiment disclosed herein.
4 is a view schematically showing a state in which a gaze indicator is displayed on a virtual camera viewpoint screen in an embodiment disclosed herein;
5 is a diagram illustrating a case in which a movement displacement of a screen from a game user's point of view is less than a preset threshold in an embodiment disclosed herein;
6 is a diagram illustrating a case in which the movement displacement of a screen from a game user's point of view is greater than or equal to a predetermined threshold value in an embodiment disclosed herein;
7 is a diagram showing movement displacement of feature points between frames of a screen from a game user's point of view in an embodiment disclosed herein;
8 is a view showing a state in which a frame of a virtual camera viewpoint screen is moved and corrected in an embodiment disclosed herein;
9 is a diagram showing the configuration of a virtual reality game broadcasting service system according to another disclosed embodiment.
10 is a block diagram showing the configuration of a broadcasting server according to the disclosed embodiment.
11 is a block diagram illustrating and describing a computing environment including a computing device suitable for use in example embodiments.

Hereinafter, specific embodiments will be described with reference to the drawings. The detailed descriptions that follow are provided to provide a comprehensive understanding of the methods, devices and/or systems described herein. However, this is only an example and is not limited thereto.

Terminology used in the detailed description is only for describing the embodiments and should in no way be limiting. Unless expressly used otherwise, singular forms of expression include plural forms. In this specification, expressions such as "comprising" or "comprising" are intended to indicate certain characteristics, numbers, steps, operations, elements, some or combinations thereof, and one or more other than those described. It should not be construed to exclude the existence or possibility of any other feature, number, step, operation, element, part or combination thereof.

In the following description, terms such as "transmission", "communication", "transmission", "reception" and other similar meanings of signals or information refer not only to direct transmission of signals or information from one component to another, but also to It also includes passing through other components. In particular, "transmitting" or "transmitting" a signal or information as a component indicates the final destination of the signal or information, and does not mean a direct destination. The same is true for "reception" of signals or information. Also, in this specification, two or more data or information being “related” means that when one data (or information) is obtained, at least a portion of other data (or information) can be obtained based thereon.

Also, terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms may be used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, the second element may also be termed a first element.

Hereinafter, "game" refers to software that one or more gamers can enjoy using an input device mounted on or connected to a gaming device. In general, it proceeds in such a way as directly manipulating a player character or manipulating the movement of an object in a game, and genres such as role-playing, adventure, and arcade are sometimes divided.

Hereinafter, "gaming device" refers to hardware that executes game software, performs calculations according to a user's input through an input device, and outputs results through an output device. For example, a personal computer, a laptop computer, a game console, a smart phone, a tablet PC, a smart band or a smart watch. It may be in the form of a wearable device. In addition, hardware that satisfies the above definition is interpreted as belonging to a gaming device.

Hereinafter, "online game" means a game that must pass through a network to play the game. Generally, two or more gaming devices at a remote location are connected to each other through a network or synchronized through a server to enjoy a game. As long as it satisfies the above definition, the type of gaming device that runs the game is not limited. In addition to games running on personal computers or game consoles, games running on mobile devices such as smartphones, or games running in the cloud or in conjunction with a plurality of different gaming devices (including different types of gaming devices) are also described above. Anything that satisfies the definition is interpreted as belonging to an online game.

Hereinafter, “input device” refers to a device for inputting data to a gaming device. For example, input devices exclusively for game operation manufactured in the form of joysticks, mice, keyboards, touch screens, hardware buttons or steering wheels, musical instruments, guns, gloves, footrests, and the like may be used. In addition, various sensors (temperature sensor, infrared sensor, motion sensor, gyro sensor, acceleration sensor, gravity sensor, etc.) mounted on or interlocked with the gaming device, Global Positioning System (GPS), Indoor Positioning System ), etc., can also be cited as an example of an input device.

1 is a diagram showing the configuration of a virtual reality game broadcasting service system according to an exemplary embodiment.

Referring to FIG. 1 , a virtual reality game broadcasting service system 100 may include a gaming device 102 , a broadcasting server 104 , and a viewer terminal 106 . The gaming device 102 and the viewer terminal 106 are communicatively connected to the broadcasting server 104 through the communication network 150, respectively.

In some embodiments, communication network 150 is the Internet, one or more local area networks, wide area networks, cellular networks, mobile networks, other types of networks, or such networks. may include a combination of

One or more game applications may be installed on the gaming device 102 . Here, the game application may refer to a program installed in the gaming device 102 to provide game services to game users. In an exemplary embodiment, gaming device 102 may be mounted on head mounted display 108 . However, it is not limited thereto, and the gaming device 102 and the head mounted display 108 may be integrally formed. In addition, the gaming device 102 may be communicatively connected to the head mounted display 108 through wireless, wired, wired or wireless, and the like.

The gaming device 102 may present a virtual reality game via the head mounted display 108 . The head mounted display 108 may be worn on the head of the game user. The gaming device 102 may provide a game screen from the viewpoint of the game user (ie, the player) according to the movement of the game user wearing the head mounted display 108 .

2 is a block diagram showing the configuration of a gaming device 102 according to one disclosed embodiment.

Referring to FIG. 2 , the gaming device 102 includes a game execution unit 111, a sensor unit 113, a first screen providing unit 115, a second screen providing unit 117, and a gaze indicator generating unit 119. , and a screen transmission unit 121 may be included.

In one embodiment, the game execution unit 111, the sensor unit 113, the first screen providing unit 115, the second screen providing unit 117, the gaze indicator generating unit 119, and the screen transmitting unit 121 may be implemented using one or more physically separated devices, or may be implemented by one or more processors or a combination of one or more processors and software, and may not be clearly distinguished in specific operations, unlike the illustrated example.

The game execution unit 111 may execute a game application according to a game user's command. The game execution unit 111 may progress the game according to a command input from the game user after the game starts. A game user may input a command for playing the game through an input device.

The sensor unit 113 may sense one or more of the position and posture of the game user. That is, the sensor unit 113 may sense the movement of the game user wearing the head mounted display 108 . While playing a virtual reality game, the game user performs actions such as turning his head, lowering his head, or moving his arm according to the game situation. The sensor unit 113 detects the position of the game user and One or more of the postures may be sensed. In an exemplary embodiment, the sensor unit 113 may include a gyro sensor, an acceleration sensor, and a gravity sensor.

The first screen providing unit 115 may provide a game screen seen from the game user's point of view. The first screen providing unit 115 provides a game screen viewed from the game user's point of view based on the sensing data sensed by the sensor unit 113 (that is, a value sensed for one or more of the game user's position and posture). can Hereinafter, a game screen viewed from the game user's point of view may be referred to as a "game user's point of view screen".

Since the game user's point of view screen is a game screen in which changes in the position and posture of the game user are reflected as they are during the game, movement or shaking occurs according to the change in the position and posture of the game user. In this case, the screen from the game user's point of view changes according to the movement of the game user based on the center of the screen from the point of view of the game user.

The second screen providing unit 117 may provide a game screen seen from the viewpoint of the virtual camera. The virtual camera is virtually provided in the game, and may be provided to view the game from a fixed position of the game user. The game screen viewed from the viewpoint of the virtual camera is a game screen separate from the screen from the viewpoint of the game user, and may be a game screen to be transmitted to the viewer terminal 106 .

That is, the game screen viewed from the viewpoint of the virtual camera may be a game screen for broadcasting to be transmitted to viewers. Hereinafter, a game screen viewed from the viewpoint of a virtual camera may be referred to as a “virtual camera viewpoint screen”. The virtual camera viewpoint screen is a screen viewed from a fixed viewpoint of the virtual camera, and is not affected by changes in the position and posture of the game user.

Here, the virtual camera viewpoint screen may be larger than the game user viewpoint screen. 3 is a diagram comparing a game user viewpoint screen and a virtual camera viewpoint screen in an embodiment disclosed herein. Referring to FIG. 3 , it can be seen that the virtual camera viewpoint screen S2 is larger than the game user viewpoint screen S1.

In an exemplary embodiment, in a state in which the centers of the game user viewpoint screen S1 and the virtual camera viewpoint screen S2 coincide, the virtual camera viewpoint screen S2 extends from each edge of the game user viewpoint screen S1 to the outside. It may be provided to have an extended area. However, it is not limited thereto, and the virtual camera viewpoint screen S2 may have the same size as the game user viewpoint screen S1.

The gaze indicator generation unit 119 may generate an indicator (ie, a gaze indicator) for a gaze direction of a game user in the game. In an exemplary embodiment, the gaze indicator generator 119 may check the gaze direction of the game user in the game based on the sensing data sensed by the sensor unit 113 . For example, the gaze indicator generator 119 may check the direction of the game user's gaze within the game by checking the direction of the game user's head through sensing data. However, it is not limited thereto, and the gaze indicator generation unit 119 may check the gaze direction of the game user in the game through a separate device such as an eye tracker.

The gaze indicator generation unit 119 may generate a gaze indicator for indicating the gaze direction of the game user after confirming the gaze direction of the game user in the game. In an exemplary embodiment, the gaze indicator generator 119 may render and display the gaze indicator on the virtual camera viewpoint screen.

4 is a diagram schematically illustrating a state in which a gaze indicator is displayed on a virtual camera viewpoint screen according to an embodiment disclosed herein. Referring to FIG. 4 , the gaze indicator I may be rendered and displayed on the virtual camera viewpoint screen S2.

Here, the gaze indicator (I) may be implemented in 3D. That is, the gaze indicator I may be implemented in 3D to indicate the direction of the gaze of the game user in three axis directions (ie, x-axis, y-axis, and z-axis).

Since the virtual camera viewpoint screen S2 is a game screen provided to viewers, by displaying the gaze indicator I on the virtual camera viewpoint screen S2, viewers can check the direction of the game user's gaze within the game. As a result, it is possible to increase the sense of immersion in virtual reality game broadcasting.

The screen transmission unit 121 may transmit the virtual camera viewpoint screen to the broadcasting server 104 . That is, in the disclosed embodiment, a virtual camera viewpoint screen is transmitted to the broadcasting server 104 instead of transmitting a game user viewpoint screen to the broadcasting server 104 . At this time, the screen transmission unit 121 may transmit the virtual camera viewpoint screen as it is (ie, transmit without movement) or move it according to the movement displacement of the game user viewpoint screen S1.

Specifically, when the movement displacement of the game user viewpoint screen is less than a preset threshold, the screen transmission unit 121 does not correct (ie, does not move) the virtual camera viewpoint screen according to the movement of the game user viewpoint screen, and broadcast server 104 ) can be sent.

That is, when the game user's point of view screen moves due to the game user's movement, but the movement displacement of the game user's point of view screen is less than a preset threshold (for example, the game user's movement is small or fine, such as eye blinking or gaze movement). If the movement displacement of the game user viewpoint screen is less than a predetermined threshold value due to motion), the screen transmission unit 121 may transmit the virtual camera viewpoint screen to the broadcasting server 104 without correcting it according to the movement of the game user viewpoint screen.

5 is a diagram illustrating a case in which a movement displacement of a screen from a game user's point of view is less than a predetermined threshold value according to an embodiment disclosed herein. Referring to FIG. 5 , when the movement displacement (d) of the game user viewpoint screen S1 is less than a predetermined threshold value, the movement displacement of the game user viewpoint screen S1 is set to be made within the virtual camera viewpoint screen S2 It can be.

For example, if the movement displacement d of the game user viewpoint screen S1 is within the virtual camera viewpoint screen S2 in a state where the centers of the game user viewpoint screen S1 and the virtual camera viewpoint screen S2 coincide. , The screen transmission unit 121 may determine that the movement displacement of the screen S1 from the game user's point of view is less than a preset threshold value. However, it is not limited to this, and if the movement displacement (d) of the game user viewpoint screen S1 is slightly out of the virtual camera viewpoint screen S2, the movement displacement of the game user viewpoint screen S1 is set in advance. It can be judged to be below the threshold.

In addition, the screen transmission unit 121 corrects (ie, moves) the virtual camera viewpoint screen according to the movement of the game user viewpoint screen when the movement displacement of the game user viewpoint screen is equal to or greater than a preset threshold value, and transmits it to the broadcasting server 104. can do. At this time, the screen transmission unit 121 may move the virtual camera viewpoint screen in response to the movement displacement of the game user viewpoint screen. That is, the screen transmission unit 121 may move the virtual camera viewpoint screen by the displacement of the game user viewpoint screen.

6 is a diagram illustrating a case in which a movement displacement of a screen from a game user's point of view is equal to or greater than a predetermined threshold value according to an embodiment disclosed herein. Referring to FIG. 6 , when the movement displacement d of the game user viewpoint screen S1 is equal to or greater than a preset threshold value, the movement displacement of the game user viewpoint screen S1 is set to deviate from the virtual camera viewpoint screen S2. can

For example, in a state where the centers of the game user viewpoint screen S1 and the virtual camera viewpoint screen S2 coincide, the movement displacement d of the game user viewpoint screen S1 deviates from the virtual camera viewpoint screen S2. , the screen transmission unit 121 may determine that the movement displacement of the screen S1 from the game user's point of view is equal to or greater than a predetermined threshold value. The screen transmission unit 121 may move the virtual camera viewpoint screen in response to the movement displacement d of the game user viewpoint screen and transmit the result to the broadcasting server 104 .

Referring back to FIG. 1 , the broadcast server 104 may be a server computing device for providing a broadcast service for virtual reality games played by game users. The broadcasting server 104 may receive a virtual camera viewpoint screen from the gaming device 102 . The broadcasting server 104 may transmit the virtual camera viewpoint screen to the viewer terminal 106 .

The viewer terminal 106 may be a terminal of a viewer who watches a broadcast about a virtual reality game. The viewer terminal 106 may access the broadcasting server 104 and receive a virtual camera viewpoint screen from the broadcasting server 104 . The viewer terminal 106 may include a smart phone, a tablet PC, a notebook computer, and a desktop computer.

According to the disclosed embodiment, a virtual camera viewpoint screen separate from the game user viewpoint screen is generated, the virtual camera viewpoint screen is transmitted as it is when the movement displacement of the game user viewpoint screen is less than a preset threshold value, and the game user viewpoint screen If the movement displacement is greater than a preset threshold, the virtual camera viewpoint screen is moved by a corresponding amount and transmitted, thereby reducing confusion or dizziness felt by viewers watching the game broadcast due to the shaking of the game screen, and at the same time keeping the eyes of the game user. As you follow along, you will be able to see the game screen.

In addition, by using a game screen separate from the game user viewpoint screen called the virtual camera viewpoint screen, even if a small shake occurs on the game user viewpoint screen, the virtual camera viewpoint screen is sent to the viewer as it is, so the game screen seen by viewers ( That is, it is also possible to reduce the occurrence of small shaking in the virtual camera viewpoint screen).

Meanwhile, here, whether the movement displacement of the game user viewpoint screen is less than or equal to a preset threshold is determined by comparing it with the virtual camera viewpoint screen, but it is not limited thereto and may be independently determined without comparison with the virtual camera viewpoint screen. That is, the screen transmission unit 121 may extract feature points from the game user's viewpoint screen, and determine whether or not to correct the virtual camera viewpoint screen according to whether the movement displacement of the feature points is less than or equal to a preset threshold value.

Specifically, the screen transmission unit 121 may extract feature points for each frame of the game user's point of view screen. The screen transmission unit 121 may check the feature point movement displacement between the current frame (n-th frame) and the previous frame (n-1-th frame) of the screen from the game user's point of view.

7 is a diagram illustrating movement displacement of feature points between frames of a game user's point of view screen according to the disclosed embodiment. Referring to FIG. 7 , the screen transmission unit 121 may extract feature points (A, B, C, D, E) for each frame (eg, frame 1 and frame 2) of the screen from the viewpoint of the game user.

The screen transmission unit 121 is responsible for the feature points (A, B, C, D) between the n-th frame (eg, frame 2) of the screen from the game user's point of view and the n-1-th frame (eg, frame 1) that is the previous frame. , E) can be confirmed. In an exemplary embodiment, the screen transmission unit 121 calculates vectors according to the movement of each feature point (A, B, C, D, and E), and averages the vectors to obtain the feature point (A, B, C, D, and E). An average vector according to their movement can be calculated. Here, when the game user's eye movement direction moves to the right, the game user's point of view screen moves to the left (ie, feature points on the game user's point of view screen move to the left).

When the size of the average vector according to the movement of feature points is less than a preset threshold, the screen transmission unit 121 broadcasts the server without correcting (ie, without moving) the virtual camera viewpoint screen corresponding to the corresponding frame of the game user viewpoint screen ( 104) can be sent.

The screen transmission unit 121 corrects (i.e., moves) the virtual camera viewpoint screen corresponding to the corresponding frame of the game user viewpoint screen when the size of the average vector according to the movement of feature points is equal to or greater than a predetermined threshold value, and broadcasts the broadcast server 104 can be sent to

8 is a diagram illustrating a state in which a frame of a virtual camera viewpoint screen is moved and corrected in an embodiment disclosed herein. Referring to FIG. 8 , the screen transmission unit 121 converts the virtual camera viewpoint screen S2 corresponding to the corresponding frame of the game user viewpoint screen to the movement displacement of the feature points when the size of the average vector according to the movement of the feature points is equal to or greater than a preset threshold. (For example, the size of the average vector according to the movement of the feature points) may be moved correspondingly.

At this time, the screen transmission unit 121 moves the feature points in the direction opposite to the movement direction (eg, the direction of the average vector according to the movement of the feature points) (ie, the same direction as the game user's eye movement direction), and moves the virtual camera viewpoint screen (S2). ) can be moved.

9 is a diagram showing the configuration of a virtual reality game broadcasting service system according to another disclosed embodiment. Here, the difference from the embodiment shown in FIG. 1 will be mainly described.

Referring to FIG. 9 , a virtual reality game broadcasting service system 200 may include a gaming device 202 , a broadcasting server 204 , and a viewer terminal 206 . The gaming device 202 and the viewer terminal 206 are communicatively connected to the broadcasting server 204 through the communication network 250, respectively.

The gaming device 202 may present a virtual reality game via the head mounted display 208 . The gaming device 202 may be mounted on the head mounted display 208 or integrally formed with the head mounted display 208, but is not limited thereto and communicates with the head mounted display 208 through wired, wireless, wired or wireless, etc. can possibly be connected.

The gaming device 202 may execute a game application according to a game user's command. The gaming device 202 may generate a screen from the game user's point of view based on sensing data obtained by sensing one or more of the position and posture of the game user. The gaming device 202 may transmit the game user's viewpoint screen to the broadcasting server 204 . The gaming device 202 may transmit the screen from the game user's point of view to the broadcasting server 204 in units of frames.

The gaming device 202 may generate gaze direction information by checking the gaze direction of the game user within the game. The gaming device 202 may transmit gaze direction information to the broadcasting server 204 .

The broadcasting server 204 may correct the game user viewpoint screen received from the gaming device 202 according to the movement displacement of the game user viewpoint screen and transmit the corrected image to the viewer terminal 206 . 10 is a block diagram showing the configuration of a broadcasting server 204 according to the disclosed embodiment. Referring to FIG. 10 , the broadcasting server 204 may include a communication unit 211, a feature point extraction unit 213, a screen correction unit 215, and a gaze indicator generation unit 217.

In one embodiment, the communication unit 211, the feature point extraction unit 213, the screen correction unit 215, and the gaze indicator generation unit 217 are implemented using one or more physically separated devices, or one or more processors or It may be implemented by a combination of one or more processors and software, and unlike the illustrated example, specific operations may not be clearly distinguished.

The communication unit 211 may communicate with the gaming device 202 and the viewer terminal 206 . The communication unit 211 may receive a game user's point of view screen from the gaming device 202 . The communication unit 211 may receive gaze direction information from the gaming device 202 . The communication unit 211 may transmit a game user viewpoint screen to the viewer terminal 206 .

The feature point extractor 213 may extract feature points from the game user's point of view screen received by the communication unit 211 . In an exemplary embodiment, the feature point extraction unit 213 may extract feature points for each frame of the game user's viewpoint screen. Feature point extraction may use various known methods for extracting feature points from an image.

The screen correction unit 215 may determine whether to correct the game user's viewpoint screen based on the movement displacement of feature points on the game user's viewpoint screen. The screen correction unit 215 may check the movement displacement of feature points between the current frame (n-th frame) and the previous frame (n-1-th frame) of the screen from the game user's point of view. In an exemplary embodiment, the screen correction unit 215 may calculate vectors according to the movement of each extracted feature point, and average the vectors to calculate an average vector according to the movement of the feature point.

The screen correction unit 215 does not correct (ie, does not move) the corresponding frame (ie, the n-th frame) of the screen from the game user's point of view when the size of the average vector according to the movement of the feature points is less than a preset threshold, and the viewer terminal It can be decided to transmit to (206). The screen correction unit 215 may transmit a corresponding frame of the screen from the game user's point of view to the viewer terminal 206 through the communication unit 211 .

The screen correction unit 215 may determine to correct (ie, move) the corresponding frame of the screen from the game user's point of view and transmit it to the viewer terminal 206 when the magnitude of the average vector according to the movement of the feature points is equal to or greater than a preset threshold. there is.

When the size of the average vector according to the movement of the feature points is equal to or greater than a predetermined threshold value, the screen correction unit 215 converts the corresponding frame of the screen from the game user's point of view to the movement displacement of the feature points (eg, the size of the average vector according to the movement of the feature points) can be moved in response to At this time, the screen correction unit 215 may move the corresponding frame in a direction opposite to the moving direction of the feature points (eg, the direction of the average vector according to the movement of the feature points).

Based on the gaze direction information received by the communication unit 211, the gaze indicator generator 217 may generate a gaze indicator for indicating the gaze direction of the game user in the game. The gaze indicator generation unit 217 may render and display the gaze indicator on the game user's point of view screen.

According to the disclosed embodiment, the broadcast server 204 post-corrects the game user's viewpoint screen according to the displacement of the game user's viewpoint screen and transmits the post-correction to the viewer terminal 206, so that viewers watching the game broadcast can experience the shaking of the game screen. This can reduce the confusion or dizziness you feel. In addition, by rendering and displaying the gaze indicator on the screen from the game user's point of view, viewers can view the game screen while following the gaze of the game user.

Meanwhile, here, it has been described that the broadcasting server 204 extracts feature points from the screen of the game user's point of view and determines whether or not to correct the screen of the game user's point of view according to the movement displacement of the feature point, but is not limited thereto.

Specifically, the broadcasting server 204 may receive sensing data obtained by sensing one or more of the position and posture of the game user from the gaming device 202 . The broadcasting server 204 may check the movement displacement of the screen from the game user's point of view based on the sensing data. That is, the broadcasting server 204 may check in which direction and by how much the game user's point of view screen has moved based on the sensing data.

The broadcasting server 204 may transmit the game user viewpoint screen to the viewer terminal 206 without correcting the game user viewpoint screen when the movement displacement of the game user viewpoint screen is less than a preset threshold value. The broadcasting server 204 may move the game user's viewpoint screen and transmit it to the viewer terminal 206 when the movement displacement of the game user's viewpoint screen is equal to or greater than a predetermined threshold value.

That is, the broadcasting server 204 may move the screen from the game user's point of view in a direction opposite to the movement direction of the screen from the game user's point of view (ie, the same direction as the movement direction of the game user's gaze). At this time, the broadcasting server 204 may move the game user viewpoint screen in the opposite direction by the displacement of the game user viewpoint screen.

11 is a block diagram illustrating and describing a computing environment 10 including a computing device suitable for use in example embodiments. In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components other than those described below.

The illustrated computing environment 10 includes a computing device 12 . In one embodiment, computing device 12 may be a gaming device 102 , 202 . Additionally, the computing device 12 may be a broadcast server 104 , 204 .

Computing device 12 includes at least one processor 14 , a computer readable storage medium 16 and a communication bus 18 . Processor 14 may cause computing device 12 to operate according to the above-mentioned example embodiments. For example, processor 14 may execute one or more programs stored on computer readable storage medium 16 . The one or more programs may include one or more computer-executable instructions, which when executed by processor 14 are configured to cause computing device 12 to perform operations in accordance with an illustrative embodiment. It can be.

Computer-readable storage medium 16 is configured to store computer-executable instructions or program code, program data, and/or other suitable form of information. Program 20 stored on computer readable storage medium 16 includes a set of instructions executable by processor 14 . In one embodiment, computer readable storage medium 16 includes memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, other forms of storage media that can be accessed by computing device 12 and store desired information, or any suitable combination thereof.

Communications bus 18 interconnects various other components of computing device 12, including processor 14 and computer-readable storage medium 16.

Computing device 12 may also include one or more input/output interfaces 22 and one or more network communication interfaces 26 that provide interfaces for one or more input/output devices 24 . An input/output interface 22 and a network communication interface 26 are connected to the communication bus 18 . Input/output device 24 may be coupled to other components of computing device 12 via input/output interface 22 . Exemplary input/output devices 24 include a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touchscreen), a voice or sound input device, various types of sensor devices, and/or a photographing device. input devices, and/or output devices such as display devices, printers, speakers, and/or network cards. The exemplary input/output device 24 may be included inside the computing device 12 as a component constituting the computing device 12, or may be connected to the computing device 12 as a separate device distinct from the computing device 12. may be

Although representative embodiments of the present invention have been described in detail above, those skilled in the art will understand that various modifications are possible to the above-described embodiments without departing from the scope of the present invention. . Therefore, the scope of the present invention should not be limited to the described embodiments and should not be defined, and should be defined by not only the claims to be described later, but also those equivalent to these claims.

100, 200: virtual reality game broadcasting service system
102, 202: gaming device
104, 204: broadcast server
106, 206: viewer terminal
108, 208: head mounted display
111: game execution unit
113: sensor unit
115: first screen providing unit
117: second screen providing unit
119: gaze indicator generation unit
121: screen transmission unit
211: Ministry of Communication
213: feature point extraction unit
215: screen correction unit
217: gaze indicator generation unit

Claims (23)

one or more processors; and
A method performed in a computing device having a memory storing one or more programs executed by the one or more processors,
obtaining sensing data by sensing a motion of a game user as the virtual reality game progresses;
obtaining a game user's viewpoint screen viewed from the game user's viewpoint based on the sensing data;
obtaining a virtual camera viewpoint screen viewed from a viewpoint of a virtual camera provided in the virtual reality game; and
Transmitting the virtual camera viewpoint screen to a broadcast screen instead of the game user viewpoint screen;
The operation of transmitting to the broadcasting screen,
and determining whether to move the virtual camera viewpoint screen according to movement displacement of the game user viewpoint screen in order to reduce shaking of the broadcast screen according to the game user's gaze movement. The method of claim 1,
The virtual camera is provided to view the game from the position of the game user in the virtual reality game, the virtual reality game broadcasting service method. delete The method of claim 1,
The virtual reality game broadcasting service method of claim 1 , wherein the virtual camera viewpoint screen is larger than the game user viewpoint screen. The method of claim 4,
The operation of transmitting to the broadcasting screen,
and transmitting without moving the virtual camera viewpoint screen when the movement displacement of the game user viewpoint screen is less than a predetermined threshold value. The method of claim 5,
and determining that the movement displacement of the game user viewpoint screen is less than the predetermined threshold when the movement displacement is made within the virtual camera viewpoint screen. The method of claim 4,
The operation of transmitting to the broadcasting screen,
and moving and transmitting the virtual camera viewpoint screen in response to the movement displacement of the game user viewpoint screen when the movement displacement of the game user viewpoint screen is equal to or greater than a predetermined threshold value. The method of claim 7,
and determining that the movement displacement of the game user's view screen deviates from the virtual camera's view screen, as being equal to or greater than the preset threshold value. The method of claim 1,
The operation of determining whether to move the virtual camera viewpoint screen,
extracting feature points for each frame of the screen from the game user's point of view;
checking a feature point movement displacement between each frame of the screen from the game user's point of view; and
and determining whether to move the virtual camera viewpoint screen according to the movement displacement of the feature point. The method of claim 9,
The operation of determining whether to move the virtual camera viewpoint screen,
and transmitting without moving the virtual camera viewpoint screen when the feature point movement displacement is less than a preset threshold value. The method of claim 9,
The operation of determining whether to move the virtual camera viewpoint screen,
and moving the virtual camera viewpoint screen in a direction opposite to the movement direction of the feature point in response to the movement displacement of the feature point when the feature point movement displacement is greater than or equal to a predetermined threshold value, and transmitting the motion. The method of claim 1,
The virtual reality game broadcast service method,
generating a gaze indicator by checking a gaze direction of the game user in the virtual reality game; and
The virtual reality game broadcasting service method further comprising displaying the gaze indicator on the virtual camera viewpoint screen. one or more processors; and
A computing device having a memory storing one or more programs executed by the one or more processors,
a sensor unit generating sensing data by sensing a motion of a game user according to progress of a virtual reality game;
a first screen providing unit generating a screen from the game user's point of view based on the sensing data;
a second screen providing unit generating a virtual camera viewpoint screen viewed from a viewpoint of a virtual camera provided in the virtual reality game; and
a screen transmission unit for transmitting the virtual camera viewpoint screen to a broadcasting screen instead of the game user viewpoint screen;
The screen transmission unit,
The computing device determining whether or not to move the virtual camera viewpoint screen according to the movement displacement of the game user viewpoint screen in order to reduce shaking of the broadcasting screen according to the game user's gaze movement, and transmits the determined movement of the virtual camera viewpoint screen. The method of claim 13,
The virtual camera is provided to view the game from the position of the game user in the virtual reality game. delete The method of claim 13,
The virtual camera viewpoint screen is provided larger than the game user viewpoint screen. The method of claim 16
The screen transmission unit,
and transmits without moving the virtual camera viewpoint screen when the movement displacement of the game user viewpoint screen is made within the virtual camera viewpoint screen. The method of claim 16
The screen transmission unit,
and moving and transmitting the virtual camera viewpoint screen in response to the movement displacement of the game user viewpoint screen when the movement displacement of the game user viewpoint screen deviates from the virtual camera viewpoint screen. The method of claim 13,
The screen transmission unit,
A computing device that extracts feature points for each frame of the screen from the game user's viewpoint, checks movement displacement of feature points between each frame of the screen from the game user's viewpoint, and determines whether or not to move the screen from the viewpoint of the virtual camera according to the movement displacement of the feature point. . The method of claim 19
The screen transmission unit,
When the feature point movement displacement is less than a preset threshold value, the computing device transmits the virtual camera viewpoint screen without moving it. The method of claim 19
The screen transmission unit,
When the feature point movement displacement is equal to or greater than a preset threshold value, the virtual camera viewpoint screen is moved in a direction opposite to the feature point movement direction in response to the movement displacement of the feature point and transmitted. The method of claim 13,
The computing device,
The computing device further includes a gaze indicator generation unit configured to check a gaze direction of the game user in the virtual reality game, generate a gaze indicator, and display the gaze indicator on the virtual camera viewpoint screen. a head-mounted display worn on the user's head;
A virtual reality game is provided through the head-mounted display, motion of the game user according to progress of the virtual reality game is sensed, and a game user's point of view screen viewed from the game user's point of view is provided to the game user, and the virtual reality game is provided. a gaming device that generates a virtual camera viewpoint screen viewed from a viewpoint of a virtual camera provided in a game and transmits the virtual camera viewpoint screen to a broadcasting screen instead of the game user viewpoint screen; and
A broadcasting server receiving the virtual camera viewpoint screen from the gaming device and transmitting it to a viewer terminal;
The gaming device,
The virtual reality game broadcasting service system of claim 1 , wherein whether or not the virtual camera viewpoint screen moves is determined and transmitted according to movement displacement of the game user viewpoint screen in order to reduce shaking of the broadcast screen according to the game user's gaze movement.

Images