KR20220049290A - Method for broadcasting service of virtual reality game, apparatus and system for executing the method - Google Patents

Abstract

Disclosed are a method for providing a virtual reality game broadcast service and an apparatus and a system for performing the same. According to an embodiment, a method for providing a virtual reality game broadcast service, which is performed in a computing device having one or more processors and a memory for storing one or more programs executed by the one or more processors, comprises the operations of: obtaining sensing data obtained by sensing the movement of a game user according to the progress of a virtual reality game; obtaining a game user viewpoint screen based on the sensing data; obtaining a virtual camera viewpoint screen viewed from the viewpoint of a virtual camera provided in the virtual reality game, and transmitting the virtual camera viewpoint screen as a broadcast screen instead of the game user viewpoint screen. Therefore, the method can reduce confusion or dizziness felt by viewers watching game broadcasts, due to shaking of a game screen.

Description

Virtual reality game broadcasting service method and apparatus and system for performing the same

The disclosed embodiment relates to a virtual reality game broadcasting service technology.

Recently, games are evolving in various ways, from local games executed only within the gaming device to online games executed through interworking with other gaming devices or servers through a network. In addition, according to the improvement of hardware performance and network technology, games are evolving to be executable in mobile devices such as smart phones and tablet PCs as well as personal computers or game consoles.

On the other hand, there is also an increasing trend in game broadcasting, in which the game that the game user is playing is transmitted through the broadcast so that the viewers can watch it. Here, in the case of VR (Virtual Reality) game broadcasting, since the game user plays while the head-mounted display is mounted, the game screen moves or shakes according to the change in the gaze of the game user. There is a problem in that you feel confused and dizzy while watching.

The disclosed embodiment is to provide a virtual reality game broadcasting service method capable of reducing shaking of a game screen according to a change in the gaze movement of a game user, and an apparatus and system for performing the same.

A virtual reality game broadcasting service method according to an embodiment of the present disclosure is a method performed in a computing device having one or more processors and a memory for storing one or more programs executed by the one or more processors, the virtual reality an operation of acquiring sensing data for sensing a motion of a game user according to the progress of the game; acquiring a game user's point of view screen viewed from the game user's point of view based on the sensing data; acquiring a virtual camera viewpoint screen viewed from a viewpoint of a virtual camera provided in the virtual reality game; and transmitting the virtual camera view screen as a broadcast screen instead of the game user view screen.

The virtual camera may be provided to view the game from the position of the game user in the virtual reality game.

The operation of transmitting to the broadcast screen may include an operation of determining whether to move the virtual camera viewpoint screen according to a movement displacement of the game user's viewpoint screen.

The virtual camera viewpoint screen may be provided larger than the game user viewpoint screen.

The operation of transmitting the broadcast screen may include an operation of transmitting the virtual camera viewpoint without moving the screen when the movement displacement of the screen from the game user's viewpoint is less than a preset threshold.

When the movement displacement of the game user's view point screen is made within the virtual camera view point screen, it may be determined that it is less than the preset threshold value.

The operation of transmitting to the broadcast screen may include an operation of moving and transmitting the virtual camera viewpoint screen in response to the movement displacement of the game user viewpoint screen when the movement displacement of the screen from the game user's point of view is equal to or greater than a preset threshold. .

When the movement displacement of the game user's view screen deviates from the virtual camera view point, it may be determined to be equal to or greater than the preset threshold.

The operation of determining whether to move the screen from the viewpoint of the virtual camera may include: extracting a feature point for each frame of the screen from the viewpoint of the game user; checking the movement displacement of the feature point between each frame of the screen from the game user's point of view; and determining whether to move the virtual camera view point screen according to the movement displacement of the feature point.

The operation of determining whether to move the virtual camera viewpoint screen may include an operation of transmitting the virtual camera viewpoint screen without moving, when the feature point movement displacement is less than a preset threshold.

The operation of determining whether to move the virtual camera viewpoint screen includes moving the virtual camera viewpoint screen in a direction opposite to the movement direction of the characteristic point in response to the movement displacement of the characteristic point when the movement displacement of the characteristic point is equal to or greater than a preset threshold and transmitting it It may include an action to

The virtual reality game broadcasting service method may include generating a gaze indicator by checking a gaze direction of the game user in the virtual reality game; and displaying the gaze indicator on the virtual camera viewpoint screen.

A computing device according to an embodiment of the disclosure is a computing device having one or more processors and a memory for storing one or more programs executed by the one or more processors, a sensor unit that senses motion and generates sensed data; a first screen providing unit for generating a screen from the point of view of the game user viewed from the point of view of the game user based on the sensing data; a second screen providing unit for generating a virtual camera viewpoint viewed from the viewpoint of the virtual camera provided in the virtual reality game; and a screen transmitter configured to transmit the virtual camera point of view screen as a broadcast screen instead of the game user's point of view screen.

The virtual camera may be provided to view the game from the position of the game user in the virtual reality game.

The screen transmitter may determine whether or not to move the virtual camera view point screen according to the movement displacement of the game user view point screen and transmit it.

The virtual camera viewpoint screen may be provided larger than the game user viewpoint screen.

The screen transmitter may transmit the virtual camera viewpoint without moving the screen when the movement displacement of the game user's viewpoint screen is within the virtual camera viewpoint screen.

The screen transmitter may be configured to move and transmit the virtual camera viewpoint screen in response to the movement displacement of the game user viewpoint screen when the movement displacement of the game user's viewpoint screen deviates from the virtual camera viewpoint screen.

The screen transmitting unit extracts a feature point for each frame of the screen from the game user's point of view, confirms the movement displacement of the feature point between each frame of the screen from the game user's point of view, and determines whether the virtual camera view screen is moved according to the movement displacement of the feature point can decide

The screen transmitter may transmit the virtual camera view point without moving the screen when the movement displacement of the feature point is less than a preset threshold.

The screen transmitter may be configured to move and transmit the virtual camera viewpoint screen in response to the movement displacement of the key point in a direction opposite to the moving direction of the key point when the movement displacement of the key point is equal to or greater than a preset threshold.

The computing device may further include a gaze indicator generator configured to generate a gaze indicator by checking a gaze direction of the game user in the virtual reality game, and to display the gaze indicator on the virtual camera viewpoint screen.

A virtual reality game broadcasting service system according to an embodiment of the present disclosure includes: a head mounted display worn on a user's head; A virtual reality game is provided through the head mounted display, and a game user's point of view screen viewed from the game user's point of view is provided to the game user by sensing the motion of the game user according to the progress of the virtual reality game, and the virtual reality a gaming device for generating a virtual camera viewpoint viewed from the viewpoint of a virtual camera provided in the game, and transmitting the virtual camera viewpoint screen as a broadcast screen instead of the game user viewpoint; and a broadcasting server that receives the virtual camera view point from the gaming device and transmits it to a viewer terminal.

According to the disclosed embodiment, a virtual camera viewpoint screen separate from the game user viewpoint screen is generated, and if the movement displacement of the game user viewpoint screen is less than a preset threshold, the virtual camera viewpoint screen is transmitted as it is, If the movement displacement is greater than a preset threshold, the virtual camera view point screen is moved and transmitted as much as it corresponds to it, thereby reducing confusion or dizziness felt by viewers watching the game broadcast due to the shaking of the game screen, and at the same time reducing the gaze of the game user. As you follow along, you will be able to see the game screen.

And, by using a game screen separate from the game user's point of view screen called the virtual camera point of view screen, even if a small shake occurs in the game user's point of view, the screen from the virtual camera point of view is transmitted to the viewer as it is, so the game screen that viewers see ( That is, it is possible to reduce the occurrence of small shakes in the virtual camera viewpoint).

In addition, by displaying the gaze indicator on the virtual camera viewpoint screen, viewers can check the gaze direction of the game user in the game, thereby increasing the immersion in the virtual reality game broadcasting.

1 is a diagram showing the configuration of a virtual reality game broadcasting service system according to an exemplary embodiment;
2 is a block diagram illustrating a configuration of a gaming device according to an embodiment of the present disclosure;
3 is a view comparing a screen from a game user's point of view and a screen from a virtual camera's point of view in the disclosed embodiment;
4 is a view schematically showing a state in which a gaze indicator is displayed on a virtual camera viewpoint screen in an disclosed embodiment;
5 is a diagram illustrating a case in which a movement displacement of a screen from a game user's point of view is less than a preset threshold in an disclosed embodiment;
6 is a diagram illustrating a case in which a movement displacement of a screen from a game user's point of view is greater than or equal to a preset threshold in an disclosed embodiment;
7 is a view showing movement displacement of feature points between frames of a screen from a game user's point of view in an embodiment to be disclosed;
8 is a diagram illustrating a state in which a frame of a virtual camera view point screen is moved and corrected according to an disclosed embodiment;
9 is a diagram showing the configuration of a virtual reality game broadcasting service system according to another disclosed embodiment;
10 is a block diagram showing the configuration of a broadcast server according to the disclosed embodiment;
11 is a block diagram illustrating and describing a computing environment including a computing device suitable for use in example embodiments;

Hereinafter, specific embodiments will be described with reference to the drawings. The following detailed description is provided to provide a comprehensive understanding of the methods, devices, and/or systems described herein. However, this is merely an example and is not limited thereto.

The terminology used in the detailed description is for the purpose of describing the embodiments only, and should in no way be limiting. Unless explicitly used otherwise, expressions in the singular include the meaning of the plural. As used herein, expressions such as “comprising” or “comprising” are intended to indicate certain features, numbers, steps, operations, elements, some or combination thereof, and one or more other than those described. It should not be construed to exclude the presence or possibility of other features, numbers, steps, acts, elements, or any part or combination thereof.

In the following description, the terms "transmission", "communication", "transmission", "reception" and other similar meanings of a signal or information are not only directly transmitted from one component to another component, but also a signal or information This includes passing through other components. In particular, to “transmit” or “transmit” a signal or information to a component indicates the final destination of the signal or information and does not imply a direct destination. The same is true for "reception" of signals or information. In addition, in this specification, when two or more data or information are "related", it means that when one data (or information) is acquired, at least a part of other data (or information) can be acquired based thereon.

Also, terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The above terms may be used for the purpose of distinguishing one component from another component. For example, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

Hereinafter, "game" means software that one or more gamers can enjoy using an input device mounted on or connected to a gaming device. Usually, it is performed by directly manipulating the player character or by manipulating the movement of objects in the game, and genres such as role-playing, adventure, and arcade are sometimes classified.

Hereinafter, the term "gaming device" refers to hardware that executes game software but performs calculations according to a user's input through an input device and outputs a result through an output device. For example, a personal computer (Personal Computer), laptop (Laptop Computer), game console (Game Console), smart phone (Smart Phone), tablet (Tablet) PC, smart band (Smart Band) or smart watch (Smart Watch) It may be in the form of a wearable device. In addition, hardware that satisfies the above definition is interpreted as belonging to a gaming device.

Hereinafter, "online game" means a game that must go through a network in order to play the game. In general, it refers to a game in which two or more gaming devices in a remote location are connected to each other through a network or synchronized through a server. As long as it satisfies the above definition, there is no limitation on the type of gaming device that executes the game. Not only games executed on personal computers or game consoles, but also games executed on mobile devices such as smartphones, or games in which a plurality of different gaming devices (including different types of gaming devices) are linked or executed in the cloud If it satisfies the definition, it is interpreted as belonging to an online game.

Hereinafter, the term “input device” refers to a device for inputting data into a gaming device. For example, a joystick, a mouse, a keyboard, a touch screen, a hardware button or a steering wheel, a musical instrument, a gun, gloves, an input device dedicated to game manipulation manufactured in the form of a footrest may be mentioned. In addition, various sensors (temperature sensor, infrared sensor, motion sensor, gyro sensor, acceleration sensor, gravity sensor, etc.) mounted on or linked to gaming devices, Global Positioning System (GPS), Indoor Positioning System ), etc., may also be used as an example of an input device.

1 is a diagram showing the configuration of a virtual reality game broadcasting service system according to an exemplary embodiment.

Referring to FIG. 1 , a virtual reality game broadcasting service system 100 may include a gaming device 102 , a broadcasting server 104 , and a viewer terminal 106 . Gaming device 102 and viewer terminal 106 are each communicatively connected to broadcast server 104 via communication network 150 .

In some embodiments, communication network 150 is the Internet, one or more local area networks, wide area networks, cellular networks, mobile networks, other types of networks, or such networks. may include combinations of these.

One or more gaming applications may be installed on the gaming device 102 . Here, the game application may refer to a program installed in the gaming device 102 to provide a game service to a game user. In an exemplary embodiment, the gaming device 102 may be mounted to a head mounted display 108 . However, the present invention is not limited thereto, and the gaming device 102 and the head mounted display 108 may be integrally formed. In addition, the gaming device 102 may be communicatively connected to the head mounted display 108 through wireless, wired, wired, or wireless, or the like.

The gaming device 102 may provide a virtual reality game via the head mounted display 108 . The head mounted display 108 may be worn on the head of the game user. The gaming device 102 may provide a game screen from the viewpoint of the game user (ie, the player) according to the movement of the game user wearing the head mounted display 108 .

2 is a block diagram illustrating a configuration of a gaming device 102 according to an disclosed embodiment.

Referring to FIG. 2 , the gaming device 102 includes a game execution unit 111 , a sensor unit 113 , a first screen providing unit 115 , a second screen providing unit 117 , and a gaze indicator generating unit 119 . , and a screen transmitter 121 .

In an embodiment, the game execution unit 111 , the sensor unit 113 , the first screen providing unit 115 , the second screen providing unit 117 , the gaze indicator generating unit 119 , and the screen transmitting unit 121 . may be implemented using one or more physically separated devices, or may be implemented by one or more processors or a combination of one or more processors and software, and may not be clearly distinguished in specific operation unlike the illustrated example.

The game execution unit 111 may execute a game application according to a command of a game user. The game execution unit 111 may proceed with the game according to a command input from the game user after the game is started. The game user may input a command to proceed with the game through the input device.

The sensor unit 113 may sense one or more of the position and posture of the game user. That is, the sensor unit 113 may sense the movement of the game user wearing the head mounted display 108 . The game user performs actions such as turning his head, bowing his head, or moving his arms depending on the game situation while playing the virtual reality game. One or more of the postures may be sensed. In an exemplary embodiment, the sensor unit 113 may include a gyro sensor, an acceleration sensor, and a gravity sensor.

The first screen providing unit 115 may provide a game screen viewed from a game user's point of view. The first screen providing unit 115 provides a game screen viewed from a game user's point of view based on the sensing data sensed by the sensor unit 113 (ie, a sensing value for one or more of the game user's position and posture). can Hereinafter, a game screen viewed from a game user's point of view may be referred to as a "game user's point of view screen".

Since the game user's view screen is a game screen in which the position and posture change of the game user are reflected as it is during the game, movement or shaking occurs according to the change in the position and posture of the game user. In this case, the screen from the game user's point of view is moved based on the center of the screen from the game user's point of view according to the movement of the game user.

The second screen providing unit 117 may provide a game screen viewed from the viewpoint of the virtual camera. The virtual camera is provided virtually in the game, and may be provided to view the game from a fixed position of the game user. The game screen viewed from the viewpoint of the virtual camera is a game screen separate from the game user's viewpoint, and may be a game screen to be transmitted to the viewer terminal 106 .

That is, the game screen viewed from the viewpoint of the virtual camera may be a broadcast game screen to be transmitted to the viewer. Hereinafter, a game screen viewed from the viewpoint of the virtual camera may be referred to as a “virtual camera viewpoint”. The virtual camera viewpoint screen is a screen viewed from a fixed viewpoint of the virtual camera, and is not affected by changes in the position and posture of the game user.

Here, the screen from the viewpoint of the virtual camera may be provided larger than the screen from the viewpoint of the game user. 3 is a diagram comparing a screen from a game user's point of view and a screen from a virtual camera's point of view according to an embodiment disclosed herein. Referring to FIG. 3 , it can be seen that the virtual camera view screen S2 is larger than the game user view screen S1 .

In an exemplary embodiment, in a state where the centers of the game user's viewpoint screen S1 and the virtual camera viewpoint screen S2 coincide with each other, the virtual camera viewpoint screen S2 moves outward from each edge of the game user viewpoint screen S1 It may be provided to have an extended area. However, the present invention is not limited thereto, and the virtual camera viewpoint screen S2 may have the same size as the game user viewpoint screen S1 .

The gaze indicator generating unit 119 may generate an indicator (ie, a gaze indicator) for the gaze direction of the game user in the game. In an exemplary embodiment, the gaze indicator generation unit 119 may check the gaze direction of the game user in the game based on the sensed data sensed by the sensor unit 113 . For example, the gaze indicator generating unit 119 may check the gaze direction of the game user in the game by checking the head direction of the game user through the sensing data. However, the present invention is not limited thereto, and the gaze indicator generating unit 119 may check the gaze direction of the game user in the game through equipment such as a separate eye tracker.

After checking the gaze direction of the game user in the game, the gaze indicator generating unit 119 may generate a gaze indicator for indicating the gaze direction of the game user. In an exemplary embodiment, the gaze indicator generator 119 may render and display the gaze indicator on the virtual camera viewpoint screen.

4 is a diagram schematically illustrating a state in which a gaze indicator is displayed on a virtual camera viewpoint screen according to an embodiment of the present disclosure. Referring to FIG. 4 , the gaze indicator I may be rendered and displayed on the virtual camera viewpoint screen S2 .

Here, the gaze indicator I may be implemented in 3D. That is, the gaze indicator I may be implemented in 3D to indicate the gaze direction of the game user in three-axis directions (ie, the x-axis, the y-axis, and the z-axis).

Since the virtual camera viewpoint screen S2 is a game screen provided to the viewer, by displaying the gaze indicator I on the virtual camera viewpoint screen S2, viewers can check the gaze direction of the game user in the game, and Therefore, it is possible to increase the immersion in virtual reality game broadcasting.

The screen transmitter 121 may transmit a virtual camera viewpoint screen to the broadcast server 104 . That is, in the disclosed embodiment, the screen from the viewpoint of the game user is not transmitted to the broadcasting server 104 , but the screen from the viewpoint of the virtual camera is transmitted to the broadcasting server 104 . In this case, the screen transmitter 121 may transmit the virtual camera view screen as it is (ie, transmit without movement) or move it according to the movement displacement of the game user view screen S1 .

Specifically, when the movement displacement of the game user's view screen is less than a preset threshold, the screen transmitter 121 does not correct the virtual camera view screen according to the movement of the game user's view screen (ie, does not move) the broadcasting server 104 ) can be sent.

That is, when the game user's view screen is moved by the game user's movement, and the movement displacement of the game user's view screen is less than a preset threshold (for example, when the game user's movement is small or minute, such as blinking eyes or moving eyes, etc.) When the movement displacement of the screen from the game user's point of view is less than a preset threshold value), the screen transmitter 121 may transmit the virtual camera point of view to the broadcasting server 104 without correcting the screen according to the movement of the screen from the game user's point of view.

5 is a diagram illustrating a case in which a movement displacement of a screen from a game user's point of view is less than a preset threshold in an embodiment disclosed herein. 5, when the movement displacement d of the game user viewpoint screen S1 is less than a preset threshold, the movement displacement of the game user viewpoint screen S1 is set to be made within the virtual camera viewpoint screen S2 can be

For example, if the center of the game user's viewpoint screen S1 and the virtual camera viewpoint screen S2 coincides with each other, the movement displacement d of the game user viewpoint screen S1 is within the virtual camera viewpoint screen S2 , the screen transmitter 121 may determine that the movement displacement of the screen S1 from the game user's point of view is less than a preset threshold. However, the present invention is not limited thereto, and if the movement displacement d of the game user viewpoint screen S1 is out of the virtual camera viewpoint screen S2, even if it is slightly deviated, the movement displacement of the game user viewpoint screen S1 is preset. It can be judged to be less than a threshold value.

In addition, when the movement displacement of the screen from the game user's point of view is equal to or greater than a preset threshold, the screen transmitter 121 corrects (ie, moves) the virtual camera point of view according to the movement of the screen of the game user's point of view and transmits it to the broadcasting server 104 . can do. In this case, the screen transmitter 121 may move the virtual camera view point screen in response to the movement displacement of the game user view point screen. That is, the screen transmitter 121 may move the virtual camera view point screen by the amount of movement displacement of the game user view point screen.

6 is a diagram illustrating a case in which a movement displacement of a screen from a game user's point of view is greater than or equal to a preset threshold in an embodiment disclosed herein. Referring to FIG. 6 , when the movement displacement d of the game user viewpoint screen S1 is greater than or equal to a preset threshold, the movement displacement of the game user viewpoint screen S1 is set to be out of the virtual camera viewpoint screen S2. can

For example, in a state where the centers of the game user's viewpoint screen S1 and the virtual camera viewpoint screen S2 coincide with each other, the movement displacement d of the game user viewpoint screen S1 deviates from the virtual camera viewpoint screen S2. If so, the screen transmitter 121 may determine that the movement displacement of the screen S1 from the game user's point of view is equal to or greater than a preset threshold. The screen transmitting unit 121 may move the virtual camera view point screen in response to the movement displacement d of the game user view point screen and then transmit it to the broadcasting server 104 .

Referring back to FIG. 1 , the broadcast server 104 may be a server computing device for providing a broadcast service for a virtual reality game played by a game user. The broadcast server 104 may receive a virtual camera viewpoint view from the gaming device 102 . The broadcast server 104 may transmit the virtual camera viewpoint screen to the viewer terminal 106 .

The viewer terminal 106 may be a terminal of a viewer watching a broadcast for a virtual reality game. The viewer terminal 106 may access the broadcast server 104 and receive a virtual camera viewpoint from the broadcast server 104 . The viewer terminal 106 may include a smart phone, a tablet PC, a notebook computer, and a desktop computer.

According to the disclosed embodiment, a virtual camera viewpoint screen separate from the game user viewpoint screen is generated, and if the movement displacement of the game user viewpoint screen is less than a preset threshold, the virtual camera viewpoint screen is transmitted as it is, If the movement displacement is greater than a preset threshold, the virtual camera view point screen is moved and transmitted as much as it corresponds to it, thereby reducing confusion or dizziness felt by viewers watching the game broadcast due to the shaking of the game screen, and at the same time reducing the gaze of the game user. As you follow along, you will be able to see the game screen.

And, by using a game screen separate from the game user's point of view screen called the virtual camera point of view screen, even if a small shake occurs in the game user's point of view, the screen from the virtual camera point of view is transmitted to the viewer as it is, so the game screen that viewers see ( That is, it is possible to reduce the occurrence of small shakes in the virtual camera viewpoint).

Meanwhile, here, it is determined whether the movement displacement of the game user's view screen is less than or greater than a preset threshold by comparing it with the virtual camera view screen, but the present invention is not limited thereto. That is, the screen transmitter 121 may extract a feature point from the game user's point of view, and determine whether to correct the virtual camera's point of view according to whether the movement displacement of the feature point is less than or greater than a preset threshold.

Specifically, the screen transmitter 121 may extract a feature point for each frame of the screen from the game user's point of view. The screen transmitter 121 may check the movement displacement of the feature point between the current frame (n-th frame) and the previous frame (n-1 th frame) of the screen from the game user's point of view.

7 is a diagram illustrating movement displacement of a feature point between frames of a screen from a game user's point of view according to an embodiment disclosed herein. Referring to FIG. 7 , the screen transmitter 121 may extract feature points A, B, C, D, and E for each frame (eg, frame 1 and frame 2) of the screen from the game user's point of view.

The screen transmitter 121 is configured to generate feature points (A, B, C, D) between the n-th frame (eg, frame 2) of the game user's point of view and the n-1 th frame (eg, frame 1) that is the previous frame. , E) can be checked for movement displacement. In an exemplary embodiment, the screen transmitter 121 calculates a vector according to the movement of each feature point (A, B, C, D, E), and averages the vector for each of the feature points (A, B, C, D, E) It is possible to calculate an average vector according to their movement. Here, when the gaze movement direction of the game user moves to the right, the screen from the game user's point of view moves to the left (that is, the feature points move to the left on the screen from the game user's point of view).

When the size of the average vector according to the movement of the feature points is less than a preset threshold, the screen transmitting unit 121 does not correct (that is, do not move) the virtual camera view screen corresponding to the frame of the game user view screen and broadcast server ( 104) can be transmitted.

When the size of the average vector according to the movement of the feature points is greater than or equal to a preset threshold, the screen transmitting unit 121 corrects (ie, moves) the virtual camera viewpoint screen corresponding to the frame of the game user viewpoint screen to broadcast server 104 can be sent to

8 is a diagram illustrating a state in which a frame of a virtual camera view point is moved and corrected according to an embodiment of the present disclosure; Referring to FIG. 8 , when the size of the average vector according to the movement of the feature points is greater than or equal to a preset threshold, the screen transmitter 121 displays the virtual camera view screen S2 corresponding to the frame of the game user view screen by moving the feature points. (For example, the size of the average vector according to the movement of the feature points) may be moved.

In this case, the screen transmitter 121 moves the virtual camera view point screen S2 in the direction opposite to the moving direction of the feature points (eg, the direction of the average vector according to the movement of the feature points) (ie, the same direction as the gaze movement direction of the game user). ) can be moved.

9 is a diagram illustrating a configuration of a virtual reality game broadcasting service system according to another disclosed embodiment. Here, a part different from the embodiment shown in FIG. 1 will be mainly described.

Referring to FIG. 9 , the virtual reality game broadcasting service system 200 may include a gaming device 202 , a broadcasting server 204 , and a viewer terminal 206 . The gaming device 202 and the viewer terminal 206 are each communicatively connected to the broadcast server 204 via a communication network 250 .

The gaming device 202 may provide a virtual reality game via the head mounted display 208 . The gaming device 202 may be mounted on the head mounted display 208 or formed integrally with the head mounted display 208 , but is not limited thereto and communicates with the head mounted display 208 through wired, wireless, wired or wireless, or the like. possible to be connected.

The gaming device 202 may execute a game application according to a command of a game user. The gaming device 202 may generate a game user's viewpoint screen based on sensing data that senses one or more of the position and posture of the game user. The gaming device 202 may transmit the game user's viewpoint screen to the broadcast server 204 . The gaming device 202 may transmit the game user's viewpoint screen to the broadcast server 204 in frame units.

The gaming device 202 may generate gaze direction information by checking the gaze direction of the game user in the game. The gaming device 202 may transmit gaze direction information to the broadcast server 204 .

The broadcast server 204 may correct the game user's viewpoint screen received from the gaming device 202 according to the movement displacement of the game user's viewpoint screen and transmit it to the viewer terminal 206 . 10 is a block diagram showing the configuration of the broadcast server 204 according to the disclosed embodiment. Referring to FIG. 10 , the broadcast server 204 may include a communication unit 211 , a feature point extraction unit 213 , a screen correction unit 215 , and a gaze indicator generation unit 217 .

In an embodiment, the communication unit 211 , the feature point extraction unit 213 , the screen correction unit 215 , and the gaze indicator generation unit 217 are implemented using one or more physically separated devices, or one or more processors or It may be implemented by a combination of one or more processors and software, and unlike the illustrated example, specific operations may not be clearly distinguished.

The communication unit 211 may communicate with the gaming device 202 and the viewer terminal 206 . The communication unit 211 may receive a game user view screen from the gaming device 202 . The communication unit 211 may receive gaze direction information from the gaming device 202 . The communication unit 211 may transmit the game user's viewpoint screen to the viewer terminal 206 .

The key point extraction unit 213 may extract the key point from the game user's viewpoint screen received by the communication unit 211 . In an exemplary embodiment, the feature point extraction unit 213 may extract a feature point for each frame of the screen from the game user's point of view. For the key point extraction, various known methods for extracting the key point from the image may be used.

The screen corrector 215 may determine whether to correct the game user's view screen based on the movement displacement of the feature point on the game user's view screen. The screen compensator 215 may check the shift displacement of the feature point between the current frame (n-th frame) and the previous frame (n-1 th frame) of the screen from the game user's point of view. In an exemplary embodiment, the screen corrector 215 may calculate an average vector according to the movement of each of the extracted feature points, respectively, and calculate an average vector according to the movement of the feature points by averaging them.

When the size of the average vector according to the movement of the feature points is less than a preset threshold, the screen corrector 215 does not correct (ie, do not move) the corresponding frame (ie, the nth frame) of the screen from the game user's point of view. It may decide to transmit to 206 . The screen compensator 215 may transmit a corresponding frame of the game user's point of view to the viewer terminal 206 through the communication unit 211 .

When the size of the average vector according to the movement of the feature points is greater than or equal to a preset threshold, the screen corrector 215 may determine to transmit the frame to the viewer terminal 206 by correcting (ie, moving) the frame of the screen from the game user's point of view. there is.

When the size of the average vector according to the movement of the characteristic points is greater than or equal to a preset threshold, the screen corrector 215 converts the frame of the screen from the game user's point of view to the movement displacement of the characteristic points (for example, the size of the average vector according to the movement of the characteristic points) can be moved in response. In this case, the screen compensator 215 may move the corresponding frame in a direction opposite to the movement direction of the feature points (eg, the direction of the average vector according to the movement of the feature points).

The gaze indicator generating unit 217 may generate a gaze indicator for indicating the gaze direction of the game user in the game based on the gaze direction information received by the communication unit 211 . The gaze indicator generating unit 217 may render and display the gaze indicator on the screen of the game user's point of view.

According to the disclosed embodiment, the broadcast server 204 post-corrects the game user's view screen according to the movement displacement of the game user's view screen and transmits it to the viewer terminal 206, so that viewers watching the game broadcast shake the game screen. This can reduce the confusion or dizziness you feel. And, by rendering and displaying the gaze indicator on the game user's point of view screen, viewers can see the game screen while following the game user's gaze.

Meanwhile, although it has been described herein that the broadcasting server 204 determines whether to correct the game user's view screen according to the movement displacement of the feature point after extracting the feature point from the game user's point of view, it is not limited thereto.

Specifically, the broadcast server 204 may receive sensing data that senses one or more of the position and posture of the game user from the gaming device 202 . The broadcast server 204 may check the movement displacement of the screen from the game user's point of view based on the sensed data. That is, the broadcast server 204 may check in which direction and how much the screen from the game user's point of view has moved based on the sensed data.

When the movement displacement of the game user's view screen is less than a preset threshold, the broadcast server 204 may transmit the game user's view screen to the viewer terminal 206 without correcting it. When the movement displacement of the game user's view screen is equal to or greater than a preset threshold, the broadcast server 204 may move the game user's view screen and transmit it to the viewer terminal 206 .

That is, the broadcast server 204 may move the game user's view screen in a direction opposite to the moving direction of the game user's view screen (ie, the same direction as the game user's gaze moving direction). In this case, the broadcasting server 204 may move the game user's viewpoint screen in the opposite direction by the amount of movement displacement of the game user's viewpoint screen.

11 is a block diagram illustrating and describing a computing environment 10 including a computing device suitable for use in example embodiments. In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components in addition to those described below.

The illustrated computing environment 10 includes a computing device 12 . In one embodiment, computing device 12 may be gaming device 102 , 202 . Computing device 12 may also be broadcast server 104 , 204 .

Computing device 12 includes at least one processor 14 , computer readable storage medium 16 , and communication bus 18 . The processor 14 may cause the computing device 12 to operate in accordance with the exemplary embodiments discussed above. For example, the processor 14 may execute one or more programs stored in the computer-readable storage medium 16 . The one or more programs may include one or more computer-executable instructions that, when executed by the processor 14, configure the computing device 12 to perform operations in accordance with the exemplary embodiment. can be

Computer-readable storage medium 16 is configured to store computer-executable instructions or program code, program data, and/or other suitable form of information. The program 20 stored in the computer readable storage medium 16 includes a set of instructions executable by the processor 14 . In one embodiment, computer-readable storage medium 16 includes memory (volatile memory, such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash It may be memory devices, other forms of storage medium accessed by computing device 12 and capable of storing desired information, or a suitable combination thereof.

Communication bus 18 interconnects various other components of computing device 12 , including processor 14 and computer readable storage medium 16 .

Computing device 12 may also include one or more input/output interfaces 22 and one or more network communication interfaces 26 that provide interfaces for one or more input/output devices 24 . The input/output interface 22 and the network communication interface 26 are coupled to the communication bus 18 . Input/output device 24 may be coupled to other components of computing device 12 via input/output interface 22 . Exemplary input/output device 24 may include a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touchscreen), a voice or sound input device, various types of sensor devices, and/or imaging devices. input devices and/or output devices such as display devices, printers, speakers and/or network cards. The exemplary input/output device 24 may be included in the computing device 12 as a component constituting the computing device 12 , and may be connected to the computing device 12 as a separate device distinct from the computing device 12 . may be

Although representative embodiments of the present invention have been described in detail above, those of ordinary skill in the art to which the present invention pertains will understand that various modifications are possible without departing from the scope of the present invention with respect to the above-described embodiments. . Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the claims described below as well as the claims and equivalents.

100, 200: Virtual reality game broadcasting service system
102, 202: gaming device
104, 204: broadcast server
106, 206: viewer terminal
108, 208: head mounted display
111: game execution unit
113: sensor unit
115: first screen providing unit
117: second screen providing unit
119: gaze indicator generating unit
121: screen transmitter
211: communication department
213: feature point extraction unit
215: screen correction unit
217: gaze indicator generation unit

Claims (23)

one or more processors, and
A method performed in a computing device having a memory storing one or more programs to be executed by the one or more processors, the method comprising:
an operation of acquiring sensing data sensing a motion of a game user according to the progress of a virtual reality game;
acquiring a game user's point of view screen viewed from the game user's point of view based on the sensing data;
acquiring a virtual camera viewpoint screen viewed from a viewpoint of a virtual camera provided in the virtual reality game; and
and transmitting the virtual camera viewpoint screen as a broadcast screen instead of the game user's viewpoint screen. The method according to claim 1,
The virtual camera, the virtual reality game broadcasting service method that is provided to view the game from the position of the game user in the virtual reality game. 3. The method according to claim 2,
The operation of transmitting to the broadcast screen is,
and determining whether to move the virtual camera view point screen according to the movement displacement of the game user view point screen. 4. The method according to claim 3,
The virtual camera viewpoint screen is provided to be larger than the game user viewpoint screen, a virtual reality game broadcasting service method. 5. The method according to claim 4,
The operation of transmitting to the broadcast screen is,
and transmitting the virtual camera viewpoint screen without moving when the movement displacement of the game user's viewpoint screen is less than a preset threshold. 6. The method of claim 5,
When the movement displacement of the game user's view screen is made within the virtual camera view screen, it is determined that it is less than the preset threshold value, the virtual reality game broadcasting service method. 5. The method according to claim 4,
The operation of transmitting to the broadcast screen is,
and transmitting and moving the virtual camera viewpoint screen corresponding to the movement displacement of the game user's viewpoint screen when the movement displacement of the game user's view screen is equal to or greater than a preset threshold. 8. The method of claim 7,
When the movement displacement of the game user's point of view screen deviates from the virtual camera view point, it is determined that it is equal to or greater than the preset threshold. 4. The method according to claim 3,
The operation of determining whether to move the virtual camera viewpoint screen,
extracting feature points for each frame of the screen from the game user's point of view;
checking the movement displacement of the feature point between each frame of the screen from the game user's point of view; and
and determining whether to move the virtual camera view point screen according to the movement displacement of the feature point. 10. The method of claim 9,
The operation of determining whether to move the virtual camera viewpoint screen,
and, when the movement displacement of the feature point is less than a preset threshold, transmitting the virtual camera viewpoint without moving the screen. 10. The method of claim 9,
The operation of determining whether to move the virtual camera viewpoint screen,
and when the movement displacement of the key point is equal to or greater than a preset threshold, moving and transmitting the virtual camera viewpoint screen in a direction opposite to the movement direction of the key point in response to the movement displacement of the key point. The method according to claim 1,
The virtual reality game broadcasting service method,
generating a gaze indicator by checking the gaze direction of the game user in the virtual reality game; and
Further comprising the operation of displaying the gaze indicator on the view point screen of the virtual camera, the virtual reality game broadcasting service method. one or more processors, and
A computing device having a memory to store one or more programs executed by the one or more processors, the computing device comprising:
a sensor unit that senses the motion of a game user according to the progress of the virtual reality game and generates sensing data;
a first screen providing unit for generating a screen from the point of view of the game user viewed from the point of view of the game user based on the sensing data;
a second screen providing unit for generating a virtual camera viewpoint viewed from the viewpoint of the virtual camera provided in the virtual reality game; and
Computing device comprising a screen transmitter for transmitting the virtual camera view screen instead of the game user view screen as a broadcast screen. 14. The method of claim 13,
wherein the virtual camera is provided to view the game from the position of the game user within the virtual reality game. 15. The method of claim 14,
The screen transmitter,
A computing device that determines whether or not to move the virtual camera view point screen according to the movement displacement of the game user view point screen and transmits it. 16. The method of claim 15,
The virtual camera viewpoint screen is provided to be larger than the game user viewpoint screen, a computing device. 17. The method of claim 16,
The screen transmitter,
When the movement displacement of the game user's viewpoint screen is made within the virtual camera viewpoint screen, the computing device transmits the virtual camera viewpoint screen without moving. 17. The method of claim 16,
The screen transmitter,
When the movement displacement of the game user's viewpoint screen deviates from the virtual camera viewpoint screen, the computing device moves and transmits the virtual camera viewpoint screen in response to the movement displacement of the game user viewpoint screen. 16. The method of claim 15,
The screen transmitter,
A computing device for extracting a feature point for each frame of the game user view screen, confirming a feature point movement displacement between each frame of the game user view point screen, and determining whether to move the virtual camera view point screen according to the feature point movement displacement . 20. The method of claim 19,
The screen transmitter,
When the feature point movement displacement is less than a preset threshold, the computing device transmits the virtual camera viewpoint without moving the screen. 20. The method of claim 19,
The screen transmitter,
When the movement displacement of the key point is equal to or greater than a preset threshold, the virtual camera view point screen is moved in a direction opposite to the movement direction of the key point in response to the movement displacement of the key point and transmitted. 14. The method of claim 13,
The computing device,
The computing device further comprising a gaze indicator generating unit generating a gaze indicator by checking the gaze direction of the game user in the virtual reality game, and displaying the gaze indicator on the virtual camera viewpoint screen. a head mounted display worn on the user's head;
A virtual reality game is provided through the head mounted display, and a game user's viewpoint screen viewed from the game user's viewpoint is provided to the game user by sensing the movement of the game user according to the progress of the virtual reality game, and the virtual reality a gaming device for generating a virtual camera viewpoint viewed from the viewpoint of a virtual camera provided in the game, and transmitting the virtual camera viewpoint screen as a broadcast screen instead of the game user viewpoint; and
A virtual reality game broadcasting service system, comprising a broadcasting server that receives the virtual camera viewpoint from the gaming device and transmits it to a viewer terminal.

Images