KR102423869B1 - Method for broadcasting service of virtual reality game, apparatus and system for executing the method - Google Patents

Abstract

Disclosed are a virtual reality game broadcasting service method and an apparatus and system for performing the same. A virtual reality game broadcasting service method according to an embodiment of the present disclosure is a method performed in a computing device having one or more processors and a memory for storing one or more programs executed by the one or more processors, It includes an operation of acquiring a game user's viewpoint screen viewed from the game user's viewpoint according to the progress of the game user, and an operation of determining whether to move the game user viewpoint screen according to the movement displacement of the game user viewpoint screen and transmitting the image.

Description

Virtual reality game broadcasting service method and apparatus and system for performing the same

The disclosed embodiment relates to a virtual reality game broadcasting service technology.

Recently, games are evolving in various ways, from local games executed only within the gaming device to online games executed through interworking with other gaming devices or servers through a network. In addition, with the improvement of hardware performance and network technology, games are evolving so that games can be executed not only on personal computers or game consoles but also on mobile devices such as smart phones and tablet PCs.

On the other hand, there is also an increasing trend in game broadcasting, in which the game that the game user is playing is transmitted through the broadcast so that the viewers can watch it. Here, in the case of VR (Virtual Reality) game broadcasting, since the game user proceeds to play while the head mounted display is mounted, the game screen moves or shakes according to the change in the gaze of the game user. There is a problem in that you feel confused and dizzy while watching.

The disclosed embodiment is to provide a virtual reality game broadcasting service method capable of reducing shaking of a game screen according to a change in the gaze movement of a game user, and an apparatus and system for performing the same.

A virtual reality game broadcasting service method according to an embodiment of the present disclosure is a method performed in a computing device having one or more processors and a memory for storing one or more programs executed by the one or more processors, the virtual reality an operation of acquiring a game user's point of view screen viewed from the game user's point of view according to the progress of the game; and determining whether or not to move the game user's view screen according to the movement displacement of the game user's view screen and transmitting it.

The operation of determining whether to move the screen from the game user's point of view and transmitting it may include: extracting a feature point for each frame of the screen from the game user's point of view; checking the movement displacement of the feature point between each frame of the screen from the game user's point of view; and determining whether to move the screen from the game user's point of view according to the movement displacement of the feature point.

The operation of determining whether to move the game user's viewpoint screen and transmitting the determination may include an operation of transmitting the game user's viewpoint without moving the screen when the movement displacement of the feature point is less than a preset threshold.

The operation of determining whether to move the screen from the game user's point of view and transmitting it may include moving the screen from the game user's point of view in a direction opposite to the moving direction of the feature point in response to the movement displacement of the feature point when the displacement of the feature point is greater than or equal to a preset threshold. and may include the operation of sending and receiving.

The operation of confirming the movement displacement of the characteristic point may include calculating a vector according to the movement of each of the extracted characteristic points; calculating an average vector according to movement of the feature points by averaging the calculated vectors; and checking the magnitude and direction of the average vector according to the movement of the feature points.

The operation of determining whether to move the screen of the game user's point of view and transmitting it may include: acquiring sensing data sensing the movement of the game user according to the progress of the virtual reality game; checking the movement displacement of the screen from the user's point of view based on the sensed data; and determining whether to move the game user's view screen according to the movement displacement of the game user's view screen.

The operation of determining whether to move the game user's viewpoint screen and transmitting the determination may include an operation of transmitting the game user's viewpoint screen without moving the screen when the movement displacement of the game user's viewpoint screen is less than a preset threshold.

The operation of determining whether to move the game user's point of view screen and transmitting the determination may include, when the moving displacement of the game user's point of view screen is equal to or greater than a preset threshold, turning the game user's point of view screen in a direction opposite to the moving direction of the game user's point of view screen. It may include an operation of moving and transmitting in response to the movement displacement of the screen of the game user's point of view.

The virtual reality game broadcasting service method may include: acquiring gaze direction information of the game user in the virtual reality game; generating a gaze indicator based on the gaze direction information; and displaying the gaze indicator on the screen from the game user's point of view.

A computing device according to an embodiment of the disclosure is a computing device having one or more processors and a memory for storing one or more programs executed by the one or more processors. a communication unit for acquiring a game user's point of view screen viewed from a point of view; and a screen compensator for determining whether to move the game user's view screen according to the movement displacement of the game user's view screen and transmitting it.

The computing device may further include a feature point extracting unit for extracting a feature point for each frame of the game user's view screen, and the screen correcting unit checks a feature point movement displacement between each frame of the game user's point of view screen, and the feature point movement displacement Accordingly, it is possible to determine whether to move the screen from the game user's point of view.

The screen compensator may transmit the game user's view screen without moving the screen when the feature point movement displacement is less than a preset threshold.

The screen corrector may be configured to move and transmit the screen from the game user's point of view in a direction opposite to the moving direction of the key point in response to the movement displacement of the key point when the displacement of the key point is greater than or equal to a preset threshold.

The screen compensator, respectively, calculates a vector according to the movement of each of the extracted characteristic points, averages the calculated vectors to calculate an average vector according to the movement of the characteristic points, and the magnitude and direction of the average vector according to the movement of the characteristic points can confirm.

The communication unit obtains sensing data for sensing a motion of a game user according to the progress of the virtual reality game, and the screen correction unit checks a movement displacement of the game user's view screen based on the sensing data, and the game Whether to move the game user's point of view screen may be determined according to the movement displacement of the user's point of view screen.

The screen corrector may transmit the game user's view screen without moving the screen when the movement displacement of the game user's view point is less than a preset threshold.

When the movement displacement of the game user's view screen is equal to or greater than a preset threshold, the game user's view screen can be moved and transmitted in the opposite direction to the moving direction of the game user's view screen in response to the movement displacement of the game user's view screen have.

The communication unit obtains gaze direction information of the game user in the virtual reality game, the computing device generates a gaze indicator based on the gaze direction information, and displays the gaze indicator on the game user viewpoint screen It may further include a gaze indicator generator.

A virtual reality game broadcasting service system according to an embodiment of the present disclosure includes: a head mounted display worn on a user's head; a gaming device that provides a virtual reality game through the head mounted display, senses a game user's movement according to the progress of the virtual reality game, and provides a game user's point of view screen viewed from the game user's point of view to the game user; and a broadcasting server that receives the game user's view screen from the gaming device, determines whether to move the game user's view screen according to a movement displacement of the game user's view screen, and transmits it to a viewer terminal.

According to the disclosed embodiment, the game user's view screen is post-corrected according to the movement displacement of the game user's view screen in the broadcasting server and transmitted to the viewer terminal, thereby causing confusion or dizziness felt by viewers watching the game broadcast due to the shaking of the game screen. can be reduced And, by rendering and displaying the gaze indicator on the game user's point of view screen, viewers can see the game screen while following the game user's gaze.

In addition, by displaying and transmitting the gaze indicator on the screen of the game user's point of view, viewers can check the gaze direction of the game user in the game, thereby increasing the sense of immersion in the virtual reality game broadcasting.

In addition, a virtual camera viewpoint screen separate from the game user viewpoint screen is generated, and if the movement displacement of the game user viewpoint screen is less than a preset threshold, the virtual camera viewpoint screen is transmitted as it is, and the movement displacement of the game user viewpoint screen is preset. If the threshold is higher than the threshold, the virtual camera view screen is moved and transmitted accordingly, thereby reducing the confusion or dizziness felt by viewers watching the game broadcast due to the shaking of the game screen. be able to see

And, by using a game screen separate from the game user's point of view screen called the virtual camera point of view screen, even if a small shake occurs in the game user's point of view, the screen from the virtual camera point of view is transmitted to the viewer as it is, so the game screen that viewers see ( That is, it is possible to reduce the occurrence of small shakes in the virtual camera viewpoint).

1 is a diagram showing the configuration of a virtual reality game broadcasting service system according to an exemplary embodiment;
2 is a block diagram illustrating a configuration of a gaming device according to an embodiment of the present disclosure;
3 is a view comparing a screen from a game user's point of view and a screen from a virtual camera's point of view in the disclosed embodiment;
4 is a view schematically showing a state in which a gaze indicator is displayed on a virtual camera viewpoint screen in an embodiment to be disclosed;
5 is a diagram illustrating a case in which a movement displacement of a screen from a game user's point of view is less than a preset threshold in an disclosed embodiment;
6 is a diagram illustrating a case in which a movement displacement of a screen from a game user's point of view is greater than or equal to a preset threshold in an disclosed embodiment;
7 is a diagram illustrating a feature point movement displacement between frames of a screen from a game user's point of view in an embodiment to be disclosed;
8 is a diagram illustrating a state in which a frame of a virtual camera view point screen is moved and corrected according to an embodiment of the present disclosure;
9 is a diagram showing the configuration of a virtual reality game broadcasting service system according to another disclosed embodiment;
10 is a block diagram showing the configuration of a broadcast server according to the disclosed embodiment;
11 is a block diagram illustrating and describing a computing environment including a computing device suitable for use in example embodiments.

Hereinafter, specific embodiments will be described with reference to the drawings. The following detailed description is provided to provide a comprehensive understanding of the methods, apparatus, and/or systems described herein. However, this is only an example and is not limited thereto.

The terminology used in the detailed description is for the purpose of describing the embodiments only, and should in no way be limiting. Unless explicitly used otherwise, expressions in the singular include the meaning of the plural. As used herein, expressions such as “comprising” or “comprising” are intended to indicate certain features, numbers, steps, acts, elements, some or a combination thereof, and one or more other than those described. It should not be construed to exclude the presence or possibility of other features, numbers, steps, acts, elements, or any part or combination thereof.

In the following description, the terms "transmission", "communication", "transmission", "reception" and other similar meanings of a signal or information are not only directly transmitted from one component to another component, but also a signal or information This includes passing through other components. In particular, to “transmit” or “transmit” a signal or information to a component indicates the final destination of the signal or information and does not imply a direct destination. The same is true for "reception" of signals or information. In addition, in this specification, when two or more data or information are "related", it means that when one data (or information) is acquired, at least a part of other data (or information) can be acquired based thereon.

Also, terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The above terms may be used for the purpose of distinguishing one component from another. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component.

Hereinafter, "game" means software that one or more gamers can enjoy using an input device mounted on or connected to a gaming device. Usually, it is performed by directly manipulating the player character or by manipulating the movement of objects in the game, and genres such as role-playing, adventure, and arcade are sometimes classified.

Hereinafter, the term "gaming device" refers to hardware that executes game software but performs calculations according to a user's input through an input device and outputs a result through an output device. For example, personal computer (Personal Computer), laptop (Laptop Computer), game console (Game Console), smart phone (Smart Phone), tablet (Tablet) PC, smart band (Smart Band) or smart watch (Smart Watch) It may be in the form of a wearable device. In addition, hardware that satisfies the above definition is interpreted as belonging to a gaming device.

Hereinafter, "online game" means a game that must go through a network in order to play the game. In general, it refers to a game in which two or more gaming devices in a remote location are connected to each other through a network or synchronized through a server. As long as it satisfies the above definition, there is no limitation on the type of gaming device that executes the game. Not only games executed on personal computers or game consoles, but also games executed on mobile devices such as smartphones, or games in which a plurality of different gaming devices (including different types of gaming devices) are linked or executed in the cloud If it satisfies the definition, it is interpreted as belonging to an online game.

Hereinafter, an “input device” refers to a device for inputting data into a gaming device. For example, a joystick, a mouse, a keyboard, a touch screen, a hardware button or a steering wheel, a musical instrument, a gun, a glove, an input device dedicated to game manipulation manufactured in the form of a footrest may be mentioned. In addition, various sensors (temperature sensor, infrared sensor, motion sensor, gyro sensor, acceleration sensor, gravity sensor, etc.) installed or linked to gaming devices, Global Positioning System (GPS), Indoor Positioning System ), etc., may also be used as an example of an input device.

1 is a diagram showing the configuration of a virtual reality game broadcasting service system according to an exemplary embodiment.

Referring to FIG. 1 , a virtual reality game broadcasting service system 100 may include a gaming device 102 , a broadcasting server 104 , and a viewer terminal 106 . Gaming device 102 and viewer terminal 106 are each communicatively connected to broadcast server 104 via communication network 150 .

In some embodiments, communication network 150 is the Internet, one or more local area networks, wide area networks, cellular networks, mobile networks, other types of networks, or such networks. may include combinations of these.

One or more gaming applications may be installed on the gaming device 102 . Here, the game application may refer to a program installed in the gaming device 102 to provide a game service to a game user. In an exemplary embodiment, the gaming device 102 may be mounted to a head mounted display 108 . However, the present invention is not limited thereto, and the gaming device 102 and the head mounted display 108 may be integrally formed. In addition, the gaming device 102 may be communicatively connected to the head mounted display 108 through wireless, wired, and wired or wireless methods.

The gaming device 102 may provide a virtual reality game via the head mounted display 108 . The head mounted display 108 may be worn on the head of the game user. The gaming device 102 may provide the game screen from the viewpoint of the game user (ie, the player) according to the movement of the game user wearing the head mounted display 108 .

2 is a block diagram illustrating a configuration of a gaming device 102 according to an disclosed embodiment.

Referring to FIG. 2 , the gaming device 102 includes a game execution unit 111 , a sensor unit 113 , a first screen providing unit 115 , a second screen providing unit 117 , and a gaze indicator generating unit 119 . , and a screen transmitter 121 .

In an embodiment, the game execution unit 111 , the sensor unit 113 , the first screen providing unit 115 , the second screen providing unit 117 , the gaze indicator generating unit 119 , and the screen transmitting unit 121 . may be implemented using one or more physically separated devices, or may be implemented by one or more processors or a combination of one or more processors and software, and may not be clearly distinguished in specific operation unlike the illustrated example.

The game execution unit 111 may execute a game application according to a command of a game user. The game execution unit 111 may proceed with the game according to a command input from the game user after the game is started. The game user may input a command to proceed with the game through the input device.

The sensor unit 113 may sense one or more of the position and posture of the game user. That is, the sensor unit 113 may sense the motion of the game user wearing the head mounted display 108 . The game user performs actions such as turning his head, bowing his head, or moving his arm depending on the game situation while playing the virtual reality game. One or more of the postures may be sensed. In an exemplary embodiment, the sensor unit 113 may include a gyro sensor, an acceleration sensor, and a gravity sensor.

The first screen providing unit 115 may provide a game screen viewed from a game user's point of view. The first screen providing unit 115 provides a game screen viewed from the game user's point of view based on the sensing data sensed by the sensor unit 113 (that is, a sensing value for one or more of the game user's position and posture). can Hereinafter, a game screen viewed from a game user's point of view may be referred to as a "game user's point of view screen".

Since the game user's view screen is a game screen in which the position and posture change of the game user are reflected as it is during the game, movement or shaking occurs according to the change in the position and posture of the game user. In this case, the screen from the game user's point of view is changed in movement based on the center of the screen from the game user's point of view according to the movement of the game user.

The second screen providing unit 117 may provide a game screen viewed from the viewpoint of the virtual camera. The virtual camera is provided virtually in the game, and may be provided to view the game from a fixed position of the game user. The game screen viewed from the viewpoint of the virtual camera is a game screen separate from the game user's viewpoint, and may be a game screen for transmission to the viewer terminal 106 .

That is, the game screen viewed from the viewpoint of the virtual camera may be a broadcast game screen to be transmitted to the viewer. Hereinafter, the game screen viewed from the viewpoint of the virtual camera may be referred to as a “virtual camera viewpoint”. The virtual camera view screen is a screen viewed from a fixed point of view of the virtual camera, and is not affected by changes in the position and posture of the game user.

Here, the virtual camera viewpoint screen may be provided larger than the game user viewpoint screen. 3 is a diagram comparing a screen from a game user's point of view and a screen from a virtual camera's point of view according to an embodiment of the present disclosure. Referring to FIG. 3 , it can be seen that the virtual camera view screen S2 is larger than the game user view screen S1 .

In an exemplary embodiment, in a state in which the centers of the game user's viewpoint screen S1 and the virtual camera viewpoint screen S2 coincide, the virtual camera viewpoint screen S2 moves outward from each edge of the game user viewpoint screen S1. It may be provided to have an extended area. However, the present invention is not limited thereto, and the virtual camera viewpoint screen S2 may have the same size as the game user viewpoint screen S1 .

The gaze indicator generating unit 119 may generate an indicator (ie, a gaze indicator) for the gaze direction of the game user in the game. In an exemplary embodiment, the gaze indicator generating unit 119 may check the gaze direction of the game user in the game based on the sensed data sensed by the sensor unit 113 . For example, the gaze indicator generating unit 119 may check the gaze direction of the game user in the game by checking the head direction of the game user through the sensing data. However, the present invention is not limited thereto, and the gaze indicator generating unit 119 may check the gaze direction of the game user in the game through equipment such as a separate eye tracker.

After checking the gaze direction of the game user in the game, the gaze indicator generating unit 119 may generate a gaze indicator for indicating the gaze direction of the game user. In an exemplary embodiment, the gaze indicator generator 119 may render and display the gaze indicator on the virtual camera viewpoint screen.

4 is a diagram schematically illustrating a state in which a gaze indicator is displayed on a virtual camera viewpoint screen according to an embodiment of the present disclosure. Referring to FIG. 4 , the gaze indicator I may be rendered and displayed on the virtual camera viewpoint screen S2 .

Here, the gaze indicator I may be implemented in 3D. That is, the gaze indicator I may be implemented in 3D to indicate the gaze direction of the game user in three-axis directions (ie, the x-axis, the y-axis, and the z-axis).

Since the virtual camera viewpoint screen S2 is a game screen provided to the viewer, by displaying the gaze indicator I on the virtual camera viewpoint screen S2, viewers can check the gaze direction of the game user in the game, and As a result, it is possible to increase the sense of immersion in virtual reality game broadcasting.

The screen transmitter 121 may transmit a virtual camera viewpoint screen to the broadcast server 104 . That is, in the disclosed embodiment, the screen from the viewpoint of the game user is not transmitted to the broadcasting server 104 , but the screen from the viewpoint of the virtual camera is transmitted to the broadcasting server 104 . In this case, the screen transmitter 121 may transmit the virtual camera view screen as it is (ie, transmit without movement) or move and transmit the virtual camera view point screen according to the movement displacement of the game user view point screen S1.

Specifically, when the movement displacement of the game user's view screen is less than a preset threshold, the screen transmitter 121 does not correct the virtual camera view screen according to the movement of the game user's view screen (ie, does not move) the broadcast server 104 ) can be sent.

That is, when the game user's view screen is moved by the game user's movement, and the movement displacement of the game user's view screen is less than a preset threshold (for example, the game user's movement is small or minute, such as blinking eyes or moving eyes, etc.) If it is movement and the displacement of the screen from the game user's point of view is less than a preset threshold), the screen transmitter 121 may transmit the virtual camera point of view to the broadcasting server 104 without correcting it according to the movement of the screen from the game user's point of view.

5 is a diagram illustrating a case in which a movement displacement of a screen from a game user's point of view is less than a preset threshold in an embodiment disclosed herein. Referring to FIG. 5 , when the movement displacement d of the game user viewpoint screen S1 is less than a preset threshold, the movement displacement of the game user viewpoint screen S1 is set to be made within the virtual camera viewpoint screen S2 can be

For example, if the center of the game user view screen S1 and the virtual camera view screen S2 coincide with each other, if the movement displacement d of the game user view screen S1 is within the virtual camera view screen S2 , the screen transmitter 121 may determine that the movement displacement of the screen S1 from the game user's point of view is less than a preset threshold. However, the present invention is not limited thereto, and if the movement displacement d of the game user viewpoint screen S1 is slightly out of the virtual camera viewpoint screen S2, the movement displacement of the game user viewpoint screen S1 is preset. It can be judged to be less than a threshold value.

In addition, when the movement displacement of the game user's view screen is greater than or equal to a preset threshold, the screen transmitter 121 corrects the virtual camera view screen according to the movement of the game user's view screen (ie, moves) and transmits it to the broadcasting server 104 . can do. In this case, the screen transmitter 121 may move the virtual camera view point screen in response to the movement displacement of the game user view point screen. That is, the screen transmitter 121 may move the virtual camera viewpoint screen by the movement displacement of the game user's viewpoint screen.

6 is a diagram illustrating a case in which a movement displacement of a screen from a game user's point of view is greater than or equal to a preset threshold in an embodiment disclosed herein. Referring to FIG. 6 , when the movement displacement d of the game user viewpoint screen S1 is greater than or equal to a preset threshold, the movement displacement of the game user viewpoint screen S1 is set to deviate from the virtual camera viewpoint screen S2. can

For example, in a state where the centers of the game user's viewpoint screen S1 and the virtual camera viewpoint screen S2 coincide with each other, the movement displacement d of the game user viewpoint screen S1 is out of the virtual camera viewpoint screen S2. surface, the screen transmitter 121 may determine that the movement displacement of the screen S1 from the game user's point of view is equal to or greater than a preset threshold. The screen transmitter 121 may move the virtual camera view point screen in response to the movement displacement d of the game user view point screen and then transmit it to the broadcast server 104 .

Referring back to FIG. 1 , the broadcast server 104 may be a server computing device for providing a broadcast service for a virtual reality game played by a game user. The broadcast server 104 may receive a virtual camera viewpoint view from the gaming device 102 . The broadcast server 104 may transmit the virtual camera viewpoint screen to the viewer terminal 106 .

The viewer terminal 106 may be a terminal of a viewer watching a broadcast for a virtual reality game. The viewer terminal 106 may access the broadcast server 104 and receive a virtual camera viewpoint from the broadcast server 104 . The viewer terminal 106 may include a smart phone, a tablet PC, a notebook computer, and a desktop computer.

According to the disclosed embodiment, a virtual camera viewpoint screen separate from the game user viewpoint screen is generated, and if the movement displacement of the game user viewpoint screen is less than a preset threshold, the virtual camera viewpoint screen is transmitted as it is, If the movement displacement is greater than or equal to a preset threshold, by moving and transmitting the virtual camera view point screen correspondingly, it is possible to reduce the confusion or dizziness felt by viewers watching the game broadcast due to the shaking of the game screen, and at the same time reduce the gaze of the game user. As you follow along, you will be able to see the game screen.

And, by using a game screen separate from the game user's point of view screen called the virtual camera point of view screen, even if a small shake occurs in the game user's point of view, the screen from the virtual camera point of view is transmitted to the viewer as it is, so the game screen that viewers see ( That is, it is possible to reduce the occurrence of small shakes in the virtual camera viewpoint).

Meanwhile, in this case, it is determined whether the movement displacement of the screen from the game user's point of view is less than or greater than a preset threshold by comparing it with the screen from the viewpoint of the virtual camera, but the present invention is not limited thereto. That is, the screen transmitter 121 may extract a feature point from the game user's viewpoint screen and determine whether to correct the virtual camera viewpoint screen according to whether the movement displacement of the feature point is less than or greater than a preset threshold.

Specifically, the screen transmitter 121 may extract a feature point for each frame of the screen from the game user's point of view. The screen transmitter 121 may check the movement displacement of the feature point between the current frame (n-th frame) and the previous frame (n-1 th frame) of the game user's view screen.

7 is a diagram illustrating movement displacement of a feature point between frames of a screen from a game user's point of view according to an embodiment disclosed herein. Referring to FIG. 7 , the screen transmitter 121 may extract feature points A, B, C, D, and E for each frame (eg, frame 1 and frame 2) of the screen from the game user's point of view.

The screen transmitting unit 121 is configured to generate the feature points A, B, C, D between the n-th frame (eg, frame 2) of the game user's point of view and the n-1 th frame (eg, frame 1) that is the previous frame. , E) can be checked for movement displacement. In an exemplary embodiment, the screen transmitter 121 calculates a vector according to the movement of each feature point (A, B, C, D, E), and averages the vector for the feature point (A, B, C, D, E) It is possible to calculate an average vector according to their movement. Here, when the movement direction of the game user's gaze moves to the right, the game user's view screen moves to the left (ie, the feature points move to the left on the game user's view screen).

When the size of the average vector according to the movement of the feature points is less than a preset threshold, the screen transmitting unit 121 does not correct (that is, do not move) the virtual camera view screen corresponding to the frame of the game user view screen and broadcast server ( 104) can be sent.

When the size of the average vector according to the movement of the feature points is greater than or equal to a preset threshold, the screen transmitting unit 121 corrects (ie, moves) the virtual camera view screen corresponding to the frame of the game user view screen to broadcast server 104 can be sent to

8 is a diagram illustrating a state in which a frame of a virtual camera view point screen is moved and corrected according to an embodiment of the present disclosure; Referring to FIG. 8 , when the size of the average vector according to the movement of the feature points is equal to or greater than a preset threshold, the screen transmitter 121 displays the virtual camera view screen S2 corresponding to the frame of the game user view screen by moving the feature points. (For example, the size of the average vector according to the movement of the feature points) may be moved.

At this time, the screen transmitter 121 moves the virtual camera view point screen S2 in the direction opposite to the moving direction of the feature points (eg, the direction of the average vector according to the movement of the feature points) (ie, the same direction as the gaze movement direction of the game user). ) can be moved.

9 is a diagram illustrating a configuration of a virtual reality game broadcasting service system according to another disclosed embodiment. Here, a part different from the embodiment shown in FIG. 1 will be mainly described.

Referring to FIG. 9 , the virtual reality game broadcasting service system 200 may include a gaming device 202 , a broadcasting server 204 , and a viewer terminal 206 . The gaming device 202 and the viewer terminal 206 are each communicatively connected to the broadcast server 204 via a communication network 250 .

The gaming device 202 may provide a virtual reality game via the head mounted display 208 . The gaming device 202 may be mounted on the head mounted display 208 or formed integrally with the head mounted display 208 , but is not limited thereto and communicates with the head mounted display 208 through wired, wireless, wired or wireless, or the like. possible to be connected.

The gaming device 202 may execute a game application according to a command of a game user. The gaming device 202 may generate a game user's viewpoint screen based on sensing data that senses one or more of the position and posture of the game user. The gaming device 202 may transmit the game user's viewpoint screen to the broadcast server 204 . The gaming device 202 may transmit the game user's view screen to the broadcast server 204 in units of frames.

The gaming device 202 may generate gaze direction information by checking the gaze direction of the game user in the game. The gaming device 202 may transmit gaze direction information to the broadcast server 204 .

The broadcast server 204 may correct the game user's viewpoint screen received from the gaming device 202 according to the movement displacement of the game user's viewpoint screen and transmit it to the viewer terminal 206 . 10 is a block diagram showing the configuration of the broadcast server 204 according to the disclosed embodiment. Referring to FIG. 10 , the broadcast server 204 may include a communication unit 211 , a feature point extraction unit 213 , a screen correction unit 215 , and a gaze indicator generation unit 217 .

In an embodiment, the communication unit 211 , the feature point extraction unit 213 , the screen correction unit 215 , and the gaze indicator generation unit 217 are implemented using one or more physically separated devices, or one or more processors or It may be implemented by a combination of one or more processors and software, and unlike the illustrated example, specific operations may not be clearly distinguished.

The communication unit 211 may communicate with the gaming device 202 and the viewer terminal 206 . The communication unit 211 may receive a game user's view screen from the gaming device 202 . The communication unit 211 may receive gaze direction information from the gaming device 202 . The communication unit 211 may transmit the game user's viewpoint screen to the viewer terminal 206 .

The key point extraction unit 213 may extract the key point from the game user's point of view screen received by the communication unit 211 . In an exemplary embodiment, the feature point extraction unit 213 may extract a feature point for each frame of the screen from the game user's point of view. For the key point extraction, various known methods for extracting the key point from the image may be used.

The screen corrector 215 may determine whether to correct the game user's view screen based on the movement displacement of the feature point on the game user's view screen. The screen compensator 215 may check the movement displacement of the feature point between the current frame (n-th frame) and the previous frame (n-1 th frame) of the screen from the game user's point of view. In an exemplary embodiment, the screen corrector 215 may calculate a vector according to the movement of each extracted feature point, and calculate an average vector according to the movement of the feature point by averaging them.

When the size of the average vector according to the movement of the feature points is less than a preset threshold, the screen corrector 215 does not correct (ie, does not move) the corresponding frame (ie, the nth frame) of the screen from the user's point of view. It may decide to transmit to 206 . The screen compensator 215 may transmit a corresponding frame of the game user's view screen to the viewer terminal 206 through the communication unit 211 .

When the size of the average vector according to the movement of the feature points is greater than or equal to a preset threshold, the screen correction unit 215 may determine to transmit the frame to the viewer terminal 206 by correcting (ie, moving) the frame of the screen from the game user's point of view. have.

When the size of the average vector according to the movement of the characteristic points is greater than or equal to a preset threshold, the screen corrector 215 converts the corresponding frame of the screen from the game user's point of view to the movement displacement of the characteristic points (for example, the size of the average vector according to the movement of the characteristic points) can be moved in response. In this case, the screen compensator 215 may move the corresponding frame in a direction opposite to the moving direction of the feature points (eg, the direction of the average vector according to the movement of the feature points).

The gaze indicator generation unit 217 may generate a gaze indicator for indicating the gaze direction of the game user in the game based on the gaze direction information received by the communication unit 211 . The gaze indicator generating unit 217 may render and display the gaze indicator on the game user's point of view screen.

According to the disclosed embodiment, the broadcast server 204 post-corrects the game user's view screen according to the movement displacement of the game user's view screen and transmits it to the viewer terminal 206, so that viewers watching the game broadcast shake the game screen. This will reduce the confusion or dizziness you may feel. And, by rendering and displaying the gaze indicator on the game user's point of view screen, viewers can see the game screen while following the game user's gaze.

Meanwhile, it has been described herein that the broadcasting server 204 determines whether to correct the game user's view screen according to the movement displacement of the feature point after extracting the feature point from the game user's point of view, but is not limited thereto.

Specifically, the broadcast server 204 may receive sensing data that senses one or more of the position and posture of the game user from the gaming device 202 . The broadcast server 204 may check the movement displacement of the screen from the game user's point of view based on the sensed data. That is, the broadcast server 204 may check in which direction and how much the screen from the game user's point of view has moved based on the sensed data.

When the movement displacement of the game user's view screen is less than a preset threshold, the broadcast server 204 may transmit the game user's view screen to the viewer terminal 206 without correcting it. When the movement displacement of the game user's view screen is equal to or greater than a preset threshold, the broadcast server 204 may move the game user's view screen and transmit it to the viewer terminal 206 .

That is, the broadcast server 204 may move the game user's view screen in a direction opposite to the moving direction of the game user's view screen (ie, the same direction as the game user's gaze moving direction). In this case, the broadcasting server 204 may move the game user's viewpoint screen in the opposite direction by the amount of movement displacement of the game user's viewpoint screen.

11 is a block diagram illustrating and describing a computing environment 10 including a computing device suitable for use in example embodiments. In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components in addition to those described below.

The illustrated computing environment 10 includes a computing device 12 . In one embodiment, computing device 12 may be gaming device 102 , 202 . Computing device 12 may also be broadcast server 104 , 204 .

Computing device 12 includes at least one processor 14 , computer readable storage medium 16 , and communication bus 18 . The processor 14 may cause the computing device 12 to operate in accordance with the exemplary embodiments discussed above. For example, the processor 14 may execute one or more programs stored in the computer-readable storage medium 16 . The one or more programs may include one or more computer-executable instructions that, when executed by the processor 14, configure the computing device 12 to perform operations in accordance with the exemplary embodiment. can be

Computer-readable storage medium 16 is configured to store computer-executable instructions or program code, program data, and/or other suitable form of information. The program 20 stored in the computer readable storage medium 16 includes a set of instructions executable by the processor 14 . In one embodiment, computer-readable storage medium 16 includes memory (volatile memory, such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash It may be memory devices, other forms of storage medium that can be accessed by computing device 12 and store desired information, or a suitable combination thereof.

Communication bus 18 interconnects various other components of computing device 12 , including processor 14 and computer readable storage medium 16 .

Computing device 12 may also include one or more input/output interfaces 22 and one or more network communication interfaces 26 that provide interfaces for one or more input/output devices 24 . The input/output interface 22 and the network communication interface 26 are coupled to the communication bus 18 . Input/output device 24 may be coupled to other components of computing device 12 via input/output interface 22 . Exemplary input/output device 24 may include a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touchscreen), a voice or sound input device, various types of sensor devices, and/or imaging devices. input devices and/or output devices such as display devices, printers, speakers and/or network cards. The exemplary input/output device 24 may be included in the computing device 12 as a component constituting the computing device 12 , and may be connected to the computing device 12 as a separate device distinct from the computing device 12 . may be

Although representative embodiments of the present invention have been described in detail above, those of ordinary skill in the art to which the present invention pertains will understand that various modifications are possible within the limits without departing from the scope of the present invention with respect to the above-described embodiments. . Therefore, the scope of the present invention should not be limited to the described embodiments, and should be defined by the claims described below as well as the claims and equivalents.

100, 200: Virtual reality game broadcasting service system
102, 202: gaming device
104, 204: broadcast server
106, 206: viewer terminal
108, 208: head mounted display
111: game execution unit
113: sensor unit
115: first screen providing unit
117: second screen providing unit
119: gaze indicator generating unit
121: screen transmitter
211: communication department
213: feature point extraction unit
215: screen correction unit
217: gaze indicator generation unit

Claims (19)

one or more processors, and
A method performed in a computing device having a memory storing one or more programs to be executed by the one or more processors, the method comprising:
an operation of acquiring a game user's point of view screen viewed from the game user's point of view according to the progress of the virtual reality game from the gaming device; and
and determining whether to move the game user's view screen according to the movement displacement of the game user's view screen and transmitting it to one or more viewer terminals,
The operation of determining whether to move the screen from the game user's point of view and transmitting it,
extracting a feature point for each frame of the screen from the game user's point of view;
checking the movement displacement of the feature point between each frame of the screen from the game user's point of view; and
and determining whether to move the screen from the game user's point of view according to the movement displacement of the feature point. delete The method according to claim 1,
The operation of determining whether to move the screen from the game user's point of view and transmitting it,
and, when the movement displacement of the feature point is less than a preset threshold, transmitting the screen without moving the game user's point of view. The method according to claim 1,
The operation of determining whether to move the screen from the game user's point of view and transmitting it,
and when the movement displacement of the key point is equal to or greater than a preset threshold, moving and transmitting the screen from the game user's point of view in a direction opposite to the movement direction of the key point in response to the movement displacement of the key point. The method according to claim 1,
The operation of confirming the movement displacement of the feature point is,
calculating each vector according to the movement of each of the extracted feature points;
calculating an average vector according to movement of the feature points by averaging the calculated vectors; and
A virtual reality game broadcasting service method comprising the operation of checking the magnitude and direction of an average vector according to the movement of the feature points. delete delete delete The method according to claim 1,
The virtual reality game broadcasting service method,
obtaining gaze direction information of the game user in the virtual reality game;
generating a gaze indicator based on the gaze direction information; and
Further comprising the operation of displaying the gaze indicator on the screen of the game user's point of view, the virtual reality game broadcasting service method. one or more processors, and
A computing device having a memory for storing one or more programs executed by the one or more processors, the computing device comprising:
a communication unit for acquiring a game user's point of view screen viewed from the game user's point of view according to the progress of the virtual reality game from the gaming device;
a screen compensator for determining whether to move the game user's view screen according to the movement displacement of the game user's view screen and transmitting it to one or more viewer terminals; and
and a feature point extraction unit for extracting a feature point for each frame of the screen from the game user's point of view,
The screen compensating unit is configured to check a movement displacement of a feature point between frames of the game user's point of view screen, and determine whether to move the game user's point of view screen according to the movement displacement of the feature point. delete 11. The method of claim 10,
The screen correction unit,
When the movement displacement of the feature point is less than a preset threshold, the computing device transmits the game user's viewpoint without moving the screen. 11. The method of claim 10,
The screen correction unit,
When the feature point movement displacement is equal to or greater than a preset threshold, the game user view screen is moved in a direction opposite to the feature point movement direction in response to the movement displacement of the feature point and transmitted. 11. The method of claim 10,
The screen correction unit,
Computing that calculates a vector according to the movement of each of the extracted feature points, calculates an average vector according to the movement of the feature points by averaging the calculated vectors, and checks the magnitude and direction of the average vector according to the movement of the feature points Device. delete delete delete 11. The method of claim 10,
The communication unit obtains gaze direction information of the game user in the virtual reality game,
The computing device is
The computing device further comprising: a gaze indicator generating unit that generates a gaze indicator based on the gaze direction information, and displays the gaze indicator on the game user's viewpoint screen. a head mounted display worn on the user's head;
a gaming device that provides a virtual reality game through the head mounted display, senses a game user's movement according to the progress of the virtual reality game, and provides a game user's point of view screen viewed from the game user's point of view to the game user; and
and a broadcasting server that receives the game user's viewpoint screen from the gaming device, determines whether to move the game user's viewpoint screen according to the movement displacement of the game user's viewpoint screen, and transmits it to a viewer terminal,
The broadcast server extracts a feature point for each frame of the game user's point of view screen, checks the feature point movement displacement between each frame of the game user's point of view screen, and determines whether the game user's point of view screen moves according to the feature point movement displacement Determining, virtual reality game broadcasting service system.

Images