KR102627662B1 - Broadcast transmission system using in-game player object matching - Google Patents

Abstract

The present invention is a broadcast transmission system using in-game player object matching that converts and displays in real time the captured images of the faces of each of the multiple gamers participating in the game in the process of transmitting broadcast images for broadcasting e-sports game competitions. It's about.

Description

Broadcast transmission system using in-game player object matching}

The present invention relates to a broadcast transmission system using in-game player object matching. More specifically, in the process of transmitting broadcast video for broadcasting an e-sports game competition, the face of each of the multiple gamers participating in the game is captured. This relates to a broadcast transmission system using in-game player object matching that converts and displays images in real time.

With the development of the Internet, online games have grown rapidly. Common MUD (Multiple User Dialogue or Multiple User Dungeon) games and MUG (Multiple User Graphic) games that can be played by installing a program on the hard disk drive and connecting to an Internet server, and web-based games that can be played using only a browser This includes games, mobile games using mobile phones, etc.

Among them, AOS genre games and FPS genre games in particular have continued to receive a lot of attention, and e-sports game broadcasts through these genres have also been receiving a lot of love.

In these e-sports game broadcasts, the use and importance of POV (Point of View) cameras are increasing.

However, due to the nature of e-sports game broadcasts, most of them are live broadcasts and the game progresses quite quickly, so in most cases, the faces of specific players and the observer PC screen cannot be broadcast as planned, and responses to the situation must be made immediately. The problem is that you cannot manually select the POV screen.

Meanwhile, the above-mentioned background technology is technical information that the inventor possessed for deriving the present invention or acquired in the process of deriving the present invention, and cannot necessarily be said to be known technology disclosed to the general public before filing the application for the present invention. .

Korean Patent No. 10-1139498

One aspect of the present invention uses in-game player object matching to convert and display in real time the captured images of the faces of each of the multiple gamers participating in the game in the process of transmitting broadcast video for broadcasting an e-sports game competition. Provides a broadcast transmission system.

The technical problem of the present invention is not limited to the technical problem mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

A broadcast transmission system using in-game player object matching according to an embodiment of the present invention, in the process of transmitting broadcast video for broadcasting an e-sports game competition, captures a captured image of the face of each of the multiple gamers participating in the game. Converts and displays in real time.

The broadcast transmission system using the in-game player object matching,

A gamer terminal that generates a game screen image generated during the gamer's game play process and a POV image that captures the gamer's face;

A spectator terminal for watching a game competition in which a large number of gamer terminals participate; and

A relay server connected to the gamer terminal and the spectator terminal and transmitting a broadcast video for broadcasting an e-sports game competition in which a plurality of gamer terminals participate to the viewer terminal,

The relay server is,

Set either the game screen video received from a gamer terminal or the spectator screen video received from the spectator terminal as the transmission screen, and create an in-game player object to identify each gamer participating in the e-sports game competition. It is generated and displayed on one side of the transmission screen, and a POV image display area is created on the other side of the transmission screen so that one POV image among a plurality of POV images is output through the POV image display area,

Either one POV image selected by the spectator terminal is output to the POV image display area, or a plurality of POV images are sequentially output to the POV image display area.

The relay server is,

Calculate the POV score for each gamer terminal using the equation below, and if it is confirmed that a gamer terminal with a calculated POV score higher than the preset standard value has occurred, the POV display area of the broadcast video is changed to the POV score of the gamer whose broadcast score is higher than the standard value. Convert to video in real time.

[Equation]

Here, POV is the POV score, w_p is the weight set differentially for each gamer terminal, d is the distance between the gamer character and the gamer character currently being transmitted to the POV screen, p_e is the preset number of event occurrences, t is the elapsed game time (minutes), r is the gamer's ranking score, h is the ratio of time transmitted to the POV screen compared to the total game time, tr is the ranking score of the team the gamer belongs to, and ev_k is the kill event occurrence score, which is 1 when a kill event occurs and is 1 when a kill event is not present. It is characterized as being 0 when it occurs.

According to one aspect of the present invention described above, it is possible to increase viewers' immersion in watching game broadcasts and provide various game screens by including the reactions and faces of players actually playing the game in the game broadcast screen.

Figure 1 is a diagram illustrating the schematic configuration of a broadcast transmission system using in-game player object matching according to an embodiment of the present invention.
Figure 2 is a diagram illustrating a specific example of a broadcast video generated by a relay server.

The detailed description of the present invention described below refers to the accompanying drawings, which show by way of example specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the invention are different from one another but are not necessarily mutually exclusive. For example, specific shapes, structures and characteristics described herein may be implemented in one embodiment without departing from the spirit and scope of the invention. Additionally, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the detailed description that follows is not intended to be taken in a limiting sense, and the scope of the invention is limited only by the appended claims, together with all equivalents to what those claims assert, if properly described. Similar reference numbers in the drawings refer to identical or similar functions across various aspects.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

Figure 1 is a diagram illustrating the schematic configuration of a broadcast transmission system using in-game player object matching according to an embodiment of the present invention.

The broadcast transmission system using in-game player object matching according to the present invention converts in real time the captured images of the faces of each of the multiple gamers participating in the game in the process of transmitting broadcast video for broadcasting an e-sports game competition. The purpose is to increase viewers' immersion in watching game broadcasts and provide a variety of game screens by including the reactions and faces of players actually playing the game on the game broadcast screen.

Specifically, the broadcast transmission system using in-game player object matching according to the present invention includes a gamer terminal 100, a spectator terminal 200, and a relay server 300.

The gamer terminal 100 is a terminal provided to gamers participating in e-sports game competitions, and is capable of wired and wireless communication with external devices and is capable of inputting, outputting and processing information, such as smartphones, PCs, laptops, tablet PCs, wearable devices, etc. It could be the same electronic device.

The gamer terminal 100 generates a game screen image generated during the gamer's game play process and a POV image that captures the gamer's face. To this end, the gamer terminal 100 may be equipped with a camera module that photographs faces participating in an e-sports game competition.

The spectator terminal 200 is a terminal provided on the side of spectators, such as referees and observers, who host and watch game competitions in which multiple gamer terminals participate. The spectator terminal 200 is a smartphone capable of wired and wireless communication with external devices and capable of inputting, outputting and processing information, It may be an electronic device such as a PC, laptop, tablet PC, wearable device, etc.

The relay server 300 is connected to the gamer terminal and the spectator terminal and transmits a broadcast video for relaying an e-sports game competition in which a plurality of gamer terminals participate to the viewer terminal.

Specifically, the relay server sets one of the game screen images received from one gamer terminal or the spectator screen image received from the spectator terminal as the transmission screen.

In this process, the relay server creates an in-game player object to identify each gamer participating in the e-sports game competition, displays it by placing it on one side of the transmission screen, and creates a POV video display area on the other side of the transmission screen. One POV image among a plurality of POV images is output through the POV image display area.

For example, the relay server causes one POV image selected by the spectator terminal to be output to the POV image display area, or to output a plurality of POV images sequentially to the POV image display area.

In other words, when one of the plurality of in-game player objects is selected on the spectator terminal, the relay server searches for the key value matched to the selected in-game player object, searches for the gamer ID mapped to the searched key value, and places it in advance. The POV video of the selected gamer can be displayed in real time in the selected layout (POV video display area).

Meanwhile, the relay server can convert POV images in real time using various techniques.

In one embodiment, the relay server calculates the POV score for each gamer terminal using Equation 1 below, and when it is confirmed that a gamer terminal with a calculated POV score higher than a preset reference value has occurred, the POV display area of the broadcast video is set to the reference value. Converts in real time to the gamer's POV video with the above broadcast score calculated.

[Equation 1]

Here, POV is the POV score, w_p is the weight set differentially for each gamer terminal, d is the ratio between the number of pixels between the gamer character and the gamer character currently being transmitted to the POV screen compared to the number of horizontal pixels on the transmission screen, and p_e is the number of preset event occurrences. , t is the elapsed game time (minutes), r is the gamer's ranking score, h is the ratio of time transmitted to the POV screen compared to the total game time, tr is the ranking score of the team to which the gamer belongs, and ev_k is the kill event occurrence score. , It is characterized as 1 when a treatment event occurs and 0 when a treatment event does not occur.

In addition, a preset event refers to a situation where, for example, a chat cheering for the gamer is entered or a cheering sign cheering for the gamer is found, the administrator can count the number of event occurrences once. .

For example, if w_p is 3, d is 0.7, p_e is 4, t is 20, r is 2, tr is 3, ev_k is 1, and h is 0.1, the POV is calculated as 96, and like this, the relay server The POV score is calculated for each gamer, and if it is confirmed that a gamer terminal has a calculated POV score higher than a preset standard value (e.g. 70), the POV display area of the broadcast video is displayed as the POV image of the gamer whose broadcast score is higher than the standard value. Convert in real time to At this time, if there are multiple gamers whose POV scores exceed the standard value, the relay server switches in real time to the POV video of the gamer with the highest POV score.

Meanwhile, the ranking score for each gamer can be calculated as follows.

[Equation 2]

Here, R is the ranking score, δ is the importance of the competition, ht is the highest event acquisition score, hb is the lowest event acquisition score, h_other is the average event acquisition score of other gamers, and n is the ranking calculation period (number of days).

For example, if δ is 10, ht is 12, hb is 3, h_other is 4.3, and n is 60, R is calculated as approximately 4.2, and the relay server calculates the ranking score for each gamer using Equation 2 above. By calculating, rankings can be assigned in descending order of the calculated ranking score.

In this way, the relay server calculates the POV score for each gamer terminal using the above-mentioned equations, and when it is confirmed that a gamer terminal has a calculated POV score higher than the preset standard value, it relays the POV display area of the broadcast video to be higher than the standard value. By switching in real time to the gamer's POV video with the score calculated, a reliable POV video can be transmitted.

In some other embodiments, the relay server may switch the POV image displayed in the POV image display area in real time using an artificial neural network.

The management server can train an artificial neural network, and can also use a trained artificial neural network. The processor can train or execute an artificial neural network stored in memory, and the memory can store a trained artificial neural network. The electronic device that trains the artificial neural network and the electronic device that uses it may be the same, but may also be separate. Artificial intelligence is a computer system that implements some of the functions of the human brain and can learn, guess, and make decisions on its own. As learning progresses, the probability of extracting an answer may increase. Artificial intelligence can be composed of learning and elemental technologies using it. Artificial intelligence learning is an algorithmic technology that classifies and learns features based on input data, and elemental technologies may be technologies that implement some of the functions of the human brain using learning algorithms.

Artificial intelligence is a technology that makes it easy to approach problems that can have multiple answers probabilistically, and can logically and probabilistically infer the optimal cycle, method, and plan according to any input data. Artificial intelligence's inference technology can include judging input data, optimization predictions, knowledge and probability-based reasoning, and preference-based planning.

Artificial neural network is one of the learning algorithms in the machine learning field and is a program that implements the connection between neurons and synapses in the brain. Artificial neural networks can be created through a program to create a neural network structure and then learn it to have the desired function. Although there may be errors, it is possible to learn based on huge data and output appropriate output data with input data. It has the advantage of being able to obtain output data with statistically good results and being similar to human reasoning.

The management server 100 can determine and provide the most important gamer screen among a plurality of gamer screens using an artificial intelligence algorithm built on big data, and may include a plurality of pre-trained artificial neural networks for this purpose. .

In this embodiment, the relay server may include a number of pre-trained artificial neural networks to perform machine learning algorithms. With machine learning, you can output output data based on input data and use the results to learn on your own, which can improve your own data processing ability. Artificial neural networks can extract features based on input data, infer regularities, and output result data. As this process accumulates, the reliability of the result data increases.

In this embodiment, the artificial neural network determines the importance from at least one of the following data: voice data of the gamer's broadcaster, kill/death/assist score, elapsed game time, occurrence of a kill/death event, and change in the gamer's facial expression through image analysis. It may be an algorithm that outputs. Artificial neural networks can infer the best output data using big data as input data or after going through a processing process to organize unnecessary data.

ve Bayes Classifier, Hidden Markov Model, K-Means Clustering 등이 사용될 수 있다.Depending on the type of learning, artificial intelligence machine learning models include Super Viser Learning, UnSuper Viser Learning, Semisupervised Learning, and Reinforcement Learning. And machine learning algorithms include Decision Tree, K-Nearest Nearest Bor, Artificial Neural Network, Support Vector Machine, Ensemble Learning, Gradient Descent, Na

ve Bayes Classifier, Hidden Markov Model, K-Means Clustering, etc. can be used.

The artificial neural network may be pre-trained on various input values that may be included in the input data. The artificial neural network can infer information based on data input by the user and select the most important POV video that should be transmitted at the current time among multiple gamer POV videos.

An artificial neural network may be an artificial neural network that is learned according to reinforcement learning, one of the learning methods. Reinforcement learning is a method of gradually increasing the probability of obtaining the correct result by setting rewards and limits. Artificial neural networks can also be modeled based on Convelutional Neural Network (CNN) or Recurrent Neural Network (RNN).

In this way, the relay server can select a reliable POV screen by building a POV screen setting model using an artificial neural network.

In this way, the survey evaluation unit calculates a rating for each review data of the non-face-to-face survey using the above-mentioned mathematical formula, and sets the average of the calculated ratings as the rating of the non-face-to-face survey, thereby determining whether the survey is meaningful. It can indicate whether it was

As such, the technology according to the present invention may be implemented as an application or in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., singly or in combination.

The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention, or may be known and usable by those skilled in the computer software field.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specifically configured to store and perform program instructions, such as ROM, RAM, flash memory, etc.

Examples of program instructions include not only machine language code such as that created by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the invention and vice versa.

Although the above has been described with reference to embodiments, those skilled in the art will understand that various modifications and changes can be made to the present invention without departing from the spirit and space of the present invention as set forth in the following patent claims. You will be able to.

100: Gamer terminal
200: Spectator terminal
300: relay server

Claims (3)

In the process of transmitting broadcast video to broadcast an e-sports game competition, a broadcast transmission system using in-game player object matching that converts and displays captured images of the faces of each of the multiple gamers participating in the game in real time. ,
The broadcast transmission system using the in-game player object matching,
A gamer terminal that generates a game screen image generated during the gamer's game play process and a POV image that captures the gamer's face;
A spectator terminal for watching a game competition in which a large number of gamer terminals participate; and
A relay server connected to the gamer terminal and the spectator terminal and transmitting a broadcast video for broadcasting an e-sports game competition in which a plurality of gamer terminals participate to the viewer terminal,
The relay server is,
Set either the game screen video received from a gamer terminal or the spectator screen video received from the spectator terminal as the transmission screen, and create an in-game player object to identify each gamer participating in the e-sports game competition. It is generated and displayed on one side of the transmission screen, and a POV image display area is created on the other side of the transmission screen so that one POV image among a plurality of POV images is output through the POV image display area,
Either one POV image selected by the spectator terminal is output to the POV image display area, or a plurality of POV images are sequentially output to the POV image display area,
The relay server is,
Calculate the POV score for each gamer terminal using the equation below, and if it is confirmed that a gamer terminal with a calculated POV score higher than the preset standard value has occurred, the POV display area of the broadcast video is changed to the POV score of the gamer whose broadcast score is higher than the standard value. A broadcast transmission system using in-game player object matching that converts to video in real time.

[Equation]

Here, POV is the POV score, w_p is the weight set differentially for each gamer terminal, d is the distance between the gamer character and the gamer character currently being transmitted to the POV screen, p_e is the preset number of event occurrences, t is the elapsed game time (minutes), r is the gamer's ranking score, h is the ratio of time transmitted to the POV screen compared to the total game time, tr is the ranking score of the team the gamer belongs to, and ev_k is the kill event occurrence score, which is 1 when a kill event occurs and is 1 when a kill event is not present. It is characterized as being 0 when it occurs.
delete delete