KR20220159571A - Deep-learning based jang-gi game service method and apparatus thereof - Google Patents

Abstract

A deep learning-based Janggi game service method according to an embodiment of the present invention is a method in which a game play server provides a deep-learning-based Janggi game service to a user terminal, and comprises the steps of: obtaining a training data set (Training data set) including chessboard states for a plurality of actions; extracting input features based on the acquired training data set; learning a deep-learning model based on the extracted input features; receiving a first chessboard state; and acquiring at least one of an action to be performed and win/loss prediction information, which is an action to be performed for the received first chess board state, based on the deep learning model. The present invention aims to provide a deep-learning-based Janggi game service method and device which utilize a trained deep learning model to predict actions and game outcomes based on game situations in a Janggi match.

Description

Long-term game service method and device based on deep learning

The present invention relates to a long-term game service method and apparatus based on deep learning. More specifically, a deep learning-based chess that learns a deep learning model of a chess game service and predicts the action to be performed and the match result according to the match situation in the chess game service using the learned deep learning model. It relates to a game service method and device therefor.

BACKGROUND ART [0002] In recent years, as the spread of wired/wireless Internet has rapidly increased with the development of communication and network technology, various types of services have been provided through a homogeneous medium called the Internet.

In particular, the game service is a service used by many users among services provided through the Internet, and various games are provided.

Among them, many battle-type games in which game points are exchanged in exchange for victory are being serviced, and games such as 'janggi', 'go', 'go-stop' or 'chess' have a large number of users. they are bad games

On the other hand, in addition to this trend, it has recently been possible to play a game with a programmed artificial intelligence computer rather than a human being through the above game.

In addition, there is a trend in which research is being actively conducted to enhance the power of artificial intelligence computers so that they can perform matches at the human level.

However, conventional artificial intelligence computers are specialized for more popular game services such as Go or chess, so technology for implementing artificial intelligence computers with high performance in long-term game services is insufficient.

In other words, the long-term game service is characterized by the characteristic that the number of possible actions is greater than that of other game services such as Baduk due to the uniquely determined row magic for each piece of the long-term game. The development of an artificial intelligence computer capable of performing deep learning optimized for it by considering the properties of the implemented game in more detail is insufficient.

KR 2007371 B1

The present invention is a deep learning-based shogi game that learns a deep learning model of a shogi game service and predicts an action to be performed and a game result according to a game situation in a shogi game service using the learned deep learning model. Its purpose is to provide a service method and its device.

In detail, the present invention is to provide a deep learning-based long game service method and apparatus for learning a deep learning model to be specialized for long game service.

In addition, the present invention is a deep learning-based long-term game service that predicts the winning rate for each action based on the match situation in the long-term game service, the action to be performed for the match situation, and the result of the match using the learned deep learning model It is intended to provide a method and an apparatus therefor.

In addition, the present invention is to provide a deep learning-based long-term game service method and apparatus for performing a long-term game based on a predicted action.

However, the technical problems to be achieved by the present invention and the embodiments of the present invention are not limited to the technical problems described above, and other technical problems may exist.

A deep learning-based long-term game service method according to an embodiment of the present invention is a method in which a game play server provides a deep-learning-based long-term game service to a user terminal. Obtaining a data set (Training data set); extracting input features based on the acquired training data set; learning a deep-learning model based on the extracted input features; Receiving a first chessboard state; and acquiring at least one of an action to be performed and win/loss prediction information, which is an action to be performed for the received first chessboard state, based on the deep learning model.

At this time, the input features include a type of long-term piece, history information on the last 8 actions of the first player, and piece location information on the last 8 actions of the first player. , History information on the last 8 actions of the second player, object location information on the last 8 actions of the second player, and whether the current player is the first player or the second player. It includes at least some of the sequence information.

In addition, in the step of learning the deep-learning model, the predetermined chess board state is used as input data, and the odds estimate value for each candidate action, which is an action that can be performed in the input predetermined chess board state, and the predetermined chess board state and learning the deep learning model so that at least one of an action to be performed for the state and prediction information for a win or loss for the predetermined checkerboard state is output data.

In addition, the action is an action of moving a specific piece on a chess board based on a magic trick for each chess piece.

In addition, the action includes a movement action outside the palace, a horse movement action, a statue movement action, a movement action within the palace, and a pass action.

In addition, the obtaining of at least one of the action performed for the first checkerboard state and win/loss prediction information may include inputting the first checkerboard state to the deep learning model, and selecting a candidate action that is an action that can be performed in the first checkerboard state. and obtaining an odds ratio estimation value for each.

In addition, the obtaining of at least one of the action performed for the first chess board state and win/loss prediction information further includes determining a candidate action having the highest odds ratio estimation value as the action performed for the first chess board state. .

In addition, the obtaining of at least one of an action performed for the first chessboard state and win/loss prediction information may include obtaining the win/loss prediction information for the first chess board state based on whether the determined action is performed and whether the game is won or lost. It further includes the steps of

In addition, the deep learning-based chess game service method according to an embodiment of the present invention further includes performing the action for the first chessboard state.

On the other hand, a long-term game service device based on deep learning according to an embodiment of the present invention includes a communication unit that receives a state of a long-term board; memory for storing deep learning models; and learning the deep learning model based on the received chess board state, and performing an action having the highest odds estimation value among a plurality of candidate actions possible in the chess board state based on the deep learning model, and the performing action and a processor for acquiring at least one of prediction information about a win or loss for the state of the chess board as it is performed.

A long-term game service method and apparatus based on deep learning according to an embodiment of the present invention considers the characteristics of a game characteristically implemented in a long-term game service in more detail by learning a deep learning model to be specialized for the long-term game service, and accordingly It is possible to implement an artificial intelligence computer capable of performing optimized deep learning.

In addition, the deep learning-based long-term game service method and apparatus according to an embodiment of the present invention predict the winning rate for each action based on the game situation in the long-term game service using the learned deep learning model, and based on this, the corresponding The performance of artificial intelligence computers for long-term game services can be further improved by predicting the actions to be performed in the game situation and the results of the game accordingly.

In addition, the deep learning-based long-term game service method and apparatus according to an embodiment of the present invention can improve the level of long-term games of artificial intelligence computers by performing long-term games based on predicted actions.

However, the effects obtainable in the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood from the description below.

1 is a conceptual diagram of a long-term game service system based on deep learning according to an embodiment of the present invention.
2 is a diagram for explaining the action model structure of a game play server for an action of an artificial intelligence computer in a long-term game service based on deep learning according to an embodiment of the present invention.
3 is a diagram for explaining a process in which an action model according to an embodiment of the present invention performs an action according to a pipeline of a search unit.
4 is a flowchart illustrating a method of providing a long-term game service based on deep learning according to an embodiment of the present invention.
5 is an example of a diagram for explaining input features according to an embodiment of the present invention.
6 is a diagram for explaining the number of possible actions based on long-term objects according to an embodiment of the present invention.
7 is an example of a diagram showing how an action model according to an embodiment of the present invention provides output data by performing deep learning based on a first chessboard state.
8 is an example of a diagram illustrating a state in which an action model according to an embodiment of the present invention performs an action determined for a first chessboard state.

Since the present invention can apply various transformations and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. Effects and features of the present invention, and methods for achieving them will become clear with reference to the embodiments described later in detail together with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms. In the following embodiments, terms such as first and second are used for the purpose of distinguishing one component from another component without limiting meaning. Also, expressions in the singular number include plural expressions unless the context clearly dictates otherwise. In addition, terms such as include or have mean that features or elements described in the specification exist, and do not preclude the possibility that one or more other features or elements may be added. In addition, in the drawings, the size of components may be exaggerated or reduced for convenience of explanation. For example, since the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of description, the present invention is not necessarily limited to the illustrated bar.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when describing with reference to the drawings, the same or corresponding components are assigned the same reference numerals, and overlapping descriptions thereof will be omitted. .

1 is a conceptual diagram of a long-term game service system based on deep learning according to an embodiment of the present invention.

Referring to FIG. 1, a deep learning-based chess game service system 10 according to an embodiment of the present invention includes a terminal 100, a chess server 200, a game play server 300, and a network ( 400) may be included.

Each component of FIG. 1 may be connected through a network 400 . It means a connection structure capable of exchanging information between nodes such as the terminal 100, the long-term server 200, and/or the game play server 300. An example of such a network is the 3rd Generation Partnership Project (3GPP) network, LTE (Long Term Evolution) network, WIMAX (World Interoperability for Microwave Access) network, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN( Personal Area Network), Bluetooth network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc. are included, but are not limited thereto.

- Terminal (100: Terminal)

First, the terminal 100 is a terminal of a user who wants to receive a long-term game service. Also, the terminal 100 is one or more computers or other electronic devices used by a user to execute applications that perform various tasks.

For example, a computer, laptop computer, smart phone, mobile phone, PDA, tablet PC, or any other device operable to communicate with organ server 200 and/or game play server 300.

However, it is not limited thereto, and the terminal 100 is executed on various machines, includes processing logic that interprets and executes commands stored in a plurality of memories, and provides a graphical user interface (GUI) on an external input/output device. It may contain various other elements, such as processes that display graphical information.

In addition, the terminal 100 may be connected to an input device (eg, mouse, keyboard, touch sensitive surface, etc.) and an output device (eg, display device, monitor, screen, etc.).

Applications executed by the terminal 100 may include game applications, web browsers, web applications running on web browsers, word processors, media players, spreadsheets, image processors, security software, or the like. .

In addition, the terminal 100 may include at least one memory 101 storing instructions, at least one processor 102 and a communication unit 103 .

The memory 101 of the terminal 100 may store a plurality of application programs or applications running in the terminal 100, data for operation of the terminal 100, and commands.

Instructions are executable by the processor 102 to cause the processor 102 to perform operations, such as transmitting a long-term game play request signal, transmitting and receiving game data, transmitting and receiving action information, transmitting and receiving odds information, and transmitting and receiving prediction information for winning and losing. , operations for transmitting and receiving various types of information such as requesting game time information and receiving game time information.

In addition, the memory 101 may be a variety of storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in terms of hardware, and the memory 130 performs the storage function of the memory 101 on the Internet. It can also be a web storage that performs.

The processor 102 of the terminal 100 may perform data processing to receive a long-term game service by controlling overall operations.

When a long game application is executed in the terminal 100, a long game environment is configured in the terminal 100. Also, the long term game application exchanges long term game data with the long term server 200 through the network 400 so that the long term game service is executed on the terminal 100 .

These processors 102 include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, and microcontrollers. It may be micro-controllers, microprocessors, or any type of processor for performing other functions.

The communication unit 103 of the terminal 100 uses the following communication methods (eg, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink (HSUPA)) Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc.), WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA ( A wireless signal may be transmitted and received with at least one of a base station, an external terminal, and a server on a network constructed according to Digital Living Network Alliance (WiBro), Wireless Broadband (WiBro), and World Interoperability for Microwave Access (WiMAX).

Such a terminal 100 may perform at least a part of the functional operation performed in at least one of the long-term server 200 and the game play server 300 to be described later.

- Long term server (200: JANG-GI server)

The long-term game service provided by the long-term server 200 may be configured in such a way that a virtual computer user provided by the long-term server 200 and a real user participate in a game together. In this, one real user and one computer user play the game together in a long-term game environment implemented on the user-side terminal 100 .

In another aspect, the long-term game service provided by the long-term server 200 may be configured in a form in which a plurality of user-side devices participate to play the long-term game.

The long-term server 200 may include at least one memory 201 storing instructions, at least one processor 202 and a communication unit 203.

The memory 201 of the long-term server 200 may store a plurality of application programs or applications running in the long-term server 200, data for the operation of the long-term server 200, and commands. .

The instructions are executable by the processor 202 to cause the processor 202 to perform operations, the operations being game play request signal reception, game data transmission and reception, action information transmission and reception, odds information transmission and reception, win/loss prediction information transmission and reception, game It may include various transmission operations such as requesting time information and receiving game time information.

In addition, the memory 201 may store a plurality of long-term notation (hereinafter referred to as notation) or a plurality of previously published notations that were played in the long-term server 200. Each of the plurality of notations may include all information from the first action, which is the first action information of the start of the game, to the final action in which the game ends. That is, the plurality of notations may include history information about actions. In addition, each notation of a plurality of notations may include each chessboard state (S) according to the order of actions in the chess game. Here, the chess board state may be a state in which chess pieces are placed on the chess board.

In addition, the long term server 200 enables the game play server 300 to provide a plurality of stored notations for training of the game play server 300 .

In addition, the memory 201 may be a variety of storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in terms of hardware, and the memory 201 performs the storage function of the memory 201 on the Internet. It can also be a web storage that performs.

The processor 202 of the long term server 200 may perform data processing to provide a long term game service by controlling overall operations. These processors 202 include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, and microcontrollers. It may be micro-controllers, microprocessors, or any type of processor for performing other functions.

The long term server 200 may communicate with the terminal 100 and the game play server 300 via the network 400 through the communication unit 203 .

- Game play server (300: Game playing server)

The game play server 300 may include a separate cloud server or a computing device. In addition, the game play server 300 may be a neural network system installed in the processor of the terminal 100 or the data processing unit of the long term server 200, but hereinafter, the game play server 300 is the terminal 100 or the long term server ( 200) and a separate device.

The game play server 300 may include at least one memory 301 storing instructions, at least one processor 302 and a communication unit 303 .

The game play server 300 builds an action model, which is a deep learning model that learns by itself according to long-term rules, and is an artificial intelligence computer capable of playing a game with the user of the terminal 100, and wins the game in its own turn. Actions in the chess game can be performed so that A detailed description of how the game play server 300 trains the action model follows the description of the action model of FIGS. 2 to 6 .

Here, an action according to an embodiment may mean an action of moving a specific piece on a chessboard based on a magic trick for each piece (hereinafter, piece). At this time, the moving magic means that the pieces move according to rules that are uniquely determined.

The memory 301 of the game play server 300 stores a plurality of application programs or applications running in the game play server 300, data for the operation of the game play server 300, and instructions. can be saved Instructions are executable by the processor 302 to cause the processor 302 to perform operations, which include action model learning (training) operation, action information transmission and reception, action preparation time reception, game time information reception, and various transmissions. Actions may be included.

Also, the memory 301 may store an action model that is a deep learning model. In addition, the memory 301 may be a variety of storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in terms of hardware, and the memory 301 performs the storage function of the memory 301 on the Internet. It can also be a web storage that performs.

The processor 302 of the game play server 300 reads the action model stored in the memory 302 and, according to the built neural network system, learns the action model described below and performs actions in the long game.

Depending on the embodiment, the processor 302 may be configured to include a main processor that controls all units and a plurality of graphics processing units (GPUs) that process large-capacity calculations required for driving a neural network according to an action model. have.

The game play server 300 may communicate with the long-term server 200 via the network 400 through the communication unit 303 .

<action model>

The action model according to an embodiment of the present invention takes a predetermined chess board state as input data, and an odds estimation value (V) for each candidate action, which is an action that can be performed in the input predetermined chess board state, and at least one candidate action An action to be performed (A) determined as an action to win in a major power and/or a result of the corresponding power, that is, win/loss prediction information (O) predicting a win or loss may be used as output data.

2 is a diagram for explaining the action model structure of the game play server 300 for actions of an artificial intelligence computer in a long-term game service based on deep learning according to an embodiment of the present invention.

Referring to FIG. 2 , an action model according to an embodiment of the present invention is a deep learning model of a game play server 300, and may include a search unit 310, a self play unit 320, and a deep learning neural network 330. can

In detail, the action model may be learned as a model for determining an action to win in a game using the search unit 310, the self-play unit 320, and the deep learning neural network 330.

)을 출력할 수 있다. Specifically, the search unit 350 may perform a Monte Carlo Tree Search (MCTS) operation according to the guide of the deep learning neural network 340 . MCTS is a heuristic search algorithm for some kind of decision making. That is, the search unit 350 may perform MCTS based on the movement probability value (p) and/or the odds ratio estimation value (V) provided by the deep learning neural network 340 . For example, the search unit 350 guided by the deep learning neural network 340 performs MCTS to search probability values that are probability distribution values for actions (

) can be output.

)에 따라 스스로 장기 대국을 할 수 있다. 셀프 플레이부(320)는 게임의 승패가 결정되는 시점까지 스스로 장기 대국을 진행하고, 자가 대국(즉, self-play)이 종료되면 장기판 상태(S), 탐색 확률값(

), 자가 플레이 가치값(z)을 딥러닝 신경망(340)에 제공할 수 있다. 여기서 장기판 상태(S)는 장기판 상에 기물이 놓여진 상태일 수 있다. 자가 플레이 가치값(z)은 장기판 상태(S)에서 자가 대국을 하였을 때 승률 값이다. The self-play unit 320 provides a search probability value (

), you can play a long-term game yourself. The self-play unit 320 proceeds with the long-term game by itself until the game is decided, and when the self-play (ie, self-play) ends, the long-term board state (S), the search probability value (

), the self-play value (z) may be provided to the deep learning neural network 340 . Here, the chess board state (S) may be a state in which pieces are placed on the chess board. The self-play value value (z) is a win rate value when a player plays a game in the chessboard state (S).

)과 동일해지도록 트레이닝되고, 승률추정값(V)이 자가 플레이 가치값(z)과 동일해지도록 트레이닝될 수 있다. 이후 트레이닝된 딥러닝 신경망(340)은 탐색부(350)를 가이드하고, 탐색부(350)는 이전 탐색 확률값(

)보다 더 좋은 액션을 찾도록 액션 준비 시간 동안 MCTS를 진행하여 새로운 탐색 확률값(

)을 출력하게 한다. 예를 들어, 액션 준비 시간은 MCTS 진행 시간에 따라 평균 액션 준비 시간, 제1 액션 준비 시간 및 제2 액션 준비 시간 중 어느 하나의 액션 준비 시간을 따를 수 있다. 액션 준비 시간은 기본적으로 평균 액션 준비 시간으로 설정되어 있을 수 있다. The deep learning neural network 340 may output a movement probability value (p) and an odds ratio estimation value (V). The movement probability value p may be a probability distribution value representing numerically which action to perform according to the checkerboard state S is a good action to win the game. The odds ratio estimation value (V) represents the odds ratio when the corresponding action is performed. For example, an action having a high movement probability value p may be a good action. The deep learning neural network 340 determines that the movement probability value (p) is the search probability value (

), and the odds estimate value (V) may be trained to be equal to the self-play value value (z). Then, the trained deep learning neural network 340 guides the search unit 350, and the search unit 350 uses the previous search probability value (

) to find a better action than the new search probability value (

) to output. For example, the action preparation time may follow one action preparation time among an average action preparation time, a first action preparation time, and a second action preparation time according to the MCTS progress time. The action preparation time may be basically set to an average action preparation time.

)에 기초하여 장기판 상태(S)에 따른 새로운 자가 플레이 가치값(z)을 출력하고 장기판 상태(S), 새로운 탐색 확률값(

), 새로운 자가 플레이 가치값(z)을 딥러닝 신경망(340)에 제공할 수 있다. 딥러닝 신경망(340)은 이동 확률값(p)과 승률추정값(V)이 새로운 탐색 확률값(

)과 새로운 자가 플레이 가치값(z)으로 출력되도록 다시 트레이닝될 수 있다. The self-play unit 320 provides a new search probability value (

), outputs a new self-play value value (z) according to the checkerboard state (S), and outputs the checkerboard state (S), new search probability value (

), the new self-play value value z may be provided to the deep learning neural network 340 . The deep learning neural network 340 determines that the movement probability value (p) and the odds ratio estimation value (V) are a new search probability value (

) and a new self play value (z) can be trained again.

That is, the action model can be trained to find a better action to win the deep learning neural network 340 by repeating this process. As an example, the action model may use action loss (l). The action loss (l) is shown in Equation 1.

[Equation 1]

는 신경망의 파라미터이고, c는 매우 작은 상수이다.

is a parameter of the neural network, and c is a very small constant.

와 p가 같아 지도록 하는 것은 크로스 엔트로피 손실(cross entropy loss) 텀에 해당되고,

에 c를 곱하는 것은 정규화 텀으로 오버피팅(overfitting)을 방지하기 위한 것이다.Making z and v equal in the action loss (l) of Equation 1 corresponds to the mean square loss term,

and p are equal to the cross entropy loss term,

Multiplying by c is to prevent overfitting with the regularization term.

Meanwhile, the deep learning neural network 340 may have a neural network structure. For example, the deep learning neural network 340 may include one convolution block and 19 residual blocks. A convolution block may have a form in which several 3X3 convolution layers are overlapped. One residual block may have a form in which several 3X3 convolutional layers are overlapped and a skip connection is included. A skip connection is a structure in which an input of a predetermined layer is outputted by summing an output value of a corresponding layer and inputted to another layer.

In addition, the input of the deep learning neural network 340 is the type of the object, the history of the last 8 actions of the first player (eg, the blue player), the location information of the object for the last 8 actions of the first player, and the first player. The history of the last eight actions of the second player (eg, the red player) and the location information of the object for the last eight actions of the first player, and the turn information on whether the current player is the first player or the second player, etc. It may be a 114*10*9 RGB image including A detailed description of this follows the description of the action model of FIGS. 2 to 8 .

3 is a diagram for explaining a process in which an action model according to an embodiment of the present invention performs an action according to a pipeline of a search unit.

Referring to FIG. 3 , the action model according to the embodiment of the present invention may perform a predetermined action by using the deep learning neural network 330 and the search unit 310 in its own turn. In detail, the action model may perform Monte Carlo Tree Search (MCTS) using the search unit 310, the self-play unit 320, and the deep learning neural network 330.

Specifically, the action model is the second checkerboard state having an action having a high activity function (Q) and a high confidence value (U) among branches not searched through MCTS in the current first checkerboard state (S1) through the selection process (a). Select (S1-2). The activity function (Q) is an average value of odds ratio estimation values (V) calculated each time a corresponding branch is passed. The confidence value (U) is inversely proportional to the number of visits (N) passing through the corresponding branch and proportional to the movement probability value (p).

The action model may be expanded to the third chessboard state (S1-2-1) in the action selected through the expansion and evaluation process (b) and calculate a movement probability value (p).

The action model calculates the odds ratio estimation value (V) of the extended third chessboard state (S1-2-1), and the activity function (Q), number of visits (N), and movement probability value of the branches passed through the backup process (c) (p) can be stored.

)을 출력할 수 있다. 액션 모델은 액션들 중 가장 높은 탐색 확률값(

)을 가지는 액션을 검출할 수 있다. 또한, 액션 모델은 상기 검출된 액션을 기초로 수행액션(A)으로 결정할 수 있다. 또한, 액션 모델은 결정된 수행액션(A)을 수행할 수 있다. 또한, 액션 모델은 결정된 일련의 수행액션(A)이 수행됨에 따른 대국의 결과 즉, 승패 여부를 판단한 승패 예측정보(O)를 제공할 수 있다. The action model repeats the process of selection (a), expansion and evaluation (b), and backup (c) during the action preparation time, and creates a probability distribution using the number of visits (N) for each action, resulting in a search probability value (

) can be output. The action model has the highest search probability value among actions (

) can be detected. Also, the action model may determine the action (A) based on the detected action. Also, the action model may perform the determined action (A). In addition, the action model may provide a result of a match according to the execution of the determined series of actions (A), that is, win/loss prediction information (O) for determining win/loss.

- Long-term game service method based on deep learning

4 is a flowchart illustrating a method of providing a long-term game service based on deep learning according to an embodiment of the present invention.

Referring to FIG. 4 , in the long-term game service based on deep learning according to an embodiment of the present invention, the game play server 300 learns an action model, which is a deep learning model of the long-term game service, and uses the learned action model. Depending on the match situation in the long-term game service, the action to be performed and the result of the match can be predicted.

Specifically, the game play server 300 may train an action model to be specialized for a long-term game service. In addition, the game play server 300 may predict the winning rate for each action based on the game situation in the long-term game service, the action performed for the game situation, and the result of the match by using the learned action model. In addition, the game play server 300 may perform a long game based on the predicted action.

Here, the action according to the embodiment may refer to an action of moving a specific piece on a checkerboard based on a magic trick for each piece. At this time, the moving magic means that the pieces move according to rules that are uniquely determined.

In detail, further referring to FIG. 4, the deep learning-based long-term game service method according to an embodiment of the present invention includes a step (S101) of receiving a training data set by the game play server 300. can

In more detail, the game play server 300 may receive a training data set for learning an action model in conjunction with the long-term server 200 . Here, the training data set may include a plurality of notations that have been played in the long-term server 200 or a plurality of notations previously published. At this time, each of the plurality of notation may include all information from the first action information of the first action of the start of the game to the final action that ends the game. That is, the plurality of notations may include history information about actions. In addition, each notation of a plurality of notations may include each chessboard state (S) according to the order of actions in the chess game. Here, the chess board state (S) may be a state in which pieces are placed on the chess board.

In addition, the deep learning-based long-term game service method according to an embodiment of the present invention may include extracting input features based on the training data set received by the game play server 300 (S103). have.

Here, the input characteristics include the type of the object, the history of the last 8 actions of the first player (eg, the blue player) and the location information of the object for the last 8 actions of the first player, and the second player (eg, the blue player). , Hong player) 114* including history of the last 8 actions and location information of the object for the last 8 actions of the first player, turn information on whether the current player is the first player or the second player, etc. It may be a 10*9 RGB image.

5 is an example of a diagram for explaining input features according to an embodiment of the present invention.

In detail, referring to FIG. 5 , the game play server 300 may perform an input feature extraction process of extracting the above-described input features based on the checkboard state (S) of a plurality of notations.

Specifically, the game play server 300 may further include an input feature extraction unit that performs the input feature extraction process. For example, the input feature extraction unit may have a neural network structure and may include a kind of encoder.

In addition, the game play server 300 may extract input features for the checkerboard state (S) using the input feature extraction unit.

At this time, the game play server 300 may use the extracted input feature as an image. In detail, the game play server 300 may convert one or more extracted input features into an image form suitable for an input form of a deep learning model (in an embodiment, an action model).

In an embodiment, the game play server 300 converts order information for indicating whether the current player is the first player or the second player into an image, in the case of the first player (eg, the second player) The turn information may be indicated by generating an image of a checkerboard state S in which the checkerboard is all filled with a plurality of pieces having the color of the first player (eg, blue). Alternatively, the game play server 300, when it is the turn of the second player (eg, the red player), the checkerboard state (S) in which the checkerboard is all filled with a plurality of pieces having the color (eg, red) of the second player An image of may be generated to indicate the order information.

That is, the game play server 300 may extract input features in the form of an image suitable for the form of input data for the action model.

In addition, the deep learning-based long-term game service method according to an embodiment of the present invention may include a step of allowing the game play server 300 to train a deep learning model based on the extracted input features (S105) .

In detail, the game play server 300 may train a deep learning-based action model based on the input features extracted as above. Specifically, the game play server 300 may train the action model by using a plurality of image-shaped input features as a training data set.

At this time, the game play server 300 takes the predetermined chess board state (S) as input data, and the odds estimation value (V) for each of the at least one candidate action based on the input predetermined chess board state (S), and at least one candidate Among the actions, the action model is designed to provide the action (A) that is determined as an action to win the game and/or the result of the game, that is, the win/loss prediction information (O) that predicts whether the player wins or loses as output data (Output features). can be learned

As an embodiment, the game play server 300 may cause the action model to obtain an action performed according to a plurality of notations and a result of the game based on the extracted input characteristics. In addition, the game play server 300 may cause the action model to generate the above-described output data based on the action performed according to the plurality of notations and the result of the game.

For example, the game play server 300 based on the input characteristics based on the plurality of notations, determines whether or not to win or lose when performing a predetermined first action Estimate the odds ratio estimate (V) for the first action The action model can be trained to do so. In addition, the game play server 300 estimates the odds ratio estimate value (V) for each of the plurality of candidate actions based on the input characteristics based on the plurality of notations, and performs the candidate action having the highest odds ratio estimate value (V). The action model may be trained to determine (A). In addition, the game play server 300, based on the input characteristics based on the plurality of notations, a series of actions (in the embodiment, the action performed (A)) having the highest odds estimate value (V) for each board state (S) The action model may be trained to generate win/loss decision information by estimating whether a major country wins or loses according to performing.

In this case, in other words, the action according to the embodiment of the present invention may refer to an action of moving a specific piece on the checkerboard based on a magic trick for each piece. Here, the row magic refers to the movement of each piece according to a uniquely determined rule.

6 is a diagram for explaining the number of possible actions based on long-term objects according to an embodiment of the present invention.

More specifically, referring to FIG. 6, the above actions are movement actions outside the palace, horse movement actions, statue movement actions, movement actions within the palace, and pass actions according to the magic of each piece. can include

In detail, the movement action outside the palace may include an action in which a car, gun, or pawn (or soldier) piece moves according to the magic method in the ground area excluding the palace area of the chessboard. can Also, the horse moving action may include an action in which a horse object moves according to a magic trick. Also, the phase movement action may include an action in which phase objects move according to a row magic. In addition, the moving action within the palace may include an action of moving a predetermined piece according to a magic trick in the palace area. Also, a pass action may mean an action that does not perform an action.

At this time, the number of actions according to the row magic for each piece can be realized as a total of 2451. In detail, the number of movement actions other than the palace is the total number of moves possible according to the magic of the car, gun, or pawn (or soldier) piece in the ground area excluding the palace area. It can be 1530. In addition, the number of horse movement actions may be 508 as the sum of the number of moves possible according to the magic of moving the horse. In addition, the number of phase movement actions may be 388 as the total number of moves possible according to the row magic of phase objects. Also, the number of movement actions within the palace may be 24 as the total number of possible movements in the palace area. Also, the number of pass actions may be one action that does not perform an action. Thus, a total of 2451 actions may be implemented.

That is, the game play server 300, as described above, based on the action performed according to the notation among the actions specific to the long game and the result of the game, the predetermined state of the board (S) as input data, and the input It is possible to learn an action model having, as output data, an odds ratio estimation value (V) for each of at least one candidate action based on a predetermined chess board state (S), an action to be performed (A), and/or win/loss prediction information (O).

In this way, the deep learning-based long-term game service device according to the embodiment can extract input features specific to long-term games from a predetermined checkerboard state (S) and use them for deep learning model learning.

Meanwhile, the deep learning-based long term game service method according to an embodiment of the present invention may include a step of receiving, by the game play server 300, a first check board state (S107) .

In detail, the game play server 300 may receive a predetermined first chess board state in conjunction with the chess server 200 .

As an example, the game play server 300 may receive the current chess board state in the current playing game as the first chess board state in conjunction with the chess server 200 .

In addition, the deep learning-based long term game service method according to an embodiment of the present invention may include a step of performing deep learning based on the state of the first check board received by the game play server 300 (S109 ).

7 is an example of a diagram showing how an action model according to an embodiment of the present invention provides output data by performing deep learning based on a first chessboard state.

In detail, referring to FIG. 7 , the game play server 300 may perform deep learning based on the received first chessboard state 1 using an action model. In addition, the game play server 300 determines the odds ratio estimation value (V) for each candidate action, which is an action that can be performed in the first chess board state (1) based on the deep learning performed, and an action to win the game among at least one candidate action. The determined action (A) and/or the result of the corresponding game, that is, win/loss prediction information (O) predicting win/loss may be acquired.

In more detail, the game play server 300 may input the received first checkerboard state 1 to the action model as input data.

At this time, the action model receiving the first chessboard state 1 may be a deep learning model learned based on a training data set including a plurality of notations. Thus, the action model performs deep learning based on the input first chessboard state 1 to obtain an odds estimate value V for each candidate action based on the first chess board state 1, an action to be performed (A), and/or Win-loss prediction information (O) may be provided as output data.

Returning again, the game play server 300 also provides the winning rate for each of the at least one candidate action based on the first checkerboard state 1 output through deep learning performed in the action model receiving the first checkerboard state 1. An estimated value (V), an action to be performed (A), and/or win/loss prediction information (O) may be acquired.

In addition, the deep learning-based long-term game service method according to an embodiment of the present invention may include a step (S111) of providing the result of the deep learning performed as above by the game play server 300.

In detail, the game play server 300 calculates the odds estimate value (V) for each candidate action for the first chess board state (1) obtained through the above-described deep learning, the action to be performed (A), and/or the win-loss prediction information (O). It can be transmitted to the user's terminal 100. At this time, the user's terminal 100 may output and provide the received information according to a predetermined method (eg, display on a checkerboard image indicating the checkerboard state S).

In this way, the deep learning-based long-term game service device according to the embodiment provides the user with information on the odds of winning for each possible action in a specific checkerboard state (S), such as the current checkerboard state, and information predicting the game win or loss at that point in time. can do.

In addition, in the long-term game service method based on deep learning according to an embodiment of the present invention, the game play server 300 performs an action on the first check board state 1 based on the deep learning performed as described above. Step S113 may be included.

FIG. 8 is an example of a diagram showing how an action model according to an embodiment of the present invention performs an action determined for a first check board state (1).

In detail, referring to FIG. 8 , the game play server 300 performs the best action for winning the game in the first checker board state 1 based on the deep learning performed on the received first checker board state 1. That is, an action to be performed (A), which is determined to be an action having the highest winning rate among a plurality of candidate actions possible in the first chess board state (1), may be determined as an action for the first chess board state (1). Also, the game play server 300 may perform the determined action. Therefore, the long-term game service device based on deep learning according to the embodiment can perform a long-term game based on the action A determined as the best action to win in the corresponding game.

As described above, the long-term game service method and apparatus based on deep learning according to an embodiment of the present invention consider the characteristics of the game characteristically implemented in the long-term game service in more detail by learning a deep learning model to be specialized for the long-term game service. It is possible to implement an artificial intelligence computer that can perform deep learning optimized for it.

In addition, the deep learning-based long-term game service method and apparatus according to an embodiment of the present invention predict the winning rate for each action based on the game situation in the long-term game service using the learned deep learning model, and based on this, the corresponding The performance of artificial intelligence computers for long-term game services can be further improved by predicting the actions performed in the game situation and the result of the game accordingly.

In addition, the deep learning-based long-term game service method and apparatus according to an embodiment of the present invention can improve the level of long-term games of artificial intelligence computers by performing long-term games based on predicted actions.

In addition, the embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer readable recording medium. The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. medium), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes generated by a compiler. A hardware device may be modified with one or more software modules to perform processing according to the present invention and vice versa.

Specific implementations described in the present invention are examples and do not limit the scope of the present invention in any way. For brevity of the specification, description of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection of lines or connecting members between the components shown in the drawings are examples of functional connections and / or physical or circuit connections, which can be replaced in actual devices or additional various functional connections, physical connection, or circuit connections. In addition, if there is no specific reference such as “essential” or “important”, it may not be a component necessarily required for the application of the present invention.

In addition, the detailed description of the present invention described has been described with reference to preferred embodiments of the present invention, but those skilled in the art or those having ordinary knowledge in the art will find the spirit of the present invention described in the claims to be described later. And it will be understood that the present invention can be variously modified and changed without departing from the technical scope. Therefore, the technical scope of the present invention is not limited to the contents described in the detailed description of the specification, but should be defined by the claims.

Claims (10)

As a method for a game play server to provide a long-term game service based on deep learning to a user terminal,
Acquiring a training data set including a checkboard state for each of a plurality of actions;
extracting input features based on the obtained training data set;
learning a deep-learning model based on the extracted input features;
Receiving a first chessboard state; and
Acquiring at least one of an action to be performed and win-loss prediction information based on the deep learning model for the received first chessboard state
Long-term game service method based on deep learning. According to claim 1,
The input features are,
Type of long-term piece, history information on the last 8 actions of the first player, position information of the piece on the last 8 actions of the first player, and information on the last 8 actions of the second player At least some of history information, piece location information for the last 8 actions of the second player, and turn information about which player is the first player or the second player
Long-term game service method based on deep learning. According to claim 1,
The step of learning the deep-learning model,
A predetermined chessboard state is used as input data,
At least one of an odds ratio estimation value for each candidate action that is an action that can be performed in the input predetermined chess board state, an action to be performed with respect to the predetermined chess board state, and win-loss prediction information for the predetermined chess board state as output data Including the step of learning the deep learning model
Long-term game service method based on deep learning. According to claim 1,
The action (action),
It is an action to start by moving a specific piece on the chess board based on the magic of each piece of chess.
Long-term game service method based on deep learning. According to claim 4,
The action (action),
Including movement action outside the palace, horse movement action, statue movement action, movement action within the palace and pass action
Long-term game service method based on deep learning. According to claim 1,
Acquiring at least one of an action performed for the first chess board state and win/loss prediction information,
Including the step of inputting the first chess board state into the deep learning model and obtaining an odds estimate value for each candidate action that is an action that can be performed in the first chess board state
Long-term game service method based on deep learning. According to claim 6,
Acquiring at least one of an action performed for the first chess board state and win/loss prediction information,
Further comprising determining a candidate action having the highest odds ratio estimate as an action to be performed for the first chessboard state.
Long-term game service method based on deep learning. According to claim 7,
Acquiring at least one of an action performed for the first chess board state and win/loss prediction information,
Further comprising the step of obtaining the win-loss prediction information for the first check board state based on whether the determined execution action is won or lost.
Long-term game service method based on deep learning. According to claim 1,
Further comprising the step of performing the action for the first check board state
Long-term game service method based on deep learning. a communication unit for receiving a chess board state;
memory for storing deep learning models; and
Learning the deep learning model based on the received chess board state;
Based on the deep learning model, at least one of a performing action, which is an action having the highest odds estimate value among a plurality of candidate actions possible in the chess board state, and win-loss prediction information for the chess board state as the performing action is performed Obtaining at least one Characterized in that it comprises a processor;
Long-term game service device based on deep learning.

Images