KR20220141979A - Method and device for deep-learning based simulation count management of go game service - Google Patents

Abstract

According to an embodiment of the present invention, a device for managing the number of go game service simulation based on deep learning comprises: a communication unit receiving at least one among a go board state, a value, the number of visits, and the number of default simulation; a memory storing a prediction unit for the number of simulation and a control model for the number of simulation; and a processor determining the optimized number of simulation with respect to the go board state by using at least one among the go board state, the value, the number of visits, and the number of default simulation by reading the prediction unit for the number of simulation, allowing the control model for the number of simulation to learn based on the determined optimized number of simulation, and determining the number of simulation according to the current go board state by reading the control model for the number of simulation.

Description

Method and device for managing the number of simulations of Go game service based on deep learning

The present invention relates to a method and apparatus for managing the number of simulations of a Go game service based on deep learning. More specifically, it relates to a deep learning-based Go game service simulation number management method and apparatus for managing the number of deep learning simulations according to the game situation performed in the Go game service.

As the use of user terminals such as smartphones, tablet PCs, personal digital assistants (PDAs), and notebook computers has become popular and information processing technology has developed, it has become possible to play Go, a type of board game, using user terminals. It is now possible to play Go games with a programmed artificial intelligence computer.

Compared to other board games such as chess and chess, there are many cases of Go, so there is a limit to how artificial intelligence computers can play matches at human level, and research to increase the energy of artificial intelligence computers is being actively conducted.

Recently, developers have applied the Monte Carlo Tree Search (MCTS) algorithm and deep learning technology to artificial intelligence computers, raising the energy of artificial intelligence computers to the level of professional engineers.

Also, Go is a time-limited board game. Each Go tournament has a different time, and players can be given various times ranging from 1 hour to 5 hours each. Therefore, knowing the remaining Go time and determining how much time is spent on a single move is an important factor in winning the game.

However, the artificial intelligence computer has a problem in that the time consumed to place a single move is always constant, so that it starts a bad move in an important phase.

In addition, in general, general or amateur or artificial intelligence computers cannot predict the remaining game length, so there is a problem in that they cannot establish a time strategy.

In addition, when training is performed by applying the Monte Carlo Tree Search (MCTS) algorithm and deep learning technology to the artificial intelligence computer as above, in most general training, a fixed number of simulations is used.

However, using a fixed number of simulations in training has the problem of forcing constraints on the performance of artificial intelligence computers.

In addition, if you rely on a fixed number of simulations even when performing MCTS for selecting the next start in the current Go board status of the current game, the circumstances related to the corresponding Go board status (e.g., win rate and / or available time, etc.) ), there is a problem that the next start should be decided using only the set number of simulations and time.

JP 4392621 B2

An object of the present invention is to provide a deep learning-based Go game service simulation number management method and apparatus for managing the number of deep learning simulations according to the game situation performed in the Go game service.

In detail, the present invention is a deep learning-based Go game service simulation number management method that dynamically adjusts the number of simulations when performing deep learning based on the Monte Carlo Tree Search (MCTS) algorithm according to the state of the checkerboard And for the purpose of providing the device.

In addition, an object of the present invention is to provide a deep learning-based Go game service simulation number management method and apparatus for changing the start preparation time in an important phase.

In addition, an object of the present invention is to provide a method and apparatus for managing the number of simulations of a Go game service based on deep learning that can effectively divide the start preparation time using the predicted remaining game length.

However, the technical problems to be achieved by the present invention and embodiments of the present invention are not limited to the technical problems as described above, and other technical problems may exist.

A deep learning-based Go game service simulation number management apparatus according to an embodiment of the present invention includes: a communication unit for receiving at least one of a Go board state, a value value, the number of visits, and the default number of simulations; a memory for storing the simulation number prediction unit and the simulation number adjustment model; and reading the number prediction unit to determine the optimal number of simulations for the checkerboard state using at least one of the checkerboard state, the value value, the number of visits, and the default number of simulations, and the number adjustment model based on the determined optimal number of simulations and a processor for learning the number of times and reading the number adjustment model to determine the number of simulations according to the current state of the checkerboard.

In this case, the optimal number of simulations includes the number of simulations that take the minimum time within a critical point in which the next starting point selected during Monte Carlo Tree Search (MCTS) simulation is not changed.

In addition, the number prediction unit, by matching the determined optimal number of simulations and the checkerboard state to generate a checkerboard state-optimal number of simulations data, the processor, the generated checkerboard state-optimal number of simulations data, a training data set (Training data set) to train the frequency adjustment model.

In addition, the number prediction unit, based on the number of default simulations, based on the first MCTS simulation performed based on the checkerboard state, obtains at least one of a first number of visits and a first value for each of a plurality of start candidates, and , based on the second MCTS simulation performed based on the checkerboard state based on the additional unit number of adding a predetermined number of times to the default simulation number, at least of a second number of visits and a second value value for each number of starting candidates get one

In addition, the number prediction unit, based on the first number of visits and the second number of visits, determines whether the next starting point is changed when the number of simulations increases, The number of simulations that take time is determined as the optimal number of simulations.

In addition, the number prediction unit determines whether to change the next starting point based on the rate of change based on the first number of visits and the second number of visits, and the rate of change is the difference between the default number of simulations and the number of additional units It is calculated based on an increase in the number of visits based on a difference between the number of first visits and the number of visits to the second compared to an increase in the number of simulations based on .

In addition, the number prediction unit, based on the ratio between the first number of visits for each of the plurality of starting candidates and the ratio between the second number of visits for each of the plurality of starting candidates determines whether to change the next starting point.

In addition, the number prediction unit determines whether the next starting point is changed when the number of simulations increases based on the increase/decrease rate based on the first value and the second value, and the increase/decrease rate is the default simulation number and It is calculated based on the increase in the value of the value based on the difference between the first value and the second value compared to the increase in the number of simulations based on the difference between the number of additional units.

In addition, when the increase/decrease rate satisfies a predetermined criterion, the number prediction unit determines the minimum number of simulations when calculating the increase/decrease rate as the optimal number of simulations.

In addition, the number adjustment model performs an adjustment process for the number of simulations according to the current checkerboard state, and the adjustment process includes a process of setting at least one of an upper limit and a lower limit of the number of simulations, and the predetermined probability. A process of increasing the number of simulations, a process of adjusting the number of simulations based on at least one of a win rate and a change in win rate according to the current checkerboard state, and adjusting the number of simulations based on a predetermined statistic on the number of simulations and at least one of the processes.

In addition, the processor provides the number of simulations according to the current checkerboard state as a set-off model for performing the MCTS simulation so that the set-off model performs the MCTS simulation based on the number of simulations.

On the other hand, the deep learning-based Go game service simulation number management method according to an embodiment of the present invention is a method of managing the number of deep learning-based Go game service simulation times in a time management model server, comprising the steps of: receiving a predetermined Go board state; performing a first MCTS based on the received checkerboard state; obtaining at least one of a first number of visits and a first value based on the performed first MCTS; performing a second MCTS based on the received checkerboard state; obtaining at least one of a second number of visits and a second value based on the performed second MCTS; Determining the optimal number of simulations for the checkerboard state based on the first number of visits and the second number of visits, or determining the optimal number of simulations for the checkerboard state based on the first value and the second value step; generating checkerboard state-optimal simulation number data by matching the determined optimal number of simulations with the checkerboard state; learning a deep learning model based on the generated checkerboard state-optimal number of simulations data; determining the number of simulations according to the current checkerboard state based on the learned deep learning model; and providing the determined number of simulations as a starting model.

Deep learning-based Go game service simulation number management method and apparatus according to an embodiment of the present invention, a Monte Carlo tree search (Monte Carlo Tree Search; MCTS) algorithm to learn the number of simulations when performing deep learning based on the can

In addition, the deep learning-based Go game service simulation number management method and apparatus according to an embodiment of the present invention can dynamically adjust the optimal number of simulations according to the state of the Go board through the learning.

In addition, the deep learning-based Go game service simulation number management method and apparatus according to an embodiment of the present invention can change the number of simulations in an important phase.

In addition, the deep learning-based Go game service simulation number management method and apparatus according to an embodiment of the present invention may determine the next start by performing MCTS based on the optimal number of simulations.

In addition, the deep learning-based Go game service simulation number management method and apparatus according to an embodiment of the present invention can change the start preparation time in an important phase.

In addition, the deep learning-based Go game service simulation number management method and apparatus according to an embodiment of the present invention can effectively distribute the start preparation time using the predicted remaining playing time.

However, the effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood from the description below.

1 is an exemplary diagram of a deep learning-based Go game service system according to an embodiment of the present invention.
Figure 2 is a view for explaining the start model structure of the start model server for the start of the artificial intelligence computer in the deep learning-based Go game service according to an embodiment of the present invention.
3 is a view for explaining the movement probability distribution for the starting point according to the policy of the starting model.
4 is a view for explaining the value value and the number of visits to the starting point of the starting model.
5 is a diagram for explaining a process in which an initiation model launches according to a pipeline of a search unit.
6 is an exemplary diagram showing a screen providing a situation determination function of a deep learning-based Go game service according to an embodiment of the present invention.
7 is a diagram for explaining the structure of a situation determination model of the situation determination model server of the present invention.
8 is a diagram for explaining one block of the neural network structure composed of a plurality of blocks of the situation determination model of the present invention.
9 is a view for explaining the first and second pre-processing steps for generating a correct answer label used for learning the situation judgment model of the present invention.
10 is a view for explaining the first and second pre-processing steps for generating a correct answer label used to learn the situation judgment model of the present invention.
11 is a view for explaining a third pre-processing step for generating a correct answer label used to learn the situation judgment model of the present invention.
12 is a view for explaining a situation determination result of the situation determination model of the present invention.
13 is a view comparing the situation determination result of the situation determination model of the present invention with the situation determination result by the deep learning model according to the prior art.
14 is a view comparing the situation determination result of the situation determination model of the present invention with the situation determination result by the deep learning model according to the prior art.
15 is a view comparing the situation determination result of the situation determination model of the present invention with the situation determination result by the deep learning model according to the prior art.
16 is an exemplary diagram of a signal flow in a deep learning-based Go game service system according to an embodiment of the present invention.
17 is a method for determining a situation in a deep learning-based Go game service method according to an embodiment of the present invention.
18 is a pre-processing method of training data for generating a correct answer label in the method of determining the situation of FIG. 17 .
19 is a diagram for explaining a time management unit of a time management model server according to an embodiment of the present invention.
20A and 20B are diagrams for explaining distributed calculation of a time management unit according to an embodiment of the present invention.
21 is an exemplary diagram of a signal flow in the Go game service system of the time management model server according to an embodiment of the present invention.
22 is a method for determining a start preparation time among the deep learning-based Go game service methods according to an embodiment of the present invention.
23 is a view for explaining a simulation number prediction unit and a simulation number adjustment model of the time management model server according to another embodiment of the present invention.
24 is a method for determining the number of MCTS simulations among the deep learning-based Go game service methods according to another embodiment of the present invention.
25 is an example of a state in which the first MCTS and the second MCTS are performed for the checkerboard state according to another embodiment of the present invention, and the value value and the number of visits in each are derived.
26 is an example of a diagram for explaining a method of determining the optimal number of simulations based on the number of visits according to another embodiment of the present invention.
27 is an example of a diagram for explaining a method of determining the optimal number of simulations based on a ratio of the number of visits according to another embodiment of the present invention.
28 is an example of a diagram for explaining a method of determining the optimal number of simulations based on a value according to another embodiment of the present invention.
29 is an example of a diagram for explaining a method of adjusting the number of simulations based on a change in win rate according to another embodiment of the present invention.
30 is a conceptual diagram for explaining a set-up model for performing MCTS based on the number of simulations according to another embodiment of the present invention.
31 is a diagram for explaining a time management model of a time management model server according to another embodiment of the present invention.
32A and 32B are diagrams for explaining the amount of change in water collection used to generate game time information according to another embodiment of the present invention.
33A and 33B are diagrams for explaining the amount of change in water collection used to generate game time information according to another embodiment of the present invention.
34A and 34B are diagrams for explaining a common multiple used to generate game time information according to another embodiment of the present invention.
35 is an exemplary diagram of a signal flow in the Go game service system of the time management model server according to another embodiment of the present invention.
36 is a method for generating game time information among the deep learning-based Go game service methods according to another embodiment of the present invention.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. Effects and features of the present invention, and a method for achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms. In the following embodiments, terms such as first, second, etc. are used for the purpose of distinguishing one component from another, not in a limiting sense. Also, the singular expression includes the plural expression unless the context clearly dictates otherwise. In addition, terms such as include or have means that the features or components described in the specification are present, and do not preclude the possibility that one or more other features or components will be added. In addition, in the drawings, the size of the components may be exaggerated or reduced for convenience of description. For example, since the size and thickness of each component shown in the drawings are arbitrarily indicated for convenience of description, the present invention is not necessarily limited to the illustrated bar.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when described with reference to the drawings, the same or corresponding components are given the same reference numerals, and the overlapping description thereof will be omitted. .

-----------------------------

1 is an exemplary diagram of a deep learning-based Go game service system according to an embodiment of the present invention.

1, the deep learning-based Go game service system according to the embodiment is a terminal 100, a Go server 200, a start model server 300, a situation determination model server 400, and a network 500) may include.

Each component of FIG. 1 may be connected through a network 500 . The terminal 100, the Go server 200, the start model server 300, the situation judgment model server 400, and the time management model server 500, such as to mean a connection structure in which information can be exchanged between each node. , Examples of such networks include a 3rd Generation Partnership Project (3GPP) network, a Long Term Evolution (LTE) network, a World Interoperability for Microwave Access (WIMAX) network, the Internet, a Local Area Network (LAN), and a Wireless LAN (Wireless LAN) network. Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth (Bluetooth) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, and the like are included, but are not limited thereto.

<Terminal (100)>

First, the terminal 100 is a terminal of a user who wants to be provided with a Go game service. Also, the terminal 100 is one or more computers or other electronic devices used by a user to execute applications that perform various tasks. For example, it includes a computer, a laptop computer, a smart phone, a mobile phone, a PDA, a tablet PC, or any other device operable to communicate with the Go server 200 . However, not limited thereto, the terminal 100 is executed on various machines, and includes processing logic for interpreting and executing commands stored in a plurality of memories, and for a graphical user interface (GUI) on an external input/output device. It may include various other elements, such as processes for displaying graphical information. In addition, the terminal 100 may be connected to an input device (eg, a mouse, a keyboard, a touch-sensitive surface, etc.) and an output device (eg, a display device, a monitor, a screen, etc.). Applications executed by the terminal 100 may include a game application, a web browser, a web application running on a web browser, word processors, media players, spreadsheets, image processors, security software or the like. .

In addition, the terminal 100 may include at least one memory 101 for storing instructions, at least one processor 102 , and a communication unit 103 .

The memory 101 of the terminal 100 may store a plurality of application programs or applications driven in the terminal 100 , data for operation of the terminal 100 , and commands. The instructions are executable by the processor 102 to cause the processor 102 to perform the operations, and the operations are to send a Go game execution request signal, send and receive game data, send and receive start information, send a situation determination request signal, the situation It may include operations of receiving a determination result, requesting game time information, receiving game time information, and receiving various types of information. In addition, in terms of hardware, the memory 101 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and the memory 130 performs the storage function of the memory 101 on the Internet. It may be a web storage that performs.

The processor 102 of the terminal 100 may perform data processing for receiving a Go game service by controlling the overall operation. When the Go game application is executed in the terminal 100 , the Go game environment is configured in the terminal 100 . And the Go game application exchanges Go game data with the Go server 200 through the network 500 so that the Go game service is executed on the terminal 100 . These processors 102 are ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers (controllers), micro It may be a controller (micro-controllers), microprocessors (microprocessors), any type of processor for performing other functions.

The communication unit 103 of the terminal 100 includes the following communication methods (eg, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink (HSUPA)) Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc.), WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA ( Digital Living Network Alliance), WiBro (Wireless Broadband), and WiMAX (World Interoperability for Microwave Access) can transmit and receive a wireless signal to and from at least one of a base station, an external terminal, and a server on a network built in accordance with.

The terminal 100 performs at least a part of the functional operation performed in at least one of the Go server 200, the start model server 300, the situation determination model server 400, and the time management model server 500 to be described later. You may.

<Go server (200)>

The Go game service provided by the Go server 200 may be configured in a form in which a virtual computer user and a real user provided by the Go server 200 participate in the game together. This means that one real user and one computer user play a game together in a Go game environment implemented on the user's terminal 100 . In another aspect, the Go game service provided by the Go server 200 may be configured in such a way that a plurality of user-side devices participate and the Go game is played.

Go server 200 may include at least one memory 201 for storing instructions, at least one processor 202 and a communication unit 203 .

The memory 201 of the Go server 200 may store a plurality of application programs or applications running in the Go server 200, data for the operation of the Go server 200, and commands. . The instructions are executable by the processor 202 to cause the processor 202 to perform operations, and the operations are game execution request signal reception, game data transmission/reception, start information transmission/reception, situation determination request signal transmission/reception, situation determination result transmission/reception , may include transmission and reception of start preparation time, transmission and reception of game information time, and various transmission operations. In addition, the memory 201 may store a plurality of notations that were played in the Go server 200 or a plurality of previously published notations. Each of the plurality of notations may include all information from the first start, which is the first start information of the start of the game, to the final start at which the game ends. That is, a plurality of notations may include history information about the start. The Go server 200 makes it possible to provide a plurality of stored notations for the training of the posture determination model server 400 to the posture determination model server 400 . In addition, in terms of hardware, the memory 201 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and the memory 201 performs the storage function of the memory 201 on the Internet. It may be a web storage that performs.

The processor 202 of the Go server 200 may perform data processing for providing a Go game service by controlling the overall operation. These processors 202 are ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers (controllers), micro It may be a controller (micro-controllers), microprocessors (microprocessors), any type of processor for performing other functions.

The Go server 200 may communicate with the terminal 100 , the start model server 300 and the situation determination model server 400 via the network 500 through the communication unit 203 .

<Start model server (300)>

Initiation model server 300 may include a separate cloud server or computing device. In addition, the set-up model server 300 may be a neural network system installed in the data processing unit of the processor or Go server 200 of the terminal 100, but below, the set-out model server 300 is the terminal 100 or the Go server ( 200) and a separate device.

Initiation model server 300 may include at least one memory 301 for storing instructions, at least one processor 302 and a communication unit 303 .

The start model server 300 is an artificial intelligence computer that can learn by itself according to the rules of Go to build a set-up model, which is a deep learning model, and play a game with the user of the terminal 100. initiation can be carried out. The detailed description of the set-up model server 300 training with the set-off model follows the description of the set-up model of FIGS. 2 to 5 .

The memory 301 of the initiation model server 300 is a plurality of applications (application program) or applications (application) driven in the initiation model server 300, data for the operation of the initiation model server 300, commands can be saved The instructions are executable by the processor 302 to cause the processor 302 to perform the operations, and the operations are a set-out model learning (training) action, send/receive set-up information, receive set-up preparation time, receive game time information, and transmit various It can include actions. In addition, the memory 301 may store an initiation model that is a deep learning model. In addition, in terms of hardware, the memory 301 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and the memory 301 performs the storage function of the memory 301 on the Internet. It may be a web storage that performs.

The processor 302 of the set-off model server 300 reads the set-off model stored in the memory 302, and performs the set-up model learning and Go start to be described below according to the built-up neural network system. According to an embodiment, the processor 302 may be configured to include a main processor that controls all units, and a plurality of graphics processing units (GPUs) that process large-capacity calculations required when driving a neural network according to an initiation model. have.

The start model server 300 may communicate with the Go server 200 via the network 500 through the communication unit 303 .

<Situation judgment model server 400>

The situation determination model server 400 may include a separate cloud server or computing device. In addition, the situation determination model server 400 may be a neural network system installed in the processor of the terminal 100 or the data processing unit of the Go server 200, but below, the situation determination model server 400 is the terminal 100 or Go It will be described as a separate device from the server 200 .

The situation determination model server 400 may include at least one memory 401 for storing instructions, at least one processor 402 and a communication unit 403 .

The situation determination model server 400 may receive the training data set from the Go server 200 through the communication unit 403 . The training data set may be a plurality of notations and situation determination information for the plurality of notations. The posture determination model server 400 supervises learning to determine the condition of the Go board on which the Go balls are placed using the received training data set to build a deep learning model, the posture determination model, and the condition of the terminal 100 user A situational judgment may be performed according to a judgment request. A detailed description of the situation determination model server 400 training with the situation determination model follows the description of the situation determination model of FIGS. 6 to 18 .

The memory 401 of the situation determination model server 400 is a plurality of application programs or applications running in the situation determination model server 400, data for the operation of the situation determination model server 400 , commands can be stored. The instructions are executable by the processor 402 to cause the processor 402 to perform the operations, and the operations include a situation determination model learning (training) operation, performing a situation determination, transmitting a situation determination result, receiving a plurality of notation information, It may include transmission of change amount information of collection water, transmission of common drainage information, and various transmission operations. Also, the memory 401 may store a situation judgment model that is a deep learning model. In addition, in terms of hardware, the memory 401 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and the memory 401 performs the storage function of the memory 301 on the Internet. It may be a web storage that performs.

The processor 402 of the situation determination model server 400 reads the situation determination model stored in the memory 402, and according to the constructed neural network system, learns the situation determination model described below and determines the situation of the checkerboard during the game. . According to the embodiment, the processor 402 may be configured to include a main processor that controls all units, and a plurality of graphics processing units (GPUs) that process large-capacity calculations required when driving a neural network according to a situation determination model. can

The situation determination model server 400 may communicate with the Go server 200 via the network 500 through the communication unit 403 .

<Time Management Model Server (500)>

The time management model server 500 may include a separate cloud server or computing device. In addition, the time management model server 500 may be a neural network system installed in the processor of the terminal 100, the memory of the Go server 200, the memory of the start model server 300, or the memory of the situation judgment model server 400, but , Hereinafter, the time management model server 500 will be described as a device separate from the terminal 100 , the Go server 200 , the start model server 300 or the situation determination model server 400 .

The time management model server 500 may include at least one memory 501 for storing instructions, at least one processor 502 and a communication unit 503 .

In addition, the time management model server 500 may receive the number of visits, a search probability value, or a value value from the start model server 300 through the communication unit 503 . The time management model server 500 may determine the start preparation time using the received number of visits, the search probability value, and the value value. A detailed description of the method for determining the start preparation time of the time management model server 500 follows the description of FIGS. 19 to 22 .

In addition, the time management model server 500 may receive a training data set from the Go server 500 through the communication unit 503 . The training data set may be a plurality of notations, situation determination information for the plurality of notations, and time information according to each checkerboard state for a plurality of notations.

In addition, the time management model server 500 is a checkerboard state (S), the default number of simulations (D), the number of visits (N) and / or receive a value (V). The time management model server 500 may determine the number of MCTS simulations using the received checkerboard state (S), the default number of simulations (D), the number of visits (N), and/or the value value (V). A detailed description of the method of determining the number of MCTS simulations by the time management model server 500 follows the description of FIGS. 23 to 30 .

In addition, the time management model server 500 may receive the situation determination information from the situation determination model server 400 through the communication unit 503 . The situation determination information may include information on a change amount of catchment, information on common drainage, and the like. The time management model server 500 builds a time management model that is a deep learning model by supervising learning so as to generate game time information using the received training data set, situation determination information, value value, etc., and starting model server 300 ) or may provide game time information according to the state of the checkerboard to the terminal 100 . The detailed description of the time management model server 50 training with the time management model follows the description of the time management model of FIGS. 35 to 36 .

The memory 501 of the time management model server 500 is a plurality of application programs or applications driven in the time management model server 500, data for the operation of the time management model server 500 , commands can be stored. The instructions are executable by the processor 502 to cause the processor 502 to perform operations, the operations comprising: a time management model learning (training) operation; , a plurality of notation information reception, default simulation number reception, collection change amount information reception, common multiple information reception, game time information generation and various transmission operations may be included. Also, the memory 501 may store a time management unit, a simulation number prediction unit, a simulation number adjustment model, or a time management model. In addition, in terms of hardware, the memory 501 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and the memory 501 performs the storage function of the memory 501 on the Internet. It may be a web storage that performs.

The processor 502 of the time management model server 500 reads the time management unit stored in the memory 501, and performs a start preparation time determination to be described below.

In addition, the processor 502 of the time management model server 500 reads the simulation number prediction unit and/or the simulation number adjustment model stored in the memory 501 to determine the number of MCTS simulations to be described later.

In addition, the processor 502 of the time management model server 500 reads the time management model stored in the memory 501 and generates game time information to be described below according to the constructed neural network system.

According to the embodiment, the processor 502 may be configured to include a main processor that controls all units, and a plurality of graphics processing units (GPUs) that process large-capacity operations required when driving a neural network according to a time management model. can

The time management model server 500 may communicate with the Go server 200 , the start model server 300 and the situation determination model server 400 via the network 500 through the communication unit 403 .

<Start model>

Figure 2 is a view for explaining the start model structure of the start model server for the start of the artificial intelligence computer in the deep learning-based Go game service according to an embodiment of the present invention, Figure 3 is a start point according to the policy of the start model It is a diagram for explaining the movement probability distribution for , FIG. 4 is a diagram for explaining the value value and number of visits to the starting point of the initiation model, and FIG. It is a drawing for explanation.

Referring to FIG. 2 , the settling model according to an embodiment of the present invention may include a search unit 310 , a self play unit 320 and a set off neural network 330 as a deep learning model of the settling model server 300 . .

)을 출력할 수 있다. 셀프 플레이부(320)는 탐색 확률값(

)에 따라 스스로 바둑 대국을 할 수 있다. 셀프 플레이부(320)는 게임의 승패가 결정되는 시점까지 스스로 바둑 대국을 진행하고, 자가 대국이 종료되면 바둑판 상태(S), 탐색 확률값(

), 자가 플레이 가치값(z)을 착수 신경망(330)에 제공할 수 있다. 바둑판 상태(S)는 착수점들에 바둑돌이 놓여진 상태이다. 자가 플레이 가치값(z)은 바둑판 상태(S)에서 자가 대국을 하였을 때 승률 값이다. 착수 신경망(330)은 이동 확률값(p)과 가치값(V)을 출력할 수 있다. 이동 확률값(p)은 바둑판 상태(S)에 따라 착수점들에 대해 어느 착수점에 착수하는 것이 게임을 이길 수 있는 좋은 수인지 수치로 나타낸 확률분포값이다. 가치값(V)은 해당 착수점에 착수시 승률을 나타낸다. 예를 들어, 이동 확률값(p)이 높은 착수점이 좋은 수일 수 있다. 착수 신경망(330)은 이동 확률값(p)이 탐색 확률값(

)과 동일해지도록 트레이닝되고, 가치값(V)이 자가 플레이 가치값(z)과 동일해지도록 트레이닝될 수 있다. 이후 트레이닝된 착수 신경망(330)은 탐색부(310)를 가이드하고, 탐색부(310)는 이전 탐색 확률값(

)보다 더 좋은 수를 찾도록 착수 준비 시간 동안 MCTS를 진행하여 새로운 탐색 확률값(

)을 출력하게 한다. 예를 들어, 착수 준비 시간은 MCTS 진행 시간에 따라 평균 착수 준비 시간, 제1 착수 준비 시간 및 제2 착수 준비 시간 중 어느 하나의 착 수 준비 시간을 따를 수 있다. 착수 준비 시간은 시간 관리 모델 서버(500)에서 제공할 수 있고 기본적으로 평균 착수 준비 시간으로 설정되어 있을 수 있다. 셀프 플레이부(320)는 새로운 탐색 확률값(

)에 기초하여 바둑판 상태(S)에 따른 새로운 자가 플레이 가치값(z)을 출력하고 바둑판 상태(S), 새로운 탐색 확률값(

), 새로운 자가 플레이 가치값(z)을 착수 신경망(330)에 제공할 수 있다. 착수 신경망(330)은 이동 확률값(p)과 가치값(V)이 새로운 탐색 확률값(

)과 새로운 자가 플레이 가치값(z)으로 출력되도록 다시 트레이닝될 수 있다. 즉, 착수 모델은 이러한 과정을 반복하여 착수 신경망(330)이 대국에서 이기기 위한 더 좋은 착수점을 찾도록 트레이닝 될 수 있다. 일 예로, 착수 모델은 착수 손실(l)을 이용할 수 있다. 착수 손실(l)은 수학식 1과 같다.The set-off model can be learned as a model set out to win the game by using the search unit 310 , the self-play unit 320 , and the set-out neural network 330 . More specifically, the search unit 310 may perform a Monte Carlo Tree Search (MCTS) operation according to the guide of the onset neural network 330 . MCTS is an experiential search algorithm for decision making. That is, the search unit 310 may perform MCTS based on the movement probability value p and/or the value value V provided by the onset neural network 330 . As an example, the search unit 310 guided by the onset neural network 330 performs MCTS to find a search probability value (

) can be printed. The self-play unit 320 is a search probability value (

), you can play Go by yourself. The self-player 320 proceeds with a Go game by itself until the time when the victory or defeat of the game is determined, and when the self-playing game ends, the Go board state (S), the search probability value (

), the self-play value value z may be provided to the onset neural network 330 . The checkerboard state (S) is a state in which Go stones are placed at the starting points. The self-play value value (z) is a win rate value when a self-playing game is played in the checkerboard state (S). The onset neural network 330 may output a movement probability value (p) and a value value (V). The moving probability value (p) is a probability distribution value expressed numerically whether it is a good number to win the game by starting at which starting point for the starting points according to the checkerboard state (S). The value (V) represents the win rate when starting the corresponding starting point. For example, the moving probability value (p) may be a good number of starting points high. The onset neural network 330 is a movement probability value (p) is a search probability value (

) and can be trained so that the value V is equal to the self-play value z. Afterwards, the trained onset neural network 330 guides the search unit 310, and the search unit 310 sets the previous search probability value (

) to find a better number than the new search probability value (

) to output. For example, the start preparation time may follow the start preparation time of any one of the average start preparation time, the first set off preparation time, and the second set off preparation time according to the MCTS progress time. The start preparation time may be provided by the time management model server 500 and may be basically set as the average start preparation time. The self-play unit 320 sets a new search probability value (

), output a new self-play value value (z) according to the checkerboard state (S) based on the checkerboard state (S), and a new search probability value (

), a new self-play value z may be provided to the initiating neural network 330 . The starting neural network 330 is a new search probability value (p) and a value value (V)

) and a new self-play value (z) can be retrained to be output. That is, the onset model may be trained to repeat this process to find a better starting point for the initiation neural network 330 to win the game. As an example, the settling model may use the settling loss (l). The onset loss (l) is the same as Equation 1.

(Equation 1)

는 신경망의 파라미터이고, c는 매우 작은 상수이다.

is a parameter of the neural network, and c is a very small constant.

와 p가 같아 지도록 하는 것은 크로스 엔트로피 손실(cross entropy loss) 텀에 해당되고,

에 c를 곱하는 것은 정규화 텀으로 오버피팅(overfitting)을 방지하기 위한 것이다.To make z and v equal in the settling loss (l) of Equation 1 corresponds to a mean square loss term,

Making p and p be equal corresponds to a cross entropy loss term,

Multiplying by c is to prevent overfitting with the regularization term.

)은 하나의 착수점에서 위에 표시된 값으로 나타낼 수 있다. 탐색 확률값(

)은 착수 후보수의 방문 횟수를 전체 횟수로 나눈 비율일 수 있다. 일 예로, MCTS 시뮬레이션 전체 횟수가 1000번이고 90.00이라고 표시되어 있으면 해당 착수 후보수에 1000번 중 900번 방문했다는 것을 의미한다. 트레이닝 된 착수 모델의 가치값(V)은 도 4의 하나의 착수점에서 아래에 표시된 값으로 나타낼 수 있다. 착수 신경망(330)은 신경망 구조로 구성될 수 있다. 일 예로, 착수 신경망(330)은 한 개의 컨볼루션(convolution) 블록과 19개의 레지듀얼(residual) 블록으로 구성될 수 있다. 컨볼루션 블록은 3X3 컨볼루션 레이어가 여러개 중첩된 형태일 있다. 하나의 레지듀얼 블록은 3X3 컨볼루션 레이어가 여러개 중첩되고 스킵 커넥션을 포함한 형태일 수 있다. 스킵 커넥션은 소정의 레이어의 입력이 해당 레이어의 출력값과 합하여서 출력되어 다른 레이어에 입력되는 구조이다. 또한, 착수 신경망(330)의 입력은 흑 플레이어의 최근 8 수에 대한 돌의 위치 정보과 백 플레이어의 최근 8 수에 대한 돌의 위치 정보와 현재 플레이어가 흑인지 백인지에 대한 차례 정보를 포함한 19*19*17의 RGB 이미지가 입력될 수 있다.For example, referring to FIG. 3 , the trained set off model may represent the moving probability value p at the set points as a probability distribution value as shown in FIG. 3 . Referring to Figure 4, the search probability value (

) can be expressed as the value indicated above at one starting point. Search probability value (

) may be a ratio obtained by dividing the number of visits by the number of start-up candidates by the total number of visits. For example, if the total number of MCTS simulations is 1000 and it is marked as 90.00, it means that 900 out of 1000 visits are made to the number of candidates for starting. The value (V) of the trained set off model may be represented by a value displayed below at one set point of FIG. 4 . The onset neural network 330 may be configured in a neural network structure. As an example, the onset neural network 330 may include one convolution block and 19 residual blocks. The convolution block may have a form in which several 3X3 convolution layers are superimposed. One residual block may have a form in which several 3X3 convolution layers are overlapped and a skip connection is included. The skip connection is a structure in which an input of a predetermined layer is output by summing an output value of the corresponding layer and input to another layer. In addition, the input of the onset neural network 330 includes the location information of the stone for the last 8 numbers of the black player, the location information of the stones for the last 8 numbers of the white player, and turn information on whether the current player is black or white 19* An RGB image of 19*17 can be input.

)을 출력할 수 있다. 착수 모델은 착수점들 중 가장 높은 탐색 확률값(

)을 선택하여 착수할 수 있다. Referring to FIG. 5 , the learned initiation model may be launched using the initiation neural network 330 and the search unit 310 in its turn. The initiation model is a second checkerboard state (S1) having a high launch point with high activity function (Q) and confidence value (U) among branches that are not currently searched through MCTS in the first checkerboard state (S1) through the selection process (a). -2) is selected. The activity function (Q) is the average value of the value values (V) calculated every time the branch passes. The confidence value (U) is inversely proportional to the number of visits (N) passing through the branch and is proportional to the movement probability value (p). The start model can be expanded to the third checkerboard state (S1-2-1) at the selected set point through the expansion and evaluation process (b) and calculate the movement probability value (p). The start model calculates the value (V) of the extended third checkerboard state (S1-2-1), and the activity function (Q), number of visits (N), and movement probability value of branches passed through the backup process (c) (p) can be saved. The initiation model repeats the process of selection (a), expansion and evaluation (b), and backup (c) during the set-up preparation time, and creates a probability distribution using the number of visits (N) for each starting point to create a search probability value (

) can be printed. The launch model has the highest search probability value (

) to start.

<Situation Judgment Model>

6 is an exemplary view showing a screen providing a situation determination function of a deep learning-based Go game service according to an embodiment of the present invention, and FIG. 7 is a diagram to explain the structure of the situation determination model of the situation determination model server of the present invention 8 is a diagram for explaining one block of the neural network structure composed of a plurality of blocks of the situation judgment model of the present invention.

Referring to FIG. 6 , the deep learning-based Go game service according to an embodiment of the present invention may determine the situation of the current Go board state. For example, as shown in FIG. 6 , when a user requests a situation determination by clicking the situation determination menu (A) during a Go game on the screen of the terminal 100, the deep learning-based Go game service provides the situation determination result in a pop-up window. can The situation judgment is to determine who wins by how many points by calculating the opponent and my house during the Go game. For example, if the user decides that the situation is favorable to me, do not overdo it any more and devise a strategy to end the game while maintaining the current advantageous situation. If it is judged unfavorable, the game phase will be changed Several strategies can be devised to do this. The standard for judging the situation is a house, a stone, a stone, a ball, and a big according to the state of the Go stones placed on the board. A stone is a stone placed on a checkerboard and is not a score in Korean rules. A house is an area made up of empty dots surrounded by one-colored Go stones, which is a score in Korean rules. Gongbae and Big are areas that are neither black nor white when Go is over, and are not points in Korean rules. Among the stones placed on the checkerboard, the Panwisa Stone is a stone that has no choice but to be caught and killed. Under Korean rules, it is used to fill the opponent's house, so it is a score. Big refers to an area that is neither black nor white when the Go game is over. Therefore, in order to judge the situation, it is necessary to accurately distinguish or predict a house, a stone, a stone, a ball, and a big in the state of the Go board on which the Go stones are placed can be an accurate judgment. At this time, to accurately distinguish a house, a stone, a stone, a public stone, and a big is to distinguish the state in which a house, a stone, a stone, a public stone, and a big are completely formed, and to accurately predict a house, a stone, a stone, a public meeting, and a big is a house , stone, stone, public meeting, or predicting a state that is likely to become a big one.

Referring to FIG. 7 , the situation determination model according to an embodiment of the present invention is a deep learning model of the situation determination model server 400 , and a situation determination neural network 410 , an input feature extractor 420 , and a correct answer label generator 430 . ) may be included.

The situation determination model may perform supervised learning to determine the situation of the current checkerboard state using the situation determination neural network 410 . More specifically, a training data set for the situation judgment model checkerboard state (S) is generated, and the situation judgment neural network 410 is trained to determine the situation according to the current checkerboard state (S) using the generated training data set. can do it The situation determination model server 400 may receive a plurality of notations from the Go server 200 . Each notation of a plurality of notations may include a respective checkerboard state (S) according to the starting order.

The input feature extraction unit 420 may extract the input feature (IF) from the checkerboard state (S) of a plurality of notations and provide it to the situation determination neural network 410 as input data for training. The input feature (IF) of the checkerboard state (S) includes the position information of the stone for the last 8 moves of the black player, the position information of the stone for the last 8 moves of the white player, and turn information on whether the current player is black or white. It can be an RGB image of 19*19*18. For example, the input feature extractor 420 may have a neural network structure and may include a kind of encoder.

)로 제공할 수 있다. 정답 레이블 생성부(430)의 정답 레이블 생성은 후술하는 도 9 내지 도 11의 설명을 따른다. 일 예로, 정답 레이블 생성부(430)는 신경망 구조의 롤아웃 또는 인코더를 포함할 수 있다.The correct answer label generator 430 generates a correct answer label (ground truth) through a pre-processing process in the current checkerboard state (S), and applies the correct answer label to the neural network 410 for training target data (

) can be provided. The correct answer label generation unit 430 generates the correct answer label according to the description of FIGS. 9 to 11, which will be described later. As an example, the correct answer label generator 430 may include a rollout of a neural network structure or an encoder.

)로 한 트레이닝 데이터 셋을 이용하여 형세 판단 신경망(410)에서 생성된 출력 데이터(o)가 타겟 데이터(

)와 동일해지도록 형세 판단 신경망(420)을 충분히 학습할 수 있다. 일 예로, 형세 판단 모델은 형세 판단 손실(

)을 이용할 수 있다. 형세 판단 손실(

)은 평균 제곱 에러(mean square error)를 이용할 수 있다. 예를 들어, 형세 판단 손실(

)은 수학식 2와 같다.The situation judgment model uses the input feature (IF) as input data and the correct answer label as the target data (

), the output data (o) generated by the situation determination neural network 410 using the training data set as the target data (

), the situation determination neural network 420 can be sufficiently trained. As an example, the situation judgment model is a situation judgment loss (

) can be used. loss of judgment (

) can use the mean square error. For example, loss of judgment (

) is the same as in Equation 2.

(Equation 2)

는 현재 바둑판 상태(S)에서 정답 레이블에 따른 소정의 교차점(i)에 대한 형세값이다. 형세값에 대한 설명은 후술하는 도 11의 설명에 따른다.

는 현재 바둑판 상태(S)에서 소정의 교차점(i)을 형세 판단 신경망(410)에 입력하였을 때에 출력되는 출력 데이터이다. 형세 판단 모델은 형세 판단 손실(

)이 최소화되도록 경사 하강법(gradient-descent)과 역전파(backpropagation)을 이용하여 형세 판단 신경망(410) 내의 가중치와 바이어스 값들을 조절하여 형세 판단 신경망(410)를 학습시킬 수 있다.B is the total number of intersections of the checkerboard. In the checkerboard, 19 horizontal and 19 vertical lines intersect each other, and 361 intersections are arranged. It is not limited thereto, and when the checkerboard has 9 horizontal and 9 vertical lines, 81 intersections may be arranged.

is a configuration value for a predetermined intersection point (i) according to the correct answer label in the current checkerboard state (S). The description of the configuration value follows the description of FIG. 11, which will be described later.

is output data output when a predetermined intersection (i) is input to the situation determination neural network 410 in the current checkerboard state (S). The situation judgment model is based on the situation judgment loss (

) can be minimized by adjusting the weights and bias values in the situation determination neural network 410 using gradient-descent and backpropagation to train the situation determination neural network 410 .

The situation determination neural network 410 may be configured in a neural network structure. As an example, the situation determination neural network 420 may be composed of 19 residual blocks. Referring to FIG. 8 , one residual block includes 256 3X3 convolutional layers, batch normalization layers, Relu activation function layers, 256 3X3 convolutional layers, batch normalization layers, skip connections, Relu activation function layers may be arranged in order. The batch normalization layer is to prevent covariate shift, a phenomenon in which the distribution of the input of the current layer is changed due to the parameter change of the previous layer during learning. Skip connection prevents the performance of the neural network from decreasing even if the block layer becomes thicker, and makes the block layer thicker to increase the overall neural network performance. The skip connection may be in a form in which the first input data of the residual block is combined with the output of the second batch normalization layer and input to the second Relu activation function layer.

9 and 10 are diagrams for explaining the first and second pre-processing steps for generating a correct answer label used for learning the situation judgment model of the present invention, and FIG. 11 is a diagram for learning the situation judgment model of the present invention It is a diagram for explaining the third pre-processing step for generating the correct answer label used for

The correct answer label generator 430 may generate a correct answer label used for learning so that the situation determination neural network 410 can accurately determine the situation.

More specifically, the correct answer label generating unit 430 receives the checkerboard state (S) that is the basis of the input data as an input, and performs a first preprocessing of ending the current checkerboard state (S) to the first preprocessing state (P1) ) can be created. The first pre-processing, ending, is a process of finishing the game by making a predetermined start so that the boundary of the house becomes clear before calculating the house. For example, referring to FIG. 9 , the correct answer label generating unit 430 generates the first pre-processing state P1 of FIG. 9(b) by ending the current checkerboard state S of FIG. can

The correct answer label generator 430 may generate the second pre-processing state P2 by performing a second pre-processing that is disposed within the house boundary in the first pre-processing state P1 and removes stones unnecessary for house classification. For example, a stone that is placed within the boundary of a house and is not necessary to classify a house may be a quarry stone. I explained earlier that the stone was a stone that died because the other stone was placed in the house, and no matter how it was placed, it could only be caught. In addition, stones that are placed within the boundary of the house and are not necessary to classify the house may be own stones placed in the house. As an example, referring to FIG. 9 , the correct answer label generating unit 430 removes stones unnecessary for house classification in the first pre-processing state P1 of FIG. (P2) can be created.

As another example, referring to FIG. 10 , the correct answer label generating unit 430 starts the red x as shown in FIG. can do. The correct answer label generating unit 430 removes the green x to remove the rubble marked with a blue x in FIG. Pre-processing can be performed.

The correct answer label generator 430 may perform a third preprocessing of changing each intersection point to a shape value (g, where g is an integer) indicated from -1 to +1 in the second preprocessing state P2 . That is, in the third pre-processing, the correct answer label generating unit 430 changes the second pre-processing state P2, which is an image feature, to a third pre-processing state P3, which is a numerical feature. For example, in the second pre-processing state P2 , if my stone is placed at the intersection, it may correspond to 0, if it is my house area, +1, if the opponent stone is disposed, it may correspond to 0, and if it is the opponent's house area, it may correspond to -1. In this case, the situation determination neural network 410 may be trained to distinguish a house, a stone, and a stone when determining the situation. As another example, in the second pre-processing state P2 , if my stone is placed at the intersection, it may correspond to 0, if it is my house area, +1, if the opponent stone is placed, 0, if it is the opponent's house area, -1, and if it is big or common, it may correspond to 0. In another example, the situation determination neural network 410 may be trained to distinguish a big or a common match when determining a situation. For example, referring to FIG. 11 , the correct answer label generating unit 430 characterizes the second pre-processing state P2 of FIG. 11 (a) as the third pre-processing state P3 of FIG. 11 (b). can be changed

)로 이용될 수 있다. The third pre-processing state (P3) becomes the correct answer label of the situation determination in the checkerboard state (S), and the target data (

) can be used as

12 is a view for explaining a situation determination result of the situation determination model of the present invention.

The learned layout determination model may provide a configuration value for all intersections of the checkerboard when the checkerboard state is input. That is, it is possible to provide an integer value of -1 to +1, which is a configuration value, for 361 points of the checkerboard intersection.

Referring to FIG. 12 , the condition determination model server 400 may determine the condition by using the condition value provided by the condition determination model, a predetermined threshold value, and the presence or absence of stones. For example, the situation determination model server 400 is a place without stones, and if the situation value exceeds the first threshold value, it is determined as a place with a high probability of becoming my home, and if the value is close to +1, it can be determined as my home area. have. The situation determination model server 400 may display in the form of a square of the same color as my stone, which increases as the probability that it is my house increases. The situation determination model server 400 may determine that there is no stone, and if the situation value is less than or equal to the second threshold value, it is highly likely to become the other house, and if the value is close to -1, it may be determined as the other house area. The situation determination model server 400 may display a square shape of the same color as the counterpart stone, which increases as the probability of the counterpart's house increases. The situation determination model server 400 is a place where there are no stones, and if the situation value is within the third threshold range or a value close to 0, it may be determined to be common or big. The situation determination model server 400 may display an X when it is determined as a common match or a big match. The shape determination model server 400 is where the stone is, and if the shape value is within the range of the third threshold value or close to 0, it may determine the stone as my stone or the opponent's stone. The situation judgment model server 400 may not display anything when it is judged to be a common match or a big game. The situation determination model server 400 is a place where the stone is located, and if the situation value exceeds the first threshold value, it is determined as a place with a high probability of becoming a stone of the opponent stone, and if the value is close to +1, it can be determined as a stone of the opponent stone. have. The situation determination model server 400 may display in the form of a square of the same color as the inner stone, which increases as the probability that the other stone is a non-stone. The situation determination model server 400 is a place where a stone is located, and if the situation value is less than the second threshold, it is determined as a place with a high probability of becoming my stone, and if the value is close to -1, it can be determined as a stone of the other stone. have. The situation determination model server 400 may display a square shape of the same color as that of the opponent stone, which increases as the probability that the opponent stone is a quarry stone increases.

In addition, the situation determination model server 300 may display the result of the calculation in the current checkerboard state by using the situation determination criterion determined at each intersection.

In addition, the situation determination model server 300 may generate information on the amount of change in water collection and information on the common drainage according to the state of the checkerboard. For example, the condition determination model server 300 may generate information on the amount of change in water collection by using the condition determination result of the checkerboard state according to the previous start and the condition determination result of the current checkerboard condition. In addition, the situation determination model server 300 may generate common multiple information using the result of determining the situation of the checkerboard state.

Therefore, the deep learning-based Go game service device according to the embodiment may determine the Go situation using a deep learning neural network. In addition, the deep learning-based Go game service apparatus according to the embodiment can accurately determine the situation of Go by accurately classifying a house, a private stone, a stone, a ball, and a big according to the Go rules. In addition, the deep learning-based Go game service device according to the embodiment may accurately determine the situation of Go by predicting a house, a stone, a stone, a ball, and a big according to the Go rules. In addition, the deep learning-based Go game service device according to the embodiment can quickly determine the situation in the Go game.

13 is a view comparing the situation determination result of the situation determination model of the present invention and the situation determination result by the deep learning model according to the prior art, and FIG. 14 is the situation determination result of the situation determination model of the present invention and the prior art It is a state of comparing the situation judgment result by the deep learning model, and FIG. 15 is a state comparing the situation judgment result of the situation judgment model of the present invention with the situation judgment result by the deep learning model according to the prior art.

Referring to FIG. 13 , the situation determination model of the present invention determines the situation by classifying houses, stones, and rocks at each intersection, as in region B of FIG. 13 (a). However, the situation judgment model by the deep learning model according to the prior art cannot distinguish a house, a stone, and a gravel at the intersection of the regions corresponding to those of FIG. 13(b) in FIG. 13(a).

Similarly, referring to FIG. 14 , the situation determination model of the present invention determines the situation by classifying houses, stones, and rocks at each intersection, as in region C of FIG. 14 (a). However, the situation judgment model by the deep learning model according to the prior art cannot distinguish a house, a stone, and a gravel with respect to the intersection of the regions corresponding to those of FIG. 14(b) to FIG. 13(a).

Referring to FIG. 15 , the situation judgment model of the present invention properly recognizes a bag as in the region D of FIG. 15 (a). However, the situation judgment model by the deep learning model according to the prior art does not distinguish the bag in the area corresponding to FIG. 15 (b) in FIG. 15 (a).

16 is an exemplary diagram of a signal flow in a deep learning-based Go game service system according to an embodiment of the present invention.

Referring to FIG. 16, the start model server 300 is an artificial intelligence computer that learns by itself according to the rules of Go so as to be able to perform the initiation of Go so that it can win the game in its own turn to train the initiation model, which is a deep learning model. can be (s11). Go server 22 may transmit a plurality of notations to the situation determination model server 400 . The situation determination model server 400 may generate a training data set. First, the situation determination model server 400 may extract input features from the checkerboard state of a plurality of notations (S13). The situation determination model server 400 may generate a correct answer label using the checkerboard state from which the input features are extracted (S14). The situation determination model server 400 may train the situation determination model using a training data set using the input characteristics as input data and the correct answer label as the target data (S15). The terminal 100 may request the Go game from the Go server 200 against the artificial intelligence computer or against another user terminal (S16). The Go server 200 may request the start model server 300 when the terminal 100 requests a Go game against the artificial intelligence computer (S17). Go server 200 proceeds a Go game, and the terminal 100 and the start model server 300 may perform a start in their turn (S18 to S20). During the game, the terminal 100 may request the Go server 200 to determine the situation (S21). The Go server 200 may request the condition determination model server 400 to determine the current condition of the Go board (S22). The situation determination model server 400 extracts the input features of the current checkerboard state, the deep learning model, the situation determination model, generates a situation value using the input features, and performs the situation determination using the checkerboard state and the situation value. can be (S23). The situation determination model server 400 may provide the situation determination result to the Go server 200 (S24). The Go server 200 may provide the result of determining the situation to the terminal 100 (S25).

17 is a method for determining a situation in the deep learning-based Go game service method according to an embodiment of the present invention, and FIG. 18 is a method for pre-processing training data for generating a correct answer label among the method for determining the situation of FIG. 17 .

Referring to FIG. 17 , the deep learning-based Go game service method according to an embodiment of the present invention may include the step (S100) of the situation determination model server receiving a plurality of notations from the Go server.

The deep learning-based Go game service method according to an embodiment of the present invention may include the step (S200) of extracting input features from the checkerboard state of a plurality of notations by the input feature extraction unit among the posture determination models of the posture determination model server. . The method of extracting the input feature follows the description of FIG. 7 .

The deep learning-based Go game service method according to an embodiment of the present invention may include generating a correct answer label based on a checkerboard state from which the correct answer label generator extracts input features from the situation determination model (S300). As an example, referring to FIG. 18 , the correct answer label generating step ( S300 ) may include a first pre-processing step ( S301 ) in which the correct answer label generating unit ends the current checkerboard state. The first pre-processing step ( S301 ) follows the description of FIGS. 9 to 10 . The correct answer label generating step (S300) may include a second pre-processing step (S302) of the correct answer label generating unit removing unnecessary stones from the first pre-processed checkerboard state. The second pre-processing step ( S302 ) follows the description of FIGS. 9 to 10 . The correct answer label generating step (S300) may include a third pre-processing step (S303) of the correct answer label generating unit changing each intersection of the second pre-processed checkerboard state to a shape value. The third pre-processing step ( S303 ) follows the description of FIG. 11 . The correct answer label generation step (S300) may include a step (S303) of providing the third preprocessing state as the correct answer label as target data to the situation determination neural network. The step of providing the target data ( S301 ) follows the description of FIGS. 7 and 11 .

The deep learning-based Go game service method according to an embodiment of the present invention may include training a situation determination neural network of a situation determination model using a training data set (S400). A method of training (learning) a situation determination neural network follows the description of FIG. 7 .

The deep learning-based Go game service method according to an embodiment of the present invention includes a step (S500) of completing the training of the situation determination neural network to build a situation determination model. As an example, the training of the situation determination neural network may be completed when the situation determination loss of FIG. 7 is less than or equal to a predetermined value.

The deep learning-based Go game service method according to an embodiment of the present invention may include a step (S600) of inputting a current Go board state into a situation determination model in response to a request for determination of a condition of a terminal.

The deep learning-based Go game service method according to an embodiment of the present invention may include a step (S700) of determining the current state of the Go board to which the situation determination model is input. The step ( S700 ) of determining the shape may follow the description in which the shape determination model described in FIG. 12 generates the shape value of the current checkerboard state.

The deep learning-based Go game service method according to an embodiment of the present invention may include the step (S800) of the situation determination model server outputting the situation determination result. The step of outputting the situation determination result ( S800 ) may follow the description in which the situation determination model server provides the situation determination result by using the situation value, the state of the checkerboard, and a predetermined threshold value described in FIG. 12 .

Therefore, the deep learning-based Go game service method according to the embodiment may determine the Go situation using a deep learning neural network. In addition, the deep learning-based Go game service method according to the embodiment can accurately determine the situation of Go by accurately classifying a house, a private stone, a stone, a ball, and a big according to the Go rules. In addition, the deep learning-based Go game service method according to the embodiment can accurately determine the situation of Go by predicting a house, a stone, a stone, a ball, and a big according to the Go rules. In addition, the deep learning-based Go game service method according to the embodiment can quickly determine the situation in the Go game.

<Time management model server according to an embodiment>

19 is a diagram for explaining a time management unit of a time management model server according to an embodiment of the present invention, and FIG. 20 is a diagram for explaining distributed calculation of the time management unit according to an embodiment of the present invention.

Deep learning-based Go game service according to an embodiment of the present invention, the time management model server 500 determines the start preparation time, which is one of the time management information, and the start model is preset or based on the set start preparation time. can get started The time management model server 500 increases the start preparation time to find a good number in order to place a better number for the artificial intelligence computer to undertake when the game is close in the game between the user and the artificial intelligence computer or the artificial intelligence computer. can do it

), 방문 횟수(N) 또는 가치값(V)을 수신하고, 착수 모델 서버(300)로 착수 준비 시간(PP)을 제공할 수 있다. 일 예로, 착수 준비 시간(PP)은 평균 착수 준비 시간, 제1 착수 준비 시간 또는 제2 착수 준비 시간을 포함할 수 있다. 제1 착수 준비 시간은 평균 착수 준비 시간보다 길고, 제2 착수 준비 시간은 제1 착수 준비 시간보다 길 수 있다. 시간 관리 모델 서버(500)는 시간 관리부(510)를 포함할 수 있다. 시간 관리부(510)는 탐색 확률값(

) 및 방문 횟수(N)에 기초하여 착수 준비 시간(PP)을 결정할 수 있다. 시간 관리부(510)는 탐색 확률값(

), 방문 횟수(N)을 이용하여 분산을 산출하고, 분산과 임계 분산 값을 비교하여 착수 준비 시간을 결정할 수 있다. 즉, 착수 후보점에 대한 분산이 낮을수록 착수 후보점이 가장 좋은 수가 아닐 가능성이 높다는 것이고 이러한 정보에 비추어 현재 경기가 막상막하일 가능성이 높을 수 있다. 이에, 시간 관리부는 분산이 임계 분산 값보다 낮을 경우 착수 준비 시간을 더 길게 하여 착수 모델이 더 좋은 수를 착도록 할 수 있다. 분산은 수학식 3과 수학식 4를 이용하여 구할 수 있다.More specifically, referring to FIG. 19 , the time management model server 500 is a search probability value (

), the number of visits (N) or a value value (V) may be received, and may provide a set-up preparation time (PP) to the set-off model server 300 . As an example, the start preparation time (PP) may include an average start preparation time, a first set off preparation time, or a second set off preparation time. The first set-out preparation time may be longer than the average set-out preparation time, and the second set-out preparation time may be longer than the first set off preparation time. The time management model server 500 may include a time management unit 510 . The time management unit 510 is a search probability value (

) and the number of visits (N) to determine the start preparation time (PP). The time management unit 510 is a search probability value (

), calculate the variance using the number of visits (N), and compare the variance with the critical variance value to determine the start preparation time. In other words, the lower the variance of the starting candidate points, the higher the possibility that the starting candidate points are not the best. Accordingly, when the variance is lower than the threshold variance value, the time management unit may lengthen the start preparation time so that the set off model can wear a better number. The variance can be calculated using Equations 3 and 4.

(Equation 3)

는 각 교차점에 대한 방문 횟수이고,

는 각 교차점에 대한 탐색 확률값이고,

는 평균 방문 횟수이다.In Equation 3, n is the number of intersections,

is the number of visits to each intersection,

is the search probability value for each intersection,

is the average number of visits.

(Equation 4)

In Equation 4, Var is variance.

The time management unit 510 may determine the start preparation time as the first set off preparation time if the variance (Var) is lower than the threshold variance value, and if the variance (Var) is not lower than the threshold variance value, the start preparation time as the average set off preparation time can be decided

For example, referring to FIG. 20 , the threshold variance value may be 0.05. In the case of FIG. 20A , the time management unit 510 may calculate the variance Var as 0.2109. In this case, since the variance (Var) is higher than the threshold variance value, the time management unit 510 may determine the start preparation time as the average start preparation time. In the case of FIG. 20B , the time management unit 510 may calculate the variance Var as 0.0145. In this case, the time management unit 510 may determine the start preparation time as the first set off preparation time because the variance (Var) is lower than the threshold variance value. Accordingly, the launch model may select better candidate points for starting by performing MCTS simulations for a longer period of time or a greater number of times.

In addition, the time management unit 510 may determine the start preparation time (PP) based on the value (V). More specifically, the time management unit 510 may determine the set off preparation time (PP) as the second set off preparation time when the value (V) is less than or equal to the threshold value after determining the first set off preparation time. For example, the threshold value may be 50.0%. That is, since the low value of the starting candidate point means that it is highly likely to lose the current game, the starting model performs MCTS simulation for a longer time or more times so that the time management unit 510 finds a better starting candidate point. is to make it possible

Therefore, the deep learning-based Go game service device according to the embodiment may manage the Go game time. In addition, the deep learning-based Go game service device according to the embodiment may change the start preparation time in an important phase.

21 is an exemplary diagram of a signal flow in the Go game service system of the time management model server according to an embodiment of the present invention.

), 방문 횟수(N), 가치값(V)를 시간 관리 모델 서버(500)에 전송할 수 있다(S2102). 시간 관리 모델 서버(500)는 수신한 탐색 확률값(

), 방문 횟수(N), 가치값(V)에 기초하여 착수 준비 시간을 결정할 수 있다(S2103). 시간 관리 모델 서버(500)는 결정된 착수 준비 시간을 착수 모델 서버(300)에 전송할 수 있다(S2104). 착수 모델 서버(300)는 수신한 착수 준비 시간 또는 기설정된 착수 준비 시간에 기초하여 착수를 수행할 수 있다(S2105). 착수 모델 서버(300)는 수신한 착수 준비 시간과 기설정된 착수 준비 시간 중 수신한 착수 준비 시간에 우선하여 착수를 수행할 수 있다. Referring to FIG. 21 , the start model server 300 may perform MCTS simulation to find a start point for winning in a game during a game (S2101). Start model server 300 is a search probability value generated as a result of MCTS simulation (

), the number of visits (N), and the value value (V) may be transmitted to the time management model server 500 (S2102). The time management model server 500 receives the search probability value (

), the number of visits (N), it is possible to determine the start preparation time based on the value (V) (S2103). The time management model server 500 may transmit the determined start preparation time to the start model server 300 (S2104). Initiation model server 300 may perform an initiation based on the received start preparation time or a preset start preparation time (S2105). Initiation model server 300 may perform the initiation in preference to the received start preparation time among the received start preparation time and the preset start preparation time.

22 is a method for determining a start preparation time among the deep learning-based Go game service methods according to an embodiment of the present invention.

), 방문 횟수(N) 또는 가치값(V)을 수신하는 단계(S2201)을 포함할 수 있다. Referring to Figure 22, in the deep learning-based Go game service method according to an embodiment of the present invention, the time management model server 500 is a search probability value (

), receiving the number of visits (N) or a value (V) (S2201).

), 방문 횟수(N)에 기초하여 분산을 산출하는 단계(S2202)를 포함할 수 있다. 분산을 산출하는 방법은 도 19 및 도 20의 설명을 따른다.In the deep learning-based Go game service method according to an embodiment of the present invention, the time management unit time management model server 500 among the time management model servers 500 searches for probability values (

), calculating a variance based on the number of visits (N) (S2202). The method of calculating the variance follows the description of FIGS. 19 and 20 .

The deep learning-based Go game service method according to an embodiment of the present invention may include a step (S2203) of determining whether the variance calculated by the time management unit among the time management model servers 500 is less than a threshold variance value (S2203).

In the deep learning-based Go game service method according to an embodiment of the present invention, if the time management unit of the time management model server 500 is not less than the threshold variance value, the step of determining the start preparation time as the average start preparation time (S2204) ) may be included. A method of determining the start preparation time as the average start preparation time follows the description of FIGS. 19 and 20 .

In the deep learning-based Go game service method according to an embodiment of the present invention, if the time management unit of the time management model server 500 is less than a threshold variance value, the step of determining the start preparation time as the first start preparation time (S2205) ) may be included. A method of determining the set-out preparation time as the first set-off preparation time follows the description of FIGS. 19 and 20 .

Deep learning-based Go game service method according to an embodiment of the present invention is to determine whether the value is less than or equal to a threshold value after the time management unit determines the start preparation time as the first start preparation time of the time management model server 500 It may include step S2206. A method of determining whether the value is equal to or less than the threshold value follows the description of FIGS. 19 and 20 . If the value is not less than the threshold value, the time management unit may determine the first start preparation time as the start preparation time.

The deep learning-based Go game service method according to an embodiment of the present invention comprises the steps of determining the start preparation time as the second start preparation time if the time management value of the time management model server 500 is less than or equal to the threshold value ( S2207) may be included. A method of determining the set-off preparation time as the second set-off preparation time follows the description of FIGS. 19 and 20 .

The deep learning-based Go game service method according to an embodiment of the present invention may include a step (S2208) of transmitting the determined start preparation time to the time management unit of the time management model server 500.

Therefore, the deep learning-based Go game service method according to an embodiment can manage the Go game time. In addition, the deep learning-based Go game service method according to an embodiment may change the start preparation time in an important phase.

<Time management model server according to another embodiment>

23 is a diagram for explaining a simulation number prediction unit and a simulation number adjustment model of a time management model server according to another embodiment of the present invention, and FIG. 24 is a deep learning-based Go game service according to another embodiment of the present invention. Among the methods, it is a method of determining the number of MCTS simulations.

Referring to Figure 23, the deep learning-based Go game service according to another embodiment of the present invention determines the number of MCTS simulations (C: hereinafter, the number of simulations) performed by the time management model server 500 by the start model described above. can do it In addition, the deep learning-based Go game service according to another embodiment of the present invention may allow the start model to determine the next start point by performing MCTS based on the determined number of simulations (C).

In detail, the deep learning-based Go game service according to another embodiment of the present invention is a deep learning model of the time management model server 500, a simulation number prediction unit (551: hereinafter, number prediction unit) and/or a simulation number adjustment model (552: hereinafter, the number of times adjustment model) based on the number of simulations (C) determined on the basis of the above-described set off model may perform MCTS to determine the next set point.

Here, the time management model server 500 may include a number prediction unit 551 . The number prediction unit 551 is based on the plurality of checkerboard states (S), the default number of simulations (D: hereinafter, the default number), the number of visits (N) and/or the value value (V), a plurality of checkerboard states-optimal simulations Count (SC) data can be generated.

In addition, the time management model server 500 may include a frequency adjustment model 552 . The number of times adjustment model 552 may be learned by using a plurality of checkerboard states-optimal simulation times SC provided by the number prediction unit 551 as a training data set. In addition, the frequency adjustment model 552 receives a predetermined checkerboard state (S) (in the embodiment, the current checkerboard state (S)) as an input through the above learning, and the number of simulations (C) according to the checkerboard state (S) can be output.

In more detail, the deep learning-based Go game service according to another embodiment of the present invention uses the number prediction unit 551 of the time management model server 500 to optimize the number of simulations (C) for a predetermined Go board state (S). ) can be determined.

In addition, the deep learning-based Go game service according to another embodiment may train the number adjustment model 552 by using at least one or more Go board state-optimal simulation times (SC) obtained as above as a training data set.

In addition, the deep learning-based Go game service according to another embodiment inputs the current Go board state (S) as input data to the learned number adjustment model 552, and in the learned number adjustment model 552, the current It is possible to output the number of simulations (C) optimized for the checkerboard state (S) as output data.

In addition, the deep learning-based Go game service according to another embodiment may allow the above-described start model to perform MCTS simulation based on the outputted number of simulations (C).

Thus, the deep learning-based Go game service according to another embodiment may allow the start model to determine the next start point by performing MCTS based on the number of simulations (C) determined as above.

Specifically, referring to FIG. 24 , the deep learning-based Go game service method according to another embodiment of the present invention includes the step (S2301) of the time management model server 500 receiving a predetermined Go board state (S) can do.

That is, the time management model server 500 may receive a plurality of notations from the Go server 200 . Each notation of a plurality of notations may include a respective checkerboard state (S) according to the starting order.

25 is an example of a state of deriving the value (V) and the number of visits (N) in each of the first MCTS and the second MCTS for the checkerboard state (S) according to another embodiment of the present invention.

In addition, referring to FIG. 25 , the deep learning-based Go game service method according to another embodiment of the present invention includes the steps of performing the first MCTS on the received Go board state (S) by the time management model server 500 ( S2302) may be included.

In detail, the time management model server 500 MCTS by a preset number of times (D) (eg, 800 times, etc.) can make it work In more detail, the time management model server 500 may provide the received checkerboard state (S) to the start model. At this time, the start model provided with the checkerboard state (S) may perform MCTS for the checkerboard state (S) as many times as a predetermined default number (D).

In addition, the deep learning-based Go game service method according to another embodiment of the present invention is a time management model server 500 based on the first MCTS performed as above the first number of visits (N1) and / or the first value It may include a step (S2303) of storing (V1).

That is, the time management model server 500 is a plurality of start candidates obtained as the output of the MCTS performed by the default number of times (D) for the checkerboard state (S) from the start model (that is, a plurality of nodes in the MCTS) The first number of visits N1 and/or the first value V1 may be received and stored.

In addition, the deep learning-based Go game service method according to another embodiment of the present invention may include the step (S2304) of the time management model server 500 performing the second MCTS for the received Go board state (S). can

In detail, the time management model server 500 adds a predetermined number of times (D) (eg, 800 times, etc.) to the default number of times (D) based on the received checkerboard state (S) in conjunction with the start model of the start model server 300 At least one second MCTS may be performed while increasing the number of times (eg, 10 times, etc.).

Specifically, the time management model server 500 increases the default number of times (D) (eg, 800 times, etc.) by a predetermined additional number (eg, 10 times, etc.) unit by interworking with the start model. Based on the number of times (eg, 810 times, 820 times, 830 times, etc.), the second MCTS may be performed at least once for the received checkerboard state (S).

In addition, in the deep learning-based Go game service method according to another embodiment of the present invention, the time management model server 500 performs at least one second number of visits (N2) and/or the second based on the second MCTS. It may include the step of storing the value value (V2) (S2305).

In detail, the time management model server 500 is at least one start candidate number (ie, MCTS) obtained as an output of MCTS performed at least once or more based on the number of additional units for the checkerboard state (S) from the start model. My plurality of nodes) may receive and store the second number of visits (N2) and/or the second value (V2) for each. That is, the second number of visits (N2) may include at least one or more number of visits (N) data for each number of starting candidates, and the second value value (V2) is at least one value value (V) for each number of starting candidates. ) may contain data.

At this time, the time management model server 500 may stop the repeated second MCTS as the number of simulations (C) is determined below.

In addition, the deep learning-based Go game service method according to another embodiment of the present invention includes the step of determining, by the time management model server 500, the optimal number of simulations based on the number of visits (N) and/or the value (V) (S2306) may be included.

In detail, the time management model server 500 operates the number prediction unit 551 to operate the first number of visits (N1) and the second number of visits (N2) and/or the first value for a predetermined checkerboard state (S). Based on (V1) and the second value (V2), it is possible to determine the number of simulations optimized for the checkerboard state (S) (hereinafter, the optimal number of simulations). In this case, the optimal number of simulations may be the number of simulations including the number of simulations (C) that take the minimum time within a critical point in which the next starting point is not changed during MCTS simulation. In an embodiment, the optimal number of simulations may be the number of simulations (C) that takes the minimum start preparation time (PP) within a critical point in which the next start point is not changed during MCTS simulation.

In general, the MCTS performs several simulations to select the number of start candidates having the largest number of visits (N) among a plurality of start candidates (ie, a plurality of nodes in the MCTS) to determine the next start point. At this time, even if the number of simulations (C) increases, it may not be necessary to increase the number of simulations (C) if the number of start-up candidates having the next start point, ie, the largest number of visits (N), does not change.

Thus, the time management model server 500 according to another embodiment determines whether the next starting point (ie, the selected node) is changed according to the increase in the number of simulations (C) of performing MCTS for a predetermined checkerboard state (S), According to the determination result, the optimal number of simulations for the checkerboard state (S) can be predicted.

In detail, the time management model server 500 may determine whether to change the next starting point (ie, the selected node) according to the increase in the number of simulations (C) for a predetermined checkerboard state (S).

Specifically, the time management model server 500 may determine whether to change the next start point based on 1) the number of visits (N) per the number of start candidates.

26 is an example of a diagram for explaining a method of determining the optimal number of simulations based on the number of visits (N) according to another embodiment of the present invention.

Referring to FIG. 26 , the time management model server 500 includes the first MCTS, that is, the first number of visits N1 for each number of candidates to be launched by performing simulation as much as the default number of times (D), and the second MCTS, that is, at least The rate of change between the first number of visits (N1) and the second number of visits (N2) based on the second number of visits (N2) for each at least one or more starting candidates obtained by performing a simulation based on each of one or more additional unit times (gradient) can be calculated.

In an embodiment, the time management model server 500 is a visit based on the increase in the number of simulations based on the difference between the default number (D) and the number of additional units, and the difference between the first number of visits (N1) and the second number of visits (N2). The change rate may be calculated using the increase in the number of times. In detail, the time management model server 500 may obtain the change rate by calculating the increase in the number of visits (ie, increase in the number of visits/increase in the number of simulations) compared to the increase in the number of simulations. However, this is only an example, and the method of calculating the change rate itself is not limited, and various embodiments may be possible.

Referring further to Figure 26, for example, the time management model server 500 is the first simulation number (C1), that is, the default number of times (D) is 500 with respect to the first number of start candidates, and the second number of simulations (C2) That is, when the predetermined number of additional units is 1000, the first number of visits (N1) is 400, and the second number of visits (N2) is 800, the rate of change (that is, the increase in the number of visits (N2-N1)/the number of simulations) The increase (C2-C1)) can be calculated as 0.8.

In addition, the time management model server 500 stops the increase in the number of simulations (C) when the calculated change rate meets a predetermined criterion (eg, a predetermined constant value (eg, 0.5) or more) stop the second MCTS).

That is, the time management model server 500 determines that the change rate (in the embodiment, increase in the number of visits (N2-N1)/increase in the number of simulations (C2-C1)) meets a predetermined criterion (eg, greater than or equal to a predetermined rate of change). , even if the number of simulations (C) for the predetermined checkerboard state (S) increases, it can be determined that the next starting point (ie, the selected node) is unchanged. In addition, the time management model server 500 may stop increasing the number of simulations (C) (ie, stop the second MCTS) when it is determined that the next starting point is unchanged. In addition, the time management model server 500 may determine the minimum number of simulations when calculating the change rate (ie, the number of simulations C when the first visit number N1 is satisfied) as the optimal number of simulations.

Alternatively, the time management model server 500 may determine whether to change the next start point based on 1) the number of visits (N) ratio for each number of start candidates.

27 is an example of a diagram for explaining a method of determining the optimal number of simulations based on a ratio of the number of visits (N) according to another embodiment of the present invention.

In detail, referring to FIG. 27 , the time management model server 500 visits a plurality of initiation candidates (in the embodiment, a plurality of child nodes existing in a predetermined first layer in the MCTS) for each first visit. Based on the ratio between the number of times (N1), it is possible to calculate a first ratio for the plurality of start candidates.

Referring further to FIG. 27 , for example, the plurality of start-up candidates includes a first start-up candidate number and a second start-up candidate number, and the first number of visits (N1) of the first start-up candidates is 400, and the When the number of first visits (N1) of the second number of start-up candidates is 100, the first ratio may be 'the number of first start-up candidates: 0.8 (400), the second number of start-up candidates: 0.2 (100)'.

In addition, the time management model server 500 may sort the plurality of start-up candidates in the ratio order based on the first ratio calculated as above.

For example, the time management model server 500 may be sorted by 'first priority: the number of first start candidates, second priority: the second number of start candidates' based on the first ratio in the above-described example.

In addition, the time management model server 500 may calculate a second ratio for the plurality of start-up candidates based on the ratio between the second number of visits N2 for each of the plurality of start-up candidates.

Referring further to FIG. 27 , for example, the number of starting candidates includes a first starting candidate number and a second starting candidate number, and the second number of visits (N2) of the first starting candidate number is 800, and the When the second number of visits (N2) of the second number of start-up candidates is 200, the second ratio may be 'the number of first start candidates: 0.8 (800), the second number of start candidates: 0.2 (200)'.

In addition, the time management model server 500 may sort the plurality of start-up candidates in the ratio order based on the calculated second ratio.

Illustratively, the time management model server 500 may be sorted by 'first priority: the number of first start candidates, second priority: the second number of start candidates' based on the second ratio in the above-described example.

At this time, the time management model server 500 may repeat the above-described second MCTS, and if the ranking according to the plurality of second ratios is unchanged by a predetermined number of times or more compared to the ranking according to the first ratio, the Even if the number of simulations (C) for a predetermined checkerboard state (S) increases, it can be determined that the next starting point (ie, the selected node) is unchanged.

For example, the time management model server 500 has a ranking according to the first ratio 'first priority: the number of first start candidates, second priority: the second number of start candidates', a predetermined number of times (eg, 10 times, etc.) ) or more, if the ranking according to the second ratio is maintained as 'first priority: the number of first start candidates, second priority: the second number of start candidates', the number of simulations (C) for the predetermined checkerboard state (S) increases However, it can be determined that the next starting point (ie, the selected node) is unchanged.

In addition, the time management model server 500 may stop increasing the number of simulations (C) (ie, stop the second MCTS) when it is determined that the next starting point is unchanged as above. In addition, the time management model server 500 may determine the minimum number of simulations in the ranking comparison (ie, the number of simulations when the first number of visits N1 is satisfied (C)) as the optimal number of simulations.

Therefore, the deep learning-based Go game service apparatus according to another embodiment increases the number of simulations (C) for a predetermined Go board state (S) while grasping the change in the number of visits (N) accordingly, and based on this It is possible to determine the number of simulations (C) optimized for the corresponding checkerboard state (S).

On the other hand, in general, when multiple simulations are performed in MCTS, the value (V) (ie, win rate) of any one of a plurality of start candidates (i.e., a plurality of nodes in MCTS) increases in a biased manner. If there is a tendency to do so, it may not be necessary to increase the number of simulations (C) anymore.

Thus, in another embodiment, the time management model server 500 may predict the optimal number of simulations for the checkerboard state S based on the first value V1 and the second value V2.

28 is an example of a view for explaining a method of determining the optimal number of simulations based on the value V according to another embodiment of the present invention.

In detail, referring to FIG. 28 , the time management model server 500 determines that a gradient between the first value V1 and at least one second value V2 is determined by a predetermined criterion (eg, a predetermined value). If the constant value or more) is satisfied, the increase in the number of simulations C may be stopped (ie, the second MCTS is stopped).

At this time, the time management model server 500 increases the simulation number based on the difference between the default number D and the number of additional units, and increases the value value based on the difference between the first value V1 and the second value V2. can be used to calculate the increase/decrease rate. In detail, the time management model server 500 may obtain the increase/decrease rate by calculating the increase in the value value (ie, increase in the value/increase in the number of simulations) compared to the increase in the number of simulations. However, this is only an example, and the method itself for calculating the increase/decrease rate is not limited, and various embodiments may be possible.

For example, referring further to Figure 28, the time management model server 500 is the first simulation number (C1), that is, the default number of times (D) is 500 with respect to the first number of candidates to start, and the second number of simulations (C2) ) that is, when the predetermined number of additional units is 1000, the first value (V1) is 40, and the second value (V2) is 100, the increase/decrease rate (that is, the increase in value (V2-V1)/number of simulations) The increment (C2-C1)) can be calculated as 0.12.

In addition, the time management model server 500 stops the increase in the number of simulations (C) when the calculated increase/decrease rate meets a predetermined criterion (eg, a predetermined constant value (eg, 0.1) or more). 2 MCTS can be stopped).

In addition, the time management model server 500 may determine the minimum number of simulations when calculating the increase/decrease rate (ie, the number of simulations when the first value V1 is satisfied (C)) as the optimal number of simulations.

Therefore, the deep learning-based Go game service apparatus according to another embodiment increases the number of simulations (C) for a predetermined Go board state (S) while grasping the change in the value value (V) accordingly, and based on this It is possible to determine the number of simulations (C) optimized for the corresponding checkerboard state (S).

In addition, the deep learning-based Go game service method according to another embodiment of the present invention includes the step of generating, by the time management model server 500, the state of the Go board-optimal number of simulations (SC) data based on the optimal number of simulations determined as above. (S2307) may be included.

That is, the time management model server 500 matches a plurality of checkerboard states (S) and the optimal number of simulations for each of the plurality of checkerboard states (S) to obtain a plurality of checkerboard states - the optimal number of simulations (SC). can create

In addition, the deep learning-based Go game service method according to another embodiment of the present invention learns the number adjustment model 552 based on the time management model server 500 generated Go board state-optimal number of simulations (SC) data. It may include a step (S2308) of doing.

In detail, the time management model server 500 is a predetermined checkerboard state (S) (in the embodiment, the current checkerboard state) based on a plurality of checkerboard states generated from the number prediction unit 551-optimal simulation number (SC) data. (S)) as an input and the number of simulations (C) according to the checkerboard state (S) as an output, the number of times adjustment model 552 can be trained. That is, here, the number adjustment model 552 may be learned by using a plurality of checkerboard states-optimal simulation times SC provided by the number prediction unit 551 as a training data set, and the above learning When a predetermined checkerboard state (S) (in the embodiment, the current checkerboard state (S)) is input through , the number of simulations (C) optimized for the checkerboard state (S) can be output.

Accordingly, the deep learning-based Go game service device according to another embodiment may train the deep learning model, that is, the number adjustment model 552 to output the simulation number C optimized for the input Go board state S. .

In addition, in the deep learning-based Go game service method according to another embodiment of the present invention, the time management model server 500 simulates the number of times according to the current Go board state (S) based on the learned number adjustment model 552 as described above. It may include a step (S2309) of determining (C).

That is, the time management model server 500 may determine the number of simulations (C) optimized for the current checkerboard state (S) in the currently ongoing game by operating the learned number adjustment model 552 .

In detail, the time management model server 500 may input the current checkerboard state (S) as input data to the number adjustment model 552 . In addition, the time management model server 500 may obtain, as output data, the number of simulations (C) optimized for the current checkerboard state (S) from the number adjustment model 552 receiving the current checkerboard state (S). . That is, the time management model server 500 uses the number of times adjustment model 552 learned based on a plurality of checkerboard state-optimal simulation times (SC) data for the optimal number of simulations (C) for the current checkerboard state (S). ) can be determined.

Therefore, the deep learning-based Go game service apparatus according to another embodiment may dynamically adjust the number of simulations (C) when performing MCTS so as to be optimized for the Go board state (S). In addition, the deep learning-based Go game service device according to another embodiment may change the number of simulations (C) in an important phase based on the Go board state (S).

Here, the time management model server 500 according to another embodiment 1) the upper limit of the number of simulations (C) and A lower limit may be preset.

In detail, the time management model server 500 may allow the number of simulations C to be determined within a preset upper limit number and a lower limit number. For example, the time management model server 500 may preset the upper limit of the number of simulations (C) to '3200' and the lower limit to '800', which is the default number (D).

In addition, the time management model server 500 may 2) randomly increase the number of simulations (C) with a certain probability. That is, the time management model server 500 probabilistically increases the number of simulations (C) by a predetermined number, thereby improving the reliability of the number of simulations (C) determined through deep learning learning.

In addition, the time management model server 500 may 3) adjust the number of simulations (C) based on the win rate and/or win rate change for the current checkerboard state (S).

In detail, the time management model server 500 is the number of simulations (C) if the winning rate (ie, value value (V)) for the current checkerboard state (S) meets a predetermined criterion (eg, less than or equal to a predetermined number, etc.) can increase That is, when it is determined that the time management model server 500 is inferior based on the current win rate, the number of simulations C may be increased by a predetermined number of times.

Alternatively, the time management model server 500 may increase the number of simulations (C) based on the change in the winning rate for the current checkerboard state (S).

29 is an example of a diagram for explaining a method of adjusting the number of simulations (C) based on a change in winning rate according to another embodiment of the present invention.

In detail, referring to FIG. 29 , the time management model server 500 may sequentially receive, accumulate, and store a plurality of value values (V) obtained for each set-off in the current game in progress from the set-off model server 300 . . In addition, the time management model server 500 may determine the win rate change based on the accumulated value (V).

Specifically, the time management model server 500 may determine the fluctuation trend and/or the fluctuation range of the value value (V) based on the accumulated value value (V). In addition, the time management model server 500 may determine a change in the win rate for the accumulated value value (V) based on the fluctuation trend and/or the fluctuation range of the identified value value (V).

Here, the change trend of the value value V may mean a rate of change of the value value V indicating whether the value value V (ie, win rate) is gradually increasing or decreasing. In detail, the change trend of the value value (V) represents the change state of a plurality of value values (V) from a predetermined start time to the current start time after the start of the game. In more detail, when the rate of change for each of the plurality of value values V is a negative number more than a predetermined number of times, that is, when the value value V decreases a predetermined number of times or more, the value value V gradually decreases and It can be judged that there is a situation. On the other hand, when the rate of change for each of the plurality of value values V is a positive number more than a predetermined number of times, that is, when the value value V increases more than a predetermined number of times, the value value V is gradually increasing. can be judged that In addition, the change trend of the value value (V) may be determined as the slope of the change rate of the value value (V). For example, referring to (a) of FIG. 29 , the change trend of the value value V is the rate of change and/or the slope (ie, the value value V) of the value value V with respect to the recently launched five numbers. is rising or the value V is falling).

Meanwhile, the variation range of the value V may mean an amount (degree of change) of the value V. The range of variation of this value value (V) may be obtained based on the difference between the value value (V) of the predetermined first start time and the value value (V) of the second start time after the first start time. At this time, the fluctuation range of the value value (V) is to determine that the win rate has decreased if the difference value obtained by subtracting the value value (V) at the first start time from the value value (V) at the second start time is negative can In addition, when the difference value having a negative number is greater than or equal to a predetermined value, the fluctuation range of the value value V may determine that the win rate has sharply decreased and detect this as an important phase. For example, referring to Figure 29 (b), the time management model server 500 is the difference between the value value (V) between the recently launched two numbers (eg, 64 numbers and 66 numbers) '35% - 58% = -23%'. In addition, the time management model server 500 may determine whether the fluctuation range of the decrease in the value V is equal to or greater than a predetermined standard based on the calculated difference value.

In addition, the time management model server 500 determines the win rate change for the plurality of value values (V) determined based on the fluctuation trend and/or the fluctuation range of the value value (V) based on a predetermined criterion (in an embodiment, If it is determined that the slope of the rate of change of the value value (V) is negative for more than a predetermined number of times, or the range of fluctuations in the decline of the value value (V) is greater than or equal to a predetermined standard), the number of simulations (C) It can be increased by a predetermined number of times.

That is, when the time management model server 500 determines the change in the win rate based on the value value (V) obtained during the game, if the win rate gradually decreases or falls significantly, it is determined that this is an important phase and the number of simulations (C) It can be increased by a predetermined number of times.

Therefore, the deep learning-based Go game service device according to another embodiment increases the number of simulations (C) in an important phase in which the win rate during a match is lowered to perform more MCTS simulations to determine a better next starting point. can do.

In addition, the time management model server 500 may increase the number of simulations (C) for the state (S) of the checkerboard at the point in time when the win rate has significantly decreased after the game is completed 4).

In detail, the time management model server 500 may detect a point in time at which the value value (V) has fallen by a large fluctuation range after the game is over. In addition, the time management model server 500 may increase the number of simulations (C) for the checkerboard state (S) at the detected time point by a predetermined number of times (eg, more than a preset number, etc.). In addition, the time management model server 500 may determine the number of simulations (C) increased as above as the optimal number of simulations for the checkerboard state (S). In addition, the time management model server 500 may generate the checkerboard state (S) and the determined optimal number of simulations as a pair, the state-optimal number of simulations (SC) data. In addition, the time management model server 500 may train the above-described number adjustment model 552 by using the generated checkerboard state-optimal simulation number (SC) data as training data.

That is, the time management model server 500 learns the number adjustment model 552 by using the training data set that increases the number of simulations (C) at the point in time when the win rate has significantly decreased after the game is completed to learn the number adjustment model 552. The number of simulations (C) in the checkerboard state (S) can be increased.

In addition, the time management model server 500 may adjust the number of simulations (C) based on statistics based on a plurality of games progressed on 5) Go game service.

In detail, the time management model server 500 interlocks with the Go server 200 and/or the start model server 300, and the like, statistics according to the win rate relationship versus the total number of simulations (C) for a plurality of games performed on the Go game service. can be obtained. For example, the time management model server 500 is based on the total number of simulations (C) for each of the plurality of powers and whether or not the result wins data, when the total number of simulations (C) is a predetermined number or more, the win rate is a predetermined percentage You can get statistics that increase more than that.

Also, the time management model server 500 may adjust the number of simulations C calculated by the number adjustment model 552 based on the obtained statistics. In an embodiment, the time management model server 500 may adjust the number of simulations (C) calculated by the number adjustment model 552 to satisfy the number of simulations (C) that satisfy the highest win rate in the statistics. For example, if the time management model server 500 determines that the win rate increases the most when the total number of simulations (C) in one game is greater than or equal to 100,000 times and less than or equal to 200,000 in the statistics, the total number of simulations (C) is To be satisfied, the number of simulations C calculated by the number adjustment model 552 may be increased or decreased.

Therefore, the deep learning-based Go game service apparatus according to another embodiment can predict the optimal number of simulations (C) from a multi-angle point of view.

In addition, the deep learning-based Go game service method according to another embodiment of the present invention may include the step (S2310) of the time management model server 500 providing the number of simulations (C) determined as above as a start-up model. .

30 is a conceptual diagram for explaining a set-up model for performing MCTS based on the number of simulations (C) according to another embodiment of the present invention.

In detail, referring to FIG. 30 , the time management model server 500 may provide the number of simulations C determined based on the number adjustment model 552 as an initiation model. Thus, the time management model server 500 may determine the next starting point by performing the MCTS simulation of the starting model based on the number of simulations (C) determined by the learned number adjustment model 552 . For example, when the number of simulations (C) determined by the number adjustment model 552 is '500 times', the time management model server 500 may provide the number of simulations (C) to the above-described start model. There, the start model can determine the next start point by performing MCTS simulation as many as '500 times' that is the number of simulations (C) received.

Therefore, the deep learning-based Go game service device according to another embodiment may determine the next start by performing MCTS based on the optimal number of simulations (C) for the Go board state (S).

<Time management model server according to another embodiment>

Figure 31 is a diagram for explaining a time management model of the time management model server 500 according to another embodiment of the present invention, Figure 32 is used to generate game time information according to another embodiment of the present invention Fig. 33 is a diagram for explaining the amount of change in water collection used to generate game time information according to another embodiment of the present invention, and Fig. 34 is another embodiment of the present invention It is a diagram for explaining a common multiple used to generate game time information according to .

Referring to FIG. 33 , the deep learning-based Go game service according to an embodiment of the present invention may generate game time information in the current Go board state (S) by using the time management model of the time management model server 500 . . The game time information may include the remaining game length predicted until the end of the game according to the checkerboard state (S). The time management model is a deep learning model of the time management model server 500 , and may include a first time management neural network 520 , a second time management neural network 530 , and a first input feature generator 540 .

The time management model may perform supervised learning to predict the remaining game length, which is game time information of the current checkerboard state (S). More specifically, the time management model server 500 generates a training data set for predicting the remaining game length according to the checkerboard state (S), and using the generated training data set, the time management model is currently in the checkerboard state (S). It can be trained to predict the remaining game length. The time management model server 500 may receive a plurality of notations from the Go server 200 . Each notation of a plurality of notations may include a respective checkerboard state (S) according to the starting order. In addition, each notation of a plurality of ups and downs may include game time information in each checkerboard state (S), in particular, the remaining game length information until the end of the game. Also, the time management model server 500 may receive the situation determination information from the situation determination model server 400 . The situation determination information is situation determination information based on a plurality of notations provided from the Go server 200 to the time management model server 500, and may include information on the amount of change in water collection and common multiple information. In addition, the time management model server 500 may receive a value value from the start model server (300). The value may be a value based on a plurality of notations provided from the Go server 200 to the time management model server 500 .

The first input feature extractor 540 may extract the first input feature IF1 from the checkerboard state S of a plurality of notations and provide the first input feature IF1 as input data for training to the time management first neural network 520 . The first input feature (IF) of the checkerboard state (S) is the position information of the stone for the last eight moves of the black player, the position information of the stone for the last eight moves of the white player, and turn information on whether the current player is black or white It may be an RGB image of 19*19*18 including . For example, the first input feature extractor 540 may have a neural network structure and may include a kind of encoder.

Also, the time management model may provide the second input feature IF2 and the third input feature IF3 as input data for training to the first time management neural network 520 . The second input feature IF2 may be information on the amount of change in water collection according to the checkerboard state S. The third input feature IF3 may be common multiple information according to the checkerboard state S. The time management model may provide the fourth input feature IF4 as input data for training to the second time management neural network 530 . The fourth input feature IF4 may be a value according to the checkerboard state S.

The first time management neural network 520 may receive the first to third input features IF1 to IF3 as inputs and provide an output value to the second time management neural network 530 . The first time management neural network 520 may be configured in a neural network structure. As an example, the first time management neural network 520 may include 20 residual blocks. Referring to FIG. 8 , one residual block includes 256 3X3 convolutional layers, batch normalization layers, Relu activation function layers, 256 3X3 convolutional layers, batch normalization layers, skip connections, Relu activation function layers may be arranged in order. The batch normalization layer is to prevent covariate shift, a phenomenon in which the distribution of the input of the current layer is changed due to the parameter change of the previous layer during learning. Skip connection prevents the performance of the neural network from decreasing even if the block layer becomes thicker, and makes the block layer thicker to increase the overall neural network performance. The skip connection may be in a form in which the first input data of the residual block is combined with the output of the second batch normalization layer and input to the second Relu activation function layer.

The second time management neural network 530 may generate game time information regarding the predicted remaining game length by inputting the output value of the first time management neural network 520 and the fourth input feature IF4 as inputs. The time management second neural network 530 may be configured in a neural network structure. As an example, the time management second neural network 530 may have a fully connected layer structure.

)로 한 트레이닝 데이터 셋을 이용하여 시간 관리 제1 신경망(520)을 거쳐 시간 관리 제2 신경망(530)에서 생성된 출력 데이터(r)가 타겟 데이터(

)와 동일해지도록 시간 관리 제1 신경망(520) 및 시간 관리 제2 신경망(530)을 충분히 학습할 수 있다. 일 예로, 시간 관리 모델은 남은 경기 길이 예측 손실(

)을 이용하여 남은 경기 길이 예측 손실(

)이 최소가 되도록 트레이닝 할 수 있다. 예를 들어, 남은 경기 길이 예측 손실(

)은 수학식 5를 따를 수 있다.The time management model uses the first to fourth input features (IF1 to IF4) as input data and the remaining game length information as target data (

), the output data (r) generated in the time management second neural network 530 through the first time management neural network 520 using the training data set as the target data (

) and the time management first neural network 520 and the time management second neural network 530 can be sufficiently trained. As an example, the time management model predicts the loss of the remaining game length (

) to predict the length of the remaining match (

) can be trained to be minimal. For example, the predicted loss of remaining match length (

) may follow Equation 5.

(Equation 5)

는 집수의 변화량 손실이다. 남은 경기 길이 예측 손실(

)은 집수의 변화량 손실(

)을 이용한다. 남은 경기 길이 예측을 위하여 집수의 변화량을 이용하는 이유는 대국중 초반의 집수의 변화가 상대적으로 크고, 후반의 집수의 변화가 상대적으로 적을 가능성이 높으므로 남은 경기 길이 예측 판단의 요소로 사용할 수 있기 때문이다. 일 예로, 도 32를 참고하면, 도 32는 임의의 기보의 초반부의 바둑판 상태(S)에 대하여 형세 판단 모델 서버(400)를 이용하여 생성된 형세 판단 정보이다. 도 32(a)는 104수까지 둔 경우로 흑이 46집이고 백이 63.5집이 된다. 도 32(b)는 흑이 한수를 더 둔 105수까지 둔 경우로 흑이 46집이고 백이 59.5집으로 백의 경우 4집의 차이가 난다. 즉, 초반부의 바둑판 상태(S)는 한 수 둘때마다 집수의 변화량이 크다. 도 33를 참고하면, 도 33는 임의의 기보의 후반부의 바둑판 상태(S)에 대하여 형세 판단 모델 서버(400)를 이용하여 생성된 형세 판단 정보이다. 도 33(a)는 222수까지 둔 경우로 흑이 64집이고 백이 63.5집이 된다. 도 33(b)는 백이 한 수를 더 둔 223수까지 둔 경우로 흑이 63집이고 백이 64.5집으로 백의 경우 1집의 차이가 난다. 즉, 후반부의 바둑판 상태(S)는 한 수 둘때마다 집수의 변화량이 작다.in Equation 5

is the change in catchment loss. Remaining Match Length Prediction Loss (

) is the change in catchment loss (

) is used. The reason for using the change in catchment to predict the remaining game length is that the change in catchment in the early part of the game is relatively large and the change in catchment in the second half is likely to be relatively small, so it can be used as a factor in judging the remaining game length. to be. As an example, referring to FIG. 32 , FIG. 32 is situation determination information generated using the situation determination model server 400 with respect to the checkerboard state (S) of the early part of any notation. In Fig. 32(a), up to 104 numbers are placed, and black is 46 and white is 63.5. Fig. 32(b) shows a case where black has an additional number of 105, with a difference of 46 in black and 59.5 in white, and 4 in the case of white. That is, in the initial stage of the checkerboard state (S), the amount of change in water collection is large every time one or two. Referring to FIG. 33 , FIG. 33 is situation determination information generated using the situation determination model server 400 for the checkerboard state (S) of the second half of any notation. 33(a) shows a case where up to 222 numbers are placed, and the number of black is 64 and the number of white is 63.5. Fig. 33(b) is a case in which white has one more number up to 223 numbers. Black has 63 sets and white has 64.5 sets, and in the case of white, there is a difference of 1 set. That is, in the second half of the checkerboard state (S), the amount of change in water collection is small every one or two.

는 공배수 손실이다. 남은 경기 길이 예측 손실(

)은 공배수 손실(

)을 이용한다. 남은 경기 길이 예측을 위하여 공배수를 이용하는 이유는 대국 중에 후반으로 갈수록 공배수가 점점 줄어들기 때문에 이를 이용하여 남은 경기 길이 예측 판단의 요소로 사용할 수 있기 때문이다. 일 예로, 도 34을 참고하면, 도 34은 임의의 기보의 초반부와 후반부의의 바둑판 상태(S)에 대하여 형세 판단 모델 서버(400)를 이용하여 생성된 형세 판단 정보이다. 공배는 도 34의 형세 판단 결과에서 붉은 선으로 X표시가 된 곳이다. 도 34(a)는 대국의 초반부이기 때문에 공배가 많다. 도 34(b)는 대국의 후반부이기 때문에 공배가 적다.in Equation 5

is the common multiple loss. Remaining Match Length Prediction Loss (

) is the common multiple loss (

) is used. The reason why common multiples are used to predict the remaining game length is that the common multiples gradually decrease toward the second half of the game, so it can be used as a factor in judging the remaining game lengths. As an example, referring to FIG. 34 , FIG. 34 is situation determination information generated using the situation determination model server 400 with respect to the checkerboard state (S) of the first half and the second half of any notation. Gongbae is a place marked with an X in the red line in the situation judgment result of FIG. 34 . Figure 34 (a) is the early part of the game, so there are many open matches. Figure 34 (b) is the second half of the game, so there is less common match.

는 가치값 손실이다. 남은 경기 길이 예측 손실(

)은 가치값 손실(

)을 이용한다. 남은 경기 길이 예측을 위하여 가치값 이용하는 이유는 대국 중에 후반으로 갈수록 어느 한쪽의 가치값이 높아지기 때문에 이를 이용하여 남은 경기 길이 예측 판단의 요소로 사용할 수 잇기 때문이다. 또한, 가치값 손실(

)은 초반에는 큰 변화가 없기 때문에 경기 후반부에 이용될 수 있다. 예를 들어, 경기 후반부는 임의의 기보에서 250수이상의 착수가 된 바둑판 상태(S)일 수 있고, 이에 제한되는 것은 아니다.in Equation 5

is the loss of value. Remaining Match Length Prediction Loss (

) is the loss of value (

) is used. The reason why the value value is used for predicting the remaining game length is that the value of either side increases toward the second half of the game, so it can be used as a factor in judging the remaining game length using this value. Also, loss of value (

) can be used in the second half of the game because there is no big change in the beginning. For example, the second half of the game may be a checkerboard state (S) with more than 250 sets in any notation, but is not limited thereto.

는 하이퍼 파라미터들이다. 사용자는 하이퍼 파라미터는 조절하여 각 손실의 상대적인 중요도를 조절할 수 있다. 예를 들어,

는 경기 후반부로 갈수록 중요도가 높아지므로 수치가 높아질 수 있고, 경기 초반부에는 중요도가 낮으므로 수치가 낮을 수 있다.In Equation 5,

are hyperparameters. The user can adjust the relative importance of each loss by adjusting the hyperparameter. for example,

may increase in importance as the game progresses toward the latter part of the game, and may have a low value in the early stage of the game because it is of low importance.

The learned time management model may generate game time information regarding the predicted remaining game length when the current checkerboard state (S) is input during a match. More specifically, in the learned time management model, when the current checkerboard state (S) is input during a game, the first input feature may be extracted by the first input feature extractor 540 . In the time management model, information on the change amount of catchment provided by the situation determination model server as a situation determination for the current checkerboard state (S) as the second input feature, and the common multiple information as the third input feature. The time management model may provide the output value generated by the first time management neural network 520 to the second time management neural network 530 by using the first input characteristic to the third input characteristic as input data. The time management model may include a fourth input feature of a value value for a starting candidate point in the current checkerboard state (S) provided by the starting model server. The time management model may generate game time information about the remaining game length predicted by the second time management neural network 530 by using the output value and the fourth input characteristic of the first time management neural network 520 as input data.

In addition, the time management model server 500 may adjust the start preparation time according to the predicted remaining game length of the game time information. The time management model server 500 may provide the adjusted start preparation time to the set off model server 300 .

Therefore, the deep learning-based Go game service device according to another embodiment may predict the remaining game length. In addition, the deep learning-based Go game service device according to another embodiment can effectively divide the start preparation time using the predicted remaining game length.

35 is an exemplary diagram of a signal flow in the Go game service system of the time management model server 500 according to another embodiment of the present invention.

Referring to FIG. 35 , the Go server 200 may transmit a plurality of notations to the start model server 300 , the situation determination model server 400 and the time management model server 500 ( S2701 ). Time management model server 500 may extract the first input feature of the checkerboard state (S) of a plurality of received notation (S2702). The situation determination model server 400 may generate the situation determination information of a plurality of received notations, and may transmit the situation determination information to the time management model server 500 (S2703, S2704). The situation determination information may be information on a change amount of catchment and information on a common drainage. The time management model server 500 may input the change amount information of the catchment as the second input feature and the common multiple information as the third input feature to the time management first neural network of the time management model ( S2705 ). Start model server 300 may generate a value value according to the checkerboard state (S) of a plurality of notations, and transmit the generated value value to the time management model server 500 (S2706, S2707). The time management model server 500 may use the received value as a fourth input feature and input it into the second neural network for time management of the time management model (S2708). The time management model server 500 may train the time management model by using the input data of the first to fourth input characteristics and the remaining game length information according to the checkerboard state (S) of a plurality of notations as target data (S2709) . The Go server 200 may perform a Go game, and the terminal 100 and the start model server 300 may perform a start in their turn (S2710 to S2712). The situation determination model server 400 extracts the input features of the current checkerboard state (S), and the situation determination model, which is a deep learning model, generates a situation value using the input features, and uses the checkerboard state (S) and the appearance value Thus, the situation can be determined (S2713). The time management model server 500 extracts the input features of the current checkerboard state (S) as the first input features, and the situation determination information as the second and third input features, which is provided by the start model server 300 By using the value as the fourth input feature, the time management model, which is a deep learning model, may generate game time information (S2714).

36 is a method for generating game time information among the deep learning-based Go game service methods according to another embodiment of the present invention.

Referring to FIG. 36 , the deep learning-based Go game service method according to another embodiment of the present invention may include a step (S2801) of the time management model server 500 receiving a plurality of notations. A description of a plurality of notations follows the description of FIG. 31 .

Deep learning-based Go game service method according to another embodiment of the present invention extracts the first input feature from the checkerboard state (S) of a plurality of notations received by the time management model server 500 (s2802) may include A method of extracting the first input feature follows the description of FIG. 31 . The time management model server 500 may provide the extracted first input feature as input data of the first neural network for time management.

The deep learning-based Go game service method according to another embodiment of the present invention may include the step (S2803) of the time management model server 500 receiving the situation determination information according to the checkerboard state (S) of a plurality of notations. can A method of receiving the situation determination information follows the description of FIG. 31 . The situation determination information may be information on a change amount of catchment and information on a common drainage. The time management model server 500 may use the change amount information of the catchment as the second input feature and the common multiple information as the third input feature.

The deep learning-based Go game service method according to another embodiment of the present invention includes the step (S2804) of the time management model server 500 inputting the second input feature and the third input feature to the first neural network for time management can do.

The deep learning-based Go game service method according to another embodiment of the present invention may include the step (S2805) of the time management model server 500 receiving a value from the start model server. The time management model server 500 may use the value as the fourth input feature.

The deep learning-based Go game service method according to another embodiment of the present invention may include the step (S2806) of the time management model server 500 inputting the fourth input characteristic to the time management second neural network.

Deep learning-based Go game service method according to another embodiment of the present invention, the time management model server 500 is the remaining game according to the input data of the first to fourth input characteristics and the checkerboard state (S) of a plurality of notations It may include training the time management model using the length information as target data (S2807). The training method of the time management model follows the description of FIGS. 31 to 34 .

Deep learning-based Go game service method according to another embodiment of the present invention, the time management model server 500 includes the step (S2809) of receiving the Go board state (S), situation determination information, and value during the game can do.

Deep learning-based Go game service method according to another embodiment of the present invention, the time management model server 500 generates game time information using the received Go board state (S), situation determination information, and value values (S2810) may be included. A method of generating game time information follows the description of FIG. 31 .

Therefore, the deep learning-based Go game service method according to another embodiment can predict the remaining playing time. In addition, the deep learning-based Go game service method according to another embodiment can effectively divide the start preparation time using the predicted remaining game time.

The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and used by those skilled in the art of computer software. Examples of the computer-readable recording medium include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floppy disks. medium), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. A hardware device may be converted into one or more software modules to perform processing in accordance with the present invention, and vice versa.

The specific implementations described in the present invention are only examples, and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connections or connecting members of the lines between the components shown in the drawings exemplify functional connections and/or physical or circuit connections, and in an actual device, various functional connections, physical connections that are replaceable or additional may be referred to as connections, or circuit connections. In addition, unless there is a specific reference such as “essential” or “importantly”, it may not be a necessary component for the application of the present invention.

In addition, although the detailed description of the present invention has been described with reference to a preferred embodiment of the present invention, those skilled in the art or those having ordinary knowledge in the art will appreciate the spirit of the present invention described in the claims to be described later. And it will be understood that various modifications and variations of the present invention can be made without departing from the technical scope. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

100 terminals
200 Go server
300 launch model server
310 search unit
320 self-play
330 Initiation Neural Network
400 Situation Judgment Model Server
410 Context Judgment Neural Network
420 input feature extractor
430 Correct answer label generator
500 hours management model server
510 Time Management Department
520 time management first neural network
530 time management second neural network
540 First input feature extraction unit
551 simulation count prediction unit
552 Simulation Count Adjustment Model

Claims (12)

a communication unit for receiving at least one of the checkerboard state, the value value, the number of visits, and the number of default simulations;
a memory for storing the simulation number prediction unit and the simulation number adjustment model; and
Reading the number prediction unit and determining the optimal number of simulations for the checkerboard state using at least one of the checkerboard state, the value value, the number of visits, and the default number of simulations,
training the number adjustment model based on the determined optimal number of simulations,
A processor that reads the number adjustment model and determines the number of simulations according to the current checkerboard state; characterized by comprising:
Simulation count management device. The method of claim 1,
The optimal number of simulations is
In Monte Carlo Tree Search (MCTS) simulation, the next starting point selected during simulation includes the number of simulations that take the minimum time within the critical point that does not change.
Simulation count management device. 3. The method of claim 2,
The number prediction unit,
By matching the determined optimal number of simulations and the checkerboard state, the checkerboard state-optimum simulation number data is generated,
The processor is
Using the generated checkerboard state-optimal simulation number data as a training data set to train the number adjustment model
Simulation count management device. 3. The method of claim 2,
The number prediction unit,
Based on the number of default simulations, based on a first MCTS simulation performed based on the checkerboard state, obtain at least one of a first number of visits and a first value for each number of start candidates,
Based on the second MCTS simulation performed based on the checkerboard state based on the additional unit number of adding a predetermined number to the default simulation number, at least one of a second number of visits and a second value value for each number of starting candidates to obtain
Simulation count management device. 5. The method of claim 4,
The number prediction unit,
Determining whether the next starting point is changed when the number of simulations increases based on the number of first visits and the second number of visits,
Determining the number of simulations that take the minimum time within a critical point in which the next starting point does not change as the optimal number of simulations
Simulation count management device. 6. The method of claim 5,
The number prediction unit,
determining whether to change the next starting point based on the rate of change based on the first number of visits and the second number of visits,
The rate of change is
calculated based on an increase in the number of visits based on a difference between the first number of visits and the second number of visits compared to an increase in the number of simulations based on a difference between the default number of simulations and the additional unit number
Simulation count management device. 6. The method of claim 5,
The number prediction unit,
Determining whether to change the next starting point based on the ratio between the first number of visits for each of the plurality of starting candidates and the ratio between the second number of visits for each of the plurality of starting candidates
Simulation count management device. 5. The method of claim 4,
The number prediction unit,
Determining whether the next starting point is changed when the number of simulations increases based on the increase/decrease rate based on the first value and the second value,
The increase/decrease rate is
Calculated based on the increase in the value of the value based on the difference between the first value and the second value compared to the increase in the number of simulations based on the difference between the default number of simulations and the number of additional units
Simulation count management device. 9. The method of claim 8,
The number prediction unit,
When the increase/decrease rate meets a predetermined criterion, the minimum number of simulations when calculating the increase/decrease rate is determined as the optimal number of simulations.
Simulation count management device. 3. The method of claim 2,
The frequency adjustment model is,
performing an adjustment process for the number of simulations according to the current checkerboard state,
The adjustment process is
A process of setting at least one of an upper limit and a lower limit of the number of simulations; a process of increasing the number of simulations with a predetermined probability; at least one of a process of adjusting the number of simulations and a process of adjusting the number of simulations based on a predetermined statistic on the number of simulations.
Simulation count management device. 11. The method of claim 10,
The processor is
By providing the number of simulations according to the current checkerboard state as a set-off model for performing the MCTS simulation, the set-off model performs the MCTS simulation based on the number of simulations
Simulation count management device. A method of managing the number of simulations of a deep learning-based Go game service in a time management model server, the method comprising:
receiving a predetermined checkerboard state;
performing a first MCTS based on the received checkerboard state;
obtaining at least one of a first number of visits and a first value based on the performed first MCTS;
performing a second MCTS based on the received checkerboard state;
obtaining at least one of a second number of visits and a second value based on the performed second MCTS;
Determining the optimal number of simulations for the checkerboard state based on the first number of visits and the second number of visits, or determining the optimal number of simulations for the checkerboard state based on the first value and the second value step;
generating checkerboard state-optimal simulation number data by matching the determined optimal number of simulations with the checkerboard state;
learning a deep learning model based on the generated checkerboard state-optimal number of simulations data;
determining the number of simulations according to the current checkerboard state based on the learned deep learning model; and
Comprising the step of providing the determined number of simulations as an initiation model
How to manage the number of simulations.

Images