KR102572342B1 - Method and device for deep-learning based simulation count management of go game service - Google Patents

Abstract

An apparatus for managing the number of simulations of a Go game service based on deep learning according to an embodiment of the present invention includes a communication unit receiving at least one of a checkerboard state, a value value, a visit number, and a default simulation number; a memory for storing a simulation number prediction unit and a simulation number adjusting model; and reading the number prediction unit to determine an optimal number of simulations for the checkerboard state using at least one of the checkerboard state, the value value, the number of visits, and the default simulation number, and the number adjustment model based on the determined optimal number of simulations. and a processor for learning and reading the number adjustment model to determine the number of simulations according to the current state of the checkerboard.

Description

Deep learning-based Go game service simulation count management method and its device

The present invention relates to a method and apparatus for managing the number of simulations of a Go game service based on deep learning. More specifically, it relates to a deep learning-based Go game service simulation number management method and apparatus for managing the number of deep learning simulations according to the game situation performed in the Go game service.

As the use of user terminals such as smartphones, tablet PCs, PDAs (Personal Digital Assistants), laptops, etc. has become popular and information processing technology has developed, it has become possible to play Go, a kind of board game, using user terminals. It became possible to play Go with programmed artificial intelligence computers.

Compared to other board games such as chess or chess, Go has a large number of cases, so artificial intelligence computers have limitations in playing a game at the human level, and research to increase the energy of artificial intelligence computers is actively progressing.

Recently, developers have applied the Monte Carlo Tree Search (MCTS) algorithm and deep learning technology to artificial intelligence computers to raise the power of artificial intelligence computers to the level of professional players.

Also, Go is a time-limited board game. The time is different for each Go tournament, but each player can be given various times from 1 hour to 5 hours, and when the given time is exceeded, a countdown rule is applied, and there is a rule to lose if the number of countdowns is exceeded. Therefore, knowing the remaining Go time and determining how much time is spent on a single move is an important factor in winning the game.

However, artificial intelligence computers have a problem in that the amount of time spent to make a move is always constant, and thus, in an important phase, a bad move is made.

In addition, general, amateur, or artificial intelligence computers generally have a problem in that they cannot set up a time strategy because they cannot predict the length of the remaining game.

In addition, when performing training by applying the Monte Carlo Tree Search (MCTS) algorithm and deep learning technology to the artificial intelligence computer as above, most general training uses a fixed number of simulations.

However, using a fixed number of simulations in training has the problem of forcing constraints on the performance of AI computers.

In addition, when performing MCTS to select the next start in the current checkerboard state of a large country, if it depends on a fixed number of simulations, the circumstances related to the corresponding checkerboard state (e.g., win rate and/or available time, etc.) ), there is a problem that the next launch must be decided using only the fixed number of simulations and time without considering it.

JP 4392621 B2

An object of the present invention is to provide a deep learning-based Go game service simulation number management method and apparatus for managing the number of deep learning simulations according to the game situation performed in the Go game service.

In detail, the present invention is a deep learning-based Go game service simulation number management method that dynamically adjusts the number of simulations according to the checkerboard state when deep learning based on the Monte Carlo Tree Search (MCTS) algorithm is performed. And it aims to provide the device.

In addition, an object of the present invention is to provide a method and apparatus for managing the number of simulations of a Go game service based on deep learning that changes the start preparation time in an important phase.

In addition, an object of the present invention is to provide a method and apparatus for managing the number of simulations of a Go game service based on deep learning that can effectively divide the start preparation time using the predicted remaining game length.

However, the technical problems to be achieved by the present invention and the embodiments of the present invention are not limited to the technical problems described above, and other technical problems may exist.

An apparatus for managing the number of simulations of a Go game service based on deep learning according to an embodiment of the present invention includes a communication unit receiving at least one of a checkerboard state, a value value, a visit number, and a default simulation number; a memory for storing a simulation number prediction unit and a simulation number adjusting model; and reading the number prediction unit to determine an optimal number of simulations for the checkerboard state using at least one of the checkerboard state, the value value, the number of visits, and the default simulation number, and the number adjustment model based on the determined optimal number of simulations. and a processor for learning and reading the number adjustment model to determine the number of simulations according to the current state of the checkerboard.

In this case, the optimal number of simulations includes the number of simulations that take the minimum time within a critical point where the next starting point selected during the Monte Carlo Tree Search (MCTS) simulation is not changed.

In addition, the number prediction unit generates checkerboard state-optimal simulation number data by matching the determined optimal simulation number with the checkerboard state, and the processor converts the generated checkerboard state-optimal simulation number data into a training data set (Training Data Set). data set) to learn the frequency adjustment model.

In addition, the number prediction unit obtains at least one of a first visit number and a first value value for each of a plurality of start candidates based on the first MCTS simulation performed based on the checkerboard state based on the default simulation number, , Based on the additional unit count obtained by adding a predetermined number to the default simulation count, based on the second MCTS simulation performed based on the checkerboard state, at least of the second visit count and second value value for each of the plurality of start candidates. get one

In addition, the number prediction unit determines whether the next starting point is changed when the simulation number is increased based on the first visit number and the second visit number, and determines the minimum The number of simulations taking time is determined as the optimum number of simulations.

In addition, the number prediction unit determines whether the next starting point is changed based on a change rate based on the first visit number and the second visit number, and the change rate is a difference between the default simulation number and the additional unit number. It is calculated based on the increase in the number of visits based on the difference between the number of first visits and the number of second visits compared to the increase in the number of simulations based on .

In addition, the number prediction unit determines whether the next starting point is changed based on the ratio between the first visit times for each of the plurality of starting candidates and the second visiting number for each of the plurality of starting candidates.

In addition, the number prediction unit determines whether the next starting point is changed when the number of simulations increases based on an increase/decrease rate based on the first value and the second value, and the increase/decrease rate is the default simulation number and It is calculated based on the value increase based on the difference between the first value and the second value compared to the increase in the number of simulations based on the difference between the number of additional units.

In addition, the number prediction unit determines the minimum number of simulations at the time of calculating the increase/decrease rate as the optimal number of simulations when the increase/decrease rate satisfies a predetermined criterion.

In addition, the number adjustment model performs an adjustment process for the number of simulations according to the current checkerboard state, and the adjustment process includes a process of setting at least one of an upper limit and a lower limit of the number of simulations, and the number of simulations with a predetermined probability. A process of increasing the number of simulations, a process of adjusting the number of simulations based on at least one of a win rate and a change in win rate according to the current checkerboard state, and adjusting the number of simulations based on a predetermined statistical value for the number of simulations. includes at least one of the processes that

In addition, the processor provides the number of simulations according to the current checkerboard state to the initiation model for performing the MCTS simulation, and causes the initiation model to perform the MCTS simulation based on the number of simulations.

Meanwhile, a deep learning-based Go game service simulation number management method according to an embodiment of the present invention is a method for managing the deep learning-based Go game service simulation number in a time management model server, comprising the steps of receiving a predetermined checkerboard state; performing a first MCTS based on the received checkerboard state; obtaining at least one of a first number of visits and a first value based on the first MCTS performed; performing a second MCTS based on the received checkerboard state; obtaining at least one of a second visit number and a second value based on the performed second MCTS; Determining the optimal number of simulations for the checkerboard state based on the first visit number and the second visit number, or determining the optimal number of simulations for the checkerboard state based on the first value and the second value step; generating checkerboard state-optimal simulation number data by matching the determined optimal simulation number with the checkerboard state; learning a deep learning model based on the generated checkerboard state-optimal simulation number data; determining the number of simulations according to the current checkerboard state based on the learned deep learning model; and providing the determined number of simulations as an undertaking model.

A deep learning-based Go game service simulation number management method and apparatus according to an embodiment of the present invention learn the number of simulations when performing deep learning based on a Monte Carlo Tree Search (MCTS) algorithm. can

In addition, the deep learning-based Go game service simulation number management method and apparatus according to an embodiment of the present invention can dynamically adjust the optimal number of simulations according to the checkerboard state through the learning.

In addition, the deep learning-based Go game service simulation number management method and apparatus according to an embodiment of the present invention may change the number of simulations in an important phase.

In addition, the deep learning-based Go game service simulation number management method and apparatus according to an embodiment of the present invention may determine the next start by performing MCTS based on the optimal number of simulations.

In addition, the deep learning-based Go game service simulation number management method and apparatus according to an embodiment of the present invention can change the starting preparation time in an important phase.

In addition, the deep learning-based Go game service simulation number management method and apparatus according to an embodiment of the present invention can effectively distribute the start preparation time using the predicted remaining game time.

However, the effects obtainable in the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood from the description below.

1 is an exemplary view of a deep learning-based Go game service system according to an embodiment of the present invention.
2 is a diagram for explaining the structure of an initiation model of an initiation model server for an initiation of an artificial intelligence computer in a deep learning-based Go game service according to an embodiment of the present invention.
3 is a diagram for explaining a movement probability distribution for an initiation point according to an initiation model policy.
4 is a diagram for explaining the value of the starting point of the starting model and the number of visits.
5 is a diagram for explaining a process in which an initiation model is initiated according to a pipeline of a search unit.
6 is an exemplary diagram showing a screen providing a situation determination function of a deep learning-based Go game service according to an embodiment of the present invention.
7 is a diagram for explaining the structure of the situation judgment model of the situation judgment model server of the present invention.
8 is a diagram for explaining one block of a neural network structure composed of a plurality of blocks of the position judgment model of the present invention.
9 is a diagram for explaining first and second preprocessing steps for generating correct answer labels used to learn a situation judgment model of the present invention.
10 is a diagram for explaining first and second pre-processing steps for generating correct answer labels used to learn a position judgment model of the present invention.
11 is a diagram for explaining a third preprocessing step for generating a correct answer label used to learn a position judgment model of the present invention.
12 is a diagram for explaining the result of the situation judgment of the situation judgment model of the present invention.
13 is a comparison between the situation judgment result of the situation judgment model of the present invention and the situation judgment result by the deep learning model according to the prior art.
14 is a comparison between the situation judgment result of the situation judgment model of the present invention and the situation judgment result by the deep learning model according to the prior art.
15 is a comparison between the situation judgment result of the situation judgment model of the present invention and the situation judgment result of the deep learning model according to the prior art.
16 is an exemplary diagram of a signal flow in a deep learning-based Go game service system according to an embodiment of the present invention.
17 is a situation judgment method among deep learning-based Go game service methods according to an embodiment of the present invention.
18 is a preprocessing method of training data for generating a correct answer label in the situation judgment method of FIG. 17 .
19 is a diagram for explaining a time management unit of a time management model server according to an embodiment of the present invention.
20A and 20B are diagrams for explaining distributed calculation of a time management unit according to an embodiment of the present invention.
21 is an exemplary diagram of a signal flow in a Go game service system of a time management model server according to an embodiment of the present invention.
22 is a method for determining start preparation time among deep learning-based Go game service methods according to an embodiment of the present invention.
23 is a diagram for explaining a simulation number estimation unit and a simulation number adjustment model of a time management model server according to another embodiment of the present invention.
24 is a method for determining the number of MCTS simulations among deep learning-based Go game service methods according to another embodiment of the present invention.
25 is an example of deriving a value value and the number of visits in each of the first MCTS and the second MCTS for a checkerboard state according to another embodiment of the present invention.
26 is an example of a diagram for explaining a method of determining the optimal number of simulations based on the number of visits according to another embodiment of the present invention.
27 is an example of a diagram for explaining a method of determining an optimal simulation number based on a visit number ratio according to another embodiment of the present invention.
28 is an example of a diagram for explaining a method of determining an optimal number of simulations based on a value according to another embodiment of the present invention.
29 is an example of a diagram for explaining a method of adjusting the number of simulations based on a change in odds according to another embodiment of the present invention.
30 is a conceptual diagram for explaining an initiation model for performing MCTS based on the number of simulations according to another embodiment of the present invention.
31 is a diagram for explaining a time management model of a time management model server according to another embodiment of the present invention.
32A and 32B are diagrams for explaining a collection change amount used to generate game time information according to another embodiment of the present invention.
33A and 33B are diagrams for explaining a collection change amount used to generate game time information according to another embodiment of the present invention.
34A and 34B are diagrams for explaining common multiples used to generate game time information according to another embodiment of the present invention.
35 is an exemplary diagram of a signal flow in a Go game service system of a time management model server according to another embodiment of the present invention.
36 is a method for generating game time information among deep learning-based Go game service methods according to another embodiment of the present invention.

Since the present invention can apply various transformations and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. Effects and features of the present invention, and methods for achieving them will become clear with reference to the embodiments described later in detail together with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms. In the following embodiments, terms such as first and second are used for the purpose of distinguishing one component from another component without limiting meaning. Also, expressions in the singular number include plural expressions unless the context clearly dictates otherwise. In addition, terms such as include or have mean that features or elements described in the specification exist, and do not preclude the possibility that one or more other features or elements may be added. In addition, in the drawings, the size of components may be exaggerated or reduced for convenience of description. For example, since the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of description, the present invention is not necessarily limited to the illustrated bar.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when describing with reference to the drawings, the same or corresponding components are assigned the same reference numerals, and overlapping descriptions thereof will be omitted. .

-----------------------------

1 is an exemplary view of a deep learning-based Go game service system according to an embodiment of the present invention.

Referring to FIG. 1, the deep learning-based Go game service system according to an embodiment includes a terminal 100, a Go server 200, a starting model server 300, a situation judgment model server 400, and a network 500. can include

Each component of FIG. 1 may be connected through a network 500 . It refers to a connection structure capable of exchanging information between nodes such as the terminal 100, the Go server 200, the start model server 300, the situation judgment model server 400, and the time management model server 500. , Examples of such networks include a 3rd Generation Partnership Project (3GPP) network, a Long Term Evolution (LTE) network, a World Interoperability for Microwave Access (WIMAX) network, the Internet, a Local Area Network (LAN), and a Wireless LAN (Wireless). Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth (Bluetooth) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc. are included, but are not limited thereto.

<Terminal (100)>

First, the terminal 100 is a terminal of a user who wants to receive a Go game service. Also, the terminal 100 is one or more computers or other electronic devices used by a user to execute applications that perform various tasks. For example, a computer, laptop computer, smart phone, mobile phone, PDA, tablet PC, or any other device operable to communicate with the Go server 200. However, it is not limited thereto, and the terminal 100 is executed on various machines, includes processing logic that interprets and executes commands stored in a plurality of memories, and provides a graphical user interface (GUI) on an external input/output device. It may contain various other elements, such as processes that display graphical information. In addition, the terminal 100 may be connected to an input device (eg, mouse, keyboard, touch sensitive surface, etc.) and an output device (eg, display device, monitor, screen, etc.). Applications executed by the terminal 100 may include game applications, web browsers, web applications running on web browsers, word processors, media players, spreadsheets, image processors, security software, or the like. .

In addition, the terminal 100 may include at least one memory 101 storing instructions, at least one processor 102 and a communication unit 103 .

The memory 101 of the terminal 100 may store a plurality of application programs or applications running in the terminal 100, data for operation of the terminal 100, and commands. The instructions are executable by the processor 102 to cause the processor 102 to perform operations, the operations sending a Go game execution request signal, sending and receiving game data, sending and receiving starting information, sending a situation judgment request signal, It may include operations for receiving a decision result, requesting game time information, receiving game time information, and receiving various types of information. In addition, the memory 101 may be a variety of storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in terms of hardware, and the memory 130 performs the storage function of the memory 101 on the Internet. It can also be a web storage that performs.

The processor 102 of the terminal 100 may perform data processing to receive a Go game service by controlling overall operations. When a Go game application is executed in the terminal 100, a Go game environment is configured in the terminal 100. In addition, the Go game application exchanges Go game data with the Go server 200 through the network 500 so that the Go game service is executed on the terminal 100 . These processors 102 include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, and microcontrollers. It may be micro-controllers, microprocessors, or any type of processor for performing other functions.

The communication unit 103 of the terminal 100 uses the following communication methods (eg, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink (HSUPA)) Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc.), WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA ( A wireless signal may be transmitted and received with at least one of a base station, an external terminal, and a server on a network constructed according to Digital Living Network Alliance (WiBro), Wireless Broadband (WiBro), and World Interoperability for Microwave Access (WiMAX).

This terminal 100 performs at least some of the functional operations performed by at least one of the Go server 200, the starting model server 300, the situation judgment model server 400, and the time management model server 500, which will be described later. You may.

<Go server (200)>

The Go game service provided by the Go server 200 may be configured in a form in which a virtual computer user provided by the Go server 200 and a real user participate in the game together. In the Go game environment implemented on the user-side terminal 100, one real user and one computer user play the game together. In another aspect, the Go game service provided by the Go server 200 may be configured in a form in which a plurality of user-side devices participate to play the Go game.

The Go server 200 may include at least one memory 201 storing instructions, at least one processor 202 and a communication unit 203.

The memory 201 of the Go server 200 may store a plurality of application programs or applications running in the Go server 200, data for the operation of the Go server 200, and commands. . Instructions are executable by the processor 202 to cause the processor 202 to perform operations, which include receiving a game execution request signal, transmitting and receiving game data, transmitting and receiving starting information, transmitting and receiving a situation judgment request signal, and transmitting and receiving a situation judgment result. , start preparation time transmission and reception, game information time transmission and reception, and various transmission operations. In addition, the memory 201 may store a plurality of notations or previously published notations that were played in the Go server 200. Each of the plurality of notation may include all information from the first start, which is the first start information of the start of the match, to the final start when the match ends. That is, a plurality of notes may include history information about the undertaking. The Go server 200 enables the position determination model server 400 to provide a plurality of notations stored for training of the position determination model server 400 . In addition, the memory 201 may be a variety of storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in terms of hardware, and the memory 201 performs the storage function of the memory 201 on the Internet. It can also be a web storage that performs.

The processor 202 of the Go server 200 may perform data processing to provide a Go game service by controlling overall operations. These processors 202 include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, and microcontrollers. It may be micro-controllers, microprocessors, or any type of processor for performing other functions.

The Go server 200 may communicate with the terminal 100, the starting model server 300, and the situation judgment model server 400 via the network 500 through the communication unit 203.

<Initiation model server 300>

The undertaking model server 300 may include a separate cloud server or computing device. In addition, the start model server 300 may be a neural network system installed in the processor of the terminal 100 or the data processing unit of the Go server 200, but hereinafter, the start model server 300 is the terminal 100 or the Go server ( 200) and a separate device.

The undertaking model server 300 may include at least one memory 301 storing instructions, at least one processor 302 and a communication unit 303.

The starting model server 300 learns by itself according to the rules of Go, builds a deep learning model, the starting model, and is an artificial intelligence computer capable of playing a game with the user of the terminal 100. launch can be carried out. A detailed description of how the initiation model server 300 trains the initiation model follows the description of the initiation model of FIGS. 2 to 5 .

The memory 301 of the initiation model server 300 stores a plurality of application programs or applications running in the initiation model server 300, data for the operation of the initiation model server 300, and instructions. can be saved Instructions are executable by the processor 302 to cause the processor 302 to perform operations, which include a set-up model learning (training) operation, set-up information transmission and reception, set-up preparation time reception, game time information reception, and various transmissions. Actions may be included. In addition, the memory 301 may store an undertaking model that is a deep learning model. In addition, the memory 301 may be a variety of storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in terms of hardware, and the memory 301 performs the storage function of the memory 301 on the Internet. It can also be a web storage that performs.

The processor 302 of the set-up model server 300 reads the set-out model stored in the memory 302, and performs the set-out model learning and Go game set-up described below according to the built neural network system. Depending on the embodiment, the processor 302 may be configured to include a main processor for controlling all units and a plurality of graphics processing units (GPUs) for processing large-capacity calculations required for driving a neural network according to an initiation model. there is.

The undertaking model server 300 may communicate with the Monarch server 200 via the network 500 through the communication unit 303 .

<Scenario judgment model server 400>

The situation judgment model server 400 may include a separate cloud server or a computing device. In addition, the situation judgment model server 400 may be a neural network system installed in the processor of the terminal 100 or the data processing unit of the Go server 200, but hereinafter, the situation judgment model server 400 is used for the terminal 100 or the Go server. It will be described as a device separate from the server 200.

The layout judgment model server 400 may include at least one memory 401 storing instructions, at least one processor 402 and a communication unit 403 .

The situation judgment model server 400 may receive a training data set from the Go server 200 through the communication unit 403 . The training data set may be a plurality of notations and situation determination information for the plurality of notations. The situation judgment model server 400 uses the received training data set to perform supervised learning to determine the situation of the state of the board on which the game of Go is placed, and builds a situation judgment model, which is a deep learning model, and determines the situation of the user of the terminal 100 Depending on the request for judgment, the situation may be judged. A detailed description of how the situation judgment model server 400 trains the situation judgment model follows the description of the situation judgment model of FIGS. 6 to 18 .

The memory 401 of the layout judgment model server 400 stores a plurality of application programs or applications running in the layout judgment model server 400 and data for the operation of the layout judgment model server 400. , can store commands. The instructions are executable by the processor 402 to cause the processor 402 to perform operations, the operations being a situation judgment model learning (training) operation, performing a situation judgment, transmitting a situation judgment result, receiving a plurality of notational information, It may include transmission of variation information of collections, transmission of common multiple information, and various transmission operations. In addition, the memory 401 may store a situation judgment model that is a deep learning model. In addition, the memory 401 may be a variety of storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in terms of hardware, and the memory 401 performs the storage function of the memory 301 on the Internet. It can also be a web storage that performs.

The processor 402 of the situation judgment model server 400 reads the situation judgment model stored in the memory 402, and performs learning of the situation judgment model described below and judgment of the situation of the checkerboard among the players according to the built neural network system. . According to an embodiment, the processor 402 is configured to include a main processor that controls all units and a plurality of graphics processing units (GPUs) that process large-capacity calculations required for driving a neural network according to a situation judgment model. can

The position judgment model server 400 may communicate with the Monarch server 200 via the network 500 through the communication unit 403 .

<Time management model server (500)>

The time management model server 500 may include a separate cloud server or a computing device. In addition, the time management model server 500 may be a neural network system installed in the processor of the terminal 100, the memory of the Go server 200, the memory of the start model server 300, or the memory of the situation judgment model server 400. Hereinafter, the time management model server 500 will be described as a device separate from the terminal 100, the Monarch server 200, the starting model server 300, or the situation judgment model server 400.

The time management model server 500 may include at least one memory 501 storing instructions, at least one processor 502 and a communication unit 503 .

In addition, the time management model server 500 may receive the number of visits, a search probability value, or a value value from the undertaking model server 300 through the communication unit 503 . The time management model server 500 may determine the start preparation time using the received number of visits, a search probability value, and a value value. A detailed description of the method for determining the start preparation time of the time management model server 500 follows the descriptions of FIGS. 19 to 22 .

In addition, the time management model server 500 may receive a training data set from the Go server 500 through the communication unit 503 . The training data set may be a plurality of notations, position judgment information for the plurality of notations, and time information according to each checkerboard state for the plurality of notations.

In addition, the time management model server 500 receives the checkerboard state (S), the default simulation number (D), the number of visits (N) and / or value value (V) can be received. The time management model server 500 may determine the number of MCTS simulations using the received checkerboard state (S), default simulation count (D), visit count (N), and/or value value (V). A detailed description of the method for determining the number of MCTS simulations by the time management model server 500 follows the descriptions of FIGS. 23 to 30 .

In addition, the time management model server 500 may receive situation determination information from the situation determination model server 400 through the communication unit 503 . The situation determination information may include change amount information of catchments, common multiple information, and the like. The time management model server 500 builds a time management model, which is a deep learning model, by conducting supervised learning to generate game time information using the received training data set, situation judgment information, value value, etc. ) or game time information according to the checkerboard state may be provided to the terminal 100 . A detailed description of how the time management model server 50 trains the time management model follows the description of the time management model of FIGS. 35 and 36 .

The memory 501 of the time management model server 500 stores a plurality of application programs or applications running in the time management model server 500 and data for operation of the time management model server 500. , can store commands. Instructions are executable by the processor 502 to cause the processor 502 to perform operations, such as time management model learning (training) operations, receiving number of visits, search probabilities or values, and determining preparation time for launching. , Receiving a plurality of notation information, receiving default simulation count, receiving variation information of collections, receiving common multiple information, generating game time information, and various transmission operations. In addition, the memory 501 may store a time management unit, a simulation number prediction unit, a simulation number adjustment model, or a time management model. In addition, the memory 501 may be a variety of storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in terms of hardware, and the memory 501 performs the storage function of the memory 501 on the Internet. It can also be a web storage that performs.

The processor 502 of the time management model server 500 reads the time management unit stored in the memory 501 and determines the start preparation time described below.

In addition, the processor 502 of the time management model server 500 reads the simulation number predictor and/or the simulation number adjustment model stored in the memory 501 to determine the number of MCTS simulations described later.

In addition, the processor 502 of the time management model server 500 reads the time management model stored in the memory 501 and generates game time information described below according to the constructed neural network system.

According to an embodiment, the processor 502 is configured to include a main processor that controls all units and a plurality of graphics processing units (GPUs) that process large-capacity calculations required for driving a neural network according to a time management model. can

The time management model server 500 may communicate with the Monarch server 200, the start model server 300, and the situation judgment model server 400 via the network 500 through the communication unit 403.

<Startup model>

2 is a diagram for explaining the start model structure of the start model server for the start of an artificial intelligence computer in the deep learning-based Go game service according to an embodiment of the present invention, and FIG. 3 is the start point according to the start model policy. Figure 4 is a diagram for explaining the movement probability distribution for , Figure 4 is a diagram for explaining the value value and the number of visits to the starting point of the initiation model, and Figure 5 shows the process initiating the initiation model according to the pipeline of the search unit. It is a drawing for explanation.

Referring to FIG. 2 , the initiation model according to an embodiment of the present invention is a deep learning model of the initiation model server 300, and may include a search unit 310, a self-play unit 320, and an initiation neural network 330. .

The starting model may be learned as a starting model to win a game by using the search unit 310, the self-play unit 320, and the starting neural network 330. More specifically, the search unit 310 may perform a Monte Carlo Tree Search (MCTS) operation according to the guidance of the initiating neural network 330 . MCTS is a heuristic search algorithm for some kind of decision making. That is, the search unit 310 may perform MCTS based on the movement probability value (p) and/or value value (V) provided by the initiating neural network 330 . For example, the search unit 310 guided by the initiating neural network 330 performs MCTS to search probability values that are probability distribution values for the initiation points ( ) can be output. The self-play unit 320 provides a search probability value ( ), you can play Go yourself. The self-play unit 320 proceeds to play Go by itself until the game is decided, and when the self-playing game ends, the checkerboard state (S), search probability value ( ), the self-play value value z may be provided to the initiating neural network 330 . The checkerboard state (S) is a state in which Go stones are placed at starting points. The self-play value value (z) is a win rate value when the player plays a game in the checkerboard state (S). The onset neural network 330 may output a movement probability value (p) and a value value (V). The moving probability value (p) is a probability distribution value numerically indicating which starting point is a good number to win the game with respect to the starting points according to the checkerboard state (S). The value value (V) represents the winning rate when starting at the starting point. For example, a starting point having a high movement probability value p may be a good number. The onset neural network 330 determines that the movement probability value (p) is the search probability value ( ), and the value (V) may be trained to be equal to the self-play value (z). Then, the trained initiating neural network 330 guides the search unit 310, and the search unit 310 uses the previous search probability value ( ) to find a better number than the new search probability value ( ) to output. For example, the start preparation time may follow any one of the start preparation time, the first start preparation time, and the second start preparation time according to the MCTS progress time. The start preparation time may be provided by the time management model server 500 and may be basically set to the average start preparation time. The self-play unit 320 provides a new search probability value ( ) Based on the checkerboard state (S), a new self-play value value (z) is output, and the checkerboard state (S), a new search probability value ( ), a new self-play value value z may be provided to the initiating neural network 330. The starting neural network 330 has a new search probability value (p) and value value (V). ) and a new self play value (z) can be trained again. That is, the starting model can be trained to find a better starting point for winning the game by repeating this process so that the starting neural network 330 can win. As an example, the launch model may use the launch loss (l). The landing loss (l) is shown in Equation 1.

(Equation 1)

is a parameter of the neural network, and c is a very small constant.

Making z and v equal in the starting loss (l) of Equation 1 corresponds to the mean square loss term, and p are equal to the cross entropy loss term, Multiplying by c is to prevent overfitting with the regularization term.

For example, referring to FIG. 3 , the trained initiation model may represent movement probability values p at the initiation points as probability distribution values as shown in FIG. 3 . Referring to FIG. 4, the search probability value of the trained initiation model ( ) can be expressed as the value shown above at one starting point. Search probability value ( ) may be a ratio of the number of visits of the starting candidates to the total number of visits. For example, if the total number of MCTS simulations is 1000 and 90.00 is displayed, it means that 900 out of 1000 visits were made to the number of starting candidates. The value (V) of the trained initiation model can be represented by the value shown below at one initiation point in FIG. 4 . The initiating neural network 330 may be composed of a neural network structure. For example, the initiating neural network 330 may include one convolution block and 19 residual blocks. A convolution block may have a form in which several 3X3 convolution layers are overlapped. One residual block may have a form in which several 3X3 convolutional layers are overlapped and a skip connection is included. A skip connection is a structure in which an input of a predetermined layer is outputted by summing an output value of a corresponding layer and inputted to another layer. In addition, the input of the starting neural network 330 is 19 * including stone position information for the last 8 moves of the black player, stone position information for the last 8 moves of the white player, and turn information on whether the current player is black or white. A 19*17 RGB image may be input.

Referring to FIG. 5 , the learned start model may start using the start neural network 330 and the search unit 310 in its own turn. Through the selection process (a), the starting model is the second checkerboard state (S1) having a starting point with a high activity function (Q) and high confidence value (U) among the branches not searched through MCTS in the current first checkerboard state (S1). -2) is selected. The activity function (Q) is the average value of value values (V) calculated every time a corresponding branch is passed. The confidence value (U) is inversely proportional to the number of visits (N) passing through the corresponding branch and proportional to the movement probability value (p). The starting model may be expanded to the third checkerboard state (S1-2-1) at the selected starting point through the expansion and evaluation process (b) and calculate a movement probability value (p). The initiation model calculates the value value (V) of the extended third checkerboard state (S1-2-1), and the activity function (Q), number of visits (N), and movement probability value of the branches passed through the backup process (c) (p) can be stored. The initiation model repeats the process of selection (a), expansion and evaluation (b), and backup (c) during the preparation time for initiation, and a probability distribution is created using the number of visits (N) for each initiation point, and the search probability value ( ) can be output. The initiation model has the highest search probability value among the initiation points ( ) can be selected.

<Scenario Judgment Model>

6 is an exemplary diagram showing a screen for providing a layout judgment function of a deep learning-based Go game service according to an embodiment of the present invention, and FIG. 7 is for explaining the layout judgment model structure of the layout judgment model server of the present invention. FIG. 8 is a diagram for explaining one block of a neural network structure composed of a plurality of blocks of the situation judgment model of the present invention.

Referring to FIG. 6 , the deep learning-based Go game service according to an embodiment of the present invention can determine the situation of the current Go board state. For example, as shown in FIG. 6 , when a user clicks a situation judgment menu (A) during a game of Go on the screen of the terminal 100 and requests a situation judgment, the deep learning-based Go game service provides the situation judgment result in a pop-up window. can The situation judgment is to determine who is winning by how many points by calculating the opponent and my house during the game of Go. For example, if the user judges that the situation is advantageous to me, he will not overdo it anymore and will set up a strategy in the direction of ending the match while maintaining the current favorable situation as it is, and if it is judged that the situation is unfavorable, the game phase will be changed to a new one. There are several strategies you can try to do this. The criteria for judging the situation are house, private stone, stone, gongbae, and big according to the state in which the stones are placed on the board. A stone is a stone placed on a checkerboard and is not a score in Korean rules. A house is an area consisting of empty dots surrounded by stones of one color, which is a score in Korean rules. Gongbae and Big are areas that are neither black nor white when the game of Baduk is over, and are not points in Korean rules. Panwisa-seok is a stone that has no choice but to be caught among the stones placed on the checkerboard, and it is used to fill the opponent's house in Korean rules, so it is a score. Big refers to the area that is neither black nor white when the game of Go is over. Therefore, the judgment of the situation can be an accurate judgment only when accurately distinguishing or predicting the house, four stones, stones, gongbae, and big in the state of the checkerboard where the baduk stones are placed. At this time, accurately distinguishing House, Four Stone, Stone, Gongbae, and Big is to distinguish the state in which the house, four stone, stone, Gongbae, and Big are completely formed, and accurately predicting House, Four Stone, Dol, Gongbae, and Big is , it may be to predict a state with a high probability of becoming a stone, a stone, a ball, or a big.

Referring to FIG. 7 , the situation judgment model according to an embodiment of the present invention is a deep learning model of the situation judgment model server 400, and includes a situation judgment neural network 410, an input feature extractor 420, and a correct answer label generator 430. ) may be included.

The situation judgment model may perform supervised learning so that the situation of the current checkerboard state can be determined using the situation judgment neural network 410 . More specifically, a training data set related to the checkerboard state (S) of the layout judgment model is created, and the layout judgment neural network 410 learns to determine the layout according to the current checkerboard state (S) using the generated training data set can make it The position judgment model server 400 may receive a plurality of notations from the Go server 200 . Each notation of a plurality of notations may include each checkerboard state (S) according to the starting order.

The input feature extraction unit 420 may extract an input feature (IF) from a plurality of checkerboard states (S) and provide the input feature (IF) to the shape judgment neural network 410 as input data for training. The input feature (IF) of the checkerboard state (S) includes stone position information for the black player's last 8 moves, stone position information for the white player's last 8 moves, and turn information on whether the current player is black or white. It may be a 19*19*18 RGB image. For example, the input feature extractor 420 may have a neural network structure and may include a kind of encoder.

The correct answer label generation unit 430 generates a correct answer label (ground truth) through a preprocessing process with the current checkerboard state (S), and transfers the correct answer label to the position judgment neural network 410 as target data for training ( ) can be provided. The generation of correct answer labels by the correct answer label generator 430 follows the description of FIGS. 9 to 11 to be described later. For example, the correct answer label generation unit 430 may include a rollout or an encoder of a neural network structure.

The position judgment model takes input features (IF) as input data and correct answer labels as target data ( ), the output data (o) generated by the situation judgment neural network 410 using the training data set is the target data ( ), the situation determination neural network 420 may be sufficiently learned. As an example, the situation judgment model is a situation judgment loss ( ) can be used. loss of judgment ( ) can use the mean square error. For example, loss of judgment ( ) is the same as Equation 2.

(Equation 2)

B is the total number of intersections on the checkerboard. In the checkerboard, 19 horizontal and 19 vertical lines intersect each other, and 361 intersections are arranged. It is not limited thereto, and when the checkerboard has 9 horizontal lines and 9 vertical lines, 81 intersection points may be arranged. is a positional value for a predetermined intersection point (i) according to the correct answer label in the current checkerboard state (S). Description of the shape value follows the description of FIG. 11 to be described later. is output data that is output when a predetermined intersection (i) is input to the situation judgment neural network 410 in the current checkerboard state (S). The layout judgment model is based on the layout loss ( ) is minimized, the situation judgment neural network 410 may be trained by adjusting weights and bias values in the situation judgment neural network 410 using gradient-descent and backpropagation.

The situation judgment neural network 410 may have a neural network structure. For example, the position judgment neural network 420 may be composed of 19 residual blocks. Referring to FIG. 8, one residual block includes 256 3X3 convolution layers, batch normalization layers, Relu activation function layers, 256 3X3 convolution layers, batch normalization layers, skip connections, Relu activation function layers may be arranged in order. The batch normalization layer is intended to prevent covariate shift, a phenomenon in which the distribution of inputs of the current layer changes due to changes in parameters of the previous layer during learning. The skip connection prevents the performance of the neural network from decreasing even when the block layer becomes thicker, and increases overall neural network performance by making the block layer thicker. The skip connection may be input to a second Relu activation function layer by summing the first input data of the residual block with the output of the second batch normalization layer.

9 and 10 are views for explaining first and second preprocessing steps for generating correct answer labels used to learn the situation judgment model of the present invention, and FIG. 11 is for learning the situation judgment model of the present invention. It is a diagram for explaining the third pre-processing step for generating the correct answer label used for the first step.

The correct answer label generation unit 430 may generate correct answer labels used for learning so that the situation judgment neural network 410 can accurately determine the situation.

More specifically, the correct answer label generation unit 430 receives the checkerboard state (S) based on the input data as an input, performs the first preprocessing of ending in the current checkerboard state (S), and performs the first preprocessing state (P1). ) can be created. The first preprocessing, the ending, is a process of finishing the game by making a predetermined start so that the boundary of the house becomes clear before calculating the house. For example, referring to FIG. 9 , the correct answer label generation unit 430 generates the first preprocessing state P1 of FIG. can

The correct answer label generation unit 430 may generate a second preprocessing state P2 by performing a second preprocessing of removing stones disposed within the house boundary and unnecessary for house classification in the first preprocessing state P1. For example, a stone that is placed within the boundary of a house and is unnecessary for house division may be a rubble stone. It was explained earlier that Sa-seok was a stone that died because the opponent's stone was placed in the house and no matter how it was placed, it was inevitable to be caught. In addition, a stone placed within the house boundary and unnecessary for house division may be a stone of one's own placed in the house. For example, referring to FIG. 9 , the correct answer label generation unit 430 removes stones unnecessary for house classification in the first preprocessing state P1 of FIG. 9(b) to the second preprocessing state of FIG. (P2) can be created.

As another example, referring to FIG. 10 , the correct answer label generation unit 430 embarks on a red x as shown in FIG. can do. The correct answer label generation unit 430 removes the rubble stones indicated by the blue x in FIG. Preprocessing can be done.

The correct answer label generation unit 430 may perform a third preprocessing in the second preprocessing state P2 to change each intersection into a shape value (g, where g is an integer) displayed from -1 to +1. That is, in the third pre-processing, the correct answer label generation unit 430 changes the second pre-processing state P2, which is an image feature, into the third pre-processing state P3, which is a numerical feature. For example, in the second preprocessing state P2, if my stone is placed at the intersection, 0, if it is my home area, +1, if the opponent's stone is placed, 0, and if it is the other house area, -1. In this case, the situation determination neural network 410 may be trained to distinguish between a house, a stone, and a private stone when determining a situation. As another example, in the second preprocessing state P2, 0 if my stone is placed at the intersection point, +1 if it is my home area, 0 if the other stone is placed, -1 if the other stone is placed, and 0 if it is big or common. In another example, the situation determination neural network 410 may be trained to distinguish a big or a bad outcome when determining a situation. For example, referring to FIG. 11, the correct answer label generator 430 characterizes the second preprocessing state P2 of FIG. 11(a) as the third preprocessing state P3 of FIG. 11(b). can be changed

The third preprocessing state (P3) becomes the correct answer label of the situation judgment in the checkerboard state (S), and the target data ( ) can be used.

12 is a diagram for explaining the result of the situation judgment of the situation judgment model of the present invention.

When a checkerboard state is input, the learned layout judgment model may provide layout values for all intersections of the checkerboard. That is, an integer value of -1 to +1, which is a shape value, can be provided for 361 points of the checkerboard intersection.

Referring to FIG. 12 , the situation judgment model server 400 may determine the situation using the situation value provided by the situation judgment model, a predetermined threshold value, and the presence or absence of stones. For example, the layout judgment model server 400 may determine a place where there is no stone, and if the value of the layout exceeds a first threshold value, it is highly likely to be my home, and if the value is close to +1, it is determined to be my home area. there is. The layout judgment model server 400 may display a square of the same color as my stone, which gradually increases as the probability of being my house increases. The layout judgment model server 400 may determine that there is no stone, and if the value of the layout is less than the second threshold, the location is highly likely to be the other's house. The layout judgment model server 400 may display a square of the same color as the opponent's stone, which gradually increases as the probability of the opponent's house increases. The position judgment model server 400 is a place where there is no stone, and if the position value is within the range of the third threshold or a value close to 0, it can be judged as an empty score or a big match. The situation judgment model server 400 may indicate an X when it is judged as a common or a big match. The position judgment model server 400 is a place where there is a stone, and if the position value is within the third threshold range or a value close to 0, it can be determined as my stone or the opponent's stone. The situation judgment model server 400 may not display anything when it is determined that the situation judgment model server 400 is an empty match or a big match. The situation judgment model server 400 is a place where there is a stone, and if the value of the situation exceeds the first threshold, it is determined that it is likely to be the opponent's stone, and if the value is close to +1, it can be determined as the opponent's stone. there is. The layout judgment model server 400 may display a square of the same color as my stone, which gradually increases as the probability of the opponent's stone being rubbish increases. The situation judgment model server 400 is a place where there is a stone, and if the value of the situation is less than the second threshold, it is determined that it is a place with a high possibility of being a stone of my stone, and if it is close to -1, it can be determined as a stone of the opponent's stone. there is. The layout judgment model server 400 may display a square of the same color as the opponent's stone, which gradually increases as the probability of the opponent's stone being a rubble stone increases.

In addition, the situation judgment model server 300 may display the result of the count in the current checkerboard state by using the situation judgment criterion determined at each intersection.

In addition, the situation judgment model server 300 may generate change amount information and common multiple information of catchments according to the checkerboard state. For example, the situation determination model server 300 may generate information on the amount of change in the catchment by using the result of determining the situation in the checkerboard state according to the previous start and the result of the determination of the situation in the current checkerboard state. In addition, the position determination model server 300 may generate common multiple information using the position determination result in a checkerboard state.

Therefore, the device of the deep learning-based Go game service according to the embodiment may determine the situation of Go using the deep learning neural network. In addition, the deep learning-based Go game service device according to the embodiment can accurately determine the situation of Go by accurately classifying house, sandstone, stone, ball game, and big according to the rules of Go. In addition, the deep learning-based Go game service device according to the embodiment can accurately determine the situation of Go by predicting House, Four Stone, Stone, Gongbae, and Big according to the rules of Go. In addition, the deep learning-based Go game service device according to the embodiment can quickly determine the situation during the game of Go.

13 is a comparison between the situation judgment result of the situation judgment model of the present invention and the situation judgment result by the deep learning model according to the prior art, and FIG. 14 is the situation judgment result of the situation judgment model of the present invention and the situation judgment result according to the prior art 15 is a comparison of the situation judgment result by the deep learning model, and FIG. 15 is a comparison between the situation judgment result of the situation judgment model of the present invention and the situation judgment result of the deep learning model according to the prior art.

Referring to FIG. 13, the situation judgment model of the present invention determines the situation by classifying houses, stones, and rubble stones at each intersection point as in region B of FIG. 13(a). However, the situation judgment model using the deep learning model according to the prior art cannot distinguish between a house, a stone, and a sandstone at the intersection of the region corresponding to FIG. 13(a) in FIG. 13(b).

Likewise, referring to FIG. 14, the situation judgment model of the present invention determines the situation by classifying houses, stones, and rubble stones at each intersection point as in area C of FIG. 14 (a). However, the situation judgment model using the deep learning model according to the prior art cannot distinguish between a house, a stone, and a sandstone at the intersection of the region corresponding to FIG. 14(b) to FIG. 13(a).

Referring to FIG. 15 , the position judgment model of the present invention correctly recognizes a bag house as shown in area D of FIG. 15 (a). However, the situation judgment model based on the deep learning model according to the prior art cannot discriminate the back house in the area corresponding to FIG. 15(a) in FIG. 15(b).

16 is an exemplary diagram of a signal flow in a deep learning-based Go game service system according to an embodiment of the present invention.

Referring to FIG. 16, the start model server 300 is an artificial intelligence computer that learns by itself according to the rules of Go and trains the start model, which is a deep learning model, so that it can perform the start of the Go stone so that it can win the game in its own turn. can (s11). The Go server 22 may transmit a plurality of notations to the situation judgment model server 400 . The situation judgment model server 400 may generate a training data set. First, the position judgment model server 400 may extract input features from a checkerboard state of a plurality of notations (S13). The position judgment model server 400 may generate a correct answer label using the checkerboard state from which input features are extracted (S14). The layout judgment model server 400 may train a layout judgment model using a training data set in which input features are input data and correct answer labels are target data (S15). The terminal 100 may request the Go server 200 to play a Go game against an artificial intelligence computer or against another user terminal (S16). The Go server 200 may request the start model server 300 to start when the terminal 100 requests a Go game against the artificial intelligence computer (S17). The Go server 200 proceeds with the Go game, and the terminal 100 and the start model server 300 may perform a start on their turn (S18 to S20). During the game, the terminal 100 may request the Go server 200 to determine the situation (S21). The Go server 200 may request the position judgment model server 400 to determine the position of the current Go board state (S22). The layout judgment model server 400 extracts the input features of the current checkerboard state, the layout judgment model, which is a deep learning model, generates a layout value using the input features, and performs the layout judgment using the checkerboard status and the layout values. It can (S23). The situation judgment model server 400 may provide the situation judgment result to the Go server 200 (S24). The Go server 200 may provide the result of the situation determination to the terminal 100 (S25).

17 is a method for determining a situation among deep learning-based Go game service methods according to an embodiment of the present invention, and FIG. 18 is a method for preprocessing training data to generate a correct answer label among the method for determining a situation in FIG. 17 .

Referring to FIG. 17, the deep learning-based Go game service method according to an embodiment of the present invention may include a step (S100) of receiving a plurality of notations from the Go server by the situation judgment model server.

The deep learning-based Go game service method according to an embodiment of the present invention may include a step (S200) of extracting input features from a plurality of notation checkerboard states by an input feature extraction unit in a layout judgment model of a layout judgment model server. . A method of extracting input features follows the description of FIG. 7 .

The deep learning-based Go game service method according to an embodiment of the present invention may include a step of generating a correct answer label based on a checkerboard state from which an input feature is extracted by a correct answer label generator in a situation judgment model (S300). For example, referring to FIG. 18 , the correct answer label generating step ( S300 ) may include a first preprocessing step ( S301 ) in which the correct answer label generating unit finishes in the current checkerboard state. The first preprocessing step (S301) follows the description of FIGS. 9 to 10. The correct answer label generation step (S300) may include a second preprocessing step (S302) of removing unnecessary stones from the first preprocessed checkerboard state by the correct answer label generator. The second pre-processing step (S302) follows the description of FIGS. 9 to 10. The correct answer label generating step (S300) may include a third preprocessing step (S303) of changing each intersection of the second preprocessed checkerboard state into a shape value by the correct answer label generating unit. The third pre-processing step (S303) follows the description of FIG. The correct answer label generation step (S300) may include a step (S303) of providing the third preprocessing state as the correct answer label to the situation judgment neural network as target data. The step of providing target data (S301) follows the description of FIGS. 7 and 11.

The deep learning-based Go game service method according to an embodiment of the present invention may include training a situation judgment neural network of a situation judgment model using a training data set (S400). A method of training (learning) the situation decision neural network follows the description of FIG. 7 .

The deep learning-based Go game service method according to an embodiment of the present invention includes a step (S500) of constructing a situation judgment model by completing training of a situation judgment neural network. For example, the training of the situation judgment neural network may be completed when the situation judgment loss of FIG. 7 becomes less than or equal to a predetermined value.

The deep learning-based Go game service method according to an embodiment of the present invention may include a step (S600) of inputting a current checkerboard state to a situation judgment model in response to a situation judgment request from a terminal.

The deep learning-based Go game service method according to an embodiment of the present invention may include a step (S700) of determining the situation of the current Go board state, to which the situation judgment model is input. The step of determining the situation (S700) may follow the description that the situation judgment model described in FIG. 12 generates the situation value of the current checkerboard state.

The deep learning-based Go game service method according to an embodiment of the present invention may include a step (S800) of outputting a situation judgment result by a situation judgment model server. The step of outputting the situation judgment result (S800) may follow the description of providing the situation judgment result by using the situation value, the state of the checkerboard, and a predetermined threshold value by the situation judgment model server described in FIG. 12 .

Therefore, the deep learning-based Go game service method according to the embodiment may determine the situation of Go using a deep learning neural network. In addition, the deep learning-based Go game service method according to the embodiment can accurately determine the situation of Go by accurately classifying house, sandstone, stone, ball game, and big according to the rules of Go. In addition, the deep learning-based Go game service method according to the embodiment can accurately determine the situation of Go by predicting House, Four Stone, Stone, Gongbae, and Big according to the Go rules. In addition, the deep learning-based Go game service method according to the embodiment can quickly determine the situation during a game of Go.

<Time management model server according to an embodiment>

19 is a diagram for explaining a time management unit of a time management model server according to an embodiment of the present invention, and FIG. 20 is a diagram for explaining distributed calculation of a time management unit according to an embodiment of the present invention.

In the deep learning-based Go game service according to an embodiment of the present invention, the time management model server 500 determines the start preparation time, which is one of time management information, and the start model is set based on the preset or determined start preparation time. can get started The time management model server 500 increases the start preparation time to find a good move so that the artificial intelligence computer can make a better move when the game is close to the game between the user and the artificial intelligence computer or the artificial intelligence computer. can make it

More specifically, referring to FIG. 19 , the time management model server 500 obtains a search probability value from the initiation model server 300 ( ), the number of visits (N) or value (V), and the start preparation time (PP) may be provided to the start model server 300. For example, the start preparation time (PP) may include an average start preparation time, a first start preparation time, or a second start preparation time. The first start preparation time may be longer than the average start preparation time, and the second start preparation time may be longer than the first start preparation time. The time management model server 500 may include a time management unit 510 . The time management unit 510 determines the search probability value ( ) and the number of visits (N), the start preparation time (PP) can be determined. The time management unit 510 determines the search probability value ( ), the number of visits (N) is used to calculate the variance, and the start preparation time can be determined by comparing the variance and the critical variance value. That is, the lower the variance of the starting candidate points, the higher the possibility that the starting candidate point is not the best number. Accordingly, when the variance is lower than the critical variance value, the time management unit may make the start preparation time longer so that the start model arrives at a better number. The variance can be obtained using Equations 3 and 4.

(Equation 3)

In Equation 3, n is the number of intersection points, is the number of visits to each intersection, is the search probability for each intersection, is the average number of visits.

(Equation 4)

In Equation 4, Var is the variance.

The time management unit 510 may determine the start preparation time as the first start preparation time when the variance (Var) is lower than the critical variance value, and the start preparation time as the average start preparation time when the variance (Var) is not lower than the critical variance value. can determine

For example, referring to FIG. 20 , the critical variance value may be 0.05. In the case of FIG. 20 (a), the time management unit 510 may calculate the variance (Var) as 0.2109. In this case, the time manager 510 may determine the start preparation time as the average start preparation time since the variance Var is higher than the critical variance value. In the case of FIG. 20 (b), the time management unit 510 may calculate the variance (Var) as 0.0145. In this case, the time manager 510 may determine the start preparation time as the first start preparation time since the variance (Var) is lower than the critical variance value. Accordingly, the initiation model may select a better initiation candidate point by performing MCTS simulations for a longer time or a greater number of times.

In addition, the time management unit 510 may determine the start preparation time (PP) based on the value (V). More specifically, the time management unit 510 may determine the start preparation time PP as the second start preparation time when the value V is less than or equal to the threshold value after determining the first start preparation time. For example, the threshold value may be 50.0%. That is, since the value of the starting candidate point is low, it is likely that the current game is lost, so the starting model needs to perform MCTS simulations for a longer time or a greater number of times so that the time management unit 510 can find a better starting point. is to make it possible

Therefore, the deep learning-based Go game service device according to the embodiment can manage the Go game time. In addition, the deep learning-based Go game service device according to the embodiment may change the starting preparation time in an important phase.

21 is an exemplary diagram of a signal flow in a Go game service system of a time management model server according to an embodiment of the present invention.

Referring to FIG. 21 , the start model server 300 may perform MCTS simulation to find a starting point for winning a game among players (S2101). The initiation model server 300 is a search probability value generated as a result of the MCTS simulation ( ), the number of visits (N), and the value (V) may be transmitted to the time management model server 500 (S2102). The time management model server 500 receives the search probability value ( ), the number of visits (N), and the value (V), the start preparation time may be determined (S2103). The time management model server 500 may transmit the determined start preparation time to the start model server 300 (S2104). The start model server 300 may perform the start based on the received start preparation time or the preset start preparation time (S2105). The start model server 300 may perform the start in priority to the received start preparation time of the received start preparation time and the preset start preparation time.

22 is a method for determining start preparation time among deep learning-based Go game service methods according to an embodiment of the present invention.

Referring to FIG. 22, in the deep learning-based Go game service method according to an embodiment of the present invention, the time management model server 500 has a search probability value ( ), receiving the number of visits (N) or value (V) (S2201).

In the deep learning-based Go game service method according to an embodiment of the present invention, the time management unit of the time management model server 500 sets the time management model server 500 to a search probability value ( ), calculating a variance based on the number of visits (N) (S2202). The method for calculating the variance follows the description of FIGS. 19 and 20 .

The deep learning-based Go game service method according to an embodiment of the present invention may include determining whether the variance calculated by the time management unit of the time management model server 500 is less than a critical variance value (S2203).

In the deep learning-based Go game service method according to an embodiment of the present invention, the time management unit of the time management model server 500 determines the start preparation time as the average start preparation time if the variance is not less than the critical variance value (S2204 ) may be included. The method of determining the start preparation time as the average start preparation time follows the description of FIGS. 19 and 20 .

In the deep learning-based Go game service method according to an embodiment of the present invention, the time management unit of the time management model server 500 determines the start preparation time as the first start preparation time when the variance is less than the critical variance value (S2205 ) may be included. A method of determining the start preparation time as the first start preparation time follows the description of FIGS. 19 and 20 .

In the deep learning-based Go game service method according to an embodiment of the present invention, the time management unit of the time management model server 500 determines the start preparation time as the first start preparation time and determines whether the value is less than or equal to the threshold value Step S2206 may be included. A method of determining whether the value is less than or equal to the threshold value follows the description of FIGS. 19 and 20 . The time management unit may determine the first start preparation time as the start preparation time when the value is not less than or equal to the threshold value.

In the deep learning-based Go game service method according to an embodiment of the present invention, the time management unit of the time management model server 500 determines the start preparation time as the second start preparation time when the value is less than the threshold value ( S2207) may be included. A method of determining the start preparation time as the second start preparation time follows the description of FIGS. 19 and 20 .

The deep learning-based Go game service method according to an embodiment of the present invention may include transmitting the start preparation time determined by the time management unit of the time management model server 500 (S2208).

Therefore, the deep learning-based Go game service method according to an embodiment can manage the Go game time. In addition, the deep learning-based Go game service method according to an embodiment may change the starting preparation time in an important phase.

<Time management model server according to another embodiment>

23 is a diagram for explaining a simulation number estimation unit and a simulation number adjustment model of a time management model server according to another embodiment of the present invention, and FIG. 24 is a deep learning-based Go game service according to another embodiment of the present invention. Among the methods, it is a method for determining the number of MCTS simulations.

Referring to FIG. 23, in the deep learning-based Go game service according to another embodiment of the present invention, the time management model server 500 determines the number of MCTS simulations (C: hereinafter, the number of simulations) performed by the above-described initiation model. can do In addition, the deep learning-based Go game service according to another embodiment of the present invention may allow the starting model to determine the next starting point by performing MCTS based on the determined number of simulations (C).

In detail, the deep learning-based Go game service according to another embodiment of the present invention is a simulation number prediction unit 551 (hereinafter referred to as a number prediction unit), which is a deep learning model of the time management model server 500, and/or a simulation number adjustment model. Based on the number of simulations (C) determined based on (552: hereinafter, number adjustment model), the above-described initiation model may perform MCTS to determine the next initiation point.

Here, the time management model server 500 may include a number prediction unit 551 . The number prediction unit 551 performs a plurality of checkerboard states-optimal simulation based on a plurality of checkerboard states S, a default simulation count (D: hereafter referred to as a default count), a visit count N, and/or a value value V. Count (SC) data can be generated.

Also, the time management model server 500 may include a frequency adjustment model 552 . The number adjustment model 552 may be learned using a plurality of checkerboard state-optimal simulation numbers SC provided by the number prediction unit 551 as a training data set. In addition, the number adjustment model 552 receives a predetermined checkerboard state (S) (in the embodiment, the current checkerboard state (S)) as an input through the above learning, and simulates the number of simulations (C) according to the checkerboard state (S). can be output.

In more detail, the deep learning-based Go game service according to another embodiment of the present invention uses the number prediction unit 551 of the time management model server 500 to optimize the simulation number (C) for a predetermined checkerboard state (S). ) can be determined.

In addition, the deep learning-based Go game service according to another embodiment may learn the number adjustment model 552 using at least one checkerboard state-optimal simulation number (SC) obtained as described above as a training data set.

In addition, the deep learning-based Go game service according to another embodiment inputs the current checkerboard state (S) to the learned number adjustment model 552 as input data, and in the learned number adjustment model 552, the current The number of simulations (C) optimized for the checkerboard state (S) can be output as output data.

In addition, the deep learning-based Go game service according to another embodiment may cause the above-described initiation model to perform MCTS simulation based on the output simulation number C.

Thus, in the deep learning-based Go game service according to another embodiment, the starting model may determine the next starting point by performing MCTS based on the number of simulations (C) determined as above.

Specifically, referring to FIG. 24 , the deep learning-based Go game service method according to another embodiment of the present invention includes a step (S2301) of receiving a predetermined checkerboard state (S) by the time management model server 500. can do.

That is, the time management model server 500 may receive a plurality of notations from the Go server 200. Each notation of a plurality of notations may include each checkerboard state (S) according to the starting order.

25 is an example of deriving the value value (V) and the number of visits (N) in each of the first MCTS and the second MCTS for the checkerboard state (S) according to another embodiment of the present invention.

In addition, referring to FIG. 25, a deep learning-based Go game service method according to another embodiment of the present invention includes the steps of the time management model server 500 performing a first MCTS for the received checkerboard state (S) ( S2302) may be included.

In detail, the time management model server 500 interlocks with the initiation model of the initiation model server 300 to generate MCTS as many times as a preset default number of times (D) (eg, 800 times, etc.) based on the received checkerboard state (S). can do it In more detail, the time management model server 500 may provide the received checkerboard state S to the starting model. At this time, the set-up model receiving the checkerboard state (S) may perform MCTS for the checkerboard state (S) for a predetermined default number of times (D).

In addition, in the deep learning-based Go game service method according to another embodiment of the present invention, the time management model server 500 determines the first number of visits (N1) and/or the first value based on the first MCTS performed as above. (S2303) of storing (V1) may be included.

That is, the time management model server 500 determines the number of initiation candidates (that is, a plurality of nodes in the MCTS) obtained as an output of the MCTS performed by the default number of times (D) for the checkerboard state (S) from the initiation model. The first number of visits N1 and/or the first value V1 may be received and stored.

In addition, the deep learning-based Go game service method according to another embodiment of the present invention may include a step (S2304) of the time management model server 500 performing a second MCTS for the received checkerboard state (S). can

In detail, the time management model server 500 interlocks with the initiation model of the initiation model server 300 to set the default number of times D (eg, 800 times, etc.) based on the received checkerboard state S. At least one second MCTS may be performed while increasing the number of times (eg, 10 times, etc.).

Specifically, the time management model server 500 interlocks with the initiation model and at least one additional unit that increases the default number D (eg, 800 times, etc.) by a predetermined additional number of times (eg, 10 times, etc.) Based on the number of times (eg, 810 times, 820 times, 830 times, etc.), at least one second MCTS may be performed for the received checkerboard state (S).

In addition, in the deep learning-based Go game service method according to another embodiment of the present invention, the time management model server 500 determines at least one second visit number N2 based on the performed second MCTS and/or the second MCTS. A step of storing the value V2 (S2305) may be included.

In detail, the time management model server 500 includes at least one starting candidate number (ie, MCTS The second number of visits (N2) and/or the second value (V2) for each of a plurality of nodes within the node may be received and stored. That is, the second number of visits (N2) may include data on the number of visits (N) for each at least one starting candidate number, and the second value value (V2) is a value value (V for each at least one starting candidate number). ) data can be included.

At this time, the time management model server 500 may stop the repeated second MCTS according to the number of simulations (C) determined below.

In addition, the deep learning-based Go game service method according to another embodiment of the present invention includes the step of determining the optimal number of simulations based on the number of visits (N) and/or the value (V) by the time management model server 500 (S2306) may be included.

In detail, the time management model server 500 operates the number prediction unit 551 to determine the first number of visits (N1), the second number of visits (N2) and/or the first value for a predetermined checkerboard state (S). Based on (V1) and the second value (V2), the number of simulations optimized for the checkerboard state (S) (hereinafter, the optimal number of simulations) may be determined. In this case, the optimal number of simulations may be the number of simulations including the number of simulations (C) that takes the minimum time within a critical point at which the next starting point is not changed during the MCTS simulation. As an example, the optimal number of simulations may be the number of simulations (C) that takes the minimum start preparation time (PP) within a critical point at which the next start point is not changed during the MCTS simulation.

In general, the MCTS performs several simulations to determine the next starting point by selecting the number of starting candidates having the largest number of visits (N) among a plurality of starting candidates (that is, a plurality of nodes in the MCTS). At this time, if the next starting point, that is, the number of starting candidates having the largest number of visits (N) is not changed even if the number of simulations (C) increases, it may not be necessary to increase the number of simulations (C).

Thus, the time management model server 500 according to another embodiment determines whether the next starting point (ie, the selected node) is changed according to the increase in the number of simulations (C) of MCTS performance for a predetermined checkerboard state (S), According to the determination result, the optimal number of simulations for the checkerboard state (S) may be predicted.

In detail, the time management model server 500 may determine whether the next starting point (ie, the selected node) is changed according to an increase in the number of simulations (C) for a predetermined checkerboard state (S).

Specifically, the time management model server 500 may determine whether to change the next starting point based on 1) the number of visits (N) for each starting candidate number.

26 is an example of a diagram for explaining a method of determining the optimal number of simulations based on the number of visits (N) according to another embodiment of the present invention.

Referring to FIG. 26, the time management model server 500 calculates the first MCTS, that is, the first number of visits N1 for each starting candidate number obtained by performing simulation as much as the default number D, and the second MCTS, that is, at least The rate of change between the first visit number N1 and the second visit number N2 based on the second visit number N2 for each at least one start candidate number obtained by performing simulation on the basis of one or more additional unit times. (gradient) can be calculated.

As an embodiment, the time management model server 500 may increase the number of simulations based on the difference between the default number D and the additional unit number, and the number of visits based on the difference between the first number of visits N1 and the second number of visits N2. The rate of change can be calculated using the number of increments. In detail, the time management model server 500 may obtain the change rate by calculating the increase in the number of visits (ie, the increase in the number of visits/the increase in the number of simulations) compared to the increase in the number of simulations. However, this is only one embodiment, and the method for calculating the change rate itself is not limited, and various embodiments may be possible.

Referring further to FIG. 26, for example, the time management model server 500 has a first number of simulations (C1), that is, a default number (D) of 500 for the first number of starting candidates, and a second number of simulations (C2). That is, when the predetermined number of additional units is 1000, the first number of visits (N1) is 400 times, and the second number of visits (N2) is 800 times, the rate of change (i.e., the increase in number of visits (N2-N1)/number of simulations) The increment (C2-C1)) can be calculated as 0.8.

In addition, the time management model server 500 stops increasing the number of simulations (C) when the calculated change rate meets a predetermined criterion (eg, a predetermined constant value (eg, 0.5) or more). The second MCTS may be stopped).

That is, the time management model server 500 determines if the change rate (in the embodiment, the increase in the number of visits (N2-N1)/the increase in the number of simulations (C2-C1)) satisfies a predetermined criterion (eg, a predetermined change rate or more). , Even if the number of simulations C for the predetermined checkerboard state S increases, it can be determined that the next starting point (ie, the selected node) is unchanged. In addition, the time management model server 500 may stop increasing the number of simulations (C) (ie, stop the second MCTS) if it is determined that the next starting point is unchanged. In addition, the time management model server 500 may determine the minimum number of simulations when calculating the change rate (ie, the number of simulations C when the first number of visits N1 is satisfied) as the optimal number of simulations.

Alternatively, the time management model server 500 may determine whether to change the next starting point based on 1) the ratio of the number of visits (N) to the number of starting candidates.

27 is an example of a diagram for explaining a method of determining an optimal number of simulations based on a ratio of the number of visits (N) according to another embodiment of the present invention.

In detail, referring to FIG. 27, the time management model server 500 first visits for each of a plurality of starting candidates (in an embodiment, a plurality of child nodes existing in a predetermined first layer in the MCTS). Based on the ratio between the number of times (N1), it is possible to calculate a first ratio for the plurality of undertaking candidates.

27, for example, the plurality of start candidates includes a first start candidate number and a second start candidate number, the first visit number N1 of the first start candidate number is 400 times, and the When the first visit number N1 of the second starting candidates is 100 times, the first ratio may be 'first starting candidates: 0.8 (400), second starting candidates: 0.2 (100)'.

In addition, the time management model server 500 may sort the plurality of starting candidates in the order of ratio based on the first ratio calculated as above.

For example, the time management model server 500 may arrange '1st priority: the number of candidates to start first, and 2nd priority: the number of candidates to start second' based on the first ratio in the above-described example.

In addition, the time management model server 500 may calculate a second ratio for the plurality of start candidates based on a ratio between the second number of visits N2 for each of the plurality of start candidates.

27, for example, the plurality of start candidates includes a first start candidate number and a second start candidate number, the second visit number N2 of the first start candidate number is 800, and the When the second visit number N2 of the second starting candidates is 200 times, the second ratio may be 'first starting candidates: 0.8 (800), second starting candidates: 0.2 (200)'.

In addition, the time management model server 500 may sort the plurality of starting candidates in a ratio order based on the calculated second ratio.

Illustratively, the time management model server 500 may arrange '1st priority: the number of candidates for the first undertaking, and 2nd priority: the number of candidates for the second undertaking' based on the second ratio in the above-described example.

At this time, the time management model server 500, if the ranks according to the plurality of second ratios obtained by repeatedly performing the above-described second MCTS do not change more than a predetermined number of times compared to the ranks according to the first ratio, the Even if the number of simulations (C) for a predetermined checkerboard state (S) increases, it can be determined that the next starting point (ie, the selected node) is unchanged.

For example, the time management model server 500 determines that the ranking according to the first ratio is 'first priority: the number of candidates for the first start, second priority: the number of candidates for the second start', and a predetermined number of times (eg, 10 times, etc.) ) or more, if the ranking according to the second ratio is maintained as '1st priority: the number of candidates for the first start, 2nd priority: the number of candidates for the second start', the number of simulations (C) for the predetermined checkerboard state (S) increases. It can be determined that the next starting point (i.e., the selected node) is invariant.

In addition, the time management model server 500 may stop increasing the number of simulations (C) (ie, stop the second MCTS) if it is determined that the next starting point is unchanged as described above. In addition, the time management model server 500 may determine the minimum number of simulations (ie, the number of simulations (C) when the first number of visits (N1) is satisfied) as the optimal number of simulations in the ranking comparison.

Therefore, the deep learning-based Go game service device according to another embodiment increases the number of simulations (C) for a predetermined checkerboard state (S) while identifying the corresponding change in the number of visits (N), and based on this The number of simulations (C) optimized for the corresponding checkerboard state (S) may be determined.

On the other hand, in general, when multiple simulations are performed in MCTS, the value value (V) (i.e., the odds ratio) for any one of the multiple start candidates (i.e., multiple nodes in the MCTS) increases biasedly. , it may not be necessary to increase the number of simulations (C) any longer.

Accordingly, in another embodiment, the time management model server 500 may predict the optimal number of simulations for the checkerboard state S based on the first value value V1 and the second value value V2.

28 is an example of a diagram for explaining a method of determining an optimal number of simulations based on a value (V) according to another embodiment of the present invention.

In detail, referring to FIG. 28 , the time management model server 500 determines the gradient between the first value value V1 and one or more second value values V2 based on a predetermined criterion (eg, a predetermined value V2). If the number of simulations (C) is satisfied (e.g., a constant value or more), the increase in the number of simulations (C) may be stopped (ie, the second MCTS is stopped).

At this time, the time management model server 500 increases the number of simulations based on the difference between the default number D and the number of additional units, and increases the value based on the difference between the first value value V1 and the second value value V2. The increase/decrease rate can be calculated using In detail, the time management model server 500 may obtain the increase/decrease rate by calculating the increase in the value compared to the increase in the number of simulations (ie, the increase in value/increase in the number of simulations). However, this is only one embodiment, and the method for calculating the increase/decrease rate itself is not limited, and various embodiments may be possible.

For example, further referring to FIG. 28 , the time management model server 500 has a first number of simulations (C1), that is, a default number (D) of 500 for the first number of starting candidates, and a second number of simulations (C2). ) That is, when the predetermined number of additional units is 1000, the first value value (V1) is 40 and the second value value (V2) is 100, the rate of increase (ie, value increase (V2-V1) / number of simulations) The increment (C2-C1)) can be calculated as 0.12.

In addition, the time management model server 500 stops increasing the number of simulations (C) when the calculated increase/decrease rate satisfies a predetermined criterion (eg, a predetermined constant value (eg, 0.1) or more). 2 MCTS may be suspended).

In addition, the time management model server 500 may determine the minimum number of simulations (ie, the number of simulations C when the first value V1 is satisfied) as the optimal number of simulations when calculating the increase/decrease rate.

Therefore, the deep learning-based Go game service device according to another embodiment increases the number of simulations (C) for a predetermined checkerboard state (S) while identifying the corresponding change in value (V), and based on this The number of simulations (C) optimized for the corresponding checkerboard state (S) may be determined.

In addition, in the deep learning-based Go game service method according to another embodiment of the present invention, the time management model server 500 generates checkerboard state-optimal simulation number (SC) data based on the optimal number of simulations determined as above. (S2307) may be included.

That is, the time management model server 500 mutually matches a plurality of checkerboard states (S) and the optimal number of simulations for each of the plurality of checkerboard states (S) to obtain a plurality of checkerboard states-optimal simulation numbers (SC). can create

In addition, in the deep learning-based Go game service method according to another embodiment of the present invention, the time management model server 500 learns the number adjustment model 552 based on the generated checkerboard state-optimal simulation number (SC) data. It may include a step (S2308).

In detail, the time management model server 500 determines a predetermined checkerboard state (S) (in the embodiment, the current checkerboard state) based on the plurality of checkerboard states-optimal simulation count (SC) data generated by the number prediction unit 551. (S)) as an input and the number of times adjustment model 552 having as an output the number of simulations (C) according to the checkerboard state (S) may be trained. That is, here, the number adjustment model 552 can be learned using a plurality of checkerboard state-optimal simulation numbers (SC) provided by the number prediction unit 551 as a training data set, and the above learning When a predetermined checkerboard state (S) (in the embodiment, the current checkerboard state (S)) is input through, the number of simulations (C) optimized for the checkerboard state (S) can be output.

Therefore, the deep learning-based Go game service device according to another embodiment may train a deep learning model, that is, the number adjustment model 552 to output the simulation number C optimized for the input checkerboard state S. .

In addition, in the deep learning-based Go game service method according to another embodiment of the present invention, the time management model server 500 simulates the number of simulations according to the current checkerboard state (S) based on the learned number adjustment model 552 as above. (C) may include determining (S2309).

That is, the time management model server 500 may operate the learned number adjustment model 552 to determine the number of simulations (C) optimized for the current checkerboard state (S) in the current game.

In detail, the time management model server 500 may input the current checkerboard state (S) to the number adjustment model 552 as input data. In addition, the time management model server 500 may obtain the number of simulations (C) optimized for the current checkerboard state (S) as output data from the number adjustment model 552 receiving the current checkerboard state (S). . That is, the time management model server 500 uses the number adjustment model 552 learned based on a plurality of checkerboard state-optimal simulation number (SC) data to optimize the number of simulations (C) for the current checkerboard state (S). ) can be determined.

Therefore, the deep learning-based Go game service apparatus according to another embodiment may dynamically adjust the number of simulations (C) during MCTS to be optimized for the Go board state (S). In addition, the deep learning-based Go game service device according to another embodiment may change the number of simulations (C) in an important phase based on the state (S) of the go board.

Here, in order to improve the accuracy and performance of the number of simulations (C) determined by the number adjustment model 552, the time management model server 500 according to another embodiment: A lower limit may be preset.

In detail, the time management model server 500 may determine the number of simulations (C) within a predetermined upper limit and lower limit. For example, the time management model server 500 may preset the upper limit of the number of simulations (C) to '3200' and the lower limit to '800', which is the default number of times (D).

In addition, the time management model server 500 may 2) randomly increase the number of simulations (C) with a constant probability. That is, the time management model server 500 may increase the reliability of the number of simulations (C) determined through deep learning by probabilistically increasing the number of simulations (C) by a predetermined number of times.

In addition, the time management model server 500 may 3) adjust the number of simulations (C) based on the win rate and/or change in the win rate for the current checkerboard state (S).

In detail, the time management model server 500 determines the number of simulations (C) when the odds ratio (ie, value (V)) for the current checkerboard state (S) meets a predetermined criterion (eg, less than or equal to a predetermined value). can increase That is, the time management model server 500 may increase the number of simulations (C) by a predetermined number of times when it is determined that the situation is inferior based on the current odds ratio.

Alternatively, the time management model server 500 may increase the number of simulations (C) based on the change in odds ratio for the current checkerboard state (S).

29 is an example of a diagram for explaining a method of adjusting the number of simulations (C) based on a change in odds according to another embodiment of the present invention.

In detail, referring to FIG. 29 , the time management model server 500 sequentially receives, accumulates, and stores a plurality of value values (V) obtained for each start in the current game in progress from the start model server 300. . Also, the time management model server 500 may determine a change in odds based on the accumulated value (V).

Specifically, the time management model server 500 may grasp a trend and/or a range of change of the value value V based on the accumulated value value V. In addition, the time management model server 500 may determine a change in the odds ratio for the accumulated value (V) based on the identified trend and/or range of change in the value (V).

Here, the change rate of the value V may mean a rate of change of the value V indicating whether the value V (ie, win rate) is gradually increasing or decreasing. In detail, the change trend of the value value (V) represents the change state of a plurality of value values (V) from a predetermined start time to the current start time after the start of the game. More specifically, when the rate of change for each of the plurality of value values (V) is a negative number more than a predetermined number of times, that is, if the value value (V) decreases more than a predetermined number of times, the value value (V) gradually decreases, It can be judged that there is a situation. On the other hand, when the rate of change for each of the plurality of value values (V) is a positive number more than a predetermined number of times, that is, if the value value (V) increases more than a predetermined number of times, the value value (V) is gradually increasing. can be judged. In addition, the change trend of the value value (V) may be determined by the slope of the change rate of the corresponding value value (V). For example, referring to (a) of FIG. 29, the trend of change of the value value V is the change rate and/or slope (ie, the value value V) of the value value V for the recently launched number 5. is rising or whether the value (V) is falling).

On the other hand, the variation range of the value value (V) may mean the amount of change (degree of change) of the value value (V). The variation range of this value value (V) may be obtained based on the difference between the value value (V) at a predetermined first start time point and the value value (V) at a second start time point after the first start time point. At this time, the fluctuation range of the value value (V) is determined that the winning rate has decreased when the difference value obtained by subtracting the value value (V) at the first start point from the value value (V) at the second start point is a negative number. can In addition, when the difference value having a negative number is greater than or equal to a predetermined value, the fluctuation range of the value value V may determine that the winning rate has rapidly decreased and detect this as an important phase. For example, referring to (b) of FIG. 29, the time management model server 500 sets the difference between the value values V between the recently launched two numbers (eg, 64 numbers and 66 numbers) to '35% - 58% = -23%'. In addition, the time management model server 500 may determine whether the range of decrease in the value V is greater than or equal to a predetermined standard based on the calculated difference.

In addition, the time management model server 500 determines the change in the odds ratio for the plurality of values V determined based on the change trend and / or change range of the value value V based on a predetermined criterion (for example, If it is determined that the situation is satisfied (e.g., when the slope of the rate of change of the value value (V) is negative for more than a predetermined number of times, or when the range of decline and variation of the value value (V) is greater than or equal to a predetermined standard), the number of simulations (C) is determined. It can be increased by a predetermined number of times.

That is, the time management model server 500 determines the change in win rate based on the value value (V) obtained during the game, and determines that the win rate gradually decreases or significantly decreases as an important phase and calculates the number of simulations (C). It can be increased by a predetermined number of times.

Therefore, the deep learning-based Go game service device according to another embodiment increases the number of simulations (C) in an important phase in which the winning rate is lowered during a match, and performs more MCTS simulations to determine a better next starting point. can do.

In addition, the time management model server 500 may increase the number of simulations (C) for the checkerboard state (S) at the time when the winning rate drops significantly after 4) the game is completed.

In detail, the time management model server 500 may detect a time point when the value (V) drops with a large fluctuation range after the game ends. In addition, the time management model server 500 may increase the number of simulations C for the checkerboard state S at the detected time by a predetermined number of times (eg, a predetermined number or more). In addition, the time management model server 500 may determine the increased number of simulations (C) as the optimal number of simulations for the checkerboard state (S). In addition, the time management model server 500 may generate checkerboard state-optimal simulation number (SC) data that pairs the checkerboard state (S) with the determined optimal number of simulations. In addition, the time management model server 500 may train the above-described number adjustment model 552 by using the checkerboard state-optimal simulation number (SC) data generated as above as training data.

That is, the time management model server 500 learns the number adjustment model 552 using a training data set in which the number of simulations (C) is increased at the time when the winning rate drops significantly after the game is completed. The number of simulations (C) in the checkerboard state (S) may be increased.

In addition, the time management model server 500 may adjust the number of simulations (C) based on statistics based on 5) a plurality of games played on the Go game service.

In detail, the time management model server 500 interlocks with the Go server 200 and/or the start model server 300, etc. to obtain statistics according to the relationship between the total number of simulations (C) and the odds ratio for each of the plurality of games played on the Go game service. can be obtained. For example, the time management model server 500, based on the total number of simulations (C) for each of a plurality of large countries and the resultant win or loss data, when the total number of simulations (C) is more than a predetermined number of times, the win rate is a predetermined percentage It is possible to obtain a statistic of an abnormal increase.

In addition, the time management model server 500 may adjust the number of simulations (C) calculated by the number adjustment model 552 based on the obtained statistical value. In an embodiment, the time management model server 500 may adjust the number of simulations (C) calculated by the number adjustment model 552 to satisfy the number of simulations (C) that satisfies the highest win rate in the statistical value. For example, when the time management model server 500 determines that the winning rate increases the most when the total number of simulations (C) in one major country is greater than or equal to 100,000 and less than or equal to 200,000 from the above statistics, the total number of simulations (C) The number of simulations (C) calculated by the number adjustment model 552 may be increased or decreased so as to be satisfied.

Therefore, the deep learning-based Go game service device according to another embodiment can predict the optimal number of simulations (C) from a multi-view point of view.

In addition, the deep learning-based Go game service method according to another embodiment of the present invention may include a step of providing, by the time management model server 500, the number of simulations (C) determined as above as a starting model (S2310). .

30 is a conceptual diagram for explaining an initiation model for performing MCTS based on the number of simulations (C) according to another embodiment of the present invention.

In detail, referring to FIG. 30 , the time management model server 500 may provide the number of simulations (C) determined based on the number adjustment model 552 as an initiation model. Thus, the time management model server 500 may allow the initiation model to perform MCTS simulation to determine the next initiation point based on the number of simulations (C) determined by the learned number adjustment model 552 . For example, when the number of simulations (C) determined by the number adjustment model 552 is '500', the time management model server 500 may provide the number of simulations (C) to the above-described initiation model. And, the initiation model can determine the next initiation point by performing MCTS simulations as much as '500 times', which is the number of simulations received (C).

Therefore, the deep learning-based Go game service apparatus according to another embodiment may determine the next start by performing MCTS based on the optimal number of simulations (C) for the Go board state (S).

<Time management model server according to another embodiment>

31 is a diagram for explaining a time management model of the time management model server 500 according to another embodiment of the present invention, and FIG. 32 is used to generate game time information according to another embodiment of the present invention. FIG. 33 is a diagram for explaining the amount of change in the collection used to generate game time information according to another embodiment of the present invention, and FIG. 34 is another embodiment of the present invention. It is a diagram for explaining common multiples used to generate game time information according to .

Referring to FIG. 33, the deep learning-based Go game service according to an embodiment of the present invention may generate game time information in the current checkerboard state (S) using the time management model of the time management model server 500. . The game time information may include the remaining game length predicted until the end of the game according to the checkerboard state (S). The time management model is a deep learning model of the time management model server 500 and may include a first time management neural network 520 , a second time management neural network 530 , and a first input feature generator 540 .

The time management model may perform supervised learning to predict the remaining game length, which is game time information of the current checkerboard state (S). More specifically, the time management model server 500 generates a training data set for predicting the remaining game length according to the checkerboard state (S), and uses the created training data set to determine the time management model in the current checkerboard state (S). It can be trained to predict the length of the remaining game. The time management model server 500 may receive a plurality of notations from the Go server 200 . Each notation of a plurality of notations may include each checkerboard state (S) according to the starting order. In addition, each notation of the plurality of ups and downs may include game time information, in particular, remaining game length information until the end of the game in each checkerboard state (S). Also, the time management model server 500 may receive situation determination information from the situation determination model server 400 . The situation judgment information is situation judgment information based on a plurality of notations provided from the Go server 200 to the time management model server 500, and may include change amount information and common multiple information of collections. Also, the time management model server 500 may receive a value from the undertaking model server 300 . The value may be a value based on a plurality of notations provided by the Go server 200 to the time management model server 500.

The first input feature extractor 540 may extract the first input feature IF1 from the checkerboard state S of a plurality of notations and provide the first input feature IF1 to the time management first neural network 520 as input data for training. The first input feature (IF) of the checkerboard state (S) is the position information of the stone for the last 8 moves of the black player, the position information of the stone for the last 8 moves of the white player, and the turn information on whether the current player is black or white. It may be a 19*19*18 RGB image including . For example, the first input feature extractor 540 may have a neural network structure and may include a kind of encoder.

In addition, the time management model may provide the second input feature IF2 and the third input feature IF3 to the first time management neural network 520 as input data for training. The second input feature IF2 may be change amount information of the catchment according to the checkerboard state S. The third input feature IF3 may be common multiple information according to the checkerboard state S. The time management model may provide the fourth input feature IF4 to the second time management neural network 530 as input data for training. The fourth input feature IF4 may be a value according to the checkerboard state S.

The first time management neural network 520 may provide an output value to the second time management neural network 530 by using the first to third input features IF1 to IF3 as inputs. The first time management neural network 520 may have a neural network structure. For example, the first time management neural network 520 may include 20 residual blocks. Referring to FIG. 8, one residual block includes 256 3X3 convolution layers, batch normalization layers, Relu activation function layers, 256 3X3 convolution layers, batch normalization layers, skip connections, Relu activation function layers may be arranged in order. The batch normalization layer is intended to prevent covariate shift, a phenomenon in which the distribution of inputs of the current layer changes due to changes in parameters of the previous layer during learning. The skip connection prevents the performance of the neural network from decreasing even when the block layer becomes thicker, and increases overall neural network performance by making the block layer thicker. The skip connection may be input to a second Relu activation function layer by summing the first input data of the residual block with the output of the second batch normalization layer.

The second time management neural network 530 may generate game time information about the predicted remaining game length by taking the output value of the first time management neural network 520 and the fourth input feature IF4 as inputs. The time management second neural network 530 may have a neural network structure. For example, the second time management neural network 530 may have a fully connected layer structure.

The time management model uses the first to fourth input features (IF1 to IF4) as input data and the remaining game length information as target data ( The output data (r) generated from the second time management neural network 530 through the first time management neural network 520 using the training data set of ) is the target data ( ), the first time management neural network 520 and the second time management neural network 530 may be sufficiently learned. As an example, the time management model predicts the remaining match length loss ( ) using remaining game length prediction loss ( ) can be trained to be a minimum. For example, remaining match length prediction loss ( ) may follow Equation 5.

(Equation 5)

in Equation 5 is the change loss of the catchment. Remaining match length prediction loss ( ) is the change in catchment loss ( ) is used. The reason for using the change in the catchment to predict the length of the remaining game is that the change in the catchment at the beginning of the game is relatively large and the change in the catchment in the second half is likely to be relatively small, so it can be used as a factor in determining the prediction of the remaining game length. am. As an example, referring to FIG. 32 , FIG. 32 is layout judgment information generated using the layout judgment model server 400 for the checkerboard state (S) at the beginning of any notation. In Fig. 32(a), in the case of up to 104 moves, black has 46 houses and white has 63.5 houses. 32(b) shows the case where black has 105 moves with one more move, black has 46 houses and white has 59.5 houses, so in the case of white, there is a difference of 4 houses. That is, in the checkerboard state (S) at the beginning of the game, the amount of change in the number of catches is large every two moves. Referring to FIG. 33 , FIG. 33 is position judgment information generated using the position determination model server 400 for the checkerboard state (S) of the second half of an arbitrary notation. In Figure 33(a), in the case of up to 222 moves, black is 64 houses and white is 63.5 houses. 33(b) shows the case where White has one more move, up to 223 moves. Black has 63 houses and White has 64.5 houses, so in the case of White, there is a difference of one house. That is, in the checkerboard state (S) in the second half, the amount of change in the catchment is small for every two moves.

in Equation 5 is the common multiple loss. Remaining match length prediction loss ( ) is the common multiple loss ( ) is used. The reason why the common multiple is used to predict the remaining game length is that it can be used as a factor in determining the remaining game length prediction as the common multiple gradually decreases toward the second half of the game. As an example, referring to FIG. 34 , FIG. 34 is the layout judgment information generated using the layout judgment model server 400 for the checkerboard state (S) of the first and second half of an arbitrary notation. Gongbae is a place marked by an X with a red line in the situation judgment result of FIG. 34 . Figure 34 (a) is the first half of the match, so there are many public cups. Figure 34 (b) is the second half of the game, so there are few public games.

in Equation 5 is the value loss. Remaining match length prediction loss ( ) is the value loss ( ) is used. The reason why the value value is used to predict the length of the remaining game is that it can be used as an element of predicting the length of the remaining game by using this value because the value of one side increases toward the second half of the game. In addition, value loss ( ) can be used later in the game because there is no significant change in the early stages. For example, the second half of the game may be a checkerboard state (S) in which more than 250 numbers have been set out in any notation, but is not limited thereto.

In Equation 5, are hyperparameters. The user can adjust the relative importance of each loss by adjusting the hyperparameters. for example, The value may increase as the importance increases towards the later part of the game, and the value may decrease due to its low importance in the early part of the game.

The learned time management model may generate game time information about the predicted remaining game length when the current checkerboard state (S) is input during a game. More specifically, in the learned time management model, the first input feature may be extracted by the first input feature extractor 540 when the current checkerboard state (S) is input during a game. The time management model may use change amount information of collections provided by the situation judgment model server as a situation judgment for the current checkerboard state (S) as a second input feature, and common multiple information as a third input feature. The time management model may provide an output value generated by the first time management neural network 520 to the second time management neural network 530 by using the first to third input characteristics as input data. The time management model may use a value for a starting candidate point in the current checkerboard state S provided by the starting model server as a fourth input feature. The time management model may generate game time information about the remaining game length predicted by the second time management neural network 530 by using the output value of the first time management neural network 520 and the fourth input feature as input data.

In addition, the time management model server 500 may adjust the starting preparation time according to the predicted remaining game length of the game time information. The time management model server 500 may provide the adjusted start preparation time to the start model server 300 .

Accordingly, the deep learning-based Go game service device according to another embodiment may predict the remaining game length. In addition, the deep learning-based Go game service apparatus according to another embodiment can effectively divide the start preparation time by using the predicted remaining game length.

35 is an exemplary diagram of a signal flow in the Go game service system of the time management model server 500 according to another embodiment of the present invention.

Referring to FIG. 35, the Monarch server 200 may transmit a plurality of notations to the starting model server 300, the situation judgment model server 400, and the time management model server 500 (S2701). The time management model server 500 may extract the first input feature of the checkerboard state (S) of the plurality of notations received (S2702). The situation judgment model server 400 may generate situation judgment information of a plurality of notations received and transmit the situation judgment information to the time management model server 500 (S2703 and S2704). The situation determination information may be information on the amount of change in the number of catchments and information on common multiples. The time management model server 500 may input the variation information of collections as the second input feature and the common multiple information as the third input feature to the time management first neural network of the time management model (S2705). Embarkation model server 300 may generate a value value according to the checkerboard state (S) of a plurality of notation, and transmit the generated value value to the time management model server 500 (S2706, S2707). The time management model server 500 may input the received value as a fourth input feature to the second neural network for time management of the time management model (S2708). The time management model server 500 may train a time management model using the input data of the first to fourth input characteristics and the remaining game length information according to the checkerboard state (S) of a plurality of notations as target data (S2709). . The Go server 200 proceeds with the Go game, and the terminal 100 and the start model server 300 may perform a start on their turn (S2710 to S2712). The layout judgment model server 400 extracts the input features of the current checkerboard state (S), the layout judgment model, which is a deep learning model, generates a layout value using the input features, and uses the checkerboard status (S) and the layout values. Thus, the situation can be judged (S2713). The time management model server 500 extracts the input feature of the current checkerboard state (S) as the first input feature, sets the situation judgment information as the second and third input features, and provides from the start model server 300 A time management model, which is a deep learning model, may generate game time information by using the value value as a fourth input feature (S2714).

36 is a method for generating game time information among deep learning-based Go game service methods according to another embodiment of the present invention.

Referring to FIG. 36, the deep learning-based Go game service method according to another embodiment of the present invention may include a step (S2801) of receiving a plurality of notations by the time management model server 500. A description of a plurality of notations follows the description of FIG. 31 .

Deep learning-based Go game service method according to another embodiment of the present invention includes extracting a first input feature from the checkerboard state (S) of a plurality of notations received by the time management model server 500 (s2802). can include A method of extracting the first input feature follows the description of FIG. 31 . The time management model server 500 may provide the extracted first input feature as input data of the first time management neural network.

The deep learning-based Go game service method according to another embodiment of the present invention includes a step (S2803) of receiving, by the time management model server 500, position judgment information according to the checkerboard state (S) of a plurality of notations. can A method of receiving situation determination information follows the description of FIG. 31 . The situation determination information may be information on the amount of change in the number of catchments and information on common multiples. The time management model server 500 may use change amount information of collections as a second input feature and common multiple information as a third input feature.

A deep learning-based Go game service method according to another embodiment of the present invention includes inputting, by the time management model server 500, a second input feature and a third input feature to a first time management neural network (S2804). can do.

The deep learning-based Go game service method according to another embodiment of the present invention may include a step of receiving, by the time management model server 500, a value from the start model server (S2805). The time management model server 500 may use a value as a fourth input feature.

The deep learning-based Go game service method according to another embodiment of the present invention may include inputting, by the time management model server 500, a fourth input characteristic to a second time management neural network (S2806).

In the deep learning-based Go game service method according to another embodiment of the present invention, the time management model server 500 determines the remaining games according to the input data of the first to fourth input characteristics and the checkerboard state (S) of a plurality of notations. A step of training a time management model using length information as target data (S2807) may be included. The training method of the time management model follows the description of FIGS. 31 to 34 .

In the deep learning-based Go game service method according to another embodiment of the present invention, the time management model server 500 includes a step of receiving a checkerboard state (S), situation judgment information, and a value value during a game (S2809). can do.

A deep learning-based Go game service method according to another embodiment of the present invention includes the step of generating, by the time management model server 500, game time information by using the received checkerboard state (S), situation judgment information, and value. (S2810) may be included. A method of generating game time information follows the description of FIG. 31 .

Therefore, the deep learning-based Go game service method according to another embodiment can predict the remaining game time. In addition, the deep learning-based Go game service method according to another embodiment can effectively divide the start preparation time using the predicted remaining game time.

Embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. medium), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes generated by a compiler. A hardware device may be modified with one or more software modules to perform processing according to the present invention and vice versa.

Specific implementations described in the present invention are examples and do not limit the scope of the present invention in any way. For brevity of the specification, description of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection of lines or connecting members between the components shown in the drawings are examples of functional connections and / or physical or circuit connections, which can be replaced in actual devices or additional various functional connections, physical connection, or circuit connections. In addition, if there is no specific reference such as “essential” or “important”, it may not be a component necessarily required for the application of the present invention.

In addition, the detailed description of the present invention described has been described with reference to preferred embodiments of the present invention, but those skilled in the art or those having ordinary knowledge in the art will find the spirit of the present invention described in the claims to be described later. And it will be understood that the present invention can be variously modified and changed without departing from the technical scope. Therefore, the technical scope of the present invention is not limited to the contents described in the detailed description of the specification, but should be defined by the claims.

100 terminals
200 Go server
300 launch model server
310 search unit
320 Self Play Department
330 Initiate Neural Networks
400 Layout Model Server
410 Neural Networks
420 input feature extraction unit
430 Correct Answer Label Generation Unit
500 hour management model server
510 Time Management Division
520 Time Management First Neural Network
530 Time Management Secondary Neural Network
540 first input feature extraction unit
551 simulation count prediction unit
552 simulation count adjustment model

Claims (12)

a communication unit receiving at least one of a checkerboard state, value, number of visits, and number of default simulations;
a memory for storing a simulation number prediction unit and a simulation number adjusting model; and
reading the number prediction unit to determine an optimal number of simulations for the checkerboard state using at least one of the checkerboard state, the value value, the number of visits, and the default simulation number;
Learning the number adjustment model based on the determined optimal number of simulations;
Characterized in that it includes; a processor for reading the number adjustment model and determining the number of simulations according to the current checkerboard state,
The optimal number of simulations is,
The next starting point selected during Monte Carlo Tree Search (MCTS) simulation includes the number of simulations that take the minimum time within the critical point that does not change,
The number prediction unit,
Matching the determined optimal number of simulations with the checkerboard state to generate checkerboard state-optimal simulation number data;
the processor,
Learning the number adjustment model using the generated checkerboard state-optimal simulation number data as a training data set
Simulation count management device. delete delete According to claim 1,
The number prediction unit,
Obtaining at least one of a first visit number and a first value value for each of a plurality of start candidates based on a first MCTS simulation performed based on the checkerboard state based on the default simulation number,
Based on the additional unit count obtained by adding a predetermined number to the default simulation count, at least one of the second visit count and second value value for each of the plurality of starting candidates based on the second MCTS simulation performed based on the checkerboard state. to acquire
Simulation count management device. According to claim 4,
The number prediction unit,
Based on the first number of visits and the second number of visits, it is determined whether the next starting point is changed when the number of simulations increases,
Determining the number of simulations that take the minimum time within the critical point where the next starting point is not changed as the optimal number of simulations
Simulation count management device. According to claim 5,
The number prediction unit,
Determine whether the next starting point is changed based on a rate of change based on the number of first visits and the number of second visits;
The rate of change is
Calculated based on the increase in the number of visits based on the difference between the number of first visits and the number of second visits compared to the increase in the number of simulations based on the difference between the default simulation number and the number of additional units
Simulation count management device. According to claim 5,
The number prediction unit,
Based on the ratio between the number of first visits for each of the plurality of starting candidates and the ratio between the number of second visits for each of the plurality of starting candidates, determining whether the next starting point is changed
Simulation count management device. According to claim 4,
The number prediction unit,
Determine whether the next starting point is changed when the number of simulations increases based on an increase/decrease rate based on the first value and the second value;
The rate of increase is
Calculated based on the value increase based on the difference between the first value and the second value compared to the increase in the number of simulations based on the difference between the default simulation number and the additional unit number
Simulation count management device. According to claim 8,
The number prediction unit,
When the increase/decrease rate satisfies a predetermined criterion, determining the minimum number of simulations when calculating the increase/decrease rate as the optimal number of simulations.
Simulation count management device. ◈Claim 10 was abandoned when the registration fee was paid.◈ According to claim 1,
The number adjustment model,
Perform an adjustment process for the number of simulations according to the current checkerboard state;
The adjustment process is
The process of setting at least one of the upper limit and the lower limit of the number of simulations, the process of increasing the number of simulations with a predetermined probability, and the number of simulations based on at least one of a win rate and a change in win rate according to the current checkerboard state. comprising at least one of a process of adjusting and a process of adjusting the number of simulations based on predetermined statistics about the number of simulations.
Simulation count management device. ◈Claim 11 was abandoned when the registration fee was paid.◈ According to claim 10,
the processor,
Providing the number of simulations according to the current checkerboard state to an initiation model that performs the MCTS simulation so that the initiation model performs the MCTS simulation based on the number of simulations
Simulation count management device. As a method for managing the number of deep learning-based Go game service simulations in a time management model server,
receiving a predetermined checkerboard state;
performing a first MCTS based on the received checkerboard state;
obtaining at least one of a first number of visits and a first value based on the first MCTS performed;
performing a second MCTS based on the received checkerboard state;
obtaining at least one of a second visit number and a second value based on the performed second MCTS;
Determining the optimal number of simulations for the checkerboard state based on the first visit number and the second visit number, or determining the optimal number of simulations for the checkerboard state based on the first value and the second value step;
generating checkerboard state-optimal simulation number data by matching the determined optimal simulation number with the checkerboard state;
learning a deep learning model based on the generated checkerboard state-optimal simulation number data;
determining the number of simulations according to the current checkerboard state based on the learned deep learning model; and
Providing the determined number of simulations as an initiation model
How to manage the number of simulations.