KR20210008757A - Method and device for baduk game service based on deep-learning - Google Patents

Abstract

According to an embodiment of the present invention, a method and an apparatus for providing a Go game service based on deep learning are provided. The apparatus comprises: a communication unit for receiving at least two candidate stone checkers based on a checkerboard state and a plurality of game records; a storage unit for storing a situation determination model and a style determination unit; and a processor which reads the situation determination model to learn the situation determination model, determines a situation of a checkerboard, to which the candidate stone checker is applied, by using the learned situation determination model, and generates style determination information, which is information obtained by determining, by the style determination unit, the style with respect to the candidate stone checker based on the determined situation. The situation determination model includes: an input feature extraction unit for extracting input features from the inputted checkerboard state; and a situation determination neural network which generates a situation value for an intersection of the input checkerboard state based on the extracted input feature. The style determination unit is a situation determination model server which generates information about points in territory based on the situation value, wherein the information about points in territory is information obtained by calculating the number of player points in territory and the number of opponent points in territory.

Description

Game service method and device thereof based on deep learning {METHOD AND DEVICE FOR BADUK GAME SERVICE BASED ON DEEP-LEARNING}

The present invention relates to a method and apparatus for providing a Go game service based on deep learning. In more detail, the present invention relates to a method and apparatus for providing a Go game service using an ethos representing a specific temperament or method when playing a Go game by determining the status of Go based on a deep learning neural network.

As the use of user terminals such as smartphones, tablet PCs, PDAs (Personal Digital Assistant), notebooks, etc. became popular and information processing technology developed, it became possible to play Go, a kind of board game, using user terminals. It became possible to play a game of Go with a programmed artificial intelligence computer.

When compared to other board games such as chess or shogi, there are a lot of cases in Baduk, so there is a limit to the ability of artificial intelligence computers to play at the level of humans, and research is being actively conducted to increase the energy of artificial intelligence computers. Recently, developers have applied the Monte Carlo Tree Search (MCTS) algorithm and deep learning technology to artificial intelligence computers to raise the energy of artificial intelligence computers to the level of professional engineers.

However, in general, since the existing Go AI program only learns how to win, it is difficult to start taking into account the difference in score between the player and the opponent. In other words, the existing Go AI program has a problem that the score difference gradually narrows toward the latter half of the Go game play, and the tendency to win with a very small score difference (defensive tendency) is remarkable.

In addition, the existing Go AI program has difficulty in realizing the player's specific temperament or ethos that is implemented during actual Go game play. Here, ethos refers to a unique way of playing or temperament that appears when playing a game such as Go or Shogi. In other words, the existing Go AI program is learned to win and implemented only with a consistent play method, so there is a limit to playing the Go game with various ethos.

In addition, in order to play an effective Go game with an artificial intelligence computer, it is necessary to precisely adjust the game difficulty according to the player's needs or ability, but the technology to implement this is insufficient, and a new technology is required.

Patent Document 1: Unexamined Patent Publication No. 10-2015-0129265

The present invention relates to a method and apparatus for providing a Go game service based on deep learning conceived to solve the above-described problem. In more detail, it is intended to propose a method and apparatus for providing a Go game service that utilizes an ethos representing a specific temperament or method when playing a Go game by determining the status of Go based on a deep learning neural network.

In detail, the present invention determines the state of Go based on predictions of houses, sandstones, stones, gongbae, and big according to the Go rule, determines the ethos of Go, and learns based on this, to determine the game where the ethos is applied when playing the Go game. The purpose of this is a deep learning based Go game service providing method and apparatus.

However, the technical problems to be achieved by the present invention and the embodiments of the present invention are not limited to the technical problems as described above, and other technical problems may exist.

A method and apparatus for a Go game service based on deep learning according to an embodiment of the present invention include: a communication unit for receiving at least two or more embarkation candidates based on a plurality of notations and a board state; A storage unit for storing a situation determination model and an ethos determination unit; And reading the situation determination model to perform learning of the situation determination model, and using the learned situation determination model to determine the configuration of the board to which the number of candidates is applied, and the ethos determination unit based on the determined situation. And a processor for generating ethos determination information, which is information for determining ethos of the number of candidates for embarkation; And a situation determination neural network that generates a position value for the intersection of the input checkerboard state based on the extracted input feature, wherein the ethos determination unit generates information on the holding house based on the position value, and the The holding house information is information obtained by calculating the number of player houses and the number of owner houses.

At this time, the ethos determination unit generates the holding house information by determining the house area of the player and the owner using the position value, a predetermined threshold value, and the presence or absence of a stone.

In addition, the air wind determination unit sets target air wind information for determining the air wind to be learned by the initiating model.

In addition, the ethos determination unit generates the ethos determination information based on the number of embarkation candidates and the target ethos information, calculates a player score value and an oponant score value based on the holding house information, and calculates the calculated player A score difference value is generated based on the score value and the oponant score value.

In addition, the airflow determination unit sets an airflow determination threshold that is a parameter of the airflow determination process for the number of launching candidates, and compares the score difference value with the airflow determination threshold, and the airflow determination information for each of the launching candidates Create

In addition, the airflow determination unit generates airflow learning data that assists self-learning of the launch model based on the airflow determination information.

In addition, the communication unit obtains the ethos learning performance information from the initiation model,

The airflow determination unit adjusts the airflow determination threshold based on airflow learning performance information obtained from the initiation model.

On the other hand, a method and apparatus for a game of Go game service based on deep learning according to an embodiment of the present invention include: a communication unit for receiving ethos learning data; A storage unit for storing the starting model; And a processor that reads out the initiation model, performs training of the initiation model, and generates two or more initiation candidates based on a checkerboard state using the learned initiation model, wherein the initiation model is a Monte Carlo tree Search (Monte Carlo Tree Search; MCTS) based on the search unit for providing the number of candidates to start; and, an initiation neural network guiding the search unit; and a self-play unit for self-learning the initiation neural network by performing self-play. ; And an ethos learning assistant for supporting the self-learning based on the ethos learning data, wherein the self-play unit performs a Go game between the higher version launching model and the lower version launching model learned based on the ethos learning data. .

In this case, the airflow learning assistant generates airflow learning performance information, which is information that diagnoses airflow determination information based on the result of the self-learning.

In addition, the method and apparatus for a Go game service based on deep learning according to an embodiment of the present invention include a communication unit, a storage unit in which a situation determination model and an ethos determination unit are stored, a processor that drives the situation determination model and the ethos determination unit. A deep learning-based Go game service method of determining an ethos by determining an ethos by determining a condition of a board state by a condition judgment model server comprising: the processor setting target ethos information; Setting, by the airflow determination unit, an airflow determination threshold; Obtaining, by the communication unit, two or more embarkation candidates based on a checkerboard state; Determining, by the processor, a situation in a checkerboard state to which the number of candidates for launch is applied using the situation determination model; Calculating, by the processor, a score difference value for each of the number of candidates for launch based on the determined situation using the ethos determination unit; Generating, by the processor, air-wind determination information for each of the number of candidates for launch based on the calculated score difference value and the air-wind determination threshold using the air-wind determination unit; And generating and transmitting, by the communication unit, the airflow learning data for each of the embarkation candidates based on the generated airflow determination information and the target airflow information, wherein the airflow determination information comprises: This is information that determines the category of the ethos implemented by the launch.

In this case, the method further includes receiving information on the airflow learning performance from the launching model server, wherein the airflow learning performance information is information obtained by diagnosing the airflow determination information by the launching model server based on the airflow learning data.

In addition, the method and apparatus for a Go game service based on deep learning according to an exemplary embodiment of the present invention further include adjusting the airflow determination threshold based on the airflow learning performance information.

In addition, the step of calculating the score difference may include generating house information, which is information obtained by calculating the number of player houses and the number of offense houses by deriving a position value based on the position determination for each of the number of candidates to be launched; and And generating the score difference value by calculating a player score value and an offense score value based on the holding house information.

In addition, the category of the ethos includes an aggressive ethos, a stable ethos, and a defensive ethos, and the aggressive ethos is a case where the score difference value is greater than the efferent determination threshold, and the stable ethos is the score difference value It is the same as the judgment threshold, and the defensive ethos is a case where the score difference value is less than the ethos judgment threshold.

The method and apparatus for a Go game service based on deep learning according to an embodiment of the present invention accurately classify houses, sandstones, stones, gongbae, and big according to the Go rule to predict the situation of Go, thereby It has the effect of accurately judging the ethos of the starting point.

In addition, the Go game service method and apparatus thereof based on deep learning according to an embodiment of the present invention promotes the progress of the game based on various ethos when playing the Go game, thereby providing a game of Go based on a diversified play method. It can be performed and the difficulty of the game can be precisely adjusted, and through this, the quality and interest of the Go game can be improved.

In addition, the deep learning-based Go game service method and device thereof according to an embodiment of the present invention determine not only the winning but also the score by determining the ethos based on the difference between the score between the player and the oponant, and performing a match based on this. There is an effect that can perform the game play of Go considering the difference of.

In addition, the deep learning-based Go game service method and device thereof according to an embodiment of the present invention can efficiently and systematically implement learning for implementing the ethos by operating to perform learning optimized for a specific ethos of a target. It can have an effect.

However, the effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood from the following description.

1 is an exemplary diagram for a deep learning-based Go game service system according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a structure of a launch model server for launching an artificial intelligence computer in a deep learning-based Go game service according to an embodiment of the present invention.
3 is a diagram for explaining a distribution of a movement probability for a starting point according to a policy of a starting model.
4 is a diagram for explaining the value of the starting point and the number of visits of the starting model.
5 is a diagram for explaining a process in which the launch model starts according to the pipeline of the search unit.
6 is an exemplary view showing a screen providing a function of determining the status of a deep learning-based Go game service according to an embodiment of the present invention.
7 is a view for explaining the structure of the situation determination model of the situation determination model server of the present invention.
8 is a diagram for explaining one block of a neural network structure composed of a plurality of blocks of the situation determination model of the present invention.
9 is a view for explaining the first and second pre-processing steps for generating the correct answer label used to learn the situation determination model of the present invention.
10 is a diagram for explaining first and second pre-processing steps for generating a correct answer label used to learn a situation determination model of the present invention.
11 is a view for explaining a third pre-processing step for generating a correct answer label used to learn the situation determination model of the present invention.
12 is a view for explaining a situation determination result of the situation determination model of the present invention.
13 is a view comparing the situation determination result of the situation determination model of the present invention and the situation determination result using a deep learning model according to the prior art.
14 is a view comparing the situation determination result of the situation determination model of the present invention and the situation determination result by a deep learning model according to the prior art.
15 is a view comparing the situation determination result of the situation determination model of the present invention and the situation determination result by a deep learning model according to the prior art.
16 is an exemplary diagram of a signal flow in a deep learning-based Go game service system according to an embodiment of the present invention.
17 is a method for determining a situation among a deep learning-based Go game service method according to an embodiment of the present invention.
18 is a method of preprocessing training data for generating a correct answer label among the method of determining the situation of FIG. 17.
19 is a diagram for explaining the structure of the ethos determination unit of the form determination model server of the present invention.
FIG. 20 is a flow chart illustrating a method of determining a situation of Go based on a deep learning neural network according to an embodiment of the present invention, and utilizing an ethos indicating a specific temperament or method when playing a Go game.
21 is a conceptual diagram for explaining a method of determining the status of Go and utilizing the ethos of playing a Go game according to an embodiment of the present invention.
22 is a view for explaining a method of generating wind determination information for each information on the number of candidates to be launched according to an embodiment of the present invention.
23 is a conceptual diagram for explaining a method of determining the status of Go and utilizing the ethos of playing a Go game according to another embodiment of the present invention.

Since the present invention can apply various transformations and have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail in the detailed description. Effects and features of the present invention, and a method of achieving them will be apparent with reference to the embodiments described later in detail together with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms. In the following embodiments, terms such as first and second are not used in a limiting meaning, but are used for the purpose of distinguishing one component from another component. In addition, expressions in the singular include plural expressions unless the context clearly indicates otherwise. In addition, terms such as include or have means that the features or elements described in the specification are present, and do not preclude the possibility of adding one or more other features or elements in advance. In addition, in the drawings, the size of components may be exaggerated or reduced for convenience of description. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of description, and the present invention is not necessarily limited to what is shown.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when describing with reference to the drawings, the same or corresponding constituent elements are assigned the same reference numerals, and redundant descriptions thereof will be omitted. .

1 is an exemplary diagram for a deep learning-based Go game service system according to an embodiment of the present invention.

Referring to FIG. 1, a deep learning-based Go game service system according to an embodiment includes a terminal 100, a Go server 200, a starting model server 300, a profile judgment model server, a profile judgment model server 400, and Network 500 may be included.

Each component of FIG. 1 may be connected through a network 500. The terminal 100, the Go server 200, the starting model server 300 and the type judgment model server type judgment model server 400, and the like, refer to a connection structure in which information can be exchanged with each other. Examples include 3GPP (3rd Generation Partnership Project) network, LTE (Long Term Evolution) network, WIMAX (World Interoperability for Microwave Access) network, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network). , Wide Area Network (WAN), Personal Area Network (PAN), Bluetooth (Bluetooth) network, satellite broadcasting network, analog broadcasting network, Digital Multimedia Broadcasting (DMB) network, and the like, but are not limited thereto.

-Terminal

First, the terminal 100 is a terminal of a user who wants to receive a Go game service. In addition, the terminal 100 is one or more computers or other electronic devices used by a user to execute applications that perform various tasks. For example, it includes a computer, laptop computer, smart phone, mobile phone, PDA, tablet PC, or any other device operable to communicate with the Go server 200. However, the present invention is not limited thereto, and the terminal 100 is executed on various machines and includes processing logic that interprets and executes instructions stored in a plurality of memories, and provides a graphic user interface (GUI) on an external input/output device. It may include a variety of other elements, such as processes for displaying graphical information. In addition, the terminal 100 may be connected to an input device (eg, a mouse, a keyboard, a touch-sensitive surface, etc.) and an output device (eg, a display device, a monitor, a screen, etc.). Applications executed by the terminal 100 may include a game application, a web browser, a web application operating in a web browser, word processors, media players, spreadsheets, image processors, security software, or the like. .

In addition, the terminal 100 may include at least one memory 101 storing instructions, at least one processor 102 and a communication unit 103.

The memory 101 of the terminal 100 may store a plurality of application programs or applications driven by the terminal 100, data for operation of the terminal 100, and commands. Commands can be executed by the processor 102 to cause the processor 102 to perform operations, and the operations transmit a request signal to execute a game of Go, transmit and receive game data, transmit and receive start information, and transmit a state determination request signal. Operations of receiving a determination result and receiving various types of information may be included. In addition, the memory 101 may be a variety of storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in hardware, and the memory 130 provides a storage function of the memory 101 on the Internet. It may be a web storage that performs.

The processor 102 of the terminal 100 may perform data processing to receive a Go game service by controlling the overall operation. When the Go game application is executed in the terminal 100, the Go game environment is configured in the terminal 100. In addition, the Go game application exchanges Go game data with the Go server 200 through the network 500 so that the Go game service is executed on the terminal 100. These processors 102 include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, and microcontrollers. Controllers (micro-controllers), microprocessors (microprocessors), may be any type of processor for performing other functions.

The communication unit 103 of the terminal 100 includes the following communication methods (eg, Global System for Mobile communication (GSM)), Code Division Multi Access (CDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink (HSUPA). Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc.), WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA ( Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), a wireless signal can be transmitted and received with at least one of a base station, an external terminal, and a server on a network.

-Go Server

The Go game service provided by the Go server 200 may be configured in a form in which a virtual computer user and an actual user provided by the Go server 200 participate in the game together. This means that one real user and one computer user play the game together in a Go game environment implemented on the user's terminal 100. In another aspect, the Go game service provided by the Go server 200 may be configured in a form in which a plurality of user-side devices participate to play the Go game.

The Go server 200 may include at least one memory 201 for storing instructions, at least one processor 202 and a communication unit 203.

The memory 201 of the Go server 200 may store a plurality of application programs or applications running in the Go server 200, data for the operation of the Go server 200, and commands. . Instructions can be executed by the processor 202 to cause the processor 202 to perform operations, and the operations are game execution request signal reception, game data transmission/reception, launch information transmission/reception, situation determination request signal transmission/reception, situation determination result transmission/reception And various transmission operations. In addition, the memory 201 may store a plurality of notations played by the Go server 200 or a plurality of previously published notations. Each of the plurality of notations may include all information from the first start, which is information on the first start of the game, to the final start, at which the game ends. That is, the plurality of notations may include history information about the initiation. The Go server 200 makes it possible to provide a plurality of notations stored for training of the shape judgment model server 400 to the shape judgment model server 400. In addition, the memory 201 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in terms of hardware, and the memory 201 may perform a storage function of the memory 201 on the Internet. It may be a web storage that performs.

The processor 202 of the Go server 200 may perform data processing to provide a Go game service by controlling the overall operation. These processors 202 include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, and microcontrollers. Controllers (micro-controllers), microprocessors (microprocessors), may be any type of processor for performing other functions.

The Go server 200 may communicate with the terminal 100, the launch model server 300, and the form judgment model server 400 via the network 500 through the communication unit 203.

-Launch model server

The launch model server 300 may include a separate cloud server or computing device. In addition, the launch model server 300 may be a neural network system installed in the processor of the terminal 100 or the data processing unit of the Go server 200, but hereinafter, the launch model server 300 is the terminal 100 or the Go server ( 200) and a separate device.

The launch model server 300 may include at least one memory 301 for storing instructions, at least one processor 302 and a communication unit 303.

The launching model server 300 is an artificial intelligence computer capable of building a deep learning model, which is a deep learning model by self-learning according to the Go rule, and playing a game with the user of the terminal 100. Initiation can be carried out. A detailed description of training in the initiation model server 300 with the initiation model follows the description of the initiation model in FIGS. 2 to 5.

The memory 301 of the initiation model server 300 includes a number of application programs or applications running in the initiation model server 300, data for the operation of the initiation model server 300, and instructions. Can be saved. The instructions are executable by the processor 302 to cause the processor 302 to perform the operations, and the operations may include an initiation model learning (training) operation, an initiation information transmission/reception, and various transmission operations. In addition, the memory 301 may store an initiation model that is a deep learning model. In addition, the memory 301 may be a variety of storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in hardware, and the memory 301 can perform a storage function of the memory 301 on the Internet. It may be a web storage that performs.

The processor 302 of the initiation model server 300 reads out the initiation model stored in the memory 302, and performs training of the initiation model and initiation of the game according to the constructed neural network system. In an embodiment, the processor 302 of the launch model server 300 may predict and derive at least two or more specific launch points determined to be the highest number in a specific checkerboard state (S). Further, the processor 302 may transmit the derived at least two or more specific starting points to the shape judgment model server 400. In addition, the processor 302 of the initiation model server 300 may perform self-play learning in which the initiation model is trained in a predetermined specific ethos. In addition, the processor 302 may diagnose the performance of whether the air wind is properly learned and determined through self-play learning.

Meanwhile, according to an embodiment, the processor 302 is configured to include a main processor that controls all units, and a plurality of graphics processing units (GPUs) that process large-capacity operations required when driving a neural network according to an initiation model. Can be.

The launch model server 300 may communicate with the Go server 200 through the network 500 through the communication unit 303.

-Form judgment model server

The form judgment model server 400 may include a separate cloud server or computing device. In addition, the shape judgment model server 400 may be a neural network system installed in the processor of the terminal 100 or the data processing unit of the Go server 200, but hereinafter, the shape judgment model server 400 is the terminal 100 or the Go It will be described as a separate device from the server 200.

The form judgment model server 400 may include at least one memory 401 storing instructions, at least one processor 402 and a communication unit 403.

The form judgment model server 400 may receive a training data set from the Go server 200 through the communication unit 403. The training data set may be a plurality of notations and information on status determination for the plurality of notations. The situation judgment model server 400 supervises and learns to determine the status of the board on which the go board is placed using the received training data set to build a status judgment model, which is a deep learning model, and the status of the terminal 100 user. The situation can be judged according to the request for judgment. A detailed description of the situation determination model server 400 training with the situation determination model follows the description of the situation determination model of FIGS. 6 to 18.

The memory 401 of the shape judgment model server 400 includes a number of application programs or applications that are driven by the shape judgment model server 400, and data for the operation of the shape judgment model server 400 , You can store commands. The instructions can be executed by the processor 402 to cause the processor 402 to perform the operations, and the operations include a situation determination model learning (training) operation, a situation determination operation, a situation determination result transmission, a plurality of notation information reception, and It may include various transmission operations. In addition, referring to FIG. 19, in the embodiment, the memory 401 determines and learns the situation of Go based on the situation determination model 400a, which is a deep learning model, and a deep learning neural network, and applies an ethos when playing a Go game. It is possible to store the airflow determination unit 400b for implementing the operation. In addition, the memory 401 may be a variety of storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in hardware, and the memory 401 provides a storage function of the memory 301 on the Internet. It may be a web storage that performs.

The processor 402 of the situation determination model server 400 reads the situation determination model 400a stored in the memory 402, and learns the situation determination model 400a described below according to the built neural network system and You will perform a judgment on the situation. In addition, the processor 402 of the situation determination model server 400 may determine the situation of Go and the ethos of a specific starting point based on the performed situation determination.

In detail, in an embodiment, the processor 402 of the shape determination model server 400 may set a target airflow to be learned among various airflows. In addition, the processor 402 may set a predetermined threshold for determining the status of the checkerboard state S and/or the ethos of a specific starting point through the status determination. In addition, the processor 402 of the layout judgment model server 400 performs a status determination on a specific checkerboard state (S) in which a specific starting point derived from the starting model server 300 is reflected, and the specific starting point and/or corresponding The ethos for a specific checkerboard state (S) can be determined. In addition, the processor 402 may transmit information on which the ethos is determined to the launch model server 300 so that self-play learning of the launching model for applying the ethos when playing a Go game is effectively performed. In addition, the processor 402 of the shape judgment model server 400 may check the performance of whether the determination of the ethos is properly made, and adjust a predetermined threshold for determining the ethos based on this.

Meanwhile, according to an embodiment, the processor 402 includes a main processor that controls all units, and a plurality of graphics processors (GPUs) that process a large amount of computation required when driving a neural network according to the situation determination model 400a. It may be configured to include.

In addition, the form judgment model server 400 may communicate with the Go server 200 via the network 500 through the communication unit 403.

-Launch model

FIG. 2 is a diagram for explaining the structure of the launching model of the launching model server 300 for launching an artificial intelligence computer in a deep learning-based Go game service according to an embodiment of the present invention. It is a diagram for explaining the distribution probability of movement for the starting point according to the following, FIG. 4 is a diagram for explaining the value of the starting point and the number of visits of the starting model, and FIG. 5 is a diagram for explaining the starting model according to the pipeline of the search unit. It is a diagram for explaining the process of doing.

Referring to FIG. 2, the initiation model according to the embodiment of the present invention is a deep learning model of the initiation model server 300, and a search unit 310, a self-play unit 320, an initiation neural network 330, and an ethos learning assistant ( 340) may be included.

)을 출력 할 수 있다. 셀프 플레이부(320)는 탐색 확률값(

)에 따라 스스로 바둑 대국을 할 수 있다. 셀프 플레이부(320)는 게임의 승패가 결정되는 시점까지 스스로 바둑 대국을 진행하고, 자가 대국이 종료되면 바둑판 상태(S), 탐색 확률값(

), 자가 플레이 가치값(z)을 착수 신경망(330)에 제공할 수 있다. 바둑판 상태(S)는 착수점들에 바둑돌이 놓여진 상태이다. 자가 플레이 가치값(z)은 바둑판 상태(S)에서 자가 대국을 하였을 때 승률 값이다. 착수 신경망(330)은 이동 확률값(p)과 가치값(v)을 출력할 수 있다. 이동 확률값(p)은 바둑판 상태(S)에 따라 착수점들에 대해 어느 착수점에 착수하는 것이 게임을 이길 수 있는 좋은 수인지 수치로 나타낸 확률분포값이다. 가치값(v)은 해당 착수점에 착수시 승률을 나타낸다. 예를 들어, 이동 확률값(p)이 높은 착수점이 좋은 수일 수 있다. 착수 신경망(330)은 이동 확률값(p)이 탐색 확률값(

)과 동일해지도록 트레이닝되고, 가치값(v)이 자가 플레이 가치값(z)과 동일해지도록 트레이닝될 수 있다. 이후 트레이닝된 착수 신경망(330)은 탐색부(310)를 가이드하고, 탐색부(310)는 이전 탐색 확률값(

)보다 더 좋은 수를 찾도록 MCTS를 진행하여 새로운 탐색 확률값(

)을 출력하게 한다. 셀프 플레이부(320)는 새로운 탐색 확률값(

)에 기초하여 바둑판 상태(S)에 따른 새로운 자가 플레이 가치값(z)을 출력하고 바둑판 상태(S), 새로운 탐색 확률값(

), 새로운 자가 플레이 가치값(z)을 착수 신경망(330)에 제공할 수 있다. 착수 신경망(330)은 이동 확률값(p)과 가치값(v)이 새로운 탐색 확률값(

)과 새로운 자가 플레이 가치값(z)으로 출력되도록 다시 트레이닝될 수 있다. 즉, 착수 모델은 이러한 과정을 반복하여 착수 신경망(330)이 대국에서 이기기 위한 더 좋은 착수점을 찾도록 트레이닝 될 수 있다. 일 예로, 착수 모델은 착수 손실(l)을 이용할 수 있다. 착수 손실(l)은 수학식 1과 같다.The launching model can be learned as a model that launches to win the game using the search unit 310, the self-play unit 320, the launch neural network 330, and the ethos learning assistant 340. You can use the ethos that represent a specific temperament or manner. More specifically, the search unit 310 may perform a Monte Carlo Tree Search (MCTS) operation according to the guide of the initiating neural network 330. MCTS is an experiential search algorithm for making some kind of decision. That is, the search unit 310 may perform the MCTS based on the movement probability value (p) and/or the value value (v) provided by the initiating neural network 330. As an example, the search unit 310 guided by the initiation neural network 330 performs an MCTS to obtain a search probability value (the probability distribution value for the initiation points).

) Can be printed. The self-play unit 320 includes a search probability value (

), you can play the Go game yourself. The self-play unit 320 proceeds with the game of Go by itself until the time when the game is determined to win or lose, and when the self-playing ends, the board state (S), the search probability value (

), the self-play value z may be provided to the initiating neural network 330. The checkerboard state (S) is a state in which go stones are placed at the starting points. The self-play value (z) is the win rate value when the player plays the game in the checkerboard state (S). The initiating neural network 330 may output a moving probability value (p) and a value value (v). The moving probability value (p) is a numerical probability distribution value indicating which starting point is a good number to win the game with respect to the starting points according to the checkerboard state (S). The value (v) represents the odds of starting at the starting point. For example, a starting point with a high moving probability value p may be a good number. Initiating neural network 330, the moving probability value (p) is the search probability value (

), and the value v may be trained to be equal to the self-play value z. After the training initiation neural network 330 guides the search unit 310, the search unit 310 is the previous search probability value (

MCTS is performed to find a number better than) and a new search probability value (

) To be printed. The self-play unit 320 is a new search probability value (

), a new self-play value (z) according to the checkerboard state (S) is output, and the checkerboard state (S), a new search probability value (

), a new self-play value z may be provided to the initiating neural network 330. The initiating neural network 330 has a moving probability value (p) and a value value (v) being a new search probability value (

) And the new self can be retrained to be output as the play value z. That is, the initiating model may be trained to repeat this process so that the initiating neural network 330 finds a better starting point to win in the game. As an example, the initiation model may use the initiation loss (l). The starting loss (l) is shown in Equation 1.

(Equation 1)

는 신경망의 파라미터이고, c는 매우 작은 상수이다.

Is a parameter of the neural network, and c is a very small constant.

와 p가 같아 지도록 하는 것은 크로스 엔트로피 손실(cross entropy loss) 텀에 해당되고,

에 c를 곱하는 것은 정규화 텀으로 오버핏을 방지하기 위한 것이다.Making z and v equal in the starting loss (l) of Equation 1 corresponds to the mean square loss term,

Making and p become equal corresponds to the term of cross entropy loss,

Multiplying by c is to prevent overfit with the normalization term.

For example, referring to FIG. 3, the trained embarkation model may represent a moving probability value p at embarkation points as a probability distribution value as shown in FIG. 3. Referring to FIG. 4, the value v of the trained embarkation model may be expressed as a value indicated above at one beheading point of FIG. 4. The initiating neural network 330 may be configured in a neural network structure. As an example, the initiation neural network 330 may include one convolutional block and 19 residual blocks. The convolution block may be in the form of overlapping several 3X3 convolution layers. One residual block may be in a form in which several 3X3 convolution layers are overlapped and skip connections are included. The skip connection is a structure in which an input of a predetermined layer is combined with an output value of a corresponding layer to be output and input to another layer. In addition, the input of the initiation neural network 330 is 19* including position information of stones for the last 8 numbers of black players, position information of stones for the last 8 numbers of white players, and turn information about whether the current player is black or white. An RGB image of 19*17 can be input.

)을 출력할 수 있다. 착수 모델은 착수점들 중 가장 높은 탐색 확률값(

)을 선택할 수 있고, 선택된 탐색 확률값(

)을 가지는 착수점을 해당 바둑판 상태(S)에서의 최고의 수로 판단할 수 있다. 이때, 착수 모델은 탐색 확률값(

)을 기반으로 최고의 수라고 판단된 착수점을 착수 후보수로 설정할 수 있다. 즉, 본 발명의 실시예에서 착수 후보수란 특정 바둑판 상태(S)에 대하여 착수 모델에 의해 예측된 최고의 착수점일 수 있다. 이때, 착수 모델은 탐색 확률값(

)을 기반으로 최고의 수라고 판단된 착수 후보수를 적어도 둘 이상 도출할 수 있고, 이에 기초하여 적어도 둘 이상의 착수 후보수 정보를 생성할 수 있다. 자세히, 착수 모델은 결정된 착수 후보수에 기반하여 해당 착수 후보수와 관련된 정보(예컨대, 바둑판 상에서의 위치 정보 등)를 포함하는 착수 후보수 정보를 적어도 둘 이상 생성할 수 있다. 여기서 착수 모델이 적어도 둘 이상의 착수 후보수 정보를 생성하는 것은, 복수의 착수 후보수를 기반으로 기풍을 판단한 결과를 통하여 특정 기풍에 최적화된 착수 후보수를 합리적으로 도출하기 위함이다. Referring to FIG. 5, the trained initiation model can be initiated using the initiation neural network 330 and the search unit 310 in their turn. The initiation model is trusted with the activity function (Q) in the current first checkerboard state (S) (S1) through the selection process (a) and the second checkerboard state (S) (S1-2), which is a branch that has not been searched through MCTS. Choose a starting point with a higher value (U). The activity function (Q) is an average value of the value values (v) calculated each time the branch passes. The confidence value (U) is proportional to the number of visits (N) passing through the branch. The starting model can be expanded to the third checkerboard state (S) (S1-2-1) at the selected starting point through the expansion and evaluation process (b), and a moving probability value (p) can be calculated. The initiation model calculates the value of the extended third checkerboard state (S) (S1-2-1) through the backup process (c), and the activity function (Q), the number of visits (N), and the probability of movement ( p) can be saved. The initiation model repeats the process of selection (a), expansion and evaluation (b), and backup (c), and creates a probability distribution using the number of visits (N) for each starting point,

) Can be printed. The initiation model is the highest search probability among the initiation points (

) Can be selected, and the selected search probability value (

The starting point with) can be determined as the highest number in the corresponding checkerboard state (S). At this time, the starting model is the search probability value (

), the starting point determined as the highest number can be set as the number of candidates for starting. That is, in an embodiment of the present invention, the number of candidates to be launched may be the highest starting point predicted by the starting model for a specific checkerboard state (S). At this time, the starting model is the search probability value (

), it is possible to derive at least two or more embarkation candidates determined to be the highest number, and generate at least two or more embarkation candidates information based on this. In detail, the launching model may generate at least two launching candidates information including information related to the launching candidates (eg, location information on a board) based on the determined launching candidates number. Here, the reason that the launching model generates information on at least two launching candidates is to reasonably derive the number of launching candidates optimized for a specific ethos through a result of determining the ethos based on a plurality of launching candidates.

In this case, the starting model server 300 may transmit the generated information on the number of embarkation candidates to the form judgment model server 400 and receive ethos training data from the form determination model server 400. Here, the ethos training data is at least one launch candidate determined by the ethos determination unit 400b of the shape judgment model server 400 to be suitable for specific ethos learning in order to learn the initiation model with a specific ethos It may be information about the number. A detailed description of this will be described later.

In addition, the initiation model may perform self-play learning based on the ethos training data obtained from the shape judgment model server 400. In detail, the initiation model may perform a Go game play between the higher version initiation model learned based on the ethos training data through the self-play unit and the lower version initiation model before training. That is, the initiation model can perform self-learning based on the ephemeris learning data, and through this, it is possible to diagnose the performance of the ethos learning data. Here, diagnosing the performance of the ethos learning data may refer to determining whether the corresponding ethos learning data is suitable data for learning a specific ethos.

Subsequently, the initiation model may generate airflow learning performance information through performance diagnosis of airflow learning data based on self-play learning. That is, the ethos learning performance information is determined whether the ethos learning data is suitable data for learning a specific ethos through self-learning of the initiation model based on the acquired ethos learning data, i.e. It may be information that diagnoses whether the ethos of about is properly determined. In this case, the start model server 300 may transmit the generated ethos training performance information to the form judgment model server 400. That is, the launching model can improve the accuracy of determining the number of candidates for launching and/or the ethos of the checkerboard state S by confirming the suitability of the training data for learning a specific ethos and providing feedback thereon.

Meanwhile, the initiation model may include an ethos learning assistant 340 to provide a Go game service that utilizes ethos representing a specific temperament or manner when playing a Go game according to an embodiment of the present invention. In detail, the ethos learning assistant 340 may receive ethos training data as input data from the shape determination model server 400. In addition, the airflow learning assistant 340 may control the self-play unit 320 so that the launch model server 300 performs self-play learning based on airflow learning data received as input data. In addition, the ethos learning assistant 340 may generate ethos learning performance information based on a result of self-play learning through the self-play unit of the undertaking model. That is, the ethos learning assistant 340 determines whether the ethos learning data is suitable data for learning a predetermined target ethos through the self-learning result of the ethos model based on ethos learning data, that is, the number of specific candidates and/or the checkerboard state. The ethos learning performance information for diagnosing whether the ethos for (S) is properly determined by the shape judgment model server 400 may be generated. For example, the ethos learning assistant 340 includes a score difference value between the player and the oponant for a specific number of embarkation candidates included in the ethos learning data, and the difference between the player and the oponant for the specific number of undertaking candidates derived through self-play. By comparing the score difference value, it is possible to generate ethos learning performance information. In addition, the ethos learning assistant 340 may transmit the generated ethos learning performance information to the shape judgment model server 400 to perform a threshold adjustment operation for the judgment of the ethos of the shape judgment model server 400 based thereon. .

A more detailed description will be described later in the detailed description of a method of determining and utilizing the ethos based on deep learning described below. In addition, in this embodiment, it is described that the ethos learning assistant 340 is included in the undertaking model server 300 and operates, but in another embodiment, the ethos learning assistant 340 is the Go server 200 or the form judgment model server ( 400) or implemented as a separate device, various embodiments are also possible.

-Position judgment model

6 is an exemplary view showing a screen that provides a position determination function of a deep learning-based Go game service according to an embodiment of the present invention, and FIG. 7 is a state determination model of the position determination model server 400 of the present invention ( 400a) is a diagram for explaining the structure, and FIG. 8 is a diagram for explaining one block of a neural network structure composed of a plurality of blocks of the state determination model 400a of the present invention.

Referring to FIG. 6, the deep learning-based Go game service according to an embodiment of the present invention can determine the status of the current Go board state (S). For example, as shown in FIG. 6, when a user requests a situation determination by clicking the situation determination menu (A) during a game of Go on the screen of the terminal 100, a deep learning-based Go game service provides the situation determination result in a pop-up window. I can. The situation judgment is to determine who is winning with how many points by calculating the opponent and my house during the game of Go. For example, if the user decides that the situation is favorable to me, he will not overdo it anymore and will develop a strategy in the direction of ending the game while maintaining the current favorable situation. You can explore different strategies to help you do it. The criteria for judging the situation are houses, sandstones, stones, gongbae, and big according to the state in which the go stones are placed on the board. A stone is a stone placed on a checkerboard and is not a score in Korean rules. A house is an area made up of empty dots surrounded by single colored Go stones, and is a score in Korean rules. Gongbae and Big are the areas that are neither black nor white when Baduk is over, and are not points in Korean rules. Panwisaseok is a stone that dies because it is bound to be caught no matter how it is placed on the board. It is a score because it is used to fill the opponent's house under Korean rules. Big refers to an area that is neither a black house nor a white house when Baduk is over. Therefore, the situation can be accurately determined only when the house, stone, stone, gongbae, and big are accurately classified or predicted in the checkerboard state (S) on which the go stones are placed. At this time, the exact classification of house, sandstone, stone, gongbae, and big is to distinguish the state where the house, sandstone, stone, gongbae, and vic are completely completed. It may be to predict a state that is likely to become a stone, stone, gongbae, or big.

Referring to FIG. 7, a situation determination model 400a according to an embodiment of the present invention is a deep learning model of a situation determination model server 400, and a situation determination neural network 410, an input feature extraction unit 420, and a correct answer label are generated. It may include a unit 430. In addition, the situation determination model 400a according to an embodiment of the present invention is interlocked with the ethos determination unit 400b to operate the process of applying ethos when playing a Go game by determining and learning the state of Go based on a deep learning neural network. Can be. A detailed description of this will be described later.

The situation determination model 400a may perform supervised learning to determine the current state of the board state S using the state determination neural network 410. More specifically, the situation determination model 400a generates a training data set for the checkerboard state (S), and the situation determination neural network 410 uses the generated training data set to determine the situation according to the current checkerboard state (S). You can learn to do it. The form judgment model server 400 may receive a plurality of notations from the Go server 200. Each notation of the plurality of notations may include a respective checkerboard state (S) according to the starting order.

The input feature extraction unit 420 may extract the input feature IF from the checkerboard state S of a plurality of notations and provide the state determination neural network 410 as input data for training. The input feature (IF) of the checkerboard status (S) includes the stone location information for the last 8 number of black players, the stone location information for the last 8 number of white players, and the turn information for whether the current player is black or white. It may be a 19*19*18 RGB image. For example, the input feature extraction unit 420 may have a neural network structure and may include a kind of encoder.

)로 제공할 수 있다. 정답 레이블 생성부(430)의 정답 레이블 생성은 후술하는 도 9 내지 도 11의 설명을 따른다. 일 예로, 정답 레이블 생성부(430)는 신경망 구조의 롤아웃 또는 인코더를 포함할 수 있다.The correct answer label generation unit 430 generates a ground truth through a pre-processing process in the current checkerboard state (S), and sends the correct answer label to the situation determination neural network 410 for training target data (

) Can be provided. The generation of the correct answer label by the correct answer label generator 430 follows the description of FIGS. 9 to 11 to be described later. As an example, the correct answer label generator 430 may include a rollout or an encoder of a neural network structure.

)로 한 트레이닝 데이터 셋을 이용하여 형세 판단 신경망(410)에서 생성된 출력 데이터(o)가 타겟 데이터(

)와 동일해지도록 형세 판단 신경망(420)을 충분히 학습할 수 있다. 일 예로, 형세 판단 모델(400a)은 형세 판단 손실(

)을 이용할 수 있다. 형세 판단 손실(

)은 평균 제곱 에러(mean square error)를 이용할 수 있다. 예를 들어, 형세 판단 손실(

)은 수학식 2와 같다.The situation determination model 400a uses the input feature (IF) as input data and the correct answer label as target data (

Using the training data set as ), the output data (o) generated by the neural network 410 is the target data (

The situation determination neural network 420 may be sufficiently learned to become equal to ). As an example, the situation judgment model 400a is the situation judgment loss (

) Can be used. Loss of judgment (

) Can be used as the mean square error. For example, loss of position judgment (

) Is the same as in Equation 2.

(Equation 2)

는 현재 바둑판 상태(S)에서 정답 레이블에 따른 소정의 교차점(i)에 대한 형세값이다. 형세값에 대한 설명은 후술하는 도 11의 설명에 따른다.

는 현재 바둑판 상태(S)에서 소정의 교차점(i)을 형세 판단 신경망(410)에 입력하였을 때에 출력되는 출력 데이터이다. 형세 판단 모델(400a)은 형세 판단 손실(

)이 최소화되도록 경사 하강법(gradient-descent)과 역전파(backpropagation)을 이용하여 형세 판단 신경망(410) 내의 가중치와 바이어스 값들을 조절하여 형세 판단 신경망(410)를 학습시킬 수 있다.B is the total number of intersections on the board. In the checkerboard, 19 horizontal and 19 vertical lines cross each other, and 361 intersections are arranged. This is not limited thereto, and 81 intersections may be arranged when the checkerboard has 9 horizontal lines and 9 vertical lines.

Is the position value for a predetermined intersection (i) according to the correct answer label in the current checkerboard state (S). The description of the position value follows the description of FIG. 11 to be described later.

Is output data that is output when a predetermined intersection (i) is input to the situation determination neural network 410 in the current checkerboard state (S). The situation judgment model 400a is the situation judgment loss (

) To minimize the situation, the situation determination neural network 410 may be trained by adjusting weights and bias values in the situation determination neural network 410 using gradient-descent and backpropagation.

The situation determination neural network 410 may be configured in a neural network structure. For example, the situation determination neural network 420 may be composed of 19 residual blocks. Referring to FIG. 8, one residual block includes 256 3X3 convolution layers, batch normalization layers, Relu activation function layers, 256 3X3 convolution layers, batch normalization layers, skip connection, and Relu activation function layers may be arranged in order. The batch normalization layer is to prevent covariate shift, which is a phenomenon in which the distribution of the input of the current layer changes due to a parameter change of the previous layer during training. The skip connection prevents a decrease in the performance of the neural network even if the block layer becomes thicker, and increases the overall neural network performance by making the block layer thicker. The skip connection may be a form in which the first input data of the residual block is added to the output of the second batch normalization layer and input to the second relu activation function layer.

9 and 10 are diagrams for explaining first and second preprocessing steps for generating a correct answer label used to learn the position determination model 400a of the present invention, and FIG. 11 is a position determination model of the present invention A diagram for explaining a third pre-processing step for generating a correct answer label used for learning (400a).

The correct answer label generation unit 430 may generate a correct answer label used for learning so that the situation determination neural network 410 can accurately determine the situation.

More specifically, the correct answer label generation unit 430 receives a checkerboard state (S) based on the input data as an input, and performs a first preprocessing to end in the current checkerboard state (S) to perform a first preprocessing state (P1). ) Can be created. The first pre-processing, finishing, is a process of completing the game by performing a predetermined start so that the boundaries of the house become clear before calculating the house. As an example, referring to FIG. 9, the correct answer label generation unit 430 may generate the first pre-processing state P1 of FIG. 9 (b) by ending in the current checkerboard state (S) of FIG. 9 (a). I can.

The correct answer label generator 430 may generate a second pre-processing state P2 by performing a second pre-processing of removing stones that are disposed within the boundary of the house in the first pre-processing state P1 and unnecessary to classify the house. For example, stones placed within the boundary of the house and unnecessary to divide the house may be sandstone. Saseok explained earlier that the other stone was placed in the house, so no matter how it was placed, it could only be caught and died. In addition, stones placed within the boundary of the house and unnecessary to divide the house may be own stones placed in the house. For example, referring to FIG. 9, the correct answer label generation unit 430 removes stones unnecessary for house division in the first preprocessing state P1 of FIG. 9B, and thus the second preprocessing state of FIG. 9C. (P2) can be created.

As another example, referring to FIG. 10, the correct answer label generation unit 430 starts the red x as shown in FIG. 10 (b) to end the first pre-processing in the current checkerboard state (S) of FIG. 10 (a). can do. The correct answer label generation unit 430 removes the rubble by embarking on the green x in order to remove the rubble indicated by the blue x in FIG. 10(b), and removes the green x used for removing the rubble. Pretreatment can be performed.

The correct answer label generation unit 430 may perform a third pre-process of changing each intersection point to a position value (g, but g is an integer) displayed from -1 to +1 in the second pre-processing state P2. That is, in the third pre-processing, the correct answer label generator 430 changes the second pre-processing state P2, which is an image feature, to a third pre-processing state P3, which is a numerical feature. For example, in the second pre-treatment state P2, if my stone is placed at the intersection, it may correspond to 0, if it is a home area, +1, if a counterpart stone is disposed, it may correspond to -1. In this case, the situation determination neural network 410 may be trained to distinguish a house, a stone, and a stone when determining the situation. As another example, in the second pretreatment state P2, if my stone is placed at the intersection, it may correspond to 0, if it is my house area, +1, if the opponent stone is disposed, it may correspond to 0, if it is a relative house area, it may correspond to -1, and if it is big or common, 0. In another example, the situation determination neural network 410 may be trained to distinguish between a big or common place when determining the situation. For example, referring to FIG. 11, the correct answer label generation unit 430 characterizes the second pre-processing state P2 of FIG. 11 (a) as the third pre-processing state P3 of FIG. 11 (b). You can change it.

)로 이용될 수 있다. The third preprocessing state (P3) becomes the correct answer label of the situation determination in the checkered state (S), and target data (

) Can be used.

12 is a diagram for explaining a result of determining the position of the position determination model 400a of the present invention.

The learned position determination model 400a may provide position values for all intersection points of the board when the checkerboard state S is input. That is, an integer value of -1 to +1, which is a layout value, can be provided for 361 points of the crossing of the checkerboard.

Referring to FIG. 12, the situation determination model server 400 may determine the situation using a situation value provided by the situation determination model 400a, a predetermined threshold value, and the presence or absence of a stone. As an example, the situation judgment model server 400 is a place where there is no stone, and if the situation value exceeds the first threshold, it is determined as a place with a high possibility of becoming my home, and a value close to +1 may be determined as my home area. have. The shape judgment model server 400 may display a square shape of the same color as my stone, which gradually increases as the likelihood of the house is high. The situation judgment model server 400 may determine a location where there is no stone, and when the location value is less than or equal to the second threshold value, it may be determined as a location that is likely to be a relative house, and when a value close to -1, it may be determined as a home area. The shape judgment model server 400 may display a square shape of the same color as the opponent stone, which gradually increases as the likelihood of the opponent's house increases. The position judgment model server 400 is a place where there is no stone, and if the position value is within the range of the third threshold value or close to 0, it may be determined as common or big. If the situation judgment model server 400 determines that it is common or big, it may be indicated by an X. The shape judgment model server 400 is a place where the stone is located, and if the shape value is within the range of the third threshold value or close to 0, it may be determined as the inner stone or the relative stone. The form judgment model server 400 may not display any display when it is determined as common or big. The situation judgment model server 400 is a place where the stone is located, and if the situation value exceeds the first threshold value, it is determined as a place that is likely to be the stone stone of the other stone, and if the value is close to +1, it can be determined as the stone stone of the other stone. have. The shape judgment model server 400 may display a square shape of the same color as my stone, which gradually increases as the probability of the stone stone of the opponent increases. The situation judgment model server 400 is a place where a stone is located, and if the situation value is less than the second threshold, it is determined as a place that is likely to be the stone of my stone, and if the value is close to -1, it can be determined as the stone of the other stone. have. The shape judgment model server 400 may display a square shape of the same color as that of the opponent stone, which gradually increases as the probability of the stone stone is higher.

In addition, the status judgment model server 400 may display the result of the calculation in the current checkerboard state (S) by using the status judgment criteria determined at each intersection.

Accordingly, the deep learning-based Go game service device according to the embodiment may determine the status of Go by using a deep learning neural network. In addition, the deep learning-based Go game service device according to the embodiment can accurately determine the status of Go by accurately classifying houses, sandstones, stones, gongbae, and big according to the Go rule. In addition, the deep learning-based Go game service apparatus according to the embodiment may accurately determine the status of Go by predicting houses, sandstones, stones, gongbae, and big according to the Go rule. In addition, the deep learning-based Go game service apparatus according to the embodiment may quickly determine the status of the Go game.

On the other hand, in order to perform a series of operations for applying a specific ethos during a game of Go based on deep learning, the situation determination model server 400 uses the first situation determination model 400a through the situation determination model 400a. To 3 It is possible to generate information about the house holdings based on the position value derived through the pre-processing process. Here, the holding house information may be information generated based on the number of holding houses of each player and owner predicted through judgment of the situation. That is, the situation judgment model server 400, through the situation determination model 400a, the position value for all intersections of the checkerboard state (S) predicted from the situation determination model 400a (in the embodiment, -1 to +1 The number of houses expected to be owned by the player and the owner can be calculated based on the value between them, and the house information can be generated. For example, the holding house information may be implemented in a form including information that predicts that the player will own 80 houses and the owner of 77 houses. The information on the house held may be used as base data for generating a score difference value, which is a parameter that is compared with a predetermined threshold for determining the wind, in the process of determining the wind, which will be described later.

In an embodiment, the situation judgment model server 400 derives the status value by performing the status determination through the status determination model 400a, and the number of houses owned by the player and the number of houses owned by the offant based on the derived status value. It is possible to generate the holding house information by calculating. In addition, the shape judgment model server 400 may obtain a player score value and an offant score value based on the information on the number of collections generated through the ethos determination unit 400b. In addition, the shape judgment model server 400 may calculate a score difference value between the two score values based on the obtained player score value and the oponant score value through the ethos determination unit 400b. Thereafter, the shape judgment model server 400 may generate ethos determination information by comparing the score difference value calculated through the ethos determination unit 400b with a predetermined ethos determination threshold. A detailed description of this will be described later.

13 is a view comparing the situation determination result of the situation determination model 400a of the present invention with the situation determination result by a deep learning model according to the prior art, and FIG. 14 is a state determination of the situation determination model 400a of the present invention The results are compared with the results of the situation determination by the deep learning model according to the prior art, and FIG. 15 shows the results of the situation determination by the situation determination model 400a of the present invention and the situation determination result by the deep learning model according to the prior art. This is a comparison.

Referring to FIG. 13, the situation determination model 400a of the present invention determines the situation by classifying houses, stones, and rubbles at each intersection as in area B of FIG. 13A. However, the situation determination model 400a based on the deep learning model according to the prior art cannot distinguish between a house, a stone, and a rubble with respect to the intersection of an area corresponding to that of FIG. 13(a) in FIG. 13(b).

Likewise, referring to FIG. 14, the situation determination model 400a according to the present invention determines the situation by classifying a house, stone, and stone at each intersection, as in area C of FIG. 14A. However, the situation determination model 400a based on the deep learning model according to the prior art cannot distinguish between a house, a stone, and a rubble with respect to the intersection of an area corresponding to that of FIG. 13(a) in FIG. 14(b).

Referring to FIG. 15, the situation determination model 400a of the present invention properly recognizes a white house as shown in area D of FIG. 15A. However, the situation determination model 400a based on the deep learning model according to the prior art cannot distinguish a bag house in a region corresponding to FIG. 15(b) to FIG. 15(a).

16 is an exemplary diagram of a signal flow in a deep learning-based Go game service system according to an embodiment of the present invention.

Referring to FIG. 16, the launching model server 300 is an artificial intelligence computer that trains the launching model, which is a deep learning model, by self-learning according to the Go rule so that the launching of Go can be performed in order to win the game in its own turn. Can be (s11). The Go server 22 may transmit a plurality of notations to the shape judgment model server 400. The form judgment model server 400 may generate a training data set. First, the form judgment model server 400 may extract input features from a checkerboard state (S) of a plurality of notations (S13). The form judgment model server 400 may generate a correct answer label by using the checkerboard state (S) from which the input features are extracted (S14). The situation determination model server 400 may train the situation determination model 400a using a training data set in which an input feature is used as input data and a correct answer label is used as target data (S15). The terminal 100 may request a game of Go from the Go server 200 against an artificial intelligence computer or against another user terminal (S16). When the terminal 100 requests a game of Go against the artificial intelligence computer, the Go server 200 may request the start model server 300 to initiate the game (S17). The Go server 200 progresses the Go game, and the terminal 100 and the start model server 300 may start on their own turn (S18 to S20). During the game, the terminal 100 may request the Go server 200 to determine the situation (S21). The Go server 200 may request the status judgment model server 400 to determine the status of the current board status (S) (S22). The layout judgment model server 400 extracts the input features of the current checkerboard state (S), the situation determination model 400a, which is a deep learning model, uses the input features to generate a status value, and the checkerboard status (S) and status The situation may be determined using the value (S23). The situation determination model server 400 may provide a result of the situation determination to the Go server 200 (S24). The Go server 200 may provide a result of determining the situation to the terminal 100 (S25).

FIG. 17 is a method for determining a situation among the deep learning based Go game service methods according to an embodiment of the present invention, and FIG. 18 is a method for preprocessing training data for generating a correct answer label among the methods for determining the situation of FIG. 17.

Referring to FIG. 17, the deep learning-based Go game service method according to an embodiment of the present invention may include a step (S100) of receiving, by the form judgment model server 400, a plurality of notations from the Go server.

In the deep learning-based Go game service method according to an embodiment of the present invention, an input feature extraction unit extracts an input feature from a checkerboard state (S) of a plurality of notations among the situation determination model 400a of the form determination model server 400. It may include step S200. A method of extracting an input feature follows the description of FIG. 7.

The deep learning-based Go game service method according to an embodiment of the present invention includes a step (S300) of generating a correct answer label based on a checkerboard state (S) in which the correct answer label generation unit extracts the input feature among the situation determination model 400a. can do. As an example, referring to FIG. 18, the step of generating a correct answer label (S300) may include a first pre-processing step (S301) in which the correct answer label generation unit ends in the current checkerboard state (S ). The first pre-processing step (S301) follows the description of FIGS. 9 to 10. The correct answer label generation step (S300) may include a second pre-processing step (S302) in which the correct answer label generator removes unnecessary stones in the first pre-processed checkerboard state (S). The second pre-processing step (S302) follows the description of FIGS. 9 to 10. The correct answer label generation step (S300) may include a third preprocessing step (S303) of changing each intersection point of the second preprocessed checkerboard state (S) into a position value by the correct answer label generation unit (S303). The third pre-processing step (S303) follows the description of FIG. 11. The step of generating the correct answer label (S300) may include a step (S303) of using the third preprocessed state as the correct answer label and providing the target data to the situation determination neural network. The step of providing target data (S301) follows the description of FIGS. 7 and 11.

The deep learning-based Go game service method according to an embodiment of the present invention may include training a situation determination neural network of the situation determination model 400a using a training data set (S400). A method of training (learning) the situation determination neural network follows the description of FIG. 7.

The deep learning-based Go game service method according to an embodiment of the present invention includes a step (S500) of constructing a situation determination model 400a after training of a situation determination neural network is completed. For example, the completion of the training of the situation determination neural network may be a case in which the loss of the situation determination in FIG. 7 becomes less than or equal to a predetermined value.

The deep learning-based Go game service method according to an embodiment of the present invention may include a step (S600) of inputting the current checkerboard state (S) into the situation determination model 400a in response to a situation determination request from the terminal.

The deep learning-based Go game service method according to an embodiment of the present invention may include a step (S700) of determining the status of the current checkerboard state (S) in which the status determination model 400a is input. The operation S700 of performing the position determination may follow the description in which the position determination model 400a described in FIG. 12 generates a position value of the current checkerboard state S.

The deep learning-based Go game service method according to an embodiment of the present invention may include the step S800 of outputting a situation determination result by the situation determination model server 400. The step of outputting the position determination result (S800) may follow the description in which the position determination model server 400 described in FIG. 12 provides the position determination result using the position value, the state of the board, and a predetermined threshold value.

Accordingly, the deep learning-based Go game service method according to the embodiment may determine the status of Go by using a deep learning neural network. In addition, the deep learning-based Go game service method according to the embodiment can accurately determine the status of Go by accurately classifying houses, sandstones, stones, gongbae, and big according to the Go rule. In addition, the deep learning-based Go game service method according to the embodiment may accurately determine the situation of Go by predicting houses, sandstones, stones, gongbae, and big according to the Go rule. In addition, the deep learning-based Go game service method according to the embodiment can quickly determine the status of the Go game.

-Emotion judgment unit

The ethos determination unit 400b according to an embodiment of the present invention is a component included in the situation determination model server 400, and is a checkerboard obtained through the situation determination model 400a in connection with the situation determination model 400a. Based on the position value of the state (S), it is possible to determine the ethos based on a specific starting point. In addition, the ethos determination unit 400b may allow the initiating model to perform ethos learning based on the determined ethos, so that the ethos representing a specific temperament or method may be utilized when playing a Go game. Here, the ethos determination is to determine which ethos is implemented, that is, in which ethos category, which is the best number predicted by the initiation model. In an embodiment, the airflow may be based on a predetermined criterion based on a 1) airflow type category or 2) a 1-n airflow stage (n=1, 2, 3, …) category. For example, the ethos may be classified into an aggressive ethos, a stable ethos, and/or a defensive ethos according to the ethos type category. In another example, the airflow is a first airflow step, a second airflow step, ... , The nth ethos may be in the mood. In detail, the situation determination model 400a may perform a situation determination on the number of embarkation candidates information obtained from the starting model server 300, and after that, the spirit determination unit 400b is based on the data derived through determination of the situation. It is possible to perform an ethos determination to determine which ethos category among the plurality of ethos categories in which the information on the number of candidates for launching is included.

In detail, the ethos determination unit 400b may first set target ethos information, which is preset information, on which ethos to perform learning in order to perform ethos determination. Here, the target wind information is preset information on which of a plurality of winds classified into various categories to be learned as a target, and 1) any one of the wind type categories (e.g., aggressive, stable, defensive, etc.) It may be information in which a target airflow is set by selecting or 2) information in which a target airflow is set by selecting any one of the 1 to n airflow step categories.

In addition, the ethos determination unit 400b is obtained from the launch model server 300 based on the score difference value between the player score value and the oponant score value according to the number of houses held by the situation determination model 400a. It is possible to set a predetermined airflow determination threshold for performing airflow determination on the number of candidates for starting. That is, the ethos determination unit 400b may perform ethos determination by comparing a predetermined ethos determination threshold value with a score difference value derived through judgment of the status of the number of candidates to be launched.

In more detail, first, the situation determination model 400a may obtain information on at least one or more embarkation candidates from the embarkation model server 300 as input data. In addition, the situation determination model 400a may generate information on the holding house by performing a situation determination on each of the acquired information on the number of candidates to be launched. Thereafter, the ethos determination unit 400b may calculate a player score value and an oponant score value based on the house information generated from the situation determination model 400a. In addition, the ethos determination unit 400b may calculate a score difference value representing a difference value between the player score value and the offer score value based on the calculated player and offer score values. In addition, the ethos determination unit 400b may generate ethos determination information for each of the at least one candidate number of embarkation information by comparing the calculated score difference value with a preset ethos determination threshold. Here, the ethos determination information may be information that determines a category of ethos implemented by the initiation of each candidate to be launched during the game of Go. A detailed description of this will be described later.

In addition, the airflow determination unit 400b may generate airflow learning data based on the generated airflow determination information. In detail, the air wind determination unit 400b may determine the air wind determination information corresponding to the predetermined target air wind information from among the air wind determination information derived for each of the received embarkation candidate number information. In addition, the ethos determination unit 400b may generate ethos learning data for implementing learning based on information on the number of embarkation candidates derived from the corresponding ethos determination information determined to correspond to the target ethos information. In addition, the air wind determination unit 400b may provide the generated air wind training data to the launch model server 300.

In addition, the air wind determination unit 400b may acquire air wind learning performance information as input data from the initiation model server 300 that has performed training based on the air wind learning data, and determines the air wind based on the obtained air wind learning performance information. Threshold can be adjusted. A more detailed description will be described later.

Meanwhile, FIG. 19 is a diagram for explaining the structure of the ethos determination unit 400b of the shape determination model server 400 of the present invention. Referring to FIG. 19, in order to perform a series of operations for determining the above ethos, the shape judgment model server 400 may include the above-described condition judgment model 400a and the ethos determination unit 400b. The determination model 400a and the ethos determination unit 400b may be operated by interworking with each other. In an embodiment, the ethos determination unit 400b sets ethos determination threshold, calculates a score difference value by deriving a player score value and an oponant score value, and generates ethos determination information based on the calculated score difference value. I can. In addition, in the embodiment, the airflow determination unit 400b may generate airflow learning data based on the generated airflow determination information, and adjust the airflow determination threshold based on the airflow learning performance information obtained from the initiation model server 300. I can.

That is, the ethos determination unit 400b of the pattern judgment model server 400 determines the ethos based on the judgment of the status of the checkerboard state (S) in connection with the status judgment model 400a according to an embodiment of the present invention. It may be a component for implementing a process that is applied when learning is performed based on a game of Go. A detailed description of this will be described later in the detailed description of a method of determining and applying the ethos based on deep learning below. In addition, in this embodiment, it is described that the ethos determination unit 400b is included in the shape determination model server 400, but in another embodiment, the ethos determination unit 400b is the Go server 200 and/or the start model server 300 Various embodiments, such as included in or implemented as a separate device, may be possible.

-How to determine and use ethos based on deep learning

Hereinafter, with reference to the drawings, to determine the ethos by determining the state of Go based on a deep learning neural network, and based on this, to explain in detail a method of providing a Go game service using ethos that represents a specific temperament or method when playing a Go game. do.

FIG. 20 is a flow chart illustrating a method of determining a situation of Go based on a deep learning neural network according to an embodiment of the present invention and utilizing an ethos indicating a specific temperament or method when playing a Go game. A conceptual diagram for explaining a method of determining the status of Go according to an embodiment and utilizing the ethos of playing a Go game.

Referring to FIGS. 20 and 21, the shape determination model server 400 may include a step (S101) of first setting target wind information. In detail, the shape judgment model server 400 may set target mood information, which is preset information, on how to perform learning through the processor 402. In this case, the target airflow information may be information for selecting any one of a plurality of airflow categories that are mood flow based on the airflow type category or the 1 to n airflow stage category. For example, the target airflow information may be any one of an aggressive airflow, a stable airflow, and/or a defensive airflow, a first airflow step, a second airflow step, ... , It may be any one of the nth airflow stage. At this time, the air type category may be implemented in various ways, such as further including a more subdivided category according to an embodiment, or including a less category by being simplified, and the 1 to n air air stage categories may be the first air flow according to the embodiment. The closer to the stage, the more defensive ethos is determined, and the closer to the nth ethos, the more aggressive ethos is determined to be determined, or vice versa.

In addition, the shape judgment model server 400 may include a step (S103) of setting an air condition judgment threshold. In detail, the shape determination model server 400 may set a predetermined airflow determination threshold for performing airflow determination through the airflow determination unit 400b. For example, the shape judgment model server 400 may preset an ethos judgment threshold to a predetermined value.

Subsequently, the situation determination model server 400 may include a step (S105) of obtaining information on the number of candidates to be initiated. In detail, the status determination model server 400 may receive information on the number of candidates for starting at least two or more derived from the starting model server 300. In addition, the situation determination model server 400 may use the received information on the number of candidates to be initiated as input data of the situation determination model 400a and/or the ethos determination unit 400b.

In addition, the situation judgment model server 400 may include a step (S107) of deriving a score value by performing a situation determination on each of the obtained information on the number of candidates to be initiated. In detail, the situation determination model server 400 performs a situation determination for each area of black and white for all intersections of the board to which the number of candidates to be launched obtained from the starting model server 300 is applied through the situation determination model 400a. You can derive the price. At this time, the criteria for determining the situation may be a house, a stone, a stone gongbae, or a big. In addition, the status judgment model server 400 may generate information on the house held based on the derived status value. In detail, the status judgment model server 400 calculates the number of houses expected to each player and the owner, through the status judgment model 400a, and determines the number of player houses and the number of owner houses. Can be created.

In addition, the status determination model server 400 that has generated the house information may derive a player score value and an offant score value based on the generated house information. In detail, the situation judgment model server 400 derives a player score value based on the number of player houses of the holding house information through the ethos determination unit 400b, and derives an offense score value based on the number of oponant houses. I can. At this time, the shape judgment model server 400 may preset a process for calculating a score value. For example, the situation judgment model server 400 may determine a score value calculation process as +1 per album. In addition, since the situation judgment model server 400 calculates a score value based on the number of candidates to be launched, the player score value may always be higher than the oponant score value. In this case, when the player score value is lower than the oponant score value, the shape judgment model server 400 may perform ethos determination after excluding the number of candidates for launch.

22 is a view for explaining a method of generating ethos determination information for each information on the number of candidates for undertaking according to an embodiment of the present invention. In this case, FIG. 22 may be displayed limited to that the player uses black go stones and the oponants use white go stones for effective explanation. However, it will be apparent that according to an embodiment, a player may use a white go stone and an oponant may use a black go stone. For example, referring to FIG. 22, the status determination model server 400 is based on the first number of candidates to be launched information, the second number of candidates to be launched information, and the third number of candidates to be launched information obtained from the launching model server 300, respectively. The score value of can be calculated. In detail, the situation judgment model server 400 may first pre-set a process for calculating a score value to determine the score value calculation process as +1 per album. In addition, the situation determination model server 400 may apply a score value calculation process to derive a player score value and an oponant score value based on information on the holding house matched with the information on the number of candidates to be launched. In detail, the situation judgment model server 400 will derive a player score value of +79 if the number of player houses is 79, when the number of candidates for launch is applied to the board, based on the information on the holding houses matched with the information on the number of candidates for launch. If the number of offense houses is 77, then the value of the offense score can be derived as +77. Likewise, the situation judgment model server 400 can derive a player score value of +80 if the number of player houses is 80 when the number of candidates for launch is applied to the checkerboard, and if the number of oponant houses is 77, the value of offense is + It can be derived as 77. In addition, when the number of third candidates is applied to the checkerboard, the situation judgment model server 400 can derive the player score value as +91 if the number of player houses is 91, and if the number of oponant houses is 68, the value is + It can be derived as 68.

Next, the situation determination model server 400 may include a step (S109) of calculating a score difference value based on the player score value and the oponant score value derived for each of the number of candidates to be initiated. In detail, the shape judgment model server 400 may calculate a score difference value for each candidate number information obtained from the launch model server 300 through the ethos determination unit 400b. In an embodiment, the situation judgment model server 400 may calculate a difference value between the player score value and the oponant score value for each embarkation candidate number information, and determine the result value as a score difference value for the embarkation candidate number. have. In this case, when the score difference value is calculated as a negative number, the situation judgment model server 400 may perform ethos determination after excluding the number of candidates for starting.

For example, referring to FIG. 22, when the player score value matched with the first embarkation candidate number information is +79, and the opposition score value is +77, the first embarkation candidate number is The difference value for the score can be calculated as +2. In addition, when the player score value matched with the second embarkation candidate number information is +80 and the oponant score value is +77, the situation judgment model server 400 increases the score difference value for the second embarkation candidate number by +3. Can be calculated as Similarly, when the player score value matched with the third embarkation candidate number information is +91, and the oponant score value is +68, the situation judgment model server 400 adds a score difference value for the third embarkation candidate number. It can be calculated as 23.

In addition, the situation judgment model server 400 that calculated the score difference value for each of the number of starting candidates includes a step (S111) of generating information about the number of starting candidates based on the calculated score difference value. I can. In detail, the shape judgment model server 400 may generate ethos determination information, which is information that determines a category of ethos implemented by the initiation of each candidate to be launched during a game of Go through the ethos determination unit 400b. In more detail, the shape judgment model server 400 may generate ethos determination information based on the calculated score difference value and a predetermined ethos determination threshold for each number of candidates for undertaking. For example, the shape judgment model server 400 may compare the score difference value with the ethos determination threshold, and determine that the score difference value is included in the aggressive ethos category when the difference value exceeds the ethos determination threshold (when the difference in score is large). have. In addition, the shape judgment model server 400 may compare the score difference value and the ethos determination threshold to determine that the difference in score is included in the stable ethos category when the difference value corresponds to the ethos determination threshold (when the difference in score is small). Likewise, the shape judgment model server 400 may compare the score difference value with the ethos determination threshold, and determine that the score difference value is included in the defensive ethos category when the difference value is less than the ethos determination threshold (when there is little difference in score).

For example, referring to FIG. 22, when the ethos determination threshold value is preset to '3', the first number of candidates to be initiated is determined if the score difference value matched with the first number of candidates to be initiated is +2. It can be judged as a defensive ethos category. In addition, if the score difference value matched with the information on the number of candidates for launching is +3, the number of candidates for launching may be determined as a stable ethos category. Likewise, if the score difference value matched with the third candidate number information is +23, the number of third candidates may be determined as an aggressive ethos category. That is, the shape judgment model server 400 may generate ethos judgment information based on a score difference value for each of the number of candidates to be launched information obtained from the launch model server 300.

Next, the shape judgment model server 400 that generates ethos determination information for each of the number of candidates for undertaking may include a step (S113) of generating ethos learning data based on the generated ethos determination information and target ethos information. have. In detail, the shape judgment model server 400 may determine, through the ethos determination unit 400b, efferent judgment information corresponding to the target ethos information from among ethos judgment information derived for each candidate number information. In addition, the shape judgment model server 400 may generate ethos training data for determining the number of candidates for launching information as learning data by selecting information on the number of embarkation candidates determined that the ethos determination information corresponds to the target ethos information. At this time, the ethos learning data is data including information generated based on the selected number of candidates to embark (e.g., information on the number of embarkation candidates, position value, player score value, oponant score value, score difference value, ethos determination information, etc.) Can be

For example, referring to FIG. 22, in the form judgment model server 400, the predetermined target air style information is an'aggressive spirit category', and the air style determination information matched with the first candidate number information is'defensive air style category', If the ethos determination information matched with the 2nd embarkation candidate information is'stable ethos category' and the ethos judgment information matched with the 3rd embarkation candidates information is'aggressive ethos category', the target ethos information corresponds to the aggressive ethos category. It is possible to generate ethos training data for determining information on the number of third candidates to be initiated as training data having ethos determination information. At this time, the situation judgment model server 400 includes information generated based on the third entrant candidate number in the generated ethos training data (e.g., information about the number of entrant candidates for the third embarkation candidate, a position value, a player score value, Effort learning data can be generated including an offense score value, score difference value, ethos determination information, etc.).

In addition, the shape judgment model server 400 may include transmitting the generated ethos training data (S115). In detail, the shape judgment model server 400 may transmit the ethos training data derived by performing ethos determination based on information on at least two or more candidates for launching to the launching model server 300.

Thereafter, the shape determination model server 400 may include a step (S117) of acquiring the ethos learning performance information from the initiation model server 300 that has received the ethos training data. In detail, the shape judgment model server 400 performs self-play learning based on the ethos learning data from the initiation model server 300, and determines whether the ethos learning data is suitable data for learning the target ethos You can receive information.

In addition, the shape judgment model server 400 may include a step (S119) of adjusting an airflow determination threshold based on the obtained airflow learning performance information. In detail, the shape judgment model server 400 may adjust the airflow determination threshold based on the airflow learning performance information through the airflow determination unit 400b, thereby improving the accuracy of airflow determination.

For example, first, the shape judgment model server 400 pre-sets the target ethos information as a'stable ethos category' and presets the ethos determination threshold to '3', and the score difference value for a specific number of candidates for launch is the corresponding ethos When it corresponds to the determination threshold value 3, it may be preset to determine that it is a stable ethos category. Thereafter, the shape judgment model server 400 may generate ethos determination information based on the score difference value between the corresponding efferent judgment threshold and the number of candidates to be launched, and generate ethos learning data based on the generated ethos judgment information. , The generated ethos training data may be transmitted to the initiation model server 300.

Subsequently, the ethos learning assistant 340 of the embarking model server 300 that received the ethos training data from the ethos model server 400 includes a score difference value for the number of candidates for embarking included in the received ethos learning data, The ethos learning performance information may be generated by comparing the score difference value for the number of candidates to be launched based on self-play learning to which the number of candidates to be launched is applied. For example, the initiation model server 300 determines that the score difference value for the number of initiation candidates on the ethos training data is +3, but the maximum difference value for the number of initiation candidates derived based on actual self-play learning is +2.5. If so, it is possible to generate ethos learning performance information including error information for determining that there is a predetermined error in the predicted score difference value. In this case, the error information may include information on the presence or absence of an error and/or degree of the error. In addition, the starting model server 300 may transmit ethos training performance information including error information to the form judgment model server 400.

Subsequently, the shape determination model server 400, which received the above-described airflow learning performance information from the launch model server 300, adjusts a preset airflow determination threshold to determine the target airflow based on the received airflow learning performance information. I can. That is, in the example, the shape judgment model server 400 may increase and adjust the ethos determination threshold preset to '3' to '3.5' based on ethos learning performance information in order to determine a stable ethos category set as the target ethos. . This is, in a situation where it is determined that it is a stable ethos when the ethos determination threshold is preset to 3 and the score difference value corresponds to 3, when it is determined that the maximum score difference value based on the result of self-play learning is 2.5, the preset ethos This operation may be due to the fact that the probability that the score difference value based on self-play learning increases when the determination threshold is further increased, thereby increasing the likelihood that it approaches 3. In this example, although the above description has been described, it will be apparent that various embodiments are possible.

In this way, the shape judgment model server 400 generates ethos learning data by performing ethos determination based on the information on the number of candidates to embark obtained from the launching model server 300, and self-learning based on the ethos learning data. It is possible to improve the accuracy of the air flow determination operation for the obtained number of launch candidates by acquiring the air wind learning performance information from the launch model server 300 and adjusting the air wind determination threshold. It is possible to increase the quality of the Go game service that determines and utilizes the ethos based on running.

On the other hand, hereinafter, a method of determining and utilizing a deep learning based ethos according to another embodiment of the present invention will be described. In the description to be described later, descriptions overlapping with those described above may be omitted.

23 is a conceptual diagram for explaining a method of determining the status of Go and utilizing the ethos of playing a Go game according to another embodiment of the present invention. Referring to FIG. 23, first, the launching model server 300 according to another embodiment may set target airflow information for determining the airflow to be learned through the processor 302. Here, as in the embodiment, the target airflow information may be information on selecting any one of a plurality of airflow categories in which mood flows are performed based on an airflow type category or 1 to n airflow stage categories.

In this case, the launching model server 300 according to another exemplary embodiment may set target wind information for each launching model including a plurality of launching models. In detail, the launching model server 300 may include at least one launching model that matches each airflow category 1:1 in proportion to the number of categories in which the airflow is classified. That is, the initiation model server 300 may match each initiation model to each ethos category in a 1:1 manner, and may set the target ethos information of each initiation model as an ethos category matched with each initiation model. Thus, the starting model server 300 may perform learning on a specific target atmosphere for each starting model.

For example, the launch model server 300 may include n launch models when the number of ethos categories is n. In addition, the launch model server 300 matches each ethos category and launching model 1:1, so that the first ethos category is set as the target ethos information of the first launching model, and the second ethos category is a second launching model. It is set as the target air wind information of, and in the same way, the nth air wind category may be set as the target air wind information of the nth launch model. As another example, the launching model server 300 may include at least three launching models that are 1:1 matched to each of the airs when there are an aggressive airflow category, a stable airflow category, and a defensive airflow category. That is, the initiation model server 300 may include a first initiation model matching the aggressive ethos category, a second initiation model matching the stable ethos category, and a third initiation model matching the defensive ethos category. In addition, the launch model server 300 sets the target air wind information of the first launch model as an aggressive air wind category, the target air wind information of the second launch model as a stable air wind category, and the target air wind information of the third launch model as a defensive air wind category. I can.

In addition, the status determination model server 400 according to another exemplary embodiment may obtain target ethos information for each of a plurality of launch models from the launch model server 300 as described above. In addition, the situation determination model server 400 may obtain information on the number of candidates to be launched, which is derived at least two or more from each model. In addition, the shape judgment model server 400 may generate ethos determination information by performing ethos determination based on the acquired information on the number of embarking candidates for each embarking model. In addition, the shape judgment model server 400 may generate ethos training data for each launching model based on the generated airflow determination information and target airflow information for each launching model. In addition, the form judgment model server 400 may collectively provide the generated ethos training data to each matched starting model.

For example, if the first initiating model sets the aggressive ethos category as the target ethos, the second initiation model uses the stable ethos category as the target ethos, and the third launching model uses the defensive ethos category as the target ethos The model server 400 may first determine the ethos of each of the starting candidates based on information on the number of starting candidates obtained at least two or more from each starting model. And the shape judgment model server 400 transmits the ethos learning data based on the number of launching candidates optimized for the learning of the aggressive ethos to the first launching model through the ethos determination, and the second launching model is optimized for stable ethos learning. The ethos learning data based on the number of initiating candidates may be transmitted, and the ethos learning data based on the number of initiating candidates optimized for learning of the defensive ethos can be transmitted to the third initiation model to provide collectively. That is, the shape judgment model server 400 may collectively perform and learn the ethos of a plurality of ethos, thereby implementing a Go game service to which ethos is applied more efficiently.

In addition, in another embodiment, the Go game service device including and/or interlocking with the launching model server 300 and the shape judgment model server 400 as described above is the processor 302 of the launching model server 300 and/or Through the processor 402 of the judgment model server 400, the game play of Go may be performed by applying different ethos according to the progress of the game of Go (eg, the degree of elapsed time) or player selection. In detail, the Go game service device is configured according to the progress of the Go game or the player selection according to the preset information for automatic wind control (e.g., information set to operate with a specific wind according to the elapsed time) or the player's selection input. Air-air conversion can be performed. In this case, the Go game service apparatus may change the starting model used according to the progress of the Go game in order to change the spirit to be applied to the Go game play.

For example, the Go game service device has a preset automatic airflow control when there is a first launching model that has learned an aggressive ethos, a second launching model that has learned an admissive airflow, and a third launching model that has learned a defensive ethos. Depending on the setting information or the player's choice, at the beginning of the Go game, the second launch model is used to play the Go game with a stable ethos, and in the middle of the Go game, the third launch model is used to play the Go game with a defensive ethos. In the second half of the game of Go, the first launch model can be used to play the Go game with an aggressive spirit. That is, the Go game service device may further improve the usability of a Go game service that utilizes the ethos when playing a Go game based on a deep learning neural network by using different ethos according to the progress of the Go game or player selection.

As described above, the method and apparatus for a Go game service based on deep learning according to an embodiment of the present invention accurately classify houses, sandstones, stones, gongbae, and big according to the Go rule to predict the situation of Go, thereby predicting the situation of Go. And there is an effect of being able to accurately determine the ethos for a specific starting point.

In addition, the Go game service method and apparatus thereof based on deep learning according to an embodiment of the present invention promotes the progress of the game based on various ethos when playing the Go game, thereby providing a game of Go based on a diversified play method. It can be performed and the difficulty of the game can be precisely adjusted, and through this, the quality and interest of the Go game can be improved.

In addition, the deep learning-based Go game service method and device thereof according to an embodiment of the present invention determine not only the winning but also the score by determining the ethos based on the difference between the score between the player and the oponant, and performing a match based on this. There is an effect that can perform the game play of Go considering the difference of.

In addition, the deep learning-based Go game service method and device thereof according to an embodiment of the present invention can efficiently and systematically implement learning for implementing the ethos by operating to perform learning optimized for a specific ethos of a target. It can have an effect.

Further, the embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded in the computer-readable recording medium may be specially designed and constructed for the present invention or may be known and usable to those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magnetic-optical media such as floptical disks. medium), and a hardware device specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of the program instructions include not only machine language codes such as those produced by a compiler but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device can be changed to one or more software modules to perform the processing according to the present invention, and vice versa.

The specific implementations described in the present invention are examples and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of the lines between the components shown in the drawings exemplarily represent functional connections and/or physical or circuit connections, and in an actual device, various functional connections that can be replaced or added Connections, or circuit connections. In addition, if there is no specific mention such as “essential” or “importantly”, it may not be an essential component for the application of the present invention.

In addition, in the detailed description of the present invention, it has been described with reference to preferred embodiments of the present invention, but those skilled in the art or those of ordinary skill in the relevant technical field will have And it will be understood that various modifications and changes can be made to the present invention within a range not departing from the technical field. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be determined by the claims.

100 terminals
200 Go Server
300 launch model server
310 Search section
320 Self Play Department
330 initiating neural network
340 ethos learning assistant
400 profile judgment model server
400a position judgment model
410 Neural Network
420 input feature extraction unit
430 correct answer label generator
400b ethos judgment unit

Claims (14)

A communication unit for receiving at least two or more embarkation candidates based on a plurality of notations and a checkerboard state;
A storage unit for storing a situation determination model and an ethos determination unit; And
The situation determination model is read out, the learning of the situation determination model is performed, the situation of the board to which the number of embarkation candidates is applied is determined by using the learned situation determination model, and the ethos determination unit is initiated based on the determined situation. Including; and a processor for generating efferent determination information that is information for determining the ethos of the number of candidates,
The situation judgment model,
An input feature extraction unit for extracting an input feature in the input checkerboard state; And
Including a situation determination neural network that generates a position value for the intersection of the input checkerboard state based on the extracted input feature,
The ethos determination unit,
Generating house information based on the above position value,
The above holding house information,
The information that calculated the number of player houses and the number of owner houses
Form judgment model server. The method of claim 1,
The ethos determination unit,
The holding house information is generated by determining the house area of the player and the owner using the position value, a predetermined threshold value, and the presence or absence of a stone.
Form judgment model server. The method of claim 1,
The ethos determination unit,
The starting model sets the target wind information that determines the wind to learn.
Form judgment model server. The method of claim 3,
The ethos determination unit,
Generate the ethos determination information based on the number of candidates for launch and the target ethos information,
Calculating a player score value and an offense score value based on the holding house information, and generating a score difference value based on the calculated player score value and the offant score value.
Form judgment model server. The method of claim 4,
The ethos determination unit,
Setting an air wind determination threshold that is a parameter of the air wind determination process with respect to the number of launch candidates, and comparing the score difference value with the air wind determination threshold to generate the air wind determination information for each of the launch candidates.
Form judgment model server. The method of claim 5,
The ethos determination unit,
Generating ethos learning data that assists self-learning of the initiation model based on the ethos determination information
Form judgment model server. The method of claim 6,
The communication unit,
Obtaining ethos learning performance information from the initiation model,
The ethos determination unit,
Adjusting the airflow determination threshold based on airflow learning performance information obtained from the initiation model
Form judgment model server. Communication unit for receiving the ethos learning data;
A storage unit for storing the starting model; And
Including; a processor that reads out the initiation model, performs training of the initiation model, and generates two or more initiation candidates based on a checkerboard state by using the learned initiation model,
The launch model,
A search unit that provides the number of starting candidates based on a Monte Carlo Tree Search (MCTS); and, an initiating neural network guiding the search unit; and performing self-play so that the initiating neural network is self-learning. Self-play unit; And an ethos learning assistant for supporting the self-learning based on the ethos learning data,
The self-play unit,
Performing a Go game between the higher version launching model and the lower version launching model learned based on the ethos learning data
Launch model server. The method of claim 8,
The ethos learning auxiliary unit,
Generating ethos learning performance information, which is information about diagnosing ephemeris determination information based on the result of the self-learning
Launch model server. The communication unit, the situation determination model, and a storage unit in which the mood determination unit are stored, the situation determination model server including a processor that drives the situation determination model and the airflow determination unit determines the state of the board by determining the state of the board to determine the state of spirit, and to generate the airborne learning data. In the deep learning-based Go game service method,
Setting, by the processor, target wind information;
Setting, by the airflow determination unit, an airflow determination threshold;
Obtaining, by the communication unit, two or more embarkation candidates based on a checkerboard state;
Determining, by the processor, a situation in a checkerboard state to which the number of candidates for launch is applied using the situation determination model;
Calculating, by the processor, a score difference value for each of the number of candidates for launch based on the determined situation using the ethos determination unit;
Generating, by the processor, air-wind determination information for each of the number of candidates for launch based on the calculated score difference value and the air-wind determination threshold using the air-wind determination unit; And
And generating and transmitting, by the communication unit, the ethos learning data for each of the number of embarkation candidates based on the generated ethos determination information and the target ethos information,
The air condition determination information,
Information that determines the category of ethos implemented by the initiation of the number of candidates
Go game service method based on deep learning. The method of claim 10,
Further comprising the step of receiving the ethos of learning performance information from the starting model server,
The ethos learning performance information,
Information obtained by diagnosing the air wind determination information based on the air wind learning data by the launch model server
Go game service method based on deep learning. The method of claim 11,
Further comprising the step of adjusting the air-wind determination threshold based on the air-air learning performance information
Go game service method based on deep learning. The method of claim 10,
The step of calculating the score difference value,
Generating house information, which is information obtained by calculating the number of player houses and the number of owners, by deriving a position value based on the position determination for each of the number of candidates for launch;
Comprising the step of generating the score difference value by calculating a player score value and an offense score value based on the holding house information
Go game service method based on deep learning. The method of claim 10,
The categories of the above ethos are,
Includes aggressive, stable, and defensive ethos,
The aggressive ethos is a case in which the score difference value is greater than the effervescence determination threshold, the stable ethos is when the score difference value is the same as the effervescence judgment threshold, and the defensive ethos is when the score difference value is the effervescence judgment threshold Less than
Go game service method based on deep learning.

Images