KR102299140B1 - Method and device for baduk game service based on deep-learning - Google Patents

Abstract

A Go game service method and apparatus based on deep learning according to an embodiment of the present invention, a communication unit for receiving at least two or more candidates based on a plurality of notations and Go board state; a storage unit for storing the situation determination model and the ethos determination unit; and reading the situation determination model to perform learning of the situation determination model, and using the learned situation determination model to determine the condition of the checkerboard to which the number of candidates to start is applied, and the ethos determination unit based on the determined situation Includes; a processor for generating wind judgment information that is information for determining the wind for the number of candidates to set out; and a situation determination neural network for generating a situation value for the intersection of the inputted checkerboard state based on the extracted input features, wherein the ethos determining unit generates holding house information based on the situation value, and the The holding house information is the situation judgment model server, which is information obtained by calculating the number of players and the number of component collections.

Description

Go game service method and device based on deep learning

The present invention relates to a method and apparatus for providing a Go game service based on deep learning. More specifically, it relates to a method for providing a Go game service and an apparatus therefor that utilize the ethos indicating a specific temperament or method during Go game play by determining Go situation based on a deep learning neural network.

As the use of user terminals such as smartphones, tablet PCs, personal digital assistants (PDA), and notebook computers has become popular and information processing technology has developed, it has become possible to play Go, a type of board game, using user terminals. It became possible to play Go games with a programmed artificial intelligence computer.

Compared to other board games such as chess and chess, there are many cases of Go, so there is a limit to how artificial intelligence computers can play matches at human level, and research to increase the energy of artificial intelligence computers is being actively conducted. Recently, developers applied the Monte Carlo Tree Search (MCTS) algorithm and deep learning technology to artificial intelligence computers, raising the energy of artificial intelligence computers to the level of professional engineers.

However, in general, existing Go AI programs have difficulty in starting considering the difference in scores between players and opponents because they only learn how to win. That is, the existing Go AI program has a problem in that the score difference gradually narrows toward the second half of Go game play, and the tendency to win with a very small score difference (defensive tendency) is prominent.

In addition, the existing Go AI program has difficulty in implementing the ethos, which is a specific temperament or method of a player that is implemented when playing the actual Go game. Here, ethos refers to a unique play style or temperament that appears in the course of a game such as Go or shogi. That is, the existing Go AI program is learned to win and implemented only with a consistent play method, so there is a limit to playing Go games with various ethos.

In addition, in order to effectively play Go with an artificial intelligence computer, it is necessary to precisely adjust the game difficulty according to the player's needs or abilities, but the technology for implementing this is insufficient, so the introduction of new technology is required.

Patent Document 1: Unexamined Patent Publication No. 10-2015-0129265

The present invention relates to a method and apparatus for providing a Go game service based on deep learning devised to solve the above problems. More specifically, it is intended to propose a method and device for providing a Go game service that utilizes the ethos indicating a specific temperament or method during Go game play by judging Go situation based on a deep learning neural network.

In detail, the present invention determines the ethos by judging the situation of Go based on the predictions of houses, stones, stones, banquets, and big according to the rules of Go, and performs learning based on the ethos to apply the ethos when playing the Go game. The purpose is to provide a method and apparatus for providing a Go game service based on deep learning in progress.

However, the technical problems to be achieved by the present invention and embodiments of the present invention are not limited to the technical problems as described above, and other technical problems may exist.

A Go game service method and apparatus based on deep learning according to an embodiment of the present invention, a communication unit for receiving at least two or more candidates based on a plurality of notations and Go board state; a storage unit for storing the situation determination model and the ethos determination unit; and reading the situation determination model to perform learning of the situation determination model, and using the learned situation determination model to determine the condition of the checkerboard to which the number of candidates to start is applied, and the ethos determination unit based on the determined situation Includes; a processor for generating wind judgment information that is information for determining the wind for the number of candidates to set out; and a situation determination neural network for generating a situation value for the intersection of the inputted checkerboard state based on the extracted input feature; the ethos determining unit generates, based on the situation value, holding house information, and The possession information is information obtained by calculating the number of players and the number of component collections.

At this time, the ethos determining unit generates the possession house information by determining the house area of the player and the Oponent by using the situation value, a predetermined threshold value, and the presence or absence of a stone.

In addition, the ethos determining unit sets target ethos information for determining the ethos to be learned by the start model.

In addition, the ethos determination unit generates the ethos determination information based on the number of candidates to start and the target ethos information, and calculates a player score value and an element score value based on the holding house information, and the calculated player A score difference value is generated based on the score value and the component score value.

In addition, the spirit determination unit sets a gi wind determination threshold, which is a parameter of the gi wind determination process for the number of start candidates, and compares the score difference value with the gi wind determination threshold value, and the gi wind determination information for each of the number of start candidates. create

In addition, the ethos determining unit generates ethos learning data that assists self-learning of the set off model based on the ethos determination information.

In addition, the communication unit, obtains the gi wind learning performance information from the start model,

The spirit determination unit adjusts the spirit determination threshold based on the wind learning performance information obtained from the set off model.

On the other hand, the Go game service method and the apparatus based on deep learning according to an embodiment of the present invention, the communication unit for receiving the ethos learning data; A storage unit for storing the launch model; and a processor that reads the set off model and performs learning of the set off model, and generates two or more start candidates based on the checkerboard state using the learned start model. A search unit that provides the number of start candidates based on a search (Monte Carlo Tree Search; MCTS); and a start neural network that guides the search unit; and a self-play unit that self-learns the start neural network by performing self-play ; And it includes a ethos learning auxiliary unit for assisting the self-learning based on the ethos learning data, wherein the self-play unit performs a Go game between the higher version start model and the lower version start model learned based on the ethos learning data. .

At this time, the gi wind learning auxiliary unit, based on the result of the self-learning generates gi wind learning performance information that is information for diagnosing the gi wind determination information.

In addition, the Go game service method and apparatus based on deep learning according to an embodiment of the present invention includes a communication unit, a storage unit in which a situation determination model and a style determination unit are stored, a processor for driving the situation determination model and the style determination unit In the deep learning-based Go game service method for determining the ethos and generating ethos learning data by determining the situation of the Go board state by a situation judgment model server comprising the steps of: setting, by the processor, target ethos information; setting, by the air wind determination unit, a wind wind determination threshold; acquiring the number of two or more start candidates based on the state of the checkerboard by the communication unit; determining, by the processor, the situation of the checkerboard state to which the number of candidates to start is applied by using the situation determination model; calculating, by the processor, a score difference value for each of the number of start candidates based on the determined situation using the ethos determination unit; generating, by the processor, the geography determination information for each of the number of candidates to set out on the basis of the calculated score difference value and the gi wind determination threshold using the gi wind determination unit; and generating and transmitting, by the communication unit, the wind learning data for each of the number of start candidates based on the generated spirit determination information and the target air wind information, wherein the spirit determination information is the number of start candidates. It is information that judges the category of ethos that is implemented by initiation.

At this time, further comprising the step of receiving the gi wind learning performance information from the starting model server, the gi wind learning performance information, the starting model server is the information for diagnosing the gi wind determination information based on the gi wind learning data.

In addition, the Go game service method and apparatus based on deep learning according to an embodiment of the present invention, further comprising the step of adjusting the spirit judgment threshold based on the spirit learning performance information.

In addition, the calculating of the score difference value includes the steps of deriving a situation value based on the situation determination for each of the number of candidates to start, and generating holding house information, which is information obtained by calculating the number of player houses and the number of oponent collections; , calculating a player score value and an element score value based on the holding house information to generate the score difference value.

In addition, the category of the ethos includes an aggressive ethos, a stable ethos, and a defensive ethos, wherein the aggressive ethos is a case in which the score difference value is greater than the ethos judgment threshold, and the stable ethos is the score difference value is the ethos. It is the same as the judgment threshold, and the defensive ethos is a case in which the score difference value is smaller than the ethos judgment threshold.

A Go game service method and device based on deep learning according to an embodiment of the present invention accurately classify a house, a stone, a stone, a ball, and a big according to the Go rules and predict the condition of Go by predicting the situation and specific It has the effect of accurately judging the ethos of the starting point.

In addition, the Go game service method and the device based on deep learning according to an embodiment of the present invention, the Go game based on a diversified play method by promoting the progress of the game based on various ethos during the Go game play. It can be performed and the difficulty of the game can be precisely adjusted, which has the effect of improving the quality and interest of the Go game.

In addition, the Go game service method and the apparatus based on deep learning according to an embodiment of the present invention, by judging the ethos based on the score difference between the player and the component, and performing a game based thereon, not only wins, but also scores It has the effect of being able to perform Go game play considering even the difference of .

In addition, the Go game service method and the apparatus based on deep learning according to an embodiment of the present invention, by operating so that learning is performed optimized for a specific ethos to be targeted, efficiently and systematically implement learning to implement ethos can have an effect.

However, the effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood from the description below.

1 is an exemplary diagram of a deep learning-based Go game service system according to an embodiment of the present invention.
Figure 2 is a diagram for explaining the initiation model structure of the initiation model server for the initiation of an artificial intelligence computer in the deep learning-based Go game service according to an embodiment of the present invention.
Figure 3 is a view for explaining the movement probability distribution for the starting point according to the policy of the starting model.
4 is a view for explaining the value value and the number of visits to the starting point of the starting model.
5 is a view for explaining a process in which the launch model is launched according to the pipeline of the search unit.
6 is an exemplary diagram showing a screen providing a situation determination function of a deep learning-based Go game service according to an embodiment of the present invention.
7 is a diagram for explaining the structure of the situation determination model of the situation determination model server of the present invention.
8 is a diagram for explaining one block of the neural network structure composed of a plurality of blocks of the situation determination model of the present invention.
9 is a view for explaining the first and second pre-processing steps for generating a correct answer label used to learn the situation judgment model of the present invention.
10 is a view for explaining the first and second pre-processing steps for generating a correct answer label used to learn the situation judgment model of the present invention.
11 is a view for explaining a third pre-processing step for generating a correct answer label used for learning the situation judgment model of the present invention.
12 is a view for explaining a situation determination result of the situation determination model of the present invention.
13 is a view comparing the situation determination result of the situation determination model of the present invention with the situation determination result by the deep learning model according to the prior art.
14 is a view comparing the situation determination result of the situation determination model of the present invention with the situation determination result by the deep learning model according to the prior art.
15 is a view comparing the situation determination result of the situation determination model of the present invention with the situation determination result by the deep learning model according to the prior art.
16 is an exemplary diagram of a signal flow in a deep learning-based Go game service system according to an embodiment of the present invention.
17 is a method for determining a situation in a deep learning-based Go game service method according to an embodiment of the present invention.
18 is a pre-processing method of training data for generating a correct answer label in the method of determining the situation of FIG. 17 .
19 is a view for explaining the structure of the ethos determining unit of the situation judgment model server of the present invention.
20 is a flowchart for explaining a method for determining a Go situation based on a deep learning neural network according to an embodiment of the present invention and using the ethos indicating a specific temperament or method during the Go game play.
21 is a conceptual diagram for explaining a method of determining a Go situation and utilizing the ethos when playing a Go game according to an embodiment of the present invention.
22 is a view for explaining a method of generating wind judgment information for each of the number of candidates to start information according to an embodiment of the present invention.
23 is a conceptual diagram for explaining a method of determining a Go situation and utilizing the ethos when playing a Go game according to another embodiment of the present invention.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. Effects and features of the present invention, and a method of achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms. In the following embodiments, terms such as first, second, etc. are used for the purpose of distinguishing one component from another, not in a limiting sense. Also, the singular expression includes the plural expression unless the context clearly dictates otherwise. In addition, terms such as include or have means that the features or components described in the specification are present, and do not preclude the possibility that one or more other features or components will be added. In addition, in the drawings, the size of the components may be exaggerated or reduced for convenience of description. For example, since the size and thickness of each component shown in the drawings are arbitrarily indicated for convenience of description, the present invention is not necessarily limited to the illustrated bar.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when described with reference to the drawings, the same or corresponding components are given the same reference numerals, and the overlapping description thereof will be omitted. .

1 is an exemplary diagram of a deep learning-based Go game service system according to an embodiment of the present invention.

Referring to Figure 1, the deep learning-based Go game service system according to the embodiment, the terminal 100, the Go server 200, the start model server 300, the layout judgment model server The layout judgment model server 400 and It may include a network 500 .

Each component of FIG. 1 may be connected through a network 500 . The terminal 100, the Go server 200, the start model server 300 and the situation judgment model server type judgment model server 400 refers to a connection structure in which information can be exchanged between each node, such as the network of An example includes a 3rd Generation Partnership Project (3GPP) network, a Long Term Evolution (LTE) network, a World Interoperability for Microwave Access (WIMAX) network, the Internet, a Local Area Network (LAN), and a Wireless Local Area Network (Wireless LAN). , WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth (Bluetooth) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc. are included, but are not limited thereto.

- terminal

First, the terminal 100 is a terminal of a user who wants to be provided with a Go game service. In addition, the terminal 100 is one or more computers or other electronic devices used by a user to execute applications that perform various tasks. For example, it includes a computer, a laptop computer, a smart phone, a mobile phone, a PDA, a tablet PC, or any other device operable to communicate with the Go server 200 . However, not limited thereto, the terminal 100 is executed on various machines, includes processing logic that interprets and executes commands stored in a plurality of memories, and provides a graphical user interface (GUI) on an external input/output device. It may include various other elements, such as processes for displaying graphical information. In addition, the terminal 100 may be connected to an input device (eg, a mouse, a keyboard, a touch-sensitive surface, etc.) and an output device (eg, a display device, a monitor, a screen, etc.). Applications executed by the terminal 100 may include a game application, a web browser, a web application running on a web browser, word processors, media players, spreadsheets, image processors, security software or the like. .

In addition, the terminal 100 may include at least one memory 101 for storing instructions, at least one processor 102 , and a communication unit 103 .

The memory 101 of the terminal 100 may store a plurality of application programs or applications driven in the terminal 100 , data for operation of the terminal 100 , and commands. The instructions are executable by the processor 102 to cause the processor 102 to perform the operations, and the operations send a Go game execution request signal, game data transmission and reception, start information transmission and reception, transmission of a situation determination request signal, the situation It may include operations of receiving the determination result and receiving various types of information. In addition, in terms of hardware, the memory 101 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and the memory 130 performs the storage function of the memory 101 on the Internet. It may be a web storage that performs.

The processor 102 of the terminal 100 may perform data processing for receiving a Go game service by controlling the overall operation. When the Go game application is executed in the terminal 100 , the Go game environment is configured in the terminal 100 . And the Go game application exchanges Go game data with the Go server 200 through the network 500 so that the Go game service is executed on the terminal 100 . These processors 102 are ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers (controllers), micro It may be a controller (micro-controllers), microprocessors (microprocessors), any type of processor for performing other functions.

The communication unit 103 of the terminal 100 includes the following communication methods (eg, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink (HSUPA)) Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc.), WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA ( Digital Living Network Alliance), WiBro (Wireless Broadband), and WiMAX (World Interoperability for Microwave Access) can transmit and receive a wireless signal to and from at least one of a base station, an external terminal, and a server on a network built in accordance with.

- Go server

The Go game service provided by the Go server 200 may be configured in a form in which a virtual computer user and a real user provided by the Go server 200 participate in the game together. This means that one real user and one computer user play the game together in the Go game environment implemented on the user-side terminal 100 . In another aspect, the Go game service provided by the Go server 200 may be configured in such a way that a plurality of user-side devices participate and the Go game is played.

Go server 200 may include at least one memory 201 for storing instructions, at least one processor 202 and a communication unit 203 .

The memory 201 of the Go server 200 may store a plurality of application programs or applications running in the Go server 200, data for the operation of the Go server 200, and commands. . The instructions are executable by the processor 202 to cause the processor 202 to perform the operations, and the operations are game execution request signal reception, game data transmission/reception, start information transmission/reception, situation determination request signal transmission/reception, situation determination result transmission/reception and various transmission operations. In addition, the memory 201 may store a plurality of notations that were played in the Go server 200 or a plurality of previously published notations. Each of the plurality of notations may include all of the information from the first start that is the first start information of the start of the game to the final start that the game ends. That is, a plurality of notations may include history information about the start. The Go server 200 allows to provide a plurality of stored notations for the training of the situation judgment model server 400 to the position judgment model server 400 . In addition, in terms of hardware, the memory 201 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and the memory 201 performs the storage function of the memory 201 on the Internet. It may be a web storage that performs.

The processor 202 of the Go server 200 may perform data processing for providing a Go game service by controlling the overall operation. These processors 202 are ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers (controllers), micro It may be a controller (micro-controllers), microprocessors (microprocessors), any type of processor for performing other functions.

The Go server 200 may communicate with the terminal 100 , the start model server 300 and the situation judgment model server 400 via the network 500 through the communication unit 203 .

- Launch model server

Initiation model server 300 may include a separate cloud server or computing device. In addition, the initiation model server 300 may be a neural network system installed in the data processing unit of the processor or Go server 200 of the terminal 100, but in the following, the initiation model server 300 is the terminal 100 or the Go server ( 200) and a separate device.

Initiation model server 300 may include at least one memory 301 for storing instructions, at least one processor 302 and a communication unit 303 .

The start model server 300 is an artificial intelligence computer that can learn by itself according to the rules of Go to build a set-up model, which is a deep learning model, and play a game with the user of the terminal 100. initiation can be carried out. The detailed description of the set-up model server 300 training with the set-off model follows the description of the set-up model of FIGS. 2 to 5 .

Memory 301 of the initiation model server 300 is a plurality of application programs (application program) or applications (application) driven in the initiation model server 300, data for the operation of the initiation model server 300, commands can be saved The instructions are executable by the processor 302 to cause the processor 302 to perform operations, and the operations may include an initiation model learning (training) operation, an initiation information transmission and reception, and various transmission operations. In addition, the memory 301 may store an initiation model that is a deep learning model. In addition, in terms of hardware, the memory 301 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and the memory 301 performs the storage function of the memory 301 on the Internet. It may be a web storage that performs.

The processor 302 of the set-off model server 300 reads the set-off model stored in the memory 302, and performs the set-up model learning and Go start to be described below according to the built-up neural network system. In an embodiment, the processor 302 of the set-off model server 300 may be derived by predicting at least two or more specific set points that are determined to be the best number in a specific checkerboard state (S). In addition, the processor 302 may transmit the derived at least two or more specific starting points to the situation judgment model server 400 . In addition, the processor 302 of the set off model server 300 may perform self-play (self-play) learning to learn the set off model in a specific specific ethos. In addition, the processor 302 may diagnose the performance of whether the spirit is properly learned and determined through self-play learning.

On the other hand, according to the embodiment, the processor 302 is configured to include a main processor for controlling all units, and a plurality of graphics processors (Graphics Processing Unit, GPU) for processing a large-capacity operation required when driving a neural network according to the starting model. can be

The start model server 300 may communicate with the Go server 200 via the network 500 through the communication unit 303 .

- Situation judgment model server

The situation judgment model server 400 may include a separate cloud server or computing device. In addition, the situation judgment model server 400 may be a neural network system installed in the data processing unit of the processor or the Go server 200 of the terminal 100, but in the following, the situation judgment model server 400 is the terminal 100 or Go It will be described as a separate device from the server 200 .

The situation judgment model server 400 may include at least one memory 401 for storing instructions, at least one processor 402 and a communication unit 403 .

The posture judgment model server 400 may receive the training data set from the Go server 200 through the communication unit 403 . The training data set may be a plurality of notations and situation determination information for the plurality of notations. The situation judgment model server 400 uses the received training data set to supervise learning so as to determine the situation for the state of the Go board on which the Go balls are placed, to build a deep learning model, the situation judgment model, and the terminal 100 user's situation A situational judgment may be performed according to a judgment request. The detailed description of the situation judgment model server 400 training with the situation judgment model follows the description of the situation judgment model of FIGS. 6 to 18 .

The memory 401 of the situation judgment model server 400 is a plurality of application programs or applications driven in the situation judgment model server 400, data for the operation of the situation judgment model server 400 , commands can be stored. The instructions are executable by the processor 402 to cause the processor 402 to perform the operations, and the operations include a situation determination model learning (training) operation, performing a situation determination, transmitting a situation determination result, receiving a plurality of notation information, and It may include various transmission operations. In addition, referring to FIG. 19, in the embodiment, the memory 401 determines and learns the Go situation based on the deep learning model, the situation determination model 400a and the deep learning neural network, and applies the ethos when playing the Go game. It is possible to store the ethos determining unit 400b for implementing the operation. In addition, in terms of hardware, the memory 401 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and the memory 401 performs the storage function of the memory 301 on the Internet. It may be a web storage that performs.

The processor 402 of the situation judgment model server 400 reads the situation judgment model 400a stored in the memory 402, and according to the constructed neural network system, the situation judgment model 400a to be described below is learned and played on the Go board. will make a judgment call. And the processor 402 of the situation judgment model server 400 may determine the ethos of the Go game and a specific starting point based on the performed situation determination.

In detail, in an embodiment, the processor 402 of the situation judgment model server 400 may set a target custom to be learned among various customs. In addition, the processor 402 may set a predetermined threshold for determining the shape of the checkerboard state (S) and / or the ethos for a specific starting point through the determination of the situation. In addition, the processor 402 of the setting model server 400 determines the situation for a specific checkerboard state (S) in which the specific starting point derived from the starting model server 300 is reflected, and the specific starting point and/or the corresponding specific starting point It is possible to determine the ethos for a specific checkerboard state (S). In addition, the processor 402 may transmit the information on the determination of the ethos to the start model server 300 so that self-play learning of the set-off model for applying the ethos when playing the Go game is effectively performed. In addition, the processor 402 of the situation judgment model server 400 may check the performance of whether the determination of the wind is properly made, and adjust a predetermined threshold value for determining the wind on the basis of this.

Meanwhile, according to the embodiment, the processor 402 includes a main processor that controls all units, and a plurality of graphics processing units (GPUs) that process large-capacity calculations necessary for driving a neural network according to the situation determination model 400a. It may be configured to include

In addition, the situation judgment model server 400 may communicate with the Go server 200 via the network 500 through the communication unit 403 .

- Launch model

2 is a view for explaining the initiation model structure of the initiation model server 300 for the initiation of an artificial intelligence computer in the deep learning-based Go game service according to an embodiment of the present invention, and FIG. 3 is a policy of the initiation model. It is a diagram for explaining the distribution of the movement probability for the starting point according to the figure, Figure 4 is a diagram for explaining the value value and the number of visits for the starting point of the starting model, Figure 5 is a starting model set out according to the pipeline of the search unit It is a drawing for explaining the process.

Referring to Figure 2, the start model according to the embodiment of the present invention as a deep learning model of the set off model server 300, the search unit 310, the self-play unit 320, the start neural network 330 and the gi-pung learning auxiliary unit ( 340) may be included.

)을 출력 할 수 있다. 셀프 플레이부(320)는 탐색 확률값(

)에 따라 스스로 바둑 대국을 할 수 있다. 셀프 플레이부(320)는 게임의 승패가 결정되는 시점까지 스스로 바둑 대국을 진행하고, 자가 대국이 종료되면 바둑판 상태(S), 탐색 확률값(

), 자가 플레이 가치값(z)을 착수 신경망(330)에 제공할 수 있다. 바둑판 상태(S)는 착수점들에 바둑돌이 놓여진 상태이다. 자가 플레이 가치값(z)은 바둑판 상태(S)에서 자가 대국을 하였을 때 승률 값이다. 착수 신경망(330)은 이동 확률값(p)과 가치값(v)을 출력할 수 있다. 이동 확률값(p)은 바둑판 상태(S)에 따라 착수점들에 대해 어느 착수점에 착수하는 것이 게임을 이길 수 있는 좋은 수인지 수치로 나타낸 확률분포값이다. 가치값(v)은 해당 착수점에 착수시 승률을 나타낸다. 예를 들어, 이동 확률값(p)이 높은 착수점이 좋은 수일 수 있다. 착수 신경망(330)은 이동 확률값(p)이 탐색 확률값(

)과 동일해지도록 트레이닝되고, 가치값(v)이 자가 플레이 가치값(z)과 동일해지도록 트레이닝될 수 있다. 이후 트레이닝된 착수 신경망(330)은 탐색부(310)를 가이드하고, 탐색부(310)는 이전 탐색 확률값(

)보다 더 좋은 수를 찾도록 MCTS를 진행하여 새로운 탐색 확률값(

)을 출력하게 한다. 셀프 플레이부(320)는 새로운 탐색 확률값(

)에 기초하여 바둑판 상태(S)에 따른 새로운 자가 플레이 가치값(z)을 출력하고 바둑판 상태(S), 새로운 탐색 확률값(

), 새로운 자가 플레이 가치값(z)을 착수 신경망(330)에 제공할 수 있다. 착수 신경망(330)은 이동 확률값(p)과 가치값(v)이 새로운 탐색 확률값(

)과 새로운 자가 플레이 가치값(z)으로 출력되도록 다시 트레이닝될 수 있다. 즉, 착수 모델은 이러한 과정을 반복하여 착수 신경망(330)이 대국에서 이기기 위한 더 좋은 착수점을 찾도록 트레이닝 될 수 있다. 일 예로, 착수 모델은 착수 손실(l)을 이용할 수 있다. 착수 손실(l)은 수학식 1과 같다.The set-off model can be learned as a model set out to win the game by using the search unit 310, the self-play unit 320, the start neural network 330 and the ethos learning assistant 340, and when proceeding with the Go game You can use the ethos that represents a particular temperament or way of doing things. More specifically, the search unit 310 may perform a Monte Carlo Tree Search (MCTS) operation according to the guide of the onset neural network 330 . MCTS is an experiential search algorithm for decision making. That is, the search unit 310 may perform MCTS based on the movement probability value (p) and/or the value value (v) provided by the onset neural network 330 . As an example, the search unit 310 guided by the onset neural network 330 performs MCTS to find a search probability value (

) can be printed. The self-play unit 320 is a search probability value (

), you can play Go by yourself. The self-player 320 performs a Go game by itself until the winning or losing of the game is determined, and when the self-playing game ends, the Go board state (S), the search probability value (

), the self-play value value z may be provided to the onset neural network 330 . The checkerboard state (S) is a state in which Go stones are placed at the starting points. The self-play value value (z) is a win rate value when a self-playing game is played in the checkerboard state (S). The onset neural network 330 may output a movement probability value (p) and a value value (v). The moving probability value (p) is a probability distribution value expressed numerically whether it is a good number to win the game by starting which starting point for the starting points according to the checkerboard state (S). The value (v) represents the win rate when starting the corresponding starting point. For example, the starting point with a high movement probability value p may be a good number. The onset neural network 330 is a movement probability value (p) is a search probability value (

) and can be trained so that the value v is equal to the self-play value z. Afterwards, the trained onset neural network 330 guides the search unit 310, and the search unit 310 sets the previous search probability value (

) to find a better number than the new search probability value (

) to output. The self-play unit 320 sets a new search probability value (

), output a new self-play value value (z) according to the checkerboard state (S) based on the checkerboard state (S), and a new search probability value (

), a new self-play value value z may be provided to the initiating neural network 330 . The starting neural network 330 is a movement probability value (p) and a value value (v) a new search probability value (

) and a new self-play value (z) can be retrained to be output. That is, the start model can be trained to repeat this process to find a better start point for the start neural network 330 to win the game. As an example, the settling model may use the settling loss (l). The onset loss (l) is the same as Equation 1.

(Equation 1)

는 신경망의 파라미터이고, c는 매우 작은 상수이다.

is a parameter of the neural network, and c is a very small constant.

와 p가 같아 지도록 하는 것은 크로스 엔트로피 손실(cross entropy loss) 텀에 해당되고,

에 c를 곱하는 것은 정규화 텀으로 오버핏을 방지하기 위한 것이다.Making z and v equal in the set-off loss (l) of Equation 1 corresponds to a mean square loss term,

To make p and p equal is a cross entropy loss term,

Multiplying by c is to prevent overfitting with the regularization term.

For example, referring to FIG. 3 , the trained set off model may represent a movement probability value p at the set points as a probability distribution value as shown in FIG. 3 . Referring to FIG. 4 , the value (v) of the trained set-off model may be represented by the value indicated above at one decapitation point of FIG. 4 . The onset neural network 330 may be configured in a neural network structure. For example, the onset neural network 330 may include one convolution block and 19 residual blocks. The convolution block may have a form in which several 3X3 convolution layers are superimposed. One residual block may have a form in which several 3X3 convolution layers are overlapped and a skip connection is included. The skip connection is a structure in which an input of a predetermined layer is output by summing it with an output value of a corresponding layer and input to another layer. In addition, the input of the onset neural network 330 includes the location information of the stone for the last 8 numbers of the black player, the location information of the stones for the last 8 numbers of the white player, and turn information on whether the current player is black or white 19* An RGB image of 19*17 can be input.

)을 출력할 수 있다. 착수 모델은 착수점들 중 가장 높은 탐색 확률값(

)을 선택할 수 있고, 선택된 탐색 확률값(

)을 가지는 착수점을 해당 바둑판 상태(S)에서의 최고의 수로 판단할 수 있다. 이때, 착수 모델은 탐색 확률값(

)을 기반으로 최고의 수라고 판단된 착수점을 착수 후보수로 설정할 수 있다. 즉, 본 발명의 실시예에서 착수 후보수란 특정 바둑판 상태(S)에 대하여 착수 모델에 의해 예측된 최고의 착수점일 수 있다. 이때, 착수 모델은 탐색 확률값(

)을 기반으로 최고의 수라고 판단된 착수 후보수를 적어도 둘 이상 도출할 수 있고, 이에 기초하여 적어도 둘 이상의 착수 후보수 정보를 생성할 수 있다. 자세히, 착수 모델은 결정된 착수 후보수에 기반하여 해당 착수 후보수와 관련된 정보(예컨대, 바둑판 상에서의 위치 정보 등)를 포함하는 착수 후보수 정보를 적어도 둘 이상 생성할 수 있다. 여기서 착수 모델이 적어도 둘 이상의 착수 후보수 정보를 생성하는 것은, 복수의 착수 후보수를 기반으로 기풍을 판단한 결과를 통하여 특정 기풍에 최적화된 착수 후보수를 합리적으로 도출하기 위함이다. Referring to FIG. 5 , the learned start-up model can start using the start-up neural network 330 and the search unit 310 in its own turn. The starting model is the activity function (Q) and confidence in the second checkerboard state (S)(S1-2), which is a branch that is not currently searched through MCTS in the first checkerboard state (S)(S1) through the selection process (a) Select the starting point with a high value (U). The activity function (Q) is the average value of the value values (v) calculated every time the branch passes. The confidence value (U) is proportional to the number of visits (N) passing through the branch. The settling model can be extended to the third checkerboard state (S) (S1-2-1) at the selected set point through the expansion and evaluation process (b), and the movement probability value (p) can be calculated. The start model calculates the value of the third checkerboard state (S) (S1-2-1) extended through the backup process (c), and the activity function (Q) of the branches that have passed, the number of visits (N), the movement probability value ( p) can be saved. The initiation model repeats the process of selection (a), expansion and evaluation (b), and backup (c), and creates a probability distribution using the number of visits (N) for each initiation point to create a search probability value (

) can be printed. The launch model has the highest search probability value (

) can be selected, and the selected search probability value (

) can be determined as the highest number in the checkerboard state (S). At this time, the starting model is a search probability value (

), the starting point determined to be the best number can be set as the number of starting candidates. That is, in an embodiment of the present invention, the number of candidates for starting may be the highest starting point predicted by the starting model for a specific checkerboard state (S). At this time, the starting model is a search probability value (

) may be derived at least two or more of the number of candidates to start, determined to be the best number, based on this, it is possible to generate information on the number of candidates for starting at least two or more. In detail, the initiation model may generate at least two or more of the number of start-up candidates information including information (eg, location information on the checkerboard, etc.) related to the corresponding number of start-up candidates based on the determined number of start-up candidates. Here, the reason that the start model generates at least two or more start-up candidates information is to rationally derive the number of start-up candidates optimized for a specific style through the result of determining the ethos based on the plurality of start-up candidates.

At this time, the starting model server 300 may transmit the generated starting candidate number information to the situation judgment model server 400 , and may receive the ethos learning data from the situation judgment model server 400 . Here, the ethos learning data is at least one start candidate that the ethos determination unit 400b of the situation judgment model server 400 judges to be suitable for learning a specific ethos based on the situation determination in order to learn the setting model in a certain specific ethos. It may be information about a number. A detailed description thereof will be provided later.

In addition, the start model may perform self-play learning based on the ethos learning data obtained from the situation judgment model server 400 . In detail, the start model can perform Go game play between the higher version start model learned based on the ethos learning data through the self-play unit, and the lower version start model before learning. That is, the start model can perform self-learning based on the ethos learning data, and through this, the performance of the ethos learning data can be diagnosed. Here, diagnosing the performance of the ethos learning data may be determining whether the ethos learning data is suitable data for learning a specific ethos.

Continuing, the start model may generate ethos learning performance information through performance diagnosis of ethos learning data based on self-play learning. That is, the gi wind learning performance information is through self-learning of the initiation model based on the acquired gi wind learning data, whether the corresponding gi wind learning data is data suitable for learning a specific ethos, that is, a specific number of candidates for starting and / or checkerboard state (S) It may be information diagnosing whether the ethos for And at this time, the start model server 300 may transmit the generated ethos learning performance information to the situation judgment model server 400 . That is, the start model can improve the accuracy of determining the ethos for a specific number of candidates and/or checkerboard state (S) by confirming the suitability of the learning data for learning a specific ethos and providing feedback thereto.

On the other hand, the initiation model may include the ethos learning auxiliary unit 340 in order to provide a Go game service that utilizes the ethos representing a specific temperament or method when playing the monarch game according to an embodiment of the present invention. In detail, the ethos learning auxiliary unit 340 may receive ethos learning data from the situation determination model server 400 as input data. In addition, the ethos learning auxiliary unit 340 may control the self-play unit 320 so that the start model server 300 performs self-play learning based on the ethos learning data received as input data. In addition, the ethos learning auxiliary unit 340 may generate ethos learning performance information based on the result of self-play learning through the self-play unit of the initiation model. That is, the spirit learning auxiliary unit 340 through the self-learning result of the start model based on the spirit learning data, whether the corresponding spirit learning data is data suitable for learning a predetermined target spirit, that is, a specific number of candidates for starting and / or a checkerboard state. It is possible to generate ethos learning performance information diagnosing whether the ethos for (S) is properly determined by the situation judgment model server 400 . For example, the ethos learning auxiliary unit 340 may include a difference value between a score between a player and an oponent for a specific number of start candidates included in the ethos learning data, and between a player and an oponant for a specific number of start candidates derived through self-play. By comparing the score difference value, it is possible to generate ethos learning performance information. And the ethos learning auxiliary unit 340 transmits the generated ethos learning performance information to the situation judgment model server 400, and the threshold adjustment operation for determining the ethos of the situation judgment model server 400 based thereon can be performed. .

A more detailed description will be given later in the detailed description of a method for determining and utilizing the ethos based on deep learning to be described below. In addition, in this embodiment, the ethos learning auxiliary unit 340 is described as being included in the start model server 300 to operate, but in another embodiment, the ethos learning auxiliary unit 340 is the Go server 200 or the situation judgment model server ( 400) or implemented as a separate device, various embodiments are also possible.

- Situation judgment model

6 is an exemplary diagram showing a screen providing a situation determination function of a deep learning-based Go game service according to an embodiment of the present invention, and FIG. 7 is a situation determination model of the situation judgment model server 400 of the present invention ( 400a) is a diagram for explaining the structure, and FIG. 8 is a diagram for explaining one block of the neural network structure composed of a plurality of blocks of the situation determination model 400a of the present invention.

Referring to FIG. 6 , the deep learning-based Go game service according to an embodiment of the present invention may determine the situation of the current Go board state (S). For example, as shown in FIG. 6 , when a user requests a situation determination by clicking the situation determination menu (A) during a Go game on the screen of the terminal 100, the deep learning-based Go game service provides the situation determination result in a pop-up window. can The situation judgment is to determine who wins by how many points by calculating the opponent and my house during the Go game. For example, if the user decides that the situation is favorable to me, do not overdo it any more and devise a strategy to end the game while maintaining the current advantageous situation. If it is judged unfavorable, the game phase will be changed Several strategies can be devised to do this. The standard for judging the situation is a house, a stone, a stone, a ball, and a big according to the state of the Go stones placed on the board. A stone is a stone placed on a checkerboard and is not a score in Korean rules. A house is an area made up of empty dots surrounded by one-colored Go stones, which is a score in Korean rules. Gongbae and Big are areas that are neither black nor white when Go is over, and are not points in Korean rules. Among the stones placed on the checkerboard, it is a stone that has no choice but to be caught and died. It is a score because it is used to fill the opponent's house under Korean rules. Big refers to an area that is neither black nor white when Go is over. Therefore, in order to judge the situation, it is necessary to accurately distinguish or predict a house, a stone, a stone, a ball, and a big in the state (S) on which the go stones are placed, in order to be an accurate judgment. At this time, to accurately distinguish a house, a stone, a stone, a public stone, and a big is to distinguish the state in which a house, a stone, a stone, a public stone, and a big are completely completed, and to accurately predict a house, a stone, a stone, a public meeting, and a big is a house , stone, stone, public meeting, or predicting a state with a high probability of becoming a big.

Referring to FIG. 7 , a situation judgment model 400a according to an embodiment of the present invention is a deep learning model of the situation judgment model server 400 , a situation judgment neural network 410 , an input feature extractor 420 , and a correct answer label generation It may include a part 430 . In addition, the situation determination model 400a according to an embodiment of the present invention is linked with the spirit determination unit 400b to operate the process of determining and learning the Go situation based on the deep learning neural network and applying the spirit when playing the Go game. can be A detailed description thereof will be provided later.

The situation determination model 400a may perform supervised learning so as to determine the situation of the current checkerboard state S using the situation determination neural network 410 . More specifically, the situation determination model 400a generates a training data set related to the checkerboard state (S), and using the generated training data set, the situation determination neural network 410 determines the situation according to the current checkerboard state (S). can be taught to do so. The situation judgment model server 400 may receive a plurality of notations from the Go server 200 . Each notation of a plurality of notations may include a respective checkerboard state (S) according to the starting order.

The input feature extraction unit 420 may extract the input feature (IF) from the checkerboard state (S) of a plurality of notations and provide it to the situation determination neural network 410 as input data for training. The input feature (IF) of the checkerboard state (S) includes the position information of the stone for the last 8 moves of the black player, the position information of the stone for the last 8 moves of the white player, and turn information on whether the current player is black or white. It can be an RGB image of 19*19*18. For example, the input feature extractor 420 may have a neural network structure and may include a kind of encoder.

)로 제공할 수 있다. 정답 레이블 생성부(430)의 정답 레이블 생성은 후술하는 도 9 내지 도 11의 설명을 따른다. 일 예로, 정답 레이블 생성부(430)는 신경망 구조의 롤아웃 또는 인코더를 포함할 수 있다.The correct answer label generator 430 generates a correct answer label (ground truth) through a preprocessing process in the current checkerboard state (S), and applies the correct answer label to the neural network 410 for training target data (

) can be provided. The correct answer label generation unit 430 generates the correct answer label according to the description of FIGS. 9 to 11 , which will be described later. As an example, the correct answer label generator 430 may include a rollout of a neural network structure or an encoder.

)로 한 트레이닝 데이터 셋을 이용하여 형세 판단 신경망(410)에서 생성된 출력 데이터(o)가 타겟 데이터(

)와 동일해지도록 형세 판단 신경망(420)을 충분히 학습할 수 있다. 일 예로, 형세 판단 모델(400a)은 형세 판단 손실(

)을 이용할 수 있다. 형세 판단 손실(

)은 평균 제곱 에러(mean square error)를 이용할 수 있다. 예를 들어, 형세 판단 손실(

)은 수학식 2와 같다.The situation judgment model 400a uses the input feature (IF) as input data and the correct answer label as the target data (

), the output data (o) generated by the situation determination neural network 410 using the training data set as the target data (

), it is possible to sufficiently learn the situation determination neural network 420 to be the same. As an example, the situation judgment model 400a is a situation judgment loss (

) can be used. loss of judgment (

) can use the mean square error. For example, loss of judgment (

) is the same as Equation 2.

(Equation 2)

는 현재 바둑판 상태(S)에서 정답 레이블에 따른 소정의 교차점(i)에 대한 형세값이다. 형세값에 대한 설명은 후술하는 도 11의 설명에 따른다.

는 현재 바둑판 상태(S)에서 소정의 교차점(i)을 형세 판단 신경망(410)에 입력하였을 때에 출력되는 출력 데이터이다. 형세 판단 모델(400a)은 형세 판단 손실(

)이 최소화되도록 경사 하강법(gradient-descent)과 역전파(backpropagation)을 이용하여 형세 판단 신경망(410) 내의 가중치와 바이어스 값들을 조절하여 형세 판단 신경망(410)를 학습시킬 수 있다.B is the total number of intersections of the checkerboard. In the checkerboard, 19 horizontal and 19 vertical lines intersect each other, and 361 intersections are arranged. It is not limited thereto, and when the checkerboard has 9 horizontal and 9 vertical lines, 81 intersections may be arranged.

is a configuration value for a predetermined intersection point (i) according to the correct answer label in the current checkerboard state (S). The description of the configuration value follows the description of FIG. 11, which will be described later.

is output data output when a predetermined intersection (i) is input to the situation determination neural network 410 in the current checkerboard state (S). The situation judgment model 400a is the situation judgment loss (

) can be minimized by adjusting the weights and bias values in the situation determination neural network 410 using gradient-descent and backpropagation to train the situation determination neural network 410 .

The situation determination neural network 410 may be configured in a neural network structure. As an example, the situation determination neural network 420 may be composed of 19 residual blocks. Referring to FIG. 8 , one residual block includes 256 3X3 convolutional layers, batch normalization layers, Relu activation function layers, 256 3X3 convolutional layers, batch normalization layers, skip connections, Relu activation function layers may be arranged in order. The batch normalization layer is to prevent covariate shift, which is a phenomenon in which the distribution of the input of the current layer is changed due to the parameter change of the previous layer during learning. Skip connection prevents the performance of the neural network from decreasing even if the block layer becomes thicker, and makes the block layer thicker to increase the overall neural network performance. The skip connection may be in a form in which the first input data of the residual block is combined with the output of the second batch normalization layer and input to the second Relu activation function layer.

9 and 10 are diagrams for explaining the first and second pre-processing steps for generating a correct answer label used to learn the situation judgment model 400a of the present invention, and FIG. 11 is the situation judgment model of the present invention It is a diagram for explaining the third pre-processing step for generating the correct answer label used to learn (400a).

The correct answer label generating unit 430 may generate a correct answer label used for learning so that the situation determination neural network 410 can accurately determine the situation.

More specifically, the correct answer label generating unit 430 receives as an input the checkerboard state (S) that is the basis of the input data, and performs a first preprocessing of ending the current checkerboard state (S) to the first preprocessing state (P1) ) can be created. The first pre-processing, ending, is a process of finishing the game by making a predetermined start so that the boundary of the house becomes clear before calculating the house. For example, referring to FIG. 9 , the correct answer label generating unit 430 generates the first pre-processing state P1 of FIG. 9(b) by ending the current checkerboard state (S) of FIG. can

The correct answer label generating unit 430 may generate the second pre-processing state P2 by performing a second pre-processing of removing stones that are disposed within the house boundary in the first pre-processing state P1 and are unnecessary for house classification. For example, a stone that is placed within the boundary of a house and is not necessary to classify the house may be a quarry stone. It was explained earlier that the stone was a stone that died because the other stone was placed in the house, and no matter how it was placed, it could only be caught. In addition, a stone that is placed within the boundary of the house and is unnecessary to classify the house may be one's own stone placed in the house. For example, referring to FIG. 9 , the correct answer label generating unit 430 removes stones unnecessary for house classification in the first pre-processing state P1 of FIG. (P2) can be created.

As another example, referring to FIG. 10 , the correct answer label generating unit 430 starts the red x as shown in FIG. can do. The correct answer label generating unit 430 removes the green x to remove the rubble indicated by the blue x in FIG. Pre-processing can be performed.

The correct answer label generator 430 may perform a third preprocessing of changing each intersection to a shape value (g, where g is an integer) indicated from -1 to +1 in the second preprocessing state P2 . That is, in the third pre-processing, the correct answer label generating unit 430 changes the second pre-processing state P2, which is an image feature, to a third pre-processing state P3, which is a numerical feature. For example, in the second pre-processing state P2 , if my stone is disposed at the intersection, it may correspond to 0, if it is my house area, +1, if the opponent stone is disposed, it may correspond to 0, and if it is the opponent's house area, it may correspond to -1. In this case, the situation determination neural network 410 may be trained to distinguish a house, a stone, and a stone when determining the situation. As another example, in the second pre-processing state P2, if my stone is placed at the intersection, it may correspond to 0, if it is my home area, +1, if the opponent stone is placed, 0, if it is the opponent's area, -1, and if it is a big or common match, 0 may correspond. In another example, the situation determination neural network 410 may be trained to distinguish a big or a common match when determining a situation. For example, referring to FIG. 11 , the correct answer label generating unit 430 characterizes the second pre-processing state P2 of FIG. 11 (a) as the third pre-processing state P3 of FIG. 11 (b). can be changed

)로 이용될 수 있다. The third preprocessing state (P3) becomes the correct label of the situation determination in the checkerboard state (S), and the target data (

) can be used as

12 is a view for explaining a situation determination result of the condition determination model 400a of the present invention.

The learned layout determination model 400a may provide a configuration value for all intersections of the checkerboard when the checkerboard state S is input. That is, it is possible to provide an integer value of -1 to +1, which is a configuration value, for 361 points of the checkerboard intersection.

Referring to FIG. 12 , the situation determination model server 400 may determine the situation using the situation value provided by the situation determination model 400a, a predetermined threshold value, and the presence or absence of stones. As an example, the situation judgment model server 400 is a place where there are no stones, and if the situation value exceeds the first threshold value, it is determined as a place with a high probability of becoming my home, and if the value is close to +1, it can be determined as my home area. have. The situation judgment model server 400 may display a square shape of the same color as my stone, which increases as the probability that it is my house increases. The situation judgment model server 400 is a place where there are no stones, and if the situation value is less than the second threshold value, it is determined as a place with a high probability of becoming a counterpart's house, and if the value is close to -1, it can be determined as the home area. The situation judgment model server 400 may display a square shape of the same color as the counterpart stone, which increases as the probability of the counterpart's house increases. The situation judgment model server 400 is a place where there are no stones, and if the situation value is within the third threshold range or a value close to 0, it can be determined as a common or big. The situation judgment model server 400 may display an X when judging that it is common or big. The shape judgment model server 400 is where the stone is, and if the shape value is within the range of the third threshold value or close to 0, it may determine the stone as my stone or the opponent's stone. The situation judgment model server 400 may not display anything when it is judged to be a common ball or a big. The situation judgment model server 400 is a place where there is a stone, and if the situation value exceeds the first threshold value, it is determined as a place with a high probability of becoming a stone of the opponent stone, and if the value is close to +1, it can be determined as a stone of the opponent stone. have. The situation judgment model server 400 may display a square shape of the same color as the inner stone, which increases as the probability that the other stone is a non-stone. The situation judgment model server 400 is a place where there is a stone, and if the position value is less than the second threshold, it is determined as a place with a high probability of becoming my stone, and if the value is close to -1, it can be determined as a stone of the other stone. have. The situation judgment model server 400 may display a square shape of the same color as that of the opponent stone, which increases as the probability that the opponent stone is a quarry stone increases.

In addition, the situation determination model server 400 may display the result of the calculation in the current checkerboard state (S) by using the situation determination criterion determined at each intersection.

Therefore, the deep learning-based Go game service device according to the embodiment may determine the Go situation using a deep learning neural network. In addition, the deep learning-based Go game service apparatus according to the embodiment can accurately determine the situation of Go by accurately classifying a house, a stone, a stone, a ball, and a big according to the Go rules. In addition, the deep learning-based Go game service device according to the embodiment may accurately determine the situation of Go by predicting a house, a stone, a stone, a ball, and a big according to the Go rules. In addition, the deep learning-based Go game service device according to the embodiment can quickly determine the situation in the Go game.

On the other hand, the situation judgment model server 400 is based on deep learning, in order to perform a series of operations for applying a specific ethos during the Go game, the first from the situation judgment model 400a through the situation judgment model 400a To 3 pre-processing, it is possible to generate the holding house information based on the value derived through the pre-processing. Here, the holding house information may be information generated based on the number of holdings of each of the player and the Oponent predicted through the situation determination. That is, the situation judgment model server 400 uses the configuration judgment model 400a, and the configuration values for all intersections of the checkerboard state S predicted from the situation judgment model 400a (in the embodiment, -1 to +1). value between), the number of houses expected to be owned by each player and component can be calculated to generate house information. For example, the holding house information may be implemented in a form including information that predicts that the player will own the 80th house and the Opponent will own the 77th house. Such holding house information may be used as base data for generating a score difference value, which is a parameter to be compared with a predetermined air wind determination threshold in the wind determination process to be described later.

In an embodiment, the situation judgment model server 400 derives a situation value by performing a situation determination through the situation determination model 400a, and the number of households owned by the player and the number of households owned by the Oponent based on the derived situation value can be calculated to generate holding house information. In addition, the situation judgment model server 400 may acquire a player score value and an oponent score value based on the holding collection information generated through the ethos judgment unit 400b. Also, the situation judgment model server 400 may calculate a score difference value between the two score values based on the obtained player score value and the component score value through the ethos judgment unit 400b. Thereafter, the situation judgment model server 400 may generate ethos determination information by comparing the score difference value calculated through the ethos determination unit 400b with a predetermined geography judgment threshold. A detailed description thereof will be provided below.

13 is a view comparing the situation determination result of the situation determination model 400a of the present invention with the situation determination result by the deep learning model according to the prior art, and FIG. 14 is the situation determination model 400a of the present invention The result and the situation judgment result by the deep learning model according to the prior art are compared, and FIG. 15 is the situation judgment result of the situation judgment model 400a of the present invention and the situation judgment result by the deep learning model according to the prior art compared view.

Referring to FIG. 13 , the condition determination model 400a of the present invention determines the condition by classifying houses, stones, and rocks at each intersection as in area B of FIG. 13 (a). However, the situation judgment model 400a by the deep learning model according to the prior art cannot distinguish a house, a stone, and a stone at the intersection of the area corresponding to FIG. 13(b) to FIG. 13(a).

Similarly, referring to FIG. 14 , the condition determination model 400a of the present invention determines the condition by classifying houses, stones, and rocks at each intersection as in area C of FIG. 14(a). However, the situation judgment model 400a by the deep learning model according to the prior art does not distinguish between houses, stones, and rocks at the intersection of the regions corresponding to those of FIG. 14(b) to FIG. 13(a).

Referring to FIG. 15 , the situation determination model 400a of the present invention properly recognizes the bag as in the region D of FIG. 15 (a). However, the situation judgment model 400a by the deep learning model according to the prior art does not distinguish the bag in the area corresponding to that of FIG. 15(b) in FIG. 15(a).

16 is an exemplary diagram of a signal flow in a deep learning-based Go game service system according to an embodiment of the present invention.

Referring to FIG. 16, the start model server 300 is an artificial intelligence computer that learns by itself according to the rules of Go so as to be able to perform the initiation of Go so that it can win the game in its own turn to train the initiation model, which is a deep learning model. can (s11). Go server 22 may transmit a plurality of notations to the situation judgment model server 400 . The situation judgment model server 400 may generate a training data set. First, the situation judgment model server 400 may extract input features from the checkerboard state (S) of a plurality of notations (S13). The layout judgment model server 400 may generate a correct answer label using the checkerboard state (S) from which the input features are extracted (S14). The situation judgment model server 400 may train the situation judgment model 400a using the training data set using the input characteristics as input data and the correct answer label as the target data (S15). The terminal 100 may request the Go game from the Go server 200 against the artificial intelligence computer or against another user terminal (S16). Go server 200 may request the start model server 300 when the terminal 100 requests a Go game against the artificial intelligence computer (S17). The Go server 200 may perform a Go game, and the terminal 100 and the start model server 300 may start in their turn (S18 to S20). During the game, the terminal 100 may request the Go server 200 to determine the situation (S21). The Go server 200 may request the condition determination model server 400 to determine the condition of the current Go board state (S) (S22). The situation judgment model server 400 extracts the input features of the current checkerboard state (S), and the situation judgment model 400a, which is a deep learning model, generates a situation value using the input features, and the checkerboard state (S) and the situation A situation may be determined using the value (S23). The situation determination model server 400 may provide the situation determination result to the Go server 200 (S24). The Go server 200 may provide the result of determining the situation to the terminal 100 (S25).

17 is a method for determining a situation among the deep learning-based Go game service methods according to an embodiment of the present invention, and FIG. 18 is a method for pre-processing training data for generating a correct answer label among the method for determining the situation of FIG. 17 .

Referring to FIG. 17 , the deep learning-based Go game service method according to an embodiment of the present invention may include the step (S100) of the situation judgment model server 400 receiving a plurality of notations from the Go server.

In the deep learning-based Go game service method according to an embodiment of the present invention, the input feature extraction unit among the situation determination model 400a of the situation judgment model server 400 extracts the input features from the checkerboard state (S) of a plurality of notations It may include a step (S200). A method of extracting input features follows the description of FIG. 7 .

The deep learning-based Go game service method according to an embodiment of the present invention includes a step (S300) of generating a correct answer label based on the checkerboard state (S) in which the correct answer label generator extracts the input features of the situation determination model (400a) can do. As an example, referring to FIG. 18 , the correct answer label generating step S300 may include the first pre-processing step S301 in which the correct answer label generating unit ends the current checkerboard state (S). The first pre-processing step ( S301 ) follows the description of FIGS. 9 to 10 . The correct answer label generation step (S300) may include a second pre-processing step (S302) in which the correct answer label generator removes unnecessary stones from the first pre-processed checkerboard state (S). The second pre-processing step ( S302 ) follows the description of FIGS. 9 to 10 . The correct answer label generation step ( S300 ) may include a third pre-processing step ( S303 ) in which the correct answer label generator changes each intersection of the second pre-processed checkerboard state (S) into a shape value. The third preprocessing step ( S303 ) follows the description of FIG. 11 . The correct answer label generation step (S300) may include a step (S303) of providing the third preprocessing state as the correct answer label as target data to the situation determination neural network. The step of providing the target data ( S301 ) follows the description of FIGS. 7 and 11 .

The deep learning-based Go game service method according to an embodiment of the present invention may include training the situation determination neural network of the situation determination model 400a using a training data set (S400). A method of training (learning) a situation determination neural network follows the description of FIG. 7 .

The deep learning-based Go game service method according to an embodiment of the present invention includes a step (S500) of completing the training of the situation determination neural network to build the situation determination model 400a (S500). As an example, the training of the situation determination neural network may be completed when the situation determination loss of FIG. 7 is less than or equal to a predetermined value.

The deep learning-based Go game service method according to an embodiment of the present invention may include a step (S600) of inputting the current Go board state (S) to the situation determination model (400a) in response to a request for determination of the condition of the terminal.

The deep learning-based Go game service method according to an embodiment of the present invention may include a step (S700) of determining the situation of the current Go board state (S) to which the situation determination model 400a is input. The step (S700) of determining the situation may follow the description in which the situation determination model 400a described with reference to FIG. 12 generates the setting value of the current checkerboard state (S).

The deep learning-based Go game service method according to an embodiment of the present invention may include the step (S800) of the situation judgment model server 400 outputting the situation determination result. The step of outputting the situation determination result ( S800 ) may follow the description in which the situation determination model server 400 provides the situation determination result using the situation value, the state of the checkerboard, and a predetermined threshold value described in FIG. 12 .

Therefore, the deep learning-based Go game service method according to the embodiment may determine the Go situation using a deep learning neural network. In addition, the deep learning-based Go game service method according to the embodiment can accurately determine the situation of Go by accurately classifying a house, a private stone, a stone, a ball, and a big according to the Go rules. In addition, the deep learning-based Go game service method according to the embodiment can accurately determine the situation of Go by predicting a house, a stone, a stone, a ball, and a big according to the Go rules. In addition, the deep learning-based Go game service method according to the embodiment can quickly determine the situation in the Go game.

- Wind judgment unit

The ethos determining unit 400b according to the embodiment of the present invention is a component included in the situation judgment model server 400, and is a checkerboard obtained through the configuration judgment model 400a in conjunction with the situation judgment model 400a. It is possible to perform ethos determination based on a specific starting point based on the shape value of the state (S). In addition, the ethos determining unit 400b may allow the start model to perform ethos learning based on the determined ethos to utilize the ethos representing a specific temperament or method when playing the Go game. Here, the ethos determination is to determine which ethos implements, that is, which ethos category, the number of candidates to start, which is the best number predicted by the set-off model. In an embodiment, the airflow may be classified according to 1) the airflow type category or 2) 1 to n airflow step (n=1, 2, 3, ...) categories based on a predetermined criterion. For example, the ethos may be classified into an aggressive ethos, a stable ethos, and/or a defensive ethos, etc. according to the ethos type category. In another example, the breeze is a first wind step, a second wind step, . . . . In detail, the situation determination model 400a may perform a situation determination on the number of candidates for starting information obtained from the starting model server 300, and then the ethos determining unit 400b is based on the data derived through the situation determination. It is possible to perform ethos determination to determine which ethos category is included among the plurality of ethic categories in which the information on the number of candidates for start-up is classified as above.

In detail, the air wind determination unit 400b may set target air wind information, which is preset information on which air style to learn first in order to perform the air wind determination. Here, the target ethos information is predetermined information on which ethos among a plurality of ethos classified into various categories to be learned as a target, 1) any one of the ethos type categories (eg, aggressive ethos, stable ethos, defensive ethos, etc.) It may be information in which a target airflow is set by selecting , or 2) information in which a target airflow is set by selecting any one of the categories 1 to n winds.

In addition, the ethos determination unit 400b acquires from the start model server 300 based on the difference value between the player score value and the oponent score value according to the number of holdings derived through the situation determination of the situation determination model 400a It is possible to set a predetermined gi wind judgment threshold for performing gi wind judgment on the number of candidates to set off. That is, the ethos determining unit 400b may perform ethic determination by comparing a preset ethic determination threshold and a score difference value derived through the determination of the situation with respect to the information on the number of candidates to set out.

In more detail, first, the situation determination model 400a may acquire at least one or more pieces of information on the number of candidates for starting as input data from the starting model server 300 . In addition, the situation determination model 400a may generate holding house information by performing a situation determination on each of the obtained information on the number of candidates to start. Thereafter, the ethos determining unit 400b may calculate a player score value and an element score value based on the house information generated from the situation judgment model 400a. Also, the ethos determining unit 400b may calculate a score difference value indicating a difference between the player score value and the oponent score value based on the calculated player and component score values. In addition, the spirit determination unit 400b may compare the calculated score difference value with a preset determination threshold to generate the wind determination information for each of at least one or more pieces of information on the number of candidates to set out. Here, the ethos determination information may be information for determining the category of ethos implemented by the start of each number of start candidates during the Go game play. A detailed description thereof will be provided below.

In addition, the air wind determination unit 400b may generate the air wind learning data based on the generated air wind determination information. In detail, the wind determination unit 400b may determine the wind determination information corresponding to the preset target air wind information from among the spirit determination information derived for each of the received start-up candidates information. In addition, the ethos determination unit 400b may generate ethos learning data for implementing learning based on the information on the number of candidates for starting the ethos determined to correspond to the target ethos information. In addition, the ethos determining unit 400b may provide the generated ethos learning data to the start model server 300 .

In addition, the ethos determining unit 400b may acquire ethos learning performance information as input data from the start model server 300 that has performed learning based on the ethos learning data, and determines the ethos based on the obtained ethos learning performance information. You can adjust the threshold. A more detailed description will be given below.

On the other hand, FIG. 19 is a view for explaining the structure of the ethos determination unit 400b of the situation determination model server 400 of the present invention. Referring to FIG. 19 , in order to perform a series of operations for determining the ethos as described above, the situation determination model server 400 may include the above-described situation determination model 400a and the ethos determination unit 400b, and the situation The determination model 400a and the air wind determination unit 400b may be operated by interworking with each other. In an embodiment, the ethos determination unit 400b sets the ethos judgment threshold, derives the player score value and the component score value to calculate a score difference value, and generates ethos judgment information based on the calculated score difference value. can In addition, in the embodiment, the wind determination unit 400b may generate the air wind learning data based on the generated air wind determination information, and adjust the air wind determination threshold based on the air wind learning performance information obtained from the set off model server 300 . can

That is, the ethos determination unit 400b of the situation judgment model server 400 determines the ethos based on the judgment of the situation of the checkerboard state (S) in conjunction with the situation judgment model 400a according to an embodiment of the present invention, and It may be a component for implementing a process that is applied when playing a Go game by performing learning based on it. A detailed description of this will be described later in the detailed description of a method for determining and applying a ethos based on deep learning. In addition, in this embodiment, it is described that the ethos determining unit 400b is included in the situation judgment model server 400, but in another embodiment, the ethos determining unit 400b is the Go server 200 and / or the settling model server 300. Various embodiments such as included in the , or implemented as a separate device will be possible.

- How to determine and utilize the ethos based on deep learning

Hereinafter, with reference to the drawings, a method of providing a Go game service that determines a Go situation based on a deep learning neural network to determine an ethos, and based on this, a ethos indicating a specific temperament or method when playing a Go game, will be described in detail. do.

20 is a flowchart for explaining a method of determining a Go situation based on a deep learning neural network according to an embodiment of the present invention and using the ethos indicating a specific temperament or method during a Go game play, FIG. 21 is a diagram of the present invention It is a conceptual diagram for explaining a method of determining a Go situation according to an embodiment and utilizing the ethos when playing a Go game.

20 and 21 , the situation judgment model server 400 may include a step (S101) of first setting target ethos information. In detail, the situation judgment model server 400 may set target ethos information, which is preset information, with respect to which ethos to perform learning through the processor 402 . In this case, the target air wind information may be information that selects any one of a plurality of air wind categories that are mood-streamed based on the air wind type category or the 1 to n air wind stage categories. For example, the target ethos information may be any one of an aggressive ethos, a stable ethos, and/or a defensive ethos, a first ethos step, a second ethos, ... , it may be any one of the nth wind phase. At this time, depending on the embodiment, the wind type category may be implemented in various ways, such as including more subdivided categories or simplified to include fewer categories. The closer to the stage, the more defensive ethos is determined, and the closer to the nth ethos stage, the more it is preset to be judged as an aggressive ethos, or vice versa.

In addition, the situation judgment model server 400 may include a step (S103) of setting the wind judgment threshold. In detail, the situation determination model server 400 may set a predetermined air wind determination threshold for performing determination of the air style through the air wind determination unit 400b. For example, the situation judgment model server 400 may preset the ethos judgment threshold to a predetermined value.

Continuing, the situation judgment model server 400 may include a step (S105) of obtaining the number of candidates to start information. In detail, the situation judgment model server 400 may receive the information on the number of candidates for starting at least two or more derived from the starting model server 300 . In addition, the situation determination model server 400 may use the received starting candidate number information as input data of the situation determination model 400a and/or the ethos determination unit 400b.

In addition, the situation judgment model server 400 may include a step (S107) of deriving a score value by performing a situation judgment on each of the obtained information on the number of candidates to set out. In detail, the situation judgment model server 400, through the situation judgment model (400a), for all intersections of the checkerboard to which the number of starting candidates obtained from the starting model server 300 is applied. It is possible to derive the format value. In this case, the standard for judging the situation may be a house, private stone, stone court, or big. And the situation judgment model server 400 may generate holding house information based on the derived configuration value. In detail, the situation judgment model server 400 calculates the number of houses expected to be owned by the player and the oponent, respectively, through the situation judgment model 400a , and determines the number of players and the oponent house, which is information about holding house information. can create

In addition, the situation judgment model server 400 that has generated the possession house information may derive the player score value and the component score value based on the generated house information. In detail, the situation judgment model server 400 derives a player score value based on the number of player sets in the holding house information, and derives an oponent score value based on the number of oponent collections through the ethos determination unit 400b. can At this time, the situation judgment model server 400 may preset a process for calculating a score value. For example, the situation judgment model server 400 may determine the score calculation process as +1 per house. In addition, since the situation judgment model server 400 calculates a score value based on the information on the number of candidates to set out, the player score value may always be higher than the component score value. At this time, when the player score value is lower than the Opponent score value, the situation judgment model server 400 may perform the ethos determination after excluding the number of candidates for starting.

22 is a view for explaining a method of generating wind judgment information for each of the number of candidates to start information according to an embodiment of the present invention. In this case, for effective explanation, FIG. 22 may be displayed by limiting that the player uses black Go stones and the Oponent uses white Go stones. However, it will be apparent that the player may use the white Go stones and the Oponent may use the black Go stones, depending on the embodiment. For example, referring to FIG. 22 , the situation judgment model server 400 is based on the first number of candidates for starting information, the second number of candidates for starting information and the third number of candidates for starting information obtained from the starting model server 300, respectively. score can be calculated. In detail, the situation judgment model server 400 may first preset a process for calculating a score value and determine the score value calculation process as +1 per house. In addition, the situation judgment model server 400 may derive the player score value and the component score value based on the holding house information matched with the information on the number of candidates to start by applying the score calculation process. In detail, the situation judgment model server 400 is based on the holding house information matched to the first start candidate number information, when the first start candidate number is applied to the checkerboard, if the number of player houses is 79, the player score value is +79. and if the number of oponent houses is 77, the oponent score value can be derived as +77. Similarly, the situation judgment model server 400 may derive the player score value as +80 if the number of player houses is 80 when the second number of candidates for starting is applied to the checkerboard, and if the number of oponent houses is 77, the value of the oponent points is + 77 can be derived. In addition, the situation judgment model server 400 can derive the player score value as +91 if the number of player houses is 91 when the number of third start candidates is applied to the checkerboard, and if the number of oponent houses is 68, the value of the oponent points is + 68 can be derived.

Next, the situation judgment model server 400 may include a step ( S109 ) of calculating a score difference value based on the player score value and the component score value derived for each of the number of start candidates information. In detail, the situation determination model server 400 may calculate a score difference value for the information on the number of candidates for each set out obtained from the set off model server 300 through the ethos determining unit 400b. In an embodiment, the situation judgment model server 400 calculates a difference value between the player score value and the component score value for each starting candidate number information, and determines the result value as the score difference value for the corresponding starting candidate number. have. In this case, the situation judgment model server 400 may perform a ethos determination after excluding the number of candidates for starting when the score difference value is calculated as a negative number.

For example, referring to FIG. 22 , the situation judgment model server 400 corresponds to the first number of candidates for starting when the player score value matched to the information on the number of first start candidates is +79, and the oponent score value is +77. The score difference value can be calculated as +2. In addition, the situation judgment model server 400, when the player score value matched to the second number of candidates for starting information is +80, and the component score value is +77, the score difference value for the second number of candidates for starting is +3 can be calculated as Similarly, when the situation judgment model server 400 matches the third number of start candidates information and the player score value is +91, and the component score value is +68, the score difference value for the third start candidate number is + 23 can be calculated.

In addition, the situation judgment model server 400 that calculates the difference in score for each of the number of candidates for starting out includes a step (S111) of generating ethos judgment information for each of the number of candidates for starting on the basis of the calculated difference in score. can In detail, the situation judgment model server 400, through the ethos determination unit 400b, may generate ethos judgment information, which is information for determining the category of ethos implemented by the start of each starting candidate number during the Go game play. In more detail, the situation judgment model server 400 may generate geography judgment information based on the calculated score difference value and a predetermined geography judgment threshold for each number of candidates to set out. For example, the situation judgment model server 400 compares the score difference value with the ethos judgment threshold, and when the score difference exceeds the ethos judgment threshold (when the score difference is large), it can be determined to be included in the aggressive ethos category. have. In addition, the situation judgment model server 400 compares the score difference value with the ethos judgment threshold, and when the score difference value corresponds to the ethos judgment threshold (when the score difference is small), it can be determined to be included in the stable ethos category. Similarly, the situation judgment model server 400 compares the score difference value with the ethos judgment threshold, and when the score difference value is less than the ethos judgment threshold (when there is little difference in scores), it can be determined to be included in the defensive ethos category.

For example, referring to FIG. 22 , when the situation judgment model server 400 presets the ethos judgment threshold to '3', if the score difference value matched with the first number of candidates for starting information is +2, the first number of candidates for starting It can be judged as a defensive ethos category. In addition, if the score difference value matched to the second number of candidates for starting information is +3, it is possible to determine the number of candidates for starting out as a stable ethos category. Similarly, if the score difference value matched to the third number of candidates for starting information is +23, it is possible to determine the third number of candidates for starting out as an aggressive ethos category. That is, the situation judgment model server 400 may generate ethos judgment information based on the score difference value for each of the starting candidates information obtained from the starting model server 300 .

Next, the situation judgment model server 400 that generates the ethos judgment information for each starting candidate number information may include a step (S113) of generating ethos learning data based on the generated ethos judgment information and the target ethos information. have. In detail, the situation judgment model server 400 may determine, through the ethos determination unit 400b, the ethos determination information corresponding to the target ethos information among the ethos determination information derived for the information on the number of candidates for each departure. And the situation determination model server 400 may generate ethos learning data for determining the number of candidates for starting information as the learning data by selecting information on the number of start candidates determined that the ethos determination information corresponds to the target ethos information. At this time, the ethos learning data is data including information generated based on the selected number of candidates for starting (eg, information on the number of candidates for starting, configuration value, player score value, oponent score value, score difference value, ethos judgment information, etc.) can be

For example, referring to FIG. 22 , the situation judgment model server 400 indicates that the preset target ethos information is an 'aggressive ethos category', and the ethos judgment information matched to the first set off candidate number information is a 'defensive ethos category', When the ethos determination information matched to the second number of candidates for starting information is a 'stable ethos category', and the ethos determining information matched to the third number of candidates for starting information is an 'aggressive ethos category', the target ethos corresponding to the aggressive ethos category It is possible to generate ethos learning data for determining the third number of candidates to start information having ethos determination information as the learning data. At this time, the situation judgment model server 400 generates information generated based on the third number of candidates for starting in the generated ethos learning data (eg, information on the number of candidates for starting out for the third number of candidates for starting, the situation value, the player score value, It is possible to generate ethos learning data including an oponent score value, a score difference value, ethos judgment information, etc.).

In addition, the situation judgment model server 400 may include a step (S115) of transmitting the generated ethos learning data. In detail, the situation judgment model server 400 may transmit the ethos learning data derived by performing ethos determination based on the information on the number of at least two or more candidates to set out to the setting model server 300 .

Thereafter, the situation judgment model server 400 may include a step (S117) of obtaining the ethos learning performance information from the starting model server 300 that has received the ethos learning data. In detail, the situation judgment model server 400 performs self-play learning based on the ethos learning data from the start model server 300 to determine whether the ethos learning data is suitable data for learning the target ethos, ethos learning performance. information can be received.

And the situation judgment model server 400 may include a step (S119) of adjusting the ethos judgment threshold based on the acquired ethos learning performance information. In detail, the situation judgment model server 400 may adjust the ethos determination threshold based on the ethos learning performance information through the ethos determination unit 400b, thereby improving the accuracy of the ethos determination.

For example, first, the situation judgment model server 400 presets the target ethos information as 'stable ethos category' and presets the ethos judgment threshold as '3', and the score difference value for the number of candidates for a specific start is the corresponding ethos. When it corresponds to the judgment threshold of 3, it may be preset to determine that it is a stable ethos category. Thereafter, the situation judgment model server 400 may generate ethos judgment information based on the difference between the score difference between the corresponding ethos judgment threshold and the number of candidates to set out, and based on the generated ethos judgment information, it is possible to generate ethos learning data. , it is possible to transmit the generated ethos learning data to the start model server 300 .

Subsequently, the ethos learning auxiliary unit 340 of the starting model server 300 that has received the ethos learning data from the situation judgment model server 400 is, The ethos learning performance information may be generated by comparing the score difference value for the number of candidates for starting based on self-play learning to which the number of candidates for starting is applied. For example, the start model server 300 determines that the score difference value for the number of start candidates on the ethos learning data is +3, but the maximum score difference value for the number of start candidates derived based on the actual self-play learning is +2.5. In this case, it is possible to generate ethos learning performance information including error information for determining that there is a predetermined error in the predicted score difference value. In this case, the error information may include information on the presence or absence of an error and/or information on the degree of the error. And the start model server 300 may transmit the ethos learning performance information including the error information to the situation judgment model server 400 .

Subsequently, the situation judgment model server 400 that has received the above ethos learning performance information from the start model server 300 adjusts the preset ethos judgment threshold to determine the target ethos based on the received ethos learning performance information. can That is, in the example, the situation judgment model server 400 increases and adjusts the ethos judgment threshold, which was previously set to '3', to '3.5' based on the ethos learning performance information in order to determine the stable geography category set as the target ethos. . This is, in a situation where the ethos judgment threshold is preset to 3 and the score difference value corresponds to 3, in a situation where it is determined to be a stable ethos, when it is determined that the maximum score difference value based on the result of self-play learning is 2.5, the preset ethos This may be due to the increase in the probability that the score difference value based on self-play learning increases and approaches 3 when the judgment threshold becomes higher. Although this example has been described as above, it will be apparent that various other embodiments are possible.

In this way, the situation determination model server 400 generates ethos learning data by performing ethos determination based on a plurality of embarkation candidates information obtained from the setting off model server 300, and self-learning based on this ethos learning data. By obtaining the gi wind learning performance information from the set-off model server 300 that has performed and performing the gi wind determination to adjust the gi wind judgment threshold, it is possible to improve the accuracy of the gi wind determination operation for the obtained number of start-up candidates, and through this It is possible to increase the quality of the Go game service that judges and utilizes ethos based on running.

Meanwhile, in the following, a deep learning-based ethos determination and utilization method according to another embodiment of the present invention will be described. In the description to be described later, descriptions overlapping those described above may be omitted.

23 is a conceptual diagram for explaining a method of determining a Go situation and utilizing the ethos when playing a Go game according to another embodiment of the present invention. Referring to FIG. 23 , first, the start model server 300 according to another embodiment may set target ethos information for determining the ethos to be learned through the processor 302 . Here, the target air wind information may be information that selects any one of a plurality of air wind categories that are mood-streamed based on the air wind type category or the 1 to n air wind stage categories, as in the embodiment.

In this case, the set off model server 300 according to another embodiment may set target ethos information for each set off model, including a plurality of set off models. In detail, the set-off model server 300 may include at least one set-up model that matches 1:1 with each gi wind category in proportion to the number of categories into which the gi wind is classified. That is, the set-off model server 300 may match each set-off model 1:1 to each set-up model, and may set the target ethos information of each set-off model as a gi wind category matched to each set off model. Thus, the initiation model server 300 may perform learning for a specific target ethos for each initiation model.

For example, the set-off model server 300 may include n set-up models when the number of gi wind categories is n. And the start model server 300, by matching each gi wind category and set off model 1:1, the first gi wind category is set as the target gi wind information of the first set off model, the second gi wind category is the second set off model It is set as the target airflow information of , and in the same way, the nth airflow category may be set as the target airflow information of the nth start model. As another example, the initiation model server 300 may include at least three launch models that match 1:1 to each ethos when there is an aggressive ethos category, a stable ethos category, and a defensive ethos category. That is, the onset model server 300 may include a first set off model matching the aggressive ethos category, a second set off model matching the stable ethos category, and a third set off model matching the defensive ethos category. And the onset model server 300 sets the target ethos information of the first onset model as an aggressive ethos category, the target ethos information of the second onset model as a stable ethos category, and set the target ethos information of the third onset model as a defensive ethos category. can

In addition, the situation judgment model server 400 according to another embodiment may acquire target ethos information for each of a plurality of set-off models from the set-up model server 300 as described above. In addition, the situation judgment model server 400 may obtain information on the number of candidates for starting at least two or more derived from each starting model. And the situation judgment model server 400 may generate ethos determination information by performing ethos determination based on the obtained information on the number of candidates for each embarkation model. In addition, the situation determination model server 400 may generate ethos learning data for each set off model based on the generated ethos determination information and target ethos information for each set off model. And the situation judgment model server 400 may collectively provide the generated ethos learning data to each matching set-up model.

For example, when the first launch model sets the aggressive ethos category as the target ethos, the second launch model sets the stable ethos category as the target ethos, and the third launch model sets the defensive ethos category as the target ethos, situation judgment The model server 400 may first perform a ethos determination for each number of candidates for starting based on the information on the number of candidates for starting at least two or more obtained from each starting model. And the situation judgment model server 400 transmits the ethos learning data based on the number of candidates to set off optimized for the learning of the aggressive ethos to the first start model through ethos determination, and the second start model is optimized for the learning of the stable ethos. It transmits the ethos learning data based on the number of start candidates, and the third start model can transmit and provide collectively the ethos learning data based on the number of start candidates optimized for learning of the defensive ethos. That is, the situation judgment model server 400 can implement a Go game service to which the ethos is applied more efficiently by collectively performing and learning the ethos for a plurality of ethos.

In addition, in another embodiment, the Go game service device including and/or interworking with the start model server 300 and the situation judgment model server 400 as above in another embodiment, the processor 302 and/or the situation of the start model server 300 Through the processor 402 of the judgment model server 400, it is possible to perform Go game play by applying different ethos according to the progress of the Go game (eg, the degree of elapsed time, etc.) or player selection. In detail, the Go game service device according to the progress of the Go game or the player selection according to the preset setting information for automatic wind control (eg, information preset to operate with a specific wind according to the elapsed time, etc.) or the player's selection input It is possible to perform a wind transformation. At this time, the Go game service device may change the starting model used according to the progress of the Go game in order to transform the ethos to be applied to the Go game play.

For example, the Go game service device includes a first set-up model that learned an aggressive ethos, a second set-off model that learned a positive ethos, and a third set-off model that learned a defensive ethos, preset automatic ethos control According to the setting information or the player's selection, in the early stage of the Go game, the game is played with a stable ethos using the second start model, and in the middle of the Go game, the game is played with a defensive ethos using the third start model. , in the second half of the Go game, you can play the Go game with an aggressive ethos using the first set-up model. That is, the Go game service device uses different ethos according to the progress of the Go game or player selection, thereby further improving the usability of the Go game service that utilizes the ethos when playing a Go game based on a deep learning neural network.

As described above, the Go game service method and device based on deep learning according to an embodiment of the present invention accurately classify a house, a stone, a stone, a ball, and a big according to the Go rules and predict the situation of Go by predicting the situation of the Go game. And there is an effect that can accurately determine the wind for a specific starting point.

In addition, the Go game service method and the device based on deep learning according to an embodiment of the present invention, the Go game based on a diversified play method by promoting the progress of the game based on various ethos during the Go game play. It can be performed and the difficulty of the game can be precisely adjusted, which has the effect of improving the quality and interest of the Go game.

In addition, the Go game service method and the apparatus based on deep learning according to an embodiment of the present invention, by judging the ethos based on the score difference between the player and the component, and performing a game based thereon, not only wins, but also scores It has the effect of being able to perform Go game play considering even the difference of .

In addition, the Go game service method and the apparatus based on deep learning according to an embodiment of the present invention, by operating so that learning is performed optimized for a specific ethos to be targeted, efficiently and systematically implement learning to implement ethos can have an effect.

In addition, the embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floppy disks. medium), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. A hardware device may be converted into one or more software modules to perform processing in accordance with the present invention, and vice versa.

The specific implementations described in the present invention are only examples, and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of the lines between the components shown in the drawings exemplarily represent functional connections and/or physical or circuit connections, and in an actual device, various functional connections, physical connections that are replaceable or additional may be referred to as connections, or circuit connections. In addition, unless there is a specific reference such as “essential” or “importantly”, it may not be a necessary component for the application of the present invention.

In addition, although the detailed description of the present invention has been described with reference to a preferred embodiment of the present invention, those skilled in the art or those having ordinary knowledge in the art will appreciate the spirit of the present invention described in the claims to be described later. And it will be understood that various modifications and variations of the present invention can be made without departing from the technical scope. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

100 terminals
200 Go server
300 launch model server
310 search unit
320 self-play
330 Initiation Neural Network
340 ethos learning assistant
400 Situation Judgment Model Server
400a situation judgment model
410 Context Judgment Neural Network
420 input feature extraction unit
430 Correct answer label generator
400b ethos judgment unit

Claims (14)

A communication unit for receiving at least two or more candidates for start based on a plurality of notations and checkerboard states;
a storage unit for storing the situation determination model and the ethos determination unit; and
Reading the situation determination model to perform learning of the situation determination model, and using the learned situation determination model to determine the situation of the checkerboard to which the number of candidates for starting is applied, and the ethos determining unit to set out based on the determined situation Including a;
The situation judgment model is,
an input feature extraction unit for extracting input features from the inputted checkerboard state; and
Containing a situation judgment neural network that generates a situation value for the intersection of the input checkerboard state based on the extracted input feature;
The ethos determining unit is configured to determine the house area of the player and the Oponent by using the situation value, a predetermined threshold value, and the presence or absence of stones to generate holding house information including information obtained by calculating the number of players and the Oponents. and setting target ethos information for determining the ethos that the starting model wants to learn, generating the ethos determination information based on the number of candidates and the target ethos information, and based on the holding house information, a player score value and calculating an oponent score value, generating a score difference value based on the calculated player score value and an oponent score value, setting a ethos judgment threshold that is a parameter of the ethos determination process for the number of candidates to be launched, and the Comparing the score difference value and the gi wind judgment threshold to generate the gi wind judgment information for each of the number of candidates for launch
Layout judgment model server. delete delete delete delete The method of claim 1,
The wind determination unit,
Creating ethos learning data that assists self-learning of the initiation model based on the ethos determination information
Layout judgment model server. 7. The method of claim 6,
The communication unit,
Obtaining gi wind learning performance information from the initiation model,
The wind determination unit,
Adjusting the gi wind judgment threshold based on gi wind learning performance information obtained from the initiation model
Layout judgment model server. Communication unit for transmitting the starting point according to the state of the Go board according to the Go game request of the user terminal;
a storage unit for storing the set-off models of different gi wind categories to determine the set-off point according to the checkerboard state; and
A processor that reads the set off model, determines a start point based on the Go board state of the Go game using the read set off model, and provides a Go game service to the user terminal; includes,
The plurality of launch models include a first launch model of an aggressive ethos for each ethos category, a second launch model of a stable ethos, and a third launch model of a defensive ethos,
The first start model compares the score difference value of the player score value and the component score value among the number of start candidates in the checkerboard state with a preset ethos judgment threshold, and the number of start-up candidates classified as the aggressive ethos is learned with the ethos learning data. It is a starting model,
The second start model compares the score difference value of the player score value and the component score value among the number of start candidates in the checkerboard state and a preset ethos judgment threshold, and is learned with the ethos learning data of the number of start candidates classified as the stable ethos. It is a starting model,
The third start model compares the score difference value of the player score value and the component score value among the number of start candidates in the checkerboard state with a preset ethos judgment threshold and learns with the ethos learning data of the number of start candidates classified as the defensive ethos. It is an established launch model,
The processor, when requesting the Go game, determines a starting model to proceed with the Go game among the plurality of starting models, and determining a starting point through the determined starting model
Launch model server. 9. The method of claim 8,
The set-out model is a search unit that provides the number of start candidates based on Monte Carlo Tree Search (MCTS); and a set-out neural network guiding the search unit; and the start-up neural network by performing self-play a self-play unit that allows self-learning; and an ethos learning auxiliary unit that assists the self-learning based on the ethos learning data,
The gi wind learning auxiliary unit,
Based on the result of the self-learning to generate ethos learning performance information, which is information diagnosing ethos judgment information,
Launch model server. The communication unit, the situation determination model and the ethos determination unit stored in the storage unit, the situation judgment model and the ethos judgment model server comprising a processor driving the ethos determination unit by determining the state of the checkerboard state to determine the ethos and generate ethos learning data In a deep learning-based Go game service method,
setting, by the processor, target airflow information;
setting, by the air wind determination unit, a wind wind determination threshold;
acquiring the number of two or more start candidates based on the state of the checkerboard by the communication unit;
determining, by the processor, the situation of the checkerboard state to which the number of candidates to start is applied by using the situation determination model;
calculating, by the processor, a score difference value for each of the number of start candidates based on the determined situation using the ethos determination unit;
generating, by the processor, the geography determination information for each of the number of candidates to set out on the basis of the calculated score difference value and the gi wind determination threshold using the gi wind determination unit; and
Comprising the step of the communication unit generating and transmitting the spirit learning data for each of the number of start candidates based on the generated wind determination information and the target air wind information,
The wind judgment information is,
Information that determines the category of the spirit implemented by the start of the number of start candidates
A Go game service method based on deep learning. 11. The method of claim 10,
Further comprising the step of receiving the learning performance information from the start model server,
The gi wind learning performance information,
It is the information that the start model server diagnoses the ethos determination information based on the ethos learning data.
A Go game service method based on deep learning. 12. The method of claim 11,
Further comprising the step of adjusting the gi wind judgment threshold based on the gi wind learning performance information
A Go game service method based on deep learning. 11. The method of claim 10,
Calculating the score difference value comprises:
generating holding house information, which is information obtained by calculating the number of player houses and the number of component houses by deriving a situation value based on the situation determination for each of the number of starting candidates;
generating the score difference value by calculating a player score value and an element score value based on the holding house information
A Go game service method based on deep learning. 11. The method of claim 10,
The category of the ethos is,
including offensive ethos, stable ethos and defensive ethos;
The aggressive ethos is a case in which the score difference value is greater than the ethos judgment threshold, the stable ethos is a case in which the score difference value is the same as the ethos judgment threshold, and the defensive ethos is the case where the score difference value is the ethos judgment threshold. in the case of smaller
A Go game service method based on deep learning.

Images