JP7051946B2 - Go game service system and service method based on deep learning - Google Patents

Description

The present invention relates to a Go game service system and a service method based on deep learning. More specifically, the present invention relates to a method for determining a Go situation using a deep learning neural network and a device thereof.

As the use of user terminals such as smartphones, tablet PCs (Personal Computers), PDAs (Personal Digital Assistants), notebooks, etc. has become popular and information processing technology has developed, it is a type of board game that uses user terminals. It has become possible to play a certain Go, and it has become possible to play a Go game with a non-human programmed artificial intelligence computer.

Go has more cases than board games such as chess and shogi, and there is a limit to how artificial intelligence computers can play human-level games. In order to overcome these limitations, research is being actively pursued to improve the strength of artificial intelligence computers. Recently, developers have applied the Monte Carlo Tree Search (MCTS) algorithm and deep learning technology to artificial intelligence computers to raise the power of artificial intelligence computers to above the level of professional players.

In particular, it is important for Go to judge the situation such as who is winning and how much while the game progresses. This is because Go is often in a number, and even if it is a disadvantageous situation, it can be changed to an advantageous position by the game strategy. However, there are many cases of Go according to the rules, and it is difficult to judge the exact situation. It is not easy for ordinary people and amateurs who play Go to judge the situation with Go, which is often used, and the accuracy is remarkable when using a recently developed artificial intelligence computer to judge the situation of Go. There was a problem that it was low. Recently, there is a method to realize an influence function as an artificial intelligence computer for judging the Go situation during a game. However, the method of determining the situation using the influence function has a problem that the Go game rules are not reflected and the accuracy of the determination is low. In addition, a method for determining a posture using a pattern has been developed, and this method is a method for predicting a posture by calculating the influence of a go stone placed on a go board on a pattern. However, the method of determining the situation using a pattern has a problem that it cannot be determined in a specific situation or an accurate determination cannot be made. In addition, the posture judgment method using the rollout neural network is to simulate the game until the end of the game for a certain number of times in the current state of the go board, and then use the average value for each stone state on the board in the end state of the game. Is a way to predict.

However, the posture judgment method using the rollout neural network requires a lot of time for simulation, and there is a problem that the posture judgment based on the simulation result is not accurate. In addition, the method of determining the situation using deep learning technology is a method using a model that learned the Go situation using CNN (GoCNN), but the accuracy of the situation prediction and the prediction of the discarded stones on the board is low, so it is useless. There is a problem that the prediction accuracy of Goban is low.

Republic of Korea Published Patent Gazette No. 10-2015-0129265

The present invention relates to a Go game service system and a service method based on deep learning for solving the above-mentioned problems. In more detail, we will propose a method and a device for determining the Go situation using a deep learning neural network.

Specifically, the present invention provides a Go game service system and a service method based on deep learning that can accurately determine the situation of Go by accurately classifying the ground, abandoned stones, stones, uselessness, and seki according to the Go rules. The purpose.

Another object of the present invention is to provide a Go game service system and a service method based on deep learning that can accurately determine the situation of Go by predicting the ground, abandoned stones, stones, uselessness, and seki according to the Go rules.

Another object of the present invention is to provide a Go game service system and a service method based on deep learning that can quickly determine the situation during a Go game.

The Go game service system based on the deep learning according to the embodiment provides a Go game service and a situation judgment in response to a user requesting a Go game via a terminal or a request for a situation judgment during a Go game. Go server, a start model server that advances a Go game with the user in response to the start request of the Go server, and a response to the user's situation judgment request based on a plurality of Go score information transmitted from the Go server. It can be equipped with a posture judgment model server that makes a posture judgment.

Further, the situation determination model server receives a storage unit for storing a plurality of game record information transmitted from the Go server and a situation determination request requested by the user from the Go server, and provides the situation determination to the Go server. It can be provided with a communication unit and a processor that reads a situation determination model from the storage unit, learns the situation determination model, and determines the situation in a go board state using the learned situation determination model.

In addition, the posture determination model is based on the input feature extraction unit that extracts the input features related to the first Go board state from the plurality of game score information, and the second Go board state of the game currently in progress based on the extracted input features. A correct answer label generator that generates a correct answer label that can be distorted, and a posture judgment neural that trains the posture judgment model so that the input data for the extracted input feature is the same as the target data for the correct answer label. It can be equipped with a network.

Further, the correct answer label may be a stance value displayed as a numerical value from -1 to +1 for an intersection or the like in the second board state (however, the stance value is a constant), and the correct answer label is the first. 2 The first pretreatment that makes the boundary of the ground clear for Yose in the Go board state, and the stones that are placed within the boundary of the ground in the first pretreatment state and are unnecessary for the division of the ground are removed. It can be obtained by performing the second preprocessing.

Further, the correct answer label is a third before changing each intersection in the second board state to a stance value consisting of any one of -1, 0, and +1 in the second preprocessing state. It can be acquired via processing, and the third pretreatment is 0 if one's stone is placed at a predetermined intersection in the second pretreatment state, +1 if one's ground area. If the opponent's stone is placed, it can be changed to 0, and if it is the opponent's ground area, it can be changed to the position value corresponding to -1.

Further, in the third pretreatment, 0 if one's stone is placed at a predetermined intersection in the second pretreatment state, +1 if one's own ground area, and 0 if an opponent's stone is placed. , If it is the other party's ground area, it can be changed to a stance value corresponding to -1, and if it is not good, it can be changed to a stance value corresponding to 0. Each of the blocks can include a convolution layer, a bulk normalization layer, a Relu activation function layer, and a skip connection.

Further, the posture judgment neural network can be trained by using the posture judgment loss L _pre according to the following mathematical formula.

Further, the input features are the position information of the stone with respect to the recent 8 moves of the black player and the position information of the stone with respect to the recent 8 moves of the white player in the first board state, and whether the player is currently black or white. Can include order information for.

The Go game service method based on deep learning according to the embodiment is a deep game in which a Go board state is determined by a communication unit, a storage unit in which a situation determination model is stored, and a situation determination model server including a processor for driving the situation determination model. In a Go game service method based on learning, a step in which the communication unit receives a plurality of Gobans, and an input feature relating to the first Go board state of the plurality of Gobans by the processor using the input feature extraction unit of the posture determination model. And the correct answer label that the processor can use in the correct answer label generation unit of the posture judgment model to generate a correct answer label that can be played in the second board state of the game currently in progress based on the input feature. Steps to train the posture judgment model so that the input data for the input feature is the same as the target data for the correct answer label, and the processor completes the training to build the posture judgment model. And a step to generate a posture value for the intersection of the second Go board state when the second board state in which the posture judgment is required is input by the processor using the trained posture judgment model. And can be included.

The Go game service system and the service method based on deep learning according to the embodiment can determine the Go situation by using the deep learning neural network.

In addition, the Go game service system and service method based on deep learning according to the embodiment can accurately classify the ground, abandoned stones, stones, uselessness, and seki according to the Go rules, and accurately judge the situation of Go.

In addition, the Go game service system and service method based on deep learning according to the embodiment can predict the ground, abandoned stones, stones, uselessness, and seki according to the Go rules, and can accurately judge the situation of Go.

Further, the Go game service system and the service method based on the deep learning according to the embodiment can quickly determine the situation during the Go game.

It is an exemplary diagram for the Go game service system based on deep learning according to the embodiment of the present invention. It is a figure for demonstrating the start model structure of the start model server for the start of the artificial intelligence computer in the Go game service based on the deep learning which concerns on embodiment of this invention. It is a figure for explaining the movement probability distribution with respect to the start point by the policy of the start model. It is a figure for demonstrating the value and the number of visits with respect to the start point of a start model. It is a figure for demonstrating the process which a start model starts along the pipeline of a search part. It is an exemplary diagram which shows the screen which provides the posture determination function of the Go game service based on the deep learning which concerns on embodiment of this invention. It is a figure for demonstrating the posture judgment model structure of the posture judgment model server of this invention. It is a figure for demonstrating one block among the neural network structure consisting of a plurality of blocks of the posture judgment model of this invention. It is a figure for demonstrating the 1st and 2nd preprocessing steps for generating the correct label used for learning the posture judgment model of this invention. It is a figure for demonstrating the 1st and 2nd preprocessing steps for generating the correct label used for learning the posture judgment model of this invention. It is a figure for demonstrating the 3rd preprocessing step for generating the correct label used for learning the posture judgment model of this invention. It is a figure for demonstrating the posture determination result of the posture determination model of this invention. It is a shape comparing the posture judgment result of the posture judgment model of the present invention and the posture judgment result by the deep learning model according to the prior art. It is a shape comparing the posture judgment result of the posture judgment model of the present invention and the posture judgment result by the deep learning model according to the prior art. It is a shape comparing the posture judgment result of the posture judgment model of the present invention and the posture judgment result by the deep learning model according to the prior art. It is an exemplary diagram for the signal flow in the Go game service system based on deep learning according to the embodiment of the present invention. Among the Go game service methods based on deep learning according to the embodiment of the present invention, this is a method for determining the situation. Among the posture determination methods of FIG. 17, this is a training data preprocessing method for generating a correct answer label.

Where various transformations can be made and various embodiments can be made, the present invention exemplifies specific embodiments in the drawings and attempts to illustrate them in detail. The effects and features of the present invention, and the methods for achieving them, will be clarified with reference to the embodiments described in detail below with reference to the drawings. However, the present invention is not limited to the embodiments disclosed below, and can be realized in various forms. In the following embodiments, terms such as first and second are used for the purpose of distinguishing one component from the other, without limiting the meaning. Also, a singular expression includes multiple expressions unless they are meant to be explicitly different in context. Also, terms such as "include" or "have" mean that the features or components described herein are present, to which one or more other features or components are added. It does not preclude the possibility. Further, in the drawings, for convenience of explanation, the size of the components and the like may be exaggerated or reduced. For example, the size and thickness of each configuration shown in the drawings are arbitrarily shown for convenience of explanation, and thus the present invention is not necessarily limited to those shown in the drawings.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when the embodiments are described with reference to the drawings, the same or corresponding components are designated by the same drawing reference numerals. Duplicate explanations will be omitted.

FIG. 1 is an exemplary diagram for a Go game service system based on deep learning according to an embodiment of the present invention.

As shown in FIG. 1, the Go game service system based on deep learning according to the embodiment can include a terminal 100, a Go server 200, a start model server 300, a posture determination model server 400, and a network 500.

Each component of FIG. 1 can be connected via the network 500. It means a connected structure in which information can be exchanged between each node such as a terminal 100, a Go server 200, a start model server 300, and a posture judgment model server 400, and is an example of such a network. 3GPP (3rd Generation Partnership Project) network, LTE (Long Term Evolution) network, WIMAX (World Intellaviity for Microwave , WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth® network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc. It's not something.

<Terminal 100>
First, the terminal 100 is a terminal of a user who wants to receive a Go game service. Also, the terminal 100 is one or more computers or other electronic devices used by the user to execute applications that perform various tasks. For example, it comprises a computer, a laptop computer, a smartphone, a mobile phone, a PDA, a tablet PC, or any other device capable of operating to communicate with the Go server 200. However, the terminal 100 is not limited to this, and the terminal 100 is an external input / output device provided with a processing logic that is executed on various machines and interprets and executes an instruction word stored in a plurality of memories. Various other elements can be included, such as the process of displaying graphic information for a graphic user interface (GUI) on top. Further, the terminal 100 can be connected to an input device (eg, mouse, keyboard, touch-sensitive surface, etc.) and an output device (eg, display device, monitor, screen, etc.). The application executed by the terminal 100 can include a game application, a web browser, a web application running on the web browser, a word processor, a media player, a spreadsheet, an image processor, security software, or the like.

Further, the terminal 100 can include at least one memory 101 for storing instructions, at least one processor 102, and a communication unit 103.

The memory 101 of the terminal 100 can store a plurality of application programs or applications driven by the terminal 100, data for the operation of the terminal 100, and command words. The instruction can be executed by the processor 102 in order to execute the operation by the processor 102, and the operation is to transmit a Go game execution request signal, game data transmission / reception, start information transmission / reception, a posture judgment request signal transmission, and a posture judgment. The operation of receiving the result and receiving various information can be included. Further, the memory 101 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in terms of hardware, and the memory 101 has a storage function of the memory 101 on the Internet. It can be a web storage that fulfills the above.

The processor 102 of the terminal 100 can control the overall operation and perform data processing for receiving the provision of the Go game service. If the Go game application is executed on the terminal 100, the Go game environment is configured on the terminal 100. Then, the Go game application exchanges the Go server 200 with the Go game data via the network 500 so that the Go game service is executed on the terminal 100. Such processor 102, ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controller (controllers) , Microcontrollers, microprocessors, and any other form of processor for performing other functions.

The communication unit 103 of the terminal 100 has the following communication methods (for example, GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), HSDPA (High Speed Downlink PackAccessPass), and the communication unit 103. , LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc.), WLAN (Wireless LAN), Wi-Fi (Wireless-Fildity), Wi-Fi (Wireless File), Wi-Fi (Wireless File) A base station, an external terminal, or a server that can transmit and receive at least one signal on a network constructed by (Digital Living Network Alliance), WiBr (Wiress Broadband), and WiMAX (World Interoperability for Wireless Access).

<Go server 200>
The Go game service provided by the Go server 200 can be configured in such a form that both the computer user of the pseudo image provided by the Go server 200 and the actual user participate in the game. In this, one actual user and one computer user play a game together in a Go game environment realized on the user-side terminal 100. In another aspect, the Go game service provided by the Go server 200 may be configured in a form in which a plurality of user-side devices participate and the Go game is played.

The Go server 200 can include at least one memory 201 for storing instructions, at least one processor 202, and a communication unit 203.

The memory 201 of the Go server 200 can store a plurality of application programs (application programs) or applications (applications) driven by the Go server 200, data for the operation of the Go server 200, and command words. The instruction can be executed by the processor 202 in order to execute the operation by the processor 202, and the operation is the game execution request signal reception, the game data transmission / reception, the start information transmission / reception, the situation judgment request signal transmission / reception, the situation judgment result transmission / reception, And various transmission operations can be included. Further, the memory 201 can store a plurality of game records played by the Go server 200 or a plurality of game records that have already been published. Each of the plurality of game records can include all the information from the first start, which is the start information at the beginning of the game, to the final start at the end of the game. That is, the plurality of game records can include history information regarding the start. The Go server 200 makes it possible to provide a plurality of stored game records to the situation determination model server 400 for training of the situation determination model server 400. Further, the memory 201 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in terms of hardware, and the memory 201 has a storage function of the memory 201 on the Internet. It can be a web storage that fulfills the above.

The processor 202 of the Go server 200 can control the overall operation and perform data processing for providing the Go game service. Such processor 202, ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controller (controllers) , Microcontrollers, microprocessors, and any other form of processor for performing other functions.

The Go server 200 can communicate with the terminal 100, the start model server 300, and the posture determination model server 400 via the network 500 via the communication unit 203.

<Start model server 300>
The start model server 300 can be provided with another cloud server or computer device. Further, the start model server 300 may be a processor of the terminal 100 or a neural network system provided in the data processing unit of the Go server 200, but in the following, the start model server 300 is the terminal 100 or the Go server 200. It will be described as another device.

The start model server 300 can include at least one memory 301 for storing instructions, at least one processor 302, and a communication unit 303.

The start model server 300 is an artificial intelligence computer that can play against the user of the terminal 100 by learning by itself according to the Go rules to build a start model that is a deep learning model, and can win the game in its own turn. You can start playing Go stones as you can. The detailed explanation of the start model server 300 training with the start model follows the description of the start model of FIGS. 2 to 5.

The memory 301 of the start model server 300 can store a plurality of application programs (application programs) or applications (applications) driven by the start model server 300, data for the operation of the start model server 300, and instruction words. .. The instructions can be executed by the processor 302 in order for the processor 302 to perform the operation, and the operation can include a start model learning (training) operation, a start information transmission / reception, and various transmission operations. Further, the memory 301 can store a start model which is a deep learning model. Further, the memory 301 can be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in terms of hardware, and the memory 301 has a storage function of the memory 301 on the Internet. It can be a web storage that fulfills the above.

The processor 302 of the start model server 300 reads out the start model stored in the memory 302, and performs the start model learning and the go stone start described below according to the constructed neural network system. According to the embodiment, the processor 302 includes a main processor that controls the entire unit, and a plurality of graphic processors (Graphics Processing Units, GPUs) that process a large amount of operations required for driving a neural network by a start model. Can be configured.

The start model server 300 can communicate with the Go server 200 via the network 500 via the communication unit 303. Further, the start model server 300 can also communicate with the posture determination model server 400 and the terminal 100 via the network 500.

<Position judgment model server 400>
The posture determination model server 400 can be provided with another cloud server or computer device. Further, the posture determination model server 400 may be a neural network system provided in the processor of the terminal 100 or the data processing unit of the Go server 200, but in the following, the posture determination model server 400 is the terminal 100 or the Go server 200. It will be described as a separate device.

The posture determination model server 400 can include at least one memory 401 for storing instructions, at least one processor 402, and a communication unit 403.

The posture determination model server 400 can receive the training data set from the Go server 200 via the communication unit 403. The training data set may be a plurality of game records and information on determining the posture of the plurality of game records. The posture judgment model server 400 constructs a posture judgment model, which is a deep learning model, by teaching and learning so that the posture for the state of the go board on which the go stones are placed can be judged using the received training data set, and the terminal 100 users. It is possible to make a situation judgment in response to a request for a situation judgment. The detailed explanation of the situation judgment model server 400 training in the situation judgment model follows the explanation of the situation judgment model of FIGS. 6 to 18.

The memory 401 of the posture judgment model server 400 stores a plurality of application programs (application programs) or applications (applications) driven by the posture judgment model server 400, data for the operation of the posture judgment model server 400, and command words. be able to. The instructions can be executed by the processor 402 in order for the processor 402 to execute the operation, and the operation includes a posture judgment model learning (training) operation, a posture judgment execution, a posture judgment result transmission, a plurality of game record information reception, and an operation. Various transmission operations can be included. Further, the memory 401 can store a posture determination model, which is a deep learning model. Further, the memory 401 may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc. in terms of hardware, and the memory 401 has a storage function of the memory 401 on the Internet. It can be a web storage that fulfills the above.

The processor 402 of the posture judgment model server 400 reads the posture judgment model stored in the memory 401, and then performs the posture judgment model learning described below according to the constructed neural network system and the posture judgment of the go board during the game. It will be like. According to the embodiment, the processor 402 includes a main processor that controls the entire unit, and a plurality of graphic processors (Graphics Processing Units, GPUs) that process a large amount of operations required for driving a neural network by a posture determination model. Can be configured in.

The situation determination model server 400 can communicate with the Go server 200 via the network 500 via the communication unit 403. Further, the posture determination model server 400 can communicate with the start model server 300 and the terminal 100 via the network 500.

<Starting model>
FIG. 2 is a diagram for explaining a start model structure of a start model server 300 for starting an artificial intelligence computer in a Go game service based on deep learning according to an embodiment of the present invention, and FIG. 3 is a diagram for explaining a start model. FIG. 4 is a diagram for explaining the movement probability distribution for the start point according to the policy of, FIG. 4 is a diagram for explaining the value and the number of visits for the start point of the start model, and FIG. 5 is a diagram in which the start model is the search unit. It is a figure for demonstrating the process of embarking along the pipeline of.

As shown in FIG. 2, the start model according to the embodiment of the present invention is a deep learning model of the start model server 300, and can include a search unit 310, a self-play unit 320, and a start neural network 330.

The start model can be learned as a model to start so that the player can win the game by using the search unit 310, the self-play unit 320, and the start neural network 330. More specifically, the search unit 310 can perform a Monte Carlo tree search (Monte Carlo Tree Search and MCTS) operation by the guide of the starting neural network 330. MCTS is a kind of experiential search algorithm for decision making. That is, the search unit 310 can perform MCTS based on the movement probability value P and / or the value V provided by the start neural network 330. As an example, the search unit 310 guided by the start neural network 330 can perform MCTS and output a search probability value π which is a probability distribution value for a start point or the like. The self-play unit 320 can play Go by itself depending on the search probability value π. The self-play unit 320 advances the Go game by itself until the victory or defeat of the game is decided, and when the self-game is completed, provides the Go board state S, the search probability value π, and the self-play value Z to the starting neural network 330. be able to. The go board state S is a state in which a go stone is placed at a starting point or the like. The self-play value Z is a winning percentage value when a self-play is played in the board state S. The start neural network 330 can output the movement probability value P and the value V. The movement probability value P is a probability distribution value that numerically expresses which starting point is a good move that can win the game with respect to the starting point or the like depending on the board state S. The value V represents the winning percentage at the time of starting the starting point. For example, a starting point with a high movement probability value P may be a good move. The start neural network 330 can be trained so that the movement probability value P is the same as the search probability value π, and the value V is the same as the self-play value Z. After that, the trained start neural network 330 guides the search unit 310, and the search unit 310 advances the MCTS so as to search for a move better than the previously search probability value π, and outputs a new search probability value π. The self-play unit 320 outputs a new self-play value Z by the board state S based on the new search probability value π, and provides the board state S, the new search probability value π, and the new self-play value Z to the starting neural network 330. be able to. The start neural network 330 can be retrained so that the movement probability value P and the value V are output with the new search probability value π and the new self-play value Z. That is, the launch model can be trained to repeat this process in search of a better launch point for the launch neural network 330 to win in the game. As an example, the start model can utilize the start loss l. The start loss l is as shown in Equation 1.

θ is a parameter of the neural network, and c is a very small constant.

Making z (self-play value) and v (value) the same in the start loss l of Equation 1 corresponds to the mean square loss term, and π and p (movement probability value). It corresponds to the cross entropy loss term, and multiplying θ by c is a normalization term to prevent overfitting.

For example, as shown in FIG. 3, in the trained start model, the movement probability value P of the start point or the like can be represented by the probability distribution value as shown in FIG. As shown in FIG. 4, the value V of the trained launch model can be represented by the value displayed above at one launch point in FIG. The starting neural network 330 can be configured with a neural network structure. As an example, the starting neural network 330 can be composed of one convolution block and 19 residal blocks. The convolution block may be a form in which a plurality of 3 × 3 convolution layers are overlapped. One recital block may have a form in which a plurality of 3 × 3 convolution layers are overlapped and a skip connection is included. The skip connection is a structure in which the input of a predetermined layer is output together with the output value of the layer and is input to another layer. Further, the input of the starting neural network 330 is the position information of the stone for the recent 8 moves of the black player, the position information of the stone for the recent 8 moves of the 100 player, and the order of whether the player is currently black or white. A 19 * 19 * 17 RGB image containing information can be input.

As shown in FIG. 5, the trained start model can be started using the start neural network 330 and the search unit 310 in its own order. In the starting model, the activity function Q and the reliability value U are high in the second board state S and S1-2, which are the branches that are not searched through the MCTS in the first board state S and S1 through the selection process a. Select a starting point. The activity function Q is an average value of the value V calculated each time the branch is passed. The confidence value U is proportional to the number of visits N through the branch. The start model can be expanded to the third board state S, S1-2-1 at the start point selected through the expansion and evaluation process b, and the movement probability value P can be calculated. The start model calculates the value of the third board state S and S1-2-1 expanded via the backup process c, and stores the activity function Q such as the passed branch, the number of visits N, and the movement probability value P. be able to. The start model can repeat the selection a, expansion and evaluation b, and backup c processes, create a probability distribution using the number of visits N for each start point, and output the search probability value π. The start model can be started by selecting the highest search probability value π from the start points and the like.

<Position judgment model>
FIG. 6 is an exemplary diagram showing a screen that provides a posture determination function of a Go game service based on deep learning according to an embodiment of the present invention, and FIG. 7 is a posture determination model structure of the posture determination model server 400 of the present invention. FIG. 8 is a diagram for explaining one block of the neural network structure composed of a plurality of blocks of the posture determination model of the present invention.

As shown in FIG. 6, the Go game service based on deep learning according to the embodiment of the present invention can determine the current state of the Go board. As an example, as shown in FIG. 6, when the user clicks the situation judgment menu A during a Go game on the screen of the terminal 100 to request the situation judgment, the Go game service based on deep learning displays the situation judgment result in the pop-up window. Can be provided. The situation judgment is to calculate the opponent and your own land during the Go game and judge who is winning by what point. For example, if a user decides that the situation is in his or her own favor, he or she should not overdo it and make a strategy to end the game while maintaining the current favorable situation. If it is judged to be disadvantageous, various strategies can be sought so that the game phase can be newly changed. The criteria for determining the situation are the ground, abandoned stones, stones, uselessness, and seki, depending on how the go stones are placed on the board. A stone is a stone placed on a board and is not a score under Korean rules. The ground is an area composed of vacant points surrounded by go stones of one color, and is a score according to Korean rules. No good and Seki are areas that are neither black nor 100 when Go is over, and are not points under Korean rules. The abandoned stone on the board is a dead stone that can only be caught by hitting it no matter how it is placed on the board. According to Korean rules, it is used to fill the opponent's land, so in terms of points be. Seki refers to an area that is neither black nor a hundred when Go is over. Therefore, the situation judgment can be made accurately only by accurately classifying or predicting the ground, abandoned stones, stones, uselessness, and seki in the state of the board on which the go stones are placed. At this time, to accurately classify the ground, abandoned stone, stone, useless, and Seki is to classify the state where the ground, abandoned stone, stone, useless, and Seki are completely completed. Accurate prediction of Seki can be to predict the ground, abandoned stones, stones, spoilage, and conditions that are likely to be Seki.

As shown in FIG. 7, the posture determination model according to the embodiment of the present invention is a deep learning model of the posture determination model server 400, and is a posture determination neural network 410, an input feature extraction unit 420, and a correct label generation unit 430. Can be provided.

The posture determination model can be supervised learning so that the posture of the current Go board state can be determined by using the posture determination neural network 410. More specifically, it is possible to generate a training data set for the Go board state S of the situation judgment model and train the situation judgment neural network 410 to judge the current state of the Go board state S using the generated training data set. can. The situation determination model server 400 can receive a plurality of game records from the Go server 200. Each game record of the plurality of game records can include each board state S according to the starting order.

The input feature extraction unit 420 can extract the input feature IF in the board state S of a plurality of game records and provide the input feature IF to the posture determination neural network 410 as input data for training. The input feature IF of the board state S is the position information of the stone for the latest 8 moves of the black player, the position information of the stone for the latest 8 moves of the white player, and the order information for whether the player is currently black or white. It can be a 19 * 19 * 18 RGB image containing. As an example, the input feature extraction unit 420 may consist of a neural network structure and may include a kind of encoder.

The posture determination neural network 410 can be configured with a neural network structure. As an example, the posture determination neural network 420 can be composed of 19 residal blocks. As shown in FIG. 8, any one resonant block (800: block n) includes 256 3 × 3 convolution layers 801, a first batch normalization layer 802, and a first Relu. The activation function layer 803, 256 3 × 3 convolution layers 804, the second batch normalization layer 805, the skip connection 807, and the second Relu activation function layer 806 may be arranged in this order. can. The first and second batch normalization layers 802 and 805 undergo covariate shift, which is a phenomenon in which the distribution of the input of the current layer changes due to the parameter change of the previous layer during learning. It is for prevention. The skip connection 807 prevents the performance of the neural network from deteriorating even if the block layer becomes thick, and makes it possible to further thicken the block layer to improve the performance of the entire neural network. The skip connection 807 is placed in the second Relu activation function layer 806 in which the first input data of the resonant block 800 is arranged second with the output of the second second batch normalization layer 805. It can be in the form of input.

9 and 10 are diagrams for explaining the first and second preprocessing steps for generating the correct label used for learning the posture determination model of the present invention, and FIG. 11 is a diagram for explaining the first and second preprocessing steps. It is a figure for demonstrating the 3rd preprocessing step for generating the correct label used for learning the posture judgment model of this invention.

The correct answer label generation unit 430 can generate a correct answer label used for learning so that the posture determination neural network 410 can perform an accurate posture determination.

More specifically, the correct answer label generation unit 430 receives the board state S based on the input data as an input, performs the first preprocessing for making a twist in the current board state S, and performs the first preprocessing state P1. Can be generated. The first pretreatment, Yose, is the process of finishing the game by making a predetermined start so that the boundary of the ground becomes clear before calculating the ground. As an example, as shown in FIG. 9, the correct label generation unit 430 can generate the first preprocessing state P1 of FIG. 9 (b) by twisting in the current board state S of FIG. 9 (a). ..

The correct label generation unit 430 is arranged within the boundary of the ground in the first pretreatment state P1, and performs the second pretreatment for removing stones unnecessary for the division of the ground to generate the second pretreatment state P2. can. For example, stones that are placed within the boundaries of the ground and are not needed to divide the ground can be abandoned stones. I explained earlier that the abandoned stone is a dead stone because the opponent's stone is placed in the ground and you have no choice but to catch it no matter how you hit it. Also, stones that are placed within the boundaries of the ground and are not needed to divide the ground can be their own stones placed within the ground. As an example, as shown in FIG. 9, the correct label generation unit 430 removes stones unnecessary for ground division in the first pretreatment state P1 of FIG. 9 (b), and (c) of FIG. The second preprocessing state P2 can be generated.

As another example, as shown in FIG. 10, the correct label generation unit 430 of FIG. 10 (b) is for Yose, which is the first preprocessing in the current board state S of FIG. 10 (a). You can start red x like this. The correct label generation unit 430 starts green × to remove the waste stones in order to remove the waste stones displayed as blue × in FIG. 10 (b), and starts the green × used for removing the waste stones. The stones that have been removed can also be removed and a second pretreatment can be performed.

The correct label generation unit 430 can perform a third preprocessing for changing each intersection to a stance value (g, where g is a constant) displayed from -1 to +1 in the second preprocessing state P2. .. That is, in the third preprocessing, the correct label generation unit 430 changes the second preprocessing state P2, which is an image feature, to the third preprocessing state P3, which is a numerical feature. As an example, in the second pretreatment state P2, 0 if your stone is placed at the intersection, +1 if your stone is placed, 0 if your opponent's stone is placed, and-if your opponent's stone is placed- It can correspond to 1. In this case, the posture determination neural network 410 can be learned so that the ground, the stone, and the abandoned stone can be classified at the time of the posture determination. As another example, in the second pretreatment state P2, if your stone is placed at the intersection, it is 0, if it is your own ground area, it is +1. If your opponent's stone is placed, it is 0. If it is -1, Seki or if it is not good, it can correspond to 0. In another example, the stance-determining neural network 410 can be learned to be able to discriminate between relations and failures during stance-determining. For example, as shown in FIG. 11, the correct label generation unit 430 changes the feature of the second preprocessing state P2 in FIG. 11A to the third preprocessing state P3 in FIG. 11B. be able to.

FIG. 12 is a diagram for explaining the posture determination result of the posture determination model of the present invention.

The trained stance determination model can provide stance values for all intersections of the board if the board state is input. That is, it is possible to provide a constant value of -1 to +1 which is a stance value for 361 points of the Go board intersection.

As shown in FIG. 12, the stance determination model server 400 can determine the stance by using the stance value provided by the stance determination model, a predetermined threshold value, and the presence or absence of stones. As an example, the situation judgment model server 400 determines that there is no stone, and if the situation value exceeds the first threshold value, it is likely to become one's own land, and if the value is close to +1 then oneself. It can be judged as the ground area of. The posture determination model server 400 can display in a square shape of the same color as its own stone, which gradually becomes larger as it is more likely to be its own ground. That is, the higher the possibility of a hundred grounds, the more a white square shape can be displayed, and the higher the possibility of a black background, the more a black square shape can be displayed. The situation judgment model server 400 is a place where there are no stones, and if the situation value is equal to or less than the second threshold value, it is determined that there is a high possibility of becoming the opponent's land, and if the value is close to -1, it is own. It can be judged as a ground area. The posture determination model server 400 can display a square shape having the same color as the opponent's stone, which gradually becomes larger as the possibility of the opponent's land is higher. If the posture determination model server 400 is a place where there are no stones and the posture value is within the third threshold value range or is close to 0, it can be determined to be useless or related. The situation determination model server 400 can display X if it determines that it is useless or related. The stance determination model server 400 can determine that it is a stone of its own or a stone of the opponent if the stance value is within the third threshold range or is close to 0 where the stone is located. The situation determination model server 400 may not display anything if it determines that it is useless or related. The stance judgment model server 400 determines that there is a stone, and if the stance value exceeds the first threshold value, there is a high possibility that it will be a waste stone of the opponent's stone. It can be judged as a discarded stone. The posture determination model server 400 can display a square shape of the same color as its own stone, which gradually becomes larger as the possibility of being a waste stone of the opponent's stone increases. The stance judgment model server 400 determines that there is a stone, and if the stance value is equal to or less than the second threshold value, it is highly likely that the stone will be discarded, and if the value is close to -1, it is likely to be a waste stone. It can be judged as a waste stone of the opponent's stone. The posture determination model server 400 can display a square shape having the same color as the opponent's stone, which gradually becomes larger as the possibility of being a waste stone of the opponent's stone increases.

In addition, the situation determination model server 400 can display the ground calculation result in the current state of the board by using the situation determination criteria determined at each intersection.

Therefore, the Go game service system based on deep learning according to the embodiment can determine the Go situation by using the deep learning neural network. In addition, the Go game service system based on deep learning according to the embodiment can accurately classify the ground, abandoned stones, stones, uselessness, and seki according to the rules of Go, and accurately judge the situation of Go. In addition, the Go game service system based on deep learning according to the embodiment can predict the ground, abandoned stones, stones, uselessness, and seki according to the rules of Go, and can accurately judge the situation of Go. In addition, the Go game service system based on deep learning according to the embodiment can quickly determine the situation during a Go game.

FIG. 13 is a shape comparing the shape judgment result of the shape judgment model of the present invention with the shape judgment result by the deep learning model according to the prior art, and FIG. 14 shows the shape judgment result of the shape judgment model of the present invention and the conventional one. It is a shape comparing the posture judgment result by the deep learning model according to the technique, and FIG. 15 is a shape comparing the posture judgment result of the posture judgment model of the present invention and the posture judgment result by the deep learning model according to the prior art. be.

As shown in FIG. 13, the posture determination model of the present invention determines the posture by classifying the ground, stones, and abandoned stones at each intersection as in the B region of FIG. 13 (a). However, in the posture determination model based on the deep learning model according to the prior art, the ground, the stone, and the abandoned stone cannot be classified with respect to the intersection of the region corresponding to the (a) of FIG. 13 in (b) of FIG.

Similarly, as shown in FIG. 14, the posture determination model of the present invention determines the posture by classifying the ground, stones, and abandoned stones at each intersection as in the C region of FIG. 14 (a). However, in the posture determination model based on the deep learning model according to the prior art, the ground, the stone, and the abandoned stone cannot be classified with respect to the intersection of the region corresponding to the (a) of FIG. 13 in (b) of FIG.

As shown in FIG. 15, the posture determination model of the present invention correctly recognizes a white background as in the D region of FIG. 15 (a). However, in the posture determination model based on the deep learning model according to the prior art, the white background cannot be classified in the region corresponding to FIG. 15 (a) in FIG. 15 (b).

FIG. 16 is an exemplary diagram for a signal flow in a Go game service system based on deep learning according to an embodiment of the present invention.

As shown in FIG. 16, the start model server 300 is an artificial intelligence computer, and learns deeply by the rules of Go so that it can start a go stone so that it can win in a game on its own turn. The starting model, which is a learning model, can be trained (S11). The Go server 200 can transmit a plurality of game records to the situation determination model server 400 (S12). The posture determination model server 400 can generate a training data set. First, the posture determination model server 400 can extract input features in a board state of a plurality of game records (S13). The situation determination model server 400 can generate a correct label by using the board state from which the input features are extracted (S14). The posture judgment model server 400 can train the posture judgment model using a training data set in which the input feature is the input data and the correct answer label is the target data (S15). The terminal 100 can request the Go server 200 to play a Go game against an artificial intelligence computer or against another user terminal (S16). The Go server 200 can request the start model server 300 to start the game if the terminal 100 requests the Go game against the artificial intelligence computer (S17). The Go server 200 advances the Go game, and the terminal 100 and the start model server 300 can start their own turn (S18 to S20). During the game, the terminal 100 can request the Go server 200 to determine the situation (S21). The Go server 200 can request the situation determination model server 400 to determine the current situation of the Go board state (S22). The posture judgment model server 400 extracts the input features of the current Go board state, the posture judgment model, which is a deep learning model, uses the input features to generate the shape value, and makes the shape judgment using the Go board state and the shape value. Can be done (S23). The situation determination model server 400 can provide the situation determination result to the Go server 200 (S24). The Go server 200 can provide the situation determination result to the terminal 100 (S25).

FIG. 17 is a posture determination method among the Go game service methods based on deep learning according to the embodiment of the present invention, and FIG. 18 is training data for generating a correct answer label among the posture determination methods of FIG. It is a pretreatment method of.

As shown in FIG. 17, the Go game service method based on deep learning according to the embodiment of the present invention can include a step (S100) in which the situation determination model server 400 receives a plurality of game records from the Go server.

The Go game service method based on deep learning according to the embodiment of the present invention is a step (S200) in which the input feature extraction unit extracts input features in a go board state of a plurality of game records in the posture judgment model of the posture judgment model server 400. Can be included. The method of extracting the input features follows the description of FIG.

The Go game service method based on deep learning according to the embodiment of the present invention includes a step (S300) of generating a correct label based on a Go board state from which an input feature is extracted by a correct label generation unit in a posture determination model. Can be done. As an example, as shown in FIG. 18, the correct answer label generation step (S300) can include a first preprocessing step (S301) in which the correct answer label generation unit makes a twist in the current board state. The first preprocessing step (S301) follows the description of FIGS. 9 to 10. The correct label generation step (S300) can include a second pretreatment step (S302) in which the correct label generation unit removes unnecessary stones in the first preprocessed Go board state. The second preprocessing step (S302) follows the description of FIGS. 9 to 10. The correct label generation step (S300) can include a third preprocessing step (S303) in which the correct label generation unit changes each intersection in the second preprocessed Go board state to a stance value. The third preprocessing step (S303) follows the description of FIG. The correct answer label generation step (S300) can include a step (S304) of providing the third preprocessing state as the correct answer label to the posture determination neural network as target data. The step (S304) for providing the target data follows the description of FIGS. 7 and 11.

The Go game service method based on deep learning according to the embodiment of the present invention can include a step (S400) of training a posture judgment neural network of a posture judgment model using a training data set. The method of training (learning) the posture judgment neural network follows the explanation of FIG.

The Go game service method based on deep learning according to the embodiment of the present invention includes a step (S500) of constructing a posture judgment model after training of the posture judgment neural network is completed. As an example, the completion of training of the posture judgment neural network may be when the posture judgment loss in FIG. 7 becomes a predetermined value or less.

The Go game service method based on deep learning according to the embodiment of the present invention can include a step (S600) in which the current Go board state is input to the posture determination model by the posture determination request of the terminal.

The Go game service method based on deep learning according to the embodiment of the present invention can include a step (S700) of determining the current state of the Go board in which the situation determination model is input. The step (S700) for determining the posture can follow the explanation that the posture determination model described in FIG. 12 generates the posture value in the current board state.

The Go game service method based on deep learning according to the embodiment of the present invention can include a step (S800) in which the situation determination model server 400 outputs a situation determination result. The step (S800) for outputting the situation determination result can follow the explanation that the situation determination model server described with reference to FIG. 12 provides the situation determination result using the situation value, the state of the board, and a predetermined threshold value.

Therefore, in the Go game service method based on deep learning according to the embodiment, the Go situation can be determined by using the deep learning neural network. In addition, the Go game service method based on deep learning according to the embodiment can accurately classify the ground, abandoned stones, stones, uselessness, and seki according to the rules of Go, and accurately determine the situation of Go. In addition, the Go game service method based on deep learning according to the embodiment can predict the ground, abandoned stones, stones, uselessness, and seki according to the rules of Go, and can accurately judge the situation of Go. In addition, the Go game service method based on deep learning according to the embodiment can quickly determine the situation during a Go game.

The embodiments according to the present invention described above can be realized in the form of program instructions that can be executed via various computer components and can be recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the computer-readable recording medium are those specially designed and configured for the present invention, or those known and usable by those skilled in the art of computer software. sell. Examples of computer-readable recording media include hard disks, floppy disks, magnetic media such as magnetic tapes, light green media such as CD-ROMs and DVDs, and magnetics such as floptic discs. -Includes optical media, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, flash memories, and the like. Examples of program instructions include not only machine language code as created by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device can be changed to one or more software modules in order to perform the processing according to the present invention, and vice versa.

The specific execution or the like described in the present invention is an embodiment and does not limit the scope of the present invention by any method. For the sake of brevity of the specification, the description of conventional electronic configurations, control systems, software, other functional aspects of the system, etc. may be omitted. In addition, the connection or connecting member such as a line between the components shown in the drawings schematically shows a functional connection and / or a physical or circuit connection, and is an alternative in an actual device. It is possible or can be represented as an additional variety of functional connections, physical connections, or circuit connections. In addition, unless there is a specific reference such as "essential" or "important", it may not be a necessary component for the application of the present invention.

Further, in the detailed description of the present invention described, although the description has been made with reference to the preferred embodiment of the present invention, any person skilled in the art or a person having ordinary knowledge in the technical field can be used. It can be understood that the present invention can be variously modified and modified without departing from the ideas and technical areas of the present invention described in the claims described later. Therefore, the technical scope of the present invention is not limited to the contents described in the detailed description of the specification, but must be determined by the scope of claims.

100 Terminal 200 Go server 300 Start model server 310 Search unit 320 Self-play unit 330 Start neural network 400 Position judgment model server 410 Position judgment neural network 420 Input feature extraction unit 430 Correct answer label generation unit

Claims (14)

A Go server that provides a Go game service and a situation judgment in response to a user requesting a Go game via a terminal and requesting a situation judgment during a Go game.
A start model server that advances a Go game with the user in response to a start request of the Go server.
A situation judgment model server that makes a situation judgment corresponding to the user's situation judgment request based on a plurality of game record information transmitted from the Go server, and a situation judgment model server.
Equipped with
The situation judgment model server is
A storage unit that stores a plurality of game record information transmitted from the Go server, and a storage unit.
A communication unit that receives a situation judgment request requested by a user from the Go server and provides the situation judgment to the Go server.
A processor that reads the posture judgment model from the storage unit, learns the posture judgment model, and judges the posture of the Go board state using the learned posture judgment model.
Equipped with
The situation judgment model is
An input feature extraction unit that extracts input features related to the current Go board state from the plurality of game record information, and an input feature extraction unit.
A correct label generation unit that generates a correct label that can be twisted in the current Go board state based on the extracted input features.
A stance judgment neural network that trains the stance judgment model so that the input data for the extracted input features is the same as the target data for the correct label.
A Go game service system based on deep learning. The correct answer label is a Go that is displayed as a numerical value from -1 to +1 for an intersection in the current Go board state (however, the Go is a constant). Go based on deep learning according to claim 1. Game service system. The correct label is the first pretreatment that makes the boundary of the ground clear for Yose in the current Go board state.
The Go game service system based on deep learning according to claim 1, which was obtained by performing the second pretreatment for removing stones unnecessary for the division of the ground, which was placed within the boundary of the ground in the first pretreatment. .. The correct label is obtained through a third preprocessing in which each intersection in the current Go board state is changed to a stance value consisting of any one of -1, 0, or +1 in the second preprocessing. The Go game service system based on the deep learning according to claim 3. In the third pretreatment, 0 if one's stone is placed at a predetermined intersection in the second pretreatment, +1 if one's own ground area, 0 if an opponent's stone is placed, and the other party. The Go game service system based on deep learning according to claim 4, wherein the value of the game is changed to the value corresponding to -1 in the case of the local area. In the third pretreatment, 0 if one's stone is placed at a predetermined intersection in the second pretreatment, +1 if one's own ground area, 0 if an opponent's stone is placed, and the other party. The Go game service system based on deep learning according to claim 4, wherein the value is changed to -1 in the case of the land area of No. 1 and 0 in the case of Seki or No. The position determination neural network includes a plurality of reciprocal blocks, and each of the plurality of reciprocal blocks includes a convolution layer, a batch normalization layer, a Relu activation function layer, and a skip connection according to claim 1. Go game service system based on deep learning. The Go game service system based on deep learning according to claim 2, wherein the posture judgment neural network is trained by using the posture judgment loss L _pre according to the following mathematical formula.

The input features are for the position information of the stone for the last 8 moves of the black player and the position information of the stone for the latest 8 moves of the white player in the current board state, and for whether the player is currently black or white. The Go game service system based on deep learning according to claim 1, which includes order information. In a Go game service method based on deep learning, in which a communication unit, a storage unit in which a situation determination model is stored, and a situation determination model server including a processor for driving the situation determination model are used to determine the situation in a go board state.
The step in which the communication unit receives a plurality of game records,
A step in which the processor uses the input feature extraction unit of the posture determination model to extract input features related to the current Go board state of the plurality of game records.
A step of generating a correct label that allows the processor to make a twist in the current Go board state based on the input feature by using the correct label generation unit of the posture determination model.
A step in which the processor trains the posture determination model so that the input data for the input feature is the same as the target data for the correct label.
The steps that the processor completes training to build a posture judgment model,
The processor includes a step of making a position determination to generate a position value for an intersection of the current board state when the current board state requiring the position determination is input using the trained position determination model.
The step of generating the correct label is
A first pretreatment step that creates a first pretreatment state that makes the boundaries of the ground clear for Yose in the current Go board state.
A second pretreatment step, which is placed within the boundary of the ground in the first pretreatment state and removes stones unnecessary for the division of the ground to generate a second pretreatment state.
A third pre-processing state that generates a third pre-processing state that changes each intersection of the current board state to a stance value consisting of any one of -1, 0, or +1 in the second pre-processing state. Pre-processing steps and
A Go game service method based on deep learning characterized by including. In the third pretreatment step, 0 if one's stone is placed at a predetermined intersection in the second pretreatment state, +1 if one's own ground area, and 0 if an opponent's stone is placed. The Go game service method based on deep learning according to claim 10, wherein if the opponent's ground area is used, the value is changed to a position value corresponding to -1. In the third pretreatment step, 0 if one's stone is placed at a predetermined intersection in the second pretreatment state, +1 if one's own ground area, and 0 if an opponent's stone is placed. The Go game service method based on deep learning according to claim 10, wherein the value is changed to -1 in the case of the opponent's land area and 0 in the case of Seki or No. The Go game service method based on deep learning according to claim 10, wherein the step of training the posture judgment model and training using the posture judgment loss L _pre according to the following mathematical formula.

The input features are for the position information of the stone for the last 8 moves of the black player and the position information of the stone for the latest 8 moves of the white player in the current board state, and for whether the player is currently black or white. The Go game service method based on deep learning according to claim 10, which includes order information.

Images