KR101108987B1 - Apparatus and method for studying pattern of game character by using separation of space - Google Patents

Abstract

The present invention relates to a pattern learning technique of a game character using space division. The present invention relates to a physical, logical, and character state space of a game in order to enable a character in a game to learn a user's behavior and perform a similar behavior to the user. Divide and divide each into neural networks to perform learning. And when learning a game character with a neural network, it is characterized in that the output of the neural network to calculate the probability and apply to the game. According to the present invention, it is possible to improve performance in terms of learning time and results through state space partitioning of the game, and the character can perform various actions in the game by allowing the output node to be selected in proportion to the size of the output node of the neural network. You can do it, and you can even do something similar to a game user.

Space Division, Game Character, Pattern Learning

Description

Apparatus and method for pattern learning of game characters using space division {APPARATUS AND METHOD FOR STUDYING PATTERN OF GAME CHARACTER BY USING SEPARATION OF SPACE}

The present invention relates to a technique for learning a pattern of a game character in a game program. Particularly, after dividing an entire state space of a game by logical, physical, and character, the present invention assigns a divided area to each neural network to learn a game character. The present invention relates to a device and method for learning a pattern of a game character using suitable spatial division.

The present invention is derived from the research conducted as part of the IT growth engine technology development project of the Ministry of Knowledge Economy and the Ministry of Information and Communication Research and Development. [Task management number: 2006-S-044-03, Task name: Multicore CPU and MPU-based crop Platform game technology development].

As is generally known, artificial life is the study of general characteristics of life by reproducing phenomena represented by living things on artificial media such as computers and robots. If biology has taken an analytical approach to biological phenomena, artificial life has a comprehensive approach. Game developers have long attempted to apply artificial life techniques to games, but they have been used on a limited basis because of the unpredictable nature of artificial life.

Recently, there have been attempts to apply artificial life to games again. This is because applying the basic characteristics of artificial life, adaptability and creativity to the game, can increase the interest of the game by adapting to a complicated environment or user's operation or by unexpected creative behavior. Until now, however, the focus of the research is still at the basic stage, focusing mainly on determining the overall strategy in games where characters are clustered.

For example, in general, game users, ie, game players, can use various objects that can be triggered through communication with players other than themselves in accordance with the rules defined by the world in the game using valid objects in games played through game machines or computers. You will have fun.

Therefore, artificial intelligence (AI) is inevitably generated at the moment of pursuing fun, the purpose of the game. For example, traditional board games come with family, friends and guests who would like to play together, but the biggest feature that emerged when these traditional games were combined with new media such as computers or game consoles played games alone. Is that.

To this end, other AI players exist in the game elements, in addition to the game characters directly manipulated by the game players, and they communicate with the game players through interactions such as relative roles and auxiliary roles.

Such AI implementation technologies include finite state machine (FSM) and fuzzy state machine (Fusm).

FSM is a rule-based system in which finite states are connected in a graph controlled by transitions between states. FSM is widely used because it can be implemented simply by if-else or switch-case statements and is easy to understand and programmatically implement. For example, if a game character finds an enemy while on the move, it moves from the moving state to the tracking state and follows the found enemy. If the enemy is within a certain distance, it will change to attack and attack the enemy. In this way, the use of the FSM has the advantage of easy implementation and precisely defined behavior. However, the progress of the game operates only in a predefined manner.

In other words, if the opponent of the game is implemented by the FSM, since the behavior of the opponent is constant, it can be easily predicted after playing the game for a certain time. Being able to predict your opponent's behavior can be a factor in halting the interest of the game. FuSM is used to make up for this drawback.

FuSM, which incorporates fuzzy theory into FSM, can be operated at random by applying fuzzy functions to inputs and outputs. In other words, since the game contains a random element, it is difficult to predict the opponent's behavior because the opponent in the game may have different actions in the same situation.

However, FSM and FuSM can be easily implemented when the number of states of the character is small. However, when the number of states increases, it is difficult to arrange the state diagram and the program is rapidly complicated. In addition, both FSM and FuSM have the disadvantage of having to code new programs to add new behavior.

The neural network, which is being researched as a new AI implementation technology, simulates the human nervous system, has a structure in which a number of simple components are connected, and produces an output based on pattern recognition of input data. The biggest advantage of the neural network applied to the game is that since the neural network has a learning ability, the intelligence can be continuously improved as the game progresses. So many attempts have been made to apply neural networks to games. Most of them are board games such as concave and Tic-Tac-Toe. Board games have the characteristic of determining the movement of individual horses by recognizing the overall situation of the horses moving on the board.

In the AI implementation technology used for the game by the conventional technique operating as described above, in the game using a neural network, the state space of the game (physical, logical, character-by-character) is considerably large when learning game characters, and efficient learning is achieved. There was a problem that this was almost impossible.

Accordingly, the present invention is to divide the entire space of the game into logical, physical, character by any number in order to efficiently learn and improve the performance of the game character, assigning each to a neural network and then to the game character of the game character that can be learned A pattern learning apparatus and method are provided.

In addition, the present invention, by dividing the entire space of the game by an arbitrary reference, each of which is assigned to the neural network, to calculate the output node value of the neural network can be applied to the game as a probability to perform more similar to humans Provided are a pattern learning apparatus and method for a game character.

An embodiment of the present invention, the device, the storage unit for transmitting the stored game program information to the control unit, a key input unit for receiving the user key operation information according to the game situation to pass to the control unit, driving the game program, the entire game After dividing the state space into a predetermined reference area, the game device operates on the allocated neural network allocated to each neural network controlled by the neural network, and the game state input from the key input unit. And a controller configured to learn a user's hardware manipulation pattern based on the user's key manipulation information corresponding to the information, and a display configured to display a signal transmitted to the controller.

According to an embodiment of the present invention, a method of driving a game program, dividing the entire state space of a driven game into a predetermined reference region, and each neural network controlled by a neural network for each of the divided reference regions And a step of learning a hardware operation pattern of the user based on the situation information of the game driven on the assigned neural network, and the user's key manipulation information corresponding to the input situation information of the game.

In the present invention, effects obtained by representative ones of the disclosed inventions will be briefly described as follows.

The present invention can improve performance in terms of learning time and results through state space partitioning of the game, and the character performs various actions in the game by allowing the output node to be selected in proportion to the size of the output node of the neural network. You can do this, and furthermore, you can perform similar actions as game users.

Hereinafter, the operating principle of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, if it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. The following terms are defined in consideration of the functions of the present invention, and may be changed according to the intentions or customs of the user, the operator, and the like. Therefore, the definition should be based on the contents throughout this specification.

The present invention divides the entire state space of the game into logical, physical, and random characters, and allocates the divided regions to respective neural networks, thereby improving performance in terms of learning time and results. Can be. When the game character is trained with a neural network, the value of the output node of the neural network is calculated as the probability that the node is selected, and when applied to an actual game, only one type of output is selected in a specific situation, but the value of the output node. The output node can be selected in proportion to the size of. This allows the character to perform a variety of actions in the game so that they can behave more like game users.

1 is a block diagram showing the structure of a user operation pattern learning apparatus according to an embodiment of the present invention.

Referring to FIG. 1, a user manipulation pattern learning apparatus includes a control unit 100, a key input unit 108, a storage unit 110, and a display unit 112, where the control unit 100 includes a game area dividing module 102. ), The neural network 104 and the learning execution unit 106 further.

The controller 100 drives the game software stored in the storage unit 110, and transmits data required for storage during game execution to the storage unit 110 to be stored.

The key input unit 108 transmits a command signal corresponding to a key input by the user to the control unit 100. The key input unit 108 that the user manipulates is a computer such as a keyboard, a mouse, a joystick, a handle, a touch screen, or the like. It is possible to apply any device that can be used as an input means of a game device. In this case, the user recognizes the current situation of the game driven by the control unit 100 through the display unit 112 and performs game control corresponding thereto.

That is, the position control and the behavior control of the game character are performed through the key input unit 108, and the input information is transmitted to the control unit 100.

The storage unit 110 stores application programs and game programs for driving the user manipulation pattern learning apparatus, and provides the controller 100 with information stored at the request of the controller 100.

Meanwhile, the controller 100 divides the entire state space of the currently used game into a random number for each logical, physical, and character through the game region dividing module 102 in order to predict a user's behavior in the driven game. The allocated area is assigned to each neural network. For example, if the logical space is divided into 10 regions, the neural network for 10 regions is allocated from the neural network 104, and if the physical space is divided into 5 regions, 5 from the neural network 104 is allocated. The neural networks for the four regions are allocated, and if four game characters exist in the game, the neural networks for the four game characters are allocated from the neural network 104.

As shown in FIG. 2, the entire game state space may be divided by each criterion, and the neural network may be allocated to each divided region.

Thereafter, the controller 100 learns information input by the user through the key input unit 108 for each game situation. That is, each game situation is set as an input to the neural network 104, and the neural network 104 receives a key input signal transmitted from the key input unit 108 to the control unit 100 as a hardware manipulation pattern of the user according to each situation. Set to) output. In this case, the neural network 104 is implemented as a neural network that is allocated and controlled for each game partition area.

The game situation is input to the neural network 104, the process of repeatedly processing the user's input pattern according to the game situation in the neural networks for each game division area as an output, and the user's for each situation performed The operation pattern is controlled to perform learning through the learning execution unit 106.

Here, the learning execution unit 106 may use a general error backpropagation algorithm. The error backpropagation algorithm is performed by the neurons of the neural network being activated to calculate the output value, calculate the error by comparing with the desired output value, and then change the connection weight of the neurons in the direction of minimizing the error. On the other hand, the learning execution unit 106 is not limited to the error back-propagation algorithm, of course, it is possible to use all learning algorithms, such as Hebian learning rules, Boltzmann learning method that can perform the learning according to the user input pattern. .

When the controller 100 controls the game character by using the learned information, the controller 100 performs a similar behavior to that of the user. That is, for each game situation, since the neural network 104 learns the game pattern based on the neural network allocated to each game partition area, the neural network 104 outputs a value close to the key value pressed by the user at that time. This allows the user to perform an action similar to a game character controlled by the user as the game progresses, and to perform an action corresponding to the predicted action of the user, so that the user can perform an action of the repetitive game that can be halved as the user progresses the game. It is possible to continue learning and responding to patterns based on the user's patterns, thereby increasing user interest and improving performance in terms of learning time and results.

3 is a diagram illustrating a method in which a game character learns a user manipulation pattern according to an exemplary embodiment of the present invention.

Referring to FIG. 3, in the neural network 104 which allocates neural networks to game regions divided by the game region dividing unit 102, sigmoid neurons are layered and a signal is inputted from an input to an output. It is a multi-layer perceptron structure that is only delivered. In addition, there are recurrent networks or a mix of experts composed of a combination of single neural networks, and a self-organizing map used for unsupervised learning may be used, but is not limited thereto. Of course, all neural networks and neural networks that can be used on the game environment are possible.

For example, the game situation is set as an input of the neural network 104. The hardware operation of the user is output as an output. In FIG. 3, ni is the number of inputs required for learning, and no is the number of outputs.

At this time, the game situation may be input as follows. For example, if the character's position becomes important information for learning, two inputs of the character's position x and y are required, and each is input as a real value. The number of game situation inputs may vary depending on the number of data required for learning. In addition, the neural network 104 is to process the learning about the allocated neural network of the divided region for the physical location, the character.

The user's hardware operation output is set as follows. For example, in a game played by keyboard operation, the number of keyboard keys required in the game is no, and no output is required. In other words, if the game uses the a, s, d, and f keys, the number of outputs is four, and in certain game situations, the key pressed by the user is set to 1, and the remaining keys not pressed by the user set to 0. And learn through the learning execution unit 106.

However, when learning a game character with a neural network through FIG. 3, by calculating and setting the value of the output node of the neural network as a probability that the corresponding node is selected, only one type of output is selected in a specific situation when applied to an actual game situation. Rather than allowing the output node to be selected in proportion to the magnitude of the value of the output node. That is, the output node may be output considering only the value of the output node within a certain range, or the same output node is not always selected in a specific situation, and various output nodes may be output in consideration of the value of the output nodes.

After learning in this way, when the behavior of the character is controlled using the neural network 104, that is, when inputting a specific game situation, the key pressed at the time of learning outputs a value close to 1, and the remaining keys are set to 0. By outputting a close value, it is possible to determine the key pressed by the user at the time of learning, and as a result, it is possible to mimic the user's operation, thereby performing an action corresponding to the user's operation. However, since the value of the output node of the neural network is calculated and output as the probability that the corresponding node is selected, it is possible to randomly select the output node in proportion to the size of the value of the output nodes.

In this case, the hardware manipulation output of the user may apply not only a keyboard but also various input means of a computer or a game machine such as a mouse, a joystick, a button, a handle, and the like.

4 is a flowchart illustrating a procedure of learning a user manipulation pattern by a game character according to an exemplary embodiment of the present invention.

Referring to FIG. 4, in step 400, the controller 100 drives a game program stored in the storage 110, and in step 402, the entire state space in the game in which the game area divider 102 in the controller 100 is driven. The area division is performed based on arbitrary criteria. That is, after dividing the entire state space of the game into any number on the basis of logical, physical, character, etc., the neural network 104 allocates neural networks to each divided region in step 404.

As a result, the entire state space of the game is divided into regions, and different neural networks are allocated to the divided regions.

Thereafter, in step 406, game situation information is set as an input to the neural network 104. In this case, the game situation information may be information about a game character, and may be location information of a game character or information for controlling the action of the game character.

In operation 408, the control unit 100 receives a signal input by a user's hardware operation, that is, a key input signal from the key input unit 108 according to the situation of the game character, and transmits the received key input signal to the neural network 104. ) And the output is calculated by the probability that the node is selected.

In this way, the neural network 104 to which different neural networks are assigned for each game area sets the current state of the game as an input, sets the user's hardware operation information as the output, and the user's hardware operated in the current state of the game. The operation information is learned using the learning execution unit 106. Accordingly, in step 410, the hardware manipulation pattern of the user is set as the output of the neural network 104 for each game situation and each game area set in the neural network 104, and the learning is performed based on the information learned in step 412. By controlling the behavior of the character using the neural network 104, it can be seen that the control is somewhat similar to the user's manipulation for each game situation.

Through this, as the user progresses the game, the controller 100 learns a user's manipulation pattern for each divided state space of the game, and controls the behavior of the game character through the neural network 104 based on the learned pattern. Since the other party's character's behavior may be different depending on the user's predicted behavior.

In other words, considering the output node value of the neural network as a probability, since the same output node is not always selected in a specific situation, and various output nodes are possible in consideration of the output node values, the game user cannot predict the opponent's behavior. Even if the user behaves differently according to the opponent's actions, this is also learned, so that the opponent character performs another action later in the same game situation.

As described above, the present invention divides the entire space of the game into logical, physical, and random numbers for each character in order to efficiently learn and improve the performance of the game character, assigns each to a neural network, and then learns the game character. In order to behave like a person, the output node value of the neural network is calculated as a probability and applied to the game.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments, but is capable of various modifications within the scope of the invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the scope of the following claims, but also by those equivalent to the scope of the claims.

1 is a block diagram showing the structure of a user operation pattern learning apparatus according to an embodiment of the present invention;

2 is a diagram showing a state space of an entire game to which a neural network is allocated according to an embodiment of the present invention;

3 is a diagram illustrating a method in which a game character learns a user manipulation pattern according to an embodiment of the present invention;

4 is a flow chart illustrating a procedure for a game character to learn a user manipulation pattern according to an embodiment of the present invention.

<Description of Signs of Major Parts of Drawings>

100: control unit 102: game area dividing unit

104: neural network 106: learning execution unit

108: key input unit 110: storage unit

 112: display unit

Claims (10)

A storage unit for transmitting the stored game program information to the control unit; A key input unit which receives user key operation information according to a game situation and delivers it to a control unit; Driving the game program, dividing the entire state space of the game into a predetermined reference region, and then assigning each neural network controlled by the neural network for each divided region and the situation information of the game driven on the allocated neural network; The controller for learning a hardware operation pattern of the user based on the key operation information of the user corresponding to the situation information of the game inputted from the key input unit; A display unit for displaying the signal transmitted to the control unit, The control unit may include: a game area dividing unit dividing the entire state space of the game by any number of physical, logical, and character characters; The neural network is assigned to all or a part of the divided regions by the game region dividing unit, and the situation information of the driven game is set as an input, and the key input information of the user corresponding to the situation information of the game is input. The neural network configured to set this as an output when received from Learning execution unit for learning the user's hardware operation pattern input according to the game situation information Pattern learning apparatus of a game character using a space partition comprising a. The method of claim 1, The neural network, Multilayer perceptron structure in which sigmoid neurons are layered and signals are transmitted in one direction only from input to output, A mix of experts consisting of recurrent networks or a combination of single neural networks, Apparatus for pattern learning of game characters using spatial segmentation, characterized in that any one of the self-organizing map used for supervised learning. 3. The method of claim 2, The neural network, The pattern learning apparatus of a game character using space division, wherein the value of the output nodes is set as a probability that the corresponding node is output and randomly selected in proportion to the size of the value of the output nodes in a specific game situation. The method of claim 3, The neural network, The pattern learning apparatus of a game character using space division, characterized in that only the value of the output node within a predetermined range of the value of the output nodes selected. The method of claim 1, The game status information, The pattern learning apparatus of the game character using the space division, characterized in that the position of the game character or the behavior control information of the game character. Driving a game program in a controller in a pattern learning device of a game character, Dividing the entire state space of the driven game into any number of physical, logical and character units; Allocating to each neural network controlled by the neural network for each of the divided reference regions; Learning the hardware operation pattern of the user based on the situation information of the game driven on the assigned neural network and the key operation information of the user corresponding to the input situation information of the game; The learning of the hardware manipulation pattern of the user may include setting the state information of the driven game as an input to the neural network; If the user's hardware operation corresponding to the situation information of the game is received, the output of the neural network comprising the step of calculating and setting the value of the output nodes as a probability that the node is output; Is a pattern learning method of game characters using space division. delete The method of claim 6, The neural network, Multilayer perceptron structure in which sigmoid neurons are layered and signals are transmitted in one direction only from input to output, A mix of experts consisting of recurrent networks or a combination of single neural networks, A method of learning a pattern of a game character using space division, characterized in that it is any one of a self-organizing map used for unsupervised learning. The method of claim 8, The neural network, By setting a value for the output nodes of the neural network as a probability, randomly selected in proportion to the size of the value of the output nodes in a specific game situation, or selecting only the value of the output node within a certain range of the value of the output nodes Pattern learning method of the game character using the spatial partition, characterized in that. The method of claim 6, The game status information, A method of learning a pattern of a game character using space division, characterized in that the position of the game character or the behavior control information of the game character.

Images