JP2020115957A - Information processing device, information processing program, and information processing method - Google Patents

Abstract

To exhibit appropriate strength in an artificial intelligence agent that plays an electronic game using a plurality of objects.SOLUTION: An information processing device includes: strategy evaluation means that has carried out machine learning for evaluating a strategy in an electronic game for use of an object as an evaluation value in the electronic game using a plurality of objects; and strategy determination means that obtains an evaluation value output result for use of the object applicable in the electronic game by using the strategy evaluation means, normalizes cumulative values obtained by arranging the evaluation values in an ascending order or a descending order and totaling them in a predetermined range, and determines use of the object actually applied to the electronic game from uses of objects corresponding to the evaluation value corresponding to a reference range in the normalized cumulative values.SELECTED DRAWING: Figure 5

Description

The present invention relates to an information processing device, an information processing program, and an information processing method, and more particularly to an information processing device, an information processing program, and an information processing method for providing an electronic game.

Conventionally, an information processing device that provides an electronic game is known. As an electronic game provided by such an information processing apparatus, an electronic game using a plurality of objects (in-game objects) such as characters and cards is known. As an example thereof, there is a competitive game in which a player uses a plurality of objects and fights against a competitor who is a computer or another player.

On the other hand, in recent years, machine learning has been actively researched. For example, research on deep learning using a multilayer neural network has been conducted. By using such deep learning, an AI agent that plays an electronic game (AI; Artificial Intelligence) agent is learned, and an AI agent having higher performance than before (that is, the game is “good”) is realized. For example, Non-Patent Document 1 discloses a technique related to a deep neural network applied to an AI agent that plays Go.

David Silver, et al, “Mastering the Game of Go with Deep Neural Networks and Tree Search” Nature 529,484-489,(28 January 2016)

In a competitive game, it is desirable that the strength can be set to an appropriate level so that the user (player) can enjoy the game as a virtual opponent (NPC) based on the evaluation value obtained by AI or the like. .. For example, if the user (player) is a beginner, the NPC may lose interest in the electronic game if the NPC is too strong. On the other hand, a highly skilled user (player) may not be satisfied with the electronic game if the NPC is too weak.

It is an object of the present invention to provide a technique capable of exerting appropriate strength in an artificial intelligence agent playing an electronic game using a plurality of objects.

One aspect of the present invention comprises a strategy evaluation unit that performs machine learning for evaluating an electronic game strategy for use of the object as an evaluation value in an electronic game using a plurality of objects, and the strategy evaluation unit. Obtaining the output result of the evaluation value for the use of the object that is applicable in the electronic game by using the evaluation value corresponding to the evaluation value corresponding to the reference range in the cumulative value obtained by arranging the evaluation values in ascending or descending order. An information processing apparatus comprising: a strategy determining unit that determines the use of the object that is actually applied to the electronic game from the use.

Here, the strategy determining means normalizes the cumulative value to a predetermined range, and actually applies to the electronic game from the use of the object corresponding to the evaluation value corresponding to the reference range in the normalized cumulative value. It is preferable to determine the use of said object.

In addition, the strategy determination means normalizes the reference range to a predetermined range, and actually applies to the electronic game from the use of the object corresponding to the evaluation value corresponding to the normalized reference range in the cumulative value. It is preferable to determine the use of said object.

In addition, it is preferable that the strategy evaluation unit performs machine learning for evaluating a strategy on the electronic game for use of the object as an evaluation value by using a past play history in the electronic game.

Further, it is preferable that the strategy determining means calculates the cumulative value by using all the evaluation values for use of the object applicable in the electronic game using the strategy evaluating means.

Further, it is preferable that the strategy determining means calculates the cumulative value by using only a part of the evaluation value for use of the object applicable in the electronic game by using the strategy evaluating means.

Further, it is preferable that the setting of the reference range can be changed according to the degree of difficulty of the electronic game.

Further, it is preferable that the width of the reference range can be changed.

Further, it is preferable that the reference range is set and changed according to the elapsed time of playing the electronic game.

Further, it is preferable that a plurality of the reference ranges are set and a use strategy of the electronic game by a plurality of players is determined by determining the use of the object for each of the reference ranges.

Further, the strategy determining means determines the use of the object corresponding to the evaluation value selected according to the uniform probability distribution from the evaluation values corresponding to the reference range as the use of the object to be actually applied to the electronic game. Is preferred.

Further, the strategy determining means determines the use of the object corresponding to the evaluation value selected according to a predetermined probability distribution from the evaluation values corresponding to the reference range as the use of the object to be actually applied to the electronic game. Is preferred. Here, it is preferable that the probability distribution is a normal distribution having the maximum value of the probability distribution at the maximum value or the minimum value of the reference range.

Another aspect of the present invention is a strategy evaluation means for causing a computer to perform machine learning for evaluating an electronic game strategy for use of an object as an evaluation value in an electronic game using a plurality of objects, and the strategy. The evaluation value output result of the evaluation value for the use of the object applicable to the electronic game is obtained using the evaluation means, and the evaluation value corresponding to the evaluation value corresponding to the reference range in the cumulative value obtained by arranging the evaluation values in ascending order or descending order. It is an information processing program characterized by causing it to function as strategy determining means for determining the use of the object to be actually applied to an electronic game from the use of the object.

Another aspect of the present invention is, in an electronic game using a plurality of objects, a strategy learning step of performing machine learning for evaluating a strategy on an electronic game for use of the object as an evaluation value, and learning in the strategy learning step. The output value of the evaluation value for the use of the object that is applicable in the electronic game is obtained by using the selected agent, and the evaluation value corresponding to the reference range is accumulated in the cumulative value obtained by arranging the evaluation value in ascending or descending order. A strategy determining step of determining the use of the object to be actually applied to the electronic game from the use of the object to perform the information processing method.

According to the present invention, it is possible to provide an information processing device and an information processing program capable of exhibiting appropriate strength in an artificial intelligence agent playing an electronic game using a plurality of objects.

FIG. 1 is a schematic configuration diagram of a game system according to an embodiment of the present invention. It is a figure which shows the example of the game screen of the game which the game system in embodiment of this invention provides. 1 is a schematic configuration diagram of a planner terminal according to an embodiment of the present invention. It is a schematic diagram of a configuration of a player terminal in the embodiment of the present invention. It is a structure schematic diagram of the game server in the embodiment of the present invention. It is a figure which shows the content example of character DB. It is a figure showing the example of contents of player log DB. FIG. 3 is a schematic configuration diagram of an AI battle processing unit in the embodiment of the present invention. It is a figure which shows the relationship between the use of a character and the evaluation value in the embodiment of the invention. It is a figure for demonstrating the normalization process with respect to the cumulative value of the evaluation value in embodiment of this invention. It is a figure which shows the example of setting of the reference range in embodiment of this invention. It is a figure for explaining selection processing of use of a character in an embodiment of the invention. It is a figure which shows the example of setting of the reference range in embodiment of this invention. It is a figure which shows the example of setting of the reference range in embodiment of this invention. It is a figure for explaining selection processing of use of a character in an embodiment of the invention. It is a figure for explaining selection processing of use of a character in an embodiment of the invention. It is a schematic block diagram of the AI battle process part in the modification of this invention. It is a figure which shows the example of setting of the reference range in the modification of this invention.

Hereinafter, embodiments of the present invention will be described.

FIG. 1 shows a schematic configuration diagram of a game system 10 according to the present embodiment. The game system 10 is a developer of an electronic game (hereinafter simply referred to as “main game”) provided by the game system 10, more specifically, a planner who is an creator of an object (character) used in the present game. The system includes a planner terminal 12 to be used, a player terminal 14 to be used by a player of this game, and a game server 16 as an information processing device. The planner terminal 12 and the game server 16 and the player terminal 14 and the game server 16 are communicably connected to each other via a communication line 18. The communication line 18 is composed of, for example, the Internet or a local area network (LAN). Although FIG. 1 shows one planner terminal 12 and one player terminal 14, respectively, a plurality of these may be provided.

In the game system 10, the planner can access the game server 16 using the planner terminal 12 to develop the main game including object creation. Further, when the user (player) uses the player terminal 14 to access the game server 16, the user can play an electronic game against an NPC (Non Player Character). Further, in the game server 16, NPCs (Non Player Characters) can be played against each other.

Here, the contents of this game will be described. FIG. 2 shows an example of the game screen displayed on the player terminal 14. In this game, a plurality of characters as a plurality of objects are used to compete with an enemy player. The enemy player may be a player other than the own player, or a computer (game server 16). Specifically, the player and the enemy player alternately arrange characters on the board 20 defining a plurality of squares as a game field in the manner of Othello (registered trademark). In FIG. 2, the rows of the cells are shown by the numbers “1” to “6” and the columns are shown by the letters “A” to “F”.

The white circles on the board 20 indicate the player's stone (white stone), and the black circles indicate the enemy player's stone (black stone). A character name (for example, “C0055”) written in a white circle on the board 20 indicates the player's character (own character), and a character name written in a black circle indicates the enemy player's character. Indicates a character (enemy character). When the enemy character or the black stone is sandwiched between the two own characters, or the one white stone and the one own character, the enemy character or the black stone can be changed (turned over) to the white stone. Further, when the own character or white stone is sandwiched between two enemy characters, or one black stone and one enemy character, the own character or white stone is changed to black stone. Note that, like the ordinary Othello, two white stones and two black stones may be placed on the four central cells of the board 20 at the start of the game. That is, white stones may be placed in the 3C squares and 4D squares, and black stones may be placed in the 3D squares and 4C squares.

The player selects a plurality of characters from among the plurality of characters defined in the present game and obtained by the player (made available in the present game). For example, in this embodiment, 16 characters are selected. A group of 16 characters selected by the player is called a deck. The player can acquire a new character according to the progress of the game or by charging. As shown in FIG. 2, four characters selected from the deck are displayed on the screen. The four characters are called the hand piece 22. The hand piece 22 may be randomly selected from the deck by the game server 16 or may be selectable from the deck by the player. The user selects an arbitrary character from the four hand pieces 22 and arranges the selected character in a square on the board 20. When one character among the hand pieces 22 is placed on the board 20, one character selected from the deck is newly added to the hand piece 22.

Although details will be described later, a plurality of parameters are set for each character. For example, parameters such as offensive power, skill effect, and skill activation condition are set. When the character is placed on the board 20, damage can be given to the opponent according to the attack power of the character. Alternatively, when the condition for activating the skill of the character is satisfied, a skill (skill) as a special effect of the character is executed, and a game effect corresponding to the skill effect (or skill effect and attack power) is activated. ..

HPs (hit points) are set for the own player and the enemy player, respectively. In FIG. 2, it is shown that the maximum HP of the player is 250 and the current HP is 180. The HP of the opponent can be reduced by arranging the character on the board 20 or by executing the skill. Further, the HP may be able to be restored by the skill of the character. The player who loses HP first loses, and the other wins.

As described above, the game provided by the game system 10 is a game in which the player plays against the opponent while using the plurality of characters.

FIG. 3 shows a schematic configuration diagram of the planner terminal 12. The planner terminal 12 may be, for example, a general computer.

The control unit 12a is configured to include, for example, a CPU (Central Processing Unit) or a microcomputer, and controls each unit of the planner terminal 12 according to a program stored in a storage unit 12e described later. The communication unit 12b is configured to include, for example, a network adapter, and has a function of communicating with the game server 16 via the communication line 18. The input unit 12c includes, for example, a mouse, a keyboard, a touch panel, or the like. The input unit 12c is for inputting a planner instruction to the planner terminal 12. The display unit 12d is configured to include, for example, a liquid crystal display, and displays various screens. In particular, a game development screen or the like is displayed on the display unit 12d. The storage unit 12e includes, for example, a hard disk, a RAM (Random Access Memory), a ROM (Read Only Memory), or the like. The storage unit 12e stores a program for operating each unit of the planner terminal 12. Alternatively, the storage unit 12e stores various control data or various processing result data.

FIG. 4 shows a schematic configuration diagram of the player terminal 14. In the present embodiment, a portable terminal such as a smartphone or a tablet terminal is used as the player terminal 14, but the player terminal 14 may be a stationary computer.

The control unit 14a includes, for example, a CPU or a microcomputer, and controls each unit of the player terminal 14 in accordance with a game client program stored in a storage unit 14e described later. The communication unit 14b includes, for example, a network adapter and the like, and has a function of communicating with the game server 16 via the communication line 18. The input unit 14c is configured to include, for example, a touch panel or buttons. The input unit 14c is for inputting a player's instruction to the player terminal 14. The display unit 14d includes, for example, a liquid crystal display, and displays various screens. In particular, a game screen or the like is displayed on the display unit 14d. The storage unit 14e is configured to include, for example, RAM or ROM. The storage unit 14e stores a game client program. Alternatively, the storage unit 14e stores various control data, various game processing result data, and the like.

FIG. 5 shows a schematic configuration diagram of the game server 16.

The communication unit 30 is configured to include, for example, a network adapter and has a function of communicating with the planner terminal 12 and the player terminal 14 via the communication line 18. The storage unit 32 includes, for example, a hard disk, a RAM, a ROM, or the like. The storage unit 32 stores a game host program as an information processing program. By executing the host program, the game proceeds and each unit of the game server 16 operates. In addition, as shown in FIG. 5, the storage unit 32 stores a character DB 34, a player log DB 36, a distributed expression DB 38, an AI log DB 40, an AI agent 42, and a statistical data group 44 including a plurality of statistical data 44a. It

The character DB 34 is a database that is defined in this game (that is, by the host program) and stores data relating to characters that can be used by the player or the enemy player in the game. Specifically, the character DB 34 stores each parameter set for each character. In this game, it is possible to use more than one thousand and hundreds of characters, and new characters are added one after another.

FIG. 6 shows an example of the contents of the character DB 34. Although the character DB 34 is shown in a table format in FIG. 6, the data format of the character DB 34 is not limited to this. In FIG. 6, one record corresponds to one character. In the present embodiment, each character has a character name, an attribute indicating the characteristic of the character, a rarity degree indicating the difficulty level of acquisition of the character, and a criterion of damage to the opponent when the character is placed on the board 20. Has an offensive power, a skill name, a skill activation condition indicating a condition for the skill of the character to be activated, and a skill effect parameter which is an effect when the skill is activated. The “normal attack” in the skill effect column means that the character is placed on the board 20 and damages the opponent based on the attack power value without activating the skill.

These parameters can be defined by the planner. Of course, the items of parameters are not limited to these, and each character may have parameters for other items.

As shown in FIG. 6, in the present embodiment, the skill activation condition of each character is, for example, "HP is 5% or less of the maximum HP" or "there are three or more own characters with the attribute P1 on the board". It is stipulated in the conditions regarding the game situation. Therefore, as the player, since the skill activation condition of each character is defined by the condition relating to the game situation, it is difficult to properly select the character to be included in the deck. That is, as a player, it is not easy to understand the probability of the character activating a skill by looking at the skill activating condition of the character.

Returning to FIG. 5, various data is stored in the player log DB 36, the distributed expression DB 38, and the AI log DB 40, respectively. Details of the data stored in these databases will be described later.

The AI agent 42 is an artificial intelligence program that plays this game. The AI agent 42 functions as a strategy evaluation means. The AI agent 42 is learned by a method using deep learning described later. Details of the learning method of the AI agent 42 will be described later. The AI agent 42 in this embodiment is configured to include a multilayer neural network. Since the multilayer neural network is defined by various parameters (layer structure, neuron structure in each layer, weight or bias of each neuron, etc.), the storage unit 32 stores these parameters.

The statistical data 44a is statistical data related to the use result of the character. In this embodiment, the statistical data 44a is generated for each character, and a plurality of statistical data 44a corresponding to each character is the statistical data group 44. The statistical data 44a is generated by a statistical data generation unit 64, which will be described later, and its details will be described later.

The control unit 50 includes, for example, a CPU, a GPU (Graphics Processing Unit), a microcomputer, or the like, and controls each unit of the game server 16 according to the game host program stored in the storage unit 32. Further, as shown in FIG. 5, the control unit 50 causes the player log collection unit 52, the expression learning unit 54, the supervised learning unit 56, the AI competition processing unit 58, the reinforcement learning unit 60, and the AI log collection unit according to the host program. 62, the statistical data generation unit 64, and the display control unit 66 also function. Hereinafter, these functions of the control unit 50 will be described.

The player log collecting unit 52 acquires a player log, which is data indicating a game progress history of the player regarding this game, and stores the player log in the player log DB 36. The player log collecting unit 52 acquires the player log when the game play by the player is completed or during the game play. Incidentally, the player referred to here may include the developer of the present game, and the player log collecting unit 52 may acquire the log regarding the test play by the developer. In any case, the player log is a log related to actual game play by a human.

FIG. 7 shows an example of the contents of the player log DB 36. In FIG. 7, the player log DB 36 is also shown in a table format, but the data format of the player log DB 36 is not limited to this. As described above, in the present game, the first player and the second player alternately arrange the characters on the board 20, so that the player log DB 36 indicates the “number of turns” and the first player at the turn. Alternatively, a "placement character" indicating the character placed on the board 20 by the second player, a "placement place" indicating the place where the character is placed on the turn, and the number of the opponent's stones or enemy characters turned over in the turn. "The number of flips", "Skill activation" indicating whether the skill of the character placed in the turn is activated, and "Skill effect" indicating the effect (for example, damage amount) caused by the skill activation. Including the "Value" item. Of course, the items shown in FIG. 7 are examples, and the player log may include other items. For example, items such as the amount of actual damage given to the opponent by the character's normal attack may be included.

As described above, in the player log, the use result (“the number of flipped sheets”, “skill activation”, or “use” when the first player or the second player uses each character (that is, the character is placed on the board 20 ). Skill effect value”) is included.

The expression learning unit 54 performs expression learning based on the player log as a game log stored in the player log DB 36. Thereby, the feature vector representing the feature of each character registered in the character DB 34 is learned. That is, the expression learning unit 54 learns the distributed expression of each character used in this game. The feature vector (distributed expression) of each character learned by the expression learning unit 54 is stored in the distributed expression DB 38. In addition, the expression learning unit 54 uses the AI log as a game log, which indicates a game progress history of the AI agent 42 by the AI competition processing unit 58, which will be described later, instead of the player log or together with the player log. You may go.

The feature vector according to the present embodiment is, for example, a five-dimensional vector, and each element indicates a strategic feature in the present game (for example, "I want to put it in a corner", "I want to use it in the beginning", etc.). ..

The expression learning unit 54 performs expression learning by using a learning device for expression learning, which is composed of, for example, a neural network. Specifically, based on the game state (such as the stones on the board 20 or the arrangement state of the characters) before the use of a certain character in the player log, and the result (the next state) of the use of the character. , The value of each element of the feature vector of the character is learned.

In this way, by representing each character in a distributed manner, it becomes possible to represent a large number of characters by a lower dimensional vector. Characters having similar strategic features in this game are represented by similar feature vectors. Note that various known learning methods can be used as the expression learning method.

The supervised learning unit 56 causes the AI agent 42 to machine-learn a strategy regarding the use of each character by using at least the player log stored in the player log DB 36 and the feature vector of each character learned by the expression learning unit 54. .. The supervised learning unit 56 functions as a strategy learning unit together with a reinforcement learning unit 60 described later.

Here, the strategy regarding the use of each character is a strategy including what characteristics the character should use in a certain game state, and which position on the board 20 the character should be placed.

Specifically, the supervised learning unit 56, the game state such as the feature vector of the character existing on the board surface or in the hand piece at a certain game point, the arrangement position, the number of turns, the actionable hand, the hit point, and the action information (specification The character is placed in a square) or the like as an input, and the AI agent 42 is learned using the learning data in which the evaluation value of the input action information in the input game state is output. The evaluation value is calculated in advance based on the player log. That is, the supervised learning unit 56 causes the AI agent 42 to learn by supervised learning.

In the learning with the supervised learning unit 56, since each character is represented by a feature vector and then the learning is performed, the amount of calculation in the learning can be reduced as compared with the case where each character is individually labeled. In addition, since characters having similar features are represented by similar feature vectors, learning about a certain character means learning about a character similar to the character. That is, the effect of generalization of learning can be expected.

The AI battle processing unit 58 uses the AI agent 42 learned by the supervised learning unit 56 or further learned by the reinforcement learning unit 60 described later to perform the battle process. That is, the AI battle processing unit 58 functions as a strategy determining unit for an electronic game using the AI learning agent 42 learned by the supervised learning unit 56 that is a strategy learning unit and the reinforcement learning unit 60 described later.

The AI battle processing section 58 performs a battle process between the AI agent 42 and the user (player). Further, the AI battle processing section 58 may perform a process of causing the AI agents 42 to battle each other. In the battle, the AI agent 42 may be left to select the character to be included in the deck. The battle process in the AI battle processing unit 58 will be described later.

The reinforcement learning unit 60 performs reinforcement learning, which is learning for strengthening the AI agent 42. Reinforcement learning is a method different from supervised learning, and for example, Q learning can be used as reinforcement learning. Specifically, in the reinforcement learning, the AI agent 42 is rewarded based on the result of a series of actions (for example, winning or losing) selected by the AI agent 42 from a certain game state. Learn what to do based on your reward.

In the present embodiment, the reinforcement learning unit 60 performs learning based on the battle result by the AI battle processing unit 58. The reinforcement learning unit 60 may perform the reinforcement learning based on the player log stored in the player log DB 36. In the reinforcement learning, the reinforcement learning unit 60 may perform the learning based on the result of the battle between the AI agent 42 and another NPC (Non Player Character; rule-based AI that is not the AI agent 42).

In this game, the action that the AI agent 42 can take differs depending on the game state. For example, the position where the character can be arranged differs depending on the state of the board 20, and the character that can be used differs depending on the type of the hand piece 22. Therefore, in the present embodiment, reinforcement learning is performed by inputting both a certain game state and the action that the AI agent 42 can take and outputting an index indicating the value of the input action.

In the reinforcement learning as well, it is preferable to perform learning while using the feature vector of each character learned by the expression learning unit 54. As a result, the same effect as that in the supervised learning unit 56 can be obtained in the reinforcement learning.

In the present embodiment, at least one of the supervised learning unit 56 and the reinforcement learning unit 60 corresponds to strategy learning means. That is, the strategy learning that causes the AI agent 42 to learn the strategy regarding the use of each character is either or both of supervised learning and reinforcement learning.

The AI battle processing unit 58 may perform the battle process based on the AI agent 42 that has undergone the reinforcement learning in the reinforcement learning unit 60. In this case, the battle process between the AI agent 42 and the user (player) further learned by the reinforcement learning unit 60 is performed. Further, the AI battle processing section 58 may perform a process of causing the AI agents 42 to battle each other. In the battle, the AI agent 42 may be left to select the character to be included in the deck. The battle process based on the evaluation value in the AI battle processing unit 58 will be described later.

The AI log collecting unit 62 indicates the game progress history of the main game in the AI battle processing unit 58, acquires an AI log that is data including the use results of a plurality of characters, and stores it in the AI log DB 40. Since the AI log has the same items as the player log, a detailed description of the AI log is omitted here. The difference between the player log and the AI log is only to show the history of game progress by the player, or at least to show the history of game progress when one of the players is the AI agent 42. It should be noted that the AI log collection unit 62 acquires not only the log of the battle between the AI agent 42 and the user (player) but also the log of the battle between the AI agents 42 and the battle between the AI agent 42 and the NPC as an AI log. You may

The statistical data generation unit 64 generates the statistical data 44a based on the AI log stored in the AI log DB 40. As described above, the statistical data 44a is generated according to each character, and relates to the use result of each character. For example, the statistical data 44a may include a winning rate when a character is used. In this case, the statistical data generation unit 64 extracts from the AI log whether or not the attention character and the AI agent 42 used when the attention character is used wins, and calculates the winning rate for the attention character. ..

Further, the statistical data 44a may include the probability of activating a skill when a character is used. In this case, the statistical data generation unit 64 extracts the attention character and whether or not the skill is activated when the attention character is used from the AI log, and calculates the skill activation probability regarding the attention character.

In addition, the statistical data 44a may include a distribution of effects that occur when a character is used. In this case, the statistical data generation unit 64 extracts the attention character and the skill effect value when the attention character activates the skill from the AI log, and calculates the distribution of the skill effect value regarding the attention character. Alternatively, the effect produced when the character is used may include the amount of damage given to the opponent when the character makes a normal attack, that is, the statistical data generation unit 64 selects from the AI log. The distribution of the damage of the normal attack of the target character is calculated by extracting the target character and the damage caused to the opponent by the normal attack of the target character.

As described above, in the present embodiment, the statistical data 44a includes at least one of the winning rate when the character is used, the probability of activating a skill when the character is used, and the distribution of the effect that occurs when the character is used. It includes one. Of course, the statistical data 44a may include other data.

The statistical data generation unit 64 generates the statistical data 44a for a plurality of characters registered in the character DB 34, ideally for all characters, by the above-described processing. As a result, the statistical data group 44 is formed.

The display control unit 66 displays the statistical data 44a generated by the statistical data generation unit 64 on the display unit 12d of the planner terminal 12 according to the request from the planner terminal 12 (that is, from the planner). The display control unit 66 also causes the display unit 14d of the player terminal 14 to display the statistical data 44a generated by the statistical data generation unit 64 in accordance with a request from the player terminal 14 (that is, from the player). The display control unit 66 does not display the statistical data 44a regarding the character under development on the display unit 14d of the player terminal 14.

[AI battle processing section]
Hereinafter, the battle process in the AI battle processing unit 58 will be described. As described above, the AI battle processing unit 58 uses the AI agent 42 learned by the supervised learning unit 56 and the reinforcement learning unit 60 to perform the battle process between the AI agent 42, the user (player), and the AI agents 42. ..

As shown in FIG. 8, the AI battle processing unit 58 includes an evaluation value calculation unit 58a, an evaluation value normalization unit 58b, a reference range setting unit 58c, and a processing determination unit 58d.

The evaluation value calculation unit 58a applies the AI agent 42 learned by the supervised learning unit 56 or the reinforcement learning unit 60 to obtain an evaluation value for a character currently usable in an electronic game.

For example, in a case where four characters (characters A to D) called a hand piece 22 out of the 16 characters included in the deck can be arranged in any of the four squares (squares B4, B5, D2, F3), As shown in FIG. 9, there are 16 variations in the use of the character, which are arranged in four squares for each of the four characters. Therefore, the evaluation value is obtained using the AI agent 42 for each of these 16 ways of using the character. In FIG. 9, for example, the character A is placed on the square B4 as the use (1) of the character, and the evaluation value is 20, and the character A is placed on the square B5 as the use (2) of the character. The evaluation value is calculated using the AI agent 42 for each use of the character such that the evaluation value is 15...

The evaluation value normalization unit 58b performs a process of normalizing the evaluation value. The evaluation value normalization unit 58b arranges and accumulates the plurality of evaluation values obtained by the evaluation value calculation unit 58a in ascending order, and performs normalization processing so that the accumulated value falls within a predetermined numerical range.

To simplify the description, an example in which the evaluation value calculation unit 58a obtains five evaluation values for five variations of character use (character use (1) to (5)) will be described. As shown in FIG. 10, the evaluation value normalization unit 58b arranges the five evaluation values in ascending order (ascending order) to obtain a cumulative value, and normalizes the cumulative value to fall within the numerical range of 0 to 100. As a result, a cumulative value obtained by normalizing the evaluation value obtained by the evaluation value calculation unit 58a is obtained.

The numerical value range for normalization in the evaluation value normalization unit 58b is not limited to 0 to 100, and may be any value that can normalize the cumulative value of a plurality of evaluation values to a fixed value. For example, the numerical range may be a range of 0 to 1, 0 to 10 and the like.

The evaluation values to be accumulated in the evaluation value normalization unit 58b may be all evaluation values obtained in the evaluation value calculation unit 58a, or may be some evaluation values. Good. For example, when it is desired to determine the use of the character actually applied to the electronic game in consideration of the use of all selectable characters in the electronic game, all evaluation values obtained by the evaluation value calculation unit 58a are accumulated. To calculate a normalized cumulative value. Further, for example, when it is desired to avoid using a character whose evaluation value is less than a predetermined reference value in an electronic game and decide to use a character that is actually applied to an electronic game, only the evaluation values above the reference value are accumulated. Alternatively, the normalized cumulative value may be calculated.

The reference range setting unit 58c sets a reference range for selecting a process for the cumulative value normalized by the evaluation value normalization unit 58b. The reference range is set within the numerical range of the normalized cumulative value in the evaluation value normalization section 58b. The AI agent 42 can select only the use of the character corresponding to the evaluation value included in the reference range in the normalized cumulative value. That is, the reference range determines the range of use of the character that the AI agent 42 can select in the electronic game.

For example, as shown in FIG. 11, when the process of normalizing the cumulative value of the evaluation values into the numerical range of 0 to 100 is performed, the reference range is set to 40 to 80 within the range of 0 to 100. In this case, the AI agent 42 uses the character corresponding to the evaluation value included in the reference range 40 to 80 (1: arranges the character A in the cell B4), (5: arranges the character B in the cell B4), ( (3: Character A is placed on the cell D2) can be determined as the use of the character to which the character is actually applied.

The process determination unit 58d determines the process actually applied in the electronic game. The determined process is, for example, a process of deciding which character is placed in which square in the electronic game. The process determination unit 58d determines the process corresponding to the evaluation value corresponding to the reference range set by the reference range setting unit 58c as the process applied to the electronic game. Specifically, a selection value is selected from the reference range set by the reference range setting unit 58c according to a predetermined probability distribution, and the use of the character corresponding to the evaluation value corresponding to the selection value is actually applied. To use.

For example, the selection value is determined based on a uniform probability distribution over the entire range of the reference range for the cumulative value normalized for the evaluation value. When the reference range for the normalized cumulative value is set to 40 to 80, as shown in FIG. 12, the selection value is determined based on a uniform probability distribution over the entire reference range. Then, the use of the character corresponding to the evaluation value corresponding to the selected value in the normalized cumulative value (5: character B is placed in the square B4) is selected as the use of the character actually applied to the electronic game. That is, the selection value is randomly determined from the entire range of the reference range, and the use of the character corresponding to the selection value is selected as the use of the character actually applied to the electronic game.

As in the present embodiment, by setting the reference range for the normalized cumulative value, the use of the character is selected from a plurality of candidates that fall within the reference range. The strength of the AI agent 42 can be adjusted.

Here, the setting of the reference range may be changed according to the difficulty level of the electronic game. For example, as shown in FIG. 13, when the opponent user (player) is a beginner, the reference range is set to 0 or more and less than 30, and when the user is an intermediate player, the reference range is set to 30 or more and less than 60. However, the reference range may be set to 60 or more and less than 90 for advanced users, and the reference range may be set to 90 or more and 100 or less for advanced users. As a result, for beginners, the use of the character corresponding to the relatively low evaluation value of the evaluation values included in the normalized cumulative value is selected, and the higher the evaluation value, the higher the evaluation value. The use of the character to play can be selected.

Specifically, by receiving an input of a level (difficulty level) desired by the user (player) from the player terminal 14 at the time of starting the play of the electronic game, and transmitting the input result to the game server 16, The game server 16 may set a reference range according to the level. Further, the reference range may be set based on the statistical data 44a obtained by the statistical data generation unit 64. For example, the reference range may be set according to the past winning rate in the battle between the AI agent 42 and the user (player).

Further, as shown in FIG. 14, the width of the reference range may be changed. The width of the reference range corresponds to the selection width of the use of the character that can be selected by the AI agent 42. Therefore, by expanding the width of the reference range, the variation in the use of the character that the AI agent 42 applies each time also expands, and the strength of the AI agent 42 can be changed. On the other hand, by narrowing the width of the reference range, the variation in the use of the character that the AI agent 42 applies each time becomes small, and the AI agent 42 can be stabilized to a constant strength. It should be noted that one reference value may be set instead of the reference range so that the strength of the AI agent 42 may be maintained completely constant.

Specifically, when the user (player) receives an input of stability of desired strength when playing an electronic game using the player terminal 14 when starting to play an electronic game, the input result is displayed. By transmitting to the game server 16, the width of the reference range may be set in the game server 16 according to the stability. Further, the width of the reference range may be set based on the statistical data 44a obtained by the statistical data generation unit 64. For example, the width of the reference range may be set according to the change in the past winning rate in the battle between the AI agent 42 and the user (player).

Further, the reference range may be changed according to the elapsed time of playing the electronic game. For example, it is preferable to set the reference range to a higher numerical range as the elapsed time of playing the electronic game becomes longer. As a result, as the playing time of the electronic game becomes longer, it is possible to make an adjustment as if the AI agent 42, which is an opponent, becomes stronger as an NPC. Further, for example, it is preferable that the reference range is set to a lower numerical range as the elapsed time of playing the electronic game becomes longer. As a result, as the playing time of the electronic game becomes longer, the AI agent 42, which is an opponent as the NPC, can be adjusted as if it was weakened due to fatigue. Further, for example, it is preferable to set the width of the reference range to a wider numerical range as the elapsed time of playing the electronic game becomes longer. As a result, as the playing time of the electronic game becomes longer, the AI agent 42, which is an opponent as the NPC, can be adjusted as if the selection was blurred due to fatigue. Further, for example, it is preferable to set the width of the reference range to be a narrower numerical range as the elapsed time of playing the electronic game becomes longer. As a result, as the playing time of the electronic game becomes longer, the concentration of the AI agent 42, which is an opponent as the NPC, increases, and it is possible to make an adjustment as if the selection is less likely to be blurred.

Further, by randomly selecting the selection value according to a uniform probability distribution over the entire reference range, any one of the plurality of characters included in the reference range is randomly selected. As a result, the strength can be varied within the strength range of the AI agent 42 set in the reference range, and the AI agent 42 can realize a natural behavior closer to that of a human.

The probability distribution set in the reference range is not particularly limited. For example, as shown in FIG. 15, the selection value may be determined from the selection range based on the normal distribution having the maximum probability at the maximum value of the reference range. In this case, it becomes easier to select the use of a character having a higher evaluation value within the set reference range, and it becomes difficult to select the use of a character having a lower evaluation value. Therefore, it is possible to realize a specification that sometimes makes an erroneous selection while maintaining the strength of the AI agent 42 substantially constant. Further, for example, as shown in FIG. 16, the selection value may be determined from the selection range based on the normal distribution having the maximum probability at the minimum value of the reference range. In this case, it becomes easy to select the use of the character having the lower evaluation value in the set reference range, and it becomes difficult to select the use of the character having the higher evaluation value. Therefore, while maintaining the strength of the AI agent 42 at a substantially low level, it is possible to realize a specification in which the evaluation value is sometimes selected to be high.

Further, the center value and standard deviation of the normal distribution may be changed. For example, by receiving an input of a level desired by the user (player) from the player terminal 14 at the time of starting the play of the electronic game, and transmitting the input result to the game server 16, the level is set in the game server 16. The center value and standard deviation of the normal distribution may be set accordingly. By changing the central value, the relative strength of the AI agent 42 can be adjusted, and by changing the standard deviation, the relative stability of the AI agent 42 can be adjusted.

Further, the AI agent 42 can also imitate a plurality of players as an NPC. In this case, the level and stability can be adjusted for each of the plurality of players to be simulated by setting the reference range for each of the plurality of players to be simulated and determining the use of the character for each of the reference ranges.

In the present embodiment, the evaluation values are arranged in ascending order to calculate the cumulative value and normalize the cumulative value. However, the evaluation values are arranged in descending order to calculate the cumulative value and normalize the cumulative value. The processing may be performed. In this case, use of a character having a higher evaluation value is selected as the value is lower in the normalized cumulative value.

In the above embodiment, the evaluation value normalization unit 58b normalizes the cumulative value of the evaluation value to a predetermined numerical range. However, the reference range is normalized according to the cumulative value of the evaluation value. It may be deformed.

FIG. 17 shows the configuration of the AI battle processing unit 58 in the modified example. The reference range setting unit 58c sets a reference range serving as a reference in a predetermined numerical range. For example, the predetermined reference value range is set to 0 to 100, and the reference reference range is set to 30 or more and 60 or less. The reference range normalization unit 58e normalizes the reference range defined by the reference range setting unit 58c according to the cumulative value of the evaluation values calculated by the evaluation value calculation unit 58a. For example, as shown in FIG. 18, when the cumulative value of the evaluation values is 300, the reference value is set so that the range 0 to 300 of the cumulative value and the predetermined numerical value range 0 to 100 that the reference range can match. The reference range defined by the range setting unit 58c is tripled and normalized to 90 or more and 180 or less.

Note that, also in this modification, the reference range may be set and changed as in the above-described embodiment. For example, the reference range and its width may be set and changed according to the degree of difficulty of the electronic game, or the reference range may be set and changed according to the elapsed time of playing the electronic game.

The outline of the game system 10 according to the present embodiment is as described above. According to the present embodiment, each character is represented by the feature vector by the expression learning unit 54, and then the AI agent 42 is learned by the supervised learning unit 56 or the reinforcement learning unit 60. This enables the AI agent 42 to learn more efficiently. Specifically, as described above, reduction of the amount of calculation for learning by compression of the dimension of the vector expressing the character and generalization of learning are realized.

Further, in the present embodiment, it is possible to exert appropriate strength in the battle between the learned AI agent 42 and the user (player) or the AI agents 42. That is, it is possible to set the AI agent 42 to an appropriate strength based on the evaluation value for the use of the character obtained by the learned AI agent 42. As a result, it is possible to give the user (player) who plays the electronic game a higher level of satisfaction.

Although the embodiment according to the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications can be made without departing from the spirit of the present invention.

10 game system, 12 planner terminal, 12a, 14a, 50 control section, 12b, 14b, 30 communication section, 12c, 14c input section, 12d, 14d display section, 12e, 14e, 32 storage section, 14 player terminal, 16 game Server, 34 character DB, 36 player log DB, 38 distributed expression DB, 40 AI log DB, 42 AI agent, 44 statistical data group, 44a statistical data, 52 player log collection unit, 54 expression learning unit, 56 supervised learning unit , 58 AI battle processing unit, 58a evaluation value calculation unit, 58b evaluation value normalization unit, 58c reference range setting unit, 58d process determination unit, 58e reference range normalization unit, 60 reinforcement learning unit, 62 AI log collection unit, 64 Statistical data generator, 66 Display controller.

Claims (15)

In an electronic game using a plurality of objects, a strategy evaluation means that allows machine learning to evaluate a strategy on an electronic game for use of the objects as an evaluation value,
Obtaining the output result of the evaluation value for the use of the object that is applicable in the electronic game using the strategy evaluation means, the evaluation value corresponding to the reference range in the cumulative value obtained by arranging the evaluation values in ascending order or descending order. Strategy determining means for determining the use of the object to be actually applied to the electronic game from the use of the corresponding object,
An information processing apparatus comprising: The information processing apparatus according to claim 1, wherein
The strategy determining unit normalizes the cumulative value to a predetermined range, and uses the object corresponding to the evaluation value corresponding to the reference range in the normalized cumulative value from the use of the object to be actually applied to the electronic game. An information processing device characterized by determining use. The information processing apparatus according to claim 1, wherein
The strategy determining means normalizes the reference range to a predetermined range, and uses the object corresponding to an evaluation value corresponding to the normalized reference range in the cumulative value to actually apply the object to an electronic game. An information processing device, characterized in that the use of the information processing device is determined. The information processing apparatus according to any one of claims 1 to 3,
The strategy evaluation means is machine learning for evaluating a strategy in an electronic game for use of the object as an evaluation value by using a past play history in an electronic game. .. The information processing apparatus according to any one of claims 1 to 4,
The information processing apparatus, wherein the strategy determining means calculates the cumulative value by using all of the evaluation values for use of the object applicable in an electronic game using the strategy evaluating means. The information processing apparatus according to any one of claims 1 to 4,
The information processing apparatus, wherein the strategy determining means calculates the cumulative value by using only part of the evaluation values for use of the object applicable in the electronic game using the strategy evaluating means. The information processing apparatus according to any one of claims 1 to 6,
An information processing apparatus, wherein the reference range can be set and changed according to a difficulty level of an electronic game. The information processing apparatus according to any one of claims 1 to 7,
An information processing apparatus, wherein the width of the reference range is changeable. The information processing apparatus according to any one of claims 1 to 8,
An information processing apparatus, wherein the reference range is set and changed according to an elapsed time of playing an electronic game. The information processing apparatus according to any one of claims 1 to 9,
An information processing apparatus, wherein a plurality of the reference ranges are set, and a strategy of playing an electronic game by a plurality of players is determined by determining the use of the object for each of the reference ranges. The information processing apparatus according to any one of claims 1 to 10, wherein
The strategy determining means determines the use of the object corresponding to the evaluation value selected according to the uniform probability distribution from the evaluation values corresponding to the reference range as the use of the object to be actually applied to the electronic game. An information processing device characterized by: The information processing apparatus according to any one of claims 1 to 10,
The strategy determining means determines the use of the object corresponding to the evaluation value selected according to a predetermined probability distribution from the evaluation values corresponding to the reference range as the use of the object actually applied to the electronic game. An information processing device characterized by: The information processing apparatus according to claim 12,
The information processing apparatus, wherein the probability distribution is a normal distribution having the maximum value of the probability distribution at the maximum value or the minimum value of the reference range. Computer,
In an electronic game using a plurality of objects, a strategy evaluation means that causes machine learning to evaluate a strategy on an electronic game for use of the objects as an evaluation value,
Obtaining the output result of the evaluation value for the use of the object applicable in the electronic game by using the strategy evaluation means, the evaluation value corresponding to the reference range in the cumulative value obtained by arranging the evaluation values in ascending order or descending order Strategy determining means for determining the use of the object to be actually applied to an electronic game from the use of the corresponding object,
An information processing program characterized by causing it to function. In an electronic game using a plurality of objects, a strategy learning step of performing machine learning for evaluating a strategy on the electronic game for use of the objects as an evaluation value,
Using the agent learned in the strategy learning step, obtain the output result of the evaluation value for the use of the object that is applicable in the electronic game, and arrange the evaluation value in the ascending or descending order to obtain a total value within the reference range. A strategy determining step of determining the use of the object actually applied to the electronic game from the use of the object corresponding to the corresponding evaluation value,
An information processing method, characterized in that the method is executed by a computer.

Images