JP7299709B2 - Information processing device, information processing program and information processing method - Google Patents

Description

The present invention relates to an information processing device, an information processing program, and an information processing method, and more particularly to an information processing device, an information processing program, and an information processing method that provide an electronic game.

2. Description of the Related Art Conventionally, information processing apparatuses that provide electronic games are known. As an electronic game provided by such an information processing apparatus, an electronic game using a plurality of objects (in-game objects) such as characters and cards is known. One example is a competitive game in which a player uses a plurality of objects to fight against an opponent who is a computer or another player.

On the other hand, in recent years, research on machine learning has been actively conducted. For example, research is being conducted on deep learning using multi-layer neural networks. Such deep learning is used to train an artificial intelligence (AI) agent that plays an electronic game, and an AI agent that has higher performance than before (that is, is "good at playing games") is realized. For example, Non-Patent Document 1 discloses a technology related to a deep neural network applied to an AI agent playing Go.

David Silver, et al, “Mastering the Game of Go with Deep Neural Networks and Tree Search”Nature 529,484-489,(28 January 2016)

In a competitive game, it is desirable to be able to set an appropriate strength in order to make the user (player) feel interesting when playing as a virtual opponent (NPC) based on the evaluation value obtained by AI or the like. . For example, a beginner user (player) may lose interest in the electronic game if the NPC is too strong. On the other hand, a highly skilled user (player) may not be satisfied with the electronic game if the NPC is too weak.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a technique that allows an artificially intelligent agent playing an electronic game using a plurality of objects to exhibit appropriate strength.

According to one aspect of the present invention, in an electronic game using a plurality of objects, a strategy evaluation means that performs machine learning for evaluating a strategy in the electronic game for using the objects as an evaluation value, and the strategy evaluation means. Obtain the output result of the evaluation value for the use of the object applicable in the electronic game using the electronic game, arrange the evaluation values in ascending or descending order, and add up the cumulative value of the object corresponding to the evaluation value falling within the reference range. and strategy determining means for determining the use of the object to be actually applied to the electronic game from the use.

Here, the strategy determination means normalizes the cumulative value to a predetermined range, and actually applies the electronic game from the use of the object corresponding to the evaluation value corresponding to the reference range in the normalized cumulative value. It is preferred to determine the use of said object.

In addition, the strategy determination means normalizes the reference range to a predetermined range, and actually applies to the electronic game from the use of the object corresponding to the evaluation value corresponding to the normalized reference range in the cumulative value It is preferred to determine the use of said object.

Moreover, it is preferable that the strategy evaluation means performs machine learning for evaluating a strategy in the electronic game for using the object as an evaluation value using a past play history in the electronic game.

In addition, it is preferable that the strategy determination means calculates the cumulative value using all evaluation values for use of the object that can be applied in the electronic game using the strategy evaluation means.

Moreover, it is preferable that the strategy determination means calculates the cumulative value using only a part of the evaluation values for the use of the object that can be applied in the electronic game using the strategy evaluation means.

Moreover, it is preferable that the reference range can be changed according to the difficulty level of the electronic game.

Also, it is preferable that the width of the reference range is changeable.

Further, it is preferable that the reference range is changed according to the elapsed time of playing the electronic game.

Further, it is preferable to determine a strategy for playing the electronic game by a plurality of players by setting a plurality of reference ranges and determining use of the object for each of the reference ranges.

Also, the strategy determination means determines the use of the object corresponding to the evaluation value selected according to the uniform probability distribution from the evaluation values corresponding to the reference range as the use of the object actually applied to the electronic game. It is preferable to

Further, the strategy determination means determines the use of the object corresponding to the evaluation value selected according to a predetermined probability distribution from the evaluation values corresponding to the reference range as the use of the object actually applied to the electronic game. It is preferable to Here, it is preferable that the probability distribution is a normal distribution that takes the maximum value of the probability distribution at the maximum value or the minimum value of the reference range.

According to another aspect of the present invention, a strategy evaluation means that causes a computer to perform machine learning for evaluating, as an evaluation value, a strategy in an electronic game for using the objects in an electronic game using a plurality of objects; An output result of an evaluation value for use of the object applicable in the electronic game is obtained using the evaluation means, and the evaluation values are arranged in ascending or descending order and summed up to correspond to the evaluation value corresponding to the reference range. An information processing program characterized by functioning as strategy determining means for determining the use of the object to be actually applied to the electronic game from the use of the object.

Another aspect of the present invention is a strategy learning step for performing machine learning for evaluating a strategy in the electronic game for using the objects as an evaluation value in an electronic game using a plurality of objects, and learning in the strategy learning step Obtaining output results of evaluation values for the use of the object applicable in the electronic game using the agent obtained, and arranging the evaluation values in ascending or descending order and totaling them to correspond to the evaluation value corresponding to the reference range. and a strategy determination step of determining the use of the object to be actually applied to the electronic game from the use of the object to be executed.

ADVANTAGE OF THE INVENTION According to this invention, the information processing apparatus and information processing program which can demonstrate appropriate strength in the artificial intelligence agent who plays the electronic game which uses several objects can be provided.

1 is a schematic configuration diagram of a game system according to an embodiment of the present invention; FIG. It is a figure which shows the example of the game screen of the game which the game system in embodiment of this invention provides. 1 is a schematic configuration diagram of a planner terminal according to an embodiment of the present invention; FIG. 1 is a schematic configuration diagram of a player terminal according to an embodiment of the present invention; FIG. 1 is a schematic configuration diagram of a game server according to an embodiment of the present invention; FIG. It is a figure which shows the example of the content of character DB. It is a figure which shows the example of the content of player log DB. FIG. 4 is a schematic configuration diagram of an AI battle processing unit according to the embodiment of the present invention; It is a figure which shows the relationship between use of a character and an evaluation value in embodiment of this invention. FIG. 4 is a diagram for explaining normalization processing for the cumulative value of evaluation values in the embodiment of the present invention; It is a figure which shows the example of a setting of the reference range in embodiment of this invention. FIG. 10 is a diagram for explaining selection processing for using a character according to the embodiment of the present invention; It is a figure which shows the example of a setting of the reference range in embodiment of this invention. It is a figure which shows the example of a setting of the reference range in embodiment of this invention. FIG. 10 is a diagram for explaining selection processing for using a character according to the embodiment of the present invention; FIG. 10 is a diagram for explaining selection processing for using a character according to the embodiment of the present invention; FIG. 11 is a schematic diagram of the configuration of an AI battle processing unit in a modified example of the present invention; It is a figure which shows the example of a setting of the reference range in the modification of this invention.

Embodiments of the present invention will be described below.

FIG. 1 shows a schematic configuration diagram of a game system 10 according to this embodiment. The game system 10 is provided by a developer of an electronic game (hereinafter simply referred to as "this game") provided by the game system 10, more specifically, a planner who is a creator of objects (characters) used in this game. It includes a planner terminal 12 to be used, a player terminal 14 to be used by a player of this game, and a game server 16 as an information processing device. The planner terminal 12 and the game server 16 and the player terminal 14 and the game server 16 are connected via a communication line 18 so as to be able to communicate with each other. The communication line 18 is configured by, for example, the Internet or a local area network (LAN). Although one planner terminal 12 and one player terminal 14 are shown in FIG. 1, a plurality of each may be provided.

In the game system 10, the planner can use the planner terminal 12 to access the game server 16, thereby developing the game including creating objects. In addition, by accessing the game server 16 using the player terminal 14, the user (player) can play an electronic game against an NPC (Non Player Character). Also, NPCs (Non Player Characters) can be played against each other in the game server 16 .

Here, the contents of this game will be explained. FIG. 2 shows an example of a game screen displayed on the player terminal 14. As shown in FIG. This game uses a plurality of characters as a plurality of objects to battle an enemy player. The enemy player may be a player other than the player, or may be a computer (game server 16). Specifically, characters are alternately arranged by the self player and the enemy player on a board 20, which serves as a game field, on which a plurality of squares are defined, in the manner of Othello (registered trademark). In FIG. 2, the rows of each square are indicated by numbers "1" to "6", and the columns are indicated by letters "A" to "F".

The white circles on the board 20 indicate the own player's stones (white stones), and the black circles indicate the enemy player's stones (black stones). A white circle on the board 20 with a character name (for example, "C0055") indicates the player's character (own character), and a black circle with a character name indicates the enemy player's character. Indicates a character (enemy character). When an enemy character or a black stone is sandwiched between two own characters or one white stone and one own character, the enemy character or the black stone can be changed (turned over) into a white stone. Also, when the self character or the white stone is sandwiched between two enemy characters or one black stone and one enemy character, the self character or the white stone is changed to a black stone. It should be noted that two white stones and two black stones may be placed in each of the four central squares of the board 20 at the start of the game, as in the case of ordinary Othello. That is, white stones may be placed in the 3C and 4D squares, and black stones may be placed in the 3D and 4C squares.

The player selects a plurality of characters from a plurality of characters defined in the game and acquired by the player (made usable in the game). For example, in this embodiment, 16 characters are selected. A group of 16 characters selected by the player is called a deck. The player can acquire new characters as the game progresses or by charging money. As shown in FIG. 2, four characters selected from the deck are displayed on the screen. The four characters are called hand pieces 22 . The hand piece 22 may be randomly selected from the deck by the game server 16, or may be selected from the deck by the player. The user selects an arbitrary character from among the four hand pieces 22 and places the selected character on the squares on the board 20. - 特許庁When one character is placed on the board 20 from the hand piece 22, one character selected from the deck is newly added to the hand piece 22.例文帳に追加

Although the details will be described later, each character is set with a plurality of parameters. For example, parameters such as attack power, skill effects, and skill activation conditions are set. When a character is placed on the board 20, the opponent can be dealt damage according to the attack power of the character. Alternatively, when the character's skill activation condition is satisfied, the character's special effect skill (technique) is executed, and a game effect corresponding to the skill effect (or skill effect and attack power) is activated. .

HP (hit points) are set for each of the self player and the enemy player. FIG. 2 shows that the player's maximum HP is 250 and the current HP is 180. In FIG. The opponent's HP can be reduced by placing a character on the board 20 or by executing a skill. Also, HP may be recovered by the character's skill. The one whose HP reaches 0 first loses, and the other wins.

In this way, the game provided by the game system 10 is a game in which the player uses a plurality of characters to battle against opponents.

FIG. 3 shows a schematic diagram of the configuration of the planner terminal 12. As shown in FIG. Planner terminal 12 may be, for example, a general computer.

The control section 12a includes, for example, a CPU (Central Processing Unit) or a microcomputer, and controls each section of the planner terminal 12 according to a program stored in a storage section 12e, which will be described later. The communication unit 12b includes, for example, a network adapter and the like, and has a function of communicating with the game server 16 via the communication line 18. FIG. The input unit 12c includes, for example, a mouse, a keyboard, or a touch panel. The input unit 12c is for inputting a planner's instruction to the planner terminal 12. FIG. The display unit 12d includes, for example, a liquid crystal display, and displays various screens. In particular, a game development screen or the like is displayed on the display unit 12d. The storage unit 12e includes, for example, a hard disk, a RAM (Random Access Memory), or a ROM (Read Only Memory). A program for operating each part of the planner terminal 12 is stored in the storage part 12e. Alternatively, the storage unit 12e stores various control data or various processing result data.

FIG. 4 shows a schematic diagram of the configuration of the player terminal 14. As shown in FIG. In this embodiment, a portable terminal such as a smart phone or a tablet terminal is used as the player terminal 14, but the player terminal 14 may be a stationary computer.

The control section 14a includes, for example, a CPU or a microcomputer, and controls each section of the player terminal 14 according to a game client program stored in a storage section 14e, which will be described later. The communication unit 14b includes, for example, a network adapter and the like, and has a function of communicating with the game server 16 via the communication line 18. FIG. The input unit 14c includes, for example, a touch panel or buttons. The input unit 14c is for inputting player instructions to the player terminal 14 . The display unit 14d includes, for example, a liquid crystal display, and displays various screens. In particular, a game screen or the like is displayed on the display unit 14d. The storage unit 14e includes, for example, a RAM or a ROM. The storage unit 14e stores a game client program. Alternatively, the storage unit 14e stores various control data, various game processing result data, and the like.

FIG. 5 shows a schematic diagram of the configuration of the game server 16. As shown in FIG.

The communication unit 30 includes, for example, a network adapter, etc., and has a function of communicating with the planner terminal 12 and the player terminal 14 via the communication line 18 . The storage unit 32 includes, for example, a hard disk, RAM, or ROM. The storage unit 32 stores a game host program as an information processing program. By executing the host program, the game progresses and each part of the game server 16 operates. 5, the storage unit 32 stores a character DB 34, a player log DB 36, a distributed expression DB 38, an AI log DB 40, an AI agent 42, and a statistical data group 44 consisting of a plurality of statistical data 44a. be.

The character DB 34 is a database that stores data relating to characters that are defined in this game (that is, by the host program) and that can be used by the player or enemy players in this game. Specifically, each parameter set for each character is stored in the character DB 34 . In addition, in this game, more than a thousand characters can be used, and new characters are added one by one.

FIG. 6 shows an example of the contents of the character DB 34. As shown in FIG. Although the character DB 34 is shown in a table format in FIG. 6, the data format of the character DB 34 is not limited to this. In FIG. 6, one record corresponds to one character. In this embodiment, each character has a character name, an attribute indicating the characteristics of the character, a rarity indicating the degree of difficulty in obtaining the character, and a reference for damage given to the opponent when the character is placed on the board 20. , a skill name, a skill activation condition that indicates the condition for activating the skill of the character, and a skill effect parameter that is the effect when the skill is activated. "Normal attack" in the skill effect column means that the character is placed on the board 20 and damages the opponent based on the attack power value without activating the skill.

These parameters can be defined by the planner. Of course, the parameter items are not limited to these, and each character may have parameters for other items.

As shown in FIG. 6, in this embodiment, the condition for activating the skill of each character is, for example, "HP is 5% or less of maximum HP" or "There are 3 or more own characters with attribute P1 on the board." are stipulated in the Game State Conditions. Therefore, it is difficult for the player to appropriately select the characters to be included in the deck because the skill activation conditions for each character are defined by the game situation conditions. In other words, it is not easy for the player to grasp the probability of the character activating the skill by looking at the skill activation conditions of the character.

Returning to FIG. 5, the player log DB 36, the distributed representation DB 38, and the AI log DB 40 each store various data. Details of the data stored in these databases will be described later.

AI agent 42 is an artificial intelligence program that plays the game. The AI agent 42 functions as strategy evaluation means. The AI agent 42 is learned by a method using deep learning, which will be described later. The details of the learning method of the AI agent 42 will be described later. The AI agent 42 in this embodiment includes a multilayer neural network. Since the multilayer neural network is defined by various parameters (layer structure, neuron structure in each layer, weight or bias of each neuron, etc.), the storage unit 32 stores these parameters.

The statistical data 44a is statistical data relating to the character usage results. In this embodiment, the statistical data 44a is generated for each character, and the statistical data group 44 is a plurality of statistical data 44a corresponding to each character. The statistical data 44a is generated by the statistical data generating section 64, which will be described later, and the details thereof will be described later.

The control section 50 includes, for example, a CPU, a GPU (Graphics Processing Unit), or a microcomputer, and controls each section of the game server 16 according to a game host program stored in the storage section 32 . In addition, as shown in FIG. 5, the control unit 50 includes a player log collection unit 52, an expression learning unit 54, a supervised learning unit 56, an AI battle processing unit 58, a reinforcement learning unit 60, an AI log collection unit, and a host program. 62 , statistical data generator 64 , and display controller 66 . These functions of the control unit 50 will be described below.

The player log collection unit 52 acquires a player log, which is data indicating the player's game progress history regarding this game, and stores it in the player log DB 36 . The player log collection unit 52 acquires the player log when the player finishes playing the game or during the game play. Incidentally, the player here may include the developer of the game, and the player log collection unit 52 may acquire logs relating to test play by the developer. In any event, player logs are logs of actual game play by humans.

FIG. 7 shows an example of the contents of the player log DB 36. As shown in FIG. Although the player log DB 36 is also shown in a table format in FIG. 7, the data format of the player log DB 36 is not limited to this. As described above, in this game, the first player and the second player alternately place characters on the board 20. Therefore, the player log DB 36 stores the "number of turns" and Or "arranged character" indicating the character arranged on the board 20 by the second player, "arranged place" indicating the place where the character was arranged in the turn, the number of opponent's stones or enemy characters overturned in the turn. , "skill activation" indicating whether or not the skill of the character placed in the turn was activated, and "skill effect" indicating the effect caused by the activation of the skill (for example, damage value” item. Of course, the items shown in FIG. 7 are an example, and the player log may include other items. For example, items such as the actual amount of damage dealt to the opponent by the normal attack of the character may be included.

In this way, in the player log, the results of use ("the number of characters turned over", "skill activation", or " Skill effect value") is included.

The expression learning unit 54 performs expression learning based on the player log as the game log stored in the player log DB 36 . Thereby, the feature vector representing the feature of each character registered in the character DB 34 is learned. That is, the expression learning unit 54 learns distributed expressions of each character used in the game. The feature vector (distributed representation) of each character learned by the representation learning unit 54 is stored in the distributed representation DB 38 . Also, the expression learning unit 54 uses an AI log as a game log, which indicates a game progress history by the AI agent 42 by the AI battle processing unit 58 to be described later, instead of the player log or in combination with the player log. may be performed.

The feature vector according to this embodiment is, for example, a five-dimensional vector, and each element indicates a strategic feature in the game (for example, "I want to put it in the corner", "I want to use it in the early stages", etc.). .

The expression learning unit 54 performs expression learning using a learning device for expression learning constituted by, for example, a neural network. Specifically, in the player log, based on the game state (stones on the board 20 or the placement state of the character, etc.) before a certain character is used, and the result of using the character (next state). , the value of each element of the feature vector of the character is learned.

In this way, by representing each character in a distributed manner, it is possible to represent a large number of characters with a lower-dimensional vector. Also, characters with similar strategic features in this game are represented by similar feature vectors. Various known learning methods can be used as the method of learning expressions.

The supervised learning unit 56 uses at least the player log stored in the player log DB 36 and the feature vector of each character learned by the expression learning unit 54 to cause the AI agent 42 to machine-learn a strategy for using each character. . The supervised learning unit 56 functions as a strategy learning means together with the reinforcement learning unit 60 described later.

Here, the strategy regarding the use of each character is a strategy including what kind of characteristics the character should have or where the character should be placed on the board 20 in a certain game state.

Specifically, the supervised learning unit 56 acquires the game state such as feature vectors, arrangement positions, number of turns, available hands, hit points, and action information (specific The AI agent 42 is trained using the learning data obtained by inputting an input such as arranging a character in a certain square, etc., and outputting an evaluation value of the input action information in the input game state. Note that the evaluation value is calculated in advance based on the player log. That is, the supervised learning unit 56 makes the AI agent 42 learn by supervised learning.

In learning in the supervised learning unit 56, each character is represented by a feature vector and then learned, so the amount of computation in learning can be reduced compared to the case where each character is individually labeled. Also, since characters with similar features are represented by similar feature vectors, learning about a certain character results in learning about characters similar to the character in question. In other words, the effect of learning generalization can be expected.

The AI battle processing unit 58 performs battle processing using the AI agent 42 learned by the supervised learning unit 56 or further learned by the reinforcement learning unit 60 described later. In other words, the AI battle processing unit 58 functions as a strategy decision means for an electronic game using the AI agent 42 trained by the supervised learning unit 56 which is the strategy learning means and the reinforcement learning unit 60 which will be described later.

The AI battle processing unit 58 performs battle processing between the AI agent 42 and the user (player). Also, the AI battle processing unit 58 may perform processing for making the AI agents 42 battle each other. In battle, the AI agent 42 may be left to select the characters to be put into the deck. The battle processing in the AI battle processing unit 58 will be described later.

The reinforcement learning unit 60 performs reinforcement learning, which is learning for strengthening the AI agent 42 . Reinforcement learning is a method different from supervised learning, and Q-learning, for example, can be used as reinforcement learning. Specifically, in reinforcement learning, AI agent 42 is rewarded based on the outcome of a series of actions (e.g., winning or losing) selected by AI agent 42 from a certain game state, and AI agent 42 Learn what actions to take based on rewards.

In this embodiment, the reinforcement learning unit 60 performs learning based on the battle results from the AI battle processing unit 58 . Note that the reinforcement learning unit 60 may perform reinforcement learning based on player logs stored in the player log DB 36 . In addition, the reinforcement learning unit 60 may perform learning based on the result of a battle between the AI agent 42 and another NPC (Non Player Character; a rule-based AI that is not the AI agent 42) in reinforcement learning.

In this game, the actions that the AI agent 42 can take differ depending on the game state. For example, the position where a character can be placed differs depending on the state of the board 20, and the characters that can be used also differ depending on the type of hand piece 22. Therefore, in this embodiment, reinforcement learning is performed by inputting both a certain game state and an action that the AI agent 42 can take, and outputting an index indicating the value of the input action.

It should be noted that, also in reinforcement learning, it is preferable to learn while using the feature vector of each character learned by the expression learning unit 54 . As a result, the same effects as those in the supervised learning unit 56 can be obtained in reinforcement learning.

In this embodiment, at least one of the supervised learning unit 56 and the reinforcement learning unit 60 corresponds to the strategy learning means. In other words, the strategy learning that causes the AI agent 42 to learn strategies regarding the use of each character is one or both of supervised learning and reinforcement learning.

Battle processing may be performed in the AI battle processing unit 58 based on the AI agent 42 that has undergone reinforcement learning in the reinforcement learning unit 60 . In this case, a battle process is performed between the AI agent 42 further learned by the reinforcement learning unit 60 and the user (player). Also, the AI battle processing unit 58 may perform processing for making the AI agents 42 battle each other. In battle, the AI agent 42 may be left to select the characters to be put into the deck. Battle processing based on the evaluation value in the AI battle processing unit 58 will be described later.

The AI log collecting unit 62 acquires an AI log, which is data indicating the game progress history of the game in the AI battle processing unit 58 and including the results of using a plurality of characters, and stores it in the AI log DB 40 . Since the AI log has the same items as the player log, detailed description of the AI log is omitted here. The only difference between the player log and the AI log is whether it indicates the history of game progress by the player or whether it indicates the history of game progress when at least one of the players is the AI agent 42 . It should be noted that the AI log collection unit 62 acquires not only the log of the match between the AI agent 42 and the user (player), but also the log of the match between the AI agents 42 and the match between the AI agent 42 and the NPC as an AI log. can be

The statistical data generator 64 generates statistical data 44a based on the AI log stored in the AI log DB 40. FIG. As described above, the statistical data 44a is generated according to each character and relates to the usage result of each character. For example, the statistical data 44a may include winning percentages when using characters. In this case, the statistical data generator 64 extracts from the AI log whether or not the character of interest and the AI agent 42 used when the character of interest was used wins, and calculates the winning rate of the character of interest. .

In addition, the statistical data 44a may include the skill activation probability when using a character. In this case, the statistical data generator 64 extracts from the AI log the character of interest and whether or not the skill was activated when the character of interest was used, and calculates the skill activation probability of the character of interest.

Statistical data 44a may also include the distribution of effects produced when characters are used. In this case, the statistical data generation unit 64 extracts the character of interest and the skill effect values when the character of interest activates a skill from the AI log, and calculates the distribution of the skill effect values of the character of interest. Alternatively, the effect produced when using a character may include the amount of damage dealt to the opponent when the character makes a normal attack. , the character of interest and the damage inflicted to the opponent by the normal attack of the character of interest are extracted, and the distribution of the damage of the normal attack of the character of interest is calculated.

As described above, in the present embodiment, the statistical data 44a includes at least one of the winning percentage when using a character, the probability of skill activation when using a character, and the distribution of effects generated when using a character. including one. Of course, the statistical data 44a may contain other data.

The statistical data generator 64 generates statistical data 44a for a plurality of characters, ideally all characters, registered in the character DB 34 through the above-described processing. A statistical data group 44 is thus formed.

The display control unit 66 causes the display unit 12d of the planner terminal 12 to display the statistical data 44a generated by the statistical data generation unit 64 in accordance with a request from the planner terminal 12 (that is, from the planner). In addition, the display control unit 66 causes the display unit 14d of the player terminal 14 to display the statistical data 44a generated by the statistical data generation unit 64 in accordance with a request from the player terminal 14 (that is, from the player). Note that the display control unit 66 does not cause the display unit 14d of the player terminal 14 to display the statistical data 44a regarding the characters under development.

[AI battle processing unit]
The battle processing in the AI battle processing unit 58 will be described below. As described above, the AI battle processing unit 58 uses the AI agent 42 trained by the supervised learning unit 56 and the reinforcement learning unit 60 to perform battle processing between the AI agent 42 and the user (player) and between the AI agents 42. .

As shown in FIG. 8, the AI battle processing section 58 includes an evaluation value calculation section 58a, an evaluation value normalization section 58b, a reference range setting section 58c, and a processing determination section 58d.

The evaluation value calculation unit 58a applies the AI agent 42 trained by the supervised learning unit 56 or the reinforcement learning unit 60 to obtain evaluation values for characters currently usable in the electronic game.

For example, if 4 characters (characters A to D) called handpieces 22 out of 16 characters included in the deck can be placed in any one of the four squares (mass B4, B5, D2, F3), As shown in FIG. 9, there are 16 variations of character usage, in which four characters are arranged in four squares. Therefore, an evaluation value is obtained using the AI agent 42 for each of these 16 character uses. In FIG. 9, for example, character use (1) corresponds to placing character A on cell B4 and the evaluation value is 20, and character use (2) corresponds to placing character A on cell B5. The evaluation value is obtained using the AI agent 42 for each use of the character, such that the evaluation value is 15 . . .

The evaluation value normalization unit 58b performs processing for normalizing evaluation values. The evaluation value normalization unit 58b arranges and accumulates the plurality of evaluation values obtained by the evaluation value calculation unit 58a in ascending order, and performs normalization processing so that the accumulated value falls within a predetermined numerical range.

To simplify the explanation, an example in which five evaluation values are obtained for five character use variations (character use (1) to (5)) in the evaluation value calculation unit 58a will be described below. As shown in FIG. 10, the evaluation value normalization unit 58b arranges the five evaluation values in descending order (ascending order) to obtain a cumulative value, and normalizes the cumulative value so that it falls within a numerical range of 0-100. As a result, a cumulative value obtained by normalizing the evaluation value obtained by the evaluation value calculation unit 58a is obtained.

Note that the normalization numerical range in the evaluation value normalization unit 58b is not limited to 0 to 100, and may be any value that can normalize the cumulative value of a plurality of evaluation values to a constant value. For example, the numerical range may be 0-1, 0-10, and so on.

The evaluation value to be accumulated in the evaluation value normalization unit 58b may be obtained by accumulating all evaluation values obtained by the evaluation value calculation unit 58a, or by accumulating a part of the evaluation values. good. For example, if it is desired to determine the use of a character that is actually applied to the electronic game in consideration of the use of all selectable characters in the electronic game, all the evaluation values obtained in the evaluation value calculation unit 58a are accumulated. Calculate the cumulative value normalized by Also, for example, when it is desired to avoid using characters whose evaluation values are less than a predetermined reference value in the electronic game and decide to use characters that are actually applied to the electronic game, only evaluation values equal to or higher than the reference value are accumulated. may be used to calculate the normalized cumulative value.

The reference range setting unit 58c sets a reference range for selecting a process for the accumulated value normalized by the evaluation value normalization unit 58b. The reference range is set within the numerical range of the normalized accumulated value in the evaluation value normalization section 58b. The AI agent 42 can select only characters whose evaluation values fall within the reference range in the normalized cumulative value. That is, the reference range determines the range of uses of characters that can be selected by the AI agent 42 in the electronic game.

For example, as shown in FIG. 11, when the process of normalizing the cumulative evaluation value to a numerical range of 0-100 is performed, the reference range is set to 40-80 within the range of 0-100. In this case, the AI agent 42 uses characters corresponding to evaluation values included in the reference range 40 to 80 (1: Place character A in square B4), (5: Place character B in square B4), ( 3: Placing character A on square D2) can be determined as the use of the character that is actually applied.

The processing determination unit 58d determines processing that is actually applied in the electronic game. The determined process is, for example, the process of placing which character in which square in the electronic game. The processing determination unit 58d determines the processing corresponding to the evaluation value falling within the reference range set by the reference range setting unit 58c as the processing to be applied to the electronic game. Specifically, a selection value is selected according to a predetermined probability distribution from the reference range set by the reference range setting unit 58c, and the character corresponding to the evaluation value corresponding to the selection value is actually applied. Select as the use of

For example, the selection value is determined based on a uniform probability distribution over the entire reference range for the accumulated value normalized for the evaluation value. When the reference range for the normalized cumulative value is set to 40 to 80, as shown in FIG. 12, the selection value is determined based on a uniform probability distribution over the entire range of the reference range. Then, the use of the character corresponding to the evaluation value corresponding to the selected value in the normalized cumulative value (5: Place character B on square B4) is selected as the use of the character actually applied to the electronic game. That is, a selection value is randomly determined from the entire reference range, and the use of the character corresponding to the selection value is selected as the use of the character actually applied to the electronic game.

As in the present embodiment, by setting a reference range for the normalized cumulative value, use of the character is selected from a plurality of candidates corresponding to the reference range. The strength of AI agent 42 can be adjusted.

Here, the setting of the reference range may be changed according to the difficulty level of the electronic game. For example, as shown in FIG. 13, if the opponent user (player) is a beginner, the reference range is set to 0 or more and less than 30, and if the user is an intermediate player, the reference range is set to 30 or more and less than 60. However, the reference range may be set to 60 or more and less than 90 in the case of advanced players, and the reference range may be set to 90 or more and 100 or less in the case of the most advanced players. As a result, a character corresponding to a relatively low evaluation value among the evaluation values included in the normalized cumulative value is selected for beginners, and a character corresponding to a relatively higher evaluation value is selected as the advanced level is used. It is possible to select the use of characters that

Specifically, by receiving input of a level (difficulty) desired by the user (player) from the player terminal 14 when starting to play the electronic game, etc., and transmitting the input result to the game server 16, The game server 16 may set a reference range according to the level. Also, the reference range may be set based on the statistical data 44a obtained by the statistical data generating section 64. FIG. For example, the reference range may be set according to past winning percentages in matches between the AI agent 42 and the user (player).

Also, as shown in FIG. 14, the width of the reference range may be changed. The width of the reference range corresponds to the selection of character uses that can be selected by the AI agent 42 . Therefore, by widening the width of the reference range, the variation in the use of the character applied by the AI agent 42 every time is also widened, and the strength of the AI agent 42 can be varied. On the other hand, by narrowing the width of the reference range, variations in the use of characters applied by the AI agent 42 every time are reduced, and the AI agent 42 can be stabilized at a constant strength. Note that a single reference value may be set instead of the reference range to keep the strength of the AI agent 42 completely constant.

Specifically, when starting to play the electronic game, the user (player) uses the player terminal 14 to receive an input of the stability of strength desired when playing the electronic game, and the input result is displayed. By transmitting to the game server 16, the width of the reference range may be set in the game server 16 according to the stability. Further, the width of the reference range may be set based on the statistical data 44a obtained by the statistical data generating section 64. FIG. For example, the width of the reference range may be set according to past fluctuations in winning percentages in matches between the AI agent 42 and the user (player).

Also, the reference range may be changed according to the elapsed time of playing the electronic game. For example, it is preferable to set the reference range to a higher numerical range as the elapsed time of playing the electronic game increases. This makes it possible to adjust as if the AI agent 42, which is the NPC's opponent, has become stronger as the electronic game has been played for a longer period of time. Also, for example, it is preferable to set the reference range to a lower numerical range as the elapsed time of playing the electronic game increases. As a result, as the electronic game is played for a longer period of time, the AI agent 42 acting as an opponent as an NPC can be adjusted as if it has become weaker due to fatigue. Also, for example, it is preferable to set the width of the reference range to a wider numerical range as the elapsed time of playing the electronic game increases. As a result, as the electronic game is played for a longer period of time, the AI agent 42, who is an opponent as an NPC, can make adjustments as if the choice has fluctuated due to fatigue. Also, for example, it is preferable to set the width of the reference range to a narrower numerical range as the elapsed time of playing the electronic game increases. As a result, as the electronic game is played for a longer period of time, the concentration of the AI agent 42, who is the opponent as an NPC, increases, and it is possible to make adjustments as if the choice is less likely to fluctuate.

Also, by randomly determining a selection value according to a uniform probability distribution over the entire reference range, one of the plurality of characters included in the reference range is randomly selected. As a result, the strength of the AI agent 42 can be varied within the strength range set in the reference range, and the AI agent 42 can realize natural behavior that is more human-like.

Note that the probability distribution set in the reference range is not particularly limited. For example, as shown in FIG. 15, the selection value may be determined from the selection range based on a normal distribution having the maximum probability at the maximum value of the reference range. In this case, a character with a higher evaluation value is more likely to be selected for use within the set reference range, and a character with a lower evaluation value is less likely to be used. Therefore, while maintaining the strength of the AI agent 42 substantially constant, it is possible to realize a specification that occasionally makes an erroneous selection. Alternatively, for example, as shown in FIG. 16, the selection value may be determined from the selection range based on a normal distribution having the maximum probability at the minimum value of the reference range. In this case, a character with a lower evaluation value is more likely to be selected for use within the set reference range, and a character with a higher evaluation value is less likely to be used. Therefore, it is possible to realize a specification in which the strength of the AI agent 42 is maintained at a substantially low level while occasionally making a selection with a high evaluation value.

Also, the center value and standard deviation of the normal distribution may be changed. For example, when starting to play an electronic game, a level desired by a user (player) is input from the player terminal 14, and by transmitting the input result to the game server 16, the level is reached in the game server 16. The center value and standard deviation of the normal distribution may be set accordingly. By changing the center value, the relative strength of AI agents 42 can be adjusted, and by changing the standard deviation, the relative stability of AI agents 42 can be adjusted.

Also, the AI agent 42 can simulate multiple players as NPCs. In this case, it is possible to adjust the level and stability for each of the plurality of simulated players by setting a reference range for each of the plurality of simulated players and determining the use of characters for each of the reference ranges.

In the present embodiment, the evaluation values are arranged in ascending order, the cumulative value is calculated, and the cumulative value is normalized. It is good also as a process to carry out. In this case, the lower the normalized cumulative value, the higher the evaluation value of the character to be used.

In the above-described embodiment, the evaluation value normalization unit 58b normalizes the accumulated evaluation value to a predetermined numerical range. It can be transformed.

FIG. 17 shows the configuration of the AI battle processing section 58 in this modified example. The reference range setting unit 58c sets a reference range that serves as a reference within a predetermined numerical range. For example, a predetermined numerical range is set to 0 to 100, and a reference range is set to 30 or more and 60 or less. The reference range normalization unit 58e normalizes the reference range determined by the reference range setting unit 58c according to the cumulative value of the evaluation values calculated by the evaluation value calculation unit 58a. For example, as shown in FIG. 18, when the cumulative evaluation value is 300, the reference range is set so that the cumulative value range 0 to 300 matches the predetermined numerical range 0 to 100 that the reference range can take. The reference range determined by the range setting unit 58c is tripled and normalized to 90 or more and 180 or less.

It should be noted that the reference range may be set and changed in the modified example as well as in the above-described embodiment. For example, the setting of the reference range and its width may be changed according to the difficulty level of the electronic game, or the setting of the reference range may be changed according to the elapsed time of playing the electronic game.

The outline of the game system 10 according to this embodiment is as described above. According to this embodiment, each character is represented by a feature vector by the expression learning unit 54, and then the AI agent 42 is learned by the supervised learning unit 56 or the reinforcement learning unit 60. FIG. This enables the AI agent 42 to learn more efficiently. Specifically, as described above, reduction in the amount of computation for learning and generalization of learning are achieved by compressing the dimensions of vectors representing characters.

In addition, in this embodiment, it is possible to exhibit appropriate strength in a match between the learned AI agent 42 and the user (player) or between the AI agents 42 . That is, it is possible to set the AI agent 42 to an appropriate strength based on the evaluation value for character use obtained by the AI agent 42 that has been learned. As a result, users (players) who play the electronic game can be given a higher sense of satisfaction.

Although the embodiments according to the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the gist of the present invention.

10 game system, 12 planner terminal, 12a, 14a, 50 control unit, 12b, 14b, 30 communication unit, 12c, 14c input unit, 12d, 14d display unit, 12e, 14e, 32 storage unit, 14 player terminal, 16 game Server, 34 Character DB, 36 Player Log DB, 38 Distributed Expression DB, 40 AI Log DB, 42 AI Agent, 44 Statistical Data Group, 44a Statistical Data, 52 Player Log Collection Unit, 54 Expression Learning Unit, 56 Supervised Learning Unit , 58 AI battle processing unit, 58a evaluation value calculation unit, 58b evaluation value normalization unit, 58c reference range setting unit, 58d processing determination unit, 58e reference range normalization unit, 60 reinforcement learning unit, 62 AI log collection unit, 64 statistical data generator, 66 display controller;

Claims (15)

Strategy evaluation means that uses machine learning to evaluate, as an evaluation value, a strategy in an electronic game for using the objects in an electronic game that uses a plurality of objects;
Using the strategy evaluation means, the output result of the evaluation value for the use of the object applicable in the electronic game is obtained, and the evaluation values are arranged in ascending or descending order and totaled to obtain the evaluation value corresponding to the reference range. a strategy determining means for determining the use of the object to be actually applied to the electronic game from the use of the corresponding object;
An information processing device comprising: The information processing device according to claim 1,
The strategy determination means normalizes the cumulative value to a predetermined range, and determines the object actually applied to the electronic game from the use of the object corresponding to the evaluation value corresponding to the reference range in the normalized cumulative value. An information processing device characterized by determining use. The information processing device according to claim 1,
The strategy determination means normalizes the reference range to a predetermined range, and the object actually applied to the electronic game from the use of the object corresponding to the evaluation value corresponding to the normalized reference range in the accumulated value An information processing apparatus characterized by determining the use of The information processing device according to any one of claims 1 to 3,
The information processing apparatus, wherein the strategy evaluation means performs machine learning for evaluating a strategy in the electronic game for using the object as an evaluation value using a past play history in the electronic game. . The information processing device according to any one of claims 1 to 4,
The information processing apparatus, wherein the strategy determination means calculates the cumulative value using all evaluation values for use of the object applicable in the electronic game using the strategy evaluation means. The information processing device according to any one of claims 1 to 4,
The information processing apparatus, wherein the strategy determination means calculates the cumulative value using only a part of evaluation values for use of the object applicable in the electronic game using the strategy evaluation means. The information processing device according to any one of claims 1 to 6,
The information processing apparatus, wherein the reference range can be changed according to the difficulty level of the electronic game. The information processing device according to any one of claims 1 to 7,
The information processing apparatus, wherein the width of the reference range is changeable. The information processing device according to any one of claims 1 to 8,
The information processing apparatus, wherein the setting of the reference range is changed according to the elapsed time of playing the electronic game. The information processing device according to any one of claims 1 to 9,
An information processing apparatus, wherein a plurality of said reference ranges are set, and a strategy for playing an electronic game by a plurality of players is determined by determining use of said object for each of said reference ranges. The information processing device according to any one of claims 1 to 10,
The strategy determination means determines the use of the object corresponding to the evaluation value selected according to a uniform probability distribution from the evaluation values corresponding to the reference range as the use of the object actually applied to the electronic game. An information processing device characterized by: The information processing device according to any one of claims 1 to 10,
The strategy determination means determines the use of the object corresponding to the evaluation value selected according to a predetermined probability distribution from the evaluation values corresponding to the reference range as the use of the object actually applied to the electronic game. An information processing device characterized by: The information processing device according to claim 12,
The information processing apparatus, wherein the probability distribution is a normal distribution that takes the maximum value of the probability distribution at the maximum value or the minimum value of the reference range. the computer,
Strategy evaluation means that uses machine learning to evaluate, as an evaluation value, a strategy in an electronic game for using the objects in an electronic game that uses a plurality of objects;
Using the strategy evaluation means, the output result of the evaluation value for the use of the object applicable in the electronic game is obtained, and the evaluation values are arranged in ascending or descending order and totaled to obtain the evaluation value corresponding to the reference range. a strategy determining means for determining the use of the object to be actually applied to the electronic game from the use of the corresponding object;
An information processing program characterized by functioning as a strategy learning step of performing machine learning for evaluating a strategy in an electronic game for using the objects as an evaluation value in an electronic game using a plurality of objects;
The agent learned in the strategy learning step is used to obtain an output result of the evaluation value for the use of the object applicable in the electronic game, and the evaluation values are arranged in ascending or descending order and totaled, and the accumulated value falls within the reference range. a strategy determination step of determining the use of the object to be actually applied to the electronic game from the use of the object corresponding to the corresponding evaluation value;
An information processing method characterized by executing by a computer.

Images