JP4804226B2 - Shooting game processing method, apparatus thereof, program thereof, and recording medium thereof - Google Patents

Description

  The present invention relates to a method of processing a shooting game for displaying an own aircraft and an enemy aircraft on a screen, operating the own aircraft, and fighting an enemy aircraft that attacks the own aircraft, its apparatus, its program, and its recording medium About.

Shooting game processing methods, particularly methods for controlling the operation of enemy aircraft, include algorithms that do not take into account the position of the aircraft, such as algorithms that keep the enemy aircraft descending in a certain direction regardless of the location of the aircraft, There was a method that adopted an algorithm that kept the enemy aircraft stationary regardless of the position of the aircraft. In addition, as an algorithm that considers the position of the own aircraft, for example, an algorithm that moves the enemy aircraft to approach the position of the own aircraft, or an algorithm that moves the enemy aircraft to always keep the position of the own aircraft at a certain distance and direction was there.
In addition, in a non-action thinking game such as chess, othello, and shogi, research on algorithms incorporating artificial intelligence technology has been conducted (for example, see Non-Patent Document 1).
Jonathan Schaeffer, H. Jaap van den Herik, “Games, computers, and artificial intelligence”, Artificial Intelligence, 2002, Vol. 134, p1-7 Hiroshi Kawano “Operation Planning and Control of Underactuated Underwater Robot Considering Navigation in Unknown and Uneven Currents”, JSAI2005, Japanese Society for Artificial Intelligence (19th), 1D1-04, 2005

However, the enemy aircraft's behavior control algorithm according to the prior art is determined in advance by a program, and the enemy aircraft deliberately avoids its own attack, or the enemy aircraft itself is advantageous for attacking its own aircraft. There was nothing that intelligently navigated to position.
Also, since the enemy attack bullet firing algorithm was not automatically updated in consideration of the trap of each player's own operation, once the user became proficient in the game, the user immediately entered the game. There was a problem of getting bored.
In order to solve such problems, it is considered effective to introduce artificial intelligence technology, but currently research on games in artificial intelligence is centered on non-action thinking games such as chess, othello and shogi. There was no attempt to apply artificial intelligence technology to action games like shooting games.

  Also, research on artificial intelligence technology for solving thinking games such as chess, othello, and shogi has concentrated on the development of methods for efficiently performing deep search calculations (see, for example, Non-Patent Document 1). Of course, even in shooting games, search technology is important to develop intelligent enemy aircraft operation algorithms. However, in the shooting game, the demand for the depth of search is low. Rather, how to achieve the level of real time required in the action game is important. However, there was no technology that could meet such requirements.

  According to the present invention, a shooting game for selecting an action of an enemy aircraft using a Markov state transition model in which the relative position of the enemy aircraft from its own position is a state variable and the movement speed of the enemy aircraft is a behavior variable. In the processing method, the displacement amount calculating means obtains the displacement amount of the relative position for each combination of each action of the enemy aircraft and each action of the own aircraft. The first state transition probability calculating means calculates a grid having the same dimension as the relative position constituting the state in a state of the Markov state transition model for each combination of each action of the enemy aircraft and each action of the own aircraft. The translation is performed by the obtained displacement amount, and the probability proportional to the area of the common part with the other lattice is obtained as the state transition probability for each combination of each action of the enemy aircraft and each action of the own aircraft. The multiplication means multiplies the state transition probability for each combination of each action of the enemy aircraft and each action of the own aircraft by a value obtained by dividing the own aircraft by the number of types of actions. The second state transition probability calculating means obtains the state transition probability when the enemy aircraft takes each action by taking the sum of the values obtained in the multiplication process for all the actions of the own machine. The reward determining means determines a reward for each combination of state, action, and transition destination state. The action planning means obtains enemy aircraft action policy data based on the action planning method in the Markov state transition model using the state transition probability when the enemy aircraft takes each action and the reward. The state acquisition means acquires the current state of the enemy aircraft. The enemy aircraft action selection means selects an action of the enemy aircraft based on the acquired current state of the enemy aircraft and the enemy aircraft action policy data.

  According to the present invention, the operation of the enemy aircraft in the shooting game has become more intelligent, and a shooting game with improved game performance has been realized. Also, it is possible to update the enemy aircraft operation algorithm in consideration of the player's own operation habits, and it is possible to realize a game that is less tired than before.

[Theoretical background]
<Shooting game>
FIG. 3 exemplifies a shooting game to be processed according to the present invention. On the screen, the own machine 1 and the attack bullet 3 of the own machine 1, and the enemy machine 2 and the attack bullet 4 of the enemy machine 2 are displayed. There may be a plurality of attack bullets 3, enemy aircraft 2, and attack bullets 4. Assume that the own aircraft 1, enemy aircraft 2, attack bullet 3, and attack bullet 4 are located on a two-dimensional plane composed of the X axis and the Y axis.
The user operates the own aircraft 1 to fight the enemy aircraft 2. The user can move his / her own device for the action unit time T at W speeds (direction and speed) in one operation. W types of moving speeds (direction and speed) (Vsx (w), Vsy (w)) that the own device 1 can take are determined in advance, and the respective moving speeds (Vsx (w), Vsy (w)) Is given a moving speed number w (w = 1,..., W). Hereinafter, the action of selecting the moving speed number w is referred to as action (w). In addition, the user can perform an operation of firing an attack bullet 3 for attacking the enemy aircraft 2 at the same time as movement at W speeds.

The enemy aircraft 2 can act at K types of moving speeds (direction and speed) for each behavior unit time T. K types of moving speeds (Vex (k), Vey (k)) that can be taken by the enemy aircraft 2 are determined in advance, and each moving speed (Vex (k), Vey (k)) has a moving speed number. It is assumed that k (k = 1,..., K) is attached. Further, the enemy aircraft 2 can fire the attack bullet 4 simultaneously with the movement. Assume that the attack bullet firing flag is lau, and that enemy aircraft 2 launches attack bullet 4 when lau = 1, and that enemy aircraft 2 does not fire attack bullet 4 when lau = 0.
When the own aircraft 1 collides with the enemy aircraft 2 and the attack bullet 4 fired by the enemy aircraft 2, the own aircraft 1 is destroyed and the game is over there. On the contrary, when the attack bullet 3 of the own aircraft 1 collides with the enemy aircraft 2, the enemy aircraft 2 is destroyed. When all the enemy aircrafts 2 are destroyed by the attack bullet 3 of the own device 1, the user wins the game.
In the present invention, an operation planning method using a Markov state transition model described below is used for controlling the operation of the enemy aircraft 2 in such a shooting game.

<Markov state transition model>
Next, a Markov state transition model and a motion planning method using the Markov state transition model, which are prerequisite knowledge of the present invention, will be described.
The Markov state transition model is a model of the environment as follows. Possible discrete _S = the set of states of the environment _{{s 1, s 2, ...} , s n}, action _A = the set of entities can take action _{{a 1, a 2, ...} a m} represents the . When an action subject executes an action a in a state s ∈ S in the environment, the environment probabilistically transitions to the state s ′ ∈ S. The transition probability is P (s, s ′, a) = Pr {s _{t + 1} = s ′ | s _t = s, a _t = a}
Is represented by This time reward r from the environment to the actors is given probabilistically, the expected value R (s, s', a ) = E {r t | s t = s, a t = a, s t + 1 = s '}
And Decision Strategies function π at each time actors _{(s, a) = Pr {} a t = a | s t = s}
Represented by π (s, a) is defined in all states s and all actions a. The policy function π (s, a) is also simply referred to as policy π.
If the values of transition probabilities P (s, s', a) and reward R (s, s', a) for all combinations of state s, action a, and transition destination state s' are determined, dynamic programming The policy π can be calculated by the (Dynamic Programming) method (for example, refer to Reference Document 1 below). Since the processing of the dynamic programming method is a well-known technique, description thereof is omitted.

The optimal action a in each state s can be determined from the policy π calculated by the dynamic programming method. The above is the outline of the motion planning method using the Markov state transition model.
(Reference 1) Sadayoshi Mikami, Masaaki Minagawa Co-translation, RSSutton, AGBarto Original work “Reinforcement Learning” Morikita Publishing, 1998, pp.94-118
In order to apply the motion planning method using the Markov state transition model to the shooting game as described above, in the embodiment described below, the Markov space in the Markov state transition model is set to a relative distance from the position of the enemy aircraft. (X, Y) and the remaining attack bullet number B of the enemy aircraft. That is, the state s is discretely expressed by the relative distance (X, Y) from the enemy aircraft's own position and the remaining attack bullet number B of the enemy aircraft. Note that the Markov space in the Markov state transition model may be configured only by the relative distance (X, Y) from the position of the enemy aircraft. Further, if the enemy aircraft character advances along the direction of the aircraft, such as an automobile, the Markov space may be configured using the azimuth angle of the enemy aircraft. That is, the Markov space may be configured with variables other than the relative distance from the enemy aircraft character's own position.

  Further, in the present embodiment described below, the action a in the Markov space is assumed to be composed of two variables, an enemy aircraft movement speed number k and an attack bullet launch flag lau. When the Markov space is configured only by the relative distance (X, Y) from the position of the enemy aircraft, the action a may be configured only by the movement speed number k of the enemy aircraft.

[Embodiment]
A shooting game processing apparatus 100 according to the present invention will be described with reference to FIGS. 1 and 2. FIG. 1 is a diagram illustrating a functional configuration of the shooting game processing apparatus 100. FIG. 2 is a diagram illustrating a processing flow of the shooting game processing apparatus 100.
The shooting game processing device 100 includes, for example, a storage unit 10, a state transition probability calculation unit 20, a reward determination unit 30, an action plan unit 40, enemy aircraft action policy data 50, an action determination unit 60, and a display unit 70.
The state transition probability calculation unit 20 includes, for example, a displacement amount calculation unit 21, a first state transition probability calculation unit 22, a multiplication unit 23, and a second state transition probability calculation unit 24. The reward determination unit 30 includes, for example, a distance calculation unit 31, a hit rate calculation unit 32, a first ray determination unit 33, a second ray determination unit 34, a setting state detection unit 35, a temporary storage unit 38, and an input unit 39. The The action determination unit 60 includes, for example, a state acquisition unit 61, an enemy aircraft action selection unit 62, and an enemy aircraft position calculation unit 63.

<Step 1>
The displacement amount calculation unit 21 of the state transition probability calculation unit 20 calculates the displacement amount of the relative position from the own aircraft position for each combination of the action (k, lau) of the enemy aircraft and the action (w) of the own aircraft. (DX, dY) is obtained. Specifically, the displacement calculation unit 21 of the state transition probability calculation unit 20 reads the speed (Vex (k), Vey (k) when the enemy aircraft selects the action (k, lau) read from the storage unit 10. )), The speed (Vsx (w), Vsy (w)) when the own aircraft selects the action (w), and the action unit time of the own aircraft and the enemy aircraft using T,
dX = (Vex (k) −Vsx (w)) × T
dY = (Vey (k) −Vsy (w)) × T
Are calculated for all combinations of the action (k, lau) of the enemy aircraft and the action (w) of the own aircraft. The calculated displacement (dX, dY) of the relative position of the enemy aircraft from its own position is output to the first state transition probability calculation unit 22.

<Step 2>
The first state transition probability calculation unit 22 performs a combination of all states s (X, Y, B), actions (w, k, lau), and transition destination states s ′ (X ′, Y ′, B ′). State transition probability Pe (w, k, lau, s, s') is calculated.
As shown in FIG. 4, the state s (X, Y, B) is composed of the X coordinate of the relative position of the enemy aircraft from its own position and the Y coordinate of the relative position of the enemy aircraft from its own position. It shall be represented by a dimensional grid. For example, here, an enemy aircraft in a certain state s (X, Y, B) exists at each point in the two-dimensional lattice representing the state s (X, Y, B) with an equal probability. To do. Under this assumption, an enemy aircraft in a certain state s (X, Y, B) takes action (k, lau), and the state transition probability Pe (w, k) when the own machine takes action (w). , Lau, s, s ′) is a lattice obtained by translating the lattice g by the displacement (dX, dY), where g is a two-dimensional lattice representing the enemy aircraft state s (X, Y, B). Assuming that gd is obtained, it can be obtained in proportion to the area in which the lattice gd moved in parallel and the other lattice overlap.

That is, the first state transition probability calculation unit 22 sets n to an integer from 1 to 4, sets the grid representing the state sn ′ to gn ′, and sets the area of the overlapping portion of the grid gd and the grid gn ′ to D (gd, gn '), The probability of transition to state sn'
D (gd, gn ′) / Σ _n D (gd, gn ′) (1)
Is obtained by calculating When the lattice gd overlaps the two lattices gn ′, the above equation (1) is calculated by setting n to an integer from 1 to 2, and the state transition probability Pe (w, k, lau, s, s ′) is calculated. Ask. When the lattice gd overlaps one lattice gn ′, the state transition probability Pe (w, k, lau, s, s ′) is obtained by calculating the above equation (1) with n = 1.

For example, when the enemy aircraft in the state s takes action (k, lau) and the own aircraft takes action (w), only the displacements dX and dY of the relative position from the position of the enemy aircraft are in the state s. When the lattice g representing is translated, it is assumed that the translated lattice gd overlaps the four lattices g1 ′ to g4 ′. The areas of the overlapping portions are D (gd, g1 ′) = 6, D (gd, g2 ′) = 6, D (gd, g3 ′) = 1, D (gd, g4 ′) = 1, respectively. Suppose that At this time, the probability that the enemy aircraft transits to the state s1 ′ is D (gd, g1 ′) / Σ _n D (gd, gn ′) = 6/14 = 0.428, and the enemy aircraft transits to the state s2 ′. D (gd, g2 ′) / Σ _n D (gd, gn ′) = 6/14 = 0.428... And the probability that the enemy aircraft will transition to state s3 ′ is D (gd, g3 ′). _{/ Σ n D (gd, gn} ') probability = 1/14 = 0.07 ..., the enemy machine state s4' transition to the, D (gd, g4 ') / Σ n D (gd, gn') = 1/14 = 0.07...

The first state transition probability calculating unit 22 changes the state transition probability Pe (w, k, lau, s) to the state sn ′ corresponding to the lattice gn ′ that overlaps the lattice gd translated by the displacement amount (dX, dY). , S ′) is obtained by calculating the above equation (1). The probability Pe (w, k, lau, s, s ′) for transition to the state s ′ other than the state sn ′ is 0.
The state transition probability Pe (w, k, lau, s, s ′) calculated in this way is output to the multiplication unit 23.

<Step S3>
The multiplication unit 23 multiplies the state transition probability Pe (w, k, lau, s, s ′) by the probability Pv (s, w) that the own device selects the action (w). The calculation results Pv (s, w) · Pe (w, k, lau, s, s ′) are output to the second state transition probability calculation unit 24.
If there is no particular data, the probability Pv (s, w) that the own device selects the action (w) can be a value obtained by dividing 1 by the number W of types of the moving speed of the own device.

<Step S4>
The second state transition probability calculation unit 24 calculates the state s by taking the sum of the action (w) of the own device with respect to the Pv (s, w) · Pe (w, k, lau, s, s ′). The state transition probability P (s, s ′, k, lau) for transitioning to the state s ′ when the enemy aircraft at (1) takes each action (k, lau) is obtained. That is, the second state transition probability calculation unit 24
_ΣwεW Pv (s, w) · Pe (w, k, lau, s, s')
To obtain the state transition probability P (s, s ′, k, lau). The calculated state transition probability P (s, s ′, k, lau) is output to the motion planning unit 40.

<Step S5>
The reward determination unit 30 makes transition to the state s ′ when the enemy aircraft in the state s takes the action (k, lau) for all combinations of the state s, the action (k, lau), and the state s ′ that is the transition destination. A reward r (s, s ′, k, lau) for time is determined.
The reward r (s, s ′, k, lau) can be arbitrarily set by the user. For example, the reward r (s, s ′, k, lau) can be set as follows.

≪How to determine reward r 1≫
The first ray determination unit 33 selects the action (k, lau), and thereby the relative position (X ′, X ′, X ′, Y ′, B ′) of the enemy aircraft that has transitioned to the state s ′ (X ′, Y ′, B ′). From Y ′), it is determined whether or not the own aircraft is on the enemy's line of sight. Whether or not the aircraft is on the enemy's line of fire is determined by assuming that the incoming direction of the attack bullet i of the enemy aircraft is (cosθi, sinθi), the lattice at the relative position (X ′, Y ′) is a line segment (t × This can be determined by whether or not cos θi, t × sin θi) is included.

The first ray determination unit 33 reads the preset flying direction (cos θi, sin θi) of the attack bullet i of the enemy aircraft from the storage unit 10, and the lattice at the relative position (X ′, Y ′) is a line segment (t It is determined whether or not x cos θi, t × sin θi) is included. When it is determined that the own aircraft is on the ray of an enemy aircraft located at the relative position (X ′, Y ′) after the transition, the first ray determination unit 33 performs such state s and action (k , Lau) and the reward r (s, s ', k, lau) for the combination of the transition destination state s', the enemy aircraft that is located at the relative position (X ', Y') after the transition Set higher than when not on line.
By determining the reward in this way, it is possible to obtain a policy π in which the enemy aircraft acts so that the enemy aircraft is positioned on the enemy's ray.

≪How to determine reward r 2≫
The second ray determination unit 34 selects the action (k, lau), and thereby the relative position (X ′, X ′, X ′, Y ′, B ′) of the enemy aircraft that has transitioned to the state s ′ (X ′, Y ′, B ′). It is determined whether Y ′) is on the ray of the aircraft. Whether or not it is on the ray of its own aircraft is determined by assuming that the flight direction of its own attack bullet j is (cos θj, sin θj), the lattice at the relative position (X ′, Y ′) is a line segment (t × cos θj, t It can be determined by whether or not xsin θj is included.

The second ray determination unit 34 reads the preset flying direction (cos θj, sin θj) of the attack bullet j of the own aircraft from the storage unit 10, and the lattice at the relative position (X ′, Y ′) is a line segment (t X cos θj, t × sin θj) is determined. In a case where it is determined that the relative position (X ′, Y ′) of the enemy aircraft after the transition is on the ray of the own aircraft, the second ray determination unit 34 determines such state s and action (k, lau). ) And the remuneration r (s, s ', k, lau) for the combination of the transition destination state s', the relative position (X ', Y') of the enemy aircraft after the transition is not on its own line of sight Set lower.
By determining the reward in this way, it is possible to obtain a policy π in which the enemy aircraft acts so as to avoid its own line of sight.

≪How to determine reward r 3≫
The distance calculation unit 31 calculates the distance r between the enemy aircraft and the own aircraft from the relative distance (X ′, Y ′) from the position of the enemy aircraft in the transition destination state s ′, and as the distance r decreases, The reward r (s, s ′, k, lau) for the state s, the action (k, lau), and the transition destination state s ′ can be set high.
By determining the reward in this way, it is possible to obtain a policy π for selecting an action in which the enemy aircraft approaches the aircraft.

≪How to determine reward r 4≫
The hit rate calculation unit 32 calculates the hit rate of the enemy aircraft attacking bullets from the relative distance (X, Y) from the enemy aircraft position in the state s. The hit rate calculation unit 32 sets the calculation result to a reward r (s, s ′, k, lau) when the enemy aircraft takes an action (k, lau = 1) in the state s. can do.

Here, when the distance between the enemy aircraft and the enemy aircraft is r, the potential probability Ps that the enemy aircraft shoots down the enemy aircraft with an attack bullet is
Ps = Aexp (−ar) (2)
It is thought that it is given by. A and a are positive constants. For example, assuming that the half of the sum of the size of the aircraft and the size of the enemy bullet attack is S, the length of the screen is L, and U is an arbitrary real number between 0.1 and 0.5, A is 1. 0 to 0.8, a
a = U × (L / S) ^−1.5
Then, from experience, it is possible to obtain a highly accurate self-shooting probability P. More specifically, for example, the game screen is 600 pixels long and 400 pixels wide, the shape of the aircraft is 32 × 32 pixels, and the size of the enemy's attack bullet is 4 pixels. At this time, it is preferable that A = 1.0 to 0.8 and a = 0.001 to 0.01. According to the above equation (2), the potential probability P that an enemy aircraft shoots down its own aircraft with an attacking bullet is exponential as the distance r between the enemy aircraft and its own aircraft becomes smaller, as shown in FIG. To rise.

The distance calculation unit 31 calculates the distance r between the enemy aircraft and the subject aircraft in the state s from the relative position (X, Y) of the enemy aircraft from the subject aircraft in the state s.
The hit rate calculation unit 32 calculates the above equation (2) from the distance r calculated by the distance calculation unit 31 and A and a read from the storage unit 10, thereby returning the attacking bullet of the enemy aircraft to its own aircraft. The hit rate Ps is calculated. The hit rate calculation unit 32 converts this Ps into a reward r (s, s ′, k, lau) when the enemy aircraft selects an action (k, lau = 1) in the state s and transitions to the state s ′. Set.
By determining the reward in this way, it is possible to obtain a policy π for selecting an action of firing an attack bullet after the enemy aircraft has sufficiently approached itself.

≪How to determine reward r 5≫
By inputting from the input unit 39, the user can freely set the relative position (X, Y) at which the number of remaining attack bullets B of the enemy aircraft is fired. . The setting state detection unit 35 of the reward determination unit 30 detects the relative position (X, Y) for each remaining attack bullet number B of the enemy aircraft determined in advance by the user setting, and the state s (X, Y, B). ) And the action (k, lau = 1) for firing an attacking bullet and the reward (s, s ′, k, lau) for the state s ′ (X ′, Y ′, B ′) of the transition destination, and other states It is set higher than the reward (s, s ′, k, lau) for s, action (k, lau), and the transition destination state s ′.

  For example, in the state s1 (X = 3, Y = 1, B = 2), a reward (s, s ′) for taking an action (k, lau = 1) to fire an attack bullet and making a transition to the state s ′ , K, lau) is set to 1, and other rewards (s, s, for the combination of the state s, the action (k, lau) and the transition state s ′ in which the remaining number B of the enemy aircraft is B = 2 s ′, k, lau) is set to zero. Further, in the state s2 (X = -3, Y = 1, B = 1), a reward (s, s) for taking an action (k, lau = 1) to fire an attack bullet and making a transition to the state s ′ ', K, lau) is set to 1, and other rewards (s for combinations of state s, action (k, lau), and transition destination state s' where the number of remaining attack bullets B of the enemy aircraft is 1 , S ′, k, lau) is set to zero.

  By setting the reward in this way, in the state where the enemy aircraft's remaining attack bullet number B = 2, first, the enemy aircraft moves to the relative position (X = 3, Y = 1), and at this position the attack bullet , And then a strategy π is obtained such that the enemy aircraft moves to the relative position (X = −3, Y = 1) and fires an attack bullet at this position.

≪How to decide reward 6≫
The rewards may be determined by combining the reward determination methods described in reward determination methods 1 to 5.
For example, first, when the second ray determination unit 34 uses the method described in Reward determination method 2 and the relative position (X ′, Y ′) of the enemy aircraft after the transition is on the ray of the own aircraft, The reward r (s, s ′, k, lau) for the combination of the state s, the action (k, lau), and the transition destination state s ′ is set to −1. Then, the hit rate calculation unit 32 uses the method described in the method 4 for determining the reward, and reward r (s, s for the combination of the other state s, the action (k, lau), and the transition destination state s ′. ', K, lau) can be defined.
Further, the reward r (s, s ′) determined by any one of the reward determination unit 30, the distance calculation unit 31, the hit rate calculation unit 32, the first ray determination unit 33, the second ray determination unit 34, and the setting state detection unit 35. , K, lau) are stored in the temporary storage unit 38. At this time, any one of the reward determination unit 30, the distance calculation unit 31, the hit rate calculation unit 32, the first ray determination unit 33, the second ray determination unit 34, and the setting state detection unit 35 other than the above-determined means, The reward r (s, s ′, k, lau) may be read from the temporary storage unit 38 and the reward may be corrected.

  For example, when it is determined that the relative position (X ′, Y ′) of the enemy aircraft after the transition is on the ray of the own aircraft, the second ray determination unit determines the state s and action (k, lau). ) And a transition r (s, s ′, k, lau) for the combination of the transition destination state s ′, 0.5 is added to the reward r (s, s ′, k, lau) read from the temporary storage unit 38. When it is determined that it is not on the line of the aircraft, the reward r (s, s ′) for the combination of the state s, the action (k, lau), and the transition destination state s ′ is set. , K, lau) can be set to a value obtained by multiplying the reward r (s, s ′, k, lau) read from the temporary storage unit 38 by one.

<Step S6>
The motion planning unit 40 determines the state transition probability P (s) for each combination of the state s (X, Y, B), the action (k, lau), and the state s ′ (X ′, Y ′, B ′) of the transition destination. , S ′, k, lau) and the reward r (s, s ′, k, lau), an enemy aircraft action policy π (s, s ′, k, lau) is obtained by a dynamic programming method. Since the processing of the dynamic programming method is a well-known technique, a description thereof will be omitted (for example, see the above-mentioned reference 1).
The calculated enemy aircraft action policy π (s, s ′, k, lau) is stored as enemy aircraft action policy data 50. The enemy aircraft action policy data 50 is, for example, a database in which an action (k, lau) that maximizes the reward r of the enemy aircraft is associated with the state s (X, Y, B) of all enemy aircraft. is there.

<Step S7>
The action determination unit 60 selects an action (k, lau) of the enemy aircraft, and updates the position of the enemy aircraft from the action (k, lau). Specifically, the enemy aircraft action selection unit 62 searches the enemy aircraft action policy data 50 using the enemy aircraft state s (X, Y, B) acquired by the state acquisition unit 61 as a key, thereby Find the action (k, lau) of the machine. The selected action (k, lau) is output to the enemy aircraft position calculation unit 63. The enemy aircraft position calculation unit 63 reads the movement speed (Vex (k), Vey (k)) and the action unit time T when the enemy aircraft takes action (k, lau) from the storage unit 10, and (T By calculating (Vex (k), TxVey (k)), the position of the updated enemy aircraft is obtained. The updated position of the enemy aircraft is output to the display unit 70.
By performing the processing of steps S1 to S7 at a constant time interval τ, the operation of the enemy aircraft becomes more intelligent, and a shooting game with a higher game performance than before can be realized.

[Modifications, etc.]
In the above embodiment, in step S3, the probability Pv (s, w) that the own device selects the action (w) is a value obtained by dividing 1 by the number W of types of the moving speed of the own device. However, the probability Pv (s, w) that the player selects the action (w) may be calculated from the operation history of the player.

For example, the operation history acquisition unit 25 of the state transition probability calculation unit 20 assumes that each action (w) is selected Dw (X, Y) at the relative position (X, Y). ) Counts Dw (X, Y) every time. The operation history Dw (X, Y) for each relative position (X, Y) from the position of the enemy aircraft is stored in the operation history database 26. The probability calculation unit 27 receives the Dw (X , Y)
Pv (s, w) = Dw (X, Y) / _ΣwεW Dw (X, Y)
To calculate the probability Pv (s, w) that the own device selects the action (w) in each state s. The multiplication unit 23 calculates Pv (s, w) · Pe (w, k, lau, s, s ′) using the Pv (s, w) calculated by the probability calculation unit 27.

In this way, by recording the operation history during game play, reflecting the result in calculating the state transition probability of the enemy aircraft in the Markov space, and performing the plan calculation by the motion planning method online during the game , The action plan π of the enemy aircraft can be updated to reflect the trap of the player's own operation.
Moreover, the difficulty level of a game can be adjusted by adjusting the time length of (tau) which is the time interval which performs the process of step S1-S7 in the said embodiment. In general, if the time length of τ is long, the action selection frequency of the enemy aircraft decreases, so the operation of the enemy aircraft is simplified, and the delay of the own-attack avoidance operation also occurs, so the difficulty of the game decreases.

Further, the processing functions of the shooting game processing apparatus 100 can be realized by a computer. In this case, the contents of the processing functions of the shooting game processing apparatus 100 are described by a program. Then, by executing this program on a computer as shown in FIG. 6, each processing function of the shooting game processing apparatus 100 is realized on the computer.
The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.
A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is provided for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

Further, in this embodiment, the shooting game processing device or the like is configured by causing a predetermined program to be executed on the computer, but at least a part of these processing contents may be realized in hardware. .
Further, the shooting game processing method, the apparatus, the program, and the recording medium according to the present invention are not limited to the above-described embodiments, and can be appropriately changed without departing from the gist of the present invention.

The figure which illustrates the function structure of the shooting game processing apparatus. The figure which illustrates the processing flow of the shooting game processing apparatus. A schematic diagram of a shooting game. The figure for assisting explanation of calculation of state transition probability Pe. The figure which illustrated the relationship between the distance r of the own machine and enemy machine, and the own machine shooting probability P. The figure which illustrated functional composition when performing the shooting game processing device by the present invention by a computer.

Explanation of symbols

1 Own aircraft 2 Enemy aircraft 3 Attack bullet 4 Attack bullet 5 CPU
6 RAM
7 output unit 8 auxiliary storage unit 9 input unit 9 ′ bus 10 storage unit 20 state transition probability calculation unit 21 displacement amount calculation unit 22 state transition probability calculation unit 23 multiplication unit 24 state transition probability calculation unit 25 operation history acquisition unit 26 operation history Database 27 Probability calculation unit 30 Reward determination unit 31 Distance calculation unit 32 Hit rate calculation unit 33 First ray determination unit 34 Second ray determination unit 35 Setting state detection unit 38 Temporary storage unit 39 Input unit 40 Operation planning unit 50 Enemy aircraft behavior Policy data 60 Action determination unit 61 State acquisition unit 62 Enemy aircraft action selection unit 63 Enemy aircraft position calculation unit 70 Display unit 100 Shooting game processing device

Claims (11)

Storage means, displacement amount calculation means, first state transition probability calculation means, multiplication means, second state transition probability calculation means, reward determination means, action planning means, state acquisition means, enemy aircraft behavior A computer equipped with a selection means selects the action of the enemy aircraft using a Markov state transition model with the relative position of the enemy aircraft from its own position as the state variable and the movement speed of the enemy aircraft as the action variable. in the method for performing the processing of the shooting game,
The storage means stores the moving speed when the enemy aircraft takes each action, and the moving speed when the own machine takes each action,
Displacement that the displacement amount calculation means obtains the displacement amount of the relative position using the corresponding enemy aircraft read from the storage means and the moving speed of the own aircraft for each combination of each action of the enemy aircraft and each action of the own aircraft Quantity calculation process,
The first state transition probability calculating means calculates a grid having the same dimension as the relative position constituting the state in a state of the Markov state transition model for each combination of each action of the enemy aircraft and each action of the own aircraft. The first state transition probability obtained by translating the obtained displacement amount and determining the probability proportional to the area of the common part with the other grid as the state transition probability for each combination of each action of the enemy aircraft and each action of the own aircraft Calculation process,
Multiplication means, the multiplication process in which the state transition probability of each combination of each behavior of each behavior and the own apparatus of the enemy, multiplying the value obtained by dividing one by the number of types of actions of its own,
The second state transition probability calculating means calculates the state transition probability when the enemy aircraft takes each action by taking the sum of the values obtained in the multiplication process for all the actions of the own machine. Probability calculation process,
A reward determination process in which a reward determination means determines a reward for each combination of a state, an action, and a transition destination state;
An action planning process for obtaining enemy aircraft action policy data based on the action planning method in the Markov state transition model using the state transition probability when the enemy aircraft takes each action and the reward,
State acquisition means, the state acquisition process to acquire the current state of the enemy aircraft,
An enemy aircraft action selection means for selecting an enemy aircraft action based on the current status of the acquired enemy aircraft and the enemy aircraft action policy data;
A shooting game processing method characterized by comprising: The shooting game processing method according to claim 1,
The reward determination process further includes a process of setting a high reward for the combination of the state, the action, and the state of the transition destination when the own aircraft is on the line of the enemy aircraft after the state transition,
A method for processing a shooting game. In shooting game processing method according to claim 1 Symbol placement,
The reward determination process further includes a process of setting a low reward for a combination of the state, the action, and the state of the transition destination when the enemy aircraft after the state transition is on the line of the own aircraft,
A method for processing a shooting game. In the shooting game processing method according to any one of claims 1 to 3,
The own machine action selection probability calculating means further includes an own machine action selection probability calculating process for obtaining a probability that the own machine selects each action from the operation history data of the player,
In the multiplication process, the state transition probability for each combination of each action of the enemy aircraft and each action of the own machine is multiplied by the probability that the own machine selects each action obtained in the calculation process of the own machine action selection probability. Process,
A method for processing a shooting game. The shooting game processing method according to any one of claims 1 to 4,
The number of attacking bullets that can be fired by enemy aircraft is further included as a state variable constituting the Markov state transition model, and whether or not the enemy aircraft is firing an attacking bullet is further included as an action variable.
A method for processing a shooting game. In the shooting game processing method according to any one of claims 1 to 5,
The reward determination process further includes a process of setting a high reward for a combination of the state, the action, and the transition destination state when the distance between the enemy aircraft after the state transition and the own aircraft after the state transition is close. ,
A method for processing a shooting game. The shooting game processing method according to claim 5,
In the above reward determination process, when an enemy aircraft takes an action to fire an attack bullet, the closer the distance between the enemy aircraft before the state transition and the own aircraft before the state transition is, the closer the state, the action, and the destination Including the process of setting a higher reward for the combination of states,
A method for processing a shooting game. In the shooting game processing method according to claim 5 or 7,
In the above reward determination process, when the enemy aircraft takes an action of firing an attack bullet at a different relative position determined in advance for each number of attack bullets that can be fired by the enemy aircraft, its state, action, and transition destination Including a process of setting a high reward value for a combination of states
A method for processing a shooting game. In a shooting game processing device that selects the action of an enemy aircraft using a Markov state transition model in which the relative position from the position of the enemy aircraft is a state variable, and the movement speed of the enemy aircraft is a behavior variable,
Storage means for storing the movement speed when the enemy aircraft took each action and the movement speed when the own machine took each action;
For each combination of each action of the enemy aircraft and each action of the own aircraft, a displacement amount calculating means for obtaining a displacement amount of the relative position using the corresponding enemy aircraft read from the storage means and the moving speed of the own aircraft ;
In a state of the Markov state transition model, the lattice having the same dimension as the relative position constituting the state is translated by a displacement amount obtained for each combination of each action of the enemy aircraft and each action of the own aircraft, First state transition probability calculating means for obtaining a probability proportional to the area of the common part with the other lattice as a state transition probability for each combination of each action of the enemy aircraft and each action of the own aircraft;
Multiplying means for multiplying the state transition probability for each combination of each action of the enemy aircraft and each action of the own aircraft by a value obtained by dividing 1 by the number of types of the action of the own aircraft;
A second state transition probability calculating means for obtaining a state transition probability when the enemy aircraft takes each action by taking the sum of the values obtained by the multiplying means for all actions of the own aircraft;
Reward determining means for determining a reward for each combination of state, action, and transition destination state;
Using the state transition probability when the enemy aircraft took each action and the reward, an action planning means for obtaining enemy action policy data based on the action planning method in the Markov state transition model,
State acquisition means to acquire the current state of the enemy aircraft,
An enemy aircraft action selection means for selecting an action of the enemy aircraft based on the current state of the acquired enemy aircraft and the enemy aircraft action policy data;
A shooting game processing device comprising: In a shooting game processing program that selects the action of an enemy aircraft using a Markov state transition model in which the relative position of the enemy aircraft from its own position is a state variable and the movement speed of the enemy aircraft is a behavior variable,
The storage means stores the moving speed when the enemy aircraft takes each action, and the moving speed when the own machine takes each action,
For each combination of each action of the enemy aircraft and each action of the own aircraft, a displacement amount calculation process for obtaining a displacement amount of the relative position using the corresponding enemy aircraft read from the storage means and the moving speed of the own aircraft;
In a state of the Markov state transition model, the lattice having the same dimension as the relative position constituting the state is translated by a displacement amount obtained for each combination of each action of the enemy aircraft and each action of the own aircraft, A first state transition probability calculation process for obtaining a probability proportional to the area of the common part with the other lattice as a state transition probability for each combination of each action of the enemy aircraft and each action of the own aircraft;
A multiplication process of multiplying the state transition probability for each combination of each action of the enemy aircraft and each action of the own aircraft by a value obtained by dividing 1 by the number of types of the action of the own aircraft;
A second state transition probability calculation process for obtaining a state transition probability when the enemy aircraft takes each action by taking the sum of the values obtained in the above multiplication process for all actions of the own aircraft,
A reward determination process for determining a reward for each combination of state, action, and destination state;
Using the state transition probability when the enemy aircraft takes each action and the reward, the action planning process for obtaining enemy action policy data based on the action planning method in the Markov state transition model,
A state acquisition process to acquire the current state of the enemy aircraft,
An enemy aircraft action selection process for selecting the action of the enemy aircraft based on the current state of the acquired enemy aircraft and the enemy aircraft action policy data;
Shooting game processing program for causing a computer to execute the. A computer-readable recording medium on which the shooting game processing program according to claim 10 is recorded.

Images