CN116886443B - Opponent action preference estimation method, device and electronic equipment for offensive and defensive games - Google Patents

Abstract

This application relates to an opponent's action preference estimation method, device and electronic equipment for offensive and defensive games. The method includes: counting the types and proportions of action sequences of both offense and defense parties in the previous round of network attack and defense; based on all action sequences of both parties. Proportion, make a two-dimensional approximate estimate of the probability of the action sequence; according to the probability of the action sequence and the joint distribution of the information sets of both offense and defense, determine the posterior distribution of the information set of the network attacker at the end of the previous round; according to the smoothed The functional relationship between the posterior distribution and the information set between rounds is to obtain the distribution of the network attacker's information set at the beginning of the next round; the attacker's opponent strategy model established based on the information set distribution is used to infer the attacker's actions and the defender's Based on this, adopt the optimal confrontation strategy for defense. This method improves the accuracy of predicting the attacker's actions, thereby improving the pertinence of our defense strategy, thereby improving the capability and effectiveness of network defense.

Description

Opponent action preference estimation method, device and electronic equipment for offensive and defensive games

Technical field

The present application relates to the field of network security technology, and in particular to an opponent action preference estimation method, device and electronic device for offensive and defensive games.

Background technique

With the continuous strengthening of informatization, the network has provided people with more conveniences, but network attacks have become increasingly frequent, causing huge losses to the attackers. Therefore, network information attack and defense confrontation has become one of the important issues of network security concern. Network attack and defense is a process of multi-stage random repeated games with incomplete information on both sides. Whether a network attack can succeed depends not only on the strength of the attack capability, but also on targeted defense measures. Therefore, the process of network attack and defense is also a process of confrontation and game between the two sides. The process of reconstructing the opponent's strategy model to predict the attack strategy of the network attacker in the network attack and defense game is challenging.

Due to incomplete information in the network attack and defense process, both network attackers and defenders cannot completely observe the opponent's hidden information, which makes it necessary to use the observation data of the completed game to explicitly build the opponent's strategy model (i.e., action probability model). Some method associates the opponent's information set that is not observed with hidden information and the opponent's information set that is observed with hidden information and the opponent's actions (in a game, the information set of any player refers to the unknowable part of the opponent's information when it acts) This leads to a collection of undetermined historical information. The essence of reconstructing the opponent's strategy model is to establish a correlation probability model between the opponent's information set and the opponent's actions, and the opponent's information set is determined by the opponent's hidden information). The current explicit reconstruction method of opponent strategy model based on decision points aggregates different information sets in the game in decision points, and uses its independent and identical distribution properties to combine the opponent information set with hidden information that has not been observed and the opponent with hidden information that has been observed. The information set is unified into a distribution and then the probability density estimation of the information set corresponding to each action is used to obtain the probability model of different actions.

However, this method assumes that the distribution of the information set within the decision point is uniform. This too strong assumption will bias the probability density estimate of the action, thus limiting the accuracy of the reconstructed opponent model. This is because (1) within a stage, the actions of both parties in the game have preferences, so the opponent's current action is affected by the previous actions of both parties in the game, so that the distribution of the information set within the opponent's current decision point is not uniformly distributed; (2) After the stage transition, due to the existence of random factors, the distribution of the information set in the current stage is affected to a certain extent by the action preferences of both parties in the previous stage, but it still cannot be completely ignored. The uneven distribution of the opponent's information set caused by the influence of both sides' action preferences causes certain deviations in the opponent's action probability model that adopts the assumption of uniform distribution.

In summary, the existing explicit reconstruction method of opponent strategy based on decision points is inconsistent with the actual situation of uneven distribution of opponent information sets affected by preferences due to the uniform distribution assumption of the information set within the decision point, which makes the accuracy of the established opponent strategy model Restrictions lead to deviations in the prediction of the opponent's actions, making the defense strategies adopted by the defender less targeted, affecting the ability and effectiveness of network defense.

Contents of the invention

Based on this, it is necessary to provide an opponent action preference estimation method, device and electronic equipment for offensive and defensive games in response to the above technical problems.

An opponent action preference estimation method for offensive and defensive games, the method includes:

Count the types and proportions of action sequences of both network attackers and defenders in the previous round of network attack and defense.

Based on the proportion of all action sequences of both parties, a two-dimensional approximate estimate of the probability of the action sequence is made.

According to the probability of the action sequence and the joint distribution of the information sets of the network attacker and defender, the posterior distribution of the network attacker's information set at the end of the previous round is determined.

According to the smoothed posterior distribution of the network attacker's information set and the functional relationship between the information sets between rounds, the distribution of the network attacker's information set at the beginning of the next round is obtained, and is used as the opponent in that round Distribution of information sets within first-order decision points.

According to the distribution of the network attacker's information set, the network attacker's opponent strategy model is established, and the network attacker's opponent strategy model is used to speculate on the network attacker's actions. The defender adopts the optimal confrontation strategy based on the network attacker's actions. defense.

In one embodiment, the types and proportions of action sequences of both network attackers and defenders in the previous round of network attack and defense are counted, including:

Statistics of the type and number of action sequences of both network attackers and defenders in the previous round in the data observed during multiple historical confrontations between network attackers and defenders;

Based on the type and number of all said action sequences, the proportion of each type of action sequence is determined as:

;

in, is the ratio of the number of action sequence acts to the total number of action sequences,/> is the number of action sequences of a type;/> is the total number of action sequences;/> It is the conditional probability of the action sequence under the joint distribution of the information sets of the network attacker and defender;/> It is the joint distribution of information sets between network attackers and defenders;/> They are the information sets of the previous round of network attack and defense respectively. The subscript P represents the defender, the subscript O represents the network attacker or opponent, and the subscript pre represents the previous round.

In one embodiment, a two-dimensional approximate estimate of the probability of the action sequence in the previous round is performed based on the proportion of all action sequences of both parties, including:

According to the general decision-making logic of the actions of both parties, the approximate probability of setting the action sequence within the two-dimensional interval of the corresponding information set is:

;

in, is the approximate probability of the action sequence in the two-dimensional space of the information set,/> ,/> They are the information sets of the network attacker and defender respectively;/> ,/> The sequence of actions of both network offensive and defensive parties/> The corresponding two-dimensional interval range of the information set, the probability of each action sequence acts should satisfy:

;

in, is the proportion of the action sequence,/> It is the prior joint distribution of the information sets of the network attacker and defender in the previous round.

According to the proportion of all action sequences of both parties and the general decision-making logic of the actions, each action sequence of the network offensive and defensive parties is obtained by iterating with two equations The corresponding two-dimensional interval range of the information set is used to obtain the approximate probability of the action sequence on the two-dimensional interval of the complete information set.

In one embodiment, based on the probability of the action sequence and the joint distribution of the information sets of the network attacker and defender, the posterior distribution of the network attacker's information set at the end of the previous round is determined, including:

Approximate probability of action sequence in two-dimensional space based on information set , and the set of action sequences Acts that can enter the next round, and obtain the posterior joint distribution of the information sets of both parties at the end of the previous round:

;

in, is the posterior joint distribution of the information sets of both parties at the end of the previous round,/> is the prior joint distribution of the information sets of both network attackers and defenders in the previous round, is the approximate probability of the action sequence in the two-dimensional space of the information set,/> Set the action sequence for the next round,/> for each sequence of actions;

The posterior distribution of the information set of the network attacker is obtained based on the marginal distribution of the joint posterior distribution:

;

in, It is the information set distribution of the network attacker in the previous round.

In one embodiment, according to the smoothed posterior distribution of the network attacker's information set and the functional relationship between the information sets between rounds, the distribution of the network attacker's information set at the beginning of the next round is obtained. ,include:

The posterior distribution of the network attacker's information set is smoothed using a preset kernel function, and a smooth nonlinear density function is obtained:

;

in, is the smooth nonlinear density function of the previous round,/> is the preset kernel function, and the subscript j is the index after uniform discretization of the information set,/> is the posterior distribution of the information set of the network attacker in the previous round,/> It is the information set of the network attacker in the previous round.

According to the smoothed posterior distribution of the network attacker's information set and the functional relationship between the information sets between rounds, the distribution of the network attacker's information set at the beginning of the next round is obtained, and is used as the first-order opponent of the round. Distribution of information sets within decision points:

;

in, is the distribution of the network attacker’s information set at the beginning of the next round, is the functional relationship between the information set between the previous round and the next round,/> It is the information set of the network attacker at the beginning of the next round.

In one of the embodiments, the method further includes a step of estimating the opponent's action preference within the second-order decision point, specifically including:

Statistics of the types and proportions of all action sequences in the network attack and defense process before the second-order decision point;

Based on the proportion of all decision point action sequences before the second-order decision point in the current round, a two-dimensional approximate estimate is made of the probability of the action sequences of both network attackers and defenders;

According to the probability of the action sequence of both sides of the network and the joint prior distribution of the information sets of both sides of the network, determine the joint posterior distribution of the information sets of both sides within the second-order decision point affected by the previous action preference, and according to the joint posterior distribution The marginal distribution determines the distribution of information sets within the second-order decision point of the network attacker. In one embodiment, based on the proportion of all action sequences before the second-order decision point in the current round, a two-dimensional approximate estimate is made of the probability of the action sequences of both the network attacker and defender, including:

According to the general decision-making logic of the actions of both parties, the action sequences are set to be uniformly distributed within the corresponding two-dimensional information set intervals. According to the proportion of the action sequences, the two-dimensional information set intervals corresponding to all action sequences are determined.

In one embodiment, based on the probability of the action sequence of the network attacker and defender and the joint prior distribution of the information sets of both network attackers and defenders, the joint posterior distribution of the information sets of both parties within the second-order decision point affected by the previous action preference is determined, And determine the information set distribution within the second-order decision point of the network attacker based on the marginal distribution of the joint posterior distribution, including:

According to the action sequence that can enter the second-order decision point , determine the joint posterior distribution of the information sets of both network attackers and defenders within the second-order decision point:

;

in, is the joint posterior distribution of the information sets of both network attackers and defenders within the second-order decision point,/> is the conditional distribution of the action sequences of both network attackers and defenders before the second-order decision point that can enter the second-order decision point,/> is the joint prior distribution of the information sets of both parties at the beginning of the current round,/> For action sequences/> The total probability of is equal to/> ;

Determine the information set distribution within the second-order decision point of the network attacker based on the marginal distribution of the joint posterior distribution:

in, is the information set distribution within the second-order decision point of the network attacker.

An opponent action preference estimation device for offensive and defensive games, the device includes:

The ratio determination module of the action sequences of both network attackers and defenders is used to count the types and ratios of the action sequences of both network attackers and defenders in the previous round of network attack and defense.

The probability estimation module of the action sequences of both sides of the network is used to make a two-dimensional approximate estimate of the probability of the action sequence in the previous round based on the proportion of all action sequences of both parties.

The posterior distribution determination module of the current round information set of the network attacker is used to determine the posterior distribution of the network attacker's information set at the end of the previous round based on the probability of the action sequence and the joint distribution of the information sets of both network attackers and defenders. empirical distribution.

The module for determining the distribution of the information set of the network attacker in the next round is used to obtain the start of the next round based on the smoothed posterior distribution of the information set of the network attacker and the functional relationship between the information sets between rounds. The distribution of the information set of the network attacker at that time is used as the distribution of the information set within the first-order decision point of the opponent in this round.

The adversary action prediction module is used to establish the adversary strategy model of the network attacker based on the information set distribution within the decision point of the network attacker, and use the adversary strategy model of the network attacker to speculate on the actions of the network attacker. The attacker adopts the optimal confrontation strategy for defense.

An electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements any of the above methods.

The above-mentioned opponent action preference estimation method, device and electronic equipment for offensive and defensive games, the method includes: counting the types and proportions of action sequences of both sides of the network in the previous round of network attack and defense; based on the proportion of all action sequences of both parties, Make a two-dimensional approximate estimate of the probability of the action sequence in the previous round; determine the posterior distribution of the network attacker's information set at the end of the previous round based on the probability of the action sequence and the joint distribution of the information sets of the network attacker and defender; According to the smoothed posterior distribution of the network attacker's information set and the functional relationship between the information sets between rounds, the distribution of the network attacker's information set at the beginning of the next round is obtained, and is used as the first-order opponent of the round. The information set distribution within the decision point; based on the information set distribution within the decision point of the network attacker, the network attacker's opponent strategy model is established, and the network attacker's opponent strategy model is used to speculate on the network attacker's actions, and the defender is based on Network attackers use optimal countermeasures to defend themselves. This method breaks through the limitations of the existing assumption of uniform distribution of information sets within decision points, can effectively improve the accuracy of explicit reconstruction of the opponent's strategy, improves the accuracy of predicting the attacker's actions, and thereby improves the pertinence of our defensive strategy. Improve the capabilities and effectiveness of network defense.

Description of the drawings

Figure 1 is a schematic flow chart of an opponent's action preference estimation method for offensive and defensive games in one embodiment;

Figure 2 is a schematic flowchart of an opponent's action preference estimation method within the second-order decision point in the network attack and defense process in one embodiment;

Figure 3 is a structural block diagram of an opponent action preference estimation device for offensive and defensive games in one embodiment;

Figure 4 is an internal structure diagram of an electronic device in one embodiment.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application and are not used to limit the present application.

In the process of network attack and defense, the network attacker's decision is actually made based on the information set. The essence of reconstructing the network attacker's strategy model is to establish the correlation probability model between the network attacker's information set and the network attacker's actions.

Due to the random factors in network attack and defense, at the beginning of a network attack and defense confrontation, the network attacker's hidden information is evenly distributed, so the information set determined by the hidden information is also evenly distributed. Therefore, when the network attack and defense just begins, the decision point within the The information set is evenly distributed. However, when both the network attacker and defender carry out certain actions, as long as the actions of the network attacker and defender are not randomly selected, that is, there is a certain action preference, then the distribution of the network attacker's information set in subsequent decision points will inevitably be affected, instead of uniformly distributed, so to build a more accurate action probability model, the impact of this preference must be considered. The result of this influence is essentially the uneven distribution of the information set within the decision point. If the specific distribution of the network attacker's information set can be accurately estimated, it is equivalent to estimating the impact of the network attacker's action preferences.

Treat the information set as a random variable, convert the action preference estimation problem into the estimation problem of the opponent's information set distribution within the decision point, distinguish the first-order decision point and the second-order decision point, and consider the actions of both parties in the previous round when the round is converted. The influence of action preferences and the preference influence of the actions of both parties before the second-order decision point in a round, through the probability statistics of the action sequence of both parties in the previous round and the action sequence of both parties before the second-order decision point in the current round, using the actions of both parties The two-dimensional approximate probability estimate of the sequence is used to obtain the joint posterior distribution of the information sets of both parties under the condition of the action sequence reaching the next round or the second-order decision point of the current round, and then using the solution marginal distribution to obtain the opponent's information set distribution. The former As the information set distribution within the first-order decision point of the opponent in this round, the latter is used as the information set distribution within the current second-order decision point of the opponent in this round. This method uses the idea of Bayesian modeling to consider the practical problem that the distribution of the opponent's information set in the game is affected by the action preferences of both parties, realizes the estimation of the distribution of the opponent's information set within the decision point, and breaks through the existing uniform information set within the decision point. The limitations of distribution assumptions can effectively improve the accuracy of explicit reconstruction of opponent strategies.

Consider the distribution of the information set as a random variable. The distribution of the information set in the previous round can be used as a density function Representation (Since the number of information sets is large, it can be represented by a continuous random variable I. The method is to map all information sets to the 0-1 real number range in order according to the size of a certain characteristic quantity (such as the expected winning rate). (above), and the information set distribution of the next round can also use a density function/> express. Due to the random factors between stages, the information set of the later round is not equal to the information set of the previous round, so there is a functional relationship between the information set of the previous round and the information set of the later round/ > , if the function is monotonic, the relationship between the distributions of the two rounds of information sets is:/> , this function can be obtained by traversing the information set. The information set distribution at the beginning of the next round can be obtained from the information set distribution at the end of the previous round by this functional relationship.

The initial distribution at the beginning of the previous round Under the influence of the action preferences of both network attackers and defenders, the distribution of information at the end of this round has changed to/> . This distribution change due to the influence of action preferences can be modeled using Bayes' principle. That is, the initial distribution at the beginning of the round can be regarded as the prior, and the distribution at the end of the round can be regarded as the posterior. The probabilities of all action sequences observed in the round can be regarded as under the prior. Conditional Probability. Since some actions of both parties in the previous round may cause the current game to end early and not enter the next round, the information set distribution of the later round is such that it can enter the later round at the end of the previous round. It is converted based on the posterior distribution of one round of action sequence observations. Therefore, we need to obtain the posterior distribution of the opponent's information set at the end of the previous round based on observations that can enter the action sequence of the next round.

due to a turn sequence of actions It is composed of both network attackers and defenders, and is affected by both network attackers and defenders. Therefore, the probability of observing a certain action sequence is under the joint distribution of the information sets of both network attackers and defenders/> conditional probability/> , where the subscript P represents the network defender, and the subscript O represents the network attacker. Due to random factors, at the beginning of a round, the information set distribution of both network attackers and defenders can be regarded as independent, so the joint distribution is through the obtained by multiplying the information set distributions.

In the observation of n game rounds, m kinds of action sequences can be observed. Some of the action sequences (the set of which is Acts) will enable the previous round to enter the next round, so those who can enter the next round The posterior distribution of the information set at the end of the previous round is:

(1)

After the joint distribution of the information sets of both parties that can enter the next round at the end of the previous round is obtained through the above posterior calculation, the information set distribution of the network attacker can be obtained by finding the edge density:

(2)

After obtaining the network attacker information set distribution at the end of the previous round, we can use the functional relationship between the information sets between rounds To get the distribution at the beginning of the next round/> .

Based on the above analysis, it can be seen that the initial distribution of the network attacker's information set in the previous round must be obtained from the initial distribution of the network attacker's information set in the next round. The focus is to obtain the information set of all different types of action sequences in the previous round. Conditional Probability , and on this basis, the posterior joint distribution that can enter the next round of action sequence observation is obtained, and finally the marginal distribution of the network attacker's information set is obtained.

In one embodiment, as shown in Figure 1, an opponent action preference estimation method for offensive and defensive games is provided. The method includes the following steps:

Step 100: Count the types and proportions of action sequences of both network attackers and defenders in the previous round of network attack and defense.

Specifically, the network attack and defense process is essentially a process of multi-stage (round) random repeated games with incomplete information involving both network attack and defense parties.

In the process of reconstructing the network attacker's strategy model, decision points are used as the basis for modeling. Reconstructing the strategy model is equivalent to reconstructing the action probability model at the decision point. Decision points represent similar decision-making scenarios composed of a type of similar information sets, which can be regarded as a collection of identically distributed information sets. Decision points distinguish between first-order decision points and second-order decision points. The first-order decision point only considers the last action before the decision point as the definition of the decision point. Therefore, it is not necessary to consider the influence of the action preferences of both parties in this stage (also called a round), but only needs to consider the influence of the action preferences of both parties in the previous stage. influence, so the first-order decision point only needs to consider the opponent's information set distribution at the beginning of each round. Since the second-order decision point considers the last two actions before the decision point as the definition of the decision point, it is necessary to consider the influence of the action preferences of both parties in the current round to determine the distribution of the information set within it.

Based on the factors that affect the opponent's decision-making in a multi-stage (round) random repeated game with incomplete information involving both network attackers and defenders, the most critical decision-making factor (usually hidden information) in network attack and defense is a variable and other decision-making factors are equal constants. The information set is compressed into a decision point, so the distribution of the information set within the decision point is determined by the most critical decision factors.

Step 102: Based on the proportion of all action sequences of both parties, make a two-dimensional approximate estimate of the probability of the action sequence in the previous round.

Step 104: Based on the probability of the action sequence and the joint distribution of the information sets of the network attacker and defender, determine the posterior distribution of the network attacker's information set at the end of the previous round.

Step 106: According to the smoothed posterior distribution of the network attacker's information set and the functional relationship between the information sets between rounds, obtain the distribution of the network attacker's information set at the beginning of the next round and use it as the round Distribution of information sets within the opponent's first-order decision point.

Step 108: Based on the distribution of the network attacker's information set, establish the network attacker's opponent strategy model, and use the network attacker's opponent strategy model to speculate on the network attacker's actions. The defender adopts the optimal strategy based on the network attacker's actions. Defend against adversarial strategies.

The above-mentioned opponent action preference estimation method for offensive and defensive games, the method includes: counting the types and proportions of action sequences of both network attackers and defenders in the previous round of network attack and defense; based on the proportion of all action sequences of both parties, the previous round Two-dimensional approximate estimation of the probability of the action sequence within The functional relationship between the posterior distribution of the network attacker's information set and the information set between rounds is obtained, and the distribution of the network attacker's information set at the beginning of the next round is obtained, and is used as the information within the first-order decision point of the opponent in that round. Set distribution; based on the information set distribution within the network attacker's decision point, establish the network attacker's opponent strategy model, and use the network attacker's opponent strategy model to speculate on the network attacker's actions. The defender adopts the strategy based on the network attacker's actions. Optimal countermeasures for defense. This method breaks through the limitations of the existing assumption of uniform distribution of information sets within decision points, can effectively improve the accuracy of explicit reconstruction of the opponent's strategy, improves the accuracy of predicting the actions of the network attacker, and thereby improves the pertinence of our defense strategy. , improve the capabilities and effectiveness of network defense.

In one embodiment, step 100 includes: counting the type and number of action series of the network attacker and defender in the previous round in the data observed during multiple historical confrontations between the network attacker and defender; based on the types and numbers of all action sequences, Determine the proportion of each type of action sequence as:

(3)

is the ratio of the number of action sequence acts to the total number of action sequences,/> is the number of action sequences of a type;/> is the total number of action sequences;/> It is the conditional probability of the action sequence under the joint distribution of the information sets of the network attacker and defender;/> It is the joint distribution of information sets between network attackers and defenders; They are the information sets of the previous round of network attack and defense respectively. The subscript P represents the defender, the subscript O represents the network attacker or opponent, and the subscript pre represents the previous round.

By counting all types of action sequences, the expected probability of a type of action sequence can be obtained by using the ratio of the number of that action sequence to the total number of action sequences.

This ratio is actually also a conditional probability of points. Since the left-hand term in equation (3) is known, the conditional probability in the integral of the right-hand term can be approximately estimated through certain assumptions.

In one embodiment, step 102 includes: setting the general decision-making logic and action sequence of the action to be uniform within the corresponding two-dimensional interval of the information set; according to the proportion of the action sequence of all decision points, the general decision-making logic and action sequence of the action The distribution within the corresponding two-dimensional interval of the information set estimates the probability of the action sequence.

Specifically, since both parties make decisions based on their own information sets, different action choices often represent different own information sets. Considering that if the value range of the information sets of both parties is known, and It is known that if the conditional probability of an action sequence can be estimated based on general action decision-making logic, the action probability estimate can be obtained over the value range of the entire information set.

In the general action decision-making logic, different actions of the network attacker and defender are selected based on the win rate of the information set (i.e., the success rate of the action). When the win rate of the information set is large, it is easier to take higher-risk actions, and vice versa. It is easier to take low-risk actions. If we consider that each action sequence will correspond to a specific information set of both parties, that is, use a interval/> Approximately/> , then in complete/> The value range is covered by the intervals corresponding to different action sequences:

(4)

The occurrence probability of each action sequence is equal to the integral within the specific interval corresponding to the action sequence:

(5)

Considering the joint distribution in equation (5) Known, conditional probability/> It is also determined by the interval of each action sequence, then according to/> You can get the area size of the two-dimensional interval of the sequence. Therefore, as long as the intervals corresponding to all sequences are arranged according to the order of actions in all sequences and the preferences of the information set corresponding to each action (that is, arranged according to the win rate of the information set), the two-dimensional completeness of the information set can be obtained The probability of a sequence of actions over a range of values.

Therefore, when all action sequences are divided, the conditional probabilities of all action sequences within the complete value range of the network defender's and attacker's information sets can be obtained .

In one embodiment, step 104 includes: determining the posterior distribution of the information set of the network attacker at the end of the previous round based on the probability of the action sequence and the joint distribution of the information sets of both network attackers and defenders, including: according to the information Approximate probabilities of action sequences in set two-dimensional space , and the set of action sequences Acts that can enter the next round, and obtain the posterior joint distribution of the information sets of both parties at the end of the previous round:

(6)

in, is the posterior joint distribution of the information sets of both parties at the end of the previous round,/> is the prior joint distribution of the information sets of both network attackers and defenders in the previous round, is the approximate probability of the action sequence in the two-dimensional space of the information set,/> Set the action sequence for the next round,/> for each sequence of actions;

The posterior distribution of the information set of the network attacker is obtained based on the marginal distribution of the joint posterior distribution:

(7)

in, It is the information set distribution of the network attacker at the end of the previous round.

In one embodiment, step 106 includes: smoothing the posterior distribution of the network attacker's information set using a preset kernel function to obtain a smooth nonlinear density function:

(8)

in, is the smooth nonlinear density function of the previous round,/> is the preset kernel function, and the subscript j is the index after uniform discretization of the information set,/> is the posterior distribution of the information set of the network attacker in the previous round,/> It is the information set of the network attacker in the previous round.

Specifically, since the two-dimensional approximation of the action sequence is assumed to be uniformly distributed within the interval, the posterior distribution of the information set of the opponent O is a piecewise linear function, so through a smoothing process, the density function of the distribution becomes Smooth nonlinear density function, as shown in equation (8).

Then we can use the functional relationship between the information sets between rounds To get the distribution at the beginning of the next round/> , and serve as the information set distribution within the first-order decision point of the opponent in this round:

(9)

in, is the distribution of the network attacker’s information set at the beginning of the next round, is the functional relationship between the information set between the previous round and the next round,/> It is the information set of the network attacker at the beginning of the next round.

In one of the embodiments, as shown in Figure 2, the method also includes a step of estimating the opponent's action preference at the second-order decision point within a round, which specifically includes the following steps:

Step 200: Count the types and proportions of all action sequences in the network attack and defense process before the second-order decision point.

Specifically, the second-order decision point uses the two actions before the current decision point as the definition of the decision point, so it is necessary to consider the influence of the action preferences of both parties before the decision point in the current stage.

Step 202: Based on the proportion of all action sequences before the second-order decision point in the current round, make a two-dimensional approximate estimate of the probability of the action sequences of both the network attacker and defender.

Step 204: Based on the probability of the action sequence of both network attackers and defenders and the joint prior distribution of the information sets of both network attackers and defenders, determine the joint posterior distribution of the information sets of both parties within the second-order decision point affected by the previous action preference, and based on the joint posterior distribution The marginal distribution of determines the information set distribution within the second-order decision point of the network attacker.

In one embodiment, step 202 includes: according to the general decision-making logic of the actions of both parties, setting the action sequences to be uniformly distributed within the two-dimensional interval of the corresponding information set, and determining the two-dimensional information corresponding to all action sequences according to the proportion of the action sequences. Set interval.

In one embodiment, step 204 includes: based on the sequence of actions that can enter the second-order decision point , determine the joint posterior distribution of the information sets of both network attackers and defenders within the second-order decision point:

(10)

in, is the joint posterior distribution of the information sets of both network attackers and defenders within the second-order decision point,/> is the conditional distribution of the action sequences of both network attackers and defenders before the second-order decision point that can enter the second-order decision point,/> is the joint prior distribution of the information sets of both parties at the beginning of the current round,/> For action sequences/> The total probability of is equal to/> ;

Determine the information set distribution within the second-order decision point of the network attacker based on the marginal distribution of the joint posterior distribution:

(11)

in, is the information set distribution within the second-order decision point of the network attacker.

The information set distribution estimation of the network attacker at the second-order decision point is similar to the preference estimate for the actions of both parties in the previous round when the round is switched, but it does not need to consider the action sequence of both parties in the entire stage, as long as the actions before the second-order decision point are considered The action sequence of both parties, and there is no need to use the functional relationship of the information set between rounds to perform variable transformation on the information set.

It should be understood that although the steps in the flowcharts of Figures 1 and 2 are shown in sequence as indicated by arrows, these steps are not necessarily executed in the order indicated by arrows. Unless explicitly stated in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in Figures 1 and 2 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. These sub-steps or The execution order of the stages is not necessarily sequential, but may be performed in turn or alternately with other steps or sub-steps of other steps or at least part of the stages.

In one embodiment, as shown in Figure 3, an opponent action preference estimation device for offensive and defensive games is provided, including: a ratio determination module 301 for the action sequences of both network attackers and defenders, and a probability estimation module for the action sequences of both network attackers and defenders. 302. The posterior distribution determination module 303 of the current round information set of the network attacker, the distribution determination module 304 of the network attacker's next round information set, and the opponent action estimation module 305, where:

The proportion determination module 301 of the action sequences of both network attackers and defenders is used to count the types and proportions of the action sequences of both network attackers and defenders in the previous round of network attack and defense.

The probability estimation module 302 of the action sequences of both sides of the network is used to make a two-dimensional approximate estimate of the probability of the action sequence in the previous round based on the proportion of all action sequences of both parties.

The posterior distribution determination module 303 of the current round information set of the network attacker is used to determine the posterior distribution of the network attacker's information set at the end of the previous round based on the probability of the action sequence and the joint distribution of the information sets of both network attackers and defenders. posterior distribution.

The next round information set distribution determination module 304 of the network attacker is used to obtain the next round based on the smoothed posterior distribution of the network attacker's information set and the functional relationship between the information sets between rounds. The distribution of the information set of the network attacker at the beginning is used as the distribution of the information set within the first-order decision point of the opponent in this round.

The adversary action prediction module 305 is used to establish the adversary strategy model of the network attacker based on the information set distribution within the decision point of the network attacker, and use the adversary strategy model of the network attacker to speculate on the actions of the network attacker. Network attackers use optimal countermeasures to defend themselves.

In one embodiment, the ratio determination module 301 of the action sequences of the network attacker and defender is also used to count the types and types of the action series of the network attacker and defender in the previous round in the data observed during the multi-game historical confrontation between the network attacker and defender. Quantity; based on the types and quantities of all action sequences, determine the proportion of each type of action sequence as shown in Equation (3).

In one embodiment, the probability estimation module 302 of the action sequences of both sides of the network is also used to set the general decision-making logic of the action and the action sequence to be uniform within the two-dimensional interval of the corresponding information set; according to all action sequences of both parties The probability of the action sequence is estimated based on the proportion, the general decision logic of the action and the distribution of the action sequence within the corresponding two-dimensional interval of the information set.

According to the general decision-making logic of the actions of both parties, the approximate probability of setting the action sequence within the two-dimensional interval of the corresponding information set is:

(12)

in, is the approximate probability of the action sequence in the two-dimensional space of the information set,/> ,/> They are the information sets of the network attacker and defender respectively;/> ,/> They are the two-dimensional interval range of the information set corresponding to the action sequences of the network attacker and defender respectively. The probability of each action sequence of acts should satisfy:

(13)

in, is the proportion of the action sequence,/> It is the prior joint distribution of the information sets of the network attacker and defender in the previous round;

According to the proportion of all action sequences of both parties and the general decision-making logic of the actions, the two-dimensional interval range of the information set corresponding to each action sequence act of both sides of the network is iteratively obtained by two equations, and then the two-dimensional interval range of the complete information set on the two-dimensional interval is obtained Approximate probabilities for action sequences.

In one of the embodiments, the posterior distribution determination module 303 of the current round information set of the network attacker is also used to determine the posterior distribution of the network attacker's information set at the end of the previous round according to the two-dimensional information set. Approximate probabilities of action sequences in space , and the action sequence set Acts that can enter the next round. The posterior joint distribution of the information sets of both parties at the end of the previous round is obtained as shown in Equation (6). The posterior distribution of the network attacker's information set is obtained based on the marginal distribution of the joint posterior distribution. The posterior distribution of the network attacker's information set is shown in Equation (7).

In one embodiment, the network attacker's information set distribution determination module 304 is also used to smooth the posterior distribution of the network attacker's information set using a preset kernel function to obtain a smooth The nonlinear density function is shown in equation (8). According to the smoothed posterior distribution of the network attacker's information set and the functional relationship between the information sets between rounds, the distribution of the network attacker's information set at the beginning of the next round is obtained, and is used as the opponent in that round The information set distribution within the first-order decision point is shown in Equation (9).

In one embodiment, the device further includes an opponent action preference estimation module at a second-order decision point within a round, which is used to count the types and proportions of all action sequences in the network attack and defense process before the second-order decision point; according to the current round The proportion of all action sequences before the intra-second order decision point is used to make a two-dimensional approximate estimate of the probability of the action sequences of both network attackers and defenders; based on the probability of the action sequences of both network attackers and defenders and the joint prior distribution of the information sets of both network attackers and defenders, Determine the joint posterior distribution of the information sets of both parties within the second-order decision point affected by the previous action preference, and determine the distribution of the information set within the second-order decision point of the network attacker based on the marginal distribution of the joint posterior distribution.

In one of the embodiments, the opponent action preference estimation module of the second-order decision point within a round is also used to set the action sequence to be uniform within the two-dimensional interval of the corresponding information set based on the general decision-making logic of the actions of both parties. According to the action The proportion of the sequence determines the two-dimensional information set interval corresponding to all action sequences. .

In one embodiment, the opponent's action preference estimation module for the second-order decision point within a round is also used to determine the action sequence that can enter the second-order decision point. , determine the joint posterior distribution of the information sets of both network attackers and defenders within the second-order decision point, as shown in Equation (10). According to the marginal distribution of the joint posterior distribution, the information set distribution of the network attacker within the second-order decision point is determined as shown in Equation (11).

Regarding the specific limitations of the opponent action preference estimation device for offensive and defensive games, please refer to the limitations of the opponent action preference estimation method for offensive and defensive games mentioned above, which will not be described again here. Each module in the above-mentioned opponent action preference estimation device for offensive and defensive games can be implemented in whole or in part by software, hardware, and combinations thereof. Each of the above modules may be embedded in or independent of the processor of the computer device in the form of hardware, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

In one embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. 4 . The electronic device includes a processor 401, a memory 402, a network interface 403, a display screen 404 and an input device 405 connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory 402 of the electronic device includes a non-volatile storage medium 4022 and an internal memory 4021. The non-volatile storage medium 4022 stores operating systems and computer programs. The internal memory 4021 provides an environment for the execution of operating systems and computer programs in non-volatile storage media. The network interface 403 of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor 401, it implements an opponent's action preference estimation method for offensive and defensive games. The display screen 404 of the electronic device may be a liquid crystal display or an electronic ink display, and the input device 405 of the electronic device may be a touch layer covered on the display screen 404, or may be a button, trackball, or The touchpad can also be an external keyboard, trackpad or mouse.

Those skilled in the art can understand that the structure shown in Figure 4 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. Specific computer equipment can May include more or fewer parts than shown, or combine certain parts, or have a different arrangement of parts.

In one embodiment, an electronic device is provided, including a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in any of the above method embodiments are implemented.

The technical features of the above embodiments can be combined in any way. To simplify the description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, all possible combinations should be used. It is considered to be within the scope of this manual.

The above-described embodiments only express several implementation modes of the present application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all fall within the protection scope of the present application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims (10)

1. An opponent action preference estimation method facing attack and defense games is characterized by comprising the following steps:

counting the types and the proportions of action sequences of both network attack and defense parties in the previous round of network attack and defense process;

according to the proportion of the action sequences of all the two parties, carrying out two-dimensional approximate estimation on the probability of the action sequence in the previous round;

determining posterior distribution of the information set of the network attack party at the end of the previous round according to the probability of the action sequence and the joint distribution of the information sets of the network attack and defense parties;

obtaining the distribution of the information set of the network attacker at the beginning of the next round according to the posterior distribution of the information set of the network attacker after the smoothing and the functional relation of the information set between rounds, and taking the distribution as the information set distribution in the first-order decision point of the opponent of the round;

according to information set distribution in decision points of the network attacker, an opponent strategy model of the network attacker is established, the opponent strategy model of the network attacker is adopted to infer the action of the network attacker, and the defender defends by adopting an optimal countermeasure strategy according to the action of the network attacker.

2. The method of claim 1, wherein counting the type and proportion of the sequence of actions of both network attacks and defenses in the previous round of network attack and defenses comprises:

Counting the types and the quantity of action sequences of the network attack and defense parties in the previous round in the data observed in the multi-office historical countermeasure process of the network attack and defense parties;

according to the types and the quantity of all the action sequences, determining the proportion of each type of action sequences as follows:

，

wherein,for the ratio of the number of action sequences acts to the total number of action sequences, +.>Number of act sequences for a class; />Is the total number of moving sequences; />The method comprises the steps of providing action sequence conditional probability under the joint distribution of information sets of both the attack and the defense of the network; />The method is the joint distribution of information sets of both the network attack and defense parties; />Respectively information sets of the previous round of network attack and defense parties, subscriptsPRepresenting the defending party, subscriptORepresenting network aggressors, i.e. adversaries, subscriptspreRepresenting the previous round.

3. The method of claim 1, wherein estimating the probability of the action sequence in the previous round in two-dimensional approximation based on the proportions of all the two-party action sequences comprises:

according to general decision logic of actions of both sides, setting approximate probability of the action sequence in the corresponding two-dimensional interval of the information set as follows:

，

wherein,approximately probability of action sequence in two-dimensional space for information set,/->、/>Respectively information sets of both network attack and defense parties; / >Action sequences of both network attack and defense parties respectivelyactsCorresponding two-dimensional interval range of information set, each action sequenceactsThe probability of (2) should satisfy:

，

wherein,for the proportion of action sequences, +.>The prior joint distribution of the information sets of the two attack and defense parties of the previous round of internal network is realized;

according to the proportion of all the action sequences of both parties and the general decision logic of actions, each action sequence of both network attack and defense parties is obtained by two-mode iterationactsThe corresponding two-dimensional interval range of the information set further obtains the approximate probability of the action sequence on the two-dimensional interval of the complete information set.

4. The method of claim 1, wherein determining the posterior distribution of the information set of the network attacker at the end of the previous round based on the probability of the sequence of actions and the joint distribution of the information set of both the network attacks and the defenders, comprises:

according to the approximate probability of the action sequence on the two-dimensional space of the information set and the action sequence set which can enter the next round, obtaining posterior joint distribution of the information sets of the two parties at the end of the previous round:

wherein (1)>For posterior joint distribution of both sets of information at the end of the previous round,for the prior joint distribution of the information sets of the two parties of the network attack and defense in the previous round, the method comprises the step of +. >Respectively information sets of the previous round of network attack and defense parties, subscriptsPRepresenting the defending party, subscriptORepresenting network aggressors, i.e. adversaries, subscriptspreRepresenting the previous round;

the posterior distribution of the information set of the network attacker is obtained according to the edge distribution of the joint posterior distribution:

，

wherein,and distributing information sets of network aggressors at the end of the previous round.

5. The method according to claim 1, wherein obtaining the distribution of the information set of the network attacker at the beginning of the next round as the information set distribution in the first-order decision point of the opponent of the round according to the posterior distribution of the information set of the network attacker after smoothing and the functional relation of the information set between rounds includes:

the posterior distribution of the information set of the network attacker is smoothed by adopting a preset kernel function, and a smooth nonlinear density function is obtained as follows:

，

wherein,for a nonlinear density function of the previous round of smoothing, +.>For a preset kernel function, subscriptsjFor the index after uniform discretization of the information set, < >>Posterior distribution of information set for previous round network aggressor, ++>The information set is the information set of the network attacker of the previous round;

according to the posterior distribution of the information set of the network attacker after the smoothing treatment and the functional relation of the information set between turns, the distribution of the information set of the network attacker at the beginning of the next turn is obtained and is used as the information set distribution in the first-order decision point of the opponent of the turn:

，

Wherein,for the distribution of the information set of the network aggressor at the beginning of the next round +.>For the functional relation of the information set between the previous round and the following round,/for the information set between the previous round and the following round>Is the information set of the network attacker at the beginning of the next round.

6. The method of claim 1, further comprising an opponent action preference estimation within a second order decision point within the current round, the steps comprising:

counting the types and proportions of all action sequences in the network attack and defense process before the second-order decision point;

according to the proportion of all the action sequences of the two parties before the second-order decision point in the current round, carrying out two-dimensional approximate estimation on the probability of the action sequences of the network attack and defense parties;

and determining the joint posterior distribution of the information sets of the two parties in the second-order decision point influenced by the previous action preference according to the probability of the action sequences of the two parties and the joint prior distribution of the information sets of the two parties, and determining the information set distribution in the second-order decision point of the network attacker according to the edge distribution of the joint posterior distribution.

7. The method of claim 6, wherein performing a two-dimensional approximate estimation of the probability of the network attack and defense party action sequences based on the proportion of all action sequences preceding the second-order decision point in the current round comprises:

According to the general decision logic of the actions of the two parties, the action sequences are set to be uniformly distributed in the two-dimensional intervals of the corresponding information sets, and according to the proportion of the action sequences, the two-dimensional information set intervals corresponding to all the action sequences are determined.

8. The method of claim 6, wherein determining a joint posterior distribution of the two party information sets within the second order decision points affected by the previous action preference based on the probability of the network attack and defense party action sequence and the joint prior distribution of the network attack and defense party information sets, and determining an information set distribution within the second order decision points of the network attack party based on an edge distribution of the joint posterior distribution, comprises:

based on a sequence of actions that can enter a second order decision pointDetermining joint posterior distribution of information sets of both the network attack and defense parties in a second-order decision point:

，

wherein,for the joint posterior distribution of information sets of both the network attack and defense parties in the second-order decision point, the ++>、/>Information sets of both network attack and defense parties respectively, < ->Is the conditional distribution of the action sequences of the network attack and defense parties which can enter the second-order decision point before the second-order decision point, +.>Combine a priori distribution for both party information sets at the beginning of the current round,/->For action sequence->Is equal to the full probability of ；

Determining information set distribution in second-order decision points of network attack parties according to edge distribution of joint posterior distribution:

，

wherein,is the information set distribution in the second order decision point of the network attacker.

9. An opponent action preference estimation device facing an attack and defense game, which is characterized by comprising:

the proportion determining module of the action sequences of the network attack and defense parties is used for counting the types and proportions of the action sequences of the network attack and defense parties in the previous round of network attack and defense process;

the probability estimation module of the action sequences of the network attack and defense parties is used for carrying out two-dimensional approximate estimation on the probability of the action sequences according to the proportion of all the action sequences of the previous round or all the action sequences before the second-order decision point of the current round;

the posterior distribution determining module of the current round information set of the network attacker is used for determining the posterior distribution of the information set of the network attacker on the current round second-order decision point or the posterior distribution of the information set of the network attacker at the end of the previous round through edge distribution solving according to the probability of the action sequence and the joint posterior distribution of the information sets of the network attack and defense parties;

the next round of information set distribution determining module of the network attacker is used for obtaining the distribution of the information set of the network attacker at the beginning of the next round according to the posterior distribution of the information set at the end of the previous round of the network attacker after the smoothing and the functional relation of the information sets between the rounds, and taking the distribution as the information set distribution in the first-order decision point of the opponent of the round;

And the opponent action presumption module is used for establishing an opponent strategy model of the network attacker according to the distribution of the information sets in the decision points of the network attacker, presuming the action of the network attacker by adopting the opponent strategy model of the network attacker, and defending by adopting an optimal countermeasure strategy according to the action of the network attacker.

10. An electronic device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the method of any one of claims 1 to 8 when executing the computer program.