CN111726192B - Communication countermeasure medium frequency decision optimization method based on log linear algorithm - Google Patents
Communication countermeasure medium frequency decision optimization method based on log linear algorithm Download PDFInfo
- Publication number
- CN111726192B CN111726192B CN202010534345.0A CN202010534345A CN111726192B CN 111726192 B CN111726192 B CN 111726192B CN 202010534345 A CN202010534345 A CN 202010534345A CN 111726192 B CN111726192 B CN 111726192B
- Authority
- CN
- China
- Prior art keywords
- user
- enemy
- party
- frequency
- communication
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004891 communication Methods 0.000 title claims abstract description 93
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000004422 calculation algorithm Methods 0.000 title claims abstract description 57
- 238000005457 optimization Methods 0.000 title claims abstract description 14
- 230000008569 process Effects 0.000 claims abstract description 34
- 238000005381 potential energy Methods 0.000 claims abstract description 29
- 238000001228 spectrum Methods 0.000 claims abstract description 16
- 230000000694 effects Effects 0.000 claims abstract description 11
- 230000006870 function Effects 0.000 claims description 40
- 230000008859 change Effects 0.000 claims description 17
- 238000005562 fading Methods 0.000 claims description 6
- 230000009191 jumping Effects 0.000 claims description 4
- 230000008447 perception Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 7
- 238000011160 research Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000000052 comparative effect Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 241000359496 Kaistibacter Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04K—SECRET COMMUNICATION; JAMMING OF COMMUNICATION
- H04K3/00—Jamming of communication; Counter-measures
- H04K3/80—Jamming or countermeasure characterized by its function
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04K—SECRET COMMUNICATION; JAMMING OF COMMUNICATION
- H04K3/00—Jamming of communication; Counter-measures
- H04K3/20—Countermeasures against jamming
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04K—SECRET COMMUNICATION; JAMMING OF COMMUNICATION
- H04K3/00—Jamming of communication; Counter-measures
- H04K3/20—Countermeasures against jamming
- H04K3/22—Countermeasures against jamming including jamming detection and monitoring
- H04K3/224—Countermeasures against jamming including jamming detection and monitoring with countermeasures at transmission and/or reception of the jammed signal, e.g. stopping operation of transmitter or receiver, nulling or enhancing transmitted power in direction of or at frequency of jammer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04K—SECRET COMMUNICATION; JAMMING OF COMMUNICATION
- H04K2203/00—Jamming of communication; Countermeasures
- H04K2203/10—Jamming or countermeasure used for a particular application
- H04K2203/24—Jamming or countermeasure used for a particular application for communication related to weapons
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
The invention discloses a medium frequency decision optimization method in communication countermeasure based on a logarithmic linear algorithm, which comprises the following steps: establishing a communication scene of the users of the enemy and the my, and setting the users of the enemy and the enemy as the game parties; the interference countermeasure problem of the users of both the enemy and the me is expressed as a Stackelberg game process by supposing that the frequency selection strategy of the current frequency spectrum environment is adjusted to maximize the effect of the current frequency spectrum environment, and a game model is constructed; the constructed game model is proved to be an accurate potential energy game; designing a distributed channel selection algorithm based on logarithmic linear learning, wherein the algorithm is used for frequency decision of enemy users and my users to obtain a balanced solution of the modeled game problem; and carrying out convergence certification on the designed algorithm. The method has better convergence, can reduce the time required by the frequency decision of the two parties, and can achieve the effect of maximizing the frequency decision of the two parties of the game according to the convergence result.
Description
Technical Field
The invention relates to the technical field of electronic battlefield wireless communication countermeasure, in particular to a medium frequency decision optimization method for communication countermeasure based on a log-linear algorithm.
Background
In recent years, due to scarcity of spectrum resources, the spectrum resources become necessary resources for military competition, and in a complex electromagnetic environment, how to guarantee the spectrum use safety is a key problem in a wireless communication network. Meanwhile, the forms of the electronic battlefield are complex and changeable, the military parties master interference technologies and anti-interference technologies of different degrees, the frequency utilization safety of the military parties faces interference threats of different types and degrees, how to attack the other party through the interference of the own party and rob the frequency spectrum resources of the enemy, or the anti-interference technology avoids the interference of the other party, the frequency utilization safety of the user of the own party is effectively protected, and the fact that the own party has a preoccupation in the electronic battlefield is a research hotspot in the current wireless communication countermeasure direction.
At present, a lot of achievements are made on the research of anti-interference problems. In terms of frequency decision, m.strasser proposed uncoordinated frequency hopping communication anti-interference in 2009, but the anti-interference efficiency of the method is low, and a more efficient method should be pursued in the environment of today with tight spectrum resources. E.k.lee proposed in 2010 a random frequency hopping interference rejection method for communication, which, although considering a dynamic spectrum environment, does not consider the intelligence of users and interference. And the H.Li researches a channel selection strategy under the condition of simulating the attack by the main user in 2010, and models the channel selection between the user and the attacker as a zero sum game. Wu in 2012 researches the anti-interference problem in the cognitive radio network and provides an effective frequency use decision scheme. Chen proposes a game theory anti-interference scheme under the unmanned aerial vehicle ad hoc network interference scene in 2013, and designs an anti-interference frequency-using decision scheme, wherein the scheme only considers the interference inside the unmanned aerial vehicle user, but ignores the malicious interference existing in an external interference machine. M.a.aref uses a standard single-user learning method in 2017, each user learns independently, but there is a certain disadvantage because Q learning requires that the state is stable, but the state transition of the Q learning is changed continuously due to the learning of other users under the multi-user condition, and when the number of users increases, it is difficult to ensure the fast convergence of the Q learning algorithm. In view of interference of a malicious jammer, l.jia proposed a hierarchical learning method for interference resistance frequency decision in 2017, which considers both internal mutual interference and external malicious interference, but does not consider the problem that the user group of the l.jia, as an jammer in the user group, may also have internal mutual interference and external malicious interference.
In an electronic battlefield, the target income obtained by both confrontation parties is determined at the moment of frequency decision, and in a highly dynamic electronic battlefield, dynamic frequency decision is indispensable. The traditional communication anti-interference decision method is difficult to meet the requirements in a battlefield. In summary, the existing electronic battlefield wireless communication countermeasure model and scheme mainly have the following problems: 1) the model is not complete enough, on an electronic battlefield, mutual interference between the users of the enemy and the my user and a task of controlling an interference machine to interfere with the enemy are indispensable, the enemy and the my party exist on the battlefield, and the internal mutual interference needs to be considered while the enemy is interfered by the my party. 2) The intelligence of all users is not considered, and intelligent users can make more intelligent frequency utilization decisions according to the result of the spectrum environment conjecture. 3) With the intelligent upgrade of equipment, the party with faster and better intelligent decision making of both parties in the battle will take the advantage of time-first, and an algorithm for effectively improving the decision making speed needs to be researched.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a communication countermeasure intermediate frequency decision optimization method based on a logarithmic linear algorithm, which considers that both enemy and my parties are intelligent agents in actual countermeasure and can conjecture the frequency of the other party as the reference of the frequency decision of the own party. Because the game of the two sides of the enemy and the my has the precedence sequence, the Stackelberg game model can be used as a frame to carry out problem modeling on the frequency utilization decision process of the two sides of the enemy and the my in turn, and the game considers the competition of two aspects: one is the competition between my jammer and the enemy user, and one is the competition between my own user (including jammer). And simultaneously, the existence of the proposed Stackelberg game equilibrium solution is proved, and the obtained equilibrium solution is a global or local optimal solution. In order to solve the established game problem equilibrium solution, the frequency decision is carried out on the enemy and the user of the enemy by adopting an algorithm based on log-linear Learning, and compared with an algorithm based on a random Learning machine SLA (stochastic Learning Automata), the log-linear Learning distributed algorithm provided by the invention can obviously improve the convergence of the algorithm.
In order to achieve the purpose, the invention adopts the following technical scheme:
a medium frequency decision optimization method in communication countermeasure based on a log linear algorithm, the optimization method comprising:
s1, establishing a friend or foe user communication scene, setting independent my user and friend or foe user as game two sides, and carrying out frequency use decision process by friend or foe in turn, wherein in each counter-countermeasure process, the frequency use condition of the other side can be used as the reference of own frequency use decision by conjecture/perception; the enemy user comprises M communication transceiving pairs;
s2, on the basis of adjusting a frequency selection strategy of a party by conjecturing/sensing a current spectrum environment and maximizing the effect of the frequency selection strategy, expressing the interference countermeasure problem of the users of the two parties of the enemy and the my as a Stackelberg game process, and constructing a game model; for the user of the same party, the game model aims to minimize the mutual interference among the users of the same party and maximize the interference of the interference machine of the same party on the user of the opposite party, and for the user of the opposite party, the game model aims to minimize the mutual interference among the users of the opposite party and minimize the interference of the interference machine of the same party on the own party;
s3, proving that the constructed game model is an accurate potential energy game;
s4, designing a distributed channel selection algorithm based on log-linear learning, wherein the algorithm is used for frequency decision of enemy users and my users to obtain a balanced solution of the established game problem;
s5, convergence verification is performed on the algorithm designed in step S4.
In order to optimize the technical scheme, the specific measures adopted further comprise:
further, in step S1, the process of constructing the communication scenario between the friend or foe user and the friend or foe user includes:
s11, it is assumed that my party user includes N communication transceiving pairs and a jammer, and the enemy party user includes M communication transceiving pairs, whereinA set of numbers representing a total of N +1 users of my party,the method comprises the steps that N communication transceiving pair sets of our party are represented, and a user of our party with the number of N +1 represents an interference machine of our party;representing a set of M communication transmit-receive pairs of enemies, and the user instantaneous channel gain is expressed asWherein d isx,yRepresenting the distance between user x and user y, alpha andrespectively representing a path fading coefficient and a Rayleigh fading coefficient;
s12, the frequency use decision process is carried out by the two sides of the enemy and the my side in sequence, wherein the enemy is used as the weak side to carry out decision firstly, the party is used as the strong side to carry out final decision, and the frequency use condition of the other side can be used as the reference of the frequency use decision of the own side by conjecture in the process of each pair of the counteractions.
Further, the frequency selection process is carried out by the two parties of the enemy and the my according to the following steps:
s121, randomly selecting frequency by the user of one party;
s122, the enemy user decides a frequency selection strategy of the enemy user according to the perceived frequency selection of the previous interference machine of the enemy;
s123, the user of the party decides the frequency selection strategy of the user of the party according to the presumed frequency selection of the user of the enemy;
and S124, circulating the steps b and c until the countermeasure is finished, wherein each circulation is a one-wheel countermeasure process.
Further, in step S2, the process of constructing the game model, which takes the principle of adjusting the frequency selection policy of the other party by inferring the current spectrum environment to maximize the utility of the other party, and expresses the interference countermeasure problem of the users of the two parties of the enemy and me as the Stackelberg game process, includes the following steps:
s21, frequency block for use in communication countermeasure of two users of friend and foeA policy Stackelberg game is defined as:whereinAndrespectively representing a set of my users and enemy users,representing respective alternative channel sets, u, of my and enemynAnd umRespectively representing the utility functions of a user n of the my party and a user m of the enemy party;
s22, for the my user, sub-gaming the my userIs defined as:in the formula, an、a-nAnd amRespectively representing the frequency selection of a user n of the my party, other users of the my party except the user n and an enemy user m;
s23, for user n, the purpose is to minimize the mutual interference between users and maximize the interference of jammers to users of enemies, and its utility unIs defined as:
wherein L isuAnd LjIs a predetermined normal number, preventing the target utility value from being less than zero, PnAnd PN+1Respectively representing the transmit power of my user n and my jammer,andrespectively representing the instantaneous channel gain between the users of the my party, the instantaneous gain between the communication pair of the my party and the jammer of the my party and the instantaneous channel gain between the jammer of the my party and the communication pair of the enemy party; zbAnd ZrRespectively representing the degree of interference caused by the antenna side lobe of the jammer at one party to the communication at one party and the degree of interference caused by the antenna main lobe of the jammer at one party to the user at the enemy party, and Zb<Zr;f(ax,ay) The indication function of whether the frequency selection results are the same is shown, and the expression is as follows:
s24, defining the established sub game model of the my user as the goal of maximizing the utility function of each user, namely
S25, for the enemy user, sub-gaming the enemy userIs defined as:am、a-mrespectively representing the frequency selection of an enemy user m and other enemy users except the user m;
s26, for enemy user m, its purpose is to minimize the mutual interference between enemy users and minimize the interference of our disturber to own party, its utility umIs defined as:
wherein L isuIs a predetermined normal number, preventing the target utility value from being less than zero, PmAnd PN+1Respectively representing enemy user n and myThe transmission power of the jammer is set,andrespectively representing instantaneous channel gain between enemy users and instantaneous channel gain between the my jammer and the enemy users;
s27, the aim of the established enemy user sub-game model is defined to maximize the utility function of each enemy user, namely
Further, in step S3, the process of proving that the constructed game model is an accurate potential energy game includes the following steps:
s31, sub-gaming on enemy usersIn (1), policy a for adversary user mmDefining a potential energy function phil:
φl=φl,1+φl,2
When enemy user m unilaterally sends policy amIs changed intoCalculating whether the change of the corresponding potential energy function is the same as the change of the utility function, and if so, establishing a game modelIs an accurate potential energy game;
s32, sub-gaming the user on my sideFor policy a of my user nnDefining a potential energy function phif:
φf=φf,1+φf,2
Wherein phif,1And phif,2H (N, N +1), y (N, N +1) and g (N, N +1) in (1) are defined as:
when one side of my user n is to make policy anIs changed intoCalculating whether the change of the corresponding potential energy function is the same as the change of the utility function, and if so, establishing a game modelIs an accurate potential energy game.
Further, in step S4, the process of designing the distributed channel selection algorithm based on log linear learning includes the following steps:
s41, initializing, making the iteration number k equal to 1, and selecting channel combination with equal probability from the available channel set according to the communication requirement of the user set
S42, the user set randomly selects a channel combination from its available channel setAnd calculates its combination in the channelOverall utility of the case (1)
S43, the user set updates its channel selection probability according to the following rule:
wherein β is a learning parameter;
and S44, making k equal to k +1, looping steps S42 and S43 until k reaches the maximum iteration number or the network utility change value is smaller than the set change threshold value, and jumping out of the loop.
Further, in step S5, the process of proving convergence of the algorithm designed in step S4 includes the following steps:
s51, converting the channel selection probability formula into:
s52, setting the value of beta, wherein beta is positive number, the specific value is determined by environment parameter, if the channel combination is randomly selectedIs less effective than the last channel combination c (k)I.e. byMuch less than 1 and close to 0, when c (k +1) equals c (k) with a probability close to 1, a randomly selected combination of channels is selectedThe probability of (a) is close to 0; if the utility value of the random selection is greater than the last value, then the random selection is favored;
and S52, optimizing the value of beta through continuous iterative learning, making a frequency-using decision with a larger utility value according to the probability, and converging the utility result to the maximum value at the moment.
The invention has the beneficial effects that:
the invention constructs the frequency decision optimization problem of communication fighting against both enemies and my parties in an electronic battlefield into a Stackelberg game model, further converts the frequency decision optimization problem into a game optimization problem, comprehensively considers the intelligence of users, adopts a reverse induction method to solve, and solves the modeled Stackelberg game problem equilibrium solution by designing a distributed channel selection algorithm based on logarithmic linear learning, and simulation shows that compared with other algorithms, the proposed algorithm has better convergence, can reduce the time required by the frequency decision of both parties, and can maximize the convergence result.
Drawings
Fig. 1 is a flow chart of a frequency-using decision optimization method in communication countermeasure based on a log-linear algorithm.
Fig. 2 is a diagram of a fighting game model for the communication of the enemy and the my.
Fig. 3 is a distribution diagram of communication users of two parties of the enemy and the my in a small scene.
Fig. 4 is a schematic diagram of frequency selection of two parties and an interferer under the force of our parties.
Fig. 5 is a schematic diagram of frequency selection between two parties and a jammer in an adversary situation.
FIG. 6 is a diagram of a distribution of friend or foe two-party communication users in a large scene.
FIG. 7 is a graph comparing the convergence effect of the linear learning algorithm of the enemy user number and the SLA algorithm.
Fig. 8 is a graph comparing the convergence effect of the linear learning algorithm of the number of users of my party with the SLA algorithm.
Fig. 9 is a graph comparing interference rejection levels for several algorithms.
FIG. 10 is a diagram of the values of parameters in the small scenario of the present invention.
FIG. 11 is a diagram of the values of parameters in a large scenario of the present invention.
Detailed Description
The present invention will now be described in further detail with reference to the accompanying drawings.
It should be noted that the terms "upper", "lower", "left", "right", "front", "back", etc. used in the present invention are for clarity of description only, and are not intended to limit the scope of the present invention, and the relative relationship between the terms and the terms is not limited by the technical contents of the essential changes.
Detailed description of the preferred embodiment
For ease of viewing, the integration section in this embodiment refers to the physical meaning of the letters, see table 1.
TABLE 1 physical meanings of letters to which the invention relates
With reference to fig. 1, the invention provides a frequency-use decision optimization method in communication countermeasure based on a log-linear learning algorithm, wherein a Stackelberg game model is used for modeling a frequency-use decision process in communication countermeasure of two enemies, and the two enemies sequentially perform the frequency-use decision process according to a set calculation method by designing a distributed channel selection algorithm based on log-linear learning. The method comprises the following steps:
step one, establishing a communication scene of users of both enemies and my parties
The game parties in the system useThe method comprises the steps that a user and an enemy user are provided, wherein the user comprises N communication transceiving pairs and a jammer, and the enemy user comprises M communication transceiving pairs. WhereinA set of numbers representing a total of N +1 users of my party,the number set represents N communication transceiving pairs of our party, and the user of our party with the number of N +1 represents the jammer of our party;the number set of M communication transceiving pairs of the enemy is represented, and the instantaneous channel gain expression of the user isWherein d isx,yRepresenting the distance between user x and user y, alpha andrespectively representing the path fading coefficients and the rayleigh fading coefficients. The method comprises the steps that two enemies and a party (including an interference machine) make a decision in sequence, wherein the enemy is taken as a weak party to make a decision firstly, and the party (including the interference machine) is taken as a strong party to make a final decision. The frequency selection process of the two parties of the enemy and the my is carried out according to the following steps:
and S121, randomly selecting frequencies by the user of the party.
And S122, the enemy user decides the frequency selection strategy of the enemy user according to the perceived frequency selection of the previous interference machine of the enemy.
And S123, the user of the party decides the frequency selection strategy of the user of the party according to the presumed frequency selection of the user of the enemy.
And S124, circulating the steps b and c until the countermeasure is finished, wherein each circulation is a one-wheel countermeasure process.
And step two, constructing a Stackelberg game model of a frequency decision problem in communication countermeasure.
In order to better study the frequency use decision problem in wireless communication countermeasure, the interference countermeasure problem is expressed as the Stackelberg game process. Specifically, the two parties of the enemy and the my are independent of each other, and the frequency selection strategy of the own party is adjusted by supposing the current spectrum environment so as to maximize the effectiveness of the own party. Specifically, the frequency decision Stackelberg game in the communication countermeasure of the two enemy and my users is defined as follows:whereinAndrespectively representing a set of my users and enemy users,representing respective alternative channel sets, u, of my and enemynAnd umRespectively representing the utility functions of a user n of the my party and a user m of the enemy party. Without loss of generality, assumeFrom the perspective of my user, my user sub-gamingCan be defined as:an、a-nand amRespectively representing the frequency selection of the user n, other users except the user n and the enemy user m. For a user n of a party, the purpose is to minimize mutual interference among the users of the party and maximize the interference of a jammer of the party on a user of an enemy party, and the utility u of the jammer of the partynIs defined as:
wherein L isuAnd LjIs a predetermined normal number, preventing the target utility value from being less than zero, PnAnd PN+1Respectively representing the transmit power of my user n and my jammer,andrespectively representing the instantaneous channel gain between my users, the instantaneous gain between my jammers and my jammers, and the instantaneous channel gain between my jammers and enemy communication pairs. Considering that my jammer uses a directional antenna channel, the direction of interference is directed towards the area where the enemy user is located, ZbAnd ZrRespectively representing the degree of interference caused by the antenna side lobe of the jammer at one party to the communication at one party and the degree of interference caused by the antenna main lobe of the jammer at one party to the user at the enemy party, and Zb<Zr。f(ax,ay) The indication function of whether the frequency selection results are the same is shown, and the expression is as follows:
the goal of my user n is to adjust its channel selection to maximize its utility, and the goal of the established my user sub-game model is to maximize the utility function for each user, i.e., theFrom the perspective of an enemy user, the enemy user subscreensCan be defined as:am、a-mrespectively represent enemy users m and other users mFrequency selection by an adversary user. For an enemy user m, the purpose of the method is to minimize the mutual interference between the enemy users and minimize the interference of the jammer of the party to the party, and the utility umIs defined as:
wherein L isuIs a predetermined normal number, preventing the target utility value from being less than zero, PmAnd PN+1Respectively representing the transmit power of the enemy user m and my jammer,andrespectively representing the instantaneous channel gain between the enemy user and the instantaneous channel gain between my jammer and the enemy user. The objective of the enemy user m is to adjust its channel selection to maximize its utility, and the objective of the established enemy user sub-game model is to maximize the utility function of each enemy user, i.e., to maximize the utility function of each enemy user
And step three, proving that the constructed game model is an accurate potential energy game.
Sub-gaming on enemy usersIn (1), policy a for adversary user mmDefining a potential energy function phil:
φl=φl,1+φl,2
when the adversary user m unilaterally sends the policy amIs changed intoThe change in the potential energy function at this time is:
due to the fact thatAnddoes not contain amThe term, the subtraction of two equations is eliminated, so:
thus, it is possible to provideAt the moment, the change of the utility function and the change of the potential energy function caused by the fact that any user changes the strategy in a unilateral way are the same, and the constructed game modelIs an accurate potential energy game.
φf=φf,1+φf,2
Wherein phif,1And phif,2H (N, N +1), y (N, N +1) and g (N, N +1) in (1) are defined as:
when one side of my user n is to make policy anIs changed intoThe change in the potential energy function at this time is:
following the previous definition of h (N, N +1), y (N, N +1) and g (N, N + 1):
when N is less than or equal to N, substituting h (N, N +1) into 0, y (N, N +1) into 1, and g (N, N +1) into 1 to obtain:
when N is N +1, h (N, N +1) is 1, y (N, N +1) is 0, and g (N, N +1) is ZbAnd substituting N ═ N +1 to obtain:
At this time, the change of the utility function and the change of the potential energy function caused by changing the strategy in one way by any user are the same. The constructed game modelIs an accurate potential energy game.
The precise potential energy game has many properties, the two most important of which are as follows:
(1) the accurate potential energy game has at least one nash balance.
(2) The global or local optimal solution of the potential energy function constitutes a nash equilibrium.
And step four, designing a distributed channel selection algorithm based on log linear learning.
First, the above-described channel selection (i.e., frequency selection) game is extended to the form of a hybrid strategy. The specific algorithm steps are as follows:
And 3, updating the channel selection probability of the user set according to the following rules:
where β is a learning parameter.
And 4, step 4: and (5) making k equal to k +1, and looping the steps 2 and 3 until the maximum iteration number is reached or the network utility is almost unchanged, and jumping out of the loop.
And step five, proving the convergence of the algorithm.
The probability update equation can be converted into:
when beta is larger, the combination of randomly selected channelsIs less effective than the last channel combination c (k), i.e.Much less than 1 and close to 0, when c (k +1) equals c (k) with a probability close to 1, a randomly selected combination of channels is selectedThe probability of (c) is close to 0. When the utility value of the random selection is greater than the last value, the random selection is favored. Through continuous iterative learning, a frequency-using decision with a larger utility value is made according to the probability, and the utility result converges to the maximum value. While a random selection may happen to have the same result as the maximum, the probabilities are 1/2, which do not converge for the probability, but converge for the utility.
Detailed description of the invention
This example builds on the adversarial distribution of user communication between the two parties of the friend and foe shown in fig. 2.
FIG. 1 shows a flow chart of the present invention, which includes the following steps
(1) And constructing a hostile-my two-party communication confrontation scene graph. Firstly, as shown in fig. 3, the position relationship of the users of the enemy and me in the small scene can be clearly seen, that the communication nodes are located in the area of 6000m × 4000m, and 8 communication nodes are located in 4 communication pairs and one jammer respectively; as shown in fig. 6, which is a position relationship between the users of the enemy and me in a large scene, it can be clearly seen that the communication nodes are located in an area of 6000m × 4000m, and a total of 20 communication nodes are located in 10 communication pairs and one jammer respectively.
(2) And constructing a Stackelberg game model of a frequency decision problem in communication countermeasure. In order to better study the frequency use decision problem in wireless communication countermeasure, the interference countermeasure problem is expressed as the Stackelberg game process. Specifically, the two parties of the enemy and the my are independent of each other, and the frequency selection strategy of the own party is adjusted by supposing the current spectrum environment so as to maximize the effectiveness of the own party. Specifically, the frequency decision Stackelberg game in the communication countermeasure of the two enemy and my users is defined as follows:whereinAndrespectively representing a set of my users and enemy users,representing respective alternative channel sets, u, of my and enemynAnd umRespectively representing the utility functions of a user n of the my party and a user m of the enemy party. Without loss of generality, assumeFrom the perspective of my user, my user sub-gamingCan be defined as:an、a-nand amRespectively representing the frequency selection of the user n, other users except the user n and the enemy user m. For a user n of a party, the purpose is to minimize mutual interference among the users of the party and maximize the interference of a jammer of the party on a user of an enemy party, and the utility u of the jammer of the partynIs defined as:
wherein L isuAnd LjIs a predetermined normal number, preventing the target utility value from being less than zero, PnAnd PN+1Respectively representing the transmit power of my user n and my jammer,andrespectively representing the instantaneous channel gain between my users, the instantaneous gain between my jammers and my jammers, and the instantaneous channel gain between my jammers and enemy communication pairs. Consider thatThe jammer to my party uses a directional antenna channel, the direction of interference is directed to the area where the enemy user is located, ZbAnd ZrRespectively representing the degree of interference caused by the antenna side lobe of the jammer at one party to the communication at one party and the degree of interference caused by the antenna main lobe of the jammer at one party to the user at the enemy party, and Zb<Zr。f(ax,ay) The indication function of whether the frequency selection results are the same is shown, and the expression is as follows:
the goal of my user n is to adjust its channel selection to maximize its utility, and the goal of the established my user sub-game model is to maximize the utility function for each user, i.e., theFrom the perspective of an enemy user, the enemy user subscreensCan be defined as:am、a-mrespectively represent the frequency selection of the enemy user m and other enemy users except the user m. For an enemy user m, the purpose of the method is to minimize the mutual interference between the enemy users and minimize the interference of the jammer of the party to the party, and the utility umIs defined as:
wherein L isuIs a predetermined normal number, preventing the target utility value from being less than zero, PmAnd PN+1Respectively representing the transmit power of the enemy user n and my jammer,andrespectively representing the instantaneous channel gain between the enemy user and the instantaneous channel gain between my jammer and the enemy user. The objective of the enemy user m is to adjust its channel selection to maximize its utility, and the objective of the established enemy user sub-game model is to maximize the utility function of each enemy user, i.e., to maximize the utility function of each enemy user
(3) And solving the established game problem by using a designed logarithmic linear learning distributed channel selection algorithm. First, the above-described channel selection (i.e., frequency selection) game is extended to the form of a hybrid strategy. The specific algorithm steps are as follows:
step 1: initialization, making iteration number k equal to 1, user set selecting channel combination with equal probability from its available channel set according to its communication requirement
Step 2: user set randomly selects one channel combination in available channel setAnd calculates its combination in the channelThe effect of
And step 3: the user set updates its channel selection probability according to the following rules:
where β is the learning parameter, and in this example, the final value of β is 11.
And 4, step 4: and (5) making k equal to k +1, and looping the steps 2 and 3 until the maximum iteration number is reached or the network utility is almost unchanged, and jumping out of the loop.
(4) Simulation analysis:
fig. 4 shows the spectrum relationship between the two enemy and my parties and the jammer in our strong situation. It can be clearly seen that: when the jammer selects a certain channel, because my party and the jammer are strong parties and enemy parties are weak parties, the my party and the jammer can always estimate the frequency selection of the enemy parties, the communication transceiver pair of the my party completely avoids the channel selected by the jammer, and the jammer can always jam the channel selected by the enemy parties. Because the mutual interference between the enemy users and the interference of the jammers need to be considered by the enemy, when the enemy users make a decision, the influence of the enemy users and the jammers needs to be considered comprehensively, and therefore the frequency band of the last interference can be almost avoided when the enemy selects the frequency, because the enemy users are also intelligent agents. In addition, if the enemy user selects the same frequency band with the my party communication transceiver pair, the jammer considers avoiding the my party communication transceiver pair, and therefore does not interfere with the frequency selection of the enemy user.
Fig. 5 shows the spectral relationship of both friend and foe and jammers under hostile forces. It can be seen that when the strong and weak relationship between the enemy and the my party is changed, the enemy user as the strong party almost avoids the interference of the jammer and the frequency mutual interference between the enemy users. Because the user of our party is still an intelligent agent, interference of an interference machine and frequency selection of other communication transceiving pairs of our party are avoided as much as possible when the frequency band is selected by the communication transceiving pair of our party.
Fig. 7 shows comparative utility convergence between the log-linear learning algorithm and the SLA algorithm for an adversary user with 4 available channels. Compared with the SLA algorithm, the logarithmic linear learning algorithm has better effect in terms of convergence rate and anti-interference level.
Fig. 8 shows comparative utility convergence between the log-linear learning algorithm and the SLA algorithm under the condition that the number of available channels of my user is 4. From the aspects of convergence speed and anti-interference level, the logarithmic linear learning algorithm has better convergence effect than the SLA algorithm no matter an enemy user or a my user.
Fig. 9 shows the comparison of the interference rejection levels of several algorithms, and it can be seen that the overall interference rejection level is improved as the number of channels increases. This is because channels selected by interference between users can be avoided as the number of available channels increases. Whether two curves of mutual interference of the users of the two enemies exist in the log-linear learning algorithm or not is clear that the interference resistance level difference of the two is not large, and the mutual interference of the users of the two enemies has small effect on the whole decision. From the log-linear learning algorithm, the optimal nash equilibrium, the worst nash equilibrium. The comparison of the SLA algorithm and the random selection of the curves shows that the logarithmic linear learning algorithm and the SLA algorithm are close to the optimal values and are obviously superior to the random selection method.
The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above-mentioned embodiments, and all technical solutions belonging to the idea of the present invention belong to the protection scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may be made by those skilled in the art without departing from the principle of the invention.
Claims (2)
1. A medium frequency decision optimization method in communication countermeasure based on a log linear algorithm is characterized by comprising the following steps:
s1, establishing a friend or foe user communication scene, setting independent my user and friend or foe user as game two sides, and carrying out frequency use decision process by friend or foe in turn, wherein in each counter-countermeasure process, the frequency use condition of the other side can be used as the reference of own frequency use decision by conjecture/perception; the enemy user comprises M communication transceiving pairs;
s2, on the basis of adjusting a frequency selection strategy of a party by conjecturing/sensing a current spectrum environment and maximizing the effect of the frequency selection strategy, expressing the interference countermeasure problem of the users of the two parties of the enemy and the my as a Stackelberg game process, and constructing a game model; for the user of the same party, the game model aims to minimize the mutual interference among the users of the same party and maximize the interference of the interference machine of the same party on the user of the opposite party, and for the user of the opposite party, the game model aims to minimize the mutual interference among the users of the opposite party and minimize the interference of the interference machine of the same party on the own party;
s3, proving that the constructed game model is an accurate potential energy game;
s4, designing a distributed channel selection algorithm based on log-linear learning, wherein the algorithm is used for frequency decision of enemy users and my users to obtain a balanced solution of the established game problem;
s5, carrying out convergence certification on the algorithm designed in the step S4;
in step S1, the process of constructing the communication scenario between the friend or foe user and the friend or foe user includes:
s11, it is assumed that my party user includes N communication transceiving pairs and a jammer, and the enemy party user includes M communication transceiving pairs, whereinA set of numbers representing a total of N +1 users of my party,the method comprises the steps that N communication transceiving pair sets of our party are represented, and a user of our party with the number of N +1 represents an interference machine of our party;representing a set of M communication transmit-receive pairs of enemies, and the user instantaneous channel gain is expressed asWherein d isx,yRepresenting the distance between user x and user y, alpha andrespectively representing a path fading coefficient and a Rayleigh fading coefficient; a isnRepresenting the frequency using strategy of the communication pair n of the my party;
s12, carrying out frequency use decision processes by the two sides of the enemy and the my, wherein the enemy is used as a weak side to carry out decision firstly, the enemy is used as a strong side to carry out final decision, and in the process of each pair of the warheads, the frequency use condition of the other side can be presumed or sensed to be used as the reference of the frequency use decision of the own side;
in step S2, the process of establishing a game model, which takes the principle of adjusting the frequency selection strategy of the own party by inferring/sensing the current spectrum environment to maximize the utility of the own party, and expresses the interference countermeasure problem of the users of both the enemy and me as a Stackelberg game process, includes the following steps:
s21, defining the frequency decision Stackelberg game in the communication countermeasure of the two enemy and my users as follows:whereinAndrespectively representing a set of my users and enemy users,representing respective alternative channel sets, u, of my and enemynAnd umRespectively representing the utility functions of a user n of the my party and a user m of the enemy party;
s22, for the my user, sub-gaming the my userIs defined as:in the formula, an、a-nAnd amRespectively representing the frequency using strategies of a user n of the same party, other users of the same party except the user n and an enemy user m;
s23, for user n, the purpose is to minimize the mutual interference between users and maximize the interference of jammers to users of enemies, and its utility unIs defined as:
wherein L isuAnd LjIs a predetermined normal number, preventing the target utility value from being less than zero, PnAnd PN+1Respectively representing the transmit power of my user n and my jammer,andrespectively representing the instantaneous channel gain between the users of the my party, the instantaneous gain between the communication pair of the my party and the jammer of the my party and the instantaneous channel gain between the jammer of the my party and the communication pair of the enemy party; zbAnd ZrRespectively representing the degree of interference caused by the antenna side lobe of the jammer at one party to the communication at one party and the degree of interference caused by the antenna main lobe of the jammer at one party to the user at the enemy party, and Zb<Zr;f(ax,ay) The indication function of whether the frequency selection results are the same is shown, and the expression is as follows:Psis the transmit power of my party communication to s, asIs the frequency usage policy of my party communication to s,representing other communication pairs than communication pair n, aN+1Is a frequency utilization strategy of the jammer of our party,is the instantaneous channel gain between the my jammer and my communication pair s, axIs the frequency usage strategy of communication to x, ayIs the frequency usage strategy of communication to y, ax,ayUsed for judging the relative relationship between any two users x and y;
s24, defining the established sub game model of the my user as the goal of maximizing the utility function of each user, namely
S25, for the enemy user, sub-gaming the enemy userIs defined as:am、a-mrespectively representing the frequency utilization strategies of an enemy user m and other enemy users except the user m;
s26, for enemy user m, its purpose is to minimize the mutual interference between enemy users and minimize the interference of our disturber to own party, its utility umIs defined as:
wherein L isuIs a predetermined normal number, preventing the target utility value from being less than zero, PmAnd PN+1Respectively representing the transmitting power of an enemy user m and the transmitting power of a my-party jammer; a isN+1The method is a frequency using strategy of the jammer of the same party; piIs the transmit power of the enemy communication pair i, aiIs the frequency usage strategy of the enemy communication pair i,representing enemy communicationsFor other communication pairs than the m-pair,is the instantaneous channel gain between the enemy communication pair i and the enemy communication pair m,is the instantaneous channel gain between my jammer and enemy communication pair m;
s27, the aim of the established enemy user sub-game model is defined to maximize the utility function of each enemy user, namely
In step S3, the process of proving that the constructed game model is an accurate potential energy game includes the following steps:
s31, sub-gaming on enemy usersIn the frequency strategy a for enemy user mmDefining a potential energy function phil:
φl=φl,1+φl,2
When enemy user m unilaterally uses frequency strategy amIs changed intoCalculating whether the change of the corresponding potential energy function is the same as the change of the utility function, and if so, establishing a game modelIs an accurate potential energy game;
s32, sub-gaming the user on my sideIn (1), frequency strategy a for my user nnDefining a potential energy function phif:
φf=φf,1+φf,2
Wherein phif,1And phif,2H (N, N +1), y (N, N +1) and g (N, N +1) in (1) are defined as:
is the instantaneous channel gain between my jammer and the enemy communication pair m,is the instantaneous channel gain between my user pair n and the enemy communication pair m;
when one side of my user n is to use frequency strategy anIs changed intoCalculating whether the change of the corresponding potential energy function is equal to the utilityThe function changes are the same, if the functions are the same, the constructed game modelIs an accurate potential energy game;
in step S4, the process of designing the distributed channel selection algorithm based on log-linear learning includes the following steps:
s41, initializing, making the iteration number k equal to 1, selecting the channel combination A with equal probability from the available channel set according to the communication requirement of the user setn={a1,a2,...,an,...,aN+1},a1Is the frequency using strategy of the user of our party to 1, a2The frequency utilization strategy of the user of our party for 2 is adopted;
s42, the user set randomly selects a channel combination from its available channel setAnd calculates its combination in the channelOverall utility of the case (1)LuIs a predetermined normal number, preventing the target utility value from being less than zero; u shapenIs the utility of my user to n, UN+1Is the effect of our jammer, LN+1Is a constant that ensures the utility of our jammer as a positive number;
s43, the user set updates its channel selection probability according to the following rule:
where β is the learning parameter, c (k) is the channel combination of the k-th generation of the iteration; u (c (k)) is the overall utility of my party when the channel combination is c (k);
s44, if k is k +1, looping steps S42 and S43 until k reaches the maximum iteration number or the network utility change value is smaller than the set change threshold, and jumping out of the loop;
in step S5, the process of proving convergence of the algorithm designed in step S4 includes the following steps:
s51, converting the channel selection probability formula into:
s52, setting the value of beta, wherein beta is positive number, the specific value is determined by environment parameter, if the channel combination is randomly selectedIs less effective than the last channel combination c (k), i.e.Much less than 1 and close to 0, when c (k +1) equals c (k) with a probability close to 1, a randomly selected combination of channels is selectedThe probability of (a) is close to 0; if the utility value of the random selection is greater than the last value, then the random selection is favored;
and S52, optimizing the value of beta through continuous iterative learning, making a frequency-using decision with a larger utility value according to the probability, and converging the utility result to the maximum value at the moment.
2. The communication countermeasure medium frequency decision optimization method based on the log linear algorithm as claimed in claim 1, wherein the frequency selection process is carried out by both the friend and foe according to the following steps:
s121, randomly selecting frequency by the user of one party;
s122, the enemy user decides the frequency selection strategy of the enemy user according to the perceived frequency utilization strategy of the interference machine of the previous party;
s123, the user of the party decides the frequency selection strategy of the user of the party according to the presumed frequency utilization strategy of the user of the enemy;
and S124, circulating the step S122 and the step S123 until the countermeasure is finished, wherein one countermeasure process is performed once in each circulation.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010534345.0A CN111726192B (en) | 2020-06-12 | 2020-06-12 | Communication countermeasure medium frequency decision optimization method based on log linear algorithm |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010534345.0A CN111726192B (en) | 2020-06-12 | 2020-06-12 | Communication countermeasure medium frequency decision optimization method based on log linear algorithm |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111726192A CN111726192A (en) | 2020-09-29 |
CN111726192B true CN111726192B (en) | 2021-10-26 |
Family
ID=72568092
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010534345.0A Active CN111726192B (en) | 2020-06-12 | 2020-06-12 | Communication countermeasure medium frequency decision optimization method based on log linear algorithm |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111726192B (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114172691A (en) * | 2021-11-11 | 2022-03-11 | 南京航空航天大学 | Anti-tracking interference system based on decoy strategy |
CN115378487B (en) * | 2022-07-05 | 2023-12-01 | 中国人民解放军战略支援部队航天工程大学 | Satellite communication dynamic spectrum access method based on electromagnetic barrier |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8545332B2 (en) * | 2012-02-02 | 2013-10-01 | International Business Machines Corporation | Optimal policy determination using repeated stackelberg games with unknown player preferences |
CN108616916A (en) * | 2018-04-28 | 2018-10-02 | 中国人民解放军陆军工程大学 | A kind of anti-interference layering betting model of cooperation and anti-interference learning algorithm |
CN108989115A (en) * | 2018-07-25 | 2018-12-11 | 湖北民族学院 | Application method of the virtual beams co-interfere in heterogeneous wireless network safety of physical layer |
CN110007278A (en) * | 2019-03-15 | 2019-07-12 | 南京航空航天大学 | A kind of more stealthy optimization methods of base radar radio frequency based on this tower Frederick Colberg game |
CN110691360A (en) * | 2019-08-21 | 2020-01-14 | 西北工业大学 | Joint resource allocation method based on user clustering and Starkelberg model |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103210622B (en) * | 2010-11-12 | 2017-03-01 | 诺基亚通信公司 | Method and apparatus for the resource allocation in communication system |
-
2020
- 2020-06-12 CN CN202010534345.0A patent/CN111726192B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8545332B2 (en) * | 2012-02-02 | 2013-10-01 | International Business Machines Corporation | Optimal policy determination using repeated stackelberg games with unknown player preferences |
CN108616916A (en) * | 2018-04-28 | 2018-10-02 | 中国人民解放军陆军工程大学 | A kind of anti-interference layering betting model of cooperation and anti-interference learning algorithm |
CN108989115A (en) * | 2018-07-25 | 2018-12-11 | 湖北民族学院 | Application method of the virtual beams co-interfere in heterogeneous wireless network safety of physical layer |
CN110007278A (en) * | 2019-03-15 | 2019-07-12 | 南京航空航天大学 | A kind of more stealthy optimization methods of base radar radio frequency based on this tower Frederick Colberg game |
CN110691360A (en) * | 2019-08-21 | 2020-01-14 | 西北工业大学 | Joint resource allocation method based on user clustering and Starkelberg model |
Non-Patent Citations (1)
Title |
---|
无线通信网用户与干扰机的动态博弈研究;周恒杰;《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》;20200331;全文 * |
Also Published As
Publication number | Publication date |
---|---|
CN111726192A (en) | 2020-09-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109743774B (en) | Power distribution method for realizing physical layer safe transmission based on non-cooperative game | |
CN108616916B (en) | Anti-interference learning method based on cooperative anti-interference layered game model | |
CN112188504B (en) | Multi-user cooperative anti-interference system and dynamic spectrum cooperative anti-interference method | |
CN111132347B (en) | Resource allocation method, device and computer for wireless energy-carrying physical layer secure transmission | |
CN111726192B (en) | Communication countermeasure medium frequency decision optimization method based on log linear algorithm | |
CN104994569B (en) | Multi-user reinforcement learning-based method for resisting hostile interference of cognitive wireless network | |
Yang et al. | Optimal transmission power control in the presence of a smart jammer | |
CN108260198B (en) | Radar networking power control method based on non-cooperative game under spectrum sharing | |
CN112566127A (en) | Physical layer secure transmission method in cognitive wireless network based on unmanned aerial vehicle assistance | |
Han et al. | Joint resource allocation in underwater acoustic communication networks: A game-based hierarchical adversarial multiplayer multiarmed bandit algorithm | |
Menon et al. | Wsn15-4: A game-theoretic framework for interference avoidance in ad hoc networks | |
Khanafer et al. | Adaptive resource allocation in jamming teams using game theory | |
CN113973362B (en) | Reinforced learning non-zero and non-cooperative multi-agent safety communication power control method | |
CN113726472B (en) | Simultaneous interference and monitoring method based on Bayesian Stackelberg game | |
CN115378487B (en) | Satellite communication dynamic spectrum access method based on electromagnetic barrier | |
Lu et al. | An efficient SIR-first adaptive power control method in cognitive radio network | |
CN116073924A (en) | Anti-interference channel allocation method and system based on Stackelberg game | |
CN113824469B (en) | MFDRL-CTDE-based intelligent anti-interference decision method for frequency hopping networking | |
CN113613337B (en) | User cooperation anti-interference method for beam forming communication | |
Chai et al. | A user-selected uplink power control algorithm in the two-tier femtocell network | |
Gouissem et al. | Toward secure IoT networks in healthcare applications: A game-theoretic anti-jamming framework | |
Li et al. | Know Thy Enemy: An Opponent Modeling-Based Anti-Intelligent Jamming Strategy Beyond Equilibrium Solutions | |
Farahmand et al. | Max-min strategies for power-limited games in the presence of correlated jamming | |
Bai et al. | A power control algorithm based on game theory in cognitive radio | |
CN114339976A (en) | Resource allocation method meeting time delay constraint in unmanned aerial vehicle ad hoc network |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |