CN114389678B - Multi-beam satellite resource allocation method based on decision performance evaluation - Google Patents
Multi-beam satellite resource allocation method based on decision performance evaluation Download PDFInfo
- Publication number
- CN114389678B CN114389678B CN202210033327.3A CN202210033327A CN114389678B CN 114389678 B CN114389678 B CN 114389678B CN 202210033327 A CN202210033327 A CN 202210033327A CN 114389678 B CN114389678 B CN 114389678B
- Authority
- CN
- China
- Prior art keywords
- user
- satellite
- communication
- total
- throughput
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B7/00—Radio transmission systems, i.e. using radiation field
- H04B7/14—Relay systems
- H04B7/15—Active relay systems
- H04B7/185—Space-based or airborne stations; Stations for satellite systems
- H04B7/1851—Systems using a satellite or space-based relay
- H04B7/18519—Operations control, administration or maintenance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B17/00—Monitoring; Testing
- H04B17/30—Monitoring; Testing of propagation channels
- H04B17/309—Measuring or estimating channel quality parameters
- H04B17/336—Signal-to-interference ratio [SIR] or carrier-to-interference ratio [CIR]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B17/00—Monitoring; Testing
- H04B17/30—Monitoring; Testing of propagation channels
- H04B17/309—Measuring or estimating channel quality parameters
- H04B17/345—Interference values
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Quality & Reliability (AREA)
- Electromagnetism (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Astronomy & Astrophysics (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radio Relay Systems (AREA)
Abstract
The invention discloses a multi-beam satellite resource allocation method based on decision performance evaluation, belonging to the field of satellite communication; firstly, aiming at M wave beams of a single satellite, respectively corresponding N users to build a communication scene of a same-frequency networking under each wave beam; the user (m, n) at the t-th time slot requests data from the satellite, and calculates the communication link [ m, n, t ]]In the same frequency interference Im,n,tSignal to interference plus noise ratio (SINR)m,n,tAnd rate of communicationThen, the communication system throughput sum C of the satellite in the time period T is calculated by using the communication rate of each user in each time slottotal(ii) a Building a multi-objective optimization model considering the throughput of a communication system and the fairness of users at the same time; and finally, solving the multi-objective optimization model by using the DDPG to obtain the joint allocation of bandwidth and power resources meeting the system throughput and the user fairness. The method keeps smaller same frequency interference among users in a full-frequency multiplexing scene, improves the total throughput of a satellite system, and simultaneously considers the fairness of the system.
Description
Technical Field
The invention belongs to the field of satellite communication, and particularly relates to a multi-beam satellite resource allocation method based on decision performance evaluation.
Background
As a significant infrastructure of the national information network, the satellite communication system can well make up for the defects of the ground network, and has become an indispensable part of the national information network.
With the rapid development of the internet of things technology, especially the increasing number of terminals, the traffic volume is increased sharply, and the spectrum resources of the satellite communication system are increasingly tense. Future communication satellites must increase their capacity to the throughput range of TB/s, and the limitation of available bandwidth is a major bottleneck to achieve such high throughput values.
The proposal of the multi-beam satellite greatly improves the spectrum utilization rate of a satellite communication system, promotes the development of a high-throughput satellite, and provides favorable conditions for meeting the increasing business requirements, realizing reliable and flexible connection and the conversion of the satellite from broadcasting to broadband tasks.
The characteristic of a multi-beam satellite is utilized to improve the frequency band utilization rate, the same frequency band must be repeatedly used among a large number of spot beams, if the whole bandwidth of the system is repeatedly used in each beam, the highest frequency band reuse rate can be realized, but the highest frequency band reuse rate causes very strong same frequency interference, and the problem can lead the resource allocation of the satellite system to be very complicated.
At present, many researches on multi-beam satellite resource allocation are carried out in a beam domain, each beam uses partial frequency, and therefore co-channel interference of a system is reduced, but the allocation formula cannot fully utilize total frequency resources of the system. Under the background that the same frequency band is used among wave beams, the frequency domain and the power domain are jointly optimized for users, so that the total throughput of the system is improved on the premise of ensuring the fairness of the users, and the method has great significance for the development of the next generation satellite communication technology.
Disclosure of Invention
Aiming at the problems, the invention provides a multi-beam satellite resource allocation method based on decision performance evaluation, which achieves the aim of improving the total throughput of a satellite system and simultaneously considering the fairness of the system by reasonably allocating bandwidth and power resources of users.
The multi-beam resource allocation method specifically comprises the following steps:
step one, building a communication scene comprising a plurality of users and a single multi-beam satellite co-frequency networking;
the multi-beam satellite has M beams with N users under each beam, where the nth user under the mth beam is denoted as (M, N); the user is located at omegam,nA location;
step two, the t time slot user (m, n) requests the data download from the multi-beam satellite, calculates the communication link [ m, n, t ] of the multi-beam satellite and the user]In the same frequency interference Im,n,t;
The calculation formula is as follows:
[m,n,t]for a communication link of the t-th timeslot user (m, n) with the multi-beam satellite:is the angle of the received signal direction from the axis of the receiving antenna. XifRepresenting the set of all co-frequency channels with frequency f, wherein f is the frequency of the t-th time slot user (m, n);representing a communication link [ m, n, t ]]The channels that cause co-channel interference are,is a link [ m, n, t]The power of the data transmission of (a),is a link [ m, n, t]The channel gain of (c).
Step three, utilizing same frequency interference Im,n,tComputing communication links [ m, n, t ]]SINR ofm,n,tAnd further calculates the communication link [ m, n, t [ ]]Communication rate of
The formula for calculating the signal to interference plus noise ratio is as follows:
Pm,n,tfor communication links [ m, n, t ]]Transmission power of, N0Is Gaussian white noise power spectral density, Bm,n,tFor communication links [ m, n, t ]]Bandwidth of hm,nIn order to obtain the gain of the satellite-to-ground link,
Gt(θ) is link [ m, n, t ]]Theta is the angle of the user (m, n) from the antenna axis of the beam in which it is located; PL is loss and fading of signal power due to a channel environment;is a link [ m, n, t]The receive antenna gain of (1).
step four, calculating the throughput sum C of the communication system of the multi-beam satellite in the time period T by using the communication rate of each user in each time slottotal;
step five, building a multi-objective optimization model considering the throughput of the communication system and the fairness of users at the same time;
firstly, calculating the fairness of the user according to the Jain index; the calculation formula is as follows:
to initialize business requirements.Representing the service satisfaction index of the subscriber (m, n) instead of the communication rate in the original formulaAnd calculating the fairness according to Jain index formula.
Then, the multi-objective optimization model is as follows:
P1:max Ctotal
P2:max F
wherein the optimization objective P1 represents maximizing the overall throughput of the communication system; the optimization objective P2 two represents maximizing fairness of the communication system.
The constraint condition C1 indicates that the throughput of the t-th time slot of the user is not greater than the residual traffic of the time slot;
after the satellite provides service for t time slots to a user (m, n), the remaining traffic of the user in the t +1 th time slot is:
the constraint C2 means that the total power of all users is not greater than the maximum transmitting power P of the satellitetotal;
The constraint C3 indicates that the total bandwidth of all users in the same beam is not greater than the total bandwidth B of the communication systemtotal。
And step six, solving the multi-target optimization model by using the deep reinforcement learning network DDPG to obtain the bandwidth and power resource joint allocation meeting the system throughput and the user fairness.
The set deep reinforcement learning network DDPG comprises: a resource allocation decision network and a decision performance evaluation network;
modeling a process of resource allocation by using DDPG as a Markov process;
defining the network state vector of the t-th time slot in the decision network as st={Ht,It,Ct,DtIn which H istRepresenting the channel gain of each user; I.C. AtThe co-channel interference strength received by the user; ctRepresents the throughput of the user; dtRepresenting the traffic of the user.
Defining actions in the decision network including allocated bandwidth and power resources for each beam, for each time slot t e {1,2,3t={Pt,BtIn which P istIndicating the data transmission power allocated by the system to the user, BtIndicating the data transmission bandwidth allocated by the system to the user.
Performance evaluation network: and calculating the current Q value according to the current resource allocation strategy output by the decision network, and updating the network parameters by calculating a loss function.
The evaluation of the total prize value of the network is designed as: r = ω1R1+ω2R2-ω3R3;
Wherein ω is1~ω3Weights corresponding to different awards; r1The reward for the total throughput of the system is: r is1=Ctotal;R2Reward corresponding to the system fairness coefficient: r2=F;R3For auxiliary reward:
wherein: x is the number ofmnThe user resource is allocated with a reasonable or not index,umnis an indicator of whether the user's throughput at the t-th time slot is greater than the traffic volume,ztas an indication of whether all users are more than the total power,
unreasonable resource allocation means that the agent has allocated bandwidth resources or power to the users, but has not allocated power or bandwidth to the users.
The invention has the advantages that:
1) The method for allocating the multi-beam satellite resources based on decision performance evaluation simultaneously considers the total throughput of a multi-beam satellite communication system and the fairness of users and simultaneously optimizes two targets; the problem that only optimizing the system throughput can cause part of users not to be served all the time under the condition of limited resources is avoided.
2) The multi-beam satellite resource allocation method based on decision performance evaluation adopts a decision performance evaluation network, adjusts parameters of the decision network according to an evaluation result of the evaluation network so as to optimize a resource allocation scheme, updates parameters of the evaluation network, and realizes accurate prediction of the decision network in an iterative optimization mode.
Drawings
Fig. 1 is a schematic diagram of a multi-beam satellite resource allocation method based on decision performance evaluation according to the present invention.
Fig. 2 is a flowchart of a multi-beam satellite resource allocation method based on decision performance evaluation according to the present invention.
FIG. 3 is a diagram of an actual multi-beam GEO satellite downlink communication scenario constructed in accordance with the present invention;
fig. 4 is a detailed design diagram of a decision-evaluation network according to the present invention.
Fig. 5 is a graph comparing throughput with average resource allocation and different bandwidths of random resources according to the present invention.
Fig. 6 is a comparison diagram of fairness of the system under different bandwidths of the average resource allocation and the random resource in the invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples.
The invention provides a multi-beam satellite resource allocation method based on decision performance evaluation, which is used for optimizing bandwidth and power resource allocation of each user, aiming at a scene that users request data downloading from a satellite in a multi-beam satellite same-frequency networking system. As shown in fig. 1, first, a user in each beam in a multi-beam satellite generates a service demand and forwards it to the satellite via a gateway station, requesting data download via the satellite. The satellite collects the position information of each user and the resource information of the system after receiving the request, then transmits the information to the resource allocation decision network, the decision network allocates power and bandwidth to the users according to the state of each user, and transmits the resource allocation strategy to the multi-beam satellite system and the resource allocation evaluation network. And the resource allocation evaluation network evaluates the decision made by the resource allocation decision network according to the actual return of the system and returns the parameters to the resource allocation decision network. The iterative loop continuously improves the accuracy of the decision made by the resource allocation decision network, thereby maximizing the system throughput and the user fairness.
The method is based on the theoretical knowledge of deep reinforcement learning, the total throughput of a satellite system and the fairness optimization modeling of a user are modeled into a multi-objective optimization problem, a continuous resource allocation process with time correlation is modeled into a Markov process, a resource allocation decision network and a decision performance evaluation network are established by using a DDPG neural network, and the multi-objective optimization problem is solved. The bandwidth and power resources of the users are reasonably distributed to the users in each time slot, so that the smaller same frequency interference among the users is kept under the full-frequency multiplexing scene, and the aim of improving the total throughput of the satellite system and simultaneously considering the fairness of the system is achieved.
As shown in fig. 2, the multi-beam satellite resource allocation method specifically includes the following steps:
step one, building a communication scene comprising a plurality of users and a single multi-beam satellite co-frequency networking;
as shown in fig. 3, consider a single multi-beam GEO satellite having M beams with N users per beam, comprising: mobile terminals, multimedia terminals, vehicle terminals, and the like; the users in each beam create service demands that are forwarded to the satellite via the gateway station requesting data download via the satellite. The satellite acquires the position information of each user and the resource information of the system after receiving the request, transmits the position information and the resource information to the gateway station through the user flow generation module and the resource allocation module, and the resource allocation decision network allocates power and bandwidth to the users according to the states of the users; wherein the nth user under the mth beam is denoted as (m, n); the user is located at omegam,nAt a location;
step two, the t time slot user (m, n) requests the data download from the multi-beam satellite, calculates the communication link [ m, n, t ] of the multi-beam satellite and the user]In the same frequency interference Im,n,t;
The t time slot satellite provides power P to user (m, n)m,n,tTransmitting data using [ m, n, t ]]Represents the communication link, hm,nIs the satellite-to-ground link gain. Assume a multi-beam satellite channel matrix of Hmatrix:
Hmatrix=[h1,1,h1,2,...,hm,n,...,hM,N]
Wherein G ist(theta) is link [ m, n, t ]]θ is the angle of the user (m, n) from the antenna axis of the beam in which it is located; PL is loss and fading of signal power due to a channel environment;is a link [ m, n, t]The gain of the receiving antenna of (1),is the angle of the received signal direction from the axis of the receiving antenna.
Considering the co-channel interference problem, the frequency of the t time slot user (m, n) is f, and the received co-channel interference is Im,n,t:
Xi thereinfRepresenting the set of all co-channel channels at frequency f,representing a communication link [ m, n, t ]]The channels that cause co-channel interference are,is a link [ m, n, t]The power of the data transmission of (a),is a link [ m, n, t]The channel gain of (c).
Step three, utilizing same frequency interference Im,n,tComputing communication links [ m, n, t [ ]]SINR ofm,n,tAnd further calculates the communication link [ m, n, t [ ]]Rate of communication
The formula for calculating the signal to interference plus noise ratio is as follows:
Pm,n,tfor communication links [ m, n, t ]]Transmission power of, N0Is Gaussian white noise power spectral density, Bm,n,tFor communication links [ m, n, t ]]The bandwidth of (d);
step four, calculating the total throughput C of the communication system of the multi-beam satellite in the time period T by using the communication rate of each user in each time slottotal;
After the satellite provides service for t time slots to the users (m, n), the remaining traffic of the users (m, n) at t +1 time slot is:
The total of the throughput of the multi-beam satellite communication system in the T time period is calculated by the formula:
step five, building a multi-objective optimization model considering the throughput of the communication system and the fairness of users at the same time;
while considering system throughput, the present invention also takes user fairness as an optimization objective. This is because without fairness constraints, it is obviously unreasonable that in resource-constrained situations there may be users that are never allocated any resources. Firstly, calculating the fairness of the user according to the Jain index; the calculation formula is as follows:
since service requests of different users are different, use is made ofRepresenting the service satisfaction index of the user (m, n) instead of the communication rate in the original formulaAnd calculating the fairness according to Jain index formula.
The complete multi-objective optimization model is then as follows:
P1:max Ctotal
P2:max F
wherein the optimization objective P1 represents maximizing the overall throughput of the communication system; the optimization objective P2 two represents maximizing fairness of the communication system.
The constraint condition C1 indicates that the throughput of the t-th time slot of the user is not greater than the residual traffic of the time slot;
the constraint C2 means that the total power of all users is not greater than the maximum transmitting power P of the satellitetotal;
The constraint C3 indicates that the total bandwidth of all users in the same beam is not greater than the total bandwidth B of the communication systemtotal;
Total bandwidth of satellite system is BtotalIs uniformly divided into NBA sub-channel.
And step six, solving the multi-target optimization model by using the deep reinforcement learning network DDPG to obtain the bandwidth and power resource joint allocation meeting the system throughput and the user fairness.
Considering that the user resource allocation decision at the t +1 moment of the system is influenced by the user resource allocation condition at the previous t moment, the method models a continuous resource allocation process with time correlation into a Markov process, and establishes the resource allocation decision by utilizing a deep reinforcement learning network so as to achieve the combined optimization target of the system throughput and the user fairness.
As shown in fig. 4, the performance evaluation scheme based on the decision-evaluation algorithm provided by the present invention is formed by combining a resource allocation decision network based on a policy gradient and a decision performance evaluation network based on a value function; the multi-beam satellite communication system is used as an environment, the output system state interacts with the DDPG network, and the DDPG network selects a resource allocation decision according to the environment state.
The network structure, state, actions and benefits are specifically designed as follows:
resource allocation decision network: and the system is responsible for selecting the current resource allocation action a according to the service requirements of users, and is used for interactively generating the total throughput of the system, the system fairness index observed value and the state with the multi-beam satellite communication system. The input is the environment state of the multi-beam satellite communication system, and the output is the resource allocation action; and the decision network corrects the network parameters according to the resource allocation decision evaluation result returned by the evaluation network.
State design
The state is a description of the external environment, and the agent needs to make subsequent decisions by means of the state parameters, and defines the state in the decision network as s. The state changes along with the change of time, and the network state vector of the t-th time slot is st={Ht,It,Ct,DtH wherein HtRepresenting the channel gain of each user; i istThe co-channel interference strength received by the user; ctRepresents the throughput of the user; dtRepresenting the traffic of the user.
Action design
The action is an output parameter of the agent, is used for adjusting variable information in the system environment, and defines the action in the decision network as a. The network action a is a resource allocation decision for the prediction situation at the next moment, and needs to be implemented into a real system to adjust resource variables.
The decision network action mainly comprises the allocated bandwidth and power resources of each beam, and the feasible solutions of the resource parameters form an action space A of the agent. For each time slot t e {1,2, 3. }, the system will allocate power and bandwidth resources to users with traffic demands under consideration of the influence of co-channel interference.
The set of actions is denoted as At={Pt,Bt}: wherein P istIndicating the data transmission power, P, allocated by the system to the usert={P1,1,t,P1,2,t,...,PM,N,t};BtRepresenting the data transmission bandwidth allocated by the system to the user, Bt={B1,1,t,B1,2,t,...,BM,N,t}。
System performance evaluation network: and calculating the current Q value according to the current resource allocation strategy output by the decision network, and updating the network parameters by calculating a loss function.
Reward design
The evaluation of the network reward value needs to reflect the quality of the system performance of the resource allocation decision performance made by the decision network. For each time slot, the environment designs a system prize value based on the current state, the action in the current state, and the next state. The design of the reward value should be related to the goal of resource allocation decision, and the evaluation of the total reward value of the network comprises the following three rewards:
r1: considering the optimization objective P1, the total system throughput is taken as the first reward:
r2: considering the optimization goal P2, the system fairness coefficient is taken as the second reward:
wherein N istotalRepresenting the number of all users in a satellite communication system, i.e. Ntotal=M×N。
wherein: x is the number ofmnThe user resource is allocated with a reasonable or not index,umnis an indication of whether the user's throughput at the t-th time slot is greater than the traffic volume,ztas an indication of whether all users are more than the total power,
unreasonable resource allocation means that the agent allocates bandwidth resources (or power) to the users, but does not allocate power (or bandwidth) to the users.
The total reward is designed to be:
the design is as follows: r = ω1R1+ω2R2-ω3R3;
Wherein ω is1~ω3Weights corresponding to different rewards.
Network design
The invention uses DDPG algorithm in deep reinforcement learning to carry out optimization distribution on the resources of the satellite system, and needs to train the parameters of the DDPG network.
The number of the hidden layers of the decision network of the DDPG network is two, the number of neurons in each layer is 128, sigmod is used as an activation function, an Adam optimizer is adopted for optimization, and the learning rate is set to be 1e-4;
and evaluating the number of the hidden layers of the network to be two layers, wherein the number of neurons in each layer is 256, using relu as an activation function, optimizing by adopting an Adam optimizer, and setting the learning rate to be 2e-4. The capacity of the experience pool was set to 10000, the size of Batch sampled from each time was 256, and the variance of the heuristic noise was 0.7.
Performance analysis
Compared with the schemes of average resource allocation and random resource allocation, as shown in fig. 5 and 6, the results show that the multi-beam satellite bandwidth and power resource joint allocation method based on decision performance evaluation and multi-objective optimization provided by the invention achieves the purpose of improving the throughput of the system on the premise of ensuring user fairness under the condition of different system resources.
Claims (5)
1. A multi-beam satellite resource allocation method based on decision performance evaluation is characterized by comprising the following steps: firstly, aiming at a single multi-beam satellite, corresponding to M beams, wherein N users are arranged under each beam, and a communication scene of the same-frequency networking of the users and the satellite is established;
at the t-th time slot, the user (m, n) requests a data download from the satellite, and the communication link m, n, t between the satellite and the user is calculated]In the same frequency interference Im,n,tSignal to interference plus noise ratio (SINR)m,n,tAnd rate of communication
Then, the communication rate of each user in each time slot is utilized to calculate the throughput sum C of the communication system of the multi-beam satellite in the time period Ttotal(ii) a Building a multi-objective optimization model considering the throughput of a communication system and the fairness of users at the same time;
the complete multi-objective optimization model is as follows:
Ρ1:max Ctotal
Ρ2:max F
wherein the optimization target Ρ 1 represents maximizing an overall throughput of the communication system;
the optimization target Ρ 2 represents a fairness F that maximizes the communication system;
the calculation formula is as follows:
in order to initiate the service requirements,representing the service satisfaction index of the subscriber (m, n) instead of the communication rate in the original formulaAnd calculating the fairness according to Jain index formula;
the constraint C1 indicates that the throughput of the t-th time slot of the user is not greater than the residual traffic of the time slot
The constraint C2 means that the total power of all users is not greater than the maximum transmitting power P of the satellitetotal;
The constraint C3 indicates that the total bandwidth of all users in the same beam is not greater than the total bandwidth B of the communication systemtotal;
And finally, solving the multi-target optimization model by using a deep reinforcement learning network DDPG (distributed data processing) to obtain the bandwidth and power resource joint distribution meeting the system throughput and the user fairness.
2. The multi-beam satellite resource allocation method based on decision performance evaluation according to claim 1, wherein said co-channel interference Im,n,tThe calculation formula is as follows:
the angle of the direction of the received signal deviating from the axis of the receiving antenna; xifRepresenting the set of all co-channel channels with frequency f, wherein f is the frequency of the t time slot user (m, n);representing a communication link [ m, n, t ]]The channels that cause co-channel interference are,is a link [ m, n, t]The power of the data transmission of (a),is a link [ m, n, t]The channel gain of (c).
3. The method of claim 1, wherein the SINR is a signal-to-interference-and-noise ratio (SINR)m,n,tThe calculation formula is as follows:
Pm,n,tfor communication links [ m, n, t ]]Transmission power of, N0Is Gaussian white noise power spectral density, Bm,n,tFor communication links [ m, n, t ]]Bandwidth of hm,nIn order to obtain the gain of the satellite-to-ground link,
5. the method according to claim 1, wherein the DDPG is a deep reinforcement learning network that comprises: a resource allocation decision network and a decision performance evaluation network;
modeling a process of resource allocation by using DDPG as a Markov process;
defining the network state vector of the t-th time slot in the decision network as st={Ht,It,Ct,DtH wherein HtRepresenting the channel gain of each user; i istThe co-channel interference strength received by the user; ctRepresents the throughput of the user; dtRepresenting the traffic of the user;
defining actions in the decision network including allocated bandwidth and power resources for each beam, for each time slot t e {1,2,3t={Pt,BtIn which P istIndicating the data transmission power allocated by the system to the user, BtRepresenting the data transmission bandwidth allocated by the system to the user;
performance evaluation network: calculating a current Q value according to a current resource allocation strategy output by the decision network, and updating network parameters by calculating a loss function;
the evaluation of the total network reward value is designed as: r = ω1R1+ω2R2-ω3R3;
Wherein ω is1~ω3Weights corresponding to different rewards; r1The reward for the total throughput of the system is: r is1=Ctotal;R2Reward for system fairness coefficient: r2=F;R3For auxiliary reward:
wherein: x is a radical of a fluorine atommnThe user resource is allocated with a reasonable or not index,umnis an indication of whether the user's throughput at the t-th time slot is greater than the traffic volume,ztas an indication of whether all users are more than the total power,
an unreasonable resource allocation is when the agent allocates bandwidth resources to the user but does not allocate power to the user, or when the agent allocates power to the user but does not allocate bandwidth to the user.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210033327.3A CN114389678B (en) | 2022-01-12 | 2022-01-12 | Multi-beam satellite resource allocation method based on decision performance evaluation |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210033327.3A CN114389678B (en) | 2022-01-12 | 2022-01-12 | Multi-beam satellite resource allocation method based on decision performance evaluation |
Publications (2)
Publication Number | Publication Date |
---|---|
CN114389678A CN114389678A (en) | 2022-04-22 |
CN114389678B true CN114389678B (en) | 2022-11-01 |
Family
ID=81201312
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210033327.3A Active CN114389678B (en) | 2022-01-12 | 2022-01-12 | Multi-beam satellite resource allocation method based on decision performance evaluation |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114389678B (en) |
Families Citing this family (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115189721B (en) * | 2022-04-29 | 2023-12-19 | 中国人民解放军国防科技大学 | Multi-beam satellite bandwidth power meter joint optimization allocation method and application |
CN115001611B (en) * | 2022-05-18 | 2023-09-26 | 西安交通大学 | Resource allocation method of beam hopping satellite spectrum sharing system based on reinforcement learning |
CN114916051A (en) * | 2022-05-24 | 2022-08-16 | 桂林电子科技大学 | LEO satellite power control method based on BP neural network |
CN114793126B (en) * | 2022-05-24 | 2023-06-23 | 北京航空航天大学 | Multi-beam low-orbit satellite user grouping and resource allocation method |
CN115441939B (en) * | 2022-09-20 | 2024-03-22 | 深圳泓越信息科技有限公司 | MADDPG algorithm-based multi-beam satellite communication system resource allocation method |
CN116156631B (en) * | 2023-01-09 | 2023-08-22 | 中国人民解放军军事科学院系统工程研究院 | Self-adaptive distribution method for satellite communication multi-beam interference power |
CN117528784B (en) * | 2023-11-09 | 2024-05-24 | 中国人民解放军军事科学院系统工程研究院 | Multi-domain cross-layer cooperative control method and device for multi-beam satellite communication network |
CN117240797B (en) * | 2023-11-15 | 2024-01-30 | 华北电力大学 | Combined resource allocation method, device, equipment and medium for electric power hybrid service |
CN117255334B (en) * | 2023-11-17 | 2024-01-26 | 国网浙江省电力有限公司信息通信分公司 | Multistage cooperative scheduling method and system for emergency satellite communication |
CN117639903B (en) * | 2024-01-23 | 2024-05-07 | 南京控维通信科技有限公司 | Multi-user satellite communication method and system based on NOMA assistance |
CN117833997B (en) * | 2024-03-01 | 2024-05-31 | 南京控维通信科技有限公司 | Multidimensional resource allocation method of NOMA multi-beam satellite communication system based on reinforcement learning |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104394596A (en) * | 2014-12-22 | 2015-03-04 | 哈尔滨工业大学 | Upstream resource distribution method capable of considering both throughput capacity and fairness in TD-LTE-Advanced (Time Division-Long Term Evolution-Advanced) relay system |
CN106507366A (en) * | 2016-11-25 | 2017-03-15 | 中国航空无线电电子研究所 | The repeater satellite space-time frequency domain resource dynamic dispatching method of facing multiple users |
CN107835528A (en) * | 2017-10-25 | 2018-03-23 | 哈尔滨工业大学 | The resource allocation methods avoided in the ground integrated network of star based on interference |
CN113644964A (en) * | 2021-08-06 | 2021-11-12 | 北京邮电大学 | Multi-dimensional resource joint allocation method of multi-beam satellite same-frequency networking system |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20130104337A (en) * | 2012-03-13 | 2013-09-25 | 한국전자통신연구원 | Apparatus and method for allocating resource in multi-beam satellite communication |
-
2022
- 2022-01-12 CN CN202210033327.3A patent/CN114389678B/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104394596A (en) * | 2014-12-22 | 2015-03-04 | 哈尔滨工业大学 | Upstream resource distribution method capable of considering both throughput capacity and fairness in TD-LTE-Advanced (Time Division-Long Term Evolution-Advanced) relay system |
CN106507366A (en) * | 2016-11-25 | 2017-03-15 | 中国航空无线电电子研究所 | The repeater satellite space-time frequency domain resource dynamic dispatching method of facing multiple users |
CN107835528A (en) * | 2017-10-25 | 2018-03-23 | 哈尔滨工业大学 | The resource allocation methods avoided in the ground integrated network of star based on interference |
CN113644964A (en) * | 2021-08-06 | 2021-11-12 | 北京邮电大学 | Multi-dimensional resource joint allocation method of multi-beam satellite same-frequency networking system |
Also Published As
Publication number | Publication date |
---|---|
CN114389678A (en) | 2022-04-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN114389678B (en) | Multi-beam satellite resource allocation method based on decision performance evaluation | |
CN111314889B (en) | Task unloading and resource allocation method based on mobile edge calculation in Internet of vehicles | |
CN112616189B (en) | Static and dynamic combined millimeter wave beam resource allocation and optimization method | |
CN111800828B (en) | Mobile edge computing resource allocation method for ultra-dense network | |
CN113572517B (en) | Beam hopping resource allocation method, system, storage medium and equipment based on deep reinforcement learning | |
CN114499629B (en) | Dynamic allocation method for jumping beam satellite system resources based on deep reinforcement learning | |
CN110167176B (en) | Wireless network resource allocation method based on distributed machine learning | |
CN115441939B (en) | MADDPG algorithm-based multi-beam satellite communication system resource allocation method | |
CN107613556B (en) | Full-duplex D2D interference management method based on power control | |
CN112583453A (en) | Downlink NOMA power distribution method of multi-beam LEO satellite communication system | |
CN114189891B (en) | Unmanned aerial vehicle heterogeneous network energy efficiency optimization method based on deep reinforcement learning | |
CN111246485B (en) | Internet of vehicles resource allocation method under high-density vehicle-mounted communication environment | |
CN114786258A (en) | Wireless resource allocation optimization method and device based on graph neural network | |
CN115412936A (en) | IRS (intelligent resource management) assisted D2D (device-to-device) system resource allocation method based on multi-agent DQN (differential Quadrature reference network) | |
CN113259944B (en) | RIS auxiliary frequency spectrum sharing method based on deep reinforcement learning | |
CN114599099A (en) | 5G satellite-ground link multi-beam dynamic power distribution method based on reinforcement learning | |
CN114630299A (en) | Information age-perceptible resource allocation method based on deep reinforcement learning | |
Leng et al. | User-level scheduling and resource allocation for multi-beam satellite systems with full frequency reuse | |
CN117412391A (en) | Enhanced dual-depth Q network-based Internet of vehicles wireless resource allocation method | |
CN116684852B (en) | Mountain land metallocene forest environment unmanned aerial vehicle communication resource and hovering position planning method | |
CN117460034A (en) | Intelligent reflection surface relay-assisted Internet of vehicles power distribution and user scheduling method, system, equipment and medium | |
CN116896407A (en) | Multi-proxy A3C algorithm-based multi-beam satellite communication system resource allocation method | |
CN116566465A (en) | Multi-domain resource allocation method for avoiding low-orbit satellite interference based on beam hopping | |
CN116505998A (en) | Multi-beam satellite communication resource distribution system and method based on deep reinforcement learning | |
Yuan et al. | Joint Multi-Ground-User Edge Caching Resource Allocation for Cache-Enabled High-Low-Altitude-Platforms Integrated Network |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |