CN115412157A - Emergency rescue oriented satellite energy-carrying Internet of things resource optimal allocation method - Google Patents
Emergency rescue oriented satellite energy-carrying Internet of things resource optimal allocation method Download PDFInfo
- Publication number
- CN115412157A CN115412157A CN202211016976.9A CN202211016976A CN115412157A CN 115412157 A CN115412157 A CN 115412157A CN 202211016976 A CN202211016976 A CN 202211016976A CN 115412157 A CN115412157 A CN 115412157A
- Authority
- CN
- China
- Prior art keywords
- low
- orbit satellite
- leo
- network
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B7/00—Radio transmission systems, i.e. using radiation field
- H04B7/14—Relay systems
- H04B7/15—Active relay systems
- H04B7/185—Space-based or airborne stations; Stations for satellite systems
- H04B7/1851—Systems using a satellite or space-based relay
- H04B7/18513—Transmission in a satellite or space-based system
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B17/00—Monitoring; Testing
- H04B17/30—Monitoring; Testing of propagation channels
- H04B17/391—Modelling the propagation channel
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B7/00—Radio transmission systems, i.e. using radiation field
- H04B7/14—Relay systems
- H04B7/15—Active relay systems
- H04B7/185—Space-based or airborne stations; Stations for satellite systems
- H04B7/1851—Systems using a satellite or space-based relay
- H04B7/18519—Operations control, administration or maintenance
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W16/00—Network planning, e.g. coverage or traffic planning tools; Network deployment, e.g. resource partitioning or cells structures
- H04W16/02—Resource partitioning among network components, e.g. reuse partitioning
- H04W16/10—Dynamic resource partitioning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W16/00—Network planning, e.g. coverage or traffic planning tools; Network deployment, e.g. resource partitioning or cells structures
- H04W16/22—Traffic simulation tools or models
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/16—Central resource management; Negotiation of resources or communication parameters, e.g. negotiating bandwidth or QoS [Quality of Service]
- H04W28/18—Negotiating wireless communication parameters
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
The invention provides an emergency rescue-oriented satellite energy-carrying Internet of things resource optimal allocation method, which comprises the following steps of: s1, constructing a network system model, including constructing a low earth orbit satellite LEO auxiliary emergency rescue network model; s2, a low orbit satellite LEO is adopted to assist an emergency rescue network resource allocation strategy; s3, combining a Dueling DQN algorithm and a SCDQN algorithm on the basis of a Markov decision process problem model and the DQN algorithm; and S4, low orbit satellite LEO assisted emergency rescue network resource allocation based on the improved D3QN algorithm. The invention provides a low-orbit satellite LEO auxiliary emergency rescue network resource allocation strategy in consideration of the influence of climate change in an emergency rescue network on network performance.
Description
Technical Field
The invention relates to the field of low-orbit satellite internet of things wireless communication, relates to a low-orbit satellite LEO auxiliary emergency rescue network model, and particularly relates to an emergency rescue-oriented satellite energy-carrying internet of things resource optimal allocation method.
Background
In a low-orbit satellite LEO auxiliary emergency rescue network, a low-orbit satellite LEO with a single antenna is used as a mobile base station to provide communication and energy supply services for a plurality of survival nodes with single antennas in an emergency rescue area, and different nodes have differences in energy consumption rate and can fluctuate under the influence of climate.
Disclosure of Invention
The invention aims to solve the technical problem of providing a low-orbit satellite LEO auxiliary emergency rescue network resource allocation strategy by considering the influence of climate change in an emergency rescue network on network performance, and providing an emergency rescue-oriented satellite energy-carrying Internet of things resource optimal allocation method.
An emergency rescue-oriented satellite energy-carrying Internet of things resource optimal allocation method comprises the following steps:
constructing a low-orbit satellite LEO auxiliary emergency rescue network model, and adopting a low-orbit satellite LEO auxiliary emergency rescue network resource allocation strategy; on the basis of a Markov decision process problem model and a DQN algorithm, a Dueling DQN algorithm and a SCDQN algorithm are combined, and low orbit satellite LEO assisted emergency rescue network resource allocation based on an improved D3QN algorithm is achieved.
S1, constructing a network system model, which mainly comprises a low orbit satellite LEO auxiliary emergency rescue network model, a node energy consumption model, a system channel model and a service model.
Step 101, constructing an LEO auxiliary emergency rescue network system model
Decomposing a low orbit satellite LEO flight period T into N time slots with the same size, wherein the duration of each time slot is delta N = T/N, the low-orbit satellite LEO traverses the nodes in the specified order within the flight period T, assuming that the position of the low-orbit satellite LEO is fixed and known within each time slot.
Each time slot is divided into two sections, in the first section of time slot, the low-orbit satellite LEO broadcasts radio frequency signals through a downlink, the surviving nodes respectively collect energy and decode information based on the power distribution ratio, in the second section of time slot, the equal division is carried out according to the number of the surviving nodes in the communication range of the low-orbit satellite LEO, and the low-orbit satellite LEO only communicates with one node in each sub-time slot.
Step 102, constructing a node energy consumption model
For surviving nodes
Indicates that the node position is [ x ] k ,y k ]. For node energy consumption, set μ k (n) represents the energy consumption rate of surviving node k at time slot n, the energy consumption rate at the beginning of each time slot is represented as:
μ k (n+1)＝μ k (n)+ξ k (n) (1)
xi therein k And (n) is the variation of the energy consumption rate of the nodes under the influence of the climate, and represents the variation of the energy consumption rate of the surviving nodes in each time slot.
Step 103, constructing a system channel model
The power gain of the communication link between the low-orbit satellite LEO and the surviving node k in the downlink channel and the uplink channel is respectively expressed as h k (n) and g k (n) weather loss induced channel fading is A air The channel power gain between the low orbit satellite LEO and the surviving node is:
wherein
Indicates the reference distance d 0 Channel power gain, f, when =1m c Representing carrier frequency, c light speed, d k (n) is the distance between the low-orbit satellite LEO and the surviving node k,
representing the path loss exponent.
(1) Sand blown by wind model
The channel fading effect of wind sand on the wireless channel is expressed as:
wherein N (r) represents the distribution function of the sand particle concentration, σ est (r) is the attenuation section of the sand particles, and the channel model under the influence of the sand-wind climate is expressed as
(2) Snowing model
The channel fading impact of dry snow weather on the wireless channel is expressed as:
A snow ＝a×R b (5)
wherein a and b are both constants, respectively a =5.42 × 10 -5 X λ +5.4958776, b =1.38, r represents the snowfall amount in mm/h, and the corresponding channel model is expressed as
Step 104, building a system service model
The horizontal position of the LEO of the low-orbit satellite in the time slot n is represented as x u (n),y u (n)]The maximum coverage radius of the low orbit satellite LEO is D. The low-orbit satellite LEO can traverse the nodes according to a set track during emergency rescue communication to serve, and simultaneously sends radio frequency signals to the surviving nodes in the current coverage range to realize communication and energy transmission.
P for downlink transmission power of low orbit satellite LEO in time slot n d Indicating that the low orbit satellite LEO transmit power is constant. All surviving nodes in the LEO coverage area of the low orbit satellite will receive the radio frequency signal and distribute the ratio beta according to the power n Respectively perform energy collection andand (5) decoding the information. The energy received power of surviving node k is expressed as
The total energy collected by surviving node k at slot n is represented as:
wherein
The efficiency of the surviving nodes in converting the received energy.
The surviving node k has the power for information decoding of
The signal-to-noise ratio of the corresponding information decoding is expressed as:
wherein
The noise figure generated in the conversion process of the received signal.
The information transmission power of the surviving node k on the uplink is through P u P representing, i.e. different surviving nodes u The same is true. At the beginning of each timeslot, the low-orbit satellite LEO moves to the target surviving node location and serves, and then sends information and energy through the downlink, informing its coverage area memory surviving node to send information through the uplink. According to the shannon formula, the data rate uploaded by the current survival node k is as follows:
where W is the bandwidth of the wireless communication,
is the noise power of the channel between the low orbit satellite LEO and the surviving node.
S2, a low-orbit satellite LEO auxiliary emergency rescue network resource allocation algorithm comprises the following steps:
step 201, defining a non-ideal network service cost function
The network survivability is analyzed from the aspects of information decoding, energy collection and information transmission of the survival nodes.
And defining omega (tau, beta) to describe the non-ideal network service cost function corresponding to three conditions of invalid information decoding, invalid energy transmission and invalid information transmission.
Step 202, defining a non-ideal network service cost function under the condition of invalid information decoding
The node ensures that the signal-to-noise ratio for information decoding is always higher than a given threshold value gamma I I.e. SINR k (n)≥γ I . If the signal-to-noise ratio of the information decoding power cannot be met, the surviving node is considered to be incapable of analyzing the information carried by the downlink signal, and a communication request cannot be actively initiated to the low orbit satellite LEO in the information transmission stage. The corresponding non-ideal network service cost function is expressed as:
step 203, defining a non-ideal network service cost function under the condition of invalid energy transmission
Suppose that the difference between the total energy obtained by the surviving node k at time slot n and the node energy consumption needs to be greater than the threshold γ E I.e. the ineffective energy transmission is denoted as E k (n)-(μ k (n)+ξ k (n))τ[n]＜γ E The corresponding non-ideal network service cost function is expressed as:
step 204, defining a non-ideal network service cost function under the condition of invalid information transmission
Surviving node k in sub-slot τ k [n]The total amount of information uploaded in the medium is R k (n)τ k [n]Assuming that the lowest information quantity uploaded by the surviving node k each time is phi, wherein phi is a constant value, when the information transmission quantity of the surviving node is less than phi, the requirement of the minimum information transmission quantity is determined to be not met, and the corresponding non-ideal network service cost function is expressed as:
the problem of guaranteeing the network survivability and network performance of the emergency rescue network under the influence of climate is converted into the problem of guaranteeing that the total omega (tau, beta) of the low-orbit satellite LEO in the process of assisting the emergency rescue network is minimum, and the target optimization problem is defined as follows:
among the constraints of the optimization problem, C1 and C2 are constraints of the service time slot allocation and power allocation ratio of the low-orbit satellite LEO during the service of the surviving node.
S3, establishing a problem model in a Markov decision process, comprising the following steps:
step 301, defining a state space model in a Markov decision process
In a low-orbit satellite LEO-assisted emergency rescue network scene, a state space is determined by K survival nodes and environment information together, in the nth time slot, the distances between all the nodes and the low-orbit satellite LEO are represented by dis (n), the energy consumption rate information of all the nodes is mu (n), and the channel fading state between all the survival nodes and the low-orbit satellite LEO is C (n). Therefore, the system state of the nth slot may be defined as:
step 302, define the motion space model in the Markov decision process
The LEO of the low orbit satellite can be based on the current system state s n Making a decision by taking action a n The next state is reached where the actions include downlink transmit power allocation and time slot allocation during service for the low orbiting satellite LEO. Namely, the actions that the low orbit satellite LEO can take as an agent are:
wherein the actions of β (n) and τ (n) are each uniformly dispersed over their range into m values, where m is a degree of freedom.
Step 303, define the reward function model in the Markov decision process
The reward function of each state is defined as a negative form of a sum of non-ideal network service cost functions of LEOs serving low-orbit satellites in the network at each time, and is expressed as follows:
r s (n)＝-(ω I (τ,β)+ω E (τ,β)+ω D (τ,β)) (15)
wherein ω is I (τ,β),ω B (τ,β),ω D (tau, beta) is a non-ideal network service cost function value during the period that the low-orbit satellite LEO service provides service for the target survival node, if non-ideal network service conditions such as invalid information decoding, energy transmission and information transmission do not exist, the reward function is 0, and if any condition occurs, the reward function is a negative value.
S4, the resource allocation algorithm based on the improved D3QN comprises the following steps:
step 401, constructing an improved SCD3QN network architecture
In the SCD3QN algorithm, an estimation value network and a target value network both adopt a DuelingDQN structure, and the state value and the action value are respectively evaluated to replace the action value evaluation in the original DQN, namely a state function V (s; theta, beta) and an advantage function A (s, a; theta, alpha) are set, and a Q function output by the network is expressed as:
and the mean deviation is introduced in the Q value calculation process to improve the stability of the algorithm.
The training of DuelingDQN is realized based on SelfCorrctingDQN, and Q value is estimated unbiased by introducing a self-correction estimator into an estimated value network and a target value network, wherein the self-correction estimator is expressed as follows:
q-based in evaluating networks γ (s n+1 ,a n ) Is calculated, i.e.
Target value y for SCD3QN target network n The calculation is as follows:
step 402, constructing an improved SCD3 QN-based resource allocation algorithm.
The technical scheme of the invention has the following technical effects:
1. constructing a model: the system mainly comprises a low-orbit satellite LEO auxiliary emergency rescue network model, a node energy consumption model, a system channel model and a service model. A low-orbit satellite LEO provided with a single antenna is taken as a mobile base station, communication and energy supply services are provided for survival nodes provided with multiple single antennas in an emergency rescue area, different nodes have differences in energy consumption rate, and fluctuation can occur under the influence of climate. (1) Because of different hardware factors and deployment locations, the energy consumption rates of different nodes are different. And (4) considering that the node energy consumption fluctuation is caused by climate influence, and constructing a node energy consumption model. (2) Considering the high-altitude flight advantage of the low-orbit satellite LEO, the low-orbit satellite LEO and the survival node can be considered to always establish a line-of-sight link for communication, and therefore, a channel link between the low-orbit satellite LEO and the survival node is modeled. (3) The low-orbit satellite LEO can traverse the nodes for service according to a set track during emergency rescue communication, and simultaneously sends radio frequency signals to the surviving nodes in the current coverage range to realize communication and energy transmission, so that system service modeling is carried out on the surviving nodes.
2. The method comprises the steps of considering the influence of climate change in the emergency rescue network on network performance, providing stable energy supply and information communication service for survival nodes to guarantee network survivability and network performance by jointly optimizing a power distribution ratio and service time slot distribution, and providing an optimal resource distribution strategy of the low-orbit satellite LEO auxiliary emergency rescue network based on a power division structure. Defining omega (tau, beta) to describe a non-ideal network service cost function corresponding to three conditions of invalid information decoding, invalid energy transmission and invalid information transmission, and ensuring that the total omega (tau, beta) of the low-orbit satellite LEO in the process of assisting the emergency rescue network reaches the minimum
3. And constructing a problem model according to a Markov decision process, and constructing a state space, an action space and a reward function of the LEO of the low-orbit satellite in the network for each service.
4. Based on a deep reinforcement learning algorithm, the resource allocation optimization in the low orbit satellite LEO auxiliary emergency rescue network is realized,
5. according to the method, nash equilibrium solutions of an FCFS (fast forward forwarding file) queuing mechanism and an LCFSSW (low-level-shift-state-switching) queuing mechanism under a queuing game model can be obtained through calculation, so that the optimal pricing strategy of the MCS network is realized. And selecting a Self-correcting DQN algorithm as a basic algorithm. And (4) combining the Dueling DQN algorithm and the SCDQN algorithm to provide an improved D3QN algorithm.
Drawings
FIG. 1 is a flow chart of a network system model building process
FIG. 2 is a flow chart of a low orbit satellite LEO assisted emergency rescue network resource allocation algorithm;
FIG. 3 is a flow diagram of a problem model in a Markov decision process;
FIG. 4 is a flow chart of a resource allocation algorithm based on the improved D3 QN;
FIG. 5 illustrates a low orbit satellite LEO assisted emergency rescue network transport protocol;
fig. 6 is a schematic diagram of SCD3QN network architecture.
Detailed Description
The following detailed description of the present invention is provided in conjunction with the accompanying drawings, but it should be understood that the scope of the present invention is not limited to the specific embodiments.
In fig. 1, constructing a network system model specifically includes:
specifically, the network system model is constructed and mainly comprises a low orbit satellite LEO auxiliary emergency rescue network model, a node energy consumption model, a system channel model and a service model.
Step 101, constructing an LEO auxiliary emergency rescue network system model
A low-orbit satellite LEO provided with a single antenna is taken as a mobile base station, communication and energy supply services are provided for survival nodes provided with multiple single antennas in an emergency rescue area, different nodes have differences in energy consumption rate, and fluctuation can occur under the influence of climate.
The transmission protocol, as shown in fig. 5, decomposes the low orbit satellite LEO flight period T into equal size N time slots, each time slot having a duration δ N = T/N, the low-orbit satellite LEO traverses the nodes in the specified order within the flight period T, assuming that the position of the low-orbit satellite LEO is fixed and known within each time slot.
Each time slot can be divided into two sections, in the first section of time slot, the low-orbit satellite LEO broadcasts radio frequency signals through a downlink, the survival nodes respectively carry out energy collection and information decoding based on the power distribution ratio, in the second section of time slot, the survival nodes are equally divided according to the number of the survival nodes in the LEO communication range of the low-orbit satellite, and the low-orbit satellite LEO only communicates with one node in each sub-time slot.
Step 102, constructing a node energy consumption model
In particular, in this scenario, the surviving node is used
Showing that the node position is [ x ] k ,y k ]. For node energy consumption, set μ k (n) represents the energy consumption rate of surviving node k in time slot n, due to different hardware factors and deployment locations, mu of different nodes k (n) there is a difference. Considering that climate influences may cause node energy consumption fluctuations, the energy consumption rate at the beginning of each time slot is expressed as:
μ k (n+1)＝μ k (n)+ξ k (n) (1)
in which ξ k And (n) is the variation of the energy consumption rate of the nodes under the influence of the climate, and represents the variation of the energy consumption rate of the surviving nodes in each time slot.
Step 103, constructing a system channel model
Specifically, considering the high-altitude flight advantage of the low-orbit satellite LEO, the low-orbit satellite LEO and the surviving node can be considered to establish a line-of-sight link for communication all the time. The power gain of a communication link between a low-orbit satellite LEO and a surviving node k in a downlink channel and an uplink channel can be respectively expressed as h k (n) and g k (n), i.e. the channel power gain between the low orbiting satellite LEO and the surviving nodes, is:
wherein
Represents a reference distance d 0 Channel power gain, f, at =1m c Representing carrier frequency, c light speed, d k (n) is the distance between the low-orbit satellite LEO and the surviving node k,
representing the path loss exponent.
The definition of the channel gain in the above equation only considers the communication link model between the low-orbit satellite LEO and the surviving node in an ideal scene, and the channel link quality may fluctuate under the influence of climate change in an actual scene. In an emergency rescue scene, common weather after disasters such as sand blown by wind, haze, rain, snow and the like frequently appears, and channel fading caused by weather loss is A air The integration of the corresponding channel fading into the channel model is expressed as:
(1) Sand blown model
Wind sand is as common post-disaster phenomenon, and a large amount of dust flies upward and can lead to the extreme deterioration of visibility, and the dust particle frequently rubs and collides and can produce strong electrostatic field in strong dust weather moreover for wireless channel receives serious influence in this environment. According to the existing research on the influence of wind, sand and weather, the visibility of wind and sand is in negative correlation with channel fading, namely, the wireless channel is subjected to fading increased along with the reduction of the visibility of wind and sand. The channel fading effect of wind sand on the wireless channel is expressed as:
wherein N (r) represents the distribution function of the sand particle concentration, σ est (r) is the attenuation section of the sand particles, and the channel model under the influence of wind-sand weather is expressed as
(2) Snowing model
Channel fading caused by snowfall is relatively complex, and researches on a snowfall model can be divided into a dry snow model and a wet snow model according to water content, wherein the influence of a dry snow scene on a wireless channel is relatively large. According to the existing research, the fading characteristics of the wireless channel in the snowing scene are similar to those in the rainfall scene, and the wireless channel is positively correlated with the snowing intensity. The channel fading impact of dry snow weather on the wireless channel is expressed as:
A snow ＝a×R b (5)
wherein a and b are both constants, respectively a =5.42 × 10 -5 X lambda +5.4958776, b =1.38, r represents the snowfall amount in mm/h, and the corresponding channel model is represented as
Step 104, building a system service model
In particular, given the limited coverage of low-orbit satellite LEO, low-orbit satellite LEO only broadcasts information, transmits energy, and provides information transmission services to surviving nodes within the coverage. Because the communication efficiency is low and stable service cannot be provided if the LEO of the low-orbit satellite is too far away from the survival node, the LEO of the low-orbit satellite has a horizontal position in the time slot n represented by x u (n),y u (n)]The maximum coverage radius of the low orbit satellite LEO is D. And the low-orbit satellite LEO can traverse the nodes according to a set track to serve during emergency rescue communication, and simultaneously sends radio frequency signals to the surviving nodes in the current coverage range to realize communication and energy transmission.
P for downlink transmission power of low orbit satellite LEO at time slot n d Indicating that the low orbit satellite LEO transmission power is constant. All surviving nodes in the LEO coverage area of the low-orbit satellite receive radio frequency signals and distribute the ratio beta according to power n Energy collection and information decoding are respectively carried out. Based on the channel model mentioned in the foregoing, the energy received power of the surviving node k is expressed as
The total energy collected by surviving node k at slot n is therefore expressed as:
wherein
The efficiency of the surviving nodes in converting the received energy.
Accordingly, the surviving node k has the power for decoding the information of
The signal-to-noise ratio of the corresponding information decoding is expressed as:
wherein
The noise figure generated in the conversion process of the received signal.
Information transmission power of surviving node k on uplink is through P u P representing, i.e. different surviving nodes u The same is true. At the beginning of each timeslot, the low-orbit satellite LEO moves to the target surviving node location and serves, and then sends information and energy through the downlink, informing its coverage area memory surviving node to send information through the uplink. According to the shannon formula, the data rate uploaded by the current survival node k is as follows:
where W is the bandwidth of the wireless communication,
is the noise power of the channel between the low orbit satellite LEO and the surviving node.
Fig. 2 is a resource allocation algorithm for providing a low-orbit satellite LEO-assisted emergency rescue network, which includes:
step 201, defining a non-ideal network service cost function
The influence of climate change in the emergency rescue network on the network performance is considered, and stable energy supply and information communication service are provided for the surviving nodes through jointly optimizing the power distribution ratio and the service time slot distribution to guarantee the network survivability and the network performance.
The network survivability is analyzed from the aspects of information decoding, energy collection and information transmission of the survival nodes.
In the aspect of information decoding, due to climate change, channel link fluctuation is caused, if the signal-to-noise ratio of information decoding is smaller than a certain threshold, a surviving node cannot acquire corresponding information from a radio frequency signal and cannot transmit information in a corresponding time slot, and invalid information decoding is caused;
in the aspect of energy collection, the influence of climate on a channel link is considered, the receiving power of a survival node is reduced, the energy consumption rate of the node is increased, if the difference value between the energy collected by the survival node and the energy consumption of the survival node is smaller than a certain threshold value, energy transmission is considered to be invalid, and the energy void problem of the survival node is caused under severe conditions;
in the uplink information transmission process of the surviving node, the influence of climate change on a channel link can be radiated to the information transmission rate of the surviving node, and the surviving node needs more time to meet the information transmission requirement.
The optimization of the low-orbit satellite LEO on service time slot allocation and power allocation ratio during service is the key to solve the above problems, and this section proposes an optimal resource allocation strategy based on a power division structure, and defines ω (τ, β) to describe non-ideal network service cost functions corresponding to three situations of invalid information decoding, invalid energy transmission, and invalid information transmission.
Step 202, defining a non-ideal network service cost function under the condition of invalid information decoding
In order to realize continuous information transmission, the node needs to ensure that the signal-to-noise ratio for information decoding is always higher than that for information decodingGiven threshold value gamma I I.e. SINR k (n)≥γ I . If the signal-to-noise ratio of the information decoding power cannot be met, the survival node is considered to be unable to analyze the information carried by the downlink signal, and a communication request cannot be actively initiated to the low orbit satellite LEO in the information transmission stage. The corresponding non-ideal network service cost function is expressed as:
step 203, defining a non-ideal network service cost function under the condition of invalid energy transmission
Under the same transmission power and power distribution ratio, under the influence of climate, energy acquired through radio frequency signals of survival nodes in the emergency rescue network may not support the normal work of the survival nodes, so that ineffective energy transmission is caused, and it is assumed that the difference between the total energy acquired by the survival nodes k at time slots n and the energy consumption of the nodes needs to be greater than a threshold value gamma E I.e. the ineffective energy transfer is denoted by E k (n)-(μ k (n)+ξ k (n))τ[n]＜γ E The corresponding non-ideal network service cost function is expressed as:
step 204 defines a non-ideal network service cost function in case of invalid information transmission
Surviving node k in sub-slot τ k [n]The total amount of information uploaded in the medium is R k (n)τ k [n]Wherein R is k (n) the information transmission rate is less than that in normal environment due to weather influence, and the same information amount needs larger tau k [n]The information transmission can be guaranteed to be completed, assuming that the lowest information quantity uploaded by the surviving node k each time is phi, wherein phi is a constant value, when the information transmission quantity of the surviving node is less than phi, it is determined that the requirement of the minimum information transmission quantity cannot be met, and the corresponding non-ideal network service cost function is represented as:
the problem of guaranteeing the network survivability and network performance of the emergency rescue network under the influence of climate can be further converted into the problem of guaranteeing that the total omega (tau, beta) of the low-orbit satellite LEO in the process of assisting the emergency rescue network is minimum, and the target optimization problem is defined as follows:
among the constraints of the optimization problem, C1 and C2 are constraints of the service time slot allocation and power allocation ratio of the low-orbit satellite LEO during the service of the surviving node. In the above optimization objective, ω I (τ, β) depends on the transmission power allocation ratio, i.e., in order to reduce invalid information decoding situations, the power allocation for information decoding needs to be increased; for omega E (τ, β), the total amount of energy collected is the downlink duration time slot τ 0 [n]Is thus a function of τ 0 [n]The smaller the probability of an ineffective energy transfer, on the other hand with the power division ratio beta n The probability of ineffective energy transfer is also reduced; omega D The change of (tau, beta) is related to the change of the current wireless channel and the time slot allocation of the uplink, and the invalid information transmission situation is correspondingly reduced along with the increase of the information transmission time slot.
From the above, it can be seen that the three sub-optimization objectives conflict with each other to some extent. Considering the complex channel fluctuation characteristics and the dynamic characteristics of the surviving nodes in the scene, finding the optimal resource allocation is a very complex problem, which results in a considerable computation cost, and the traditional model-based methods, such as a dynamic planning method, cannot effectively solve the problem.
In recent years, deep reinforcement learning DRL has shown superior ability to solve complex problems, with greater comprehension and decision-making ability relative to RL. DQN is used as a classic DRL algorithm, is suitable for the resource allocation problem under a discrete action space, considers the defects of the traditional DQN algorithm and the DoubleDQN algorithm, realizes the Dueling-SCDQN algorithm based on the Self-correcting DQN algorithm in the section, and provides a low orbit satellite LEO auxiliary emergency rescue network resource allocation algorithm based on an improved D3QN algorithm.
FIG. 3, presenting a problem model in a Markov decision process, includes:
step 301, defining a state space model in a Markov decision process
In a low-orbit satellite LEO assisted emergency rescue network scene, a state space is determined by K survival nodes and environment information together, in an nth time slot, the distances between all the nodes and the low-orbit satellite LEO are represented by dis (n), the energy consumption rate information of all the nodes is mu (n), and the channel fading state between all the survival nodes and the low-orbit satellite LEO is C (n). Therefore, the system state of the nth slot may be defined as:
step 302, define motion space model in Markov decision process
The LEO of the low orbit satellite can be based on the current system state s n Making a decision by taking action a n The next state is reached where actions include downlink transmit power allocation and time slot allocation during service by the low orbit satellite LEO. Namely, the actions that the low orbit satellite LEO can take as an agent are:
wherein the actions of β (n) and τ (n) are each uniformly dispersed over their range into m values, where m is a degree of freedom.
Step 303, define the reward function model in the Markov decision process
Since the environment is partially observable, low orbit satellite LEO relies on rewards to evaluate its decisions, infer state distributions, and obtain environmental information. In addition, the agent also relies on the reward function to learn the control strategy. According to the optimal resource allocation strategy in the previous section, the reward function of each state is defined as the negative form of the sum of the non-ideal network service cost functions of the LEO of the low-orbit satellite in the network for each service, and is expressed as:
r s (n)＝-(ω I (τ,β)+ω E (τ,β)+ω D (τ,β)) (15)
wherein ω is I (τ,β),ω B (τ,β),ω D (tau, beta) is a non-ideal network service cost function value during the period that the low-orbit satellite LEO service provides service for the target survival node, if non-ideal network service conditions such as invalid information decoding, energy transmission and information transmission do not exist, the reward function is 0, and if any condition occurs, the reward function is a negative value.
Fig. 4 proposes a resource allocation algorithm based on the improved D3QN, which includes:
step 401, constructing an improved SCD3QN network architecture
For the selection of the algorithm, the state influenced by climate is considered to be special, the low-orbit satellite LEO adopts different decisions to obtain similar reward values, and the reward values are all at a lower level, and aiming at the problem, the DuelingDQN algorithm which can independently predict the action and state value is selected [73] As a base algorithm. Considering that calculation of a target Q value in a DQN algorithm is realized based on a greedy method, which causes an over-estimation problem, so that a large deviation exists in an algorithm model, a DoubleDQN algorithm adopts two separately updated action cost functions to respectively carry out action selection and value evaluation, although the over-estimation problem is solved, the under-estimation problem of value evaluation is introduced to a certain extent, the convergence speed is reduced, and the defects of the algorithm are overcomeAnd selecting a Self-correcting Self-correcting DQN algorithm as a basic algorithm. Based on the method, the improved D3QN algorithm is provided by combining the DuelingDQN algorithm and the SCDQN algorithm so as to realize efficient and stable convergence.
As shown in fig. 6, in the SCD3QN algorithm, the estimation value network and the target value network both adopt a dulingdqn structure, and the state value and the action value are evaluated respectively to replace the action value evaluation in the original DQN, that is, a state function V (s; θ, β) and an advantage function a (s, a; θ, α) are set, and a Q function output by the network is expressed as:
and the mean deviation is introduced in the Q value calculation process to improve the stability of the algorithm.
The training of DuelingDQN is realized based on SelfCorrctingDQN, and Q value is estimated unbiased by introducing a self-correcting estimator in an estimated value and target value network, wherein the self-correcting estimator is expressed as:
q-based in evaluating networks γ (s n+1 ,a n ) Is calculated, i.e.
Target value y for SCD3QN target network n The calculation is as follows:
step 402, constructing an improved SCD3 QN-based resource allocation algorithm
The complete low orbit satellite LEO assisted emergency rescue network resource allocation algorithm based on the SCD3QN is as follows:
it will be evident to those skilled in the art that the application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Claims (6)
1. An emergency rescue-oriented satellite energy-carrying Internet of things resource optimal allocation method is characterized by comprising the following steps:
s1, constructing a network system model, including constructing a low earth orbit satellite LEO auxiliary emergency rescue network model;
s2, a low orbit satellite LEO is adopted to assist an emergency rescue network resource allocation strategy;
s3, combining a Dueling DQN algorithm and a SCDQN algorithm on the basis of a Markov decision process problem model and the DQN algorithm;
and S4, low orbit satellite LEO assisted emergency rescue network resource allocation based on the improved D3QN algorithm.
2. The emergency rescue oriented satellite energy-carrying internet of things resource optimization allocation method according to claim 1, characterized by comprising the steps of S1, constructing a network system model, wherein the network system model mainly comprises a low-orbit satellite LEO auxiliary emergency rescue network model, a node energy consumption model, a system channel model and a service model;
step 101, constructing an LEO auxiliary emergency rescue network system model
Decomposing a low orbit satellite LEO flight period T into N time slots with the same size, wherein the duration of each time slot is delta N = T/N, low orbitTraversing nodes of the satellite LEO in a designated order in a flight period T, and assuming that the position of the low-orbit satellite LEO is fixed and known in each time slot;
each time slot is divided into two sections, in the first section of time slot, the low-orbit satellite LEO broadcasts radio frequency signals through a downlink, the survival nodes respectively carry out energy collection and information decoding based on a power distribution ratio, in the second section of time slot, the survival nodes are equally divided according to the number of the survival nodes in the LEO communication range of the low-orbit satellite, and the low-orbit satellite LEO only communicates with one node in each sub-time slot;
step 102, constructing a node energy consumption model
For surviving nodes
Indicates that the node position is [ x ] k ,y k ](ii) a For node energy consumption, set μ k (n) represents the energy consumption rate of surviving node k at time slot n, the energy consumption rate at the beginning of each time slot is represented as:
μ k (n+1)＝μ k (n)+ξ k (n) (1)
in which ξ k (n) is the variable quantity of the energy consumption rate of the nodes under the influence of climate, and represents the energy consumption rate change condition of the surviving nodes in each time slot;
step 103, constructing a system channel model
The power gain of the communication link between the low-orbit satellite LEO and the surviving node k in the downlink channel and the uplink channel is respectively expressed as h k (n) and g k (n) weather loss induced channel fading is A air The channel power gain between the low orbit satellite LEO and the surviving nodes is:
wherein
Represents a reference distance d 0 When =1mPower gain of the channel, f c Representing carrier frequency, c light speed, d k (n) is the distance between the low-orbit satellite LEO and the surviving node k,
represents a path loss exponent;
step 104, building a system service model
The horizontal position of the LEO of the low-orbit satellite in the time slot n is represented as x u (n),y u (n)]The maximum coverage radius of the low orbit satellite LEO is D; the low-orbit satellite LEO can traverse the nodes for service according to a set track during emergency rescue communication, and simultaneously sends radio frequency signals to the surviving nodes in the current coverage range to realize communication and energy transmission;
p for downlink transmission power of low orbit satellite LEO in time slot n d Indicating that the LEO transmitting power of the low-orbit satellite is constant; all surviving nodes in the LEO coverage area of the low orbit satellite will receive the radio frequency signal and distribute the ratio beta according to the power n Respectively carrying out energy collection and information decoding; the energy received power of surviving node k is expressed as
The total energy collected by surviving node k at slot n is represented as:
wherein
Efficiency of conversion of received energy for surviving nodes;
the surviving node k has the power for information decoding of
The signal-to-noise ratio of the corresponding information decoding is expressed as:
wherein
Noise coefficient generated in the conversion process of the received signal;
the information transmission power of the surviving node k on the uplink is through P u Representing, i.e. P, of different surviving nodes u The same; when each time slot starts, the low-orbit satellite LEO moves to the position of a target survival node and serves, then information and energy are sent through a downlink, and the active node in the coverage area of the LEO is informed to send information through an uplink; according to the shannon formula, the data rate uploaded by the current survival node k is as follows:
where W is the bandwidth of the wireless communication,
is the noise power of the channel between the low orbit satellite LEO and the surviving node.
3. The emergency rescue oriented satellite energy-carrying Internet of things resource optimal allocation method as claimed in claim 1,
in step 103, the channel fading caused by weather loss is a air The method comprises the following steps:
(1) Sand blown model
The channel fading effect of wind sand on the wireless channel is expressed as:
whereinN (r) represents the distribution function of the sand particle concentration, σ est (r) is the attenuation section of the sand particles, and the channel model under the influence of the sand-wind climate is expressed as
(2) Snowing model
The channel fading impact of dry snow weather on the wireless channel is expressed as:
A snow ＝a×R b (5)
wherein a and b are both constants, respectively a =5.42 × 10 -5 X lambda +5.4958776, b =1.38, r represents the snowfall amount in mm/h, and the corresponding channel model is represented as
4. The optimal resource allocation method for the energy-carrying internet of things of the satellite for emergency rescue as claimed in claim 1, wherein the S2 low-orbit satellite LEO assisted emergency rescue network resource allocation algorithm comprises:
step 201, defining a non-ideal network service cost function
The survivability of the network is analyzed from the three aspects of information decoding, energy collection and information transmission of the survivable nodes;
defining omega (tau, beta) to describe non-ideal network service cost functions corresponding to three conditions of invalid information decoding, invalid energy transmission and invalid information transmission;
step 202, defining a non-ideal network service cost function under the condition of invalid information decoding
The node ensures that the signal-to-noise ratio for information decoding is always above a given threshold gamma I I.e. SINR k (n)≥γ I (ii) a If the signal-to-noise ratio of the information decoding power cannot be met, the survival node is considered to be incapable of analyzing the information carried by the downlink signal, and a communication request cannot be actively initiated to the low orbit satellite LEO in the information transmission stage; corresponding to non-ideal network service cost functionExpressed as:
step 203, defining a non-ideal network service cost function under the condition of invalid energy transmission
Suppose that the difference between the total energy obtained by the surviving node k at the time slot n and the node energy consumption needs to be greater than the threshold value gamma E I.e. the ineffective energy transfer is denoted by E k (n)-(μ k (n)+ξ k (n))τ[n]＜γ E The corresponding non-ideal network service cost function is expressed as:
step 204, defining a non-ideal network service cost function under the condition of invalid information transmission
Surviving node k in sub-slot τ k [n]The total amount of information uploaded in the medium is R k (n)τ k [n]Assuming that the lowest information quantity uploaded by the survival node k each time is phi, wherein phi is a constant value, when the information transmission quantity of the survival node is less than phi, the requirement of the minimum information transmission quantity is determined to be not met, and the corresponding non-ideal network service cost function is represented as:
the problem of guaranteeing the network survivability and network performance of the emergency rescue network under the influence of climate is converted into the problem of guaranteeing that the total omega (tau, beta) of the low-orbit satellite LEO in the process of assisting the emergency rescue network is minimum, and the target optimization problem is defined as follows:
P1:
s.t.C1:
C2:
among the constraints of the optimization problem, C1 and C2 are constraints of the service time slot allocation and power allocation ratio of the low-orbit satellite LEO during the service of the surviving node.
5. The emergency rescue oriented satellite energy-carrying internet of things resource optimization allocation method according to claim 1, wherein S3, establishing a problem model in a Markov decision process comprises:
step 301, defining a state space model in a Markov decision process
In a low-orbit satellite LEO auxiliary emergency rescue network scene, a state space is determined by K survival nodes and environment information together, in the nth time slot, the distances between all the nodes and the low-orbit satellite LEO are represented by dis (n), the energy consumption rate information of all the nodes is mu (n), and the channel fading state between all the survival nodes and the low-orbit satellite LEO is C (n); therefore, the system state of the nth slot may be defined as:
step 302, define motion space model in Markov decision process
The LEO of the low orbit satellite can be based on the current system state s n Make a decision by taking action a n Reaching a next state in which actions include downlink transmit power allocation and time slot allocation during service by low orbit satellite LEO; namely, the actions that the low orbit satellite LEO can take as an agent are:
wherein the actions of β (n) and τ (n) are each uniformly dispersed over their range into m values, where m is a degree of freedom;
step 303, define the reward function model in the Markov decision process
The reward function of each state is defined as a negative form of the sum of the cost functions of the non-ideal network services of the LEO of the low-orbit satellite in the network for each service, and is expressed as:
r s (n)＝-(ω I (τ,β)+ω E (τ,β)+ω D (τ,β)) (15)
wherein omega I (τ,β),ω B (τ,β),ω D (tau, beta) is a non-ideal network service cost function value during the period that the low orbit satellite LEO service provides service for the target survival node, if the non-ideal network service conditions such as invalid information decoding, energy transmission, information transmission and the like do not exist, the reward function is 0, and if any condition occurs, the reward function is a negative value.
6. The emergency rescue-oriented satellite energy-carrying internet of things resource optimal allocation method according to claim 1, wherein the S4 resource allocation algorithm based on the improved D3QN comprises the following steps:
step 401, constructing an improved SCD3QN network architecture
In the SCD3QN algorithm, an estimation value network and a target value network both adopt a DuelingDQN structure, and the state value and the action value are respectively evaluated to replace the action value evaluation in the original DQN, namely a state function V (s; theta, beta) and an advantage function A (s, a; theta, alpha) are set, and a Q function output by the network is expressed as:
wherein, the mean deviation is introduced in the Q value calculation process to improve the stability of the algorithm;
the training of DuelingDQN is realized based on SelfCorrctingDQN, and Q value is estimated unbiased by introducing a self-correcting estimator in an estimated value and target value network, wherein the self-correcting estimator is expressed as:
q-based in evaluating networks γ (s n+1 ,a n ) Is calculated, i.e.
Target value y for SCD3QN target network n The calculation is as follows:
step 402, constructing an improved SCD3 QN-based resource allocation algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211016976.9A CN115412157A (en) | 2022-08-22 | 2022-08-22 | Emergency rescue oriented satellite energy-carrying Internet of things resource optimal allocation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211016976.9A CN115412157A (en) | 2022-08-22 | 2022-08-22 | Emergency rescue oriented satellite energy-carrying Internet of things resource optimal allocation method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115412157A true CN115412157A (en) | 2022-11-29 |
Family
ID=84161971
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211016976.9A Pending CN115412157A (en) | 2022-08-22 | 2022-08-22 | Emergency rescue oriented satellite energy-carrying Internet of things resource optimal allocation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115412157A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116827515A (en) * | 2023-06-28 | 2023-09-29 | 苏州中析生物信息有限公司 | Fog computing system performance optimization algorithm based on blockchain and reinforcement learning |
-
2022
- 2022-08-22 CN CN202211016976.9A patent/CN115412157A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116827515A (en) * | 2023-06-28 | 2023-09-29 | 苏州中析生物信息有限公司 | Fog computing system performance optimization algorithm based on blockchain and reinforcement learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8509098B2 (en) | Method and apparatus for identifying network connectivity changes in dynamic networks | |
| CN102413535B (en) | Route cognizing method of interstellar links of multi-level satellite communication system | |
| CN114048689B (en) | Multi-unmanned aerial vehicle aerial charging and task scheduling method based on deep reinforcement learning | |
| CN111148069A (en) | Air-ground integrated Internet of vehicles information transmission method based on fog calculation and intelligent traffic | |
| CN115441939B (en) | MADDPG algorithm-based multi-beam satellite communication system resource allocation method | |
| CN113255218B (en) | Unmanned aerial vehicle autonomous navigation and resource scheduling method of wireless self-powered communication network | |
| CN111479239A (en) | Sensor emission energy consumption optimization method of multi-antenna unmanned aerial vehicle data acquisition system | |
| CN114650567B (en) | Unmanned aerial vehicle auxiliary V2I network task unloading method | |
| CN115412157A (en) | Emergency rescue oriented satellite energy-carrying Internet of things resource optimal allocation method | |
| CN113206701A (en) | Three-dimensional deployment and power distribution joint optimization method for unmanned aerial vehicle flight base station | |
| CN114050855A (en) | Channel information self-adaptive oriented intelligent cooperative transmission method between low-orbit satellites | |
| CN115173922B (en) | Multi-beam satellite communication system resource allocation method based on CMADDQN network | |
| CN114980169A (en) | Unmanned aerial vehicle auxiliary ground communication method based on combined optimization of track and phase | |
| CN114205769A (en) | Joint trajectory optimization and bandwidth allocation method based on unmanned aerial vehicle data acquisition system | |
| CN117614507A (en) | Self-adaptive flow unloading method of high-dynamic topology heaven-earth integrated network | |
| CN115483964B (en) | Air-space-ground integrated Internet of things communication resource joint allocation method | |
| CN116847460A (en) | Unmanned aerial vehicle-assisted general sense integrated system resource scheduling method | |
| US20230247484A1 (en) | Adaptive load balancing in a satellite network | |
| CN113079559B (en) | Inter-satellite link power distribution method for medium and low orbit satellite combined networking | |
| CN112243239B (en) | Unmanned aerial vehicle deployment method based on overpass and related device | |
| CN114900827A (en) | Covert communication system in D2D heterogeneous cellular network based on deep reinforcement learning | |
| CN104753783A (en) | Vehicle-mounted short-distance communication network based quickly-converged ant colony routing establishment method | |
| CN110831204B (en) | Power distribution method and system for downlink of high-speed mobile train | |
| Dai et al. | Heuristic computing methods for contact plan design in the spatial-node-based Internet of Everything | |
| CN115412156B (en) | Urban monitoring-oriented satellite energy-carrying Internet of things resource optimal allocation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |