CN113613339B - Channel access method of multi-priority wireless terminal based on deep reinforcement learning - Google Patents
Channel access method of multi-priority wireless terminal based on deep reinforcement learning Download PDFInfo
- Publication number
- CN113613339B CN113613339B CN202110781263.0A CN202110781263A CN113613339B CN 113613339 B CN113613339 B CN 113613339B CN 202110781263 A CN202110781263 A CN 202110781263A CN 113613339 B CN113613339 B CN 113613339B
- Authority
- CN
- China
- Prior art keywords
- channel
- experience
- network
- priority
- protocol
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 230000002787 reinforcement Effects 0.000 title claims abstract description 39
- 230000009471 action Effects 0.000 claims abstract description 62
- 238000012549 training Methods 0.000 claims abstract description 36
- 230000006870 function Effects 0.000 claims abstract description 35
- 238000003062 neural network model Methods 0.000 claims abstract description 27
- 238000004088 simulation Methods 0.000 claims abstract description 25
- 238000013461 design Methods 0.000 claims abstract description 16
- 238000012795 verification Methods 0.000 claims abstract description 13
- 238000004891 communication Methods 0.000 claims abstract description 9
- 230000005540 biological transmission Effects 0.000 claims description 48
- 238000013528 artificial neural network Methods 0.000 claims description 10
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 5
- 238000005516 engineering process Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 2
- 239000003795 chemical substances by application Substances 0.000 description 10
- 229920006395 saturated elastomer Polymers 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000007246 mechanism Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 229910003460 diamond Inorganic materials 0.000 description 3
- 239000010432 diamond Substances 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 108700026140 MAC combination Proteins 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 101100172132 Mus musculus Eif3a gene Proteins 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W74/00—Wireless channel access
- H04W74/04—Scheduled access
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Computer Hardware Design (AREA)
- Geometry (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
The invention belongs to the technical field of wireless communication, and discloses a channel access method of a multi-priority wireless terminal based on deep reinforcement learning, which comprises the following steps: establishing network scenes with different priority services; designing and defining a system model of the protocol, carrying out state space modeling and action space modeling according to the network scene of the protocol, and designing reward functions aiming at different scenes; defining and establishing a neural network model used by the protocol, and training the network model through an experience tuple; and performing performance verification on the trained model through simulation comparison of multiple scenes. The invention designs the channel access method of the multi-priority service wireless terminal by using deep reinforcement learning, is more suitable for wireless networks with different priority services, improves the throughput of the system and the utilization rate of wireless channel resources, reduces the scheduling delay of high priority services and improves the opportunity of accessing channels of low priority services.
Description
Technical Field
The invention belongs to the technical field of wireless communication, and particularly relates to a channel access method of a multi-priority wireless terminal based on deep reinforcement learning.
Background
Currently, with the rapid development of wireless communication technology, the demand for wireless channels for new services such as data transmission and exchange is increasing. In a wireless network, when a plurality of users contend for the use right of a specific resource (e.g., the use right of a shared channel) at the same time, the users transmit data packets by acquiring the use right of the channel, and at this time, since information from different users needs to occupy the channel for transmission, collision of the data packets may be caused, thereby causing communication failure. In order to improve communication efficiency, a multiple access protocol needs to be introduced to determine the use authority of a user on resources, so that the problem that multiple users share the same physical link resource is solved.
YIding Yu et al propose a Deep-reinforcement learning based multiple access protocol (Deep-reinforcement Learning Multiple Access, DLMA) in the literature in wireless heterogeneous networks. Different multiple access algorithms based on deep reinforcement learning are provided for different optimization targets in the literature, and system simulation comparison analysis is performed on the algorithms corresponding to the different targets. Simulation results show that DLMA can achieve the expected goal without knowing the multiple access protocols adopted by other coexisting networks, thereby improving the proportional fairness and the throughput of the system. However, the existing channel access method based on deep reinforcement learning does not consider the service quality requirement difference of the services with different priorities, and cannot well guarantee the service quality requirement of the service with high priority.
Conventional multiple access protocols such as time division multiple access (Time Division Multiple Access, TDMA), ALOHA protocol, code division multiple access (Code Division Multiple Access, CDMA), carrier sense multiple access (Carrier Sense Multiple Access, CSMA), etc. all have a problem of low channel utilization. Therefore, the invention mainly aims at the problems of the multiple access protocol, designs a channel access method of the multiple priority wireless terminal based on deep reinforcement learning in a network scene with multiple priority services, and reduces the scheduling delay of the high priority services under the constraint of ensuring the better throughput of the system.
Through the above analysis, the problems and defects existing in the prior art are as follows:
(1) In a wireless network, when a plurality of users contend for the use of a specific resource at the same time, the users transmit data packets by acquiring the use of a channel, and since information from different users needs to occupy the channel for transmission, collision of the data packets may be caused, resulting in communication failure.
(2) The existing channel access method based on deep reinforcement learning does not consider the service quality requirement difference of the services with different priorities, and cannot well ensure the service quality requirement of the high-priority service.
(3) The traditional multiple access method based on the multiple priority service can not fully utilize wireless channel resources, so that the resource waste is caused.
The difficulty and meaning for solving the problems and the defects are as follows: the invention provides a channel access method of a multi-priority wireless terminal based on deep reinforcement learning, which endows the wireless terminal with learning capability, and interacts with the environment by using a reinforcement learning mechanism so as to further improve the utilization rate of wireless channel resources; in the design of the rewarding function, by setting different rewards for the services with different priorities, the scheduling delay of the service with high priority can be reduced, and meanwhile, the opportunity of accessing the channel to the service with low priority can be improved.
Disclosure of Invention
Aiming at the problems of the existing multiple access protocol, the invention provides a channel access method of a multi-priority wireless terminal based on deep reinforcement learning, in particular to a channel access method of a multi-priority wireless terminal based on deep reinforcement learning.
The invention is realized in such a way that the channel access method of the multi-priority wireless terminal based on the deep reinforcement learning comprises the following steps:
Step one, establishing network scenes with different priority services; and (3) defining network scenes, and modifying the system model and the neural network model according to different network scenes by a user so as to be deployed in different network scenes.
Step two, designing and defining a system model of the protocol, carrying out state space modeling and action space modeling according to the network scene of the protocol, and designing reward functions aiming at different scenes; according to different network scenes, the state space and the action space are subjected to fine tuning and use, and the form of the reward function is designed and modified according to different points (the core of the reward function design is the priority), so that the requirements of different points in different network scenes are better met, and the requirements of actual deployment are more met.
Step three, a neural network model used by the protocol is definitely and established, and the network model is trained through an experience tuple; aiming at different neural network model training processes, the model convergence rates are not very same, and the training process is faster and more accurate by comparing and selecting the better neural network model.
And step four, performing performance verification on the trained model through simulation comparison of multiple scenes. The feasibility and superiority of the invention are verified and illustrated by comparing the multiple access protocol with the service without priority in multiple scenes.
Further, in the first step, the establishing a network scenario with services of different priorities includes:
establishing a transmission network with k priority services, wherein k is greater than 0; the network scene comprises a base station, N DRL-MAC nodes (the nodes adopting the invention), M time division multiple access (Time Division Multiple Access, TDMA) nodes and X q-ALOHA nodes, wherein (N >1; M+X > 1) at least one DRL-MAC node and one other protocol node exist.
The base station is used for acquiring data from a wireless channel between the node and the base station and transmitting the data; the DRL-MAC node adopts a multiple access technology based on deep reinforcement learning, and if the node sends services with different priorities, a transmission result fed back by a base station is obtained and different rewards are obtained according to the services with different priorities; if the node does not send the service, the node will monitor the channel and obtain the transmission state of other nodes in a certain time slot through the channel observation result; the time division multiple access node adopts a TDMA protocol and is used for carrying out service transmission according to regularly and periodically occupying the allocated time slot; the q-ALOHA node adopts q-ALOHA for carrying out service transmission in each time slot with fixed transmission probability q according to q values under different scenes.
Further, in the second step, each DRL-MAC node is equivalent to one agent in reinforcement learning; in each time slot, the agent calculates the Q value in the current state:
wherein q(s) t A, θ) is an approximation of the deep neural network model calculation, selecting action a from the set of actions according to a greedy strategy to maximize rewards overall expectations or better accommodate dynamically changing wireless network environments.
In the second step, the system model of the protocol is designed and defined, state space modeling and action space modeling are performed according to the network scene of the protocol, and reward functions are designed according to different scenes, including:
(1) Motion space modeling
System action set A t ＝{a 0 ,a 1 ,a 2 ......a k Where k is the number of classes of priority traffic in the network scenario. In time slot t, DRL-MAC node needs to make decision a through deep neural network to determine whether to access data packet into channel in current time slot; wherein a is 0 Indicating that the channel is not accessed; a, a 1 ......a k Representing the service access channels with different priorities corresponding to k;
the channel observation result after taking action is Z t E { SUCCESS, COLLISION, IDLENESS }, obtaining channel observations for the construction of the experience tuples by listening to the channel; wherein SUCCESS indicates that the node accesses the channel and transmits the data packet successfully; COLLISION means that a plurality of nodes access channels simultaneously for transmission, resulting in COLLISION; IDLENESS indicates that the channel is idle, i.e., no node accesses the channel; the Agent determines a channel observation result according to the acknowledgement signal from the access point and the interception channel;
(2) State space modeling
State setWherein M history states to be tracked are contained, each history state is observed by action>Composition, of which there are a total of 2k+3 combinations:
(3) Reward function
The principle that the reward function always follows for network scenes of different priority services is as follows: the higher the priority of the service, the higher the rewards brought by successful transmission, and the more the penalties brought by failed transmission; the bonus function is set to sum_repeat=α+s (1- α) ×delay/T; wherein alpha and T are controllable variables, the parameter alpha is used for adjusting the influence value of the time delay on the overall rewards, and when the influence of the time delay on the overall rewards is not considered, the parameter alpha is initialized to 1; the parameter T is used for unifying the influence range of time delay on rewards and is initialized to 50; delay is the time delay of a certain service from generation to an accessed channel, namely, scheduling time delay; rewards are prize values, r, presented for different priority traffic 1 ......r k Rewards corresponding to successful transmission of different priority services, r -1 ......r -k Penalty corresponding to transmission failure for different priority service:
further, in the third step, the defining and establishing the neural network model used by the protocol, and training the network model through the experience tuple, includes:
The DQN is introduced to enable the DRL-MAC node to better learn the use condition of the channel by other nodes to make a next decision, the intelligent agent adopts a depth residual error network architecture for training, the depth residual error network is used for approximating the Q value, the current state s is input, the action strategy a is output, and then an experience tuple is formed by combining other information for training the depth residual error network.
Further, in the third step, the defining and establishing the neural network model used by the protocol, and training the network model through the experience tuple, further includes:
(1) Initializing an experience pool, setting the capacity of the experience pool and initializing parameters;
(2) Starting from time slot t=0;
(3) The Q value of the current state s is calculated in the afferent neural network; selecting an action to be executed through a greedy strategy, and recording a channel observation result z and total rewards sum_forwards obtained after the action is taken; putting the obtained state s, the rewards r obtained after taking action a and the experience tuples (s, a, r, s ') which are formed by the next state s' into an experience pool;
(4) If the current generated experience tuple is larger than the experience pool capacity, discarding the experience tuple which enters the experience pool earliest, and placing the latest experience tuple into the experience pool; otherwise, sequentially entering the experience tuples into an experience pool according to the sequence;
(5) If the current time slot t is a multiple of 10, randomly extracting N experience tuples from the experience pool, and sequentially calculating y values of the experience tuples:where r represents the current prize obtained by taking action a in the current state s, gamma E (0, 1) is the discount factor, ++>Selecting rewards obtained by the action with the maximum Q value for future prediction, otherwise, entering the step (8);
(6) Updating the Q-estimated network parameter θ using a half gradient descent algorithm;
(7) If the current time slot t is a multiple of the parameter F, the Q-estimated network parameter theta is assigned to the target network parameter theta - Otherwise, entering the step (8);
(8) If the time slot t is greater than or equal to the set training round, exiting the training process; otherwise, entering the next time slot t=t+1, and entering the step (3).
Another object of the present invention is to provide a channel access system of a multi-priority service wireless terminal to which the channel access method of a multi-priority service wireless terminal based on deep reinforcement learning is applied, the channel access system of the multi-priority service wireless terminal comprising:
the network scene establishing module is used for establishing network scenes with different priority services;
the system model design module is used for designing and defining a system model of the protocol;
The space modeling module is used for carrying out state space modeling and action space modeling according to the protocol network scene;
the rewarding function design module is used for designing rewarding functions aiming at different scenes;
the neural network model building module is used for definitely and building a neural network model used by the protocol;
the network model training module is used for training the network model through the experience tuple;
and the performance verification module is used for verifying the performance of the trained model through simulation comparison of multiple scenes.
It is a further object of the present invention to provide a computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of:
establishing network scenes with different priority services; designing and defining a system model of the protocol, carrying out state space modeling and action space modeling according to the network scene of the protocol, and designing reward functions aiming at different scenes; defining and establishing a neural network model used by the protocol, and training the network model through an experience tuple; and performing performance verification on the trained model through simulation comparison of multiple scenes.
Another object of the present invention is to provide a computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of:
establishing network scenes with different priority services; designing and defining a system model of the protocol, carrying out state space modeling and action space modeling according to the network scene of the protocol, and designing reward functions aiming at different scenes; defining and establishing a neural network model used by the protocol, and training the network model through an experience tuple; and performing performance verification on the trained model through simulation comparison of multiple scenes.
Another object of the present invention is to provide a wireless communication information data processing terminal for implementing a channel access system of the multi-priority service wireless terminal.
By combining all the technical schemes, the invention has the advantages and positive effects that: the invention provides a channel access method (Deep Reinforcement Learning-based Medium Access Control, DRL-MAC) of a multi-priority service wireless terminal based on deep reinforcement learning, which is implemented by establishing network scenes with different priority services; designing and defining a system model of the protocol, carrying out state space modeling and action space modeling according to the network scene of the protocol, and designing reward functions aiming at different scenes; defining and establishing a neural network model used by the protocol, and training the network model through an experience tuple; and performing performance verification on the trained model through simulation comparison of multiple scenes so as to reduce the scheduling delay of the high-priority service under the constraint of ensuring the throughput of the system. The invention designs the channel access method of the multi-priority service wireless terminal by using deep reinforcement learning, is more suitable for wireless networks with different priority services, improves the throughput of the system and reduces the scheduling delay of high priority services.
Aiming at wireless networks with different priority services, the invention provides a channel access method of a multi-priority service wireless terminal based on deep reinforcement learning, which endows the wireless terminal with learning capability, and interacts with the environment by using a reinforcement learning mechanism so as to further improve the utilization rate of wireless channel resources; in the design of the rewarding function, by setting different rewards for the services with different priorities, the scheduling delay of the service with high priority can be reduced, and meanwhile, the opportunity of accessing the channel to the service with low priority can be improved. The comparison with the multiple access protocol of the non-prioritized service shows that the channel access method of the multi-priority service wireless terminal based on the deep reinforcement learning has better system throughput and scheduling delay of the high-priority service.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments of the present invention will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flow chart of a channel access method of a multi-priority service wireless terminal according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of a channel access method of a multi-priority service wireless terminal according to an embodiment of the present invention.
Fig. 3 is a block diagram of a channel access system of a multi-priority service wireless terminal according to an embodiment of the present invention;
in the figure: 1. a network scene building module; 2. a system model design module; 3. a spatial modeling module; 4. a reward function design module; 5. the neural network model building module; 6. a network model training module; 7. and a performance verification module.
Fig. 4 is a flowchart of a training deep neural network based on deep reinforcement learning according to an embodiment of the present invention.
Fig. 5 is a schematic diagram of a network model according to an embodiment of the present invention.
Fig. 6 is a schematic diagram of simulation results provided by an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the following examples in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Aiming at the problems existing in the prior art, the invention provides a channel access method of a multi-priority wireless terminal based on deep reinforcement learning, and the invention is described in detail below with reference to the accompanying drawings.
As shown in fig. 1, the channel access method of the multi-priority wireless terminal based on deep reinforcement learning provided by the embodiment of the invention comprises the following steps:
s101, establishing network scenes with different priority services;
s102, designing and defining a system model of the protocol, carrying out state space modeling and action space modeling according to a network scene of the protocol, and designing reward functions aiming at different scenes;
s103, defining and establishing a neural network model used by the protocol, and training the network model through an experience tuple;
s104, performing performance verification on the trained model through simulation comparison of multiple scenes.
The schematic diagram of the channel access method of the multi-priority wireless terminal based on deep reinforcement learning provided by the embodiment of the invention is shown in fig. 2.
As shown in fig. 3, a channel access system of a multi-priority service wireless terminal according to an embodiment of the present invention includes:
the network scene establishment module 1 is used for establishing network scenes with different priority services;
a system model design module 2 for designing and specifying a system model of the protocol;
the space modeling module 3 is used for carrying out state space modeling and action space modeling according to the protocol network scene;
A reward function design module 4, configured to design a reward function for different scenes;
the neural network model building module 5 is used for definitely and building a neural network model used by the protocol;
a network model training module 6 for training the network model through the experience tuple;
and the performance verification module 7 is used for performing performance verification on the trained model through multi-scene simulation comparison.
The technical scheme of the invention is further described below by combining the embodiments.
As shown in fig. 1, the DRL-MAC protocol provided by the embodiment of the present invention includes the following steps:
(1) Establishing a wireless network scene containing a plurality of different priority services;
establishing a wireless network with two priority services, wherein the network scene comprises the following steps: a base station, N DRL-MAC nodes (the nodes adopting the invention), M time division multiple access (Time Division Multiple Access, TDMA) nodes, X q-ALOHA nodes, wherein (N >1; M+X > 1) at least one DRL-MAC node and one other protocol node exist.
The base station acquires data from a wireless channel between the node and the base station and transmits the data; the DRL-MAC node adopts a multiple access technology based on deep reinforcement learning, and if the node sends the service with different priority, a transmission result fed back by the base station is obtained and different rewards are obtained according to the service with different priority; if the node does not send the service, the node will monitor the channel and obtain the transmission state of other nodes in a certain time slot through the channel observation result; the time division multiple access node adopts a TDMA protocol, and performs service transmission according to regularly and periodically occupying the allocated time slot; the illustrated q-ALOHA node employs q-ALOHA, which performs traffic transmission with a fixed transmission probability q at each slot according to q values in different scenarios.
(2) Designing and defining a system model of the protocol, carrying out state space modeling and action space modeling according to the network scene of the protocol, and designing reward functions aiming at different scenes;
each DRL-MAC node is equivalent to one agent in reinforcement learning; in each time slot, the agent calculates the Q value in the current state:
wherein q(s) t A, θ) is an approximation of the deep neural network model calculation and action a is selected from the set of actions according to a greedy strategy to maximize rewards overall expectations or better accommodate dynamically changing wireless network environments.
(2.1) motion space modeling
For a network scene with two priority services, the system action set is A t ＝{a 0 ,a 1 ,a 2 The DRL-MAC node needs to make a decision a via the deep neural network at time slot t to determine whether to access the packet to the channel at the current time slot, where a 0 Indicating that the channel is not accessed; a, a 1 Indicating to access the high priority traffic to the channel; a, a 2 Indicating that the low priority traffic is to be accessed to the channel.
The channel observation result after taking action is Z t E { SUCCESS, COLLISION, IDLENESS }, obtaining a channel observation result through a listening channel for the formation of an experience tuple, wherein SUCCESS indicates that a node accesses the channel and transmits a data packet successfully; COLLISION means that a plurality of nodes access channels simultaneously for transmission, resulting in COLLISION; IDLENESS indicates that the channel is idle, i.e., no node has access to the channel. The Agent determines the channel observations based on the acknowledgement signal from the access point (if it is transmitting) and the listening channel (if it is waiting).
(2.2) State space modeling
State setWherein M history states to be tracked are contained, each history state is observed by action>The composition, for a network scenario with two priority traffic, of action-observation pairs is seven in total:
(2.3) reward function
For a network scenario with two priority services, the principle that the reward function always follows is: the higher the priority of the service, the higher the rewards brought by successful transmission; the more penalty it incurs from a transmission failure. The reward function is set as sum_reward=α+1- α (delay/T), where α and T are controllable variables, and the parameter α is used to adjust the impact value of the delay on the overall reward, and when the impact of the delay on the overall reward is not considered, it is initialized to 1; the parameter T is mainly used for unifying the influence range of time delay on rewards and initializing the influence range to 50; delay is the time delay of a certain service from generation to an accessed channel, namely, scheduling time delay; the forwards is a prize value proposed for a network scenario with two priority traffic.
(3) Defining and establishing a neural network model used by the protocol, and training the network model through an experience tuple; FIG. 4 is a flow chart of training a deep neural network based on deep reinforcement learning according to the present invention.
The DQN is introduced to enable the DRL-MAC node to better learn the use condition of the channel by other nodes to make a next decision, the intelligent agent adopts a depth residual error network architecture for training, the depth residual error network is used for approximating the Q value, the current state s is input, the action strategy a is output, and then an experience tuple is formed by combining other information for training the depth residual error network.
And (3.1) initializing an experience pool, setting the capacity of the experience pool and initializing parameters.
(3.2) starting from time slot t=0;
(3.3) afferent the current state s into a neural network to calculate the Q value of the state; selecting an action to be executed through a greedy strategy, and recording a channel observation result z and total rewards sum_forwards obtained after the action is taken; the obtained state s, the prize r obtained after taking action a and the next state s 'are put into an experience pool to form an experience tuple (s, a, r, s').
(3.4) if the currently generated experience tuple is greater than the experience pool capacity, discarding the experience tuple that was first entered into the experience pool, and placing the latest experience tuple into the experience pool; otherwise, the experience tuples are sequentially put into the experience pool.
(3.5) if the current time slot t is a multiple of 10, randomly extracting N experience tuples from the experience pool, and sequentially calculating y values of the experience tuples: Where r represents the current prize obtained by taking action a in the current state s, gamma E (0, 1) is the discount factor, ++>And selecting the rewards obtained by the action with the maximum Q value for future prediction, otherwise, entering the step (3.8).
(3.6) updating the Q-estimated network parameter θ using a half gradient descent algorithm.
(3.7) if the current time slot t is a multiple of the parameter F, assigning the Q-estimated network parameter θ to the target network parameter θ - Otherwise, the step (3.8) is entered.
(3.8) if the time slot t is greater than or equal to the set training round, exiting the training process; otherwise, entering the next time slot t=t+1; and then enter step (3.3).
(4) And performing performance verification on the trained model through simulation comparison of multiple scenes so as to reduce the scheduling delay of the high-priority service under the constraint of ensuring the throughput of the system.
Fig. 5 is a network scenario used in the simulation experiment of the present invention, where the network scenario includes a base station, N DRL-MAC nodes (nodes adopting the present invention) (N > 1), M TDMA nodes, and X q-ALOHA nodes (m+x > 1), where data packets are transmitted between the nodes and the base station through a shared wireless channel.
The technical effects of the present invention will be described in detail with reference to simulation experiments.
1. Simulation conditions:
the simulation experiment of the invention is implemented on a Windows platform, and is mainly configured as follows: CPU is Intel (R) Core (TM) i7-7500U,2.70GHz; the memory is 8G; the operating system is Windows10; the simulation software is PyCharm.
2. Simulation content and result analysis:
the simulation experiment is compared with a model-aware node, wherein the model-aware node is aware of multiple access protocol (multiple access control, MAC) mechanisms of other coexisting nodes, and obtains the optimal MAC protocol coexisting with the node by utilizing the mechanism of the known MAC protocol. The simulation experiment results are shown in fig. 6.
Example one: in a transmission network with two priority services, a network scene comprises a base station, a DRL-MAC node and a TDMA node; the DRL-MAC node is always in a state with traffic to be transmitted (saturated traffic scenario).
Fig. 6 (a) is a throughput result when a DRL-MAC node and a TDMA node coexist in a saturated traffic scenario, and the goal is to achieve the optimal throughput of the system.
The throughput result when the TDMA occupied time slot N is changed from 2 to 9 when the frame length is 10 can be seen from fig. 6 (a). The diagonal filled-in portion and the solid filled-in portion of the histogram represent the throughput of the DRL-MAC node and TDMA node, respectively. The circular marked dashed line is the simulated total throughput, i.e., the total throughput of the system, in coexistence of the DRL-MAC node and the TDMA node. The diamond marked dashed line represents the value of the theoretical optimal system throughput verified by the model aware node. It can be seen from fig. 6 (a) that the circular marked dashed line and the diamond marked dashed line almost coincide. This means that the DRL-MAC node can discover the unused time slots of TDMA by learning without knowing the protocol used by another node.
Fig. 6 (b) is a throughput result when the DRL-MAC node and one TDMA node coexist in a saturated traffic scenario, with the goal of achieving the optimal throughput of the system.
Fig. 6 (b) shows access probabilities of high-priority packets in a scenario where the DRL-MAC node and the TDMA node consider and do not consider traffic priorities. The square marked solid line of fig. 6 (b) shows the access probability of the high priority packet in the scenario where traffic priority is considered; the circular marked solid line indicates the probability of access of high priority packets without considering traffic priority scenarios. From fig. 6 (b) it can be seen that the blue line is in most cases above the red line. It can be concluded that the DRL-MAC node can send the high-priority service more timely in the service priority scenario, so as to ensure the priority communication of the high-priority service.
Example two: under the transmission network with two priority services, the network scene comprises a base station, a DRL-MAC node and a q-ALOHA node; the DRL-MAC node is always in a state with traffic to be transmitted (saturated traffic scenario).
Fig. 6 (c) is a throughput result when the DRL-MAC node and one q-ALOHA node coexist in a saturated traffic scenario, with the goal of achieving the optimal throughput of the system.
Fig. 6 (c) shows the throughput result of q-ALOHA when the access probability q is changed from 0.2 to 0.9 when the q-ALOHA node coexist with the DRL-MAC node in the saturated traffic scenario. The diagonal line filled-in portion and the solid color filled-in portion in fig. 6 (c) represent throughput of the DRL-MAC node and the q-ALOHA node, respectively. The circular marked dashed line is the simulated total throughput in the coexistence of the DRL-MAC node and the q-ALOHA node, i.e. the total throughput of the system. The diamond marked dashed line represents the value of the theoretical optimal system throughput verified by the model aware node. From fig. 6 (c), it can be seen that the circular marked dashed line and the circular marked dashed line almost coincide in most cases. This shows that the DRL-MAC node can obtain the best throughput by learning the policy without knowing that the other node is a q-ALOHA node and the transmission probability q.
Fig. 6 (d) is a throughput result when a DRL-MAC node and a q-ALOHA node coexist in a saturated traffic scenario, and the goal is to achieve fair transmission between nodes.
Fig. 6 (d) is a throughput result of q-ALOHA when the access probability q is changed from 0.2 to 0.6 in the case of implementing the proportional fairness index. The actual throughput of the q-ALOHA node, the DRL-MAC node and the system are respectively represented by a circular marked solid line, a triangular marked solid line and a square marked solid line, and the theoretical optimal throughput of the q-ALOHA node, the DRL-MAC node and the system obtained by the model perception node is respectively represented by a circular marked dotted line, a triangular marked dotted line and a square marked dotted line. As can be seen from fig. 6 (d), in the case of implementing the proportional fairness index, there is some relatively small error between the actual throughput and the theoretical optimal throughput, which may indicate that the DRL-MAC node may implement the proportional fairness index by learning a policy without knowing that another node is a q-ALOHA node and the transmission probability q.
Fig. 6 (e) is a throughput result when the DRL-MAC node and one q-ALOHA node coexist in a saturated traffic scenario, and the goal is to achieve the optimal throughput of the system.
Fig. 6 (e) is a simulation result of q-ALOHA accessing probability for high priority packet when the access probability q is changed from 0.2 to 0.9. The solid line in the figure shows the duty cycle of the Agent access high priority packet action in all access channel actions at different access probabilities q. As can be seen from fig. 6 (e), in a scenario where DRL-MAC and q-ALOHA coexist (in case of unstable environment), high priority traffic can be achieved with a larger access probability than low priority traffic.
Example three: in a transmission network with two priority services, a network scene comprises a base station, a DRL-MAC node and a TDMA node; the DRL-MAC is in an unsaturated traffic scenario.
The unsaturated service scene refers to: each time slot has a data packet arrival, the arrival rates of the data packets with different priorities are different, the arrival probability of the data packets with high priority is defined as 0.3, the arrival probability of the data packets with low priority is defined as 0.7, the data packets with different priorities enter the corresponding data packet queues with different priorities respectively for queuing and waiting, in order to avoid the condition that the queues are empty when access action operation is taken, when the service queues are initialized, a certain amount of data packets with corresponding priorities are respectively enqueued to the service queues with different priorities (in the invention, the queues with high and low priorities are respectively initialized to enqueue 5 data packets), and the condition that the access service queues are empty when training is just started is avoided.
The channel access criteria under the unsaturated service scene are as follows: when action a is taken 1 Namely, accessing low-priority service in the channel, and dequeuing the data packet and discarding the data packet when collision occurs in the transmission process; when action a is taken 2 Namely, high-priority service is accessed in the channel, collision is generated in the transmission process, the data packet is re-queued and placed at the head of the queue; high priority traffic is sent in greater demand due to the greater rewards of high priority traffic, but action a is taken herein for Agent by learning due to the fixed probability of arrival of traffic 2 I.e. when accessing high priority service, when there is no service in the high priority service queue, the action a of accessing high priority service is still executed 2 Actually taking out the low-priority service from the low-priority queue for access, and giving rewards corresponding to the low-priority service if the data packet is successfully transmitted at the moment; if the transmission fails, the corresponding punishment is given to the high-priority data packet, and the data packet is re-accessed into the low-priority queue head.
Fig. 6 (f) is a result obtained in a simulation scenario corresponding to different parameters α under coexistence of a DRL-MAC node and a TDMA node in an unsaturated service scenario, where each simulation result is divided into three parts in the diagram: system throughput, high priority service access probability, and system delay.
It can be seen from fig. 6 (f) that the time delays for the transmission of the different priority traffic for the different parameter a in the lower left corner of each simulation graph are seen to remain almost unchanged when decreasing from 1 to 0.8, while the maximum time delay for the low priority traffic is stepped down from a value close to 80 to a value close to 60. It can be shown from fig. 6 (f) that in this scenario, by adjusting the parameter α, it is ensured that the optimal throughput of the system is achieved by the learning strategy; on the premise that the access probability of the high-priority service is larger than that of the low-priority data packets, the overall time delay of the system is reduced by sacrificing the access probability of some high-priority data packets.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When used in whole or in part, is implemented in the form of a computer program product comprising one or more computer instructions. When loaded or executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL), or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.
The foregoing is merely illustrative of specific embodiments of the present invention, and the scope of the invention is not limited thereto, but any modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present invention will be apparent to those skilled in the art within the scope of the present invention.
Claims (8)
1. The channel access method of the multi-priority wireless terminal based on the deep reinforcement learning is characterized by comprising the following steps of:
step one, establishing network scenes with different priority services;
step two, designing and defining a system model of a protocol, carrying out state space modeling and action space modeling according to a network scene of the protocol, and designing reward functions aiming at different scenes;
step three, a neural network model used by the protocol is definitely and established, and the network model is trained through an experience tuple;
step four, performing performance verification on the trained model through simulation comparison of multiple scenes;
in the second step, the system model of the designed and defined protocol performs state space modeling and action space modeling according to the network scene of the protocol, and designs reward functions aiming at different scenes, including:
(1) Motion space modeling
System action set A t ＝{a 0 ,a 1 ,a 2 ......a k The method comprises the steps that k is the number of types of priority service in a network scene, and in a time slot t, a DRL-MAC node needs to make a decision a through a deep neural network for determining whether to access a data packet into a channel in the current time slot; wherein a is 0 Indicating that the channel is not accessed; a, a 1 ......a k Representing the service access channels with different priorities corresponding to k;
the channel observation result after taking action is Z t E { SUCCESS, COLLISION, IDLENESS }, obtaining channel observations for the construction of the experience tuples by listening to the channel; wherein SUCCESS indicates that the node accesses the channel and transmits the data packet successfully; COLLISION means that a plurality of nodes access channels simultaneously for transmission, resulting in COLLISION; IDLENESS indicates that the channel is idle, i.e., no node accesses the channel; the Agent determines a channel observation result according to the acknowledgement signal from the access point and the interception channel;
(2) State space modeling
State setWherein M history states to be tracked are contained, each history state is observed by action>Composition, of which there are a total of 2k+3 combinations:
(3) Reward function
The principle that the reward function always follows for network scenes of different priority services is as follows: the higher the priority of the service, the higher the rewards brought by successful transmission, and the more the penalties brought by failed transmission; the bonus function is set to sum_repeat=α+s (1- α) ×delay/T; wherein alpha and T are controllable variables, the parameter alpha is used for adjusting the influence value of the time delay on the overall rewards, and when the influence of the time delay on the overall rewards is not considered, the parameter alpha is initialized to 1; the parameter T is used for unifying the influence range of time delay on rewards and is initialized to 50; delay is the time delay of a certain service from generation to an accessed channel, namely, scheduling time delay; rewards are prize values, r, presented for different priority traffic 1 ......r k Rewards corresponding to successful transmission of different priority services, r -1 ......r -k Penalty corresponding to transmission failure for different priority service:
in the third step, the neural network model used by the protocol is defined and built, and the neural network model is trained through experience tuples, and the method further comprises the following steps:
(1) Initializing an experience pool, setting the capacity of the experience pool and initializing parameters;
(2) Starting from time slot t=0;
(3) The Q value of the current state s is calculated in the afferent neural network; selecting an action to be executed through a greedy strategy, and recording a channel observation result z and total rewards sum_forwards obtained after the action is taken; putting the obtained state s, the rewards r obtained after taking action a and the experience tuples (s, a, r, s ') which are formed by the next state s' into an experience pool;
(4) If the current generated experience tuple is larger than the experience pool capacity, discarding the experience tuple which enters the experience pool earliest, and placing the latest experience tuple into the experience pool; otherwise, sequentially entering the experience tuples into an experience pool according to the sequence;
(5) If the current time slot t is a multiple of 10, randomly extracting N experience tuples from the experience pool, and sequentially calculating y values of the experience tuples:where r represents the current prize obtained by taking action a in the current state s, gamma E (0, 1) is the discount factor, ++ >Selecting rewards obtained by the action with the maximum Q value for future prediction, otherwise, entering the step (8);
(6) Updating the Q-estimated network parameter θ using a half gradient descent algorithm;
(7) If the current time slot t is a multiple of the parameter F, the Q-estimated network parameter theta is assigned to the target network parameter theta - Otherwise, entering the step (8);
(8) If the time slot t is greater than or equal to the set training round, exiting the training process; otherwise, entering the next time slot t=t+1, and entering the step (3).
2. The method for channel access in a multi-priority wireless terminal based on deep reinforcement learning as set forth in claim 1, wherein in the first step, the establishing a network scenario with different priority services includes:
establishing a transmission network with k priority services, wherein k is greater than 0; the network scene comprises a base station, N DRL-MAC nodes, M time division multiple access (Time DivisionMultiple Access, TDMA) nodes and X q-ALOHA nodes, wherein N is greater than 1; m+x >1, i.e. there is at least one DRL-MAC node and one other protocol node;
the base station is used for acquiring data from a wireless channel between the node and the base station and transmitting the data; the DRL-MAC node adopts a multiple access technology based on deep reinforcement learning, and if the node sends services with different priorities, a transmission result fed back by a base station is obtained and different rewards are obtained according to the services with different priorities; if the node does not send the service, the node will monitor the channel and obtain the transmission state of other nodes in a certain time slot through the channel observation result; the time division multiple access node adopts a TDMA protocol and is used for carrying out service transmission according to regularly and periodically occupying the allocated time slot; the q-ALOHA node adopts q-ALOHA for carrying out service transmission in each time slot with fixed transmission probability q according to q values under different scenes.
3. The method for channel access in a multi-priority wireless terminal based on deep reinforcement learning of claim 1, wherein in step two, each DRL-MAC node corresponds to an agent in reinforcement learning; in each time slot, the agent calculates the Q value in the current state:
wherein q(s) t A, θ) is an approximation of the deep neural network model calculation, selecting action a from the set of actions according to a greedy strategy to maximize rewards overall expectations or better accommodate dynamically changing wireless network environments.
4. The method for channel access in a multi-priority wireless terminal based on deep reinforcement learning as set forth in claim 1, wherein in step three, the defining and establishing a neural network model used by the protocol and training the network model through an experience tuple comprises:
the DQN is introduced to enable the DRL-MAC node to better learn the use condition of the channel by other nodes to make a next decision, the intelligent agent adopts a depth residual error network architecture for training, the depth residual error network is used for approximating the Q value, the current state s is input, the action strategy a is output, and then an experience tuple is formed by combining other information for training the depth residual error network.
5. A channel access system of a multi-priority service wireless terminal implementing the channel access method of a multi-priority wireless terminal based on deep reinforcement learning as set forth in any one of claims 1 to 4, wherein the channel access system of a multi-priority service wireless terminal comprises:
the network scene establishing module is used for establishing network scenes with different priority services;
the system model design module is used for designing and defining a system model of the protocol;
the space modeling module is used for carrying out state space modeling and action space modeling according to the protocol network scene;
the rewarding function design module is used for designing rewarding functions aiming at different scenes;
the neural network model building module is used for definitely and building a neural network model used by the protocol;
the network model training module is used for training the network model through the experience tuple;
the performance verification module is used for verifying the performance of the trained model through simulation comparison of multiple scenes;
designing and defining a system model of the protocol, carrying out state space modeling and action space modeling according to the network scene of the protocol, and designing reward functions aiming at different scenes, wherein the method comprises the following steps:
(1) Motion space modeling
System action set A t ＝{a 0 ,a 1 ,a 2 ......a k The method comprises the steps that k is the number of types of priority service in a network scene, and in a time slot t, a DRL-MAC node needs to make a decision a through a deep neural network for determining whether to access a data packet into a channel in the current time slot; wherein a is 0 Indicating that the channel is not accessed; a, a 1 ......a k Representing the service access channels with different priorities corresponding to k;
taking ofThe channel observation result obtained after the operation is Z t E { SUCCESS, COLLISION, IDLENESS }, obtaining channel observations for the construction of the experience tuples by listening to the channel; wherein SUCCESS indicates that the node accesses the channel and transmits the data packet successfully; COLLISION means that a plurality of nodes access channels simultaneously for transmission, resulting in COLLISION; IDLENESS indicates that the channel is idle, i.e., no node accesses the channel; the Agent determines a channel observation result according to the acknowledgement signal from the access point and the interception channel;
(2) State space modeling
State setWherein M history states to be tracked are contained, each history state is observed by action>Composition, of which there are a total of 2k+3 combinations:
(3) Reward function
The principle that the reward function always follows for network scenes of different priority services is as follows: the higher the priority of the service, the higher the rewards brought by successful transmission, and the more the penalties brought by failed transmission; the bonus function is set to sum_repeat=α+s (1- α) ×delay/T; wherein alpha and T are controllable variables, the parameter alpha is used for adjusting the influence value of the time delay on the overall rewards, and when the influence of the time delay on the overall rewards is not considered, the parameter alpha is initialized to 1; the parameter T is used for unifying the influence range of time delay on rewards and is initialized to 50; delay is the time delay of a certain service from generation to an accessed channel, namely, scheduling time delay; rewards are prize values, r, presented for different priority traffic 1 ......r k Rewards corresponding to successful transmission of different priority services, r -1 ......r -k Penalty corresponding to transmission failure for different priority service:
defining and establishing a neural network model used by the protocol, training the network model through experience tuples, and further comprising:
(1) Initializing an experience pool, setting the capacity of the experience pool and initializing parameters;
(2) Starting from time slot t=0;
(3) The Q value of the current state s is calculated in the afferent neural network; selecting an action to be executed through a greedy strategy, and recording a channel observation result z and total rewards sum_forwards obtained after the action is taken; putting the obtained state s, the rewards r obtained after taking action a and the experience tuples (s, a, r, s ') which are formed by the next state s' into an experience pool;
(4) If the current generated experience tuple is larger than the experience pool capacity, discarding the experience tuple which enters the experience pool earliest, and placing the latest experience tuple into the experience pool; otherwise, sequentially entering the experience tuples into an experience pool according to the sequence;
(5) If the current time slot t is a multiple of 10, randomly extracting N experience tuples from the experience pool, and sequentially calculating y values of the experience tuples:where r represents the current prize obtained by taking action a in the current state s, gamma E (0, 1) is the discount factor, ++ >Selecting rewards obtained by the action with the maximum Q value for future prediction, otherwise, entering the step (8);
(6) Updating the Q-estimated network parameter θ using a half gradient descent algorithm;
(7) If the current time slot t is a multiple of the parameter F, the Q-estimated network parameter theta is assigned to the target network parameter theta - Otherwise, entering the step (8);
(8) If the time slot t is greater than or equal to the set training round, exiting the training process; otherwise, entering the next time slot t=t+1, and entering the step (3).
6. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the channel access method of a deep reinforcement learning based multi-priority wireless terminal of any of claims 1-4.
7. A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the channel access method of a deep reinforcement learning based multi-priority wireless terminal of any of claims 1 to 4.
8. A wireless communication data processing terminal for implementing a channel access system for a multi-priority service wireless terminal according to claim 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110781263.0A CN113613339B (en) | 2021-07-10 | 2021-07-10 | Channel access method of multi-priority wireless terminal based on deep reinforcement learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110781263.0A CN113613339B (en) | 2021-07-10 | 2021-07-10 | Channel access method of multi-priority wireless terminal based on deep reinforcement learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113613339A CN113613339A (en) | 2021-11-05 |
| CN113613339B true CN113613339B (en) | 2023-10-17 |
Family
ID=78304401
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110781263.0A Active CN113613339B (en) | 2021-07-10 | 2021-07-10 | Channel access method of multi-priority wireless terminal based on deep reinforcement learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113613339B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114024639B (en) * | 2021-11-09 | 2024-01-05 | 成都天软信息技术有限公司 | Distributed channel allocation method in wireless multi-hop network |
| CN114375022B (en) * | 2022-01-08 | 2024-03-12 | 山东大学 | Channel preemption method based on multi-agent reinforcement learning in wireless network |
| CN114826986B (en) * | 2022-03-30 | 2023-11-03 | 西安电子科技大学 | Performance analysis method for ALOHA protocol with priority frameless structure |
| CN114938530B (en) * | 2022-06-10 | 2023-03-21 | 电子科技大学 | Wireless ad hoc network intelligent networking method based on deep reinforcement learning |
| CN115134060A (en) * | 2022-06-20 | 2022-09-30 | 京东科技控股股份有限公司 | Data transmission method and device, electronic equipment and computer readable medium |
| CN115315020A (en) * | 2022-08-08 | 2022-11-08 | 重庆邮电大学 | Intelligent CSMA/CA (Carrier sense multiple Access/Carrier aggregation) backoff method based on IEEE (institute of Electrical and electronics Engineers) 802.15.4 protocol of differentiated services |
| CN115767785B (en) * | 2022-10-22 | 2024-02-27 | 西安电子科技大学 | MAC protocol switching method based on deep reinforcement learning in self-organizing network |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110049018A (en) * | 2019-03-25 | 2019-07-23 | 上海交通大学 | SPMA protocol parameter optimization method, system and medium based on enhancing study |
| CN110809306A (en) * | 2019-11-04 | 2020-02-18 | 电子科技大学 | Terminal access selection method based on deep reinforcement learning |
| CN111628855A (en) * | 2020-05-09 | 2020-09-04 | 中国科学院沈阳自动化研究所 | Industrial 5G dynamic multi-priority multi-access method based on deep reinforcement learning |
| CN111711666A (en) * | 2020-05-27 | 2020-09-25 | 梁宏斌 | Internet of vehicles cloud computing resource optimization method based on reinforcement learning |
| CN111786713A (en) * | 2020-06-04 | 2020-10-16 | 大连理工大学 | Unmanned aerial vehicle network hovering position optimization method based on multi-agent deep reinforcement learning |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10375585B2 (en) * | 2017-07-06 | 2019-08-06 | Futurwei Technologies, Inc. | System and method for deep learning and wireless network optimization using deep learning |
-
2021
- 2021-07-10 CN CN202110781263.0A patent/CN113613339B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110049018A (en) * | 2019-03-25 | 2019-07-23 | 上海交通大学 | SPMA protocol parameter optimization method, system and medium based on enhancing study |
| CN110809306A (en) * | 2019-11-04 | 2020-02-18 | 电子科技大学 | Terminal access selection method based on deep reinforcement learning |
| CN111628855A (en) * | 2020-05-09 | 2020-09-04 | 中国科学院沈阳自动化研究所 | Industrial 5G dynamic multi-priority multi-access method based on deep reinforcement learning |
| CN111711666A (en) * | 2020-05-27 | 2020-09-25 | 梁宏斌 | Internet of vehicles cloud computing resource optimization method based on reinforcement learning |
| CN111786713A (en) * | 2020-06-04 | 2020-10-16 | 大连理工大学 | Unmanned aerial vehicle network hovering position optimization method based on multi-agent deep reinforcement learning |
Non-Patent Citations (2)
| Title |
|---|
| He Jiang ; Haibo He.Q-Learning for Non-Cooperative Channel Access Game of Cognitive Radio Networks.IEEE.2018,全文. * |
| 动态频谱接入的Q学习优化算法;黄影;严定宇;李男;;西安电子科技大学学报(第06期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113613339A (en) | 2021-11-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113613339B (en) | Channel access method of multi-priority wireless terminal based on deep reinforcement learning | |
| Kong et al. | Performance analysis of IEEE 802.11 e contention-based channel access | |
| EP3637708B1 (en) | Network congestion processing method, device, and system | |
| Ali et al. | Performance evaluation of heterogeneous IoT nodes with differentiated QoS in IEEE 802.11 ah RAW mechanism | |
| US7843943B2 (en) | Differentiation for bandwidth request contention | |
| CN111294775B (en) | Resource allocation method based on H2H dynamic characteristics in large-scale MTC and H2H coexistence scene | |
| Inan et al. | Analysis of the 802.11 e enhanced distributed channel access function | |
| CN112543508A (en) | Wireless resource allocation method and network architecture for 5G network slice | |
| CN111601398B (en) | Ad hoc network medium access control method based on reinforcement learning | |
| CN107835517B (en) | Long-distance CSMA/CA method with QoS guarantee | |
| Hu et al. | Performance and reliability analysis of prioritized safety messages broadcasting in DSRC with hidden terminals | |
| Mancuso et al. | Slicing cell resources: The case of HTC and MTC coexistence | |
| Shoaei et al. | Reconfigurable and traffic-aware MAC design for virtualized wireless networks via reinforcement learning | |
| AlQahtani | Performance analysis of cognitive‐based radio resource allocation in multi‐channel LTE‐A networks with M2M/H2H coexistence | |
| CN117118855A (en) | Data link SPMA access method based on machine learning priority prediction | |
| CN114845338A (en) | Random back-off method for user access | |
| Bensaou et al. | A measurement-assisted, model-based admission control algorithm for IEEE 802.11 e | |
| Ahmed et al. | A QoS-aware scheduling with node grouping for IEEE 802.11 ah | |
| Hui et al. | Delay analysis of reservation based random access: A tandem queue model | |
| Mehri et al. | RACH Traffic Prediction in Massive Machine Type Communications | |
| Raeis et al. | Distributed fair scheduling for information exchange in multi-agent systems | |
| Li et al. | Optimal parameter selection for discrete-time throughput-optimal MAC protocols | |
| Conti et al. | Design and performance evaluation of a MAC protocol for wireless local area networks | |
| Peng et al. | An enhanced reliable access scheme for massive IoT applications in ubiquitous IoT systems | |
| CN114980353A (en) | Ordered competition large-scale access learning method for machine type communication system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |