CN110519020B

CN110519020B - Intelligent cross-layer data transmission method and system for unmanned system network

Info

Publication number: CN110519020B
Application number: CN201910744399.7A
Authority: CN
Inventors: 王琪; 何晨涛; 刘建敏; 徐勇军; 徐亦达
Original assignee: Institute of Computing Technology of CAS
Current assignee: Institute of Computing Technology of CAS
Priority date: 2019-08-13
Filing date: 2019-08-13
Publication date: 2020-09-11
Anticipated expiration: 2039-08-13
Also published as: CN110519020A

Abstract

The invention provides an intelligent cross-layer data transmission method and system for an unmanned system network, which comprises a search strategy that an upper confidence interval is adopted as a node to select a forwarding node in a routing layer; the problem of inaccurate behavior selection evaluation caused by over-high Q value estimation is solved by adopting a double-Q learning technology; the calculation of the return value comprehensively considers MAC delay, link error probability and position error, so that the return value can reflect the benefit of action better; and according to the node speed and the MAC delay, different learning rates and discount factors are given to each link, so that the self-adaptive adjustment of the learning rates and the discount factors is realized. And finally, combining the MAC layer and the routing layer, realizing cross-layer optimization by using MAC delay parameter sharing, and providing an intelligent cross-layer data transmission method and system of the unmanned system network.

Description

Intelligent cross-layer data transmission method and system for unmanned system network

Technical Field

The invention relates to the technical field of information, in particular to an intelligent cross-layer data transmission protocol of an unmanned system network.

Background

An Unmanned System (Unmanned System) is composed of a plurality of necessary data processing units, sensors, automatic control units and a communication System, can automatically complete machines or devices of specific tasks without human intervention, and the Unmanned machines or devices can be Unmanned planes, Unmanned vehicles, ground robots, underwater robots, water surface robots, satellites and the like.

Networks established by unmanned systems in an ad hoc fashion or based on a network infrastructure are collectively referred to as unmanned system networks. The invention mainly develops the unmanned system self-organizing network (hereinafter referred to as the unmanned system network) by focusing on the unmanned system self-organizing network. In the unmanned system, nodes (unmanned machines or devices) move at a high speed, the link state is unstable, and the unmanned system has the characteristics of three-dimensional large-range operation space and the like, so that the data transmission of the unmanned system is difficult to a certain extent.

On a Medium Access Control (MAC) layer, most of the MAC layers are based on reservation competition, that is, an IEEE 802.11 DCF protocol is improved, and an adaptive MAC protocol (amav) for an unmanned aerial vehicle network transmits different types of data packets from different antennas on the basis of DCF, so that although throughput is improved and end-to-end delay is reduced, link uncertainty in the unmanned system network still has a great influence on the network.

At the network layer, conventional network-based routing protocols do not work well in an unmanned system network because of the need to maintain fixed routing tables. In the routing protocol based on the geographic information, the positioning auxiliary route needs to establish a routing path before sending a data packet, and the data packet sending has hysteresis. Greedy peripheral stateless routing simplifies the topology structure, and cannot be used when the height difference of nodes is large, and the right-hand rule adopted by routing forwarding may cause the hop count of a data packet to increase. Because the nodes in the unmanned system network have richer perception means, the unmanned system network can carry out all-dimensional perception on the situation of multiple dimensions such as the external working environment, the working state of the unmanned system network and the like, and the unmanned system network can select the optimal transmission path capable of achieving the target by self-learning by utilizing a Q-learning (belonging to one of reinforcement learning) method. In recent years, research works have been carried out to realize data transmission and transmission of an unmanned system network by using Q-learning, and although results show that the network performance is greatly improved compared with the traditional routing protocol, the method still has several problems, a fixed learning rate and a discount factor cannot be adaptively adjusted according to the dynamic change of the network, the problem of overhigh estimation value of the Q-learning and a search strategy of an algorithm in initialization are not well solved.

In addition, for the network of the unmanned system, the single-layer optimization of the protocol is not enough to solve the communication problem of the network of the unmanned system, and the idea of cross-layer optimization is utilized to realize information interaction between layers, such as a network layer and an MAC layer, a transmission layer and an MAC layer, and the like, so that the method is an effective method for improving the network performance of the network of the unmanned system. The MAC layer and the network layer have information sharing relation: the MAC layer will pass the MAC delay of each packet to the routing algorithm of the network layer for use.

Disclosure of Invention

In order to solve the technical problems, the invention firstly proposes a network self-synchronization MAC protocol (CSMA/MAS) under multiple channels at an MAC layer, and then adopts an upper confidence interval as a search strategy for selecting a forwarding node by a node at a routing layer; the existing Q-learning routing (QGeo) is improved by adopting a Double-Q learning (Double-Q-learning) technology, so that the problem of inaccurate behavior selection evaluation caused by over-high Q value estimation is solved; the calculation of the return value comprehensively considers MAC delay, link error probability and position error, so that the return value can reflect the benefit of action better; and according to the node speed and the MAC delay, different learning rates and discount factors are given to each link, so that the self-adaptive adjustment of the learning rates and the discount factors is realized. And finally, combining the MAC layer and the routing layer, realizing cross-layer optimization by using MAC delay parameter sharing, and providing an intelligent cross-layer data transmission method and system of the unmanned system network.

Aiming at the defects of the prior art, the invention provides an intelligent cross-layer data transmission method of an unmanned system network, which comprises the steps of constructing the unmanned system network with a plurality of unmanned devices as nodes, wherein the nodes periodically send HELLO packets, the nodes receiving the HELLO packets establish a neighbor table according to the HELLO packets, the neighbor table records the position, link information, moving speed and UCB value of each neighbor node, and the transmission of the data packets from a source node to a destination node in the unmanned system network comprises a channel access step and a data routing step;

the channel access step realizes the access of node channels through a network self-synchronization MAC protocol under multiple channels, wherein the multiple channels consist of 1 control channel and 15 data channels, the control channel is used for transmitting RTS and CTS packets, and the data channels are used for transmitting data packets and ACK packets; when a node needs to send a data packet, on a control channel, a data channel used for transmitting the data packet is determined by utilizing RTS and CTS packet negotiation; when a node selects a data channel, firstly selecting the latest data channel used for completing data transmission successfully recently, if the latest data channel is in an idle state when the current node is about to send a data packet, selecting the latest data channel as the data channel used for transmitting the data packet, otherwise selecting the data channel with the earliest idle time from the data channels as the data channel used for transmitting the data packet, and determining the data channel used for transmitting the data packet by both communication nodes through RTS and CTS so as to transmit the data packet and an ACK packet;

the data routing step is to judge whether a node in the neighbor node table is not selected as a forwarding node by the current node, if so, a probe data packet is sent to obtain a UCB value, otherwise, the node with the maximum UCB value in the neighbor node table is selected and a data packet is sent; and meanwhile, according to the moving speed and the MAC delay of the neighbor node, the learning rate and the discount factor are adjusted so as to update the UCB value of the corresponding forwarding node.

The intelligent cross-layer data transmission method of the unmanned system network, wherein the channel access step further comprises: before starting a new data transmission process, the node accesses the channel through the continuous back-off time so as to achieve collision-free channel access.

The intelligent cross-layer data transmission method of the unmanned system network, wherein the data routing step calculates the UCB value through the following formula:

wherein i is the node number, m is the number of neighbor nodes in the neighbor node table of the node i, n_jIs the number of times that the neighbor node is selected as the forwarding node by the current node i, n is the sum of the number of times that all neighbor nodes are selected as the forwarding nodes in the neighbor node table of the node i, and Q_iIs the action effective value of the node i.

The intelligent cross-layer data transmission method of the unmanned system network comprises the following steps:

where i is the number of retransmissions, C is 3N, N is the number of nodes in a single-hop range of the node, and context Window is the Contention Window, rand (context Window)_i) Means that the generation interval is [0, context Window_i-1]The random number in (c).

The invention also provides an intelligent cross-layer data transmission system of the unmanned system network, which comprises the steps of constructing the unmanned system network with a plurality of unmanned devices as nodes, wherein the nodes periodically send HELLO packets, the nodes receiving the HELLO packets establish a neighbor table according to the HELLO packets, the neighbor table records the position, link information, moving speed and UCB value of each neighbor node, and the transmission of the data packets from the source node to the destination node in the unmanned system network comprises a channel access module and a data routing module;

the channel access module realizes node channel access through a network self-synchronization MAC protocol under multiple channels, wherein the multiple channels comprise 1 control channel and 15 data channels, the control channels are used for transmitting RTS and CTS packets, and the data channels are used for transmitting data packets and ACK packets; when a node needs to send a data packet, on a control channel, a data channel used for transmitting the data packet is determined by utilizing RTS and CTS packet negotiation; when a node selects a data channel, firstly selecting the latest data channel used for completing data transmission successfully recently, if the latest data channel is in an idle state when the current node is about to send a data packet, selecting the latest data channel as the data channel used for transmitting the data packet, otherwise selecting the data channel with the earliest idle time from the data channels as the data channel used for transmitting the data packet, and determining the data channel used for transmitting the data packet by both communication nodes through RTS and CTS so as to transmit the data packet and an ACK packet;

the data routing module judges whether a node in the neighbor node table is not selected as a forwarding node by the current node, if so, a probe data packet is sent to obtain a UCB value, otherwise, the node with the maximum UCB value in the neighbor node table is selected and a data packet is sent; and meanwhile, according to the moving speed and the MAC delay of the neighbor node, the learning rate and the discount factor are adjusted so as to update the UCB value of the corresponding forwarding node.

The intelligent cross-layer data transmission system of the unmanned system network, wherein the channel access module further comprises: before starting a new data transmission process, the node accesses the channel through the continuous back-off time so as to achieve collision-free channel access.

The process of the node for realizing the channel access without rush is as follows: when a plurality of nodes in the network need to access the channel, when one node successfully accesses the channel to transmit data, according to a calculation formula of the continuous back-off time, no competition for channel use is carried out in the next C time slots, only the node which fails to compete for the channel use right, according to the method, the number of nodes competing for the channel use right is continuously reduced, and after all the nodes obtain the channel use right, because the nodes transmit data packets, the persistent back-off counters of other nodes need to be suspended and, according to the data channel selection policy, the node selects the data channel that was successfully transmitted before as the data channel to be used for transmitting the data packet, and therefore, at the later time, the nodes in the network will sequentially acquire the channel use right according to the sequence of the data packets sent by the previous nodes, so as to reach the synchronous state, and the period of the nodes acquiring the channel use right is represented by the following formula:

T_period＝T_c+(N-1)×T_{transmit-control}+T_DIFS

wherein, T_CIs the time occupied by C time slots, T_{transmit-control}Is the time occupied on the control channel during data transmission, which is equal to the time occupied by RTS + SIFS + CTS + SIFS, T_DIFSIs the time taken for DIFS and N is the number of nodes in a single hop range.

The intelligent cross-layer data transmission system of the unmanned system network, wherein the data routing module calculates the UCB value according to the following formula:

The intelligent cross-layer data transmission system of the unmanned system network comprises the following steps:

According to the scheme, the invention has the advantages that:

the reserved competition type of RTS/CTS in the prior art only works under a single channel, and the sending process of the data packet can be only carried out on one channel; the random backspacing value calculating method provided by the invention enables a network to realize automatic synchronization under multiple channels, which cannot be realized in the prior art. The invention changes the network from the traditional conflict transmission state to the new conflict-free transmission state, avoids the retransmission caused by conflict collision, and reduces the MAC time delay.

Drawings

FIG. 1 is a flow diagram of an intelligent cross-layer data transfer protocol for an unmanned system network, according to one embodiment of the invention;

fig. 2 to 11 are graphs showing results of simulation experiments of the example of the present invention.

Detailed Description

The invention aims to overcome the defects in the prior art, and provides a network self-synchronization MAC protocol (CSMA/MAS) under multiple channels at an MAC layer; then, adopting an upper confidence interval balance node as a searching strategy when the node selects a forwarding node in a routing layer; the existing Q-learning routing (QGeo) is improved by adopting a Double-Q learning (Double-Q-learning) technology, and the problem of inaccurate behavior selection evaluation caused by over-high Q value estimation is solved; the calculation of the return value comprehensively considers MAC delay, link error probability and position error, so that the return value can reflect the benefit of action better; and according to the node speed and the MAC delay, different learning rates and discount factors are given to each link, so that the self-adaptive adjustment of the learning rates and the discount factors is realized. And finally, combining the MAC layer and the routing layer, realizing cross-layer optimization by using MAC delay parameter sharing, and providing an intelligent cross-layer data transmission method and system of the unmanned system network. 1. Network self-synchronization MAC protocol (CSMA/MAS) under multiple channels

1) The invention divides the wireless channel into 16 channels, including 1 control channel and 15 data channels. The node has dual transceivers, wherein 1 transceiver is a control channel transceiver for transceiving data packets on a control channel, and the other 1 transceiver is a data channel transceiver for transceiving data packets on a data channel. The time of a node in the network runs in time slots.

2) Both communication nodes use a reserved competition type mode similar to RTS/CTS to transmit data packets, and the calculation formula of the random backoff count value is as follows:

wherein i is the number of retransmissions, C is 3N, and N is the number of nodes in a single hop range of the node. rand (contentionWindow)_i) Means that the generation interval is [0, context Window_i-1]The random number in (2) and the content Window are competition windows, and the calculation formula is as follows:

where m is the maximum number of increases of the contention window, when the number of retransmissions exceeds this value, the contention window is unchanged, R represents the maximum number of retransmissions, when the number of retransmissions of the node exceeds this value,the packet is discarded. Wherein W is 1, the minimum Contention Window, Contention Window_iIs the contention window value at the i-th retransmission.

The source node sends an RTS (Request To Send Request To Send) packet, the time occupied by the control channel, the earliest available time of the data channel and the data channel selected according To the data channel selection strategy are recorded in the RTS packet. The calculation formula of the time that the control channel recorded in the RTS will occupy and the earliest available time of the data channel is as follows:

Control_rts＝2×SIFS+CTS

Data_rts＝3×SIFS+CTS+DATA+ACK+DIFS

the SIFS and DIFS respectively represent the time occupied by the short interframe space and the distributed interframe space. CTS, DATA, ACK represent the time taken to transmit CTS, DATA, ACK packets, respectively.

After receiving the RTS packet, the destination node sends a CTS (Clear To Send allowed) packet if the data channel in the RTS is available To the destination node, and records the time that the control channel will occupy, the earliest available time of the data channel, and the selected data channel in the CTS packet. The calculation formula of the time that the control channel recorded in the CTS will occupy and the earliest available time of the data channel is as follows:

Control_cts＝SIFS

Data_cts＝2×SIFS+DATA+ACK+DIFS

after the RTS/CTS DATA packet is sent, the DATA channel transceivers of the source node and the destination node are switched to the previously negotiated DATA channel, the control channel transceiver remains unchanged, when the source node receives the CTS DATA packet and wants to send a DATA packet, the source node sends the DATA packet on the previously negotiated DATA channel, the destination node sends an ACK packet on the previously negotiated DATA channel after receiving the DATA packet, and the source node indicates that the transmission process is completed when receiving the ACK packet. In the whole transmission process, when a node receives an RTS or CTS packet and is not a destination node, the information in the RTS or CTS packet, that is, the data channel and the time period that the data channel will be occupied by other nodes are recorded, so that the node can know the use condition of each data channel.

3) And (3) a DATA channel selection strategy, wherein both communication nodes select a DATA channel used when the DATA packet is transmitted by utilizing RTS/CTS negotiation. For any node, after finishing data transmission, the node records the time of the channel finishing data transmission, when the node selects data channel, it selects the channel used for finishing data transmission, if the data channel is available, it uses the data channel, if not, it selects the data channel with earliest available time from the available data channels. The conditions under which the data channel is available are:

T_data≥T_spare

wherein, T_dataIndicates the time T at which the node communication will transmit the data packet_spareRepresenting the earliest point in time of availability of the data channel. The same is true if the destination node judges whether the data channel in the RTS is available.

The node accesses the channel through the continuous back-off time before starting a new data transmission process so as to achieve collision-free channel access.

T_period＝T_c+(N-1)×T_{transmit-control}+T_DIFS

2. Improved Q-learning routing

1) The method comprises the steps that a node periodically sends a HELLO packet to a neighbor node in a broadcasting mode, the position, the speed, the maximum Q value and link information of a current node are recorded in the HELLO packet, the node receiving the HELLO packet establishes a neighbor node table by using the information of the HELLO packet, the node endows a timer for each record in the neighbor node table, and when the timer is overtime and the channel is not updated, the node recorded by the information is considered not to be the neighbor node of the current node, and the information is deleted. The Q value is used to evaluate whether a certain action is taken in a specific state. In an unmanned aerial vehicle network, different neighbor nodes are selected as a comprehensive evaluation value of the forwarding node, and the higher the evaluation value is, the better the node selects the neighbor node as the forwarding node.

2) Each neighbor node in the neighbor table of the node records the times of selecting the current node as the next node, namely the forwarding node, and the current node also records the total times of selecting all the neighbor nodes as the next hop in the current neighbor node table. When a node has a data packet to be sent or forwarded, if a neighbor node exists in a neighbor table and has not been selected as a forwarding node, the node is selected as the forwarding node, but the node does not send the data packet but sends a probe data packet to calculate the value of UCB. If all the neighbor nodes in the current node have been selected as forwarding nodes, calculating a value of an upper confidence interval (UCB) of each neighbor node, and selecting the node with the maximum UCB value as the forwarding node. The calculation formula is as follows:

wherein i is the number of the node, m is the number of neighbor nodes in the neighbor node table of the node i, nj is the number of times that the neighbor node is selected as a forwarding node by the current node i, n is the sum of the number of times that all neighbor nodes are selected as forwarding nodes in the neighbor node table of the node i, and the calculation formula is as follows:

when the node selects the next hop, the node with the largest UCB value is selected as the forwarding node.

3) After the node sends the data packet, the node calculates a return value, and the calculation formula is as follows:

when the next hop of the node is the destination node, it is the maximum value, when the next hop of the node is empty (i.e. routing hole problem), it is the minimum value, otherwise, it is f_pts/R_pts。

f_ptsCalled the packet travel speed, is a value which comprehensively considers the link state, the position error and the MAC delay, and the calculation formula is as follows:

wherein, diff_i，jThe distance between the node i and the node j, and the T data packet travel time are calculated according to the following formula:

wherein, MAC delay is the MAC delay of the data packet sent this time, E_linkIs link state information, is 1 value between 0 and 1, E_locIs a position error, also 1 value between 0 and 1. Since the node acquires other nodes through the HELLO packet, the information in the HELLO packet has a certain hysteresis for the receiving node, assuming that at t1At this time, the node 2 will locate its own position l₂And velocity v₁Recorded in a HELLO packet, sent out in a broadcast manner, and node 1 at t₂The HELLO packet is received at the moment, and the information of the node 2 is written into a neighbor information table at t₃At the moment, the node 1 needs to send a data packet to the neighboring node 2, at the moment, the node 1 needs to estimate the position of the node 2, and the HELLO has a short sending period, so that it can be considered that the node is at t₁To t₂At that moment, the motion of the node 2 can be approximately regarded as a uniform motion. Then at this time, the node 2 may be located at a position l₂As a center of circle, then v₁×(t₃-t₁) As a sphere with a center, if the node 2 is located within the propagation range of the node 1, after the node 1 sends a data packet, the node 2 can receive the data packet only by considering the propagation distance, otherwise, the data packet cannot be received. Therefore, it is required that the computing node 2 be at t₃Probability that the time instant is not within the propagation range of node 1. The problem can be converted into a relation problem of two spheres, the first sphere being l₂As the center of circle, with v₁×(t₃-t₁) A sphere as a circle center. The second sphere is at t with node 1₃Position of time l₁A sphere taking the propagation range as the radius and taking the propagation range as the center of the circle. With E_locIndicating a position error, V₁Representing the volume of the first sphere, V₂Representing the volume of the second sphere. Then E_locThe calculation formula of (a) is as follows:

R_ptsis a normalization factor which is f_ptsIn consideration of the values in the ideal state, the calculation formula is as follows:

wherein range is the signal propagation range of the node. Combining the above equations, we can derive the third calculation formula of reward as follows:

4) the node stores two Q value tables, the Q value table is composed of actions and states, the actions refer to forwarding nodes selectable by the node, and the states refer to nodes where the data packets are located. The union of the actions and states recorded by the nodes is the set of actions and states of the entire network. Namely:

a′₁∪a′₂∪a′₃∪…∪a′_n＝A

s′₁∪s′₂∪s′₃∪…∪s′_n＝5

where n is the number of nodes in the network, a_i' refers to the action that a node can take, s_i' is the state that the node needs to record. A is all the actions that can be performed in the network and S is all the presence states in the network.

When the node updates the Q value table, 1 of the tables is selected for updating, and the probability of each table being selected is 50%. When the 1 st Q-value table is selected, the formula is updated as follows:

Q₁(s，a)＝Q₁(s，a)+α×(reward+γ×maxQ₂(s′，a^*)-Q₁)

when the 2 nd Q-value table is selected, the formula is updated as follows:

Q₂(s，a)＝Q₂(s，a)+α×(reward+γ×maxQ₁(s′，a^*)-Q₂)

in the above two equations, Q (s, a) represents the expectation of the benefit that would be obtained by taking action a at a certain time s state, s' represents the next state after taking action a at state s, a^*Represents one of the set of actions that can be taken in state s ', maxQ (s', a)^*) Representing the maximum utility value that can be achieved by taking action at s', α is the learning rate, which is 1 number between 0 and 1, and γ is the discount factor, which is 1 number between 0 and 1.

5) The node endows each record in the neighbor table with different learning rate and discount factor, and the calculation formula of the learning rate is as follows:

where j is the number of the neighbor node in the neighbor node table of node i (the same applies below),

is the variance of the velocity of the neighboring node j,

is the maximum value of the velocity variance, LEARNING, in the neighbor node table of node i_MAXAnd LEARNINGs_MINThe maximum and minimum values of the learning rate, respectively.

The formula for the discount factor is as follows:

wherein the content of the first and second substances,

is the MAC delay variance of the neighbor node j,

is the maximum value of MAC delay variance in the neighbor node table of node i, DISCOUNT_MAXAnd DESCOUNT_MINRespectively, a maximum value and a minimum value of the discount factor.

3. Cross-layer data transmission

When a data packet enters a sending queue of the MAC layer from the routing layer, the data packet records the time of entering the sending queue. Due to channel contention, the transmission of the data packet may succeed or fail. When the node receives the ACK packet, the data transmission is successful, and at this time, the MAC delay calculation formula is as follows:

MAC delay_success＝T_re-ack-T_in-queue

wherein, T_re-ackIndicating that the node received the ACK packetTime of (T)_in-queueIs the time at which the packet enters the transmit queue.

When the retransmission times of the data packet exceed the maximum retransmission times specified by the MAC layer, the data transmission is indicated to fail, and the MAC delay calculation formula is as follows:

MAC delay_fail＝T_max-retry-T_in-queue

wherein, T_max-retryIndicating the time when the retransmission times of the MAC layer data packet exceed the maximum retransmission times.

The MAC layer calculates its MAC delay for each packet in the transmit queue and passes the delay up to the routing layer. In the routing layer, there are two places where MAC delay needs to be used, first in the calculation formula of the return value:

secondly, in the calculation formula of the learning rate:

the symbols of the above two formulae are explained in the above.

In order to make the aforementioned features and effects of the present invention more comprehensible, embodiments accompanied with figures are described in detail below.

In order to solve the problems in the prior art, the invention provides an intelligent cross-layer data transmission protocol of an unmanned system network, and provides a network self-synchronizing MAC protocol under multiple channels at an MAC layer; then, an upper confidence interval is adopted as a searching strategy when the node selects a forwarding node in a routing layer; the existing Q-learning routing (QGeo) is improved by adopting a Double-Q learning (Double-Q-learning) technology, and the problem of inaccurate behavior selection evaluation caused by over-high Q value estimation is solved; the calculation of the return value comprehensively considers MAC delay, link error probability and position error, so that the return value can reflect the benefit of action better; and according to the node speed and the MAC delay, different learning rates and discount factors are given to each link, so that the self-adaptive adjustment of the learning rates and the discount factors is realized. And finally, combining the MAC layer and the routing layer, realizing cross-layer optimization by using MAC delay parameter sharing, and providing an intelligent cross-layer data transmission protocol of the unmanned system network.

According to the MAC protocol, wireless channels are divided into 16 channels, the 16 channels are divided into 1 control channel and 15 data channels, the sending process of a data packet is sent in channels, two communication nodes delay a plurality of time slots after the first transmission of the data packet and still carry out a rollback process when the nodes do not send the data packet, and each node can automatically realize a collision-free data transmission process under the conditions.

The routing protocol balances the relation between 'exploration' and 'utilization' of the nodes when the nodes are selected to be forwarded by utilizing a search strategy of a confidence upper limit interval optimization algorithm; solving the problem of over-estimation of the Q value by using a Double-Q-learning formula; in the return value, MAC delay, link state and position error are comprehensively considered; and endowing different learning rates and discount factors for each link by using the speed change of the neighbor nodes and the MAC delay change of the data packet, and realizing the self-adaptive routing of the learning rates and the discount factors.

Before describing the intelligent cross-layer data transmission protocol of the unmanned system network provided by the invention in detail, a simple explanation is firstly made on some concepts and terms related to the invention.

"network synchronization": nodes within the network in turn have the right to send data packets for a period of time without collision.

"continuous backoff procedure": after a rollback process is finished, if no data is sent, the node restarts a new rollback process according to a value formula of a random rollback count value instead of being in an idle state.

"MAC delay": the time from the data packet entering the MAC layer to the completion of the whole transmission process is composed of queuing delay and channel access delay.

"queuing delay": refers to the time taken from the entry of a data packet into the transmit queue until the data packet reaches the head of the transmit queue.

"channel access delay": the time from the time when the data packet arrives at the head of the queue to the time when the data packet is successfully transmitted and the destination node ACK packet is received is referred to as the time spent on retransmitting the data packet for the maximum number of times under the condition of failed transmission.

"end-to-end delay": the time that elapses from the generation of the data packet by the source node until the reception by the destination node.

Based on the above analysis, the intelligent cross-layer data transmission protocol of the unmanned system network according to the invention will be described in detail by a specific embodiment, and with reference to fig. 1, the method includes:

step 101: and after the node starts to operate, assigning the backspacing counter according to the following formula.

Wherein i is the retransmission times, C is 3N, N is the number of nodes in the node single-hop range, rand (contentionWindow)_i) Finger generation is in the interval [0, context Window_i-1]The random number in the random number, the content Window, is a Contention Window, and the calculation formula is as follows:

wherein m is the maximum increasing number of the contention window, when the retransmission number exceeds the value, the contention window is not changed, R represents the maximum retransmission number, W is the minimum contention window, and 1 is taken.

Step 102: the node checks whether the control channel is busy every other time slot, and the detection method has two methods, one is physical carrier monitoring, and the other is virtual carrier monitoring realized through the recorded nav value of the control channel. If the monitored result is that the wireless channel is busy, the monitoring is carried out after another time interval. If the monitored channel is idle, subtracting 1 from the backoff count value, if the backoff count value is changed into 0, checking whether the transmission queue is empty, namely whether data is to be transmitted, and if not, continuing the backoff process according to the formula of the previous contention window and the assignment formula of the backoff counter. And if the sending queue is not empty, namely a data packet is sent, sending an RTS packet, and recording the time occupied by the control channel, the data channel selected for communication and the earliest available time of the data channel in the RTS packet. The time that the control channel will occupy is calculated as follows:

Control_rts＝2×SIFS+CTS

where SIFS denotes a time occupied by a short interframe space, and CTS denotes a time taken to transmit a CTS packet.

The calculation formula of the time that the data channel will occupy is as follows:

Data_rts＝3×SIFS+CTS+DATA+ACK+DIFS

where DIFS denotes the time taken for the distributed inter-frame space and ACK, DATA denotes the time taken to transmit ACK and DATA.

The node selects the data channel used in communication by using the following data channel selection strategy:

after the node finishes data transmission once, the node records the time of finishing data transmission of the channel, when the node needs to select a data channel, the channel used for finishing data transmission successfully recently is selected, if the data channel is available, the data channel is used, and if the data channel is unavailable, the data channel with the earliest available time is selected from the available data channels. The conditions under which the data channel is available are:

T_data≥T_spare

wherein, T_dataIndicates the time T at which the node communication will transmit the data packet_spareRepresenting the earliest point in time of availability of the recorded data channel.

Step 103: after receiving the RTS packet, the node waits for an SIFS and then sends a CTS packet if the node is a destination node and the data channel recorded in the RTS packet is available for the destination node, and records the control channel and the occupation time of the corresponding data into the corresponding nav value if the node is not the destination node. At this time, the time that the control channel will occupy is:

Control_cts＝SIFS

the earliest time of availability of the data channel is:

Data_cts＝2×SIFS+DATA PACKET+ACK+DIFS

step 104: after receiving the CTS packet, if the node is the destination node, the node waits for a SIFS and then sends a DATA packet, and if the node is not the destination node, the node records the occupied time of the control channel and the corresponding DATA channel into the corresponding nav value.

Step 105: and after receiving the DATA packet, if the node is the destination node, the node waits for an SIFS and then sends an ACK, otherwise, the node discards the DATA packet.

Step 106: due to channel contention, the transmission of the data packet may succeed or fail. When the node receives the ACK packet, the data transmission is successful, and at this time, the MAC delay calculation formula is as follows:

MAC delay_success＝T_re-ack-T_in-queue

wherein, T_re-ackIndicating the time, T, at which the node received the ACK packet_in-queueIs the time at which the packet enters the transmit queue.

MAC delay_fail＝T_max-retry-T_in-queue

The MAC layer calculates the MAC delay of each data packet, calculates according to the two formulas and transmits the MAC delay upwards to the routing layer.

Step 107: and (3) neighbor discovery, wherein the node periodically sends HELLO packets, the position, the speed, the maximum Q value and link information of the node are recorded in the HELLO packets, the node establishes a neighbor node table according to the received HELLO packets, a timer is assigned to each record, and when the timer exceeds 300ms and the node does not receive the HELLO packet sent by the neighbor node corresponding to the new record, the neighbor node is considered not to be in the propagation range of the current node, and the record is deleted.

Step 108: and (4) making a routing decision, namely selecting a next hop node from the neighbor table when the node needs to send a data packet, and if the neighbor node in the neighbor table of the node is not selected as the neighbor node, selecting the next hop node as the next hop node, but not sending the data packet, but sending a probe data packet. If all the neighbor nodes in the neighbor table of the node are selected as the neighbor nodes, the UCB value of each neighbor node is calculated by using the following formula, and the node with the highest UCB value is selected as a forwarding node.

Where m is the number of neighbor nodes of node i, n_jThe number of times that the neighbor node j is selected as a forwarding node by the node i is referred to, n is the sum of the number of times that all neighbor nodes of the node i are selected as forwarding nodes, namely:

step 110: updating the Q value, calculating a return value, and calculating the return value after the node sends a data packet, wherein the return value calculation formula is as follows:

wherein, MAX takes on 1, MIN takes on 0. The first case is to return a maximum value when the selected forwarding node is the destination node. The second case is when no forwarding node can be found, the minimum is returned. The third case is when the selected forwarding node is neither null nor destination. F_ptsCalled the travel speed of the data packet, the formula is as follows:

wherein, diff_i，jIs the distance between nodes i and j, and T is called the travel time of the data packet, and the calculation formula is as follows:

where MAC delay is MAC delay, provided by the MAC layer, E_linkIs the link error probability and is 1 value between 0 and 1. E_locIs a position error, also 1 value between 0 and 1. Since the node acquires other nodes through the HELLO packet, the information in the HELLO packet has a certain hysteresis for the receiving node, assuming t₁At that moment, the node 2 will locate its own position l₂And velocity v₁Recorded in a HELLO packet, sent out in a broadcast manner, and node 1 at t₂The HELLO packet is received at the moment, and the information of the node 2 is written into a neighbor information table at t₃At the moment, the node 1 needs to send a data packet to the neighboring node 2, at the moment, the node 1 needs to estimate the position of the node 2, and the HELLO has a short sending period, so that it can be considered that the node is at t₁To t₂At that moment, the motion of the node 2 can be approximately regarded as a uniform motion. Then at this time, the node 2 may be located at a position l₂As a center of circle, then v₁×(t₃-t₁) As a sphere with a center, if the node 2 is located within the propagation range of the node 1, after the node 1 sends a data packet, the node 2 can receive the data packet only by considering the propagation distance, otherwise, the data packet cannot be received. Therefore, it is required that the computing node 2 be at t₃Probability that the time instant is not within the propagation range of node 1. The problem can be converted into a relation problem of two spheres, the first sphere being l₂As the center of circle, with v₁×(t₃-t₁) A sphere as a circle center. The second sphere is at t with node 1₃Position of time l₁A sphere taking the propagation range as the radius and taking the propagation range as the center of the circle. With E_locIndicating a position error, V₁Representing the volume of the first sphere, V₂Representing the volume of the second sphere. Then E_locThe calculation formula of (a) is as follows:

R_ptsis a normalization factor, is f in the ideal case_ptsValue of (A), R_ptsThe calculation formula is as follows:

step 110: updating the Q value, calculating the learning rate, dynamically adjusting the learning rate of each link by the node according to the speed condition of the neighbor node, wherein the calculation formula of the learning rate is as follows:

is the variance of the velocity of the neighboring node j,

Step 111: updating the Q value, calculating a discount factor, wherein the node dynamically adjusts the discount factor of each link according to the delay change condition of a transmitted data packet and the MAC delay of the data packet provided by an MAC layer, and the calculation formula of the discount factor is as follows:

wherein the content of the first and second substances,

is the MAC delay variance of the neighbor node j,

Step 112: and updating the Q value table, wherein two Q value tables are stored in the node, one Q value table is randomly selected to be updated after each data packet is sent, and the probability of each Q value table being selected is 50%. When the first Q value table is selected, the update formula is as follows:

Q₁(s，a)＝Q₁(s，a)+α×(reward+γ×maxQ₂(s′，a^*)-Q₁)

when the 2 nd Q-value table is selected, the formula is updated as follows:

Q₂(s，a)＝Q₂(s，a)+α×(reward+γ×maxQ₁(s′，a^*)-Q₂)

After the Q value table is updated, the work of the routing layer in the sending process is completed, and the next data packet sending is waited.

The details of the specific mathematical calculations and variables involved in the calculations in steps 101-112 are described below.

< equation for step 101 >

When the random back-off counter of the node becomes 0 or when the node runs for the first time, a new value is assigned to the random back-off counter, and the assignment formula is as follows:

wherein, C takes 3N, N is the number of nodes in the node single-hop range, rand (context Window)₀) Finger generation is in the interval [0, context Window₀-1]The random number in the random number, the content Window, is a Contention Window, and the calculation formula is as follows:

wherein i is the number of retransmissions, m is the maximum number of increases of the contention window, when the number of retransmissions exceeds this value, the contention window is unchanged, R represents the maximum number of retransmissions, W is the minimum contention window, and 1 is taken.

< equation for step 102 >

When the node sends RTS and CTS data packets, it needs to record the occupied time of the control channel and the earliest available time of the data channel into the data packets to notify the neighboring nodes near both sides of the communication node, so as to avoid collision.

When a node sends an RTS packet, the control channel occupies the following time:

Control_rts＝2×SIFS+CTS

where SIFS denotes a time occupied by a short interframe space, and CTS denotes a time required to transmit a CTS packet.

The earliest time of availability of the data channel is:

Data_rts＝3×SIFS+CTS+DATA+ACK+DIFS

wherein, DIFS represents the time occupied by the distributed interframe space, ACK, and DATA represents the time required for transmitting the ACK packet and DATA.

The two sides of the communication node also need to select the data channel used in communication, when sending RTS packet, the source node will record the selected data channel into RTS packet, the principle of selection is that the data channel used for completing transmission successfully recently is selected preferentially, if the data channel is not available, the data channel with earliest available time is selected from the available data channels. The conditions under which the data channel is available are:

T_data≥T_spare

< equation for step 103 >

When a node is to send a CTS packet, the time that the control channel will occupy and the earliest time of availability of the data channel are recorded in the CTS. The time that the control channel will occupy is calculated as follows:

Control_cts＝SIFS

the earliest time available for a data channel is calculated as follows:

Data_cts＝2×SIFS+DATA+ACK+DIFS

< equation for calculation in step 106 >

Due to channel contention, the transmission of the data packet may succeed or fail. When the node receives the ACK packet, the data transmission is successful, and at this time, the MAC delay calculation formula is as follows:

MAC delay_success＝T_re-ack-T_in-gueue

MAC delay_fail＝T_max-retry-T_in-queue

wherein, T_max-retryIndicating the time when the number of MAC layer packets exceeds the maximum number of retransmissions.

< equation for calculation in step 107 >

The method comprises the steps that a node periodically sends HELLO packets, the positions, the speeds, the maximum Q values and link information of the node are recorded in the HELLO packets, the node establishes a neighbor node table according to the received HELLO packets, a timer is given to each record, and when the timer exceeds 300ms and the node does not receive the HELLO packet sent by a neighbor node corresponding to a new record, the neighbor node is considered not to be in the propagation range of the current node, and the record is deleted.

< equation for calculation in step 108 >

When a node selects a forwarding node, if all nodes in a neighbor table of the node are selected as neighbor nodes, selecting the neighbor node with the highest UCB value, wherein the calculation formula of the UCB value is as follows:

< equation for calculation in step 109 >

After the node sends the data packet, calculating a return value, wherein a return value calculation formula is as follows:

< equation for calculation in step 110 >

The node dynamically adjusts the learning rate of each link according to the speed condition of the neighbor node, and the calculation formula of the learning rate is as follows:

is the variance of the velocity of the neighboring node j,

< equation for calculation in step 111 >

The node dynamically adjusts the discount factor of each link according to the MAC delay change condition of the sent data packet, and the calculation formula of the discount factor is as follows:

wherein the content of the first and second substances,

is the MAC delay variance of the neighbor node j,

< equation for step 112 >

Two Q value tables are stored in the node, one Q value table is randomly selected to be updated after a data packet is sent each time, and the probability of each Q value table being selected is 50%. When the first Q value table is selected, the update formula is as follows:

Q₁(s，a)＝Q₁(s，a)+α×(reward+γ×maxQ₂(s′，a^*)-Q₁)

when the 2 nd Q-value table is selected, the formula is updated as follows:

Q₂(s，a)＝Q₂(s，a)+α×(reward+γ×maxQ₁(s′，a^*)-Q₂)

in the above two equations, Q (s, a) represents the expectation of the benefit that would be obtained by taking action a at a certain time s state, s ' represents the next state after taking action a at state s, a x represents one of the set of actions that can be taken at state s ', maxQ (s ', a)^*) Representing the maximum utility value that can be achieved by taking action at s', α is the learning rate, which is 1 number between 0 and 1, and γ is the discount factor, which is 1 number between 0 and 1.

The intelligent cross-layer data transmission protocol of the unmanned system network of the invention is subjected to simulation experiments by specific examples and is explained below.

Firstly, comparing MAC protocols, the example simulates an experiment in a wireless network simulator WSNet environment, in the example, nodes are distributed in the area of 500m x500m x500m, and other nodes are randomly distributed. Table 1 describes the following detailed information of MAC protocol versus experimental common parameters.

In this example, an antenna model of antenna _ omni _ directional is adopted, and each node communicates by using a propagation _ range model, and the communication range is 250 m. Under a static condition, 3 groups of experiments are carried out according to the difference of the number of nodes, under a dynamic condition, 5 groups of experiments are carried out according to the difference of packet sending intervals of source nodes, under the condition that the packet sending interval of the source nodes is 10ms, 3 groups of experiments are carried out according to the difference of the maximum moving speeds of the nodes, and under the condition that the packet sending interval of the source nodes is 45ms, 3 groups of experiments are carried out according to the difference of the maximum moving speeds of the nodes. The network topology adopts a star topology structure, the center is a receiving node, and the other nodes are source nodes.

In the experiment, the MAC protocol experiment of the invention is compared with other two MAC protocols, namely a classic DCF protocol and a DC-DCF protocol.

We evaluate the MAC protocol of the present invention from this performance indicator of MAC delay. Before analyzing the experimental results, the performance indexes related to the experiment are briefly explained:

MAC time delay: the time from the time when the data packet enters the sending queue of the MAC layer to the time when the destination node receives the ACK is up, if the transmission fails, the time is the time from the time when the data packet enters the sending queue of the MAC layer to the time when the data packet is retransmitted for the maximum times;

fig. 2 shows the relationship between the MAC delay and the number of nodes in the case of a source node packet transmission interval of 10ms under static conditions. In a network saturation state, the MAC delays of the three algorithms are increased along with the increase of the number of nodes, because the number of nodes competing for a channel at the same time in the network is increased along with the increase of the number of nodes, so that the number of collisions is increased, and finally the MAC delay is increased. The MAC delay of the DCF is obviously higher than that of the DC-DCF and the CSMA/MAS, the average MAC delay of the DCF is 3.6 times that of the DC-DCF and 9.5 times that of the CSMA/MAS, and because the networks of the DC-DCF and the CSMA/MAS can realize self-synchronization, the number of times of collision in the network can be greatly reduced. And the CSMA/MAS adopts a multi-channel technology, so that the utilization rate of the channel is increased, and the MAC average time delay is reduced by 55.73 percent compared with that of the DC-DCF and is reduced by 90.5 percent compared with that of the DCF.

Fig. 3 shows the relationship between the MAC delay and the source node packet transmission interval under dynamic conditions, in case that the maximum moving speed of the node is 40m/s and the number of nodes is 35. It can be seen that as the packet transmission interval increases, the MAC delay decreases continuously, and after decreasing to a certain extent, the MAC delay is in a relatively stable state (for example, the MAC delay at 45ms is not much different from that at 60 ms), and in general, the average delay of CSMA/MAS is still better than that of DCF and DC-DCF, and the average delay of DCF is 3.3 times that of DC-DCF and 10.9 times that of CSMA/MAS. And the CSMA/MAS average time delay is reduced by 63.6 percent compared with the DC-DCF and 91.6 percent compared with the DCF.

Fig. 4 shows the relationship between the MAC delay and the maximum moving speed of the node when the source node packet interval is 10ms and the number of nodes is 35. It can be seen that when the maximum moving speed of the node increases, the MAC delay also increases, because the larger the speed change is, the more times the node fails to send the data packet increases, and the contention window also increases, thereby increasing the MAC delay. And the average time delay of the CSMA/MAS is still better than that of the DCF and the DC-DCF under different speeds. The average delay of DCF is 5.2 times that of DC-DCF and 2.4 times that of CSMA/MAS. The average delay of CSMA/MAS is reduced by 53.6% compared with DC-DCF and 91.9% compared with DCF.

Fig. 5 shows the relationship between the MAC delay and the maximum moving speed of the node when the packet transmission interval is 45ms and the number of nodes is 35. It can be seen that, because the packet transmission interval is larger, the number of nodes competing for the channel at the same time in the network is smaller, so the MAC delays of the three MAC protocols are all smaller than those of the packet transmission interval of 10ms, and meanwhile, the average delay of the CSMA/MAS is still better than that of the DCF and the DC-DCF at different speeds. The average delay of DCF is 4.4 times of that of CSMA/MAS and 2.8 times of that of DC-DCF, and the average delay of CSMA/MAS is reduced by 37.3% and 92.1% compared with that of DC-DCF.

The experimental results of this example demonstrate that the CSMA/MAS protocol described in this invention has a lower MAC delay.

Then, the routing protocol comparison and the cross-layer protocol comparison are carried out, the example simulates an experiment in a wireless network simulator WSNet environment, in the example, the nodes are distributed in the area of 500m x500m x500m, and other nodes are distributed randomly. Table 2 describes the detailed information of the experimental parameters of the present invention.

TABLE 2 parameter configuration Table

In this example, an antenna model of antenna _ omni _ directivity is adopted, each node communicates by using a propagation _ range model, the communication range is 250m, the time interval of one HELLO packet is 100ms, and the life cycle of each neighbor node is 300 ms. In the experiment, only the source node sends data, the destination node receives data, and other nodes forward the received data. In addition to the destination node, other nodes randomly change the speed and direction of movement. 10 groups of experiments are carried out according to the maximum moving speed of the nodes, 3 groups of experiments are carried out according to the number of the nodes, 3 groups of experiments are carried out according to the time interval of sending data by the source node, each group of experiments are simulated for 100 times, the source node sends 10000 data packets, and the size of each data packet is 250 Bytes.

In this experiment, the inventive examples were compared to two other protocols, three of which are shown below:

among the above protocols, QGeo is the existing Routing Protocol (QGeo: Q-Learning based geographic Ad-Hoc Routing Protocol for Unmanned Routing Networks, Jung WS, 2017), CSMA/MAS is the abbreviation of MAC Protocol (CSMA/MAS) proposed in the present invention, and DCF is the classic CSMA/CA MAC Protocol.

The intelligent data transmission protocol of the unmanned system is evaluated according to 2 performance indexes of end-to-end average delay and successful arrival rate of data packets. Before analyzing the experimental results, 2 performance indexes related to the experiment are briefly explained:

end-to-end average delay: average time delay for a data packet to successfully arrive at the destination node D from the source node S;

packet successful arrival rate: the data packet number received by the destination node D accounts for the percentage of the data packet number sent by the source node S;

fig. 6 shows the relationship between the successful arrival rate of the data packets and the maximum speed of the node when the packet sending interval is 30ms and the number of the nodes is 35. When the maximum speed of the node is increased, the successful arrival rates of the data packets of the three protocols are reduced to different degrees, because the faster the moving speed of the node is, the more frequent the network topology changes, and the neighbor nodes of the node continuously change, so that the data packets cannot be received by the corresponding neighbor nodes, and the successful arrival rates of the data packets are reduced. It can be seen that protocol 3 has better performance than protocol 2 and protocol 1. The successful arrival rate of the data packets of the protocol 3 is improved by 5.4 percent compared with the average arrival rate of the data packets of the protocol 2 and is improved by 10.8 percent compared with the average arrival rate of the data packets of the protocol 1. Protocol 2 is a 5.2% improvement over protocol 1. It can be seen that in the high speed state, the packet successful arrival rate of protocol 3 is lower than that of protocol 1 and protocol 2. Taking the section of the maximum speed of the node of 70m/s-100m/s as an example, the average descending degree of the successful arrival rate of the data packet of the protocol 3 is 1.9%, and the average descending degrees of the protocol 2 and the protocol 1 are 4.6% and 7.4% respectively, which shows that the designed cross-layer protocol can better adapt to the node speed change in a larger range, and can still keep the higher successful arrival rate of the data packet in a high-speed state.

Fig. 7 shows the relationship between the end-to-end delay and the maximum node speed at a packet transmission interval of 30ms and a node number of 35. The end-to-end delay of the protocol 1 is obviously higher than that of the

protocols

2 and 3, the average end-to-end delay of the protocol 1 is 2.8 times of that of the protocol 2 and 3.3 times of that of the protocol 3, because the MAC layers of the

protocols

2 and 3 can realize conflict-free network self-synchronization, the MAC delay is greatly reduced, and the end-to-end delay of a data packet is reduced. When the node speed is increased, the end-to-end delay rise amplitude of the protocol 2 and the protocol 3 is not large. Comparing protocol 3 with protocol 2, it can be seen that the end-to-end delay of protocol 3 is better than that of protocol 2, and the average end-to-end delay of protocol 3 is reduced by 15.9% compared with that of protocol 2.

FIG. 8 shows the relationship between the successful arrival rate of data packets and the node density at a maximum node speed of 40m/s at a packet transmission interval of 30 ms. When the node density is increased, the successful arrival rate of the data packet is correspondingly increased, because as the node density is increased, the neighbor nodes of the node are increased, the selectivity of the next hop of the node is increased, and the data packet is easier to reach the destination node. As can be seen from the figure, the successful arrival rates of the packets are not much different when the number of nodes is 50 and the number of nodes is 35, which indicates that the network is close to a saturation state. Comparing different protocols, it can be found that the successful arrival rate of the data packets of protocol 3 is better than that of protocol 2 and protocol 1. The average successful arrival rate of the protocol 3 is improved by 8.0 percent compared with the protocol 1 and is improved by 3.4 percent compared with the protocol 2. Protocol 2 is a 4.5% improvement over protocol 1.

FIG. 9 shows the relationship between end-to-end delay and node density at 30ms interval and 40m/s maximum node speed. As node density increases, end-to-end delay increases. The average end-to-end delay of protocol 3 is 16.6% lower than that of protocol 2 and 63.7% lower than that of protocol 1. The average end-to-end delay of protocol 2 is reduced by 56.5% compared to protocol 1.

Fig. 10 shows the relationship between the successful arrival rate of packets and the packet transmission interval at the node number of 35 and the maximum moving speed of the node of 40 m/s. The successful arrival rate of the data packets rises along with the rise of the packet sending interval, and the shorter the packet sending interval of the source node is, the more the number of the data packets of the MAC layer sending queue is within a certain time, and the number of times of channel competition among the nodes is increased, so that the data packets can not be successfully sent, and the successful arrival rate of the data packets is reduced. Particularly, when the packet sending interval is smaller, the successful arrival rate of the data packet is reduced more, and as can be seen from the figure, when the packet sending interval is changed from 30ms to 15ms, the successful arrival rate of the data packet in the three protocols is reduced more than that when the packet sending interval is changed from 45ms to 30 ms. Comparing different protocol architectures, it can be seen that the successful arrival rate of the data packets of protocol 3 is better than that of the other two protocols. The successful arrival rate of the data packets of the protocol 3 is improved by 19.9 percent compared with the protocol 1 and is improved by 11.2 percent compared with the protocol 2. The successful arrival rate of the data packets of the protocol 2 is improved by 7.8 percent compared with that of the protocol 1.

Fig. 11 shows a graph showing the relationship between the end-to-end delay and the packet transmission interval at the node number of 35 and the maximum moving speed of the node of 40 m/s. The smaller the packet sending interval of the source node is, the larger the end-to-end delay is due to the increased collision times of the MAC layer, and for protocol 1, when the packet sending interval is changed from 30ms to 15ms, the rise amplitude of the end-to-end delay is much higher than that of

protocols

2 and 3, because the MAC layer of protocol 1 uses the classic DCF mode, the increased collision times continuously increases the contention window, and even causes the packet to be discarded. And the MAC layers of the protocol 2 and the protocol 3 can realize automatic synchronization, thereby greatly reducing the number of times of conflict. Compared with different protocols, the average end-to-end delay of the protocol 3 is reduced by 73.5% compared with the protocol 1 and 14.8% compared with the protocol 2. While protocol 2 has an average end-to-end delay that is 69.0% lower than protocol 1.

The experimental results of this example demonstrate that the intelligent data transmission protocol for an unmanned system according to the present invention has a lower end-to-end delay and a higher successful arrival rate of data packets than the existing protocol.

The following are system examples corresponding to the above method examples, and this embodiment can be implemented in cooperation with the above embodiments. The related technical details mentioned in the above embodiments are still valid in this embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related-art details mentioned in the present embodiment can also be applied to the above-described embodiments.

Claims

1. An intelligent cross-layer data transmission method of an unmanned system network comprises the steps of constructing the unmanned system network with a plurality of unmanned devices as nodes, wherein the nodes periodically send HELLO packets, the nodes receiving the HELLO packets establish a neighbor table according to the HELLO packets, the neighbor table records the position, link information, moving speed and UCB value of each neighbor node, and the transmission of the data packets from a source node to a destination node in the unmanned system network comprises a channel access step and a data routing step;

the data routing step is to judge whether a node in the neighbor node table is not selected as a forwarding node by the current node, if so, a probe data packet is sent to obtain a UCB value, otherwise, the node with the maximum UCB value in the neighbor node table is selected and a data packet is sent; meanwhile, according to the moving speed and the MAC delay of the neighbor node, the learning rate and the discount factor are adjusted so as to update the UCB value of the corresponding forwarding node;

wherein the data routing step comprises: and calculating a learning rate and a discount factor by using the node moving speed and the MAC delay in the MAC layer, substituting the calculated learning rate and discount factor into an improved Q learning algorithm to update two Q value tables, and updating the UCB value by using the updated Q value.

2. The intelligent cross-layer data transmission method for the unmanned system network as claimed in claim 1, wherein the channel accessing step further comprises: before starting a new data transmission process, the node accesses the channel through the continuous back-off time so as to achieve collision-free channel access.

3. The intelligent cross-layer data transmission method of the unmanned system network of claim 2, wherein the data routing step calculates the UCB value by the following formula:

4. The intelligent cross-layer data transmission method for the unmanned system network of claim 2, wherein the continuous back-off time is calculated by:

5. The intelligent cross-layer data transmission method for the unmanned system network of claim 2, wherein the node implementing the collision-free channel access comprises:

when a plurality of nodes in the network need to access the channel, whenever one of the nodes successfully accesses the channel to send data, according to the continuous backoff time, the one node forbids re-competing for the channel to use only in the next C time slots, and only the node which fails in competition loses competition for the channel use right; after the plurality of nodes all obtain the usage right of the channel, because the persistent backoff counters of other nodes need to be suspended when the nodes send the data packets, and the nodes select the data channel successfully transmitted before as the data channel used for transmitting the data packets according to the data channel selection policy, at a later time, the nodes in the network sequentially obtain the usage right of the channel according to the sequence that the previous nodes send the data packets, so as to reach a synchronous state, and the period for the nodes to obtain the usage right of the channel is represented by the following formula:

T_period＝T_c+(N-1)×T_{transmit-control}+T_DIFS

wherein, T_CIs the time occupied by the C time slots, T_{transmit-control}Is the time, T, occupied on the control channel during data transmission_DIFSIs the time taken by the distributed inter-frame spacing, and N is the number of nodes in the single-hop range.

6. An intelligent cross-layer data transmission system of an unmanned system network comprises the unmanned system network which takes a plurality of unmanned devices as nodes, and is characterized in that the nodes periodically send HELLO packets, the nodes which receive the HELLO packets establish a neighbor table according to the HELLO packets, the neighbor table records the position, link information, moving speed and UCB value of each neighbor node, and the transmission of the data packets from a source node to a destination node in the unmanned system network comprises a channel access module and a data routing module;

the data routing module judges whether a node in the neighbor node table is not selected as a forwarding node by the current node, if so, a probe data packet is sent to obtain a UCB value, otherwise, the node with the maximum UCB value in the neighbor node table is selected and a data packet is sent; meanwhile, according to the moving speed and the MAC delay of the neighbor node, the learning rate and the discount factor are adjusted so as to update the UCB value of the corresponding forwarding node;

wherein the data routing module includes: and calculating a learning rate and a discount factor by using the node moving speed and the MAC delay in the MAC layer, substituting the calculated learning rate and discount factor into an improved Q learning algorithm to update two Q value tables, and updating the UCB value by using the updated Q value.

7. The intelligent cross-layer data transmission system of claim 6, wherein the channel access module further comprises: before starting a new data transmission process, the node accesses the channel through the continuous back-off time so as to achieve collision-free channel access.

8. The intelligent cross-layer data transmission system of claim 7, wherein the data routing module calculates the UCB value by the following formula:

9. The intelligent cross-layer data transmission system of claim 7, wherein the duration back-off time is calculated by:

wherein i is retransmission, C is 3N, and N is the number of nodes in a node single-hop rangeThe Contention Window is a Contention Window rand (Contention Window)_i) Means that the generation interval is [0, context Window_i-1]The random number in (c).

10. The intelligent cross-layer data transmission system of unmanned system network of claim 7, wherein the nodes implementing collision-free channel access comprises:

T_period＝T_c+(N-1)×T_{transmit-control}+T_DIFS