CN116419290B - Underwater acoustic communication energy optimization method based on cross-layer design combined depth Q network - Google Patents

Underwater acoustic communication energy optimization method based on cross-layer design combined depth Q network Download PDF

Info

Publication number
CN116419290B
CN116419290B CN202310504832.6A CN202310504832A CN116419290B CN 116419290 B CN116419290 B CN 116419290B CN 202310504832 A CN202310504832 A CN 202310504832A CN 116419290 B CN116419290 B CN 116419290B
Authority
CN
China
Prior art keywords
network
layer
energy
energy optimization
parameters
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310504832.6A
Other languages
Chinese (zh)
Other versions
CN116419290A (en
Inventor
刘帅
王渝
郁泽慧
曹润琪
王猛
矫禄禄
刘钊
王景景
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qingdao University of Science and Technology
Original Assignee
Qingdao University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qingdao University of Science and Technology filed Critical Qingdao University of Science and Technology
Priority to CN202310504832.6A priority Critical patent/CN116419290B/en
Publication of CN116419290A publication Critical patent/CN116419290A/en
Application granted granted Critical
Publication of CN116419290B publication Critical patent/CN116419290B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W24/00—Supervisory, monitoring or testing arrangements
    • H04W24/02—Arrangements for optimising operational condition
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B13/00—Transmission systems characterised by the medium used for transmission, not provided for in groups H04B3/00 - H04B11/00
    • H04B13/02—Transmission systems in which the medium consists of the earth or a large mass of water thereon, e.g. earth telegraphy
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W16/00—Network planning, e.g. coverage or traffic planning tools; Network deployment, e.g. resource partitioning or cells structures
    • H04W16/22—Traffic simulation tools or models
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W52/00—Power management, e.g. TPC [Transmission Power Control], power saving or power classes
    • H04W52/04—TPC
    • H04W52/18—TPC being performed according to specific parameters
    • H04W52/24—TPC being performed according to specific parameters using SIR [Signal to Interference Ratio] or other wireless path parameters
    • H04W52/241—TPC being performed according to specific parameters using SIR [Signal to Interference Ratio] or other wireless path parameters taking into account channel quality metrics, e.g. SIR, SNR, CIR, Eb/lo
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W52/00—Power management, e.g. TPC [Transmission Power Control], power saving or power classes
    • H04W52/04—TPC
    • H04W52/18—TPC being performed according to specific parameters
    • H04W52/26—TPC being performed according to specific parameters using transmission rate or quality of service QoS [Quality of Service]
    • H04W52/267—TPC being performed according to specific parameters using transmission rate or quality of service QoS [Quality of Service] taking into account the information rate
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W84/00—Network topologies
    • H04W84/18—Self-organising networks, e.g. ad-hoc networks or sensor networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Quality & Reliability (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

The invention discloses an underwater acoustic communication energy optimization method based on a cross-layer design joint depth Q network. Firstly, underwater acoustic sensor networks are arranged, and transmission power constraint of an underwater network physical layer, channel access rate constraint of an MAC layer and data packet throughput constraint of a routing layer are established on the basis of considering energy; then, based on the constraint, performing cross-layer design on the network, and establishing an energy optimization model crossing three layers; and finally, solving the energy optimization model parameters by adopting a depth Q network, and iteratively updating the cross-layer energy optimization model parameters until the optimal parameters of network energy optimization are output. The invention can effectively cope with the environment with limited energy when the underwater acoustic sensor network is in communication, and reduces the energy consumption of network communication.

Description

Underwater acoustic communication energy optimization method based on cross-layer design combined depth Q network
Technical Field
The invention belongs to the technical field of underwater acoustic communication and ocean information, and particularly relates to an underwater acoustic communication energy optimization method based on a cross-layer design combined depth Q network.
Background
The underwater communication network has wide application in the fields of marine disaster early warning, underwater military operations, offshore oil exploration and the like.
Underwater sensor networks face energy efficiency challenges because of limited battery power and difficulty in charging underwater. Unlike land networks, the underwater protocol stack is mostly divided into three layers, namely a physical layer, a data link (MAC) layer, and a routing layer. The layers operate independently in principle, information interaction cannot occur, however, information parameters of each layer in the protocol stack can have great influence on network energy consumption, information of the layers are mutually coupled, information of other layers can influence the layer, and if energy optimization is only performed on a single-layer protocol stack without considering information influence of other layers, low-power-consumption operation of network energy is difficult to realize.
In summary, although the conventional method can reduce the energy consumption of the underwater acoustic communication network to a certain extent, the optimization problem often only considers the single-layer information design, and the energy optimization is not performed by fully combining the information of each sub-layer in the network.
Disclosure of Invention
The invention aims to provide an underwater acoustic communication energy optimization method based on a cross-layer design combined depth Q network, so as to make up for the defects of the prior art.
In order to achieve the aim of the invention, the invention is realized by adopting the following technical scheme:
an underwater acoustic communication energy optimization method based on a cross-layer design joint depth Q network comprises the following steps:
s1: laying an underwater sound sensing network, and establishing transmission power constraint of an underwater network physical layer, channel access rate constraint of an MAC layer and data packet throughput constraint of a routing layer;
s2: based on the constraint of each layer of network of the S1, performing cross-layer design on the underwater sound sensing network, and establishing a cross-layer energy optimization model;
s3: solving the cross-layer energy optimization model parameters by using DQN, and iteratively updating the model parameters;
s4: verifying the model parameters obtained in the step S3, and returning to the step S3 to iteratively update the cross-layer energy optimization model after the parameters if the obtained energy consumption value is greater than the threshold value; and if the energy consumption value is smaller than the threshold value, outputting the optimal parameters of the current cross-layer energy optimization model.
Further, the S1 specifically includes the following steps:
s1-1: establishing physical layer transmission power constraints:
laying a sensing network underwater, and defining a water surface base station asAnd has a physical layer capacity of +.>The power required by the node of (a) in transmitting the data packet is +.>Use->To represent the power consumption between two consecutive transmissions, a transmission power constraint model is built as:
；
wherein ,representing physical layer bit capacityDecay index of quantity with distance, +.>The fluctuation coefficient representing the physical layer bit capacity is calculated by the following formula:
；
；
wherein ,representation->Is satisfied by the discount factor of->，/>Representation->Is satisfied by the discount factor of->；
S1-2: establishing a channel access rate constraint of the MAC layer:
firstly, establishing a channel utilization model of a MAC layer，/>The calculation formula of (2) is as follows:
；
wherein ,representing the probability of successful transmission of the MAC layer flow, +.>And representing the waiting time of the traffic transmission failure, and the maximum channel access rate constraint formula is as follows:
；
wherein ,the number and the size of the transmission traffic of the MAC layer;
s1-3: establishing a packet throughput constraint of a routing layer:
assume that the transmission probability of the routing layer data packet isThen by transmission probability->Represented total throughput of network data packets +.>The method comprises the following steps:
；
wherein ,indicating the number of routing layer packet transmissions, +.>The probability of successful transmission of the routing layer data packet is represented; total throughput for network packets>Deriving to obtain the best transmission probability->The method comprises the following steps:
；
will beSubstitution network packet total throughput +.>In the calculation formula of (2), the relation between the node residual energy and the throughput is considered to obtain the optimal routing layer data packet throughput constraint +.>The method comprises the following steps:
；
wherein ,indicating the number of routing layer packet transmissions, +.>Indicating the probability of successful transmission of the routing layer packet.
Further, in the step S2, a cross-layer energy optimization model is established:
based on the network constraints of each layer of S1, an energy optimization model crossing three layers is established:
；
wherein ,indicate->The energy consumption of each node, and the objective of cross-layer energy optimization is the mostMinimizing the energy consumption of all nodes in the network, < >>Representing the maximum throughput of the network +.>Indicating maximum channel access rate,/->Representing channel bandwidth, +.>Representing the signal to noise ratio +.>Representing network runtime.
Further, the S3 specifically includes the following steps:
s3-1: establishing a Q value iteration function model:
the energy optimization problem is established as a triplet, />, />) The method comprises the steps of obtaining a group of parameters with the maximum Q value through an existing energy optimization data training network, and updating a formula of a Q value iteration function to be:
；
wherein ,representing execution of an action->Transition to State->Probability of->Representing discount factors->Is the reward obtained by performing the action, +.>Is indicated in the state->Performs +.>The maximum Q value obtained;
s3-2: establishing a deep Q network model rewarding strategy:
in a hydroacoustic sensor network, the energy of the sensor nodes is limited, and in order to balance the consumed energy among the nodes, the DQN model rewarding strategy takes the energy of the nodes into account, and the depth Q network model rewarding functionThe calculation formula is as follows:
；
wherein , representing the remaining energy of nodes in the network;
s3-3: solving energy optimization model parameters:
obtaining a group of parameters with the maximum Q value through the existing energy optimization data training network, calculating a loss value by means of mean square error between the parameters and ideal parameters, and solving the optimal parameters of an energy optimization model by continuously adjusting the parameters to reduce the loss value, wherein a solving formula of the loss function is as follows:
；
wherein ,representing weight(s)>Representing ideal parameters, the calculation formula is as follows:
；
wherein ,is the reward obtained by performing the action, +.>Representing discount factors->Is indicated in the state->In all actions of (2), the weight is +.>Execution +.>The maximum Q value obtained.
Further, in S4, the optimal parameters for energy optimization are iteratively output:
the obtained model parameters are carried into a test set for verification, the energy consumption value is calculated, and the network energy consumption value is calculatedThe calculation formula is as follows:
；
wherein ,representing network runtime, +.>Indicate->Channel capacity of individual nodes,/->Indicate->Probability of successful transmission of the data packet of each node; and if the obtained energy consumption value is larger than the threshold value, returning to the S3, and if the energy consumption value of the energy optimization model after the iteration updating of the parameters is continued is smaller than the threshold value, outputting the optimal parameters of the current energy optimization model.
The invention has the following advantages and technical effects:
firstly, establishing transmission power constraint of an underwater network physical layer, channel access rate constraint of an MAC layer and data packet throughput constraint of a routing layer on the basis of considering energy, and effectively establishing constraint relation among three layers of protocol stacks; then, based on the constraint, cross-layer design is carried out, a cross-layer energy optimization model is established, the limitation among layers is broken, and information interaction among layers is realized; and finally, solving the energy optimization model parameters by adopting a depth Q network, and iteratively updating the cross-layer energy optimization model parameters until the optimal parameters of network energy optimization are output, thereby obtaining the optimal value of network energy optimization.
Through verification, the method can effectively cope with the special environment with limited energy of the water residue sensor network, reduce the whole energy consumption and reduce the death speed of nodes in the network.
Drawings
FIG. 1 is an overall flow chart of one embodiment of the present invention.
Fig. 2 is a diagram of the deep Q network model architecture of one embodiment of the present invention.
FIG. 3 is a graph comparing simulation results of the present invention with the prior art method regarding energy consumption as a function of node density for one embodiment of the present invention.
Fig. 4 is a graph comparing simulation results of the present invention with the prior art method regarding the number of surviving nodes over time according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings and examples.
Example 1:
the sensor network is deployed underwater, and because the capacity of the sensor node battery is limited and the power supply battery cannot be replaced, the requirement that a low-power-consumption network is difficult to reach by considering single-layer protocol stack optimization is considered. How to effectively cope with the special environment with limited energy of the underwater acoustic sensor network, reduce the overall energy consumption of the network and reduce the death speed of nodes in the network is a technical problem to be solved by the embodiment.
The embodiment provides an underwater acoustic communication energy optimization method based on a cross-layer design joint depth Q network, the whole flow chart of which is shown in figure 1, comprising the following steps:
s1: the method comprises the following specific steps of establishing an underwater network physical layer transmission power constraint, a channel access rate constraint of an MAC layer and a data packet throughput constraint of a routing layer on the basis of considering energy by laying a sensor network underwater:
s1-1: establishing transmission power constraint of an underwater network physical layer:
underwater laying underwater acoustic sensor network, the base station at a specified distance from the water surface isAnd has a physical layer capacity of +.>The power required by the node of (a) in transmitting the data packet is +.>Use->To represent the power consumption between two consecutive transmissions, a transmission power constraint model is built as:
；
wherein ,an attenuation index indicating physical layer bit capacity with distance, < >>The fluctuation coefficient representing the physical layer bit capacity is calculated as:
；
；
wherein ,representation->Is satisfied by the discount factor of->，/>Representation->Is satisfied by the discount factor of->。
S1-2: establishing channel access rate constraints of an underwater network MAC layer:
firstly, establishing a channel utilization model of a MAC layer，/>The calculation formula of (2) is as follows:
；
wherein ,representing the probability of successful transmission of the MAC layer flow, +.>And representing the waiting time of the traffic transmission failure, and the maximum channel access rate constraint formula is as follows:
；
wherein ,the number of traffic is transmitted for the MAC layer.
S1-3: establishing a packet throughput constraint of a routing layer:
assume that the transmission probability of the routing layer data packet isThen by transmission probability->Represented total throughput of network data packets +.>The method comprises the following steps:
；
wherein ,indicating the number of routing layer packet transmissions, +.>Indicating the probability of successful transmission of the routing layer packet. Total throughput for network packets>Deriving to obtain the best transmission probability->The method comprises the following steps:
；
will beSubstitution network packet total throughput +.>In the calculation formula of (2), the relation between the node residual energy and the throughput is considered to obtain the optimal routing layer data packet throughput constraint +.>The method comprises the following steps:
；
wherein ,indicating the number of routing layer packet transmissions, +.>Indicating the probability of successful transmission of the routing layer packet.
S2: based on the constraint of each layer of network of S1, the cross-layer design is carried out on the network, and an energy optimization model of the cross-layer is built, and the specific steps are as follows:
s2-1: building an energy optimization model crossing three layers:
based on the constraint of each layer of network of S1, the cross-layer design is carried out on the network, and an energy optimization model of three layers is built:
；
wherein ,indicate->The energy consumption of individual nodes, the goal of cross-layer energy optimization is to minimize the energy consumption of all nodes in the network, +.>Representing the maximum throughput of the network +.>Indicating maximum channel access rate,/->Representing channel bandwidth, +.>Representing the signal to noise ratio +.>Representing network runtime.
S3: solving the energy optimization model parameters in the S2 by using a Depth Q Network (DQN), and iteratively updating the cross-layer energy optimization model parameters, wherein the method comprises the following specific steps of:
s3-1: establishing a network Q value iteration function model:
the energy optimization problem is established as a triplet, />, />) The method comprises the steps of obtaining a group of parameters with the maximum Q value through an existing energy optimization data training network, and updating a formula of a Q value iteration function to be:
；
wherein ,representing execution of an action->Transition to State->Probability of->Representing discount factors->Is the reward that is obtained by performing the action. />Is indicated in the state->Performs +.>The maximum Q value obtained.
S3-2: establishing a deep Q network model rewarding strategy:
in the underwater acoustic sensor network, the energy of the sensor nodes is limited, and in order to balance the consumed energy among the nodes, the DQN model rewarding strategy considers the energy of the nodes,deep Q network model rewarding functionThe calculation formula is as follows:
；
wherein , representing the remaining energy of the nodes in the network.
S3-3: solving energy optimization model parameters:
obtaining a group of parameters with the maximum Q value through the existing energy optimization data training network, calculating a loss value by means of mean square error between the parameters and ideal parameters, and solving the optimal parameters of an energy optimization model by continuously adjusting the parameters to reduce the loss value, wherein a solving formula of the loss function is as follows:
；
wherein ,representing weight(s)>Representing ideal parameters, the calculation formula is as follows:
；
wherein ,is the reward obtained by performing the action, +.>Representing discount factors->Is expressed in the shape ofStatus->In all actions of (2), the weight is +.>Execution +.>The maximum Q value obtained.
The structure diagram of the deep Q network model used in the embodiment of the present invention is shown in fig. 2. The network gives a deep Q network model reward according to environment constraint information under the condition of considering energy, the parameter estimation network selects a convolutional neural network model (CNN), the input is parameters of each layer of the network, the output is an optimal energy optimization model with the maximum Q value, and the network is continuously and iteratively updated according to the test set network model until the optimal energy optimization parameters are output.
S4: and carrying the obtained model parameters into a test set for verification, and returning to S3 to iterate the energy optimization model after updating the parameters if the obtained energy consumption value is greater than the threshold value. If the energy consumption value is smaller than the threshold value, outputting the optimal parameters of the current energy optimization model, wherein the method comprises the following specific steps:
s4-1: iterative output of optimal parameters for energy optimization:
the obtained model parameters are carried into a test set for verification, the energy consumption value is calculated, and the network energy consumption value is calculatedThe calculation formula is as follows:
；
wherein ,indicate->Channel capacity of individual nodes,/->Indicate->Probability of successful transmission of the data packet of each node. And if the obtained energy consumption value is larger than the threshold value, returning to S3 to continue iterating the energy optimization model after updating the parameters. And if the energy consumption value is smaller than the threshold value, outputting the optimal parameters of the current energy optimization model.
Example 2:
simulation experiments were performed to verify the method proposed in example 1:
simulation results of network energy consumption as a function of node density using the method provided by the embodiments and using the existing single layer energy optimization method are shown in fig. 3. Simulation was performed in Aqua-Sim-NG (NS-3 based underwater sensor network simulator), and the specific parameters of all simulations in this example are shown in table 1.
Table 1 simulation parameters
Simulation parameters Numerical value
Network size 1200×1200×1200m 3
Speed of acoustic signal 1.5Km/s
Number of nodes 100-600
Data packet size 5Kb
Node communication range 50m
Node initial energy 800J
Data transmission rate of node 4kbps
Node data transmission consumption 0.3J/s
。
As can be seen from the simulation results of FIG. 3, the energy consumption of the underwater sensor network is continuously reduced along with the increase of the number of nodes, and when the number of nodes reaches 600, the method saves about 340J of energy compared with a single-layer energy optimization method.
Simulation results of the number of surviving nodes over time in a network using the present invention and existing methods of single layer energy optimization are shown in fig. 4. As can be seen from the simulation results of fig. 4, when the underwater network was operated for 10 hours. The node survival number of the invention is about 360, and the node survival number of the single-layer energy optimization method is about 180.
In summary, the invention can effectively reduce network energy consumption, effectively balance energy consumption and reduce the death speed of the sensor node.
The above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be apparent to one skilled in the art that modifications may be made to the technical solutions described in the foregoing embodiments, or equivalents may be substituted for some of the technical features thereof; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.

Claims (4)

1. The underwater acoustic communication energy optimization method based on the cross-layer design joint depth Q network is characterized by comprising the following steps of:
s1: laying an underwater sound sensing network, and establishing transmission power constraint of an underwater network physical layer, channel access rate constraint of an MAC layer and data packet throughput constraint of a routing layer;
s2: based on the constraint of each layer of network of S1, the underwater sound sensing network is subjected to cross-layer design, and a cross-layer energy optimization model is established:
wherein ,Ei (R ag_al ，λ max ) Represents the energy consumption of the ith node, T represents the maximum throughput of the network, lambda max Represents maximum channel access rate, B represents channel bandwidth, SNR represents signal-to-noise ratio, t represents network run time, R ag_al Representing optimal routing layer packet throughput, P t (l, C) represents the power required by a node l from the surface base station and having a physical layer capacity C when transmitting a data packet;
s3: solving the cross-layer energy optimization model parameters by using DQN, and iteratively updating the model parameters;
s4: verifying the model parameters obtained in the step S3, and returning to the step S3 to iteratively update the cross-layer energy optimization model after the parameters if the obtained energy consumption value is greater than the threshold value; and if the energy consumption value is smaller than the threshold value, outputting the optimal parameters of the current cross-layer energy optimization model.
2. The underwater acoustic communication energy optimization method as claimed in claim 1, wherein S1 is specifically as follows:
s1-1: establishing physical layer transmission power constraints:
the sensor network is laid underwater, the power required by a node with a prescribed distance from a water surface base station to be l and a physical layer capacity to be C when transmitting a data packet is P t (l, C), with P s To represent the power consumption between two consecutive transmissions, a transmission power constraint model is built as:
wherein ,a1 (C) An attenuation index, a, representing the physical layer bit capacity with distance 2 (C) The fluctuation coefficient representing the physical layer bit capacity is calculated by the following formula:
a 2 (C)＝β 3 +β 2 10log 10 C+β 1 (10log 10 (C+1)) 2 ；
wherein ,representation a 1 (C) Is satisfied by the discount factor of->β 1 ，β 2 ，β 3 Representation a 2 (C) Is a discount factor of (1) satisfying beta 1 +β 2 +β 3 ＝1；
S1-2: establishing a channel access rate constraint of the MAC layer:
establishing a channel utilization model U (lambda) of the MAC layer, wherein the calculation formula of U (lambda) is as follows:
wherein ,e-2Nλ Representing the probability of successful transmission of the MAC layer flow, and 1/lambda represents the waiting time of failed transmission of the flow, the maximum channel access rate constraint formula is as follows:
wherein, N is the number of the transmission flow of the MAC layer;
s1-3: establishing a packet throughput constraint of a routing layer:
assuming that the transmission probability of the routing layer packet is p, the total throughput R of the network packet is represented by the transmission probability p ag_al (p) is:
R ag_al (p)＝(n-1)q(1-q) n-2 (1-P)+p(1-q) n-1 ；
wherein n represents the number of times of transmission of the routing layer data packet, and q represents the probability of successful transmission of the routing layer data packet; total throughput R for network packets ag_al (p) deriving the best probability of transmission p ＊ The method comprises the following steps:
will p ＊ Substituting the total throughput R of network data packets ag_al In the calculation formula of (p), obtaining the optimal routing layer data packet throughput constraint R ag_al The method comprises the following steps:
wherein n represents the number of times of transmission of the routing layer data packet, and q represents the probability of successful transmission of the routing layer data packet.
3. The underwater acoustic communication energy optimization method as claimed in claim 1, wherein S3 is specifically as follows:
s3-1: establishing a Q value iteration function model:
the data training network is optimized through the existing energy to obtain a group of parameters with the maximum Q value, and the formula for updating the Q value iteration function is as follows:
wherein ,P(St+1 |s t ，a t ) Representing execution of action a t Transition to state S t+1 And γ represents the discount factor, r t Is the reward that is obtained by performing the action,is shown in state S t+1 Performing the maximum Q value obtained in a';
s3-2: establishing a deep Q network model rewarding strategy:
deep Q network model rewarding function r t The calculation formula is as follows:
wherein ,Eres Representing the remaining energy of nodes in the network;
s3-3: solving energy optimization model parameters:
the solution formula of the loss function is:
L(ω)＝[y(s t+1 ，r t )-Q(s t ，a t ；ω)] 2 ；
wherein ω represents a weight, y(s) t+1 ，r t ) Representing ideal parameters, the calculation formula is as follows:
wherein ,rt Is the reward earned by performing the action, gamma represents the discount factor,is shown in state S t+1 In (2) when the weight is ω, performing the maximum obtained by aQ value.
4. The underwater acoustic communication energy optimization method as claimed in claim 1, wherein in S4, the optimal parameters for energy optimization are iteratively output: verifying the obtained model parameters, calculating the energy consumption value, and calculating the network energy consumption value E i (R ag_al ，λ max ) The calculation formula is as follows:
wherein t represents network runtime, C i Representing the channel capacity of the i-th node,representing the probability of successful transmission of the data packet of the ith node; and if the obtained energy consumption value is larger than the threshold value, returning to the S3, and if the energy consumption value of the energy optimization model after the iteration updating of the parameters is continued is smaller than the threshold value, outputting the optimal parameters of the current energy optimization model.
CN202310504832.6A 2023-05-08 2023-05-08 Underwater acoustic communication energy optimization method based on cross-layer design combined depth Q network Active CN116419290B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310504832.6A CN116419290B (en) 2023-05-08 2023-05-08 Underwater acoustic communication energy optimization method based on cross-layer design combined depth Q network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310504832.6A CN116419290B (en) 2023-05-08 2023-05-08 Underwater acoustic communication energy optimization method based on cross-layer design combined depth Q network

Publications (2)

Publication Number Publication Date
CN116419290A CN116419290A (en) 2023-07-11
CN116419290B true CN116419290B (en) 2023-10-27

Family

ID=87057934

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310504832.6A Active CN116419290B (en) 2023-05-08 2023-05-08 Underwater acoustic communication energy optimization method based on cross-layer design combined depth Q network

Country Status (1)

Country Link
CN (1) CN116419290B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117560093B (en) * 2024-01-11 2024-04-02 汉江国家实验室 Distributed underwater acoustic network multiple access method, device, equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106788782A (en) * 2016-12-06 2017-05-31 哈尔滨工程大学 Underwater sound communication network OFDM Link Physical Layers and MAC layer cross-layer communication method
CN114390469A (en) * 2022-03-23 2022-04-22 青岛科技大学 Service life prolonging method of three-dimensional columnar underwater acoustic sensor network based on cross-layer cooperation
CN114980178A (en) * 2022-06-06 2022-08-30 厦门大学马来西亚分校 Distributed PD-NOMA underwater acoustic network communication method and system based on reinforcement learning
CN115987886A (en) * 2022-12-22 2023-04-18 厦门大学 Underwater acoustic network Q learning routing method based on meta-learning parameter optimization

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050251725A1 (en) * 2004-05-06 2005-11-10 Genieview Inc. Signal processing methods and systems

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106788782A (en) * 2016-12-06 2017-05-31 哈尔滨工程大学 Underwater sound communication network OFDM Link Physical Layers and MAC layer cross-layer communication method
CN114390469A (en) * 2022-03-23 2022-04-22 青岛科技大学 Service life prolonging method of three-dimensional columnar underwater acoustic sensor network based on cross-layer cooperation
CN114980178A (en) * 2022-06-06 2022-08-30 厦门大学马来西亚分校 Distributed PD-NOMA underwater acoustic network communication method and system based on reinforcement learning
CN115987886A (en) * 2022-12-22 2023-04-18 厦门大学 Underwater acoustic network Q learning routing method based on meta-learning parameter optimization

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
仲崇显 ; 李春国 ; 杨绿溪 ; .多业务MIMO-OFDMA/SDMA系统跨层调度与动态资源分配.通信学报.2010,(第09期),全文. *
多业务MIMO-OFDMA/SDMA系统跨层调度与动态资源分配;仲崇显;李春国;杨绿溪;;通信学报(第09期);全文 *
水声网络中的跨层设计研究;许肖梅;邹哲光;;声学技术(第03期);全文 *
移动性无线传感器网络吞吐量跨层优化;丁凡;周永明;;电子技术应用(第02期);全文 *
能量受限的水下无线传感器网络设计研究;杨鸿宇;中国优秀硕士学位论文全文数据库;正文第2-4章 *
许肖梅 ; 邹哲光 ; .水声网络中的跨层设计研究.声学技术.2012,(第03期),全文. *

Also Published As

Publication number Publication date
CN116419290A (en) 2023-07-11

Similar Documents

Publication Publication Date Title
US11748628B2 (en) Method for optimizing reservoir operation for multiple objectives based on graph convolutional neural network and NSGA-II algorithm
CN116419290B (en) Underwater acoustic communication energy optimization method based on cross-layer design combined depth Q network
Chen et al. QMCR: A Q-learning-based multi-hop cooperative routing protocol for underwater acoustic sensor networks
CN109803291B (en) Robust topology generation method based on underwater acoustic sensor network
Fei et al. Energy-efficient clustering algorithm in underwater sensor networks based on fuzzy C means and Moth-flame optimization method
CN111132200B (en) Three-dimensional underwater network topology control method based on potential game and rigid subgraph
Wang et al. Node energy consumption balanced multi-hop transmission for underwater acoustic sensor networks based on clustering algorithm
CN111065107A (en) Quantum genetic algorithm-based safe routing control method for underwater wireless sensor network
CN111526555B (en) Multi-hop routing path selection method based on genetic algorithm
CN109362048B (en) Underground pipe gallery detection method based on wireless sensor network
CN115866735A (en) Cross-layer topology control method based on super-mode game underwater sensor network
CN115099133A (en) TLMPA-BP-based cluster system reliability evaluation method
CN113923123B (en) Underwater wireless sensor network topology control method based on deep reinforcement learning
CN103701647B (en) A kind of wireless network topology optimization generates method
CN108650030B (en) Water surface multi-sink node deployment method of underwater wireless sensor network
CN106879044A (en) The underwater sensor network method for routing that cavity perceives
Zhang et al. A Non-Uniform Clustering Routing Algorithm based on a Virtual Gravitational Potential Field in Underwater Acoustic Sensor Network
CN115243212B (en) Ocean data acquisition method based on AUV assistance and improved cross-layer clustering
CN115843083B (en) Underwater wireless sensor network routing method based on multi-agent reinforcement learning
CN110391851B (en) Underwater acoustic sensor network trust model updating method based on complex network theory
CN114390469B (en) Service life prolonging method of three-dimensional columnar underwater acoustic sensor network based on cross-layer cooperation
CN115987886A (en) Underwater acoustic network Q learning routing method based on meta-learning parameter optimization
Zhu et al. A Q-Learning and Data Priority-Based Routing Protocol with Dynamic Computing Cluster Head for Underwater Acoustic Sensor Networks
Liu et al. IQWOA: improved quantum whale optimization algorithm for clustering in industrial wireless sensor network
CN102882727B (en) Monitoring area partition method for hierarchical monitoring network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant