CN110430547A - More AUV collaboration data collection algorithms in UASNs based on Q-learning - Google Patents
More AUV collaboration data collection algorithms in UASNs based on Q-learning Download PDFInfo
- Publication number
- CN110430547A CN110430547A CN201910670534.8A CN201910670534A CN110430547A CN 110430547 A CN110430547 A CN 110430547A CN 201910670534 A CN201910670534 A CN 201910670534A CN 110430547 A CN110430547 A CN 110430547A
- Authority
- CN
- China
- Prior art keywords
- auv
- cluster
- cluster head
- node
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B13/00—Transmission systems characterised by the medium used for transmission, not provided for in groups H04B3/00 - H04B11/00
- H04B13/02—Transmission systems in which the medium consists of the earth or a large mass of water thereon, e.g. earth telegraphy
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/46—Cluster building
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
- H04L47/2425—Traffic characterised by specific attributes, e.g. priority or QoS for supporting services specification, e.g. SLA
- H04L47/2433—Allocation of priorities to traffic types
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/38—Services specially adapted for particular environments, situations or purposes for collecting sensor information
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W40/00—Communication routing or communication path finding
- H04W40/24—Connectivity information management, e.g. connectivity discovery or connectivity update
- H04W40/32—Connectivity information management, e.g. connectivity discovery or connectivity update for defining a routing cluster membership
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W84/00—Network topologies
- H04W84/18—Self-organising networks, e.g. ad-hoc networks or sensor networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
The invention discloses more AUV collaboration data collection algorithms in a kind of UASNs based on Q-learning, include the following steps: to select cluster head according to certain condition, and cluster head is added in other node self-adaptings nearby, form node cluster；The distribution of AUV task is carried out based on contract net algorithm is improved；Path planning is carried out based on Q-learning algorithm, AUV completes data collection according to the path of planning.For the present invention by carrying out reasonable task distribution to multiple AUV, the improving AUV of the task completes efficiency, reduces data collection delay；The message level that data packet is considered in data collection carries out preferential collection to emergency data, realizes the fast and effective processing for emergency data；Path planning is carried out to AUV by using Q-learning, reduces the distance to go and energy consumption of AUV.
Description
Technical field
The invention belongs to water sound sensor network fields, and in particular to more AUV based on Q-learning in a kind of UASNs
Collaboration data collection algorithm.
Background technique
Water sound sensor network is a kind of emerging and promising network technology, can be widely applied to answer under water
With, such as underwater environment observation, coastline monitor and protect, take precautions against natural calamities, assisting navigation and mine detect.In recent years, underwater wireless
Sensor network is widely applied the concern with many advantages by more and more oceanographers because of it, it can be helped
Human perception and monitor wide unplumbed marine environment, the generation of monitoring and early warning Oceanic disasters, for exploration, using and
Protection marine resources provide important information and support.
The a large amount of perception datas of transmission are responsible on basis one of of the data collection techniques as UASNs, but incident is high
Energy consumption problem.Therefore, how underwater environment feature is combined, equilibrium data efficiency of transmission and node energy consumption have far-reaching significance.
Larger due to underwater sensor network, the communication range of underwater node is limited and the spies such as network topology dynamic change
Point, underwater data collection techniques carry out data collection usually using AUV.
The method for carrying out data collection using AUV at present, is received according to using the quantity of AUV to be broadly divided into single AUV
Collection and multiple AUV carry out two class of data collection.
The first kind is to carry out data collection using single AUV, is collected under water by disposing underwater sensor node
Relevant information, AUV navigate by water collector node data in a network, and are transmitted to the Sink of the water surface.
Ilyas et al. publishes an article " AEDG:AUV- on " Procedia Computer Science " in 2015
aided Efficient Data Gathering Routing Protocol for Underwater Wireless
Sensor Networks " devises a kind of AUV auxiliary high efficient data capture Routing Protocol (AEDG).One AUV is along scheduled
Elliptic orbit is mobile, broadcasts a HP to change information, and select gateway node according to RSSI value and the dump energy of node
(GNS).Meanwhile member node (MNS) is associated with shortest path by gateway node, and is transmitted data packet by gateway node
To AUV.The article " A Multi-hop Approach " that Khan.JU et al. is delivered on " Sensors " in 2016 proposes
A kind of AUV cooperative data transmission gathering algorithm divided based on network.In the algorithm, according to voronoi, she generates a general to Sink
Whole network plane is divided into multiple Thiessen polygons.Each polygon distributes an AUV, and plans road according to cluster situation
Diameter, while AUV carries out cooperative transmission by the way that an agent node is arranged in each region.SEYED MOHAMMAD GHOREYSHI
Et al. article " the Mobile Data Gathering With Hop-Constrained that is delivered on " IEEE ACCESS "
Clustering in Underwater Sensor Networks " proposes a kind of Large-scale Mobile data based on cluster and adopts
Collect (CMDG) scheme, acquires in data and weighed between delay and energy conservation.It is clustered for acoustic sensor and with shortest
Stroke covers its cluster head problem and proposes two kinds of effective algorithms, to obtain approximate optimal solution within the shorter calculating time, discusses
Influence of the chance variation to ECMDG performance is adopted in terms of energy-delay tradeoff when network working time and existing mobile data
When the time of collection scheme compares long, CMDG can effectively keep shorter path length.
Second class is to carry out data collection using multiple AUV, completes relatively large field by the cooperation between multiple AUV
The data collection of scape, the situation for being suitble to network range larger.
" the IEEE International Conference on Mechatronics of Yan, ZP et al. in 2017
And Automation (ICMA) " on publish an article " An Improved Multi-AW Patrol Path Planning
Method " considers the unreliability of subsurface communication link, by increasing the number in path, so that path generates redundancy, guarantees number
According to being able to carry out timely, accurate transmission.Particle swarm algorithm is constrained, two-staged prediction is introduced, realizes the update to particle
Monitoring, avoids the generation of infeasible particle.Weidong Zhou et al. was in " Transactions of the in 2019
Institute of Measurement and Control " on article " the Route planning algorithm that delivers
for autonomous underwater vehicles based on the hybrid of particle swarm
Optimization algorithm and radial basis function " proposes a kind of based on particle group optimizing
(PSO) the underwater robot paths planning method that algorithm is combined with radial basis function (RBF).In improved PSO algorithm, adopt
It prevents improved PSO algorithm from falling into local optimum with big city's criterion, is carried out using path of the RBF to PSO algorithmic rule flat
It is sliding, to avoid falling into local optimum.Bing Sun et al. was in " IEEE Transactions on Cognitive in 2019
And Developmental Systems " on publish an article " Complete Coverage Autonomous Underwater
Vehicles Path Planning Based on Glasius Bio-Inspired Neural Network Algorithm
For Discrete and Centralized Programming " proposes a kind of Ge Lasi based on discrete centralization programming
Bionic neural network (GBNN) algorithm.Basic modeling is carried out based on more AUV all standing problems of grid chart and neural network, then
Devise single AUV complete coverage algorithm based on GBNN algorithm, propose the underwater multi-robot all standing based on GBNN algorithm from
Centralized planning method is dissipated, devises collisionless path for the underwater navigation of multiple underwater robots.
Data collection is carried out using single AUV to be suitable in the lesser situation of network size, when network size increases, is held
It is also easy to produce biggish data collection delay.Therefore corresponding number is completed usually using multiple AUV cooperation when network size is larger
According to the task of collection, but multiple AUV collect need reasonable task divide could completion task more efficiently, while each AUV
Path planning be also problem research emphasis.
Summary of the invention
Present invention mainly solves the problem of have: the task of multiple AUV is reasonably distributed, guarantee task can be efficient
The completion of rate；Efficient path planning is carried out to AUV, the delay and the energy consumption of AUV of data collection is reduced, extends the network longevity
Life.
It realizes above-mentioned technical purpose, reaches above-mentioned technical effect, the invention is realized by the following technical scheme:
More AUV collaboration data collection algorithms based on Q-learning in a kind of UASNs, comprise the following steps:
Step 1: node cluster
Under water in wireless sensor network, node random placement picks out cluster from these nodes according to picking rule
Head node is responsible for that the data of cluster interior nodes are collected and are integrated；After cluster head is selected, it can not received as the node of cluster head
To the statement message from different cluster heads, the node for receiving message sends addition message to nearest cluster head and nearest cluster head is added
Form different node clusters；
Step 2: AUV task distribution
After node clustering, regard each cluster as a collection task；In bidding phase, by cluster head as bid person
Information on bidding is issued, the size information including cluster head position, data packet waits bidder to submit a tender；
Bidding period, AUV is as prospective tenderer；After AUV receives information on bidding, according to the content of information on bidding and certainly
The location of body and energy state, the cost assessing the income of completion task and paying, decide whether to submit a tender, but each
The bid quantity of AUV is limited；The successful AUV that submits a tender becomes the competition that bidder participates in task；
Acceptance of the bid stage, cluster head carry out analytical calculation to the information on bidding being collected into, and select satisfied bidder to complete to appoint
Business, and sign contract, it is specified that bid person can only sign contract with a bidder, but bidder can be less than cluster
Quantity half bid person signing；
Step 3: AUV path planning
After task is distributed, each AUV has been assigned to task, and AUV determines the access order of task cluster, uses Q-
Learning algorithm make rational planning for AUV reach task cluster path.
The picking rule of leader cluster node in above-mentioned steps one are as follows: energy is greater than node primary powerTo prevent cluster head mistake
It is early dead；The distance between two neighboring cluster head is greater than network deployment widthIt is relatively equal to guarantee that node is formed by cluster size
Weighing apparatus；The node for meeting above-mentioned two condition has 40% probability to be picked as cluster head.
In above-mentioned steps two, when cluster head issues information on bidding as bid person, the initial range of transmission of bidding documents is less than node
Communication radiusAnd provide bidding documents effective time be task distribute a time slot length, when beyond effective time not yet
When bidder's competitive bidding, then former bidding documents failure, the range of transmission that bid person needs to expand bidding documents retransmit bidding documents.
In above-mentioned steps two, the bid quantity of bidder is limited to Pt, be according to itself shape after bidder receives bidding documents
State and the cost for completing task carry out comprehensive assessment, decide whether the competitive bidding for participating in the task；It to be thrown after bidder's assessment
When target quantity Mt < Pt, bidding documents information can be continued to, otherwise will not be received again.
In above-mentioned steps three, method that AUV determines the access order of task cluster are as follows:
(3-1) cluster head in t time slot broadcasts a beacon, includes cluster head number V in beaconk, cluster head coordinate Ck(t), itself
Maximum communication radius CRk, the moving range MR of packet information rank DI, node kkInformation, the message level of data packet is divided into
A, b, c three classes, wherein a class is emergence message, needs quickly to handle；
(3-2) if not receiving the beacon from cluster head, the contact probability of AUV and cluster is 0, if AUV receives the letter of cluster head
Mark, then the contact probability for calculating the cluster head of the beacon received is Pt(k, Ai)=Pen (dt(Vk, Ai), CRk, MRk, DI)
Wherein dt(Vk, Ai) indicate the distance between leader cluster node and AUV,
Wherein (Ax, Ay, Az) indicate t moment AUV position coordinates, (Vx, Vy, Vz) indicate t moment leader cluster node position
Coordinate；
After (3-3) AUV obtains the distance between cluster head of each task cluster, compare big with each cluster contact probability
It is small, select the maximum cluster head of contact probability as the next target point of oneself；Pay the utmost attention to the packet information grade in beacon
Not, the high data of preferential collection levels of information carry out data collection according to the size of contact probability if levels of information is consistent；
After (3-4) determines next target cluster of AUV through the above steps, the road of part is carried out using Q-learning
Diameter planning, updates Q-table, selects the movement of Income Maximum；
(3-5) after AUV reaches target cluster, AUV sends acknowledgement frame, cluster, AUV where notifying other clusters AUV at present
Position coordinates and it is expected that the residence time；Due to the influence of water flow, the position of node can change, after collecting to AUV, respectively
The cluster head of a cluster retransmits beacon to AUV, after AUV receives beacon, repeats step (3-2), (3-3), (3-4).
In above-mentioned steps (3-4), in Q-table renewal process, the setting method of reward value be according to AUV previous state and
Current state is set at a distance from target cluster, and it is as follows to update rule:
Wherein, dt(Vk, Aj) indicate the distance between t moment AUV and target cluster head, dt-1(Vk, Aj) indicate t-1 moment AUV
The distance between target cluster head.
In above-mentioned steps (3-4), select the movement of Income Maximum as follows:
The movement of setting AUV is selected as centered on AUV, by the actual act spatial model A of AUV be defined as it is upper and lower,
The discrete movement of left, right, front and rear 6 respectively represents floating, dive, left-hand rotation, right-hand rotation, advance, several fortune of retrogressing of AUV in water
Dynamic state.
Compared with existing water sound sensor network method of data capture, good effect possessed by the present invention are as follows:
(1) cluster head selection is more reasonable, and the size of cluster is relatively uniform, and the quantity of cluster is suitable；
(2) traditional contract net algorithm is improved, the task distribution of AUV is more rationally efficient；
(3) levels of information for considering data packet when data collection is carried out, realizes the priority processing to emergency data；
(4) Q-learning algorithm is used when AUV path planning, selects to reward maximum path by Q-table, have
Conducive to the cruise path for reducing AUV, the energy consumption of AUV is reduced.
Detailed description of the invention
Fig. 1 is the network model figure in the present invention Jing Guo sub-clustering；
Fig. 2 is the block diagram of AUV task distribution in the present invention；
Fig. 3 is the flow chart of AUV path planning；
Fig. 4 is AUV movement selection schematic diagram.
Specific embodiment
Below in conjunction with the accompanying drawings and embodiment the invention will be further described.
More AUV collaboration data collection algorithms based on Q-learning in a kind of UASNs, comprise the following steps:
Step 1: node cluster
As shown in Figure 1, under water in wireless sensor network, node random placement, according to picking rule from these nodes
In pick out leader cluster node, be responsible for the data of cluster interior nodes are collected and are integrated；After cluster head is selected, do not become cluster head
Node will receive the statement message from different cluster heads, receive the node of message and sent to nearest cluster head and message is added is added
Nearest cluster head forms different node clusters；
Step 2: AUV task distribution
As shown in Fig. 2, regarding each cluster as a collection task after node clustering；In bidding phase, by cluster head
Information on bidding is issued as bid person, the size information including cluster head position, data packet waits bidder to submit a tender；
Bidding period, AUV is as prospective tenderer；After AUV receives information on bidding, according to the content of information on bidding and certainly
The location of body and energy state, the cost assessing the income of completion task and paying, decide whether to submit a tender, but each
The bid quantity of AUV is limited；The successful AUV that submits a tender becomes the competition that bidder participates in task；
Acceptance of the bid stage, cluster head carry out analytical calculation to the information on bidding being collected into, and select satisfied bidder to complete to appoint
Business, and sign contract, it is specified that bid person can only sign contract with a bidder, but bidder can be less than cluster
Quantity half bid person signing；
Step 3: AUV path planning
After task is distributed, each AUV has been assigned to task, and AUV determines the access order of task cluster, uses Q-
Learning algorithm make rational planning for AUV reach task cluster path.
The picking rule of leader cluster node in above-mentioned steps one are as follows: energy is greater than node primary powerTo prevent cluster head mistake
It is early dead；The distance between two neighboring cluster head is greater than network deployment widthIt is relatively equal to guarantee that node is formed by cluster size
Weighing apparatus；The node for meeting above-mentioned two condition has 40% probability to be picked as cluster head.
In above-mentioned steps two, when cluster head issues information on bidding as bid person, the initial range of transmission of bidding documents is less than node
Communication radiusAnd provide bidding documents effective time be task distribute a time slot length, when beyond effective time not yet
When bidder's competitive bidding, then former bidding documents failure, the range of transmission that bid person needs to expand bidding documents retransmit bidding documents.
In above-mentioned steps two, the bid quantity of bidder is limited to Pt, be according to itself shape after bidder receives bidding documents
State and the cost for completing task carry out comprehensive assessment, decide whether the competitive bidding for participating in the task；It to be thrown after bidder's assessment
When target quantity Mt < Pt, bidding documents information can be continued to, otherwise will not be received again.
In above-mentioned steps three, as shown in figure 3, the method that AUV determines the access order of task cluster are as follows:
(3-1) cluster head in t time slot broadcasts a beacon, includes cluster head number V in beaconk, cluster head coordinate Ck(t), itself
Maximum communication radius CRk, the moving range MR of packet information rank DI, node kkInformation, the message level of data packet is divided into
A, b, c three classes, wherein a class is emergence message, needs quickly to handle；
(3-2) if not receiving the beacon from cluster head, the contact probability of AUV and cluster is 0, if AUV receives the letter of cluster head
Mark, then the contact probability for calculating the cluster head of the beacon received is Pt(k, Ai)=Pen (dt(Vk, Ai), CRk, MRk, DI)
Wherein dt(Vk, Ai) indicate the distance between leader cluster node and AUV,
Wherein (Ax, Ay, Az) indicate t moment AUV position coordinates, (Vx, Vy, Vz) indicate t moment leader cluster node position
Coordinate；
After (3-3) AUV obtains the distance between cluster head of each task cluster, compare big with each cluster contact probability
It is small, select the maximum cluster head of contact probability as the next target point of oneself；Pay the utmost attention to the packet information grade in beacon
Not, the high data of preferential collection levels of information carry out data collection according to the size of contact probability if levels of information is consistent；
After (3-4) determines next target cluster of AUV through the above steps, the road of part is carried out using Q-learning
Diameter planning, updates Q-table, selects the movement of Income Maximum；
(3-5) after AUV reaches target cluster, AUV sends acknowledgement frame, cluster, AUV where notifying other clusters AUV at present
Position coordinates and it is expected that the residence time；Due to the influence of water flow, the position of node can change, after collecting to AUV, respectively
The cluster head of a cluster retransmits beacon to AUV, after AUV receives beacon, repeats step (3-2), (3-3), (3-4).
In above-mentioned steps (3-4), in Q-table renewal process, the setting method of reward value be according to AUV previous state and
Current state is set at a distance from target cluster, and it is as follows to update rule:
Wherein, dt(Vk, Aj) indicate the distance between t moment AUV and target cluster head, dt-1(Vk, Aj) indicate t-1 moment AUV
The distance between target cluster head.
In above-mentioned steps (3-4), select the movement of Income Maximum as follows:
As shown in figure 4, the movement of setting AUV is selected as centered on AUV, the actual act spatial model A of AUV is defined
For 6 discrete movements of up, down, left, right, before and after, respectively represent AUV floating in water, dive, left-hand rotation, right-hand rotation, advance, after
Move back several motion states.
Claims (7)
1. more AUV collaboration data collection algorithms in a kind of UASNs based on Q-learning, which is characterized in that include following step
It is rapid:
Step 1: node cluster
Under water in wireless sensor network, node random placement picks out cluster head section according to picking rule from these nodes
Point is responsible for that the data of cluster interior nodes are collected and are integrated；After cluster head is selected, it not will receive to come as the node of cluster head
From the statement message of different cluster heads, the node for receiving message is added nearest cluster head to nearest cluster head transmission addition message and is formed
Different node clusters；
Step 2: AUV task distribution
After node clustering, regard each cluster as a collection task；In bidding phase, issued by cluster head as bid person
Information on bidding, the size information including cluster head position, data packet wait bidder to submit a tender；
Bidding period, AUV is as prospective tenderer；After AUV receives information on bidding, according to the content of information on bidding and itself institute
The position at place and energy state, the cost assessing the income of completion task and paying, decide whether to submit a tender, but each AUV
Bid quantity it is limited；The successful AUV that submits a tender becomes the competition that bidder participates in task；
Acceptance of the bid stage, cluster head carry out analytical calculation to the information on bidding being collected into, select satisfied bidder to complete task, and
Contract is signed, it is specified that a bid person can only sign contract with a bidder, but a bidder can be with the number less than cluster
The bid person of the half of amount contracts；
Step 3: AUV path planning
After task is distributed, each AUV has been assigned to task, and AUV determines the access order of task cluster, uses Q-learning
Algorithm make rational planning for AUV reach task cluster path.
2. more AUV collaboration data collection algorithms in UASNs according to claim 1 based on Q-learning, feature
It is, the picking rule of leader cluster node in the step 1 are as follows: energy is greater than node primary powerTo prevent cluster head too early
It is dead；The distance between two neighboring cluster head is greater than network deployment widthIt is relatively equal to guarantee that node is formed by cluster size
Weighing apparatus；The node for meeting above-mentioned two condition has 40% probability to be picked as cluster head.
3. more AUV collaboration data collection algorithms in UASNs according to claim 1 based on Q-learning, feature
It is, in the step 2, when cluster head issues information on bidding as bid person, the initial range of transmission of bidding documents is communicated less than node
RadiusAnd provide that the effective time of bidding documents is the length that task distributes a time slot, when beyond effective time bid not yet
When person's competitive bidding, then former bidding documents failure, the range of transmission that bid person needs to expand bidding documents retransmit bidding documents.
4. more AUV collaboration data collection algorithms in UASNs according to claim 1 based on Q-learning, feature
It is, in the step 2, the bid quantity of bidder is limited to Pt, be according to itself state after bidder receives bidding documents
And the cost progress comprehensive assessment of task is completed, decide whether the competitive bidding for participating in the task；It to submit a tender after bidder's assessment
Quantity Mt < Pt when, bidding documents information can be continued to, otherwise will not be received again.
5. more AUV collaboration data collection algorithms in UASNs according to claim 1 based on Q-learning, feature
It is, in the step 3, method that AUV determines the access order of task cluster are as follows:
(3-1) cluster head in t time slot broadcasts a beacon, includes cluster head number V in beaconk, cluster head coordinate Ck(t), itself is maximum
Communication radius CRk, the moving range MR of packet information rank DI, node kkInformation, the message level of data packet is divided into a, b,
C three classes, wherein a class is emergence message, needs quickly to handle；
(3-2) if not receiving the beacon from cluster head, the contact probability of AUV and cluster is 0, if AUV receives the beacon of cluster head,
The contact probability for calculating the cluster head of beacon received is
Pt(k, Ai)=pen (dt(Vk,Ai),CRk,MRk, DI)
Wherein dt(Vk,Ai) indicate the distance between leader cluster node and AUV,
Wherein (Ax, Ay, Az) indicate t moment AUV position coordinates, (Vx, Vy, Vz) indicate t moment leader cluster node position coordinates；
After (3-3) AUV obtains the distance between cluster head of each task cluster, compare the size with each cluster contact probability, selects
The maximum cluster head of contact probability is selected as the next target point of oneself；The packet information rank in beacon is paid the utmost attention to, it is excellent
The high data of levels of information are first collected, if levels of information is consistent, carry out data collection according to the size of contact probability；
After (3-4) determines next target cluster of AUV through the above steps, advised using the path that Q-learning carries out part
It draws, updates Q-table, select the movement of Income Maximum；
(3-5) after AUV reaches target cluster, AUV sends acknowledgement frame, the position of cluster, AUV where notifying other clusters AUV at present
Coordinate and it is expected that the residence time；Due to the influence of water flow, the position of node can change, after being collected to AUV, each cluster
Cluster head retransmit beacon to AUV, after AUV receives beacon, repeat step (3-2), (3-3), (3-4).
6. more AUV collaboration data collection algorithms in UASNs according to claim 5 based on Q-learning, feature
It is, in the step (3-4), in Q-table renewal process, the setting method of reward value is according to AUV previous state and to work as
Preceding state is set at a distance from target cluster, and it is as follows to update rule:
Wherein, dt(Vk,Aj) indicate the distance between t moment AUV and target cluster head, dt-1(Vk,Aj) indicate t-1 moment AUV and mesh
Mark the distance between cluster head.
7. more AUV collaboration data collection algorithms in UASNs according to claim 5 based on Q-learning, feature
It is, in the step (3-4), selects the movement of Income Maximum as follows:
Setting AUV movement be selected as centered on AUV, by the actual act spatial model A of AUV be defined as upper and lower, left and right,
Forward and backward 6 discrete movements respectively represent AUV floating in water, dive, left-hand rotation, right-hand rotation, advance, retreat several movement shapes
State.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910670534.8A CN110430547B (en) | 2019-07-24 | 2019-07-24 | Q-learning-based multi-AUV cooperative data collection method in UASNs |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910670534.8A CN110430547B (en) | 2019-07-24 | 2019-07-24 | Q-learning-based multi-AUV cooperative data collection method in UASNs |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110430547A true CN110430547A (en) | 2019-11-08 |
| CN110430547B CN110430547B (en) | 2022-07-15 |
Family
ID=68410598
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910670534.8A Active CN110430547B (en) | 2019-07-24 | 2019-07-24 | Q-learning-based multi-AUV cooperative data collection method in UASNs |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110430547B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111542020A (en) * | 2020-05-06 | 2020-08-14 | 河海大学常州校区 | Multi-AUV cooperative data collection method based on region division in underwater acoustic sensor network |
| CN111541494A (en) * | 2020-06-15 | 2020-08-14 | 河海大学常州校区 | Location privacy protection method based on clustering structure in underwater acoustic sensor network |
| CN111612162A (en) * | 2020-06-02 | 2020-09-01 | 中国人民解放军军事科学院国防科技创新研究院 | Reinforced learning method and device, electronic equipment and storage medium |
| CN112866911A (en) * | 2021-01-11 | 2021-05-28 | 燕山大学 | Underwater data collection method assisted by autonomous underwater vehicle based on Q learning |
| CN114282645A (en) * | 2021-11-24 | 2022-04-05 | 杭州电子科技大学 | DQN-based space-time crowdsourcing task allocation method |
| CN115361744A (en) * | 2022-08-10 | 2022-11-18 | 广西财经学院 | UWSNs medium access control method suitable for AUV |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101815326A (en) * | 2010-01-11 | 2010-08-25 | 北京邮电大学 | Method for allocating tasks in wireless sensor network based on negotiation |
| US8107397B1 (en) * | 2006-06-05 | 2012-01-31 | Purdue Research Foundation | Protocol for secure and energy-efficient reprogramming of wireless multi-hop sensor networks |
| CN105828287A (en) * | 2016-03-11 | 2016-08-03 | 南京航空航天大学 | Reinforcement learning collaborative tracking algorithm (RLTCA) of wireless sensor network |
| CN107708089A (en) * | 2017-10-30 | 2018-02-16 | 吴豪 | Data forwarding method and data forwarding device based on sub-clustering |
| CN108684005A (en) * | 2018-04-02 | 2018-10-19 | 河海大学常州校区 | More AUV efficient data collections methods in underwater sensing net based on SOM |
| CN109275099A (en) * | 2018-09-26 | 2019-01-25 | 河海大学常州校区 | More AUV efficient data collection methods in underwater wireless sensor network based on VOI |
| CN109286961A (en) * | 2018-09-27 | 2019-01-29 | 天津大学 | The energy-optimised routing resource of underwater sensor network based on machine learning |
| CN109540151A (en) * | 2018-03-25 | 2019-03-29 | 哈尔滨工程大学 | A kind of AUV three-dimensional path planning method based on intensified learning |
-
2019
- 2019-07-24 CN CN201910670534.8A patent/CN110430547B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8107397B1 (en) * | 2006-06-05 | 2012-01-31 | Purdue Research Foundation | Protocol for secure and energy-efficient reprogramming of wireless multi-hop sensor networks |
| CN101815326A (en) * | 2010-01-11 | 2010-08-25 | 北京邮电大学 | Method for allocating tasks in wireless sensor network based on negotiation |
| CN105828287A (en) * | 2016-03-11 | 2016-08-03 | 南京航空航天大学 | Reinforcement learning collaborative tracking algorithm (RLTCA) of wireless sensor network |
| CN107708089A (en) * | 2017-10-30 | 2018-02-16 | 吴豪 | Data forwarding method and data forwarding device based on sub-clustering |
| CN109540151A (en) * | 2018-03-25 | 2019-03-29 | 哈尔滨工程大学 | A kind of AUV three-dimensional path planning method based on intensified learning |
| CN108684005A (en) * | 2018-04-02 | 2018-10-19 | 河海大学常州校区 | More AUV efficient data collections methods in underwater sensing net based on SOM |
| CN109275099A (en) * | 2018-09-26 | 2019-01-25 | 河海大学常州校区 | More AUV efficient data collection methods in underwater wireless sensor network based on VOI |
| CN109286961A (en) * | 2018-09-27 | 2019-01-29 | 天津大学 | The energy-optimised routing resource of underwater sensor network based on machine learning |
Non-Patent Citations (4)
| Title |
|---|
| GUANGJIE HAN; ZHENGKAI TANG: "District Partition-Based Data Collection Algorithm With Event Dynamic Competition in Underwater Acoustic Sensor Networks", 《IEEE》 * |
| YANG YANG; XUE-SONG QIU: "A self-adaptive method of task allocation in clustering-based MANETs", 《IEEE》 * |
| 先兴平: "博弈论在无线传感器网络路由中的应用研究", 《信息科技辑》 * |
| 栾筱林: "多自主式水下航行器协同控制的人工物理法研究", 《中国海洋大学学报(自然科学版)》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111542020A (en) * | 2020-05-06 | 2020-08-14 | 河海大学常州校区 | Multi-AUV cooperative data collection method based on region division in underwater acoustic sensor network |
| CN111612162A (en) * | 2020-06-02 | 2020-09-01 | 中国人民解放军军事科学院国防科技创新研究院 | Reinforced learning method and device, electronic equipment and storage medium |
| CN111612162B (en) * | 2020-06-02 | 2021-08-27 | 中国人民解放军军事科学院国防科技创新研究院 | Reinforced learning method and device, electronic equipment and storage medium |
| CN111541494A (en) * | 2020-06-15 | 2020-08-14 | 河海大学常州校区 | Location privacy protection method based on clustering structure in underwater acoustic sensor network |
| CN112866911A (en) * | 2021-01-11 | 2021-05-28 | 燕山大学 | Underwater data collection method assisted by autonomous underwater vehicle based on Q learning |
| CN112866911B (en) * | 2021-01-11 | 2022-02-25 | 燕山大学 | Underwater data collection method assisted by autonomous underwater vehicle based on Q learning |
| CN114282645A (en) * | 2021-11-24 | 2022-04-05 | 杭州电子科技大学 | DQN-based space-time crowdsourcing task allocation method |
| CN114282645B (en) * | 2021-11-24 | 2023-04-21 | 杭州电子科技大学 | Space-time crowdsourcing task allocation method based on DQN |
| CN115361744A (en) * | 2022-08-10 | 2022-11-18 | 广西财经学院 | UWSNs medium access control method suitable for AUV |
| CN115361744B (en) * | 2022-08-10 | 2024-04-23 | 广西财经学院 | UWSNs medium access control method suitable for AUV |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110430547B (en) | 2022-07-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110430547A (en) | More AUV collaboration data collection algorithms in UASNs based on Q-learning | |
| Liu et al. | Distributed and energy-efficient mobile crowdsensing with charging stations by deep reinforcement learning | |
| Lin et al. | TADP: Enabling temporal and distantial priority scheduling for on-demand charging architecture in wireless rechargeable sensor networks | |
| Cao et al. | Swarm intelligence-based performance optimization for mobile wireless sensor networks: survey, challenges, and future directions | |
| CN100583900C (en) | Radio sensor network data convergence path planning method based on the intelligent agent | |
| CN112738752B (en) | WRSN multi-mobile charger optimal scheduling method based on reinforcement learning | |
| CN100505763C (en) | Wireless sensor network distributed cooperated information processing method | |
| CN107831685B (en) | Group robot control method and system | |
| CN109982452B (en) | Quasi-array-based wireless chargeable sensor network charging scheduling method | |
| Lin et al. | Maximizing energy efficiency of period-area coverage with a UAV for wireless rechargeable sensor networks | |
| CN109886509A (en) | A kind of mobile poly concourse nodes paths planning method, system and electronic equipment | |
| CN102014344A (en) | Clustering control method of intelligent wireless sensor network based on DPSO (Discrete Particle Swarm Optimization) | |
| CN109451556A (en) | The method to be charged based on UAV to wireless sense network | |
| Lin et al. | Hybrid charging scheduling schemes for three-dimensional underwater wireless rechargeable sensor networks | |
| Cheikhrouhou et al. | Move and improve: A distributed multi-robot coordination approach for multiple depots multiple travelling salesmen problem | |
| CN103702384B (en) | Application-oriented clustering routing method of wireless sensor network | |
| Guo et al. | Distributed task allocation algorithm based on connected dominating set for WSANs | |
| Dong et al. | Instant on-demand charging strategy with multiple chargers in wireless rechargeable sensor networks | |
| Zhao et al. | Dronesense: Leveraging drones for sustainable urban-scale sensing of open parking spaces | |
| Srinivas et al. | Delay-tolerant charging scheduling by multiple mobile chargers in wireless sensor network using hybrid GSFO | |
| CN105636081A (en) | Improved EAST (Efficient Data Collection Aware of Spatio-Temporal Correlation) algorithm based on grey model prediction | |
| Wei et al. | A novel on-demand charging strategy based on swarm reinforcement learning in WRSNs | |
| Azizi et al. | A novel energy aware node clustering algorithm for wireless sensor networks using a modified artificial fish swarm algorithm | |
| Kumar et al. | Area exploration by flocking of multi robot | |
| Ou et al. | Modeling wireless sensor network coverage based on random radius |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |