CN110602167B - Distributed data storage system under wireless ad hoc network environment - Google Patents
Distributed data storage system under wireless ad hoc network environment Download PDFInfo
- Publication number
- CN110602167B CN110602167B CN201910736516.5A CN201910736516A CN110602167B CN 110602167 B CN110602167 B CN 110602167B CN 201910736516 A CN201910736516 A CN 201910736516A CN 110602167 B CN110602167 B CN 110602167B
- Authority
- CN
- China
- Prior art keywords
- node
- marginal
- storage
- cluster
- storage node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
- H04L41/0668—Management of faults, events, alarms or notifications using network fault recovery by dynamic selection of recovery network elements, e.g. replacement by the most appropriate element after failure
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/04—Arrangements for maintaining operational condition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W84/00—Network topologies
- H04W84/18—Self-organising networks, e.g. ad-hoc networks or sensor networks
Abstract
The invention discloses a distributed data storage system in a wireless ad hoc network environment, which comprises a clustering scheme, an election scheme and a downtime recovery scheme and is suitable for a small wireless ad hoc network. The storage nodes are divided into a plurality of clusters, and the best performance index G is selected from each cluster as a marginal node of the cluster; the clusters communicate through the marginal nodes; all storage nodes in the cluster back up self storage data to the marginal node of the cluster; each marginal node maintains position distribution information of the data of the whole network; defining a feature vector C of a storage node; each storage node maintains the feature vector matching degree D of the storage node and the marginal node and the performance index G of the storage node; when a new storage node is accessed, judging the feature vector matching degree D of the new storage node and the marginal node of each cluster, and adding the new storage node into the cluster with the maximum matching degree; and when the marginal node in one cluster is down, selecting the storage node with the maximum comprehensive value of the feature vector matching degree D and the performance index G as a new marginal node.
Description
Technical Field
The invention relates to the field of data storage, in particular to a distributed data storage system in a wireless ad hoc network environment.
Background
In recent years, the popularity of machine learning is continuously rising, and the classification algorithms also include various classification algorithms such as a decision tree classification method, a naive Bayes classification algorithm, a knowledge vector machine-based classifier, a neural network method, a k-nearest neighbor method, a fuzzy classification method and the like; however, these methods require a large number of training data sets and high-performance computation of the machine, and are therefore not suitable for use in small-sized wireless ad hoc networks.
There are many election algorithms in distributed systems: paxos election algorithm, Raft election algorithm, Zookeeper election algorithm, motorway election algorithm, etc. But Paxos is named as being difficult to understand and difficult to realize; zookeeper is extremely sensitive to network isolation, resulting in the Zookeeper reacting violently to any wind-blown grass on the network, which makes the Zookeeper unusable for a relatively long time. The ideas of the Raft and the extra road election algorithm are not matched with the existing wireless ad hoc network.
One problem that any system architecture has to consider is the problem of downtime recovery. The Paxos election algorithm, the Raft election algorithm, the Zookeeper election algorithm, the extra election algorithm and the like mentioned in the election algorithms all face the condition that the leader node is down, and have corresponding down coping strategies. But since election algorithms are not suitable for distributed ad-hoc architectures, the present invention does not employ recovery strategies that are too complex for them.
In summary, in order to implement a distributed data storage system in a wireless ad hoc network, relevant influencing factors must be considered comprehensively in the design of a clustering mechanism, an election algorithm and a quick recovery after downtime, so as to ensure that the system can operate efficiently and reliably.
Disclosure of Invention
In view of this, the present invention provides a distributed data storage system in a wireless ad hoc network environment, which includes a clustering scheme, an election scheme, and a downtime recovery scheme, and is suitable for a small wireless ad hoc network.
In order to solve the technical problem, the invention is realized as follows:
a distributed data storage system under the wireless self-organizing network environment, the storage node in the data storage system is based on the wireless self-organizing network protocol networking; the method comprises the following steps:
the storage nodes are divided into a plurality of clusters, and the best performance index G is selected from each cluster as a marginal node of the cluster; the clusters communicate through the marginal nodes; all storage nodes in the cluster back up self storage data to the marginal nodes of the cluster, and the data is acquired from the marginal nodes of the cluster when the data needs to be acquired, and if the data is not acquired from the marginal nodes of other clusters; each marginal node maintains position distribution information of the data of the whole network;
defining a feature vector C (C) of a storage node1,c2…cn) The feature vector consists of n features; each storage node maintains the feature vector matching degree D of the storage node and the marginal node and the performance index G of the storage node;
when a new storage node is accessed, judging the feature vector matching degree D of the new storage node and the marginal node of each cluster, and adding the new storage node into the cluster with the maximum matching degree;
and when the marginal node in one cluster is down, selecting the storage node with the maximum comprehensive value of the feature vector matching degree D and the performance index G as a new marginal node.
Preferably, during initial clustering, the storage nodes are partitioned according to the matching degree of the feature vectors, and the storage nodes with similar matching degree are partitioned into a cluster.
Preferably, during initial clustering, the storage node with the largest performance index G is elected as the marginal node.
Preferably, the performance index G is achieved by computing a weighted sum over a plurality of performance parameters; the performance parameters comprise the residual cache space of the storage node, the CPU frequency, the number of computing cores of the storage node equipment and the memory capacity, and the performance parameters all reflect that the larger the performance of the node is, the better the node is.
Preferably, the integrated value of the feature vector matching degree D and the performance index G refers to a weighted sum of D and G.
Preferably, when the data is backed up to the marginal node, the data is further backed up to the marginal node of the selected other cluster according to the set backup rate.
Has the advantages that:
under the wireless ad hoc network environment, each node can be rapidly added into a correct cluster through a clustering strategy based on the feature vector matching degree D; each cluster can quickly select a marginal node as a main node of the cluster for saving the state of the whole network through an election algorithm based on the performance index G, and can realize access among the clusters through the main node. If the marginal node of a certain cluster is down, the cluster can be quickly restored to a normal state through a quick restoration algorithm based on the feature vector matching degree D and the performance index G. The scheme of the invention not only can realize the distributed storage of the data, but also can realize the quick request and response of the data, and has good fault tolerance.
Drawings
FIG. 1 is a diagram of a distributed data storage system in a wireless ad hoc network environment according to the present invention.
Detailed Description
The invention is described in detail below by way of example with reference to the accompanying drawings.
The invention provides a distributed data storage system in a wireless ad hoc network environment, as shown in fig. 1, the data storage system comprises a plurality of storage nodes. The storage nodes are networked based on a wireless self-organizing network protocol, and the storage nodes are communicated through a wireless network. The storage nodes are divided into a plurality of clusters, and one storage node is selected from each cluster to serve as a marginal node. The storage nodes in the cluster communicate through full connection or through marginal routing. And the clusters are communicated through the marginal nodes, so that the network condition is updated in real time.
The invention customizes a clustering strategy, an election algorithm, a downtime recovery strategy and a data backup scheme for the system.
(1) Clustering strategy
In order to achieve a correct and efficient classification effect, the invention provides a clustering decision mechanism applied to a distributed data storage system, and the nodes to be classified can be quickly classified under the conditions of the small wireless ad hoc network and limited node computing capacity. The method specifically comprises the following steps:
the invention defines a feature vector C (C) for each storage node1,c2…cn) The feature vector consists of n features. The node functions can be distinguished by the characteristics. Examples may include: temperature sensor characteristics, image acquisition characteristics, voiceprint acquisition characteristics, electromagnetic pulse perception characteristics. If the function is included, the corresponding characteristic value is 1, and if the function is not included, the corresponding characteristic value is 0. E.g., C (1, 0, 0, 1): indicating that the node has the functions of a temperature sensor and sensing electromagnetic pulses. So that nodes of the same or similar functionality will be grouped together.
When clustering is initially performed, preferably, a feature vector of each storage node is calculated, and partitioning is performed according to the feature vector matching degree D, and storage nodes with similar matching degrees are partitioned into a cluster. Whether the matching degrees D are close or not can be realized by calculating the Hamming distance of the two vectors. In practice, the initial clustering may also be determined in a specified manner.
And when a new storage node is accessed, judging the matching degree D of the new storage node and the feature vector of the marginal node of each cluster, and adding the new storage node into the cluster with the highest matching degree.
Meanwhile, each storage node also maintains the matching degree D of the feature vectors of the storage node and the marginal node, and preparation is made for later recovery.
(2) Election algorithm
The invention provides a proper election algorithm in consideration of the existing environment, and marginal nodes can be rapidly selected in respective clusters to serve as main nodes. Other nodes in each cluster can request data from the main node in the cluster, thereby achieving the effect of quick request response and reducing delay. The main nodes of each cluster are communicated with each other and used for caching the network condition of the wireless ad hoc network.
A performance index G is defined for each storage node. The performance index reflects the storage and computing performance of the storage node. The performance index G may be implemented by computing a weighted sum of a plurality of performance parameters.
In a preferred embodiment, a performance indicator function g (n) of a node is defined as P · W:
wherein: p (Rcs, CPUp, Kn; RAM)Evaluating performance parameters of the node performance; w (W)r,wc,wk,wram) Representing the weight occupied by each parameter.
Rcs is a Remaining cache space (Remaining cache space), and the larger the space, the better the performance;
CPUp is CPU frequency (CPU frequency), the higher the frequency, the better the performance;
kn is the number of the computing kernels (Kernel number) of the node equipment, and the more kernels, the better performance;
the RAM is a memory device, and the larger the memory, the better the performance.
The weight occupied by each parameter can be determined according to the specific environment applied by the system, for example, the requirement on the storage amount is higher, the larger the weight of Rcs is, the better the weight is; if the requirement on the calculation speed is higher, CPUp and Kn are carried out; the larger the weight of the RAM, the better.
And during election, calculating the performance value G of each storage node according to the performance function, wherein one node with the highest selective performance value is used as a marginal node of the cluster.
Meanwhile, each storage node also maintains the performance index value G of the storage node for later recovery.
(3) Downtime recovery strategy
The invention establishes a 'quick recovery algorithm' suitable for a small wireless ad hoc network, so that when the marginal node of a certain cluster goes down due to some external reasons, the cluster can quickly select a new marginal node in the cluster by using the 'quick recovery algorithm', and the normal operation of the network of the cluster is maintained.
As mentioned above, each storage node maintains the respective performance index value G and the feature vector matching degree D of the storage node and the marginal node. When a marginal node in a cluster is down, the storage node with the largest comprehensive value of the feature vector matching degree D and the performance index G is elected as a new marginal node.
The invention requires that the performance of the new marginal node is better, and the new marginal node is considered to be more similar to the original marginal node, so that the new marginal node can take over the task of the original marginal node more smoothly.
In calculating the integrated value of D and G, it can be realized by calculating a weighted sum of D and G. The participation degree of D and G can be flexibly controlled by adjusting the weight.
The new boundary node that is pushed out is obtained from other marginal results.
(4) Data backup scheme
When the storage node collects or obtains a new piece of data from the outside, the data is stored to the storage node, and meanwhile, the data is backed up to the marginal node in the cluster. Therefore, the marginal nodes can store the data of all the nodes in the cluster, and when the nodes need to read the stored data, the data can be directly read from the marginal nodes. If the marginal nodes in the cluster do not exist, each marginal node maintains the position distribution information of the data of the whole network, so that the data can be acquired from other nodes through inter-cluster communication.
In order to ensure the data security, the invention adopts multiple backups, when backing up the data to the marginal node, the data is further backed up to the marginal node of other selected clusters according to the set backup rate. Even if one cluster of equipment is down, data can be recovered from other clusters. When the marginal node needing to be backed up is selected according to the backup rate, the marginal node with large residual storage space of the marginal node is set to be preferentially selected as the backup node.
In summary, the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (7)
1. A distributed data storage system under the wireless self-organizing network environment, the storage node in the data storage system is based on the wireless self-organizing network protocol networking; it is characterized in that the preparation method is characterized in that,
the storage nodes are divided into a plurality of clusters, and the best performance index G is selected from each cluster as a marginal node of the cluster; the clusters communicate through the marginal nodes; all storage nodes in the cluster back up self storage data to the marginal nodes of the cluster, and the data is acquired from the marginal nodes of the cluster when the data needs to be acquired, and if the data is not acquired from the marginal nodes of other clusters; each marginal node maintains position distribution information of the data of the whole network;
defining a feature vector C (C) of a storage node1,c2…cn) The feature vector consists of n features, and the functions of the storage nodes can be distinguished through the features; each storage node maintains the feature vector matching degree D of the storage node and the marginal node and the performance index G of the storage node;
when a new storage node is accessed, judging the feature vector matching degree D of the new storage node and the marginal node of each cluster, and adding the new storage node into the cluster with the maximum matching degree;
and when the marginal node in one cluster is down, selecting the storage node with the maximum comprehensive value of the feature vector matching degree D and the performance index G as a new marginal node.
2. The system of claim 1, wherein in the initial clustering, the division is performed according to the matching degree of the feature vectors, and the storage nodes with similar matching degree are divided into a cluster.
3. The system of claim 1, wherein the storage node with the largest performance metric G is elected as the marginal node at the time of initial clustering.
4. The system of claim 1, wherein the performance metric G is achieved by computing a weighted sum over a plurality of performance parameters; the performance parameters comprise the residual cache space of the storage node, the CPU frequency, the number of computing cores of the storage node equipment and the memory capacity, and the performance parameters all reflect that the larger the performance of the node is, the better the node is.
5. The system of claim 1, wherein the integrated value of the eigenvector matching metric D and the performance metric G is a weighted sum of D and G.
6. The system of claim 1, wherein when backing up data to the marginal node, the data is further backed up to the marginal node of the selected other cluster according to a set backup rate.
7. The system of claim 1, wherein the feature vector expresses functions that a storage node can provide, one function for each feature; the corresponding characteristic value of a function included in the storage node is 1, and the corresponding characteristic value of a function not included is 0.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910736516.5A CN110602167B (en) | 2019-08-09 | 2019-08-09 | Distributed data storage system under wireless ad hoc network environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910736516.5A CN110602167B (en) | 2019-08-09 | 2019-08-09 | Distributed data storage system under wireless ad hoc network environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110602167A CN110602167A (en) | 2019-12-20 |
| CN110602167B true CN110602167B (en) | 2021-11-05 |
Family
ID=68854035
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910736516.5A Active CN110602167B (en) | 2019-08-09 | 2019-08-09 | Distributed data storage system under wireless ad hoc network environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110602167B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113642697B (en) * | 2020-04-27 | 2024-02-20 | 郑州芯兰德网络科技有限公司 | Distributed multi-level graph network training method and system |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101207572A (en) * | 2007-12-14 | 2008-06-25 | 北京科技大学 | Method for clustering vehicle mounted Ad hoc network based on signal strength |
| EP2120181A1 (en) * | 2008-05-15 | 2009-11-18 | Deutsche Telekom AG | A sensor node, a sensor network and a method for autonomous decision-making in sensor networks |
| CN101896002A (en) * | 2010-06-26 | 2010-11-24 | 上海交通大学 | Rapid data aggregation-oriented wireless sensor and actor network clustering method |
| CN102970722A (en) * | 2012-12-10 | 2013-03-13 | 北京科技大学 | Multicasting route algorithm of low-time-delay delay tolerant and disruption tolerant sensor network |
| CN103415054A (en) * | 2013-09-03 | 2013-11-27 | 中南大学 | Hierarchical routing algorithm based on distributed type wireless aggregation network |
| CN103491590A (en) * | 2013-09-26 | 2014-01-01 | 常熟理工学院 | Method for achieving next-generation wireless sensor network route |
| CN104581862A (en) * | 2014-12-27 | 2015-04-29 | 中国人民解放军63655部队 | Measurement and control communication method and system based on low-altitude unmanned aerial vehicle self-network |
| CN107426794A (en) * | 2017-03-17 | 2017-12-01 | 兰州交通大学 | A kind of combined weighted cluster head choosing method suitable for VANET networks |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11544288B2 (en) * | 2010-12-23 | 2023-01-03 | Mongodb, Inc. | Systems and methods for managing distributed database deployments |
-
2019
- 2019-08-09 CN CN201910736516.5A patent/CN110602167B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101207572A (en) * | 2007-12-14 | 2008-06-25 | 北京科技大学 | Method for clustering vehicle mounted Ad hoc network based on signal strength |
| EP2120181A1 (en) * | 2008-05-15 | 2009-11-18 | Deutsche Telekom AG | A sensor node, a sensor network and a method for autonomous decision-making in sensor networks |
| CN101896002A (en) * | 2010-06-26 | 2010-11-24 | 上海交通大学 | Rapid data aggregation-oriented wireless sensor and actor network clustering method |
| CN102970722A (en) * | 2012-12-10 | 2013-03-13 | 北京科技大学 | Multicasting route algorithm of low-time-delay delay tolerant and disruption tolerant sensor network |
| CN103415054A (en) * | 2013-09-03 | 2013-11-27 | 中南大学 | Hierarchical routing algorithm based on distributed type wireless aggregation network |
| CN103491590A (en) * | 2013-09-26 | 2014-01-01 | 常熟理工学院 | Method for achieving next-generation wireless sensor network route |
| CN104581862A (en) * | 2014-12-27 | 2015-04-29 | 中国人民解放军63655部队 | Measurement and control communication method and system based on low-altitude unmanned aerial vehicle self-network |
| CN107426794A (en) * | 2017-03-17 | 2017-12-01 | 兰州交通大学 | A kind of combined weighted cluster head choosing method suitable for VANET networks |
Non-Patent Citations (2)
| Title |
|---|
| An Improved Cluster Maintenance Scheme for Mobile AdHoc Networks;Sunil Pathak;《2014 International Conference on Advances in Computing, Communicaitons and Informatics》;20141201;全文 * |
| 无线自组网的分簇算法研究;常来花;《中国优秀硕士学位论文全文数据库信息科技辑》;20180515;第I136-152页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110602167A (en) | 2019-12-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | Variational few-shot learning | |
| Isaksson et al. | SOStream: Self organizing density-based clustering over data stream | |
| CN109859054B (en) | Network community mining method and device, computer equipment and storage medium | |
| Meyer et al. | Information-theoretic inference of large transcriptional regulatory networks | |
| Mulvey et al. | Cell fault management using machine learning techniques | |
| CN110389866A (en) | Disk failure prediction technique, device, computer equipment and computer storage medium | |
| CN115358487A (en) | Federal learning aggregation optimization system and method for power data sharing | |
| CN110602167B (en) | Distributed data storage system under wireless ad hoc network environment | |
| WO2021084623A1 (en) | Degradation suppression program, degradation suppression method, and information processing device | |
| CN108289285A (en) | A kind of ocean wireless sensor network is lost data and is restored and reconstructing method | |
| Pereira et al. | Transfer learning for Bayesian networks with application on hard disk drives failure prediction | |
| CN117241295B (en) | Wireless communication network performance optimization method, device and storage medium | |
| Titouna et al. | Outlier detection algorithm based on mahalanobis distance for wireless sensor networks | |
| EP4193304A1 (en) | Normalization in deep convolutional neural networks | |
| CN115277249B (en) | Network security situation perception method based on cooperation of multi-layer heterogeneous network | |
| CN114675789B (en) | Big data analysis and storage system and method based on computer system | |
| Qin et al. | Adaptive in-network collaborative caching for enhanced ensemble deep learning at edge | |
| Bhardwaj et al. | Pecc: Prediction-error correcting cache | |
| CN115545106A (en) | AoI sensitive data collection method and system in multiple unmanned aerial vehicles | |
| CN117376084A (en) | Fault detection method, electronic equipment and medium thereof | |
| KR101617074B1 (en) | Method and Apparatus for Context-aware Recommendation to Distribute Water in Smart Water Grid | |
| Estiri et al. | Attentive federated learning for concept drift in distributed 5g edge networks | |
| de Melo et al. | Cost-sensitive measures of algorithm similarity for meta-learning | |
| CN107316056B (en) | Automatic evaluation system and automatic evaluation method for network security level | |
| JP2021536047A (en) | Autonomous Hardware Computation Resilience Systems and Methods |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |