CN111064641B - Node performance detection system and method for decentralized storage network - Google Patents

Node performance detection system and method for decentralized storage network Download PDF

Info

Publication number
CN111064641B
CN111064641B CN201911408673.XA CN201911408673A CN111064641B CN 111064641 B CN111064641 B CN 111064641B CN 201911408673 A CN201911408673 A CN 201911408673A CN 111064641 B CN111064641 B CN 111064641B
Authority
CN
China
Prior art keywords
node
nodes
detection
point
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911408673.XA
Other languages
Chinese (zh)
Other versions
CN111064641A (en
Inventor
李昕
李涛
段云瑞
达宇
赵李波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Kunyao Network Technology Co ltd
Original Assignee
Shanghai Kunyao Network Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Kunyao Network Technology Co ltd filed Critical Shanghai Kunyao Network Technology Co ltd
Priority to CN201911408673.XA priority Critical patent/CN111064641B/en
Publication of CN111064641A publication Critical patent/CN111064641A/en
Application granted granted Critical
Publication of CN111064641B publication Critical patent/CN111064641B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
    • H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L61/00—Network arrangements, protocols or services for addressing or naming
    • H04L61/30—Managing network names, e.g. use of aliases or nicknames
    • H04L61/3005—Mechanisms for avoiding name conflicts
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/3236—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/3247—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving digital signatures

Abstract

The invention belongs to the technical field of distributed networks, and particularly relates to a node performance detection system and a node performance detection method for a decentralized storage network, wherein the system comprises the following steps: a number of peer-to-peer network nodes; the network nodes are connected through a point-to-point network; the node comprises: the system comprises a node detection unit, a random data access unit, a detection response unit, a data statistics recording unit and a control unit; the node detection unit is used for periodically and actively detecting the accessibility and the availability among the nodes in the point-to-point decentralized storage network; the random data access unit is used for detecting the storage access performance among the nodes in the point-to-point decentralized storage network, thereby realizing the distributed detection and updating of the node performance.

Description

Node performance detection system and method for decentralized storage network
Technical Field
The invention belongs to the technical field of distributed networks, and particularly relates to a node performance detection system and method for a decentralized storage network.
Background
The distributed storage technology is a main technology adopted by current cloud storage, and the distributed storage technology stores data into a plurality of nodes by using a network formed by a plurality of nodes so as to realize redundancy and backup of the data, eliminate data loss caused by single-point failure and improve the access performance of a user by multi-node storage.
The traditional distributed storage technology mainly organizes nodes in a cluster mode, and performs unified management in a close-coupled mode of manageable nodes to provide services for users. The deployment, upgrading, detection and management of the service nodes are uniformly carried out, and the method is easy to realize and mature.
Decentralized storage networks are a recent development in the development of distributed storage. The storage and management mode and the centralized distributed storage are greatly different, and the key difference lies in that:
firstly, service nodes are more decentralized, the nodes are connected through a point-to-point network, and the nodes are managed in a centralized mode, so that the cost is high and the efficiency is low;
the addition and the quitting of the nodes are free, and are not controlled by centralized management, so that the system needs to be more flexible;
thirdly, the nodes are connected through the Internet, the network conditions of users are different, and the management of the user nodes and the optimization of performance detection are particularly important;
a decentralized storage network is still under development, and various technologies have been proposed at present, including: IPFS (InterPlanetary File System), BitTorrent, Sia, Swarm (an etherhouse-based decentralized File System). However, these technologies are all basic file systems built on peer-to-peer networks, and there is no complete node performance detection mechanism and application optimization based on the performance detection mechanism. The invention provides an efficient performance detection method for a decentralized storage network, which comprises the reachability, the availability, the service performance and the like of nodes.
Disclosure of Invention
In view of the above, the main objective of the present invention is to provide a node performance detection system and method for decentralized storage networks, which have the advantages of accurate result and high security.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
a node performance detection system for a decentralized storage network, the system comprising: a number of peer-to-peer network nodes; the network nodes are connected through a point-to-point network; the method is characterized in that: the node comprises: the system comprises a node detection unit, a random data access unit, a detection response unit, a data statistics recording unit and a control unit; the node detection unit is used for periodically and actively detecting the accessibility and the availability among the nodes in the point-to-point decentralized storage network; the random data access unit is used for detecting the storage access performance among the nodes in the point-to-point decentralized storage network; the detection response unit is used for receiving the detection message and responding in time; the data statistics recording unit: the system is used for carrying out data exchange and data processing between nodes, carrying out process recording and forming a statistical report of the node performance; the control unit is used for constructing a node ID, carrying out encryption signature on the detection data, providing the constructed node ID for other units in the system to use, and controlling the operation of other units in the system; the method for constructing the node ID comprises the following steps: generating a private key according to the anti-collision hash function and the random number generated by the random number generator; generating a public key corresponding to the private key based on an asymmetric cryptographic algorithm; and constructing the node ID according to the hash value and the check code corresponding to the public key.
Further, the node further includes a node connection unit, configured to:
enabling the node to periodically acquire node ID information of other nodes;
enabling the node to determine the ID contact ratio with other nodes, and keeping the ID information of the relevant nodes with the ID contact ratio larger than a preset threshold value;
determining the proximity of the node and each related node according to the ID information of the related node;
and enabling the nodes to sequentially select the relative nodes with the maximum proximity for connection until the maximum connection number of the nodes is reached.
Further, the node detection unit is specifically configured to:
randomly selecting a preset number of nodes to be detected from the nodes in the distributed hash table in the nodes;
and sequentially selecting the nodes to be detected with the maximum detectability from the nodes to be detected for detection.
Further, the nodes transmit information among the nodes through point-to-point network broadcasting and Gossip protocol.
Further, the nodes maintain the detection period and the node information together; storing the node interrelation and the node information through the distributed hash table; each node only needs to store part of the content of the distributed Hash table.
Correspondingly, the application also provides a node performance detection method of the decentralized storage network based on the system, and the method executes the following steps: a plurality of peer-to-peer network nodes are connected through a point-to-point network to form a network; the node comprises: the system comprises a node detection unit, a random data access unit, a detection response unit, a data statistics recording unit and a control unit; the node detection unit is used for periodically and actively detecting the accessibility and the availability among nodes in the point-to-point decentralized storage network; the random data access unit is used for detecting the storage access performance among nodes in the point-to-point decentralized storage network; the detection response unit is used for receiving the detection message and responding in time; a data statistics recording unit: carrying out data exchange and data processing between nodes, recording the process and forming a statistical report of the node performance; the control unit is used for constructing a node ID, carrying out encryption signature on the detection data, providing the constructed node ID for other units in the system to use, and controlling the operation of other units in the system, wherein the construction method of the node ID comprises the following steps: generating a private key according to the anti-collision hash function and the random number generated by the random number generator; generating a public key corresponding to the private key based on an asymmetric cryptographic algorithm; and constructing the node ID according to the hash value and the check code corresponding to the public key.
Further, after constructing the node ID, the method further includes:
the node periodically acquires node ID information of other nodes;
the node determines the ID contact ratio with other nodes, and retains the ID information of the relevant nodes with the ID contact ratio larger than a preset threshold value;
determining the proximity of the node and each related node according to the ID information of the related node;
and the nodes sequentially select the relative nodes with the maximum proximity for connection until the maximum connection number of the nodes is reached.
Further, the node detection unit performs periodic active detection of reachability and availability among nodes in a point-to-point decentralized storage network, specifically:
randomly selecting a preset number of nodes to be detected from the nodes in the distributed hash table in the nodes;
and sequentially selecting the nodes to be detected with the maximum detectability from the nodes to be detected for detection.
Further, the nodes transmit information among the nodes through point-to-point network broadcasting and Gossip protocol.
Further, the nodes maintain the detection period and the node information together; storing the node interrelation and the node information through the distributed hash table; each node only needs to store part of the content of the distributed Hash table.
The invention discloses a node performance detection system and method of a decentralized storage network, which have the following beneficial effects: the invention provides a node performance detection system and method of a decentralized storage network, which are used for detecting the accessibility and the availability of nodes and the related storage performance, realizing the distributed detection and the update of the node performance, providing the data information of node service for the upper-layer application of decentralized storage, optimizing the distributed storage application and improving the performance of the decentralized storage network.
Drawings
Fig. 1 is a schematic system structure diagram of a node performance detection system of a decentralized storage network according to an embodiment of the present invention;
fig. 2 is a schematic method flow diagram of a node performance detection method of a decentralized storage network according to an embodiment of the present invention.
Detailed Description
The method of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments of the invention.
Example 1
A node performance detection system for a decentralized storage network, the system comprising: a number of peer-to-peer network nodes; the network nodes are connected through a point-to-point network; the node comprises: the system comprises a node detection unit, a random data access unit, a detection response unit, a data statistics recording unit and a control unit; the node detection unit is used for periodically and actively detecting the accessibility and the availability among the nodes in the point-to-point decentralized storage network; the random data access unit is used for detecting the storage access performance among the nodes in the point-to-point decentralized storage network; the detection response unit is used for receiving the detection message and responding in time; the data statistics recording unit: the system is used for carrying out data exchange and data processing between nodes, carrying out process recording and forming a statistical report of the node performance; the control unit is used for constructing a node ID, carrying out encryption signature on the detection data, providing the constructed node ID for other units in the system to use, and controlling the operation of other units in the system; the method for constructing the node ID comprises the following steps: generating a private key according to the anti-collision hash function and the random number generated by the random number generator; generating a public key corresponding to the private key based on an asymmetric cryptographic algorithm; and constructing the node ID according to the hash value and the check code corresponding to the public key.
In a specific application scenario, the method for constructing the node ID may further be implemented by performing the following steps: randomly generating a sequence X through a pseudo-random algorithm, and designing a filter according to an autocorrelation function of the given sequence X and the Gaussian white noise power spectral density; generating 3 power spectral densities N with the same length as the sequence X to be issued0The gaussian white noise sequence of (1); 3 Gaussian white noise sequences pass through a filter to obtain 3 correlation Gaussian noise sequences with autocorrelation functions; splicing the 3 Gaussian white noise sequences to form a final sequence, wherein the final sequence is used as a node ID; the step of cryptographically signing the detection data performs the steps of: and extracting two sequences from the 3 white gaussian noise sequences, taking one of the two extracted sequences as a first key and the other as a second key, taking the first key as a parameter A and the second key as a parameter B, and constructing an encryption matrix:
Figure BDA0002349382870000061
recording a known encryption factor
Figure BDA0002349382870000062
Combining the encryption matrix and the encryption factor
Figure BDA0002349382870000063
Matrix multiplication to obtain an encrypted string matrix X:
Figure BDA0002349382870000064
and taking letters in the alphabet sequentially corresponding to the numerical values of the P parameter and the Q parameter in the X matrix as final encryption key values, forming a final encryption key by all the encryption key values, and encrypting the detection data by using the final encryption key.
The method for randomly generating a sequence X through a pseudo-random algorithm and designing a filter according to an autocorrelation function and a Gaussian white noise power spectral density of the given sequence X comprises the following steps: condition R of autocorrelation function satisfied by sequence X and Laplace noiseZ(m)＝RXX(m) substituting the Laplace noise generation formula to obtain an autocorrelation function of the correlated Gaussian noise as follows:
Figure BDA0002349382870000071
wherein R isXX(m) is the autocorrelation function of the sequence; fourier transform is carried out on the autocorrelation function of the correlation Gaussian noise, and the power spectral density of the correlation Gaussian noise is obtained as follows:
Figure BDA0002349382870000072
wherein δ (ω) is a pulse function; after gaussian noise passes through the existing system, the power spectral density is: pG′(ω)＝PG(ω)·|H(ω)|2(ii) a By the above formula, the system function of the filter is obtained as:
Figure BDA0002349382870000073
wherein N is0Is the known power spectral density of gaussian noise.
Specifically, the decentralized storage network is composed of a plurality of distributed peer nodes, and the nodes are connected through a point-to-point network to form a mesh network. In order to realize expandability, the nodes transmit information among the nodes through point-to-point network broadcasting and Gossip (Gossip) protocol. Each node constructs a node ID through a random number by a cryptographic algorithm, and the node ID collision is prevented by using the quasi-limitless space of the random number; and a pair of public and private keys is generated by this random number for signing the message. Each node stores the node interrelationship and the node information of the stored file content through a distributed Hash table.
The nodes in the network randomly detect and count each other to obtain the performance index of the node, and the node comprises: the node detection unit is used for periodically and actively detecting reachability and availability among nodes in the point-to-point decentralized storage network; the random data access unit is used for detecting the storage access performance among the nodes in the point-to-point decentralized storage network; the detection response unit is used for receiving the detection message and responding in time so as to achieve the purpose of truly reflecting the performance of the node, and the performance parameters of each node can directly influence the storage capacity and the income of the node, so that the node needs to respond in the fastest time; a data statistics recording unit: for exchanging and processing data among nodes, and finally recording and forming a statistical report of the node performance
A random detection protocol comprising: the link between peer nodes of the peer-to-peer network is determined by the relationship between the node IDs, and the node IDs are randomly generated, so that the peer link has no meaningful rule and has the largest coverage; the selection of the peer-to-peer detection nodes is randomly generated in each round, and the generation of random numbers is based on high-intensity cryptography, so that fairness and wide coverage are realized; meanwhile, the number of peer detection can be effectively reduced; the peer-to-peer detection result is automatically transmitted by the nodes through a publish-subscribe mode by a network Gossip protocol so as to realize the summary of the node detection information, and the peer-to-peer nodes of the nodes record the node detection information.
Node ID generation is done entirely by the node independently without any external communication. The generation mode mainly adopts an asymmetric password system, an anti-collision Hash algorithm and the like, and comprises the following steps:
1. generating a node private key: the method is solved at one time by using a collision-proof hash function by using Gaussian white noise (a thermal noise source of a system can be used) and system correlation attributes.
PriKey＝H(Randomness||System Attributes)
2. And solving the node public key by using the asymmetric cryptographic function. Different asymmetric cryptographic algorithms may be employed. An elliptic curve algorithm is generally selected for generation. The specific method is to obtain the product by multiplication in a limited cyclic domain with prime number P as the price.
PubKey＝PriKey*G
Here G is the generator of the cyclic group.
3. Generating a node ID: the method is formed by combining public key hashing and error correction codes.
IDraw＝H(PubKey)
IDcheck＝CheckSum(IDraw)
ID＝IDraw||IDcheck。
Example 2
On the basis of the above embodiment, the node further includes a node connection unit configured to:
enabling the node to periodically acquire node ID information of other nodes;
enabling the node to determine the ID contact ratio with other nodes, and keeping the ID information of the relevant nodes with the ID contact ratio larger than a preset threshold value;
determining the proximity of the node and each related node according to the ID information of the related node;
and enabling the nodes to sequentially select the relative nodes with the maximum proximity for connection until the maximum connection number of the nodes is reached.
Specifically, in the case of node ID determination, the point-to-point connection node selects: how each node selects other nodes that need to be connected, comprising the steps of:
a) each node can set the size of the distributed hash table to store the state of the network part node;
b) the node periodically acquires other node information through the connected node;
c) but the node only keeps the node information with high contact ratio with the ID of the node, and the contact ratio calculation method comprises the following steps:
Figure BDA0002349382870000091
d) when selecting connection, selecting the device with ID closer to the self ID, and calculating the proximity as follows:
Figure BDA0002349382870000092
the larger value of Vpref is selected for connection up to the maximum number of connections. Due to the randomness of the node IDs, the connections between the nodes are evenly distributed throughout the network.
Example 3
On the basis of the above embodiment, the node detection unit is specifically configured to:
randomly selecting a preset number of nodes to be detected from the nodes in the distributed hash table in the nodes;
and sequentially selecting the nodes to be detected with the maximum detectability from the nodes to be detected for detection.
Specifically, the point-to-point probing node selects: how each node selects the node to be detected in each round comprises the following steps:
a) the detection node is selected from nodes in a distributed hash table in the node, and the detection node does not refer to a connected node here;
b) each round is independently selected, a random factor is added, and the random factor can consider a time factor so as to ensure the randomness and the detectability of each round of detection:
Figure BDA0002349382870000101
selecting VdetectThe larger one is used for detection, and the number of detection in each round can be configured, such as 10.
Example 4
On the basis of the above embodiment, the nodes transmit information between the nodes through point-to-point network broadcasting and Gossip protocol.
Specifically, decentralization accessibility, availability and performance detection of decentralization storage network nodes are carried out by introducing a cryptography technology and the propagation characteristic of a point-to-point network, so that decentralization management is realized, the coverage is wide, the accuracy and the objectivity are realized, the resources are saved, the efficiency is improved, and the specific expression is as follows: the randomness of the node ID and the randomness of the selection of the peer detection nodes of the nodes greatly improve the detection universality and universality through random replacement of each round; through long-time detection, the coverage of the whole network is well balanced and covered, and the result is more objective; the detection data is signed by using an asymmetric password mechanism, so that the authenticity and the non-tampering property of the data are ensured, and the objective safety protection effect is achieved on the performance detection of a decentralized and untrusted network; the performance characteristics of the single node are realized through statistics, and the random coverage is added, so that the malicious attack resisting effect is achieved, and the accuracy of decentralized network detection is guaranteed.
Example 5
On the basis of the previous embodiment, the system further comprises a node monitoring unit, which is used for monitoring illegal information transmitted between nodes; the method for monitoring the information transmitted between the nodes by the node monitoring unit comprises the following steps: preprocessing the transmitted data and then establishing an index, wherein the index item is extracted after training the data with known sensitivity level; the method for obtaining the weight adopts a weighting strategy, specifically adopts a sensitive information detection algorithm based on a vector space, and comprises the following steps: representing transmission data as a vector of weights W according to a weighting strategyj＝<w1j，W2j，...，WMj>Wherein W isijThe weight of the index item in the transmission data is represented, and a specific calculation formula can be represented as follows:
Figure BDA0002349382870000111
wherein tf (t)i，dj) Is a word tiThe number of words present in the transmission data; n is the number of all texts to be clustered; df (t)i) To include the word tiThe number of transmission data of (1); expressing query P as vector of weight to calculate similarity between query and transmitted data, wherein the expression formula of query weight vector is Q ═ w1q，w2q，...，wMq>(ii) a Compute query P and document djMiddle index item tiThe similarity is calculated according to the following formula:
Figure BDA0002349382870000121
and comparing the similarity value with a set threshold, if the similarity value is greater than the set threshold, the information is sensitive information, and the monitoring channel controls the transmission channel to be closed, so that the information cannot be transmitted.
Example 6
On the basis of the embodiment 4, the nodes commonly maintain the detection period and the node information; storing the node interrelation and the node information through the distributed hash table; each node only needs to store part of the content of the distributed Hash table.
Example 6
A method for node performance detection in a decentralized storage network, the method comprising the steps of: a plurality of peer-to-peer network nodes are connected through a point-to-point network to form a network; the node comprises: the system comprises a node detection unit, a random data access unit, a detection response unit, a data statistics recording unit and a control unit; the node detection unit is used for periodically and actively detecting the accessibility and the availability among nodes in the point-to-point decentralized storage network; the random data access unit is used for detecting the storage access performance among nodes in the point-to-point decentralized storage network; the detection response unit is used for receiving the detection message and responding in time; a data statistics recording unit: carrying out data exchange and data processing between nodes, recording the process and forming a statistical report of the node performance; the control unit is used for constructing a node ID, carrying out encryption signature on the detection data, providing the constructed node ID for other units in the system to use, and controlling the operation of other units in the system, wherein the construction method of the node ID comprises the following steps: generating a private key according to the anti-collision hash function and the random number generated by the random number generator; generating a public key corresponding to the private key based on an asymmetric cryptographic algorithm; and constructing the node ID according to the hash value and the check code corresponding to the public key.
Specifically, the nodes in the system are peer-to-peer, and the node performance detection is periodically performed, so that the node information maintained in the system has timeliness. All nodes maintain the detection period and the node information together, and each round of detection is guaranteed to be orderly and correctly carried out. Each node stores node interrelations and node information through a distributed Hash table. The system is structured as follows: the performance detection is carried out periodically, the period is maintained by all nodes together, and all detection information which is not carried out in the detection period is not adopted uniformly; in one round of detection, each node should be used as a detection unit to avoid that part of nodes are not detected. Each node randomly generates a node list to be detected, the information is reported to a network and is commonly maintained by all the nodes, and therefore, part of the nodes are effectively prevented from being united and uploading unrealistic information.
The single node detection order is reachability, availability, and performance detection. The availability is the detection of the node service performance, and a corresponding data protocol, such as a request mode, a data format, a response reply and the like, should be established according to the service provided by the system. The detecting unit and the detected unit need to follow the data protocol to carry out mutual data interaction, otherwise, the detection result has no meaning.
Example 7
On the basis of the above embodiment, after the node ID is constructed, the method further includes:
enabling the node to periodically acquire node ID information of other nodes;
enabling the node to determine the ID contact ratio with other nodes, and keeping the ID information of the relevant nodes with the ID contact ratio larger than a preset threshold value;
determining the proximity of the node and each related node according to the ID information of the related node;
and enabling the nodes to sequentially select the relative nodes with the maximum proximity for connection until the maximum connection number of the nodes is reached.
Specifically, in the case of node ID determination, the point-to-point connection node selects: how each node selects other nodes that need to be connected, comprising the steps of:
a) each node can set the size of the distributed hash table to store the state of the network part node;
b) the node periodically acquires other node information through the connected node;
c) but the node only keeps the node information with high contact ratio with the ID of the node, and the contact ratio calculation method comprises the following steps:
Figure BDA0002349382870000141
d) when selecting connection, selecting the device with ID closer to the self ID, and calculating the proximity as follows:
Figure BDA0002349382870000142
the larger value of Vpref is selected for connection up to the maximum number of connections. Due to the randomness of the node IDs, the connections between the nodes are evenly distributed throughout the network.
Example 8
On the basis of the above embodiment, the node probing unit performs periodic active probing on reachability and availability among nodes in a point-to-point decentralized storage network, specifically:
randomly selecting a preset number of nodes to be detected from the nodes in the distributed hash table in the nodes;
and sequentially selecting the nodes to be detected with the maximum detectability from the nodes to be detected for detection.
Specifically, the point-to-point probing node selects: how each node selects the node to be detected in each round comprises the following steps:
a) the detection node is selected from nodes in a distributed hash table in the node, and the detection node does not refer to a connected node here;
b) each round is independently selected, a random factor is added, and the random factor can consider a time factor so as to ensure the randomness and the detectability of each round of detection:
Figure BDA0002349382870000151
selecting VdetectThe larger one is used for detection, and the number of detection in each round can be configured, such as 10.
Example 9
On the basis of the above embodiment, the nodes transmit information between the nodes through point-to-point network broadcasting and Gossip protocol.
Specifically, the detection result of the single node is issued to the whole network through the Gossip protocol, and all the nodes maintain the distributed hash table of the node performance information together. The Gossip process is initiated by the node to be reported, several nodes around the node are randomly selected in each period to broadcast messages, and the node which has received the messages is not broadcasted; the node receiving the message will repeat the process until eventually all nodes in the network receive the message, and ideally eventually all nodes will receive the message, so it is a final coherency protocol.
Example 10
On the basis of the previous embodiment, the nodes commonly maintain the detection period and the node information; storing the node interrelation and the node information through the distributed hash table; each node only needs to store part of the content of the distributed Hash table.
Example 11
On the basis of embodiment 6, the number of peer-to-peer network nodes; the network nodes are connected through a point-to-point network; the method is characterized in that: the node comprises: the system comprises a node detection unit, a random data access unit, a detection response unit, a data statistics recording unit and a control unit; the node detection unit is used for periodically and actively detecting the accessibility and the availability among the nodes in the point-to-point decentralized storage network; the random data access unit is used for detecting the storage access performance among the nodes in the point-to-point decentralized storage network; the detection response unit is used for receiving the detection message and responding in time; the data statistics recording unit: the system is used for carrying out data exchange and data processing between nodes, carrying out process recording and forming a statistical report of the node performance; the control unit is used for constructing the node ID, carrying out encryption signature on the detection data, providing the constructed node ID for other units in the system to use, and controlling the operation of other units in the system.
Specifically, the detection result of the single node is issued to the whole network through the Gossip protocol, and all the nodes maintain the distributed hash table of the node performance information together. The Gossip process is initiated by the node to be reported, several nodes around the node are randomly selected in each period to broadcast messages, and the node which has received the messages is not broadcasted; the node receiving the message will repeat the process until eventually all nodes in the network receive the message, and ideally eventually all nodes will receive the message, so it is a final coherency protocol.
Example 12
On the basis of the above embodiment, the method for randomly generating a sequence X by a pseudo-random algorithm and designing a filter according to the autocorrelation function and the gaussian white noise power spectral density of the given sequence X performs the following steps: condition R of autocorrelation function satisfied by sequence X and Laplace noiseZ(m)＝RXX(m) substituting the Laplace noise generation formula to obtain an autocorrelation function of the correlated Gaussian noise as follows:
Figure BDA0002349382870000161
wherein R isXX(m) is the autocorrelation function of the sequence; fourier function of autocorrelation function of correlated Gaussian noiseAnd (3) performing inner-leaf transformation to obtain the power spectral density of the correlated Gaussian noise as follows:
Figure BDA0002349382870000171
wherein δ (ω) is a pulse function; after gaussian noise passes through the existing system, the power spectral density is: pG′(ω)＝PG(ω)·|H(ω)|2(ii) a By the above formula, the system function of the filter is obtained as:
Figure BDA0002349382870000172
wherein N is0Is the known power spectral density of gaussian noise.
Example 13
On the basis of the above embodiment, the nodes transmit information between the nodes through point-to-point network broadcasting and Gossip protocol.
Example 14
On the basis of the previous embodiment, the system further comprises a node monitoring unit, which is used for monitoring illegal information transmitted between nodes; the method for monitoring the information transmitted between the nodes by the node monitoring unit comprises the following steps: preprocessing the transmitted data and then establishing an index, wherein the index item is extracted after training the data with known sensitivity level; the method for obtaining the weight adopts a weighting strategy, specifically adopts a sensitive information detection algorithm based on a vector space, and comprises the following steps: representing transmission data as a vector of weights W according to a weighting strategyj＝<w1j，W2j，...，WMj>Wherein W isijThe weight of the index item in the transmission data is represented, and a specific calculation formula can be represented as follows:
Figure BDA0002349382870000173
wherein tf (t)i，dj) Is a word tiThe number of words present in the transmission data; n is the number of all texts to be clustered; df (t)i) To include the word tiThe number of transmission data of (1); representing queries P as vectors of weights to compute query and transmit dataThe expression formula of the query weight vector is Q ═<w1q，w2q，...，wMqAbove (S); compute query P and document djMiddle index item tiThe similarity is calculated according to the following formula:
Figure BDA0002349382870000181
and comparing the similarity value with a set threshold, if the similarity value is greater than the set threshold, the information is sensitive information, and the monitoring channel controls the transmission channel to be closed, so that the information cannot be transmitted.
The above description is only an embodiment of the present invention, but not intended to limit the scope of the present invention, and any structural changes made according to the present invention should be considered as being limited within the scope of the present invention without departing from the spirit of the present invention.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process and related description of the system described above may refer to the corresponding process in the foregoing method embodiments, and will not be described herein again.
It should be noted that, the system provided in the foregoing embodiment is only illustrated by dividing the functional modules, and in practical applications, the functions may be distributed by different functional modules according to needs, that is, the modules or steps in the embodiment of the present invention are further decomposed or combined, for example, the modules in the foregoing embodiment may be combined into one module, or may be further split into multiple sub-modules, so as to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not to be construed as unduly limiting the present invention.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes and related descriptions of the storage device and the processing device described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
Those of skill in the art would appreciate that the various illustrative modules, method steps, and modules described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that programs corresponding to the software modules, method steps may be located in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. To clearly illustrate this interchangeability of electronic hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as electronic hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.
The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.

Claims (10)

1. A node performance detection system for a decentralized storage network, the system comprising: a number of peer-to-peer network nodes; the network nodes are connected through a point-to-point network; the method is characterized in that: the node comprises: the system comprises a node detection unit, a random data access unit, a detection response unit, a data statistics recording unit and a control unit; the node detection unit is used for periodically and actively detecting the accessibility and the availability among the nodes in the point-to-point decentralized storage network; the random data access unit is used for detecting the storage access performance among nodes in a point-to-point decentralized storage network, wherein the storage access performance is detected, the link among peer nodes of the point-to-point network is determined by the relation among node IDs, and the peer detection result is automatically transmitted by a network Gossip protocol through a publish-subscribe mode by the nodes and is recorded by the peer nodes of the nodes; the detection response unit is used for receiving the detection message and responding in time; the data statistics recording unit: the system is used for carrying out data exchange and data processing between nodes, carrying out process recording and forming a statistical report of the node performance; the control unit is used for constructing a node ID, carrying out encryption signature on the detection data, providing the constructed node ID for other units in the system to use, and controlling the operation of other units in the system; the method for constructing the node ID comprises the following steps: generating a private key according to the anti-collision hash function and the random number generated by the random number generator; generating a public key corresponding to the private key based on an asymmetric cryptographic algorithm; and constructing the node ID according to the hash value and the check code corresponding to the public key.
2. The system of claim 1, wherein the node further comprises a node connection unit to:
enabling the node to periodically acquire node ID information of other nodes;
enabling the node to determine the ID contact ratio with other nodes, and keeping the ID information of the relevant nodes with the ID contact ratio larger than a preset threshold value;
determining the proximity of the node and each related node according to the ID information of the related node;
and enabling the nodes to sequentially select the relative nodes with the maximum proximity for connection until the maximum connection number of the nodes is reached.
3. The system of claim 1, wherein the node probing unit is specifically configured to:
randomly selecting a preset number of nodes to be detected from the nodes in the distributed hash table in the nodes;
and sequentially selecting the nodes to be detected with the maximum detectability from the nodes to be detected for detection.
4. The system of claim 2, wherein the nodes communicate inter-node information therebetween via point-to-point network broadcast and Gossip protocols.
5. The system of claim 4, wherein the nodes collectively maintain a detection period and node information; storing the node interrelation and the node information through a distributed hash table; each node only needs to store part of the content of the distributed Hash table.
6. A method for detecting node performance of a decentralized storage network based on the system of one of claims 1 to 5, characterized in that the method performs the following steps: a plurality of peer-to-peer network nodes are connected through a point-to-point network to form a network; the node comprises: the system comprises a node detection unit, a random data access unit, a detection response unit, a data statistics recording unit and a control unit; the node detection unit is used for periodically and actively detecting the accessibility and the availability among nodes in the point-to-point decentralized storage network; the random data access unit is used for detecting the storage access performance among nodes in the point-to-point decentralized storage network; the detection response unit is used for receiving the detection message and responding in time; a data statistics recording unit: carrying out data exchange and data processing between nodes, recording the process and forming a statistical report of the node performance; the control unit is used for constructing a node ID, carrying out encryption signature on the detection data, providing the constructed node ID for other units in the system to use, and controlling the operation of other units in the system, wherein the construction method of the node ID comprises the following steps: generating a private key according to the anti-collision hash function and the random number generated by the random number generator; generating a public key corresponding to the private key based on an asymmetric cryptographic algorithm; and constructing the node ID according to the hash value and the check code corresponding to the public key.
7. The method of claim 6, after constructing the node ID, further comprising:
the node periodically acquires node ID information of other nodes;
the node determines the ID contact ratio with other nodes, and retains the ID information of the relevant nodes with the ID contact ratio larger than a preset threshold value;
determining the proximity of the node and each related node according to the ID information of the related node;
and the nodes sequentially select the relative nodes with the maximum proximity for connection until the maximum connection number of the nodes is reached.
8. The method according to claim 7, wherein the node probing unit performs periodic active probing of reachability and availability among nodes in the peer-to-peer decentralized storage network, specifically:
randomly selecting a preset number of nodes to be detected from the nodes in the distributed hash table in the nodes;
and sequentially selecting the nodes to be detected with the maximum detectability from the nodes to be detected for detection.
9. The method of claim 8, wherein the nodes transmit inter-node information therebetween via point-to-point network broadcast and Gossip protocol.
10. The method of claim 9, wherein the nodes maintain a detection period and node information in common; storing the node interrelation and the node information through the distributed hash table; each node only needs to store part of the content of the distributed Hash table.
CN201911408673.XA 2019-12-31 2019-12-31 Node performance detection system and method for decentralized storage network Active CN111064641B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911408673.XA CN111064641B (en) 2019-12-31 2019-12-31 Node performance detection system and method for decentralized storage network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911408673.XA CN111064641B (en) 2019-12-31 2019-12-31 Node performance detection system and method for decentralized storage network

Publications (2)

Publication Number Publication Date
CN111064641A CN111064641A (en) 2020-04-24
CN111064641B true CN111064641B (en) 2021-07-02

Family

ID=70305420

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911408673.XA Active CN111064641B (en) 2019-12-31 2019-12-31 Node performance detection system and method for decentralized storage network

Country Status (1)

Country Link
CN (1) CN111064641B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112737864A (en) * 2021-01-15 2021-04-30 北京开物数智科技有限公司 Edge cloud network division monitoring method
CN114363357B (en) * 2021-12-28 2024-01-19 上海沄熹科技有限公司 Distributed database network connection management method based on Gossip

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101119271A (en) * 2007-07-05 2008-02-06 中国科学技术大学 Structured P2P based application service platform and implementing method thereof
CN101378325A (en) * 2007-08-31 2009-03-04 华为技术有限公司 Superposition network and building method thereof
CN102104518A (en) * 2011-03-14 2011-06-22 南京邮电大学 Hybrid Pastry network for voice over Internet protocol (VoIP) service
CN102769673A (en) * 2012-07-25 2012-11-07 楚云汉智武汉网络存储系统有限公司 Failure detection method suitable to large-scale storage cluster
CN102932451A (en) * 2012-10-31 2013-02-13 北龙中网（北京）科技有限责任公司 Solving system specific to content distribution network
CN105335229A (en) * 2014-07-25 2016-02-17 杭州华三通信技术有限公司 Business resource scheduling method and apparatus
US9369360B1 (en) * 2014-05-12 2016-06-14 Google Inc. Systems and methods for fault detection in large scale networks

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10614098B2 (en) * 2010-12-23 2020-04-07 Mongodb, Inc. System and method for determining consensus within a distributed database
CN104348641A (en) * 2013-07-30 2015-02-11 华为技术有限公司 Fault detection method and fault detection device
CN106559166B (en) * 2015-09-25 2020-07-17 伊姆西Ip控股有限责任公司 Fingerprint-based state detection method and equipment for distributed processing system
US10972269B2 (en) * 2017-09-11 2021-04-06 Jpmorgan Chase Bank, N.A. Systems and methods for token vault synchronization using a distributed ledger
CN110162424B (en) * 2019-05-23 2022-03-22 腾讯科技（深圳）有限公司 Fault processing method, system, device and storage medium

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101119271A (en) * 2007-07-05 2008-02-06 中国科学技术大学 Structured P2P based application service platform and implementing method thereof
CN101378325A (en) * 2007-08-31 2009-03-04 华为技术有限公司 Superposition network and building method thereof
CN102104518A (en) * 2011-03-14 2011-06-22 南京邮电大学 Hybrid Pastry network for voice over Internet protocol (VoIP) service
CN102769673A (en) * 2012-07-25 2012-11-07 楚云汉智武汉网络存储系统有限公司 Failure detection method suitable to large-scale storage cluster
CN102932451A (en) * 2012-10-31 2013-02-13 北龙中网（北京）科技有限责任公司 Solving system specific to content distribution network
US9369360B1 (en) * 2014-05-12 2016-06-14 Google Inc. Systems and methods for fault detection in large scale networks
CN105335229A (en) * 2014-07-25 2016-02-17 杭州华三通信技术有限公司 Business resource scheduling method and apparatus

Also Published As

Publication number Publication date
CN111064641A (en) 2020-04-24

Similar Documents

Publication Publication Date Title
CN101304321B (en) Method for defending equity network virus based on trust
CN105247529A (en) Synchronizing credential hashes between directory services
Shahsavari et al. Performance modeling and analysis of the bitcoin inventory protocol
Yu et al. Practical and secure multidimensional query framework in tiered sensor networks
CN111064641B (en) Node performance detection system and method for decentralized storage network
CN112865962B (en) Distributed identification authentication method and device based on block chain and electronic equipment
Chang et al. Protecting mobile crowd sensing against sybil attacks using cloud based trust management system
CN115499249B (en) File storage method and system based on block chain distributed encryption
Balakrishnan et al. Subjective logic based trust model for mobile ad hoc networks
Danner et al. Robust fully distributed minibatch gradient descent with privacy preservation
Wang et al. Bpfl: A blockchain based privacy-preserving federated learning scheme
JP2006109413A (en) Authentication method of group element
Liu et al. A distributed attack-resistant trust model for automatic modulation classification
Sharma et al. A sustainable Ethereum merge-based Big-Data gathering and dissemination in IIoT System
Ge et al. A privacy protection method of lightweight nodes in blockchain
Boumerzoug et al. A key management method based on an AVL tree and ECC cryptography for wireless sensor networks
WO2023003847A2 (en) System and method for quantum-secure microgrids
Lim et al. Secure and structured IoT smart grid system management
Shibu et al. Random bit extraction for secret key generation in MANETs
Nallakannu et al. PSO‐based optimal peer selection approach for highly secure and trusted P2P system
Almuzaini et al. Secured wireless sensor networks using hybrid Rivest Shamir Adleman with ant lion optimization algorithm
Mirval et al. Federated learning on personal data management systems: Decentralized and reliable secure aggregation protocols
Huang et al. A secure and efficient WSN by employing symmetric key matrix and rectangular frame scheme
Yuan et al. AME Blockchain: An Architecture Design for Closed-Loop Fluid Economy Token System
Doraipandian et al. An Efficient Key Management Scheme in Multi-Tier and Multi-Cluster Wireless Sensor Networks.

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant