CN110336875B - Method for improving computing and storing speed of Internet of things application - Google Patents

Method for improving computing and storing speed of Internet of things application Download PDF

Info

Publication number
CN110336875B
CN110336875B CN201910603422.0A CN201910603422A CN110336875B CN 110336875 B CN110336875 B CN 110336875B CN 201910603422 A CN201910603422 A CN 201910603422A CN 110336875 B CN110336875 B CN 110336875B
Authority
CN
China
Prior art keywords
node
data
computing
edge
storage
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910603422.0A
Other languages
Chinese (zh)
Other versions
CN110336875A (en
Inventor
崔伟
佘高元
柴文坚
孙瑞华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Jiuchain Technology Co ltd
Original Assignee
Shenzhen Jiuchain Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Jiuchain Technology Co ltd filed Critical Shenzhen Jiuchain Technology Co ltd
Priority to CN201910603422.0A priority Critical patent/CN110336875B/en
Publication of CN110336875A publication Critical patent/CN110336875A/en
Application granted granted Critical
Publication of CN110336875B publication Critical patent/CN110336875B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H04L67/1061—Peer-to-peer [P2P] networks using node-based peer discovery mechanisms
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks

Abstract

The invention discloses a method for improving the computing and storing speed of the application of the Internet of things, wherein an edge node is used as a source of mass computing and mass storage resources of the Internet of things and can be all terminal equipment with certain computing or storing capacity in the future, and a central computing node is used for guaranteeing the edge node, so that the edge node is ensured to provide efficient, reliable and credible block chain network service for large data storage and ultrahigh-speed intelligent contract edge computing processing. By forming a distributed computing and storage platform by using the idle computing power and the storage power of massive edge nodes, computing tasks with long time consumption are executed, including scenes using the edge nodes, such as AI application, picture processing, gene sequencing and the like, after intensive computing tasks are unloaded from a cloud to the edge, the consumption of the whole system on energy is reduced by more than 40%, and the time of data in the aspects of integration, migration and the like can be reduced by more than 90%.

Description

Method for improving computing and storing speed of Internet of things application
Technical Field
The invention relates to the technical field of block chains, in particular to a method for improving the computing and storing speed of application of an internet of things by means of edge nodes and block chain technologies.
Background
In large-scale application of the internet of things, a large amount of computing requirements can bring serious load to a central computing node. Through the pressure test of the central computing node mode, the central computing node is not suitable for processing complex long-time computing tasks in the internet of everything environment, and therefore edge computing nodes are required to be accessed to execute computing-intensive tasks. On the other hand, in some applications of the internet of things with high response requirements, the overall efficiency is low due to the delay of cloud response.
Therefore, the concept of Edge nodes (Edge nodes) is provided, and an Edge Node mechanism is used as a necessary supplement of a central computing Node, so that intensive computing services can be sunk to the Edge nodes, response delay and bandwidth cost are reduced, and the requirements of various intelligent scenes under a decentralized architecture model are met.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to solve the technical problem of improving the computing and storing speed of the application of the Internet of things by means of edge node and block chain technology.
In order to solve the technical problem, the invention is realized by the following scheme: the invention relates to a method for improving the calculation and storage speed of Internet of things application, which adopts a kademlia algorithm to realize data storage and retrieval, wherein in a block chain, the node Nid length is 512, and the node redundancy parameter K is 32;
the method comprises the following steps:
step one, an edge node E is present 1 After joining the network, requesting a node id pool from the central computing node, and selecting an unused node id from the node id pool when the central computing node is close to H1 1 To the edge node E 1 ；
Step two, the edge node E 1 Receiving node id 1 Then, nid is calculated 1 ＝sha3-512(id 1 )，Nid 1 Identifying the node;
step three, the edge node E 1 Receiving Data sent by terminal equipment 1 When the storage is required, the Data is used 1 Dividing the obtained product into small pieces;
let edge node E 1 Data to be processed 1 The length is L, and the length result after segmentation is expressed as L =sigman i ·b i；
Wherein b is i Sequentially taking 256M and 256K; n is 0 Is 256M number of blocks, n 1 256K number of blocks; corresponding data partition as B j I.e. Data 1 ＝∑B j ；
Step four, selecting a matrix M according to the HSN certification condition paid by the user, and generating redundant data according to the matrix M:
a. the user selects the normal safe mode, and the system generates redundant data according to the proportion of 10 percent: m is i ＝n i *10％，(i＝0,1)；
b. The user selects the medium safety mode, and the system generates redundant data according to the proportion of 20 percent: m is i ＝n i *20％，(i＝0,1)；
c. The user selects a strong security mode, and the system generates redundant data according to the proportion of 30 percent: m is i ＝n i *30％，(i＝0,1)；
d. The newly generated overall data is recorded as: data ’ 1 ＝∑B k Wherein k is less than or equal to n 0 +n 1 When, B k ＝B j ，k﹥n 0 +n 1 When, B k Is redundant data;
step five, calculating for each k
Figure GDA0003868572370000021
According to h j At the edge node E 1 Inquiring whether a data table item exists, wherein when the data table item exists, continuing the next k processing after executing the block chaining certification operation; if the data table item exists, the following steps a-d are carried out;
a. computing nodes to be stored
Figure GDA0003868572370000022
And the local node E 1 Distance between two adjacent plates
Figure GDA0003868572370000023
Looking up the position of the kademlia binary tree routing table;
b. seeking by ping command
Figure GDA0003868572370000024
Whether to survive;
c. if there is a direction of activity
Figure GDA0003868572370000025
Send store B k Instructions; next k treatment;
d. if node
Figure GDA0003868572370000026
Non-survival, reporting the non-survival node to a central computing node, and then collecting the non-survival node in a corresponding K-kucket set
Figure GDA0003868572370000027
Selecting a first survival node for storage;
let i t Indicating the distance between the node to be stored and the edge node, 2 i ≤i t <2 i+1 T =1, 2.. K max is chosen to be 32, and if not found, the data is discarded.
Furthermore, after the node id is allocated, the node id is kept unchanged in the whole life cycle.
Compared with the prior art, the invention has the beneficial effects that: the edge node is used as a source of mass computing and mass storage resources of the Internet of things, can be terminal equipment with certain computing or storage capacity in the future, and ensures that the edge node provides efficient, reliable and credible block chain network service for large data storage and ultrahigh-speed intelligent contract edge computing processing by ensuring that the central computing node guarantees the edge node. By forming a distributed computing and storage platform by using the idle computing power and the storage power of massive edge nodes, computing tasks which are long in time consumption are executed, including scenes using the edge nodes, such as AI application, picture processing, gene sequencing and the like, after intensive computing tasks are unloaded from a cloud to the edge, the energy consumption of the whole system is reduced by more than 40%, and the time of data in the aspects of integration, migration and the like can be reduced by more than 90%.
Drawings
FIG. 1 is a schematic diagram of a kademlia binary tree structure according to the present invention.
FIG. 2 is a schematic diagram of k-bucket storage inside each node according to the present invention.
FIG. 3 is a schematic diagram of Read-solomon data recovery according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, and thus, the protection scope of the present invention is more clearly and clearly defined. It should be apparent that the described embodiments of the present invention are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplification of description, but do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; the two elements may be directly connected or indirectly connected through an intermediate medium, or may be connected through the inside of the two elements, or may be connected wirelessly or through a wire. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
Furthermore, the technical features involved in the different embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Example 1:
referring to fig. 1-3, the invention provides a method for improving the calculation and storage speed of internet of things application, which adopts kademlia algorithm to realize data storage and retrieval, and is characterized in that in a block chain, the node Nid length value is 512, and the node redundancy parameter K value is 32;
the method comprises the following steps:
step one, an edge node E is present 1 After joining the network, requesting a node id pool from the central computing node, and selecting an unused node id from the node id pool when the central computing node is close to H1 1 To said edge node E 1 ；
Step two, the edge node E 1 Receiving node id 1 Then, nid is calculated 1 ＝sha3-512(id 1 )，Nid 1 Identifying the node;
step three, the edge node E 1 Receiving Data sent by terminal equipment 1 When the storage is required, the Data is used 1 Dividing the obtained product into small pieces;
let edge node E 1 Data to be processed 1 The length is L, and the length result after segmentation is expressed as L =sigman i ·b i；
Wherein b is i Sequentially taking 256M and 256K; n is 0 Is 256M number of blocks, n 1 256K number of blocks; corresponding data partition as B j I.e. Data 1 ＝∑B j ；
Step four, selecting a matrix M according to the HSN certification condition paid by the user, and generating redundant data according to the matrix M:
a. the user selects a common safety mode, and the system generates redundant data according to the proportion of 10 percent: m is i ＝n i *10％，(i＝0,1)；
b. The user selects the medium safety mode, and the system generates redundant data according to the proportion of 20 percent: m is a unit of i ＝n i *20％，(i＝0,1)；
c. The user selects a strong safety mode, and the system generates redundant data according to the proportion of 30 percent: m is i ＝n i *30％，(i＝0,1)；
d. The newly generated overall data is recorded as: data ’ 1 ＝∑B k Wherein k is less than or equal to n 0 +n 1 When, B k ＝B j ，k﹥n 0 +n 1 When, B k Is redundant data;
step five, calculating for each k
Figure GDA0003868572370000041
According to h j At the edge node E 1 Inquiring whether a data table item exists, wherein when the data table item exists, continuing the next k processing after executing the block chaining certification operation; if the data table item exists, the following steps a-d are carried out;
a. computing nodes to be stored
Figure GDA0003868572370000042
And the local node E 1 Distance between two adjacent plates
Figure GDA0003868572370000043
Looking up the position of the kademlia binary tree routing table;
b. seeking by ping command
Figure GDA0003868572370000044
Whether to survive;
c. if there is a direction of activity
Figure GDA0003868572370000045
Send store B k Instructions; next k treatment;
d. if node
Figure GDA0003868572370000046
Non-survival, reporting the non-survival node to a central computing node, and then collecting the non-survival node in a corresponding K-kucket set
Figure GDA0003868572370000047
Selecting a first survival node for storage;
let i t Indicating the distance between the node to be stored and the edge node, 2 i ≤i t <2 i+1 T =1, 2.. K max is chosen to be 32, and if not found, the data is discarded.
A preferred technical solution of this embodiment: after the node id is distributed, the node id is kept unchanged in the whole life cycle.
The edge node is used as a source of mass computing and mass storage resources of the Internet of things, can be terminal equipment with certain computing or storage capacity in the future, and ensures that the edge node provides efficient, reliable and credible block chain network service for large data storage and ultrahigh-speed intelligent contract edge computing processing by ensuring that the central computing node guarantees the edge node. By forming a distributed computing and storage platform by using the idle computing power and the storage power of massive edge nodes, computing tasks which are long in time consumption are executed, including scenes using the edge nodes, such as AI application, picture processing, gene sequencing and the like, after intensive computing tasks are unloaded from a cloud to the edge, the energy consumption of the whole system is reduced by more than 40%, and the time of data in the aspects of integration, migration and the like can be reduced by more than 90%.
Example 2:
as shown in fig. 1, fig. 1 is a schematic diagram of a kademlia binary tree structure according to the present invention. Kademlia is a distributed hash table, and DHT technology is a most core routing addressing technology in a decentralized P2P network, and can quickly find a target node in the network without a central server. Fig. 1 is a simple distribution diagram of nodes.
Example 3:
as shown in fig. 2, fig. 2 is a schematic view of the internal k-bucket storage of each node according to the present invention.
The node distance of K-bucket0 is (2) 0 ，2 1 )；
The node distance of K-bucket1 is (2) 1 ，2 2 )；
The node distance of K-bucket3 is (2) 2 ，2 3 )；
The node distance of K-bucket511 is (2) 511 ，2 512 ) And the K-bucket511 node is provided with an oldest updating node and a newest updating node.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and all equivalent structures or equivalent processes performed by the present invention or directly or indirectly applied to other related technical fields are also included in the scope of the present invention.

Claims (2)

1. A method for improving the calculation and storage speed of Internet of things application adopts a kademlia algorithm to realize data storage and retrieval, and is characterized in that in a block chain, the node Nid length value is 512, and the node redundancy parameter K value is 32;
the method comprises the following steps:
step one, an edge node E is present 1 After joining the network, requesting a node id pool from the central computing node, and selecting an unused node id from the node id pool when the central computing node is close to H1 1 To said edge node E 1 ；
Step two, the edge node E 1 Receiving node id 1 Then, the Nid is calculated 1 ＝sha3-512(id 1 )，Nid 1 Identifying the node;
step three, the edge node E 1 Receiving Data sent by terminal equipment 1 When the storage is required, the Data is used 1 Dividing the obtained product into small pieces;
let edge node E 1 Data to be processed 1 The length is L, and the length result after segmentation is expressed as L =sigman i ·b i；
Wherein b is i Sequentially taking 256M and 256K; n is a radical of an alkyl radical 0 Is 256M number of blocks, n 1 256K number of partitions; the corresponding data is divided into B j I.e. Data 1 ＝∑B j ；
Step four, selecting a matrix M according to the HSN certification condition paid by the user, and generating redundant data according to the matrix M:
a. the user selects the normal safe mode, and the system generates redundant data according to the proportion of 10 percent: m is i ＝n i *10％，(i＝0,1)；
b. The user selects the medium safety mode, and the system generates redundant data according to the proportion of 20 percent: m is i ＝n i *20％，(i＝0,1)；
c. The user selects the strong security mode and,the system generates redundant data at a 30% ratio: m is i ＝n i *30％，(i＝0,1)；
d. The newly generated overall data is recorded as: data' 1 ＝∑B k Wherein k is less than or equal to n 0 +n 1 When, B k ＝B j ，k﹥n 0 +n 1 When, B k Is redundant data;
step five, calculating for each k
Figure FDA0003868572360000011
According to h j At the edge node E 1 Inquiring whether a data table item exists, wherein when the data table item exists, continuing the next k processing after executing the block chaining certification operation; if the data table item exists, the following steps a-d are carried out;
a. computing nodes to be stored
Figure FDA0003868572360000012
And the local node E 1 Distance between two adjacent plates
Figure FDA0003868572360000013
Searching the position of the kademlia binary tree routing table;
b. seeking by ping command
Figure FDA0003868572360000014
Whether to survive;
c. if there is a direction of activity
Figure FDA0003868572360000015
Send store B k Instructions; next k treatment;
d. if node
Figure FDA0003868572360000016
Non-survival, reporting the non-survival node to a central computing node, and then collecting the non-survival node in a corresponding K-kucket set
Figure FDA0003868572360000017
Selecting a first survival node for storage;
let i t Indicating the distance between the node to be stored and the edge node, 2 i ≤i t <2 i+1 T =1,2.. K max, 32 is selected, and if not found, the data is discarded.
2. The method for increasing the computing and storage speed of the internet of things application according to claim 1, wherein the method comprises the following steps: after the node id is allocated, the node id is kept unchanged in the whole life cycle.
CN201910603422.0A 2019-07-05 2019-07-05 Method for improving computing and storing speed of Internet of things application Active CN110336875B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910603422.0A CN110336875B (en) 2019-07-05 2019-07-05 Method for improving computing and storing speed of Internet of things application

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910603422.0A CN110336875B (en) 2019-07-05 2019-07-05 Method for improving computing and storing speed of Internet of things application

Publications (2)

Publication Number Publication Date
CN110336875A CN110336875A (en) 2019-10-15
CN110336875B true CN110336875B (en) 2022-12-02

Family

ID=68144409

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910603422.0A Active CN110336875B (en) 2019-07-05 2019-07-05 Method for improving computing and storing speed of Internet of things application

Country Status (1)

Country Link
CN (1) CN110336875B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112055052B (en) * 2020-08-05 2022-06-14 中国建设银行股份有限公司 Service detection and information distribution method, system and medium for internet of things system
CN113852601B (en) * 2021-08-11 2023-04-28 杭州师范大学 Intelligent contract-based internet of vehicles matrix computing security unloading verifiable method
CN116248680B (en) * 2023-05-11 2023-08-01 湖南工商大学 De novo peptide sequencing method, de novo peptide sequencing device and related equipment

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10362517B2 (en) * 2017-04-07 2019-07-23 Vapor IO Inc. Distributed handoff-related processing for wireless networks
CN109523243A (en) * 2018-11-19 2019-03-26 济南浪潮高新科技投资发展有限公司 A kind of mist calculates the date storage method based on block chain under environment
CN109688211A (en) * 2018-12-18 2019-04-26 杭州茂财网络技术有限公司 Data distribution formula processing method
CN109873879A (en) * 2019-04-17 2019-06-11 上海沄界信息科技有限公司 It can the distributed cloud storage method and device really weighed of data

Also Published As

Publication number Publication date
CN110336875A (en) 2019-10-15

Similar Documents

Publication Publication Date Title
CN110336875B (en) Method for improving computing and storing speed of Internet of things application
US9218220B2 (en) Elastic and scalable publish/subscribe service
CN106101232A (en) Load-balancing method and device
CN103595805A (en) Data placement method based on distributed cluster
Lin et al. Research on resource self-organizing model for cloud computing
CN111083179B (en) Internet of things cloud platform, and equipment interaction method and device based on same
CN104504147A (en) Resource coordination method, device and system for database cluster
CN112104558A (en) Method, system, terminal and medium for implementing block chain distribution network
CN113655969A (en) Data balanced storage method based on streaming distributed storage system
CN103326925A (en) Message push method and device
CN103297547B (en) The P2P system constructing cloud based on DHT is used to store the method for auxiliary system
Gheorghe et al. Decentralized storage system for edge computing
CN113488996A (en) Power distribution network protogram modeling method based on distributed parallel graph computing framework
CN111225045B (en) HIVE high-availability early warning method, device and computer readable storage medium
CN115878729B (en) Node block storage allocation optimization method and system based on alliance chain
CN112748996A (en) Load balancing strategy method and system of non-centralized topology system
CN109254844B (en) Triangle calculation method of large-scale graph
Beaumont et al. Allocation of clients to multiple servers on large scale heterogeneous platforms
CN107623586B (en) Network link prediction method and device
Souravlas et al. Dynamic Load Balancing on All-to-All Personalized Communications Using the NNLB Principal
CN108923937A (en) A kind of similar port power distribution method of electric energy router group system and device
CN105574134B (en) A kind of document handling method and device
CN107341241A (en) A kind of wind-powered electricity generation big data analysis system based on cloud computing
CN117439655B (en) Space terahertz information center network lightweight caching method and device
CN115277416B (en) Container deployment method and system for edge computing in industrial Internet

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant