CN113535850A - Data collaboration method, device and system based on block chain - Google Patents
Data collaboration method, device and system based on block chain Download PDFInfo
- Publication number
- CN113535850A CN113535850A CN202110789801.0A CN202110789801A CN113535850A CN 113535850 A CN113535850 A CN 113535850A CN 202110789801 A CN202110789801 A CN 202110789801A CN 113535850 A CN113535850 A CN 113535850A
- Authority
- CN
- China
- Prior art keywords
- data
- block chain
- module
- message queue
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Quality & Reliability (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a data collaboration method, device and system based on a block chain. The method comprises the following steps: acquiring data from a resource layer; performing calculation analysis on the data, and dividing the data into local storage data and cloud storage data according to calculation analysis results; uploading the cloud storage data to a cloud storage, and locally storing the local storage data; and uploading key data in the data and/or characteristic values of the key data to a block chain publishing node in a block chain, and performing uplink by the block chain publishing node. The scheme provided by the application can greatly reduce the storage burden and simultaneously can enhance the expandability of the system.
Description
Technical Field
The application relates to the technical field of Internet of things and block chains, in particular to a data collaboration method, device and system based on a block chain.
Background
At present, the block chain technology is rapidly developed and is applied to different fields, such as the fields of smart power grids and internet of things. Blockchains as decentralized common ledgers can store data (i.e., transaction records) that outperforms the centralized ledger approach, the data being recorded by blockchains as blocks (e.g., collections of transactions) that form linked-list data structures to indicate logical relationships between data added to blockchains, without requiring a central entity or intermediary to maintain the data blocks.
Edge computing is introduced in blockchain technology as an extension of the cloud. Edge computing helps users to have location awareness, keep latency low, support heterogeneity, and improve application Quality Of Service (QoS) by providing computing power, data storage, and application services. The distributed structure of edge computation has many advantages. The block chain and the edge calculation are integrated into a system, and the network, the storage and the calculation distributed at the edge can be reliably accessed and controlled, so that a large-scale network server, data storage and effectiveness calculation can be provided under the safe condition.
However, the combination of blockchain and edge techniques still has some problems in the related art. In the related art, each edge node needs to store a complete data book, because the edge nodes need to consider the data storing and calculating functions of the edge nodes in addition to the data uplink function, which results in a considerable storage burden of the edge nodes. In addition, all IOT (Internet of Things) data is stored on the blockchain, which in turn results in a lack of scalability of the system.
Disclosure of Invention
In order to solve or partially solve the problems in the related art, the application provides a data collaboration method, device and system based on a block chain, and the method, device and system can greatly reduce the storage burden and enhance the expandability of the system.
A first aspect of the present application provides a data coordination method based on a block chain, including:
acquiring data from a resource layer;
performing calculation analysis on the data, and dividing the data into local storage data and cloud storage data according to calculation analysis results; uploading the cloud storage data to a cloud storage, and locally storing the local storage data;
and uploading key data in the data and/or characteristic values of the key data to a block chain publishing node in a block chain, and performing uplink by the block chain publishing node.
Preferably, the uploading key data in the data and/or characteristic values of the key data to a blockchain publishing node in a blockchain includes:
and packaging key data in the data and/or a hash value obtained by carrying out hash operation on the key data into a block in a block data format, and uploading the block to a block chain release node in a block chain.
Preferably, before the performing the computational analysis on the data, the method further includes: and analyzing and cleaning the data.
Preferably, the performing uplink by the block chain issuing node includes:
and after the block chain release node adds the set head information to the encapsulated block, performing uplink.
Preferably, the header information includes at least one of: version number, parent hash value, merkel root, timestamp, difficulty value, and random number.
A second aspect of the present application provides an edge node apparatus, including an equipment communication module, a data calculation module, and a block chain module;
the equipment communication module is used for acquiring data from a resource layer;
the data calculation module is used for calculating and analyzing the data acquired by the equipment communication module and dividing the data into local storage data and cloud storage data according to the calculation and analysis result; uploading the cloud storage data to a cloud storage, and locally storing the local storage data;
the block chain module is configured to upload key data in the data acquired by the device communication module and/or a feature value of the key data to a block chain publishing node in a block chain, and the block chain publishing node performs uplink.
Preferably, the apparatus further comprises: and the message queue module is used for analyzing and cleaning the data acquired by the equipment communication module.
A third aspect of the present application provides a data collaboration system based on a block chain, including a resource layer, an edge node device, a block chain publishing node, and a cloud;
the resource layer is used for acquiring data and uploading the data to the edge node device;
the edge node device is used for acquiring data from the resource layer; performing calculation analysis on the data, and dividing the data into local storage data and cloud storage data according to calculation analysis results; uploading the cloud storage data to a cloud storage, and locally storing the local storage data; uploading key data in the data and/or characteristic values of the key data to a block chain publishing node in a block chain;
the block chain publishing node is configured to receive key data in the data uploaded by the edge node device and/or a feature value of the key data, and perform data uplink;
and the cloud end is used for receiving and storing the cloud end storage data uploaded by the edge node device.
Preferably, the edge node apparatus includes:
the equipment communication module is used for acquiring data from a resource layer;
the message queue module is used for analyzing and cleaning the data acquired by the equipment communication module;
the data calculation module is used for calculating and analyzing the data processed by the message queue module and dividing the data into local storage data and cloud storage data according to the calculation and analysis result; uploading the cloud storage data to a cloud storage, and locally storing the local storage data;
and the block chain module is used for uploading key data in the data acquired by the equipment communication module and/or the characteristic value of the key data to a block chain publishing node in a block chain, and the block chain publishing node executes uplink.
Preferably, the message queue module is provided with a message queue service process;
the message queue service process comprises a message queue switch and a message queue, the message queue switch analyzes and cleans data acquired from the equipment communication module and forwards the processed data to the message queue, and the message queue transmits the data to the data calculation module and the block chain module.
The technical scheme provided by the application can comprise the following beneficial effects:
the method comprises the steps that based on a mode of combining union links and edge computing in block links, a multi-level block link network node structure is used, and after data from a resource layer are obtained by edge nodes, the data are divided into local storage data and cloud storage data; uploading the cloud storage data to a cloud storage, and locally storing the local storage data; and uploading key data in the data and/or characteristic values of the key data to a block chain publishing node in a block chain, and performing uplink by the block chain publishing node. Through the processing, the edge node does not need to pay attention to the data condition of the whole block chain, and the data storage and calculation pressure of the edge node and the block chain publishing node in the block chain is reduced; in addition, a storage mode of the data is a storage framework of a block chain, an edge node and a cloud, the characteristic values of key data and/or data are stored on the block chain, and original data files are stored on the cloud and the edge node, so that the storage capacity of the block chain can be expanded, and the consensus efficiency is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The foregoing and other objects, features and advantages of the application will be apparent from the following more particular descriptions of exemplary embodiments of the application, as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts throughout the exemplary embodiments of the application.
Fig. 1 is a schematic flowchart illustrating a block chain-based data collaboration method according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of an edge node apparatus according to an embodiment of the present application;
FIG. 3 is a schematic structural diagram of a data collaboration system according to an embodiment of the present application;
fig. 4 is another schematic flowchart of a data collaboration method based on a blockchain according to an embodiment of the present application;
FIG. 5 is a schematic diagram illustrating a flow of work processing in the data collaboration system according to an embodiment of the present application;
fig. 6 is a schematic flow chart illustrating a flow of stream data processing in the data collaboration system according to the embodiment of the present application;
FIG. 7 is a schematic workflow diagram of a message queue service process according to an embodiment of the present application;
fig. 8 is a flowchart illustrating a data uplink operation according to an embodiment of the present application.
Detailed Description
Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While embodiments of the present application are illustrated in the accompanying drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It should be understood that although the terms "first," "second," "third," etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.
In the related art, each edge node needs to store a complete data book, which results in a considerable storage burden on the edge node, and in addition, all IOT data is stored on a block chain, which results in a lack of scalability of the system. In view of the above problems, the present application provides a data coordination method based on a block chain, which can greatly reduce storage overhead and enhance system scalability.
The technical solutions of the embodiments of the present application are described in detail below with reference to the accompanying drawings.
Referring to fig. 1, the present application provides a data coordination method based on a block chain, including the following steps:
and step S10, acquiring data from the resource layer.
The edge node device (edge node for short) receives the data from the resource layer, and analyzes and packs the data from the resource layer. The resource layer comprises a plurality of terminal devices, and the number of the terminal devices can be 1 or more. The terminal equipment collects data, and the data can be uploaded to the edge node device after being subjected to some simple processing.
Step S20, carrying out calculation analysis on the data, and dividing the data into local storage data and cloud storage data according to calculation analysis results; and uploading the cloud storage data to a cloud storage, and locally storing the local storage data.
It should be noted that the edge node device analyzes the data, divides the data into two parts, one part is local storage data, the other part is cloud storage data, and processes the two parts of data in different ways. And for the local storage data, the local storage data is stored in an internal memory, and for the cloud storage data, the cloud storage data is uploaded into the cloud for storage.
Step S30, the key data in the data and/or the characteristic value of the key data are uploaded to the blockchain publishing node in the blockchain, and the blockchain publishing node performs uplink.
In this step, the key data in the data and/or the hash value obtained by performing hash operation on the key data may be encapsulated into a block in a block data format, and uploaded to a block chain publishing node in a block chain.
To better understand the method of the present application, please refer to fig. 2, which also provides an edge node apparatus 200, wherein the edge node apparatus 200 is also referred to as an edge node, and includes a device communication module 210, a message queue module 220, a data calculation module 230, and a block chain module 240.
The device communication module 210 is configured to establish communication with a resource layer, obtain data in the resource layer, and parse and package the data.
It should be noted that the device communication module 210 is responsible for communicating with each terminal device in the resource layer, collecting heterogeneous network data, and the device communication module 210 may parse and package the data according to a protocol, and then synchronize the data to the message queue module 220.
The message queue module 220 is configured to parse and clean the data acquired by the device communication module 210, and transmit the data to the data calculation module 230 and the block chain module 240, for example, the data may be correspondingly transmitted to the data calculation module 230 and the block chain module 240 according to a subscription rule.
It should be noted that, after the message queue module 220 receives the analyzed and packaged data, the message queue module 220 may issue the data after analyzing and cleaning, and deliver the data to the relevant function modules in the edge node device 200 for subsequent processing, and the message queue module 220 is an important system for implementing the decoupling of the data calculation module 230 and the block chain module 240 in the edge node device 200, and ensures the asynchronization of the data request. The message queue module 220 may also publish a subscription rule to ensure that data can flow to the data computation module 230 or the blockchain module 240 accordingly.
The data calculation module 230 is configured to analyze data, divide the data into local storage data and cloud storage data, and upload the cloud storage data to the cloud.
It should be noted that, after the data calculation module 230 acquires data from the message queue module 220 according to a predetermined subscription rule, the acquired data is calculated and analyzed in real time, and the data is divided into cloud storage data and local storage data according to types. The cloud storage data are important data, and the important data need to be uploaded to the cloud for storage, so that on one hand, the important data can be stored in the cloud for preventing loss, on the other hand, the problem of transmission bandwidth can be solved, and storage resources (namely storage pressure of a block chain) are improved for the block chain. The locally stored data is part of the hot data, and is stored in the memory of the data calculation module 230, so that the hot data can be called at any time.
It should be further noted that the data calculation module 230 performs data analysis and processing, deploys a time sequence database and a streaming calculation service to perform real-time calculation and analysis on data, senses environmental risks, provides an alarm function, accelerates data transmission, storage and processing, and improves the real-time performance of data reading. In addition, the data calculation module 230 may further perform data cleansing to ensure that the data provides data consistency correctly.
The blockchain module 240 uploads the key data and/or the feature value of the key data in the data acquired by the device communication module to a blockchain publishing node in the blockchain. For example, the critical data and/or the characteristic value of the critical data may be encapsulated into a block according to a specified uplink rule, and the block is uploaded to the blockchain distribution node.
It should be noted that the block chain module 240 is configured to specify a data uplink rule, and after the block chain module 240 obtains data from the message queue according to a predetermined subscription rule, the block chain module encapsulates the key data into a block, and transmits the block into the block chain for publishing. It is emphasized that the blockchain module 240 is an important functional module for separating the data uplink work from the work of the edge node, the work of data analysis, data storage and data uplink rule assignment is mainly performed by the edge node, and the data uplink is handed over to the blockchain publishing node of the blockchain to complete the processing, so that the data uplink work is separated from the work of the edge node while the data storage and calculation pressure of the edge node and the blockchain publishing node is relieved.
By the method, the data are divided into the local storage data and the cloud storage data by the edge node after the edge node acquires the data from the resource layer based on a mode of combining the alliance chain and the edge calculation in the block chain and by using a multi-level block chain network node structure; uploading the cloud storage data to a cloud storage, and locally storing the local storage data; and uploading key data in the data and/or characteristic values of the key data to a block chain publishing node in the block chain, and performing uplink by the block chain publishing node. Through the processing, the edge node does not need to pay attention to the data condition of the whole block chain, and the data storage and calculation pressure of the edge node and the block chain publishing node in the block chain is reduced; in addition, a storage mode of the data is a storage framework of a block chain, an edge node and a cloud, the characteristic values of key data and/or data are stored on the block chain, and original data files are stored on the cloud and the edge node, so that the storage capacity of the block chain can be expanded, and the consensus efficiency is improved.
Referring to fig. 3, 4, 5 and 6, an embodiment of the present invention provides a data collaboration system 10, which includes four parts of a resource layer 100, an edge node device 200, a block chain publishing node 300 and a cloud 400.
Referring to fig. 4, the data collaboration method of the present application includes:
and S401, the resource layer collects data and uploads the data to the edge node device.
The resource layer 100 is configured to collect data and upload the data to the edge node device 200.
It should be noted that the resource layer 100 may include a plurality of terminal devices, where the terminal devices are various sensor nodes, and data information can be collected through the terminal devices. For example, the terminal device may acquire data information such as temperature, humidity, air pressure, illumination, pressure, and the like. However, it should be emphasized that the terminal device only completes simple data acquisition and simple data processing tasks, and the more complex or extremely complex computing tasks need to be transferred to each function module of the data coordination system to complete, and the terminal device completes processing by requesting the edge node and receives various responses of the processing results.
Step S402, the edge node device analyzes and processes data uploaded by the resource layer, divides the processed data into local storage data and cloud storage data, uploads the cloud storage data to the cloud, and encapsulates key data and/or characteristic values of the key data in the processed data into block bodies according to a specified uplink rule and uploads the block bodies to the block chain publishing node.
The edge node device 200 is configured to analyze and process data uploaded by the resource layer 100, divide the processed data into local storage data and cloud storage data, upload the cloud storage data to the cloud 400, and package, according to a specified uplink rule, key data and/or feature values of the key data in the processed data into a block and upload the block to the block chain publishing node 300.
It should be noted that an edge node is a node closest to a user, for example, an edge node may be an edge device such as a certain computer room, a physical device, an edge network manager, a home gateway, or an IOT gateway.
In the present application, the edge node apparatus 200 includes a device communication module 210, a message queue module 220, a data calculation module 230, and a block chain module 240. The device communication module 210 is responsible for communicating with the terminal devices in the resource layer 100, collecting data, and performing operations such as parsing and packaging on the data. It should be emphasized that when the device communication module 210 communicates with the terminal device and collects data, the collected data is heterogeneous network data, and the heterogeneous network is a type of network, which is composed of computers, network devices and systems produced by different manufacturers, and in most cases operates on different protocols to support different functions or references. The terminal equipment is equipment produced by different manufacturers, and communicates with each other to perform data interaction under the condition of supporting different protocols. The device communication module 210 collects data in the terminal device to implement the convergence of heterogeneous networks.
The message queue module 220 is used for decoupling each functional module in an edge node, ensuring asynchronization of data requests and enabling data to be interacted among the functional modules in the edge node, and the message queue module 220 is a distributed real-time processing system.
The data calculation module 230 is used for analyzing and processing data, deploying a time sequence database and a streaming calculation service to calculate and analyze the data in real time, sensing environmental risks, providing an alarm function, accelerating data transmission, storage and processing, and improving the real-time performance of data reading. In addition, the data calculation module 230 may further perform data cleansing to ensure that the data provides data consistency correctly.
The blockchain module 240 is used to formulate a data uplink rule, and encapsulate data into blocks for data uplink by the blockchain 300, and the blockchain module 240 is an important function module for separating data uplink work from work of an edge node.
Step S403, the block chain publishing node receives the block uploaded by the edge node device, and performs data chaining on the block.
The blockchain publishing node 300 is configured to receive a block uploaded by the edge node apparatus 200, and perform data uplink on the block. It should be noted that the blockchain issuing node 300 is used for data uplink of encapsulated data.
Step S404, the cloud receives the cloud storage data uploaded by the edge node device for storage.
It should be noted that there is no necessary order relationship between step S403 and step S404.
The cloud 400 is configured to receive and store cloud storage data uploaded by the edge node device 200. It should be noted that the cloud 400 may be a large cloud server, and may perform AI (Artificial Intelligence) big data processing on the received and transmitted data, analyze, calculate, and store the data, so as to construct a complete digital portrait, and meanwhile, the cloud 400 may further establish a cloud management operation system to provide management for the terminal nodes and the edge nodes. The cloud 400 may also establish a cloud management operation system to provide management for the terminal nodes and edge nodes. It is emphasized that cloud 400400 is where the core data is stored, and the cloud is able to interact with the blockchain to verify the authenticity and integrity of the data.
Referring to fig. 7, in an alternative embodiment, the message Queue module 220 includes a message Queue service process Broker, where the message Queue service process Broker is composed of a message Queue switch Exchange and a message Queue, the message Queue switch Exchange cleans and filters data synchronized from the device communication module 210 according to a preset rule, and forwards the cleaned and filtered data to the message Queue, and the message Queue module 220 forwards the corresponding message Queue to the data calculation module 230 or the block chain module 240 for processing according to the subscription rule of the data calculation module 230 and the block chain module 240 respectively.
Thus, it should be noted that the message Queue module 220 includes a message Queue service process Broker, and the message Queue service process Broker is composed of a message Queue switch Exchange and a message Queue. The message Queue Exchange may perform cleaning and filtering on data according to a preset rule, divide the data into different types, and forward the data into the message Queue for storage.
It should be further noted that, as shown in fig. 7, a message Producer, that is, a terminal device corresponding to the present application, and a message Consumer, that is, a data calculation module 230 and a blockchain module 240 corresponding to the present application are also shown, the terminal device transfers the collected data (that is, data produced by the message Producer) to a message Queue switch Exchange, the message Queue switch Exchange performs cleansing filtering on the data according to a preset rule, and forwards the cleansed and filtered data to a message Queue, and finally, the data calculation module 230 and the blockchain module 240 respectively obtain corresponding data with a subscription rule of the message Queue module 220 (the data calculation module 230 and the blockchain module 240, that is, the message Consumer.
Referring to fig. 7, specifically, when the data calculation module 230 and the block chain module 240 respectively subscribe to the message queue to acquire data:
the data calculation module 230 and the block chain module 240 respectively establish connection and channel with the message Queue service process Broker, and call a function to monitor the specified message Queue through an interface provided by the message Queue module 220, a working thread in the message Queue service process Broker continuously and circularly obtains information from the message Queue, and converts the information into an event object, and triggers a corresponding event processing function, so as to push data to the data calculation module 230 or the block chain module 240 through the channel, and after the data calculation module 230 or the block chain module 240 receives the subscribed data, the data is calculated, stored, and uploaded according to respective service logic.
Thus, it should be noted that, before the data calculation module 230 acquires data from the message queue module 220, the terminal device starts to access the edge node apparatus 200, and the device communication module 210 acquires data from the terminal device according to a protocol, analyzes the data, and synchronizes the data to the message queue module 220. The message queue module 220 then parses the data, and the message queue module 220 begins aggregating the connections of the functional modules within the edge node. When the data calculation module 230 needs to acquire data from the message Queue according to the subscription rule, the data calculation module 230 establishes a connection and a channel with the message Queue service process Broker, and calls a function to monitor the specified message Queue through an interface provided by the message Queue module 220, a working thread in the message Queue service process Broker continuously and circularly acquires information from the message Queue, converts the information into an event object, triggers a corresponding event processing function, and pushes the data to the data calculation module 230 through the channel, and finally the data calculation module 230 calculates, stores and uploads the information according to the service logic.
It should be further noted that the principle flow of the blockchain module 240 acquiring data from the message queue module 200 is the same as the principle flow of the data calculation module 230 acquiring data from the message queue module 200, and is not repeated.
In an optional embodiment, when the data calculation module 230 performs real-time calculation analysis on the acquired data, the acquired data may be further divided into service flow data, where the service flow data is data that is stored or circulated by the edge calculation server.
Therefore, it should be noted that, when the data calculation module 230 performs real-time calculation and analysis on the acquired data, the data may be divided into local storage data and cloud storage data, and may also be divided into service flow data, where the service flow data is data that is handed over to the edge calculation server for storage or flow, and if necessary, the edge calculation server may conveniently store the characteristic value of the data/data on the block chain platform without additional data docking between platforms/systems.
Referring to fig. 8, in an alternative embodiment, when the block chain module 240 encapsulates data:
and packaging the key data of the acquired data and/or the characteristic value of the key data into a block body in a block data format, and uploading the block body to the block chain publishing node 300. The characteristic value of the critical data may be a hash value.
Therefore, it should be noted that, in the present application, the hash value of the key data is stored in the blockchain publishing node 300, the hash value obtained by calculation may be compared with the stored hash value after the data of the block is subsequently acquired, the integrity of the data is verified based on the unidirectionality and the collision resistance of the hash algorithm, and the storage burden of the node is effectively reduced. In an alternative embodiment, after the node 300 issues the blockchain, the block uploaded and encapsulated by the blockchain module 240 is uplink data after adding the header information.
As described above, the header information is representative feature data in the block body, and the addition of the header information can play a role of tracing back, and since only the representative feature data is stored in the block body, the storage pressure of the block chain can be greatly reduced. Specifically, the header information may include one or a combination of a version number, a parent hash, a merkel root, a timestamp, a difficulty value, and a random number. Of course, the header information may include other types of header information in addition to the above.
The technical solution of the present application is described below with reference to a specific application scenario:
in a student health management system (namely, a corresponding data collaboration system) in colleges and universities, student health data are acquired through terminal equipment (the terminal equipment can be a mobile phone, an intelligent bracelet, a temperature sensor and the like) of a resource layer, and the student health data comprise heart rate, heart oxygen, sleep quality, exercise capacity, body temperature and the like. And then, the equipment communication module of the edge node device establishes communication with terminal equipment such as a mobile phone, an intelligent bracelet and a temperature sensor, so as to acquire student health data, and analyze and package the student health data. The message queue module issues subscription rules in real time, the data calculation module and the block chain module acquire related student health data from the message queue module according to the subscription rules of the data calculation module and the message queue module, the data calculation module subscribes, analyzes and processes the student health data, the student health data are divided into local storage data and cloud storage data, the local storage data are stored by a local storage, and the cloud storage data are uploaded to the cloud storage. The cloud end can carry out big data analysis and calculation on the data to form a student health file and construct a complete digital portrait. The block chain module subscribes personal privacy information of student health, encrypts and packages the block data into block data, and then delivers the block data to a block chain publishing node in the block chain to complete data chaining.
According to the application example, the technical scheme of the application can find that the edge node does not need to pay attention to the data condition of the whole block chain due to the adoption of a multi-level block chain network node structure, and the data storage and calculation pressure of the edge node and the block chain release node is reduced; in addition, a storage mode of the data is a storage framework of a block chain, an edge node and a cloud, key original data and characteristic values of the data are stored on the block chain, original data files are stored on the cloud and the edge node, and therefore storage capacity of the block chain can be expanded, and consensus efficiency is improved.
Having described embodiments of the present application, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (10)
1. A data collaboration method based on a block chain is characterized by comprising the following steps:
acquiring data from a resource layer;
performing calculation analysis on the data, and dividing the data into local storage data and cloud storage data according to calculation analysis results; uploading the cloud storage data to a cloud storage, and locally storing the local storage data;
and uploading key data in the data and/or characteristic values of the key data to a block chain publishing node in a block chain, and performing uplink by the block chain publishing node.
2. The method according to claim 1, wherein the uploading key data in the data and/or characteristic values of the key data to a blockchain publishing node in a blockchain comprises:
and packaging key data in the data and/or a hash value obtained by carrying out hash operation on the key data into a block in a block data format, and uploading the block to a block chain release node in a block chain.
3. The method of claim 1, wherein prior to said computationally analyzing said data, further comprising:
and analyzing and cleaning the data.
4. The method of claim 2, wherein the performing uplink by the blockchain issuing node comprises:
and after the block chain release node adds the set head information to the encapsulated block, performing uplink.
5. The method of claim 4, wherein:
the header information includes at least one of: version number, parent hash value, merkel root, timestamp, difficulty value, and random number.
6. An edge node device is characterized by comprising an equipment communication module, a data calculation module and a block chain module;
the equipment communication module is used for acquiring data from a resource layer;
the data calculation module is used for calculating and analyzing the data acquired by the equipment communication module and dividing the data into local storage data and cloud storage data according to the calculation and analysis result; uploading the cloud storage data to a cloud storage, and locally storing the local storage data;
the block chain module is configured to upload key data in the data acquired by the device communication module and/or a feature value of the key data to a block chain publishing node in a block chain, and the block chain publishing node performs uplink.
7. The edge distribution apparatus of claim 6, further comprising:
and the message queue module is used for analyzing and cleaning the data acquired by the equipment communication module.
8. A data collaboration system based on a block chain is characterized by comprising a resource layer, an edge node device, a block chain publishing node and a cloud end;
the resource layer is used for acquiring data and uploading the data to the edge node device;
the edge node device is used for acquiring data from the resource layer; performing calculation analysis on the data, and dividing the data into local storage data and cloud storage data according to calculation analysis results; uploading the cloud storage data to a cloud storage, and locally storing the local storage data; uploading key data in the data and/or characteristic values of the key data to a block chain publishing node in a block chain;
the block chain publishing node is configured to receive key data in the data uploaded by the edge node device and/or a feature value of the key data, and perform data uplink;
and the cloud end is used for receiving and storing the cloud end storage data uploaded by the edge node device.
9. The blockchain-based data collaboration system of claim 8 wherein the edge node means comprises:
the equipment communication module is used for acquiring data from a resource layer;
the message queue module is used for analyzing and cleaning the data acquired by the equipment communication module;
the data calculation module is used for calculating and analyzing the data processed by the message queue module and dividing the data into local storage data and cloud storage data according to the calculation and analysis result; uploading the cloud storage data to a cloud storage, and locally storing the local storage data;
and the block chain module is used for uploading key data in the data acquired by the equipment communication module and/or the characteristic value of the key data to a block chain publishing node in a block chain, and the block chain publishing node executes uplink.
10. The blockchain-based data collaboration system as claimed in claim 9 wherein:
the message queue module is provided with a message queue service process;
the message queue service process comprises a message queue switch and a message queue, the message queue switch analyzes and cleans data acquired from the equipment communication module and forwards the processed data to the message queue, and the message queue transmits the data to the data calculation module and the block chain module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110789801.0A CN113535850A (en) | 2021-07-13 | 2021-07-13 | Data collaboration method, device and system based on block chain |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110789801.0A CN113535850A (en) | 2021-07-13 | 2021-07-13 | Data collaboration method, device and system based on block chain |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113535850A true CN113535850A (en) | 2021-10-22 |
Family
ID=78127672
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110789801.0A Pending CN113535850A (en) | 2021-07-13 | 2021-07-13 | Data collaboration method, device and system based on block chain |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113535850A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114726877A (en) * | 2022-03-18 | 2022-07-08 | 扬州大学 | Block chain-based data storage updating method under edge calculation |
| CN115391098A (en) * | 2022-08-25 | 2022-11-25 | 北京有竹居网络技术有限公司 | Big data analysis method and device, edge node and cloud server |
| CN115460280A (en) * | 2022-08-31 | 2022-12-09 | 云南电网有限责任公司信息中心 | Main side chain data interaction system, method, equipment and medium based on cloud edge cooperation |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112163045A (en) * | 2020-10-29 | 2021-01-01 | 杭州谐云科技有限公司 | Block chain-based edge data secure storage system |
| CN112463881A (en) * | 2020-11-10 | 2021-03-09 | 南京邮电大学 | Efficient block chain system and processing method for cloud edge aggregation computing environment |
| CN112600892A (en) * | 2020-12-07 | 2021-04-02 | 北京邮电大学 | Block chain equipment and system for Internet of things and working method |
| CN113065961A (en) * | 2021-03-22 | 2021-07-02 | 江苏派智信息科技有限公司 | Power block chain data management system |
-
2021
- 2021-07-13 CN CN202110789801.0A patent/CN113535850A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112163045A (en) * | 2020-10-29 | 2021-01-01 | 杭州谐云科技有限公司 | Block chain-based edge data secure storage system |
| CN112463881A (en) * | 2020-11-10 | 2021-03-09 | 南京邮电大学 | Efficient block chain system and processing method for cloud edge aggregation computing environment |
| CN112600892A (en) * | 2020-12-07 | 2021-04-02 | 北京邮电大学 | Block chain equipment and system for Internet of things and working method |
| CN113065961A (en) * | 2021-03-22 | 2021-07-02 | 江苏派智信息科技有限公司 | Power block chain data management system |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114726877A (en) * | 2022-03-18 | 2022-07-08 | 扬州大学 | Block chain-based data storage updating method under edge calculation |
| CN115391098A (en) * | 2022-08-25 | 2022-11-25 | 北京有竹居网络技术有限公司 | Big data analysis method and device, edge node and cloud server |
| CN115460280A (en) * | 2022-08-31 | 2022-12-09 | 云南电网有限责任公司信息中心 | Main side chain data interaction system, method, equipment and medium based on cloud edge cooperation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113535850A (en) | Data collaboration method, device and system based on block chain | |
| CN112099948B (en) | Method for standardizing digital twin manufacturing unit protocol and integrating industrial big data in real time | |
| CN109167829B (en) | Internet of things communication method and cloud gateway | |
| CN107390650B (en) | A kind of data collection system based on Internet of Things and the data compression method based on the system | |
| Tse et al. | Global zoom/pan estimation and compensation for video compression | |
| CN111131379B (en) | Distributed flow acquisition system and edge calculation method | |
| WO2020147336A1 (en) | Micro-service full-link monitoring system and method | |
| CN108471366A (en) | A kind of stereoscopic monitoring system of facing cloud native applications | |
| CN107463582A (en) | The method and device of distributed deployment Hadoop clusters | |
| CN106656846A (en) | Construction method of coordination layer in software defined network (SDN) architecture | |
| CN109831524A (en) | A kind of load balance process method and device | |
| CN108833267A (en) | Intelligent industrial gateway | |
| CN107979498B (en) | Mesh network cluster and large file transmission method based on cluster | |
| CN114401284B (en) | Real-time data acquisition and transmission system and method for fixed pollution source treatment working condition | |
| CN107067325A (en) | Filter method, filter and the supplying system of stock market data | |
| CN108881817A (en) | A kind of methods, devices and systems that data are synchronous | |
| CN114945032B (en) | Electric power internet of things terminal data access system, method, device, equipment and medium | |
| Wang et al. | Performance modeling and suitability assessment of data center based on fog computing in smart systems | |
| CN114926163A (en) | Resource cross-chain transfer method and device | |
| CN110502391A (en) | The grasping means and system of SQL information in a kind of MySQL database | |
| CN102655480B (en) | Similar mail treatment system and method | |
| CN107181701B (en) | The collection method and device of common gateway interface data | |
| CN112883110A (en) | Terminal big data distribution method, storage medium and system based on NIFI | |
| CN100547560C (en) | A kind of computers group monitoring and method | |
| Sang et al. | High-performance cluster computing over gigabit/fast ethernet |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |