CN112511515B

CN112511515B - Chain number cube for data chaining

Info

Publication number: CN112511515B
Application number: CN202011307270.9A
Authority: CN
Inventors: 汪林川; 王梦雪; 张�浩
Original assignee: Chengdu Wuyou Blockchain Technology Co ltd
Current assignee: Chengdu Wuyou Blockchain Technology Co ltd
Priority date: 2020-11-19
Filing date: 2020-11-19
Publication date: 2023-03-10
Anticipated expiration: 2040-11-19
Also published as: CN112511515A

Abstract

The invention relates to the technical field of block chain information sharing, and particularly discloses a chain number cube for data chaining, wherein the functions of data encryption, data storage, data chaining and data access are realized through the chain number cube; the chain number cube comprises a data directory, a search module, a data lockbox, an intelligent contract, an ETL execution engine, a data lens, a chaining engine and a blockchain edge cloud node. The user can not only realize the encrypted storage of the local original data, but also carry out the uplink storage of the data information through the chain number cube, and particularly, the uplink storage is carried out on the ETL process data imported and exported by the data, so that the traceability of the data circulation is realized.

Description

Chain number cube for data chaining

Technical Field

The invention relates to the technical field of block chain information sharing, and particularly discloses a chain number cube for data uplink.

Background

The data exchange system is widely established in each field and each layer, interconnection and intercommunication of data are realized, wide sharing is realized, a multi-layer data exchange system with a management node, a core exchange node, a virtual front node and a database node is established, access specifications and various data sharing and exchange modes are provided for heterogeneous and homogeneous systems, data exchange of cross-department, cross-region, cross-service, multi-layer and multi-service application domains is realized, and support and service are provided for cooperation and interaction among all departments, among all layers and among all service application systems.

Generally, a data exchange system can utilize a tamper-resistant mechanism of a block chain, and is widely applied to various fields, and correspondingly, a data uplink server needs to be equipped, and a data uplink method is provided in a patent "user operation data uplink method and system based on a service data block chain" (chinese patent application CN 202010270234.3): the block chain node receives a user operation data uplink request sent by a server; the user operation data is the operation of a user on service data, and the user operation data uplink request is the user data uplink request sent by the server receiving user equipment; storing user operation data through a chain structure of the block, and storing a state after the user operation by using a tree structure; and sending the user operation data uplink response to the server so that the server can send the user operation data uplink response to the client. The server is arranged between the client and the block chain link points, the server can store user operation data, and the user operation data can be sent to the block chain link points for multiple times, so that the problem of user operation data loss is avoided, and the uplink of the user operation data is ensured.

In summary, to construct a multi-level data switching system capable of implementing data uplink, the required devices are various, and the devices need to be configured, including but not limited to servers for storing data, servers for data transmission management, servers for data uplink, clients for data uplink, switches, virtual front-end nodes, and so on. Each device has single function, the system is complex to construct, and the system integration difficulty is high.

Disclosure of Invention

The invention aims to solve the problems that each device in the multi-level data exchange system capable of realizing data uplink has single function, the system is complex to construct and the system integration difficulty is high. A chain number cube for data chaining is proposed.

In order to achieve the above purpose, the invention provides the following technical scheme:

a chain number cube for data chaining realizes the functions of data encryption, data storage, data chaining and data access through the chain number cube;

the chain number cube comprises a data directory, a search module, a data box, an intelligent contract, an ETL execution engine, a data lens, a chaining engine and a block chain edge cloud node;

the data directory stores the accessed data field of the chain number cube, and is used for providing access for other chain number cubes;

the searching module is used for accessing data directories of other chain number cubes in the system;

the data cipher box is used for encrypting the data imported into the chain number cube and storing the encrypted data;

the intelligent contract is used for the data provider and the data buyer to reach an agreement and realize the data transaction;

the ETL execution engine is used for synchronizing external data into the chain number cube in an extraction, conversion and loading mode and/or synchronizing data in the chain number cube into other chain number cubes in an extraction, conversion and loading mode;

the data lens is used for decrypting external access, and the data stored in the data encryption box can only be accessed through the data lens;

the uplink engine is used for storing ETL process information of the imported data and/or ETL process information of the distributed data into the block chain ledger;

the block chain edge cloud node stores a block chain account book, and the ETL process information of each chain number cube in the system is stored in the block chain account book.

As a preferred embodiment of the present invention, the specific steps of simultaneously implementing the functions of data encryption, data storage and data chaining by the chain number cube include:

a1, an ETL execution engine synchronously imports original data into a data cipher box in an ETL mode, and in the import process, the data cipher box encrypts the original data to obtain encrypted data and stores the encrypted data into the data cipher box;

and A2, the uplink engine imports the original data into ETL process information of the data cipher box for uplink storage, and stores the ETL process information into a block chain account book.

As a preferred aspect of the present invention, the ETL process information and the encrypted data are in a one-to-one correspondence relationship, and the ETL process information includes, but is not limited to, the following: the hash value of the time stamp, the source database name, the source table name, the target database name, the target table name and the original data of the operation.

In a preferred embodiment of the present invention, the data cipher box encrypts the data using both AE and TDE, and the data lens decrypts the encrypted data using both AE and TDE.

As a preferred aspect of the present invention, when the data of the chain number cube is in abnormal access, the data lens is further configured to fuse the external access link, and the factors triggering the data lens to perform the fusing operation include: data volume over-query, misuse of SQL frequency or full table query.

The invention further comprises an analysis module, wherein the analysis module is used for carrying out forward tracing or backward tracing on the data source of the data stored in the local data lockbox, and the forward tracing or backward tracing information is realized through ETL process information in the block chain ledger.

Based on the same concept, an apparatus for a chain number cube for data chaining is also provided, including the chain number cube of any one of the above; the data encryption device further comprises a JDBC or ODBC interface, and data are synchronized into the chain number cube device through the JDBC or ODBC interface, so that the functions of data encryption, data storage, data uplink and data access are realized; the system also comprises a JDBC or ODBC interface, and data are synchronized in the device of the chain number cube through the JDBC or ODBC interface, so that the functions of data encryption, data storage, data uplink and data access are realized.

Compared with the prior art, the invention has the beneficial effects that:

1. the invention discloses a chain number cube for data chaining, which comprises a data directory, a search module, a data box, an intelligent contract, an ETL execution engine, a data lens, a chaining engine and a block chain edge cloud node. The user can not only realize the encrypted storage of the local original data, but also carry out the uplink storage of the data information through the chain number cube, and particularly, the uplink storage is carried out on the ETL process data imported and exported by the data, so that the traceability of the data circulation is realized. The user can join the distributed block chain data circulation network only through the chain number cube without purchasing other equipment, and the use is simple and convenient.

2. A user directly links local original data into a cabinet through a chain number cube, all data can be completed in a chain form no matter in future inquiry or use, and the like, and the data monitorability is guaranteed. The data entering the chain number cube is encrypted end to end, and cannot be modified by any user; the data owner has absolute control right and recovery and destruction rights on the data which are linked into the cabinet, and meanwhile, according to a fusing protection mechanism during data use, the data are prevented from being used maliciously, so that the problem of data double-flower (double-flower problem, namely double payment, which means the phenomenon that one piece of digital cash is reused in a transaction) is effectively prevented.

Description of the drawings:

fig. 1 is an architecture diagram of a big data management application system based on a chain number cube in embodiment 1 of the present invention;

fig. 2 is a functional architecture diagram of a chain number cube for data uplink in embodiment 1 of the present invention;

fig. 3 is a schematic diagram of a technical architecture in embodiment 1 of the present invention;

fig. 4 is a first physical architecture of the system according to embodiment 1 of the present invention;

fig. 5 is a second physical architecture of the system according to embodiment 1 of the present invention;

fig. 6 is a typical screenshot of a data directory in embodiment 1 of the present invention;

FIG. 7 is an exemplary search module interface diagram according to embodiment 1 of the present invention;

FIG. 8 is a flow chart of an exemplary transaction of a smart contract according to embodiment 1 of the present invention;

FIG. 9 is a schematic diagram of the working principle of an ETL execution engine in embodiment 1 of the present invention;

fig. 10 is a diagram illustrating the content of ETL process information in embodiment 1 of the present invention;

fig. 11 is a schematic diagram of a user implementing data encryption, data storage, and data chaining through the chain number cube according to embodiment 1 of the present invention;

FIG. 12 is a Delta data message standard diagram in embodiment 1 of the present invention;

fig. 13 is a diagram illustrating an example of the application of a data message in embodiment 1 of the present invention;

fig. 14 is a management interface diagram of a chain number cube (safe) in embodiment 1 of the present invention;

FIG. 15 is a diagram of an unstructured-data management interface in embodiment 1 of the present invention;

fig. 16 is a diagram of a data locker management interface in embodiment 1 of the present invention;

FIG. 17 is a data application management interface diagram in embodiment 1 of the present invention;

fig. 18 is an interface diagram of a data open management unit in embodiment 1 of the present invention;

fig. 19 is a diagram of a data locker access function interface in embodiment 1 of the present invention;

FIG. 20 is a view showing an interface of a personal console in embodiment 1 of the present invention;

fig. 21 is a diagram of a real-time network monitoring interface in embodiment 1 of the present invention;

FIG. 22 is a diagram of an intelligent posture monitoring interface in embodiment 1 of the present invention;

FIG. 23 is a diagram illustrating state monitoring of a chain number cube in embodiment 1;

FIG. 24 is a data redirection perception diagram in accordance with example 1 of the present invention;

FIG. 25 is a data asset transition map in embodiment 1 of the present invention;

FIG. 26 is a large screen diagram of the data asset security supervision in embodiment 1 of the present invention;

fig. 27 is a diagram of data exchange sharing monitoring in embodiment 1 of the present invention;

FIG. 28 is an ETL management interface diagram according to embodiment 1 of the present invention;

FIG. 29 is a schematic diagram illustrating a data lens fuse configuration in accordance with embodiment 1 of the present invention;

FIG. 30 is a diagram of the components of a chain-number cube PaaS layer in example 2 of the present invention;

fig. 31 is a hardware configuration diagram of a chain-number cube in embodiment 2 of the present invention.

Detailed Description

The present invention will be described in further detail with reference to test examples and specific embodiments. It should be understood that the scope of the above-described subject matter is not limited to the following examples, and any techniques implemented based on the disclosure of the present invention are within the scope of the present invention.

Example 1

An architecture diagram of a big data management application system based on a chain number cube is shown in fig. 1, and includes a plurality of chain number cubes, each chain number cube establishes a bridge connection with a local database, a user imports local original data into the chain number cube in an ETL synchronization mode, encrypts and stores the imported chain number cube, and in addition, uplink of data ETL process information is also realized, and data interaction is performed between the chain number cubes through data messages. Each chain number cube includes functional modules such as a data directory, a search module, a data box, an intelligent contract, an ETL execution engine, a data lens, a chaining engine, and a block chain edge cloud node, and a functional architecture diagram of the chain number cube for data chaining is shown in fig. 2.

The overall architecture of the system takes a BD-WAN (block defined network) and a chain number cube as bottom-layer physical network support, integrates chain number search (data sharing), chain number intelligence (data visual detection), WBaaS (block chain service platform), and software products of an operation management background (basic data and system management), and all data are circulated and exchanged based on a trusted data environment. The system adopts a springclosed-based cloud native micro-service architecture as a technical architecture, and a schematic diagram of the technical architecture is shown in fig. 3. The server of the system comprises a portal, a chain number search, an execution engine, an uplink engine, a chain number intelligence, an operation background and vbaas. The server is linked with the service database and used for accessing mysql data and the elasticsearch data in the service database, and the server reads data in the data safety cabinet through the data lens; in addition, the servers are linked with the peer node, the orderer node and the ca node through special gateways. Accordingly, the physical architecture of the system is shown in fig. 4 and 5.

The system adopts a distributed structure to carry out networking, has easy expandability, can transversely expand an execution engine and a network, and adopts a BD-WAN block chain to define the network.

An important component of the system architecture is the chain number cube. The user can link the local original data into the cabinet through the chain number cube and perform data circulation with other chain number cubes. The uplink is ETL process information, and the entering into the cabinet means that original data is stored into a data encryption box of a chain number cube after being encrypted. After the hardware network is connected to the system, original data is not directly uplink, uplink is ETL process data of the original data, ETL process information is stored in a distributed account book of the block chain, and the original data is stored in a local data cipher box after being encrypted.

Each chain number cube includes a data directory, a search module, a data lockbox, a smart contract, an ETL execution engine, a data lens, a chaining engine, and a blockchain edge cloud node.

The data directory stores fields of accessible data of the chain number cube, and a user can select to display all the fields corresponding to the data in the local data cipher box in the data directory through setting, or not display all the fields, or display all the fields in the data directory. The information stored in the data directory is used as data share and can be accessed by the chain number cube in other systems. One established screenshot of the data directory is shown in fig. 6. The list comprises a self-defined data set name, a chain number cube where the data are located, a data close box where the data set is located, a state, the number of applied times, the issuing time and the operation. Wherein the administrator may set the data set as retrievable or not retrievable, and when selected as retrievable, other chain number cube users in the network may search the data set through a search function. The applied times refer to the times that the data set is applied for access or export by other chain number cubes.

The search module is used for accessing data directories of other chain number cubes in the system. Before all data in the chain number cube are shared and exchanged, a user can perform high-level retrieval on the data in a simple and easy-to-use search engine mode through a search module, and a data directory is accessed to realize data awareness. A user needs to search target data in an engine first and then exchange and apply for the data; the traditional data directory index is replaced by a quick search mode, so that the data can be more conveniently known, and a typical search module interface is shown in FIG. 7. Through keyword search, a list of data sets in the network matching the keyword can be found, and the name of the chain number cube (data provider) where each data set in the list is located, the update date, the data description, the charging standard and the content of the data can be seen.

The data cipher box is used for encrypting the data and storing the encrypted data. The encryption adopts end-to-end constant encryption AE and TDE transparent encryption. The end-to-end constant encryption is an asymmetric encryption technology, and the transparent encryption is a symmetric encryption technology. In the local chain number cube, a plurality of data close boxes can be arranged according to requirements, and data in a data set can come from different data close boxes of the same chain number cube.

The intelligent contract is used for the data provider and the data buyer to reach an agreement and realize the data transaction. In the field of blockchain technology, smart contracts refer to computer programs that are triggered, non-tampered, and automatically executed based on a predetermined event. During the data transaction process, based on the trusted and non-tampered data transaction information, an automatically executable program (smart contract) can be deployed on the blockchain, and the coverage range of the automatically executable program comprises a programming language, a compiler, a virtual machine, time, a fault tolerance mechanism and the like. The data asset supplier issues the data asset in the data asset trading platform based on the block chain to create an intelligent contract; triggering an intelligent contract through a buyer, and purchasing and paying to complete a transaction; and in the data asset transaction process, the uplink of data asset information, the uplink of data asset transaction users, the uplink of data asset transaction information and distributed storage are synchronously performed to each endpoint function in real time. In the technical scheme of the invention, the data transmission in the system is managed on the basis of the operation mechanism of the intelligent contract.

A typical transaction flow diagram for a smart contract is shown in fig. 8. The blockchain data asset transaction flow diagram illustrates:

1) The 'data asset supplier' issues 'data assets', firstly, the 'data assets' is registered, an intelligent contract is created, a 'data auditor' is waited for auditing, and the intelligent contract is put on shelf and written into a block chain after passing;

2) The data asset purchasing party purchases data assets, browses a data asset catalogue, selects required data, creates an order, triggers an intelligent contract, completes payment and writes transaction information into a block chain;

3) The intelligent contract comprises a data asset use role, time, times, fields and other limiting algorithms.

The ETL execution engine is used for extracting, converting and loading data. Importing external data into the chain number cube and exporting the data of the chain number cube are realized by adopting an ETL execution engine. Inputting data in the relational database, data in the nosql database, data on real-time stream, file stream and http into an ETL execution engine for ETL conversion, wherein the ETL conversion content comprises the following steps: data connection, data conversion, data desensitization, data encryption, data mapping, data statistics, data verification, data cleaning and the like. The working principle diagram of the ETL execution engine is shown in fig. 9. The data generated after the ETL conversion is completed is encrypted and stored in a data lockbox, and the whole ETL process information is loaded into the block chain through a data uplink engine entry message queue. The ETL process information includes a timestamp of the current operation, a source database name or a source data address, a source table name, a target database name or a target database address, a target table name, a hash value of the current operation data, the number of fields, the number of pieces, and the like, and the content of the ETL process information is shown in fig. 10, for example. When the ETL process information is uplinked, it is clear that the source and destination of the batch of data are recorded on the blockchain distributed ledger.

In particular, during data import and export, the ETL execution engine checks the data that performs the operation to ensure that the data has not been tampered with before import and export.

The data lens is used for decrypting external access, and the data stored in the data encryption box can only be accessed through the data lens. Besides end-to-end asymmetric technology encryption, data in the data cipher box is also transparently encrypted by a symmetric technology, and a data user needs to preview or use the data in a limited environment, so that data leakage accidents are prevented. Meanwhile, data API sharing and library-level circulation are supported, and users can perform table association, table-level operation and the like in an encrypted database through authorization of a data lens. Because the data cipher box adopts end-to-end constant encryption AE plus TDE transparent encryption, the corresponding data lens decrypts the encrypted data through AE and TDE. Node key management based on TEE (trusted execution environment) and whole-course encrypted data transmission are adopted, the safety of data is guaranteed, and only a data owner or an approved data applicant calls a data lens through a unique key of the data owner or the approved data applicant, the data can be previewed and used, so that the data is safer and more secret; meanwhile, in the calling process of the data lens, a related fusing mechanism is arranged to prevent behaviors such as illegal calling of data and the like.

The uplink engine is used for storing the ETL process information of the stored data or the ETL process information of the distributed data into the block chain account book.

The block chain edge cloud node stores a block chain account book, and the block chain account book is used for storing ETL process information of each chain number cube in the network.

Preferably, the user realizes data encryption, data storage and data uplink through the chain number cube. A schematic diagram of implementing data encryption, data storage, and data chaining by a chain number cube is shown in fig. 11, and the specific steps include:

a1, an ETL execution engine synchronously imports original data into a data cipher box in an ETL mode, and in the importing process, the data cipher box encrypts the original data to obtain encrypted data and stores the encrypted data into the data cipher box;

a2, the uplink engine imports the original data into ETL process information of a data box to carry out uplink operation, and stores the generated block chain account book into a block chain edge cloud node;

the ETL process information is in a one-to-one correspondence with the encrypted data, and the ETL process information includes, but is not limited to, the following: the operation comprises a timestamp, a source database name, a source table name, a target database name, a target table name and a hash value of the original data.

And the user realizes data access through the chain number cube, and the data access specifically comprises three modes of local access, data distribution and intelligent contract access.

The specific steps of local access include: and the local user directly decrypts the encrypted data in the data encryption box through the data lens in the own chain number cube and checks the decrypted original data.

The data distribution method specifically comprises the following steps:

c1, a user accesses data directories of other chain number cubes in the system through a search module in the access end chain number cube, and obtains data information and a target chain number cube where the data information is located from the data directories;

c2, establishing point-to-point link between the target chain number cube and the access end chain number cube through a handshake protocol of the data message;

c3, the access end chain number cube decrypts the encrypted data in the data encryption box through the data lens of the target chain number cube to obtain data detail corresponding to the data information;

and C4, the target chain number cube distributes data details corresponding to the data information to the access terminal chain number cube through the ETL execution engine, and the uplink engine performs uplink storage on the ETL process information of the data details distribution.

The specific steps of intelligent contract access comprise:

b1, accessing data directories of other chain number cubes in the system by a user through a search module in the access end chain number cube, and acquiring data information and a target chain number cube where the data information is located from the data directories;

b2, after mutual verification is carried out on the intelligent contract in the access end chain number cube and the intelligent contract in the target chain number cube by the user, a point-to-point link is established;

and B3, the access end chain number cube decrypts the encrypted data in the data encryption box through the data lens of the target chain number cube, and checks the data detail corresponding to the data information.

Another important component in the system is a data message, and data interaction is realized among chain number cubes through the data message.

The data distribution is performed through ETL operation, but a plurality of mature ETL tools such as ODI, informatica, datax, kettle and the like exist in the market, different scripting languages used by different tools are different, codes are different, if only one tool can be used for analysis, the ETL tool cannot be fully utilized, but the workload is large when all scripts are analyzed. Therefore, a set of standard models is constructed, different types of ETL operation can be analyzed into a unified model, and networking transmission among chain number cubes is facilitated. Before data distribution, a handshake protocol carries out identity authentication of both sides, and both sides of data distribution are confirmed to be chain number cube main bodies. After the chain number cube is deployed, the chain number cube network has a unique number, whether the IP is in the chain number cube network or not is judged during identity verification, and communication connection is established after the verification is passed. Delta messages specify the contents of exchange data format, data type, expression mode and the like, and contain the specification of a communication model. The message consists of a message header and a message body, wherein the message header comprises some basic information, such as a message unique identification number, a reference message number, a resource version, a source node code, a target node code, basic operation, composite operation and the like. The Delta data message standard is shown in FIG. 12. The data message comprises a source node, a target node and an operation node; the source node is a node for accessing data, the target node is a node for accessing data, the operation node comprises basic operation and composite operation, and the basic operation comprises extraction, loading, filtering, conversion and duplication removal; compound operations include split, merge, join, associate, and custom. Fig. 13 shows an example of the application of the Delat datagram.

The above description has been made on the system architecture and the components, and since the system has the above architecture and components, the system has the following typical functions:

1. search engine

All data sharing and exchanging in the system are based on a search module, and are more in line with the use habits of users. A user needs to search target data in an engine first to exchange and apply for the data; the traditional data directory index is replaced by a quick searching mode, and the data can be known more conveniently. The search engine supports multiple conditions to meet the query; and the method supports the description information preview of the open data set, and the like.

2. Chain number cube (safe) management

The chain number cube management interface is shown in FIG. 14. The chain number cube mainly provides a function of encrypting and safely storing data, and all data entering the cube needs to be previewed or used under a limited condition, so that data leakage accidents are prevented. The safe comprises two types: a third-party cloud server cube and a chain number cube (all-in-one machine) type safe; and the user purchases the safe according to the actual requirements of the user. Enterprise users may assign purchased cube management rights to other members within the enterprise, enabling collaboration within the organization to operate more efficiently. For a user who purchases the third-party cloud server cube, the user can apply for the function of upgrading configuration on line, and after upgrading is completed, prompting is carried out in the cube state.

3. Unstructured data management

The unstructured data management interface is shown in fig. 15, and the chain number also supports the storage capacity of unstructured data in addition to the processing capacity of structured data. The method has the advantages that unstructured data are stored in a HDFS cluster distributed storage mode, and the method has the advantages of being high in fault tolerance, computing efficiency, access convenience, reliability and the like; the supported file types comprise compressed packets, MS documents, pictures, videos, audios and the like; and a batch uploading function is supported.

4. Data in cabinet management

Data in-cabinet management interface as shown in fig. 16, a user can load local data directly into a cube through a cabinet in function. The system will automatically generate the corresponding ETL job according to the connection information input by the user. The system supports the way and frequency of synchronizing the data, and if the chosen way is periodic synchronization, the system will incrementally transfer the data into the cube. The cache record of the connection information of multiple databases is supported, and a user can enter the cabinet through the record without repeated operation. The granularity of the data synchronization function supports the table level, and the condition that the whole database is synchronized and occupies too many resources is avoided.

5. Data application management

As shown in fig. 17, the data application management interface exchanges data in an application manner after the data requesting party finds the target data, and the data requesting party can access the data after agreeing to apply the data. For the agreed application, there are two ways in which the system provides access: through authorization, entering a cube of a data owner for data query; or directly transfer the owner's data to the applicant's cube in the manner of ETL. During the service handling process, the relevant users receive message prompts from the system. And supporting the inquiry of the details of the applicant and the application data set.

6. Data open management

As shown in fig. 18, the data opening management unit interface is configured such that data entering the chain number cube can be searched by other users only after the opening operation is completed, and a service of data exchange is completed. The granularity for open data reaches the field level. The sharing mode may be set personalizedly, such as one-time synchronization or periodic synchronization, whether to charge a fee, whether to allow resale, etc.

7. Data in cabinet access function

The data locker access function interface is shown in fig. 19, and a user applying for data can directly call an intelligent contract and remotely enter a chain number cube of a data owner to check and calculate data. And the self-defined calling of the intelligent contract is supported, and the data owner checks the contract content. And judging the authority (data volume, query frequency, invocation of data lens and the like) of the data visitor according to the selected intelligent contract.

8. Personal console

As shown in fig. 20, the personal console interface provides overview perception of the data sharing situation of the user in a visual chart manner, including a balance situation, a consumption situation, an organization situation, a data transaction situation, a task to be handled situation, and the like.

9. Real-time network monitoring

The real-time network monitoring interface is shown in fig. 21, and supports management-type users to perform visual detection on the real-time conditions of the current network, including block-out time and transaction number, average block size of time period, node distribution map, data asset account ranking, ETL transaction statistical conditions, and node connection overview.

10. Intelligent situation monitoring

The intelligent situation monitoring interface is shown in fig. 22, and the situation of the whole operation condition of the chain data communication is sensed by five dimensions of a network, data assets, data stream conversion, ETL operation and an intelligent contract. According to the selected report type, the user can define the query condition by user and preview the report online. And supporting the downloading function of report details.

11. Key information monitoring

As shown in fig. 23, the chain number cube state monitoring supports managed setting of monitoring information, and sets an alarm notification for an object to be monitored so as to avoid occurrence of irregular data operation or abnormal user behavior. Supporting multi-target monitoring; behavior capture to support ETL jobs and data asset changes; supporting the alarm prompting mode of the mailbox and the mobile phone; and supporting the inquiry of the alarm detail.

12. Data redirection awareness

As shown in fig. 24, through data analysis on the ETL uplink, a user can clearly understand the context of data, and the system analyzes the entire data stream forwarding process in a visual scenario. The whole process analysis of data flow is supported; resolving in a manner that supports streaming processes; and the resolution of the conversion relation between the table and the field is supported. As a preferred embodiment of the present invention, a data asset circulation can be traced, and a data asset circulation map is shown in fig. 25, which can obtain nodes from which data in a local chain number cube comes through statistics, and obtain the proportion of each source node data in the local data, so that the data circulation can be clearly seen. The statistical method of which nodes the data comes from is as follows: the ETL process information of the data during import or export is found from the blockchain account book, and the timestamp, the source database name or source data address, the source table name, the target database name or target database address, the target table name, the hash value, the number of fields, the number of pieces and the like of the current operation of the data are definitely recorded in the process information, so that the source data address and the source table name of the batch of data can be obtained by reading the ETL process information, the source of the data is obtained, and in addition, the target database address and the target table name of the batch of data can also be obtained, and the destination of the data is obtained. After the data source or destination is obtained, the number of fields of the batch of data can be obtained, after the number of fields of each batch of data is counted, the number of fields of each source data is divided by the total number of fields, and the proportion of the batch of data in the data set is obtained. The proportion of each source data in the local chain number cube can be counted. Therefore, the statistical information of the data flow direction of each chain number cube in the network is clearly obtained. A data asset stream map may be drawn based on the statistical data. And drawing a data asset security supervision large screen and a data exchange sharing monitoring graph according to the statistical data, wherein the data asset security supervision large screen is shown in FIG. 26, and the data exchange sharing monitoring graph is shown in FIG. 27.

13. ETL management

The ETL management interface is shown in fig. 28, and the ETL is a data transaction, and provides an analysis basis for a data circulation process through maintenance of the ETL. Support maintenance functions and log tracking for ETL jobs; support maintenance functions that perform engines on the ETL; support maintenance functions for the ETL uplink engine.

14. Data lens fusing arrangement

The level of data security is increased by fusing the configuration. The system judges according to the four dimensions of fusing, and if the SQL statement reaches the fusing standard, the behavior operation is blocked, and a user is prompted to fuse and trigger warning. The data lens blown configuration is shown in fig. 29. The fusing triggering conditions include the following: fusing whole table query or calendar data, fusing too much query quantity, fusing too high SQL execution frequency, and fusing SQL statements of editing/newly adding/damaging data.

15. The BaaS is WBaaS, and a block chain service platform can enable managers to easily complete alliance chain construction through the WBaaS, so that the operation and maintenance implementation cost is reduced. Supporting the maintenance of organization information and the association of user information; the state of the support server maintains an association with the cube; the construction of multiple alliances and multiple channels is supported; supporting the maintenance of peer and orderer nodes; the development and maintenance of the multi-language intelligent contract are supported; supporting key management and cloud storage.

Example 2

The chain number cube is a basic bearing node of the block chain network, is also a distributed basic data node, and also bears the service of the chain number communication software. As a blockchain data network server, functions such as data uplink (ETL uplink engine), data encryption storage (safe), data decryption access (data lens), etc. are provided. And the asset attributes of the data are more emphasized, and a more finely designed data governance application is provided.

The chain number universal chain number cube adopts a plug-and-play chain-up mode and has the following characteristics.

Chip consensus

For the user adopting the chain number cube in the uplink mode, the calculation capability, the data transmission capability and the like are greatly improved, the CPU calculation resources are released, and the hardware acceleration is realized through the FPGA.

Flexible networking

Networking the chain number cube in a BD-WAN mode makes the geographic range no longer a difficult problem for server deployment. Meanwhile, multi-cloud multi-network interconnection and multi-wide area network access are supported.

Fuse protection

Safety protection is provided for data assets, and accidents of data leakage are avoided.

Data uplink

Block chain techniques are employed to ensure authenticity and authority of the data.

Credible calculation storage

And establishing a safe and credible computing and storing environment for the system by combining TEE technology and TDE technology.

1. Overall architecture

The chain number stereo machine comprises IaaS and PaaS components.

2. Layer cloud management platform

The WStack is used as a cloud management platform for the self-research of the number of chains, and management and scheduling of resources such as calculation, storage, network and the like of a user data center are provided. The private cloud environment can be rapidly configured, and cloud hosts can be rapidly created, cloud disks can be rapidly distributed, and a cloud host network can be automatically configured.

WStack is based on cloud computing IaaS (infrastructure as a service) software. The intelligent data center oriented Application Programming Interface (API) manages data center resources including calculation, storage and network by providing flexible and complete APIs. The WStack can be used for quickly constructing an intelligent cloud data center of the WStack, and flexible cloud application scenes such as PaaS (platform as a service), saaS (software as a service) and the like can also be constructed on the stable WStack.

WStack is the next generation cloud platform IaaS software designed based on the proprietary cloud platform 4S (Simple, strong, scalable elastic, smart) standard.

The WStack provides management of resources such as computing, storage and network of an enterprise data center infrastructure, the bottom layer supports KVM and VMware virtualization technologies, supports storage types such as DAS/NAS/SAN/DFS, supports local storage, NFS storage, SAN storage and distributed block storage, and supports network models such as VLAN/VXLAN.

The core cloud engine of the WStack uses a message bus to communicate with the MariaDB database and each service module, and provides functions of cloud host management, physical host management and control, storage scheduling, network functions, account number charging, real-time monitoring and the like. WStack also provides SDKs for Java and Python, and supports RESTful APIs for resource scheduling management. The proprietary cloud management platform based on WStack fully embodies the 4S advantages of the proprietary cloud, namely: simple, robust Strong, flexible Scalable, smart.

3. PaaS layer component

The PaaS layer component on the chain number cube is the bottom core software service of the whole chain number product system. The system comprises an ETL operation engine, an ETL transaction ledger, an abstract uplink submission, an asset safe, a data confidential box, a data lens, a console client and the like. The PaaS layer component of the chain number cube is shown in FIG. 30.

1) ETL job engine

The corresponding ETL job intelligence contract may be triggered down the chain using the network public API.

2) ETL Account book

Additional ETL transaction types are abstracted for tracing, and a unique ETL ledger is formed.

3) Abstract uplink submission

And chaining the downlink untrusted data digests to form a trust root.

4) Data cipher box

A secure web repository that stores data as assets.

5) Data lens

According to the identity management mechanism of the alliance chain, data can be directly accessed according to the strategy.

6) Console client

And the chain number cube can be debugged or controlled through the control console client.

The hardware configuration of the chain number cube is shown in fig. 31, and includes a CPU, a memory, a system disk, a RAID/SAS card, a gigabit network card, an FPGA, and the like.

The characteristics of the chain number cube include:

1) And (4) intelligentizing. An ETL ontology model is built, a set of ETL description standards are built, the transaction process of data assets is represented by a unified data structure through automatic analysis of ETL script contents, and transaction traceability is simpler and more intelligent; machine learning is carried out on behaviors of the user accessing data to form an SQL grammar rule base, abnormal user behaviors are detected and timely fused through SQL semantic analysis and behavior feature analysis, data safety is protected, and data leakage is prevented.

2) Easy to use. The data input in the system has corresponding prompt function, and simultaneously supports data monitoring and prevents the input of abnormal data; meanwhile, the system adopts humanized page design, a conventional use process and 0 code management operation, so that the use threshold of the whole system is reduced.

3) And (4) safety. Node key management based on TEE (trusted execution environment), and whole-course encryption data transmission; the product uses AE (encryption all the time) and TDE (transparent data encryption) technology to ensure the safety of the data, only a data owner or an approved data applicant calls a data lens through a unique secret key, the data can be previewed and used, and the data is safer and more secret; meanwhile, in the calling process of the data lens, a related fusing mechanism is arranged to prevent illegal calling and other behaviors of data.

4) And (4) expansibility. The intelligent contract in the chain number communication supports multi-language writing, and provides efficient demand solving capability for developers.

5) High performance. The execution engine supports cluster deployment, so that the TPS capacity is larger than 10 ten thousand; meanwhile, the chain number cube is accelerated through the FPGA chip level, so that the performance is greatly improved.

6) Compatibility. The system basic B/S architecture is developed to support the access and operation of conventional browsers such as chrome,360, IE, firefox and the like; meanwhile, in addition to a conventional structured database (such as MySql, oracle, postgreSQL, DB2 and the like), the system also provides corresponding storage and computing power for unstructured data, including XML, video, picture, document type data and the like, and supports HDFS clustered distributed storage.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A chain number cube for data chaining is characterized in that functions of data encryption, data storage, data chaining and data access are realized through the chain number cube;

the data directory stores fields of accessed data of the chain number cube, and the fields are used for providing access for other chain number cubes;

the intelligent contract is used for the data provider and the data purchasing party to reach an agreement, so that data transaction is realized, and data asset information uplink, data asset transaction user uplink and data asset transaction information uplink are carried out in the data transaction process;

the uplink engine is used for storing ETL process information of imported data and/or ETL process information of distributed data into a block chain account book;

the block chain edge cloud node stores a block chain account book, and the block chain account book stores ETL process information of each chain number cube in the system;

the specific steps of simultaneously realizing the functions of data encryption, data storage and data chaining through the chain number cube comprise:

a1, the ETL execution engine synchronously imports original data into the data cipher box in an ETL mode, and in the importing process, the data cipher box encrypts the original data to obtain encrypted data and stores the encrypted data into the data cipher box;

a2, the uplink engine imports the original data into ETL process information of the data cipher box for uplink storage, and stores the ETL process information into the block chain account book;

data access is realized through the chain number cube, and the three modes specifically comprise local access, data distribution and intelligent contract access:

the specific steps of local access include: the local user directly decrypts the encrypted data in the data encryption box through the data lens in the own chain number cube and checks the decrypted original data;

the data distribution method specifically comprises the following steps:

c4, the target chain number cube distributes data details corresponding to the data information to the access terminal chain number cube through the ETL execution engine, and meanwhile, the uplink engine carries out uplink storage on the ETL process information distributed by the data details;

the specific steps of intelligent contract access comprise:

2. The chain number cube of claim 1, wherein the ETL process information is in a one-to-one correspondence with the encrypted data, and wherein the ETL process information includes, but is not limited to, the following: the operation comprises a timestamp, a source database name, a source table name, a target database name, a target table name and a hash value of the original data.

3. The chain number cube of claim 1, wherein the data lockbox encrypts data using both AE and TDE, and wherein the data lens decrypts encrypted data using both AE and TDE.

4. The chain count cube for data chaining of claim 1, wherein when data of said chain count cube is in abnormal access, said data lens is further adapted to blow an external access link, and factors triggering said data lens to blow include: data volume over-query, abuse SQL frequency or full table query.

5. The chain number cube for data chaining according to any one of claims 1-4, further comprising an analysis module for performing forward or backward trace of data origin on data stored in a local data lockbox, wherein the forward or backward trace information is implemented by ETL process information in a block chain ledger.

6. An apparatus for a chain count cube for data chaining, comprising the chain count cube of any of claims 1-5; the data encryption device also comprises a JDBC or ODBC interface, and data are synchronized to the device of the chain number cube through the JDBC or ODBC interface, so that the functions of data encryption, data storage, data uplink and data access are realized.