CN112417517B - A data standard management method based on blockchain - Google Patents

Abstract

The invention relates to the technical field of data standard management and blockchain, and particularly discloses a blockchain-based data standard management method. The method comprises the following steps: A. when the data standard is created, the metadata information in the data standard is uploaded to the blockchain, the unique identification code of each piece of metadata information is obtained, and the data standard maker and the user do not need to perform centralized access on the data standard any more, so that the effect of decentralized data standard management is achieved. The step A specifically comprises the following steps: 1, converting original metadata into a target format; and 2, operating the metadata in the target format to obtain a character string and an encrypted character string corresponding to the metadata in each target format. The invention carries out data conversion and uplink on the metadata information through the blockchain technology, so that the metadata is traceable and can not be tampered, and therefore, the used organization can completely trust the data standard obtained from the central library.

Description

A data standard management method based on blockchain

Technical field

The invention relates to the technical fields of data standard management and blockchain, and specifically discloses a data standard management method based on blockchain.

Background technique

With the advancement of informatization in various industries, a large amount of data has accumulated in various systems. To use this data, unified standards are needed, so there is a need for data standard management.

Data standards refer to normative constraints that ensure the consistency and accuracy of internal and external use and exchange of data. The specific form of data standards is usually a collection of one or more metadata, that is, metadata is the basic unit of data standards. What is metadata? Metadata is data used to describe data. For example, the description data of the name, type, length, etc. of each field in a data table is called metadata. What is standard metadata? Standard metadata is metadata jointly developed and recognized by an organization or group. In the process of establishing a data warehouse, it is used as a rule to perform data conversion; in the process of data sharing and exchange between departments, it is used to describe the format of the data and improve the efficiency of sharing and exchange.

Since metadata in data standards plays a key role in the data governance process, metadata must require extremely high uniformity, recognition, and credibility. Traditional data standard management is generally led by designated departments or organizations (such as IT departments), and data standards are formed through research. Data standards include data sets, data subcategories, and specifically metadata describing data subcategories. Then Metadata information is stored in a central database, and data standard maintenance and management are completed by designated departments. This is a centralized data standard management model and system. However, it is difficult for all parties using data standards to participate in the formulation, maintenance and upgrade of data standards. The process is complicated and responsibilities and rights are unclear. Often after an organization spends great efforts to specify data standards, as the business changes, the data standards soon become no longer applicable. The progress of revising the data standards cannot keep up with the speed of changes in the business, eventually making the data standards useless. Second, there are credibility issues with data standards in large organizations or cross-organizational environments. However, organizations using centralized data standards management cannot identify whether the metadata is credible, and only have unconditional trust in the central database. A distributed and decentralized data standard management method and system is needed to make the metadata in the data standard credible, and relevant organizations can participate in the process of formulating and maintaining the metadata in the data standard.

Contents of the invention

The purpose of this invention is to address the situation where organizations or business departments using data standards cannot identify whether the metadata in the data standard is trustworthy in data standard management in a data center or data warehouse, and it is difficult to participate in centralized data standard management and maintenance. Current situation, taking advantage of the characteristics of blockchain data that cannot be tampered with and are traceable, a standard data management method based on blockchain is proposed, including a metadata conversion method based on blockchain, which enables multiple parties to participate in the creation and maintenance of data standards. .

In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

A data standard management method based on blockchain, the steps include:

A. When creating metadata, upload the metadata information to the blockchain and obtain the unique identification code of each piece of metadata information;

B. When calling the metadata in the data standard, verify the obtained metadata on the blockchain based on the unique identification code to determine whether the obtained metadata has been tampered with;

Step A specifically includes:

The first step is to convert the original metadata into the target format and obtain the metadata in the target format;

The second step is to perform operations on the metadata in the target format to obtain the string corresponding to each piece of metadata in the target format;

The third step is to encrypt the string and obtain the encrypted string;

The fourth step is to combine the string, the encrypted string corresponding to the string, the table name, the relevant description, the source of information before the change and the current metadata version into the metadata information on the blockchain, and the encrypted string corresponding to the string. It is the unique identification code of metadata information.

Further, the method also includes step B: when calling the metadata in the data standard, verify the obtained metadata on the blockchain according to the unique identification code to determine whether the obtained metadata has been tampered with;

Specifically, it includes the following steps:

When obtaining off-chain metadata, look for the encrypted string in the off-chain metadata;

By encrypting the string, search the blockchain to see if there is corresponding on-chain metadata. If so, the obtained off-chain metadata has not been tampered with. If not, the obtained off-chain metadata has been tampered with.

As a preferred solution, the steps also include: the maintenance method of metadata in the block. The maintenance method of metadata in the block includes the following steps: 1) Preparation of data; 2) Reaching consensus on the chain; 3) Synchronous changes of off-chain data .

Further, data preparation specifically includes the following steps:

Find the encrypted string before the metadata change that needs to be changed;

Follow the method in step A to convert the changed metadata information into an on-chain data structure to obtain the changed metadata information;

Add the pre-change encrypted string to the "Pre-change information source" field in the post-change metadata information.

As a preferred solution, the steps to reach consensus on the chain include:

C1. Build an organization channel. The organization channel contains multiple organization nodes. The organization nodes correspond one-to-one with the metadata organizations in the usage data standard, and the metadata organization is allowed to initiate metadata modification requests in the organization channel;

C2. One of the organization nodes initiates a request to modify the metadata in the data standard and constructs a modification proposal;

C3. The endorsement node in the organization channel verifies the modification proposal based on the content of the modification proposal and the construction information of the organization channel. When the verification passes, the transaction result is generated;

C4. Generate a proposal response based on the transaction results and the signature of the endorsement node;

C5. Determine whether the proposal response meets the pre-endorsement policy. If so, package the modified proposal into blocks in chronological order;

C6. Each organization node verifies the modification proposal in the block. If the modification proposal in the block has not changed, the transaction in the block is marked as valid;

C7. When the transaction of the block is marked as valid, the metadata on the chain is modified.

As a preferred solution, the organization channels and organization nodes meet the following conditions:

1) The organization node corresponding to the organization that uses the metadata in the data standard has completed the registration and registration of the metadata standard organization, and obtained the encryption material to confirm the identity;

2) The blockchain whose metadata is modified has an endorsement policy and defines a method for modifying the metadata;

3) The organization node that initiates the metadata modification request has the authority to initiate metadata modification;

4) Only when all organization nodes on the organization channel agree, can the blockchain be triggered to modify the metadata.

As a preferred solution, in step C3, the modification proposal is verified. The verification content includes:

1) Whether the format of the modification proposal is complete; 2) Whether the modification proposal is submitted for the first time; 3) Verify whether the signature of the endorsement node is valid; 4) Whether the organization node that constructed the modification proposal has the authority to initiate modification of metadata.

As a preferred solution, the transaction result includes the response value and the write set. The write set is the key-value pair of the metadata to be modified.

As a preferred solution, the pre-endorsement policy is that all nodes in the organization channel must endorse.

Based on the same concept, the present invention also proposes a data standard management system based on blockchain, including at least one processor, and a memory communicatively connected to the at least one processor; the memory stores information that can be executed by at least one processor. Instructions are executed by at least one processor to enable at least one processor to execute any of the above methods.

Compared with the existing technology, the beneficial effects of the present invention are:

1. The present invention converts metadata information onto the chain through blockchain technology, making the metadata traceable and non-tamperable. Therefore, the organizations using it can fully trust the standard metadata obtained from the central library;

2. The present invention also introduces the chain code and channel in the alliance chain into the metadata blockchain to achieve data non-tamperability, traceability, and multi-party participation in the creation and maintenance of data standards.

Picture description:

Figure 1 is a flow chart of a blockchain-based data standard management method in Embodiment 1 of the present invention;

Figure 2 is a schematic diagram of the off-chain data storage method of metadata information in the data standard in Embodiment 1 of the present invention;

Figure 3 is a schematic diagram of the json string in Embodiment 1 of the present invention;

Figure 4 is a structural diagram of metadata information on a block in Embodiment 1 of the present invention;

Figure 5 is a schematic diagram of the off-chain data storage method of metadata information added to the MD5 encrypted string in Embodiment 1 of the present invention;

Figure 6 is a structural diagram of metadata in blocks in Embodiment 1 of the present invention;

Figure 7 is a diagram of converting modified metadata information into on-chain data structure in Embodiment 1 of the present invention;

Figure 8 is a process diagram for the accounting node to put the received data into the block in Embodiment 1 of the present invention;

Figure 9 is a flow chart of specific steps for reaching consensus on the Fabric chain in Embodiment 1 of the present invention.

Detailed ways

The present invention will be described in further detail below in conjunction with test examples and specific implementations. However, this should not be understood to mean that the scope of the above-mentioned subject matter of the present invention is limited to the following embodiments. All technologies implemented based on the contents of the present invention belong to the scope of the present invention.

Example 1

In the solution of the present invention, the central library is not completely abandoned, but the blockchain is transformed on the original basis. Taking the metadata information in Table 1 as an example, first when creating the metadata in the data standard, the metadata information is uploaded to the blockchain. In the blockchain, the metadata information is regarded as an asset. After each piece of data is uploaded, a unique identifier will be obtained, and the unique identifier will be stored in the corresponding metadata of the central library, as shown in Table 2. Therefore, when someone uses these metadata, they can use the unique identifier on the block to obtain the data on the blockchain for comparison. Since the characteristic data of the blockchain cannot be tampered with, this solves the problem. Credible questions.

Table 1 Example of metadata information

serial number Data item name length constraint Reference number GXXX020101 wxya 8 O GXXX020102 WHMC 60 O GXXX020103 wxya 12 O

Table 2 Example of metadata format with block identification

serial number Data item name length constraint Reference number Uniquely identified on the block GXXX020101 wxya 8 O 87f34b75650f4848b2891e7cce9dfaa2 GXXX020102 WHMC 60 O 5d85355111f34f029d3fead832decd52 GXXX020103 wxya 12 O a4dec2c8521a469c85a75077a74f0b56

The flow chart of a blockchain-based data standard management method is shown in Figure 1. The method includes the following steps:

When creating metadata, upload the metadata information to the blockchain and obtain the unique identification code of each piece of metadata information.

The first step is to convert the original metadata into json format to obtain metadata in json format;

The second step is to perform operations on the metadata in json format to obtain the json string corresponding to each piece of metadata in json format;

The third step is to perform the md5 encryption algorithm on the json string to obtain the md5 encrypted string;

The fourth step is to combine the json string, the corresponding md5 encrypted string of the json string, the table name, the relevant description, the source of the information before the change and the current metadata version into the metadata information on the blockchain, and the corresponding json string The md5 encrypted string is a unique identification encoding of the metadata information.

Change the method. The way metadata information is stored off-chain is shown in Figure 2. Each piece of data includes nine types of data, namely "table_code", "meta_code", "meta_name", "meta_cn_name", "meta_type", "meta_len", "meta_constraint", and "meta_valuie_space".

The first step is by converting the metadata of the committee table into json format;

In the second step, use the JSON.stringify(metaJson) function to get the json string corresponding to each piece of data;

The third step is to perform the md5 encryption algorithm on the json string to obtain the md5 encrypted string, for example:

The json string is shown in Figure 3. Through the functional formula Var md5Str=md5(metaJsonStr), the corresponding md5 encrypted string is obtained:

md5Str = F5B89BEC77BD75374AED6F13BFD8798D

The fourth step is the metadata json string (data) of the committee table, and the encrypted string (tx_id) obtained by the corresponding MD5 encryption algorithm, plus the table name (table_name), related description (desc), and the data from the block. Which piece of metadata information was changed from (update_from), and the current metadata version (version). These information together constitute the metadata information on the block (that is, a transaction information in the block), and its structure is shown in Figure 4. table_name is defined as the name of the table described by the metadata; Desc is defined as the description of the metadata; data is defined as the detailed data of the metadata, stored in the form of a json string; tx_id is the MD5 encrypted string of the metadata, which is also The unique identifier of metadata on the block; update_from is the metadata tx_id before the update, which can be used to find the address of the metadata information before the update; Version is the version information of the current metadata.

As a preferred solution, every time when metadata information is added to the chain, update_from is null. Every time when the metadata information on the chain is changed, update_from saves the metadata tx_id before the metadata is updated.

As a preferred solution, after the new metadata is added to the blockchain through the above steps one to four, the metadata information shown in Figure 2 adds a tx_id field and saves the MD5 encrypted string of the metadata. That is, the unique identifier of the saved metadata on the block. The storage format is shown in Figure 5.

The structure of metadata in the block is shown in Figure 6. The metadata of the committee table is a piece of transaction data in the block. Every time its metadata information is changed, a new transaction data is formed in the block, which can be passed update_from forms a metadata change chain, and the chain code can be used under the chain to access the metadata information on the chain through tx_id. If you want to access the metadata of TX02, call the fabric's chaincode get method and pass in tx_id=md-02 as a parameter to query the metadata information in TX02 from the chain.

When an organization using metadata needs to verify the authenticity of the acquired metadata, the specific method is as follows:

For example, the committee table in the example needs to verify whether the data stored off-chain has been tampered with, then query all metadata information with table_code GXXX0201, obtain the corresponding tx_id content, and then query the corresponding metadata information on the chain. , if the metadata information is not successfully obtained on the chain through tx_id, then the metadata under the chain has been tampered with. If it is successfully obtained, then follow steps 1, 2, and 3 of the metadata initialization in the previous block. Calculate the md5 encrypted string of the offline metadata and compare it with the metadata stored offline. If they are different, it means that the data has been tampered with. Otherwise, it proves that the data has not been tampered with, and you can use the metadata information with confidence.

As a preferred solution, the present invention also includes the maintenance of metadata in blocks. Metadata maintenance is mainly divided into three steps:

1) Data preparation

2) Reaching consensus on the chain

3) Synchronous changes to off-chain data

For example, the length of the committee responsibility field (meta_len) in the metadata of the committee table is changed from 200 to 1000. The modification and maintenance process is as follows:

1) Data preparation

a) Find the md5 string whose metadata you want to change, that is, tx_id. The tx_id found here is:

F5B89BEC77BD75374AED6F13BFD8798D, tx_id will be used in update_from of the new on-chain metadata transaction information.

b) Convert the changed metadata information into an on-chain data structure according to the metadata on-chain initialization process, as shown in Figure 7.

2) Reaching consensus on the chain

When an organization initiates changes to metadata on the chain, the mechanism of the block on the chain will ask other endorsing organizations whether modifications are allowed. Only when the conditions allowed in the endorsement policy are met, the data will be broadcast to each accounting node through the sorting node. , the accounting node puts the received data into the block, and the process is shown in Figure 8.

The specific step process is shown in Figure 9. The steps include:

C1. Build an organization channel. The organization channel contains multiple organization nodes. The organization nodes correspond one-to-one with organizations that use metadata in the data standard, and organizations are allowed to initiate metadata modification requests in the organization channel;

C2. One of the organization nodes initiates a request to modify metadata and constructs a modification proposal;

C3. The endorsement node in the organization channel verifies the modification proposal based on the content of the modification proposal and the construction information of the organization channel. When the verification passes, the transaction result is generated;

C4. Generate a proposal response based on the transaction results and the signature of the endorsement node;

C5. Determine whether the proposal response meets the pre-endorsement policy. If so, package the modified proposal into blocks in chronological order;

C6. Each organization node verifies the modification proposal in the block. If the modification proposal in the block has not changed, the transaction in the block is marked as valid;

C7. When the transaction of the block is marked as valid, the metadata on the chain is modified.

Taking Figure 8 as an example to specifically illustrate the process of the alliance chain implementing on-chain auditing and metadata changes, the process is as follows:

2.1. Assumptions:

1) A channel MDChannel has been set up. The channel is running normally. There are organizations org1, org2, org3, org4, and org5 on the channel, which correspond to nodes peerA, peerB, peerC, peerD, and peerE respectively. All five organizations have used the organization's The CA registration and registration are completed, and the encryption materials used to confirm identity in the network are obtained.

2) The chain code MDChiancode that modifies the original data on the chain has been installed and initialized on peerA-peerE, and has been recognized by the above five organizations. The chaincode MDChiancode defines the method of modifying metadata, and the chaincode also sets an endorsement. The policy is that every metadata modification must be signed by the nodes of the five organizations.

3) Permission definition: org1 and org2 can initiate metadata change requests, org3, org4, and org5 can only view it.

4) A request to modify metadata will trigger the chain code MDChiancode to modify the metadata only if all five organizations agree.

2.2. Organization org1 initiates a metadata modification request

Organization org1 sends a metadata modification request. The request will reach peerA, peerB, peerC, peerD, peerE. Then, a revision proposal is constructed. A proposal is a request to invoke a chaincode method with certain input parameters. The purpose of this request is to modify the metadata on the chaincode.

2.3. The endorsement node verifies the signature and executes the transaction

The endorsing node verifies (1) that the format of the modified proposal is complete, (2) that the proposal has not been submitted before, (3) that the signature is valid (using MSP), and (4) that the initiator (org1 in this example) ) has the authority to perform the operation on the channel (that is, each endorsing node ensures that the initiator satisfies the channel Writers policy). The endorsing node takes the proposal input as an argument to the chaincode function called. Then the chain code is executed based on the current state of the database to generate transaction results, including response values and write sets (that is, key-value pairs representing the metadata to be updated). But the ledger was not updated. These values, along with the signature of the endorsing node, are returned to the SDK as a "proposal response", which is parsed by the SDK for the application.

2.4. Check the proposal response and broadcast to the sorting service

The application verifies the signature of the endorsing node and compares these modification proposal responses to determine whether they are identical and satisfy the specified endorsement policy (i.e. peerA -peerE all endorse). After the conditions are met, the modified proposal is sent to the sorting service. The sorting service accepts the proposal, sorts it by time and packages it into blocks.

2.5. Verify and submit transactions

The blocks packaged by the ordering service are "sent" to all Peer nodes on the channel. Modification proposals within the block are verified to ensure that the endorsement policy is met and that the ledger status of the variables in the read set has not changed since step 3 when the proposal was executed to generate the read set. If there are no changes, the transactions in the block are marked as valid.

2.6. Modify metadata on the chain

Each peer appends a block to the channel's chain, and for each valid modification, a write set is committed to the current state database. The system will emit an event to notify the client application that this metadata modification has been immutably attached to the chain, and will also notify the transaction verification result that it is valid.

The above method enables multiple parties to jointly maintain and manage standard metadata, and every modification process of metadata will be recorded on the chain.

1) Synchronous changes to off-chain data

After the on-chain data modification is completed, the fabric sends the result message to the off-chain. If the modification is successful, the off-chain data will be modified together, and the tx_id will be changed to the latest version of tx_id on the chain. In the example, tx_id will be changed from

F5B89BEC77BD75374AED6F13BFD8798D

Change to

B19542B9C3A8D9EAD40401787415A653.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are included in the protection scope of the present invention.

Claims (8)

1. A data standard management method based on blockchain, characterized in that the steps include: A. When creating metadata, upload the metadata information to the blockchain and obtain the unique identification code of each piece of metadata information; B. When calling the metadata in the data standard, verify the obtained metadata on the blockchain based on the unique identification code to determine whether the obtained metadata has been tampered with; Step A specifically includes: The first step is to convert the original metadata into the target format and obtain the metadata in the target format; The second step is to perform operations on the metadata in the target format to obtain the string corresponding to each piece of metadata in the target format; The third step is to encrypt the string to obtain the encrypted string; The fourth step is to combine the string, the corresponding encrypted string, the table name, the relevant description, the source of information before the change and the current metadata version into metadata information on the blockchain. The corresponding encrypted string is the unique identification code of the metadata information; Step B specifically includes the following steps: When obtaining off-chain metadata, search for the encrypted string in the off-chain metadata; Through the encrypted string, search the blockchain to see if there is corresponding on-chain metadata. If there is, the obtained off-chain metadata has not been tampered with. If not, the obtained off-chain metadata has been tampered with; It also includes: a method for maintaining metadata in blocks. The method for maintaining metadata in blocks includes the following steps: 1) Preparation of data; 2) Reaching consensus on the chain; 3) Synchronous changes to data off the chain. 2. A blockchain-based data standard management method as claimed in claim 1, characterized in that the preparation of the data specifically includes the following steps: Find the encrypted string before the metadata change that needs to be changed; Convert the changed metadata information into an on-chain data structure according to the method in step A, and obtain the changed metadata information; Add the pre-change encrypted string to the "Pre-change information source" field in said changed metadata information. 3. A data standard management method based on blockchain as claimed in claim 2, characterized in that the step of reaching consensus on the chain includes: C1. Construct an organization channel. The organization channel contains multiple organization nodes. The organization nodes correspond one-to-one to the metadata organizations in the usage data standard, and the metadata organization is allowed to initiate metadata modification requests in the organization channel; C2. One of the organization nodes initiates a request to modify the metadata in the data standard and constructs a modification proposal; C3. The endorsement node in the organization channel verifies the modification proposal based on the content of the modification proposal and the construction information of the organization channel. When the verification passes, a transaction result is generated; C4. Generate a proposal response based on the transaction result and the signature of the endorsement node; C5. Determine whether the proposal response meets the pre-endorsement policy. If so, package the modified proposal into blocks in chronological order; C6. Each organization node verifies the modification proposal in the block. If the modification proposal in the block has not changed, the transaction in the block is marked as valid; C7. When the transaction of the block is marked as valid, modify the metadata on the chain. 4. A data standard management method based on blockchain as claimed in claim 3, characterized in that the organization channel and the organization node meet the following conditions: 1) The organization node corresponding to the organization that uses the metadata in the data standard has completed the registration and registration of the metadata standard organization where it is located, and has obtained the encryption material to confirm the identity; 2) The blockchain whose metadata is modified has an endorsement policy and defines a method for modifying the metadata; 3) The organization node that initiates the metadata modification request has the authority to initiate metadata modification; 4) Only when all organization nodes on the organization channel agree can the blockchain be triggered to modify the metadata. 5. A data standard management method based on blockchain as claimed in claim 3, characterized in that, in step C3, the modification proposal is verified, and the verification content includes: 1) Whether the format of the modification proposal is complete; 2) Whether the modification proposal is submitted for the first time; 3) Verify whether the signature of the endorsing node is valid; 4) Whether the organizational node constructing the modification proposal has the ability to initiate modification metadata permissions. 6. A data standard management method based on blockchain according to claim 3, characterized in that the transaction result includes a response value and a write set, and the write set is a key-value pair of metadata to be modified. 7. A data standard management method based on blockchain according to claim 3, characterized in that the pre-endorsement policy is that all nodes in the organization channel must endorse. 8. A data standard management system based on blockchain, characterized by including at least one processor, and a memory communicatively connected to the at least one processor; the memory stores instructions that can be executed by at least one processor, the The instructions are executed by at least one processor to enable at least one processor to execute the method of any one of claims 1-7.