CN115114314A - Data probe-based data detection and extraction method and system - Google Patents
Data probe-based data detection and extraction method and system Download PDFInfo
- Publication number
- CN115114314A CN115114314A CN202211037187.3A CN202211037187A CN115114314A CN 115114314 A CN115114314 A CN 115114314A CN 202211037187 A CN202211037187 A CN 202211037187A CN 115114314 A CN115114314 A CN 115114314A
- Authority
- CN
- China
- Prior art keywords
- data
- information
- block chain
- probe
- party
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/176—Support for shared access to files; File sharing support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/12—Network monitoring probes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L61/00—Network arrangements, protocols or services for addressing or naming
- H04L61/30—Managing network names, e.g. use of aliases or nicknames
- H04L61/3015—Name registration, generation or assignment
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data probe-based data detection and extraction method and a system, wherein the system comprises the following steps: the system comprises a block chain building module, a DID registration module, a block chain network addressing and addressing platform, an identity mutual-recognition module, a data probe and a storage module; the method comprises the steps of building a block chain; all data parties carry out DID registration and generate DID documents of all the parties, and chain deposit certificates are linked in the DID document information; based on BNS technology, carrying out network addressing on each database data resource, adding information of related resources on a chain, and generating a corresponding address according to the related information; sending a data sharing application, finishing an authorization process of data interaction based on database DID document information, and chaining a chain storage certificate in authorization information; the data probe executes the reading operation of the target data set, extracts the required data and converts the data into storage; the invention combines the block chain, the data probe, the distributed digital identity authentication and the data routing technology, and the whole data circulation process is efficient, safe, stable and traceable.
Description
Technical Field
The invention relates to the technical field of data probes, in particular to a data probe-based data detection and extraction method and system.
Background
With the development of information technology, the important roles of digitalization and informatization are gradually recognized in various social fields, and the society of today has completely entered a big data age. Big data technology has become one of the driving forces for the development of the social productivity progress. Data products and data services formed based on big data enable production of enterprises and organizations, a traditional production economic operation mechanism is changed, and economic operation efficiency is further improved.
But large data does not efficiently support the vigorous development of socio-economy due to lack of synchronicity and integrity in digital economy. Data islanding phenomenon exists in data in a large number of organizations and entities. Effective synergy between the individual data sets is difficult due to the lack of organic integrated organizational structures, which ultimately exacerbates the effects of data islanding.
Moreover, the conventional data sharing system generally adopts a centralized storage manner. This approach has several problems: 1. this can easily lead to leakage of internal data if the administrator of the centralized storage has an extremely high level of rights. 2. The server requirement of the centralized storage mode is extremely high, and the effect of single point failure is easy to occur. That is, if a problem arises with one server running, the entire data sharing system will no longer be available. 3. The service of the centralized storage mode is likely to stop due to operation or management conditions, and data of all users may completely disappear, so that data security will not be effectively guaranteed.
At present, a main mode for solving a data isolated island is data sharing, a data probe technology is widely used in various technical fields of data perception and data extraction, a source database of a data provider is accurately detected through a data probe according to the requirements of a data demander, data can be extracted and stored in a target database of the data demander, and the data probe technology ensures high efficiency and accuracy of a data sharing process.
The blockchain technology is a powerful means for solving the problem of data security of centralized storage, and the organic combination of the blockchain technology and the data sharing technology can effectively solve the problems of data security and single-point failure in the centralized storage; in addition, authorization related to personal information identities of data sharing parties is required in the data sharing process, including credit of the data parties, and the like, and the participating parties need to exchange respective personal information in the data sharing process so as to realize exchange services based on the respective information.
The patent (CN 113961571A) proposes a data sensing scheme based on a data probe method, however, this scheme does not consider the data security aspect (data omission, data abuse), which easily causes the challenge of data resource security; furthermore, this approach does not consider data extraction with a data probe.
The technical solution for anti-fraud detection based on block-chain combined data probe proposed by the patent (CN 113971248A) is a scenario solution combining block-chain and data probe. However, the scheme does not discuss the identity of each node on the access chain, so that the detection process of the data probe based on which technology can be finished cannot be confirmed
Therefore, it is an urgent need to solve the problem of providing a data probe-based data detection and extraction method and system.
Disclosure of Invention
In view of the above, the present invention provides a data probing and extracting method and system based on a data probe to solve the problems mentioned in the background art.
In order to achieve the purpose, the invention adopts the following technical scheme:
a data probe-based data detection and extraction method comprises the following steps:
s1, building a block chain;
s2, carrying out DID registration on each data party, generating DID documents of each data party, wherein the DID documents of each data party have respective public keys, uploading block link deposit certificate information, wherein the DID document information of the uploaded block link deposit certificate comprises a DID short address and a public key, and each data party comprises a data demand party, a data supply party and a data probe;
s3, based on a block chain domain name BNS technology, carrying out network addressing on each database data resource, including naming service and related address setting service, adding information of related resources on a block chain, generating a corresponding address according to the related resource information, and uploading address information to a block chain deposit certificate;
s4, sending a data sharing application, mutually authenticating the identities of all data parties based on the DID document information of the database to finish the authorization process of data interaction, and entering S5 if the data interaction passes the authentication successfully, and uploading authorization information to a block link storage card;
s5, analyzing the application of data sharing, setting an access rule, selecting an optimal resource or service path according to a source and a routing strategy of a data demand party, executing reading operation on a target data set by a data probe system according to the routing strategy, extracting required data, and uploading data interaction information and access rule information to a block chain for storage;
and S6, after data are extracted, the conversion of data formats and the data storage are completed according to the data requirements and the access rules of the target database.
Preferably, the block chain in S1 is a federation chain.
Preferably, the specific contents of S3 include:
s31, establishing a uniform specification to finish naming of different resources on the chain;
s32, registering the user domain name based on a block chain domain name system, and storing the domain name and related domain name resolution information on a block chain;
and S33, the data provider adds information of related resources on the block chain, including related service information, interface information, intelligent contract information and architecture information, generates a corresponding address according to the related information provided by the data provider, stores the corresponding address in the block chain, and binds the generated corresponding address with the domain name.
Preferably, the specific contents of S4 include:
s41, a data demand side sends a data sharing application to a data provider;
s42, the data provider determines the identity of the data demander according to the DID document information of the data demander on the block chain, generates a verifiable certificate VC and sends the verification certificate VC to the data demander;
and S43, after the data demand party receives the VC information of the verifiable certificates, packaging the VC information of the multiple verifiable certificates and generating verifiable expression VP information, and sending the verifiable expression VP information to the data probe for confirmation.
Preferably, the specific contents of S5 include:
s51, resolving the domain name of the data demand party into related information corresponding to resources on a block chain, wherein the related information comprises related service information, architecture information, interface information, address information and intelligent contract information;
s52, according to the address information obtained after analysis, accessing data resources on the block chain and feeding back the data resources to a data demand side;
s53, selecting an optimal resource or service path based on a simple routing strategy, a balanced routing strategy, a weighted routing strategy and a delay routing strategy;
s54, according to the optimal resource or service path, performing reading operation on the target data set and extracting required data, generating a data directory of a single data set type and performing writing operation on the data directory to a new database.
A data probe-based data detection and extraction system, comprising: the system comprises a block chain building module, a DID registration module, a block chain network addressing and addressing platform, an identity mutual-recognition module and a data probe;
the block chain building module is used for building a block chain;
the DID registration module is used for carrying out DID registration on each data party and generating a DID document of each data party, the DID document of each data party has a respective public key, the DID document information is uploaded to the block link deposit certificate, the DID document information of the uploaded block link deposit certificate comprises a DID short address and a public key, and each data party comprises a data demand party, a data supply party and a data probe;
the block chain network addressing and addressing platform is used for carrying out network addressing on each database data resource based on a block chain domain name BNS technology, comprises naming service and related address setting service, adds information of related resources on a block chain, generates a corresponding address according to the related resource information, and uploads the block chain storage certificate by the address information; the system is also used for analyzing the application of data sharing after successful authentication and authorization, setting an access rule, and selecting an optimal resource or service path according to the source and the routing strategy of a data demand party;
the identity mutual-authentication module is used for carrying out identity mutual-authentication of all data parties to finish the authorization process of data interaction based on database DID document information after sending the application of data sharing, and uploading authorization information to the block link deposit certificate;
and the data probe is used for executing reading operation on the target data set according to the routing strategy and extracting required data, uploading the data interaction information and the access rule information to the block chain for storage, and finishing data format conversion and data storage according to the data requirement and the access rule of the target database after data extraction.
Preferably, the blockchain is a federation chain.
Preferably, the block chain network addressing and addressing platform comprises a name management unit, an application management unit, an addressing management unit, an analysis management unit, an addressing management unit and a routing policy unit;
the name management unit is used for establishing a uniform standard and finishing naming of different resources on the chain;
the application management unit is used for registering the user domain name based on the block chain domain name system and storing the domain name and related domain name resolution information on the block chain;
the addressing management unit is used for adding information of related resources on the block chain by the data provider, including related service information, interface information, intelligent contract information and architecture information, generating a corresponding address according to the related information provided by the data provider, storing the corresponding address in the block chain, and binding the generated corresponding address with the domain name;
the analysis management unit is used for analyzing the domain name of the data demand party into related information corresponding to resources on the block chain, wherein the related information comprises related service information, architecture information, interface information, address information and intelligent contract information;
the addressing management unit is used for accessing data resources on the block chain and feeding back the data resources to the data demand side according to the address information obtained after analysis;
and the routing strategy unit is used for providing a simple routing strategy, a balanced routing strategy, a weighted routing strategy and a delay routing strategy to select the optimal resource or service path.
Preferably, the process of authorizing the data interaction by mutually authenticating the identities of the data parties specifically includes:
the data demand side sends a data sharing application to the data provider;
the data provider determines the identity of the data demander according to the DID document information of the data demander on the block chain, produces a verifiable certificate VC and sends the VC to the data demander;
and after the data demand party receives the verifiable voucher VC information, packaging a plurality of verifiable voucher VC information of the data demand party, generating verifiable expression VP information, and sending the verifiable expression VP information to the data probe for confirmation.
Preferably, the data probe performs a "read" operation on the target data set and extracts the required data according to the optimal resource or service path, generates a data catalog of a single data set type and performs a "write" operation to the new database.
According to the technical scheme, compared with the prior art, the data probe-based data detection and extraction method and system are provided, the fact that the node entering and exiting and the data uplink are within a controllable range is guaranteed based on the characteristics of the alliance chain, a chain manager exists in the chain, the stability and the controllability of the structure of the whole chain are guaranteed, evidence and trace are stored in the chain in all steps in the whole data probe process, and the whole-process traceability of the data using process is guaranteed; the distributed digital identity authentication technology is adopted at the initial stage of data query, so that the low efficiency and inaccuracy of centralized identity authentication are prevented, and the personal information of the data interaction participants is effectively ensured; the data probe extraction process is based on a block chain network addressing technology, all resources on a chain can be endowed with a legal and unique domain name address, and a user can more conveniently search corresponding resources on the chain; the block chain, the data probe, the distributed digital identity authentication and the data routing technology are combined, a general model is designed, and the whole data circulation process is efficient, safe, stable and traceable.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a schematic diagram of a data probe extraction method according to the present invention;
fig. 2 is a schematic diagram of mutual identity authentication of various data parties provided by the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention discloses a data probe-based data detection and extraction method, as shown in figure 1, comprising the following steps:
s1, building a block chain;
s2, carrying out DID registration on each data party, generating DID documents of each data party, wherein the DID documents of each data party have respective public keys, uploading block link deposit certificate information, wherein the DID document information of the uploaded block link deposit certificate comprises a DID short address and a public key, and each data party comprises a data demand party, a data supply party and a data probe;
s3, based on a block chain domain name BNS technology, carrying out network addressing on each database data resource, including naming service and related address setting service, adding information of related resources on a block chain, generating a corresponding address according to the related resource information, and uploading address information to a block chain deposit certificate;
s4, sending a data sharing application, mutually authenticating the identities of all data parties based on the DID document information of the database to finish the authorization process of data interaction, and entering S5 if the data interaction passes the authentication successfully, and uploading authorization information to a block link storage card;
s5, analyzing the application of data sharing, setting an access rule, selecting an optimal resource or service path according to a source and a routing strategy of a data demand party, executing reading operation on a target data set by a data probe system according to the routing strategy, extracting required data, and uploading data interaction information and access rule information to a block chain for storage;
and S6, after data are extracted, the conversion of data formats and the data storage are completed according to the data requirements and the access rules of the target database.
In this embodiment, the established blockchain is a multi-node alliance chain, and there is an administrator on the chain.
In order to further implement the above technical solution, the block chain in S1 is a federation chain.
In this embodiment, only the manager of the blockchain has the right to parse the short address of the DID and directly access the DID document, thereby ensuring the security of the whole model.
In order to further implement the above technical solution, the specific content of S3 includes:
s31, establishing a uniform specification to finish naming of different resources on the chain;
s32, registering the user domain name based on a block chain domain name system, and storing the domain name and related domain name resolution information on a block chain;
and S33, the data provider adds information of related resources on the block chain, including related service information, interface information, intelligent contract information and architecture information, generates a corresponding address according to the related information provided by the data provider, stores the corresponding address in the block chain, and binds the generated corresponding address with the domain name.
In order to further implement the above technical solution, as shown in fig. 2, the specific content of S4 includes:
s41, a data demand side sends a data sharing application to a data provider;
s42, the data provider determines the identity of the data demander according to the DID document information of the data demander on the block chain, produces a verifiable certificate VC and sends the VC to the data demander;
and S43, after the data demand party receives the VC information of the verifiable certificates, packaging the VC information of the multiple verifiable certificates and generating verifiable expression VP information, and sending the verifiable expression VP information to the data probe for confirmation.
In order to further implement the above technical solution, the specific content of S5 includes:
s51, resolving the domain name of the data demand party into related information corresponding to the resources on the block chain, wherein the related information comprises related service information, architecture information, interface information, address information and intelligent contract information;
s52, according to the address information obtained after analysis, accessing data resources on the block chain and feeding back the data resources to a data demand side;
s53, selecting an optimal resource or service path based on a simple routing strategy, a balanced routing strategy, a weighted routing strategy and a delay routing strategy;
s54, according to the optimal resource or service path, performing reading operation on the target data set and extracting required data, generating a data directory of a single data set type and performing writing operation on the data directory to a new database.
In this embodiment, the data probe system includes data probing and data extraction, where the data probing includes two types of library table probing and file probing, and the data extraction includes two types of library table extraction and file extraction.
The data detection process through the data probe has the following capabilities for detecting the library table and the file: database connectivity detection and base table structure and data update detection capability, various types of database detection capability, and more functional detection capability based on requirements.
For the extraction of the base table and the extraction of the file, the following capabilities are provided: the ability to extract and aggregate from source data to destination addresses, multi-type data extraction capability, more functionality extraction capability based on demand.
The method can realize the requirement-based data detection and extraction from the field level and the base table level, analyze the data from the data dimension, and comprises the analysis and diagnosis of the data dimension, such as the analysis and diagnosis of the problems of abnormal content length, uniform coding and the like, and also comprises the analysis and diagnosis of the measurement value of the data, such as the general overall situation of the data values such as large value, minimum value, average value, summary value, variance, median and the like, and also comprises other problems, such as the analysis and diagnosis of the data precision and the data range.
A data probe-based data detection and extraction system, comprising: the system comprises a block chain building module, a DID registration module, a block chain network addressing and addressing platform, an identity mutual-recognition module and a data probe;
the block chain building module is used for building a block chain;
the DID registration module is used for carrying out DID registration on each data party and generating a DID document of each data party, the DID document of each data party has a respective public key, the DID document information is uploaded to the block link deposit certificate, the DID document information of the uploaded block link deposit certificate comprises a DID short address and a public key, and each data party comprises a data demand party, a data supply party and a data probe;
the block chain network addressing and addressing platform is used for carrying out network addressing on each database data resource based on a block chain domain name BNS technology, comprises naming service and related address setting service, adds information of related resources on a block chain, generates a corresponding address according to the related resource information, and uploads the block chain storage certificate by the address information; the system is also used for analyzing the application of data sharing after successful authentication and authorization, setting an access rule, and selecting an optimal resource or service path according to the source and the routing strategy of a data demand party;
the identity mutual-authentication module is used for carrying out identity mutual-authentication of all data parties to finish the authorization process of data interaction based on database DID document information after sending the application of data sharing, and uploading authorization information to the block link deposit certificate;
and the data probe is used for executing reading operation on the target data set according to the routing strategy and extracting required data, uploading the data interaction information and the access rule information to the block chain for storage, and finishing data format conversion and data storage according to the data requirement and the access rule of the target database after data extraction.
To further implement the above technical solution, the block chain is a federation chain.
In order to further implement the technical scheme, the block chain network addressing and addressing platform comprises a name management unit, an application management unit, an addressing management unit, an analysis management unit, an addressing management unit and a routing strategy unit;
the name management unit is used for establishing a uniform standard and finishing naming of different resources on the chain;
the application management unit is used for registering the user domain name based on the block chain domain name system and storing the domain name and related domain name resolution information on the block chain;
the addressing management unit is used for adding information of related resources on the block chain by the data provider, including related service information, interface information, intelligent contract information and architecture information, generating a corresponding address according to the related information provided by the data provider, storing the corresponding address in the block chain, and binding the generated corresponding address with the domain name;
the analysis management unit is used for analyzing the domain name of the data demand party into related information corresponding to resources on the block chain, wherein the related information comprises related service information, architecture information, interface information, address information and intelligent contract information;
the addressing management unit is used for accessing resources on the block chain and feeding back the resources to the data demand side according to the address information obtained after analysis;
and the routing strategy unit is used for providing a simple routing strategy, a balanced routing strategy, a weighted routing strategy and a delay routing strategy to select the optimal resource or service path.
In this embodiment, the simple routing policy is the simplest way, and when the client receives a plurality of resources and needs to select one of them, the client has complete autonomy and can select any one of them to send a request.
The balanced routing strategy means that a plurality of addresses of a certain type are received by a client, and at the moment, an algorithm (such as a random algorithm) is adopted by the client to send requests to a target address in a balanced manner, so that the request quantity received by a server is basically the same, and the single service load is avoided from being overlarge.
The weighted routing strategy depends on the address configuration of the server side, and the server side allocates a weight to each resource. The resource records received by the client contain the weight value, the client can sum the ownership weight values of the same resource to obtain n, then a random value from 0 to n is generated, and a resource address is selected according to the space where the random value is located.
The delay routing strategy means that after receiving a plurality of target addresses corresponding to a domain name, a BNS client randomly selects 2 configurable addresses when accessing resources for the first time, then pings the two addresses, records the RTT time of the two addresses, and selects a sending request with the shortest RTT return time; the second time of accessing the resource is to select another address to ping, compare with the previous RTT and select the sending request with the shortest RTT time; by analogy, when all addresses are found to be ping before sending the request, the request is directly sent to the shortest RTT.
In order to further implement the above technical solution, as shown in fig. 2, the process of authorizing data interaction by mutually authenticating identities of data parties specifically includes:
the data demand side sends a data sharing application to the data provider;
the data provider determines the identity of the data demander according to the DID document information of the data demander on the block chain, produces a verifiable certificate VC and sends the VC to the data demander;
and after the data demand party receives the VC information of the verifiable certificates, packaging the VC information of the multiple verifiable certificates of the data demand party, generating verification expression VP information, and sending the verification expression VP information to the data probe for confirmation.
In order to further implement the above technical solution, the data probe performs a "read" operation on the target data set according to the optimal resource or service path and extracts the required data, generates a data directory of a single data set type and performs a "write number" operation to a new database.
In the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A data detection and extraction method based on a data probe is characterized by comprising the following steps:
s1, building a block chain;
s2, carrying out DID registration on each data party, generating DID documents of each data party, wherein the DID documents of each data party have respective public keys, uploading block link deposit certificate information, wherein the DID document information of the uploaded block link deposit certificate comprises a DID short address and a public key, and each data party comprises a data demand party, a data supply party and a data probe;
s3, based on a block chain domain name BNS technology, carrying out network addressing on each database data resource, including naming service and related address setting service, adding information of related resources on a block chain, generating a corresponding address according to the related resource information, and uploading address information to a block chain deposit certificate;
s4, sending a data sharing application, mutually authenticating the identities of all data parties based on the DID document information of the database to finish the authorization process of data interaction, and entering S5 if the data interaction passes the authentication successfully, and uploading authorization information to a block link storage card;
s5, analyzing the application of data sharing, setting an access rule, selecting an optimal resource or service path according to a source and a routing strategy of a data demand party, executing reading operation on a target data set by a data probe according to the routing strategy, extracting required data, and uploading data interaction information and access rule information to a block chain for evidence storage;
and S6, after data are extracted, the conversion of data formats and the data storage are completed according to the data requirements and the access rules of the target database.
2. The data probe-based data probing and extracting method as claimed in claim 1, wherein the block chain in S1 is a federation chain.
3. The data probe-based data detection and extraction method according to claim 1, wherein the details of S3 include:
s31, establishing a uniform specification to finish naming of different resources on the chain;
s32, registering the user domain name based on a block chain domain name system, and storing the domain name and related domain name resolution information on a block chain;
and S33, the data provider adds information of related resources on the block chain, including related service information, interface information, intelligent contract information and architecture information, generates a corresponding address according to the related information provided by the data provider, stores the corresponding address in the block chain, and binds the generated corresponding address with the domain name.
4. The data probe-based data detection and extraction method according to claim 1, wherein the details of S4 include:
s41, a data demand side sends a data sharing application to a data provider;
s42, the data provider determines the identity of the data demander according to DID document information of the data demander on the block chain, produces a verifiable certificate VC and sends the VC to the data demander;
and S43, after the data demand party receives the VC information of the verifiable certificates, packaging the VC information of the multiple verifiable certificates and generating verifiable expression VP information, and sending the verifiable expression VP information to the data probe for confirmation.
5. The data probe-based data detection and extraction method according to claim 1, wherein the details of S5 include:
s51, resolving the domain name of the data demand party into related information corresponding to resources on a block chain, wherein the related information comprises related service information, architecture information, interface information, address information and intelligent contract information;
s52, according to the address information obtained after analysis, accessing data resources on the block chain and feeding back the data resources to a data demand side;
s53, selecting an optimal resource or service path based on a simple routing strategy, a balanced routing strategy, a weighted routing strategy and a delay routing strategy;
s54, according to the optimal resource or service path, performing reading operation on the target data set and extracting required data, generating a data directory of a single data set type and performing writing operation on the data directory to a new database.
6. A data probe-based data detection and extraction system, comprising: the system comprises a block chain building module, a DID registration module, a block chain network addressing and addressing platform, an identity mutual-recognition module and a data probe;
the block chain building module is used for building a block chain;
the DID registration module is used for carrying out DID registration on each data party and generating a DID document of each data party, the DID document of each data party has a respective public key, the DID document information is uploaded to the block link deposit certificate, the DID document information of the uploaded block link deposit certificate comprises a DID short address and a public key, and each data party comprises a data demand party, a data supply party and a data probe;
the block chain network addressing and addressing platform is used for carrying out network addressing on each database data resource based on a block chain domain name BNS technology, comprises naming service and related address setting service, adds information of related resources on a block chain, generates a corresponding address according to the related resource information, and uploads the address information to the block chain for evidence storage; the system is also used for analyzing the application of data sharing after successful authentication and authorization, setting an access rule, and selecting an optimal resource or service path according to the source and the routing strategy of a data demand party;
the identity mutual-authentication module is used for carrying out identity mutual-authentication of all data parties to finish the authorization process of data interaction based on database DID document information after sending the application of data sharing, and uploading authorization information to the block link deposit certificate;
and the data probe is used for executing reading operation on the target data set according to the routing strategy and extracting required data, uploading the data interaction information and the access rule information to the block chain for storage, and finishing data format conversion and data storage according to the data requirement and the access rule of the target database after data extraction.
7. The data probe-based data detection and extraction system of claim 6, wherein the blockchain is a federation chain.
8. The data probe-based data detection and extraction system of claim 6, wherein the blockchain network addressing and addressing platform comprises a name management unit, an application management unit, an addressing management unit, an analysis management unit, an addressing management unit and a routing policy unit;
the name management unit is used for establishing a uniform standard and finishing naming of different resources on the chain;
the application management unit is used for registering the user domain name based on the block chain domain name system and storing the domain name and related domain name resolution information on the block chain;
the addressing management unit is used for adding information of related resources on the block chain by the data provider, including related service information, interface information, intelligent contract information and architecture information, generating a corresponding address according to the related information provided by the data provider, storing the corresponding address in the block chain, and binding the generated corresponding address with the domain name;
the analysis management unit is used for analyzing the domain name of the data demand party into related information corresponding to resources on the block chain, wherein the related information comprises related service information, architecture information, interface information, address information and intelligent contract information;
the addressing management unit is used for accessing resources on the block chain and feeding back the resources to the data demand side according to the address information obtained after analysis;
and the routing strategy unit is used for providing a simple routing strategy, a balanced routing strategy, a weighted routing strategy and a delay routing strategy to select the optimal resource or service path.
9. The data detection and extraction system based on the data probe as claimed in claim 6, wherein the authorization process of mutual identity authentication of data parties to complete data interaction specifically includes:
the data demand side sends a data sharing application to the data provider;
the data provider determines the identity of the data demander according to the DID document information of the data demander on the block chain, produces a verifiable certificate VC and sends the VC to the data demander;
and after the data demand party receives the VC information of the verifiable certificates, packaging the VC information of the multiple verifiable certificates of the data demand party, generating verification expression VP information, and sending the verification expression VP information to the data probe for confirmation.
10. The data probe-based data detection and extraction system as claimed in claim 8, wherein the data probe performs a read operation on the target data set according to the optimal resource or service path and extracts the required data, generates a data directory of a single data set type and performs a write operation to a new database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211037187.3A CN115114314A (en) | 2022-08-29 | 2022-08-29 | Data probe-based data detection and extraction method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211037187.3A CN115114314A (en) | 2022-08-29 | 2022-08-29 | Data probe-based data detection and extraction method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115114314A true CN115114314A (en) | 2022-09-27 |
Family
ID=83336017
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211037187.3A Pending CN115114314A (en) | 2022-08-29 | 2022-08-29 | Data probe-based data detection and extraction method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115114314A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109787815A (en) * | 2018-12-27 | 2019-05-21 | 云南财经大学 | Government information resources shared system based on block chain |
| CN110417739A (en) * | 2019-06-27 | 2019-11-05 | 华东师范大学 | It is a kind of based on block chain technology safety Netowrk tape in measurement method |
| CN114116904A (en) * | 2021-11-25 | 2022-03-01 | 国网江苏省电力有限公司电力科学研究院 | Asset account chain storage system and method for information security |
-
2022
- 2022-08-29 CN CN202211037187.3A patent/CN115114314A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109787815A (en) * | 2018-12-27 | 2019-05-21 | 云南财经大学 | Government information resources shared system based on block chain |
| CN110417739A (en) * | 2019-06-27 | 2019-11-05 | 华东师范大学 | It is a kind of based on block chain technology safety Netowrk tape in measurement method |
| CN114116904A (en) * | 2021-11-25 | 2022-03-01 | 国网江苏省电力有限公司电力科学研究院 | Asset account chain storage system and method for information security |
Non-Patent Citations (1)
| Title |
|---|
| XIAOFENG JIA 等: "Cross‐organisational data sharing framework based on blockchain‐probes", 《IET NETWORKS PUBLISHED BY JOHN WILEY & SONS LTD ON BEHALF OF THE INSTITUTION OF ENGINEERING AND TECHNOLOGY》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11651109B2 (en) | Permission management method, permission verification method, and related apparatus | |
| CN109522735B (en) | Data permission verification method and device based on intelligent contract | |
| Gimenez-Aguilar et al. | Achieving cybersecurity in blockchain-based systems: A survey | |
| CN106910051B (en) | DNS resource record notarization method and system based on alliance chain | |
| Pradhan et al. | A flexible permission ascription (FPA)-based blockchain framework for peer-to-peer energy trading with performance evaluation | |
| CN108876669B (en) | Course notarization system and method applied to multi-platform education resource sharing | |
| Li et al. | A hybrid blockchain ledger for supply chain visibility | |
| CN110493347A (en) | Data access control method and system in large-scale cloud storage based on block chain | |
| Wang et al. | Security Analysis on dBFT protocol of NEO | |
| CN108881167A (en) | A kind of intelligent contract of finite field block catenary system | |
| Zhang et al. | Cycledger: A scalable and secure parallel protocol for distributed ledger via sharding | |
| CN109218391A (en) | A method of distributed memory system audit and decentralization based on block chain | |
| Han et al. | Jupiter: A blockchain platform for mobile devices | |
| CN108959457A (en) | A kind of inquiry of certificate, verification method and system | |
| CN112053271B (en) | Public service platform data evidence management method and system based on block chain | |
| CN111612452A (en) | Intellectual property management system and method based on block chain | |
| CN111506589A (en) | Block chain data service system based on alliance chain, access method and storage medium | |
| WO2023040496A1 (en) | Virtual resource allocation based on blockchain | |
| Li et al. | EdgeWatch: Collaborative investigation of data integrity at the edge based on blockchain | |
| WO2022206431A1 (en) | Method and apparatus for querying ledger data of fabric blockchain | |
| Hei et al. | Themis: An accountable blockchain-based P2P cloud storage scheme | |
| Yoon et al. | Blockchain-based personal data trading system using decentralized identifiers and verifiable credentials | |
| US20120066497A1 (en) | Method and device for enabling portable user reputation | |
| Zhang et al. | Access control and trust management for emerging multidomain environments | |
| CN114022285A (en) | Cross-enterprise cloud computing resource metering method and system based on alliance block chain |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220927 |
|
| RJ01 | Rejection of invention patent application after publication |