CN113806342A - System for extracting data at bottom of ether house block chain - Google Patents

Abstract

The invention discloses a system for extracting data at the bottom of an Ether house block chain, belonging to the technical field of Ether house data extraction; the system comprises a levelDB bottom layer connecting module, a data structure analyzing module and a data formatting module, wherein the levelDB bottom layer module has the following main functions: firstly, directly accessing a level DB database of an Etherhouse; secondly, controlling a database lock during multi-thread analysis; the data analysis module has the main functions as follows: firstly, analyzing block header data; secondly, analyzing transaction data in the block; thirdly, calling Log data in the analysis block; the data formatting module has the main functions as follows: a decimal language of the first hexadecimal trading value; secondly, formatting a transaction address; thirdly, formatting a transaction calling function; the invention leads out the Ether house block chain data by directly accessing the Ether house node bottom level DB and formatting binary data. The invention can greatly shorten the time for exporting the data of the Ethernet house block chain and provide an efficient and reliable data source for the data analysis on the Ethernet house chain.

Description

System for extracting data at bottom of ether house block chain

Technical Field

The invention provides a system for extracting data at the bottom of an Ether house block chain, and belongs to the technical field of Ether house data extraction.

Background

Ethereum (ethernet in english) is an open-source public blockchain platform with intelligent contract functions, and provides an ethernet Virtual Machine (ethernet Virtual Machine) with decentralized center through its dedicated ethernet currency (ETH, abbreviated as "ETH") to process point-to-point contracts. Billions of levels of historical transaction data and contract calling data are stored on an Ethernet main chain, compressed binary block data needs to be analyzed into a data input high-performance database with clear fields, strong indexing and strong analyzability to efficiently and comprehensively analyze the data and records on a block chain, and accordingly, the capacity of performing big data mining analysis and artificial intelligent clustering on the data is accelerated.

There are two distinct pieces of data in the ether house; permanent data and temporary data. An example of persistent data is a transaction, which will be recorded in a transaction trie after it is fully validated, and will never change. An example of temporary data is a balance on a particular etherhouse account address. The balance of the account address is stored in a state trie and is changed whenever the transaction for that particular account changes. In an etherhouse, where persistent data and temporary data (such as account balances) are stored separately, the etherhouse uses a trie data structure to manage the data.

EtherFang uses level DB as its underlying data storage database, level DB is an open source key-value pair database, besides database main functions, also provides forward and backward iterations of data, ordered mapping from key to value, custom comparison function and automatic compression function. Data may be automatically compressed using the open source Google compression/decompression library "Snappy". Snappy aims at faster compression speeds than for maximum compression. The LevelDB provides an important storage and retrieval mechanism for managing the state of the ethernet network. Because of the complexity of the Etherhouse data structure, the existing Etherhouse ETL tools in the market all cross the level DB and directly derive the historical data by requesting the RPC interface carried by the Etherhouse node program.

The existing ether mill data export technology is to export historical data by requesting an RPC interface carried by an ether mill node program, so that the efficiency of an extraction mode is low, a server is greatly loaded, the export period of the full data is long (about one month long), and the requirement of quickly analyzing the ether mill data cannot be met.

Disclosure of Invention

The system for extracting the bottom data of the Ethernet workshop block chain can solve the problem of low extraction efficiency of the existing Ethernet workshop data export technology.

In order to solve the above problems, the present invention provides the following technical solutions: the system comprises a levelDB bottom layer connecting module, a data structure analyzing module and a data formatting module, wherein the levelDB bottom layer module has the following main functions: firstly, directly accessing a level DB database of an Etherhouse; secondly, controlling a database lock during multi-thread analysis; the data analysis module has the main functions as follows: firstly, analyzing block header data; secondly, analyzing transaction data in the block; thirdly, calling Log data in the analysis block; the data formatting module has the main functions as follows: a decimal language of the first hexadecimal trading value; secondly, formatting a transaction address; thirdly, formatting a transaction calling function;

the operation mode is as follows:

a: data extraction, namely directly analyzing the Ethenhouse LevelDB and formatting binary data to obtain the bottom layer data of the Ethenhouse, wherein different extraction methods need to be selected in the extraction process, so that the operation efficiency is improved as much as possible;

b: cleaning and converting data;

c: loading data;

the invention has the beneficial effects that:

the invention leads out the Ethernet workshop block chain data by directly accessing the Ethernet workshop node bottom level DB and formatting binary data. The method can greatly shorten the time for exporting the data of the Ethernet workshop block chain (the general one-month period is shortened to 2 days), and provides an efficient and reliable data source for data analysis on the Ethernet workshop block chain. The invention can improve the extraction efficiency of the data at the bottom layer of the Ethernet block chain and restore the original format of the data at the bottom layer of the Ethernet block chain to the maximum extent.

Drawings

Fig. 1 is a flow chart of ethernet bay data export of a system for extracting data at the bottom of an ethernet bay block chain according to the present invention.

Detailed Description

The invention is further described below with reference to the accompanying drawings.

According to the illustration of FIG. 1: the invention provides a system for extracting data at the bottom of an Ether house block chain, which comprises the following steps: the system comprises a levelDB bottom layer connecting module, a data structure analyzing module and a data formatting module, wherein the levelDB bottom layer module has the main functions as follows: firstly, directly accessing a level DB database of an Etherhouse; secondly, controlling a database lock during multi-thread analysis; the data analysis module has the main functions as follows: firstly, analyzing block header data; secondly, analyzing transaction data in the block; thirdly, calling Log data in the analysis block; the data formatting module has the main functions as follows: a decimal language of the first hexadecimal trading value; secondly, formatting a transaction address; thirdly, formatting a transaction calling function; the operation mode is as follows: a: data extraction, namely directly analyzing the Ethenhouse LevelDB and formatting binary data to obtain the bottom layer data of the Ethenhouse, wherein different extraction methods need to be selected in the extraction process, so that the operation efficiency is improved as much as possible; b: cleaning and converting data; c: loading data;

and deriving the Ethernet block chain data by directly accessing the underlying level DB of the Ethernet node and formatting binary data. The method can greatly shorten the time for exporting the data of the Ethernet workshop blockchain (the general one-month period is shortened to 2 days), and provides an efficient and reliable data source for data analysis on the Ethernet workshop blockchain. The specific mode is that firstly, data extraction is carried out, the Ethenhouse LevelDB is directly analyzed and binary data is formatted to obtain the bottom layer data of the Ethenhouse, different extraction methods need to be selected in the extraction process, therefore, the operation efficiency is improved as much as possible, then the data is cleaned and converted, and finally the data is loaded.

The present invention and its embodiments have been described above, and the description is not intended to be limiting, and the drawings are only one embodiment of the present invention, and the actual structure is not limited thereto. In summary, those skilled in the art should appreciate that they can readily use the disclosed conception and specific embodiments as a basis for designing or modifying other structures for carrying out the same purposes of the present invention without departing from the spirit and scope of the invention as defined by the appended claims.

Claims (2)

1. A system for extracting data at the bottom of an Ethernet house block chain is characterized in that: the system comprises a levelDB bottom layer connecting module, a data structure analyzing module and a data formatting module, wherein the levelDB bottom layer module has the following main functions: firstly, directly accessing a level DB database of an Etherhouse; secondly, controlling a database lock during multi-thread analysis; the data analysis module has the main functions as follows: firstly, analyzing block header data; secondly, analyzing transaction data in the block; thirdly, calling Log data in the analysis block; the data formatting module has the main functions as follows: a decimal language of the first hexadecimal trading value; secondly, formatting a transaction address; and thirdly, formatting a transaction calling function.

2. The system for extracting data at the bottom of an etherhouse blockchain according to claim 1, wherein: the operation mode is as follows:

a: data extraction, namely directly analyzing the Ethenhouse LevelDB and formatting binary data to obtain the bottom layer data of the Ethenhouse, wherein different extraction methods need to be selected in the extraction process, so that the operation efficiency is improved as much as possible;

b: cleaning and converting data;

c: and (4) loading data.

Images