WO2020118725A1

WO2020118725A1 - Blockchain performance test system and method

Info

Publication number: WO2020118725A1
Application number: PCT/CN2018/121333
Authority: WO
Inventors: 叶可江; 王锐; 须成忠
Original assignee: 深圳先进技术研究院
Priority date: 2018-12-14
Filing date: 2018-12-14
Publication date: 2020-06-18

Abstract

A blockchain performance test system, the system comprising an adaptation layer, a core layer and a test layer, wherein the adaptation layer is used for adapting a blockchain to the test system and integrating the blockchain to be tested into the test system; the core layer is used for transferring the adapted blockchain to a test unit, monitoring the situation of the test layer and performing performance analysis, and integrating test results; and the test layer is used for distinguishing between same according to a consensus mechanism of the blockchain to be tested, and performing a performance test on the blockchain according to a set configuration file. The system is suitable for a variety of blockchain platforms, and is simple in configuration and convenient in operation, and can achieve the universality of blockchain testing.

Description

Blockchain performance testing system and method

Technical field

This application belongs to the technical field of computer software testing, and particularly relates to a blockchain performance testing system and method.

Background technique

Blockchain, also known as distributed ledger, is essentially a node that the system does not trust each other maintains a data structure that can only be attached. From the perspective of the database, the blockchain can be regarded as a solution to the problem of distributed transaction management: all nodes save copies of data and agree on the order of execution of transactions. But traditional databases operate in a trusted environment and use well-known concurrency control techniques to sequence transactions. The key advantage of the blockchain is that it does not assume that nodes trust each other, so it aims to achieve Byzantine fault tolerance. Blockchain has the potential to subvert many existing technologies because it can bring lower infrastructure and labor costs. In particular, the immutability and transparency of the blockchain reduces the need for human error and manual intervention in conflicting data. The blockchain can simplify the business process by eliminating duplicate work.

At present, the performance of the blockchain is far from meeting the requirements, and it cannot replace the applications running on the database system, such as banking, financial and transaction applications. The performance of blockchain solutions is one of the most concerned functions of blockchain users. At present, there is no performance evaluation for different blockchain solutions according to a set of neutral and generally accepted rules. Some reports have different situations. The performance of different blockchain implementations, but there is no universal blockchain testing system.

Summary of the invention

1. Technical problems to be solved

Based on the current situation, the performance of the blockchain is far from meeting the requirements, and it cannot replace the applications running on the database system, such as banking, financial and transactional applications. The performance of blockchain solutions is one of the most concerned functions of blockchain users. At present, there is no performance evaluation for different blockchain solutions according to a set of neutral and generally accepted rules. Some reports have different situations. The performance of different blockchain implementations, but there is no problem with a general blockchain testing system, this application provides a blockchain performance testing system and method.

2. Technical solutions

In order to achieve the above purpose, the present application provides a blockchain performance testing system, which includes an adaptation layer, a core layer and a test layer;

The adaptation layer is used to adapt the blockchain to the test system and integrate the blockchain to be tested into the test system;

The core layer is used to deliver the adapted blockchain to the test unit, monitor and analyze the performance of the test layer, and integrate the results of the test;

The test layer is used for distinguishing according to the blockchain consensus mechanism to be tested, and performing performance tests on the blockchain according to the set configuration file.

Optionally, the adaptation layer includes a blockchain native software development kit.

Optionally, the core layer includes a blockchain calling module, a resource monitoring module, a performance analysis module, and a report generation module;

The blockchain calling module is used to compile and deploy smart contracts, call smart contracts, and query the status from the ledger;

The resource monitoring module is used to start and stop the monitor and obtain the resource consumption status of the blockchain;

The performance analysis module is used to read predefined performance statistics and print test results, and record key indicators;

The report generation module is used to summarize statistical results and generate a report.

Optionally, the resources in the resource consumption state include CPU, memory, and network input and output.

Optionally, the predefined performance statistics include the number of transactions per second, transaction delay, and resource utilization; the key indicators include the creation time, commit time, and transaction result of the transaction.

Optionally, the report format is HTML format.

This application also provides a blockchain performance test method, which includes the following steps:

Step 1. Adapt the blockchain to be tested to the test system and select the blockchain test scenario to be tested;

Step 2. Run the blockchain test scenario. After the test is completed, return the test data;

Step 3. Analyze the returned test data and generate a report.

Optionally, the step 1 includes the following steps:

101. Analyze the blockchain to be tested and abstract its framework into three layers;

102. After the layering of the blockchain to be tested, different test scenarios are selected for different consensus mechanisms;

103. According to the selection of the test scenario, integrate the blockchain to be tested into the test system through adaptation.

Optionally, the step 2 includes the following steps:

201. Build a distributed test environment in the test system;

202. After the blockchain under test enters the distributed test environment, a blockchain configuration file is used to create and initialize the blockchain under test;

203. Read and compile the smart contract according to the method specified in the configuration file;

204. Deploy the compiled smart contract to the test system and start the monitor object to monitor the key performance indicators of the blockchain to be tested;

205. Start a test client according to the configuration file of the test system;

206. After the test is completed, create a node containing performance statistics;

207. Return the performance statistics node that stores the blockchain to be tested for analysis.

Optionally, the step 3 includes the following steps:

301. Analyze statistical data and summarize the data;

302. Establish a table for the summarized data and generate a report.

3. Beneficial effect

Compared with the prior art, the beneficial effects of a blockchain performance testing system and method provided by this application are:

The blockchain performance test system provided by this application analyzes and compares different blockchain solutions by setting an adaptation layer, a core layer, and a test layer. It is used to evaluate the blockchain as another distributed data processing platform, and can help developers find bottlenecks and better help developers analyze and improve the blockchain system. The blockchain performance test system involved in this application is applicable to a variety of blockchain platforms, and is simple to configure and easy to operate. The versatility of blockchain testing can be achieved.

BRIEF DESCRIPTION

Figure 1 is a schematic diagram of a blockchain performance test system of this application;

2 is a schematic diagram of the core layer of a blockchain performance testing system of the present application.

detailed description

Hereinafter, specific embodiments of the present application will be described in detail with reference to the accompanying drawings. According to these detailed descriptions, those skilled in the art can clearly understand the present application and can implement the present application. Without violating the principles of the present application, the features in the different embodiments may be combined to obtain new implementations, or to replace certain features in certain embodiments to obtain other preferred implementations.

In the original design, the Bitcoin blockchain stored bitcoin as a system state shared by all participants. For this simple application, the Bitcoin node implements a simple replication state machine model, transferring Bitcoin from one address to another. Since then, the blockchain has rapidly developed into a user-defined state and Turing complete state machine model. Ethereum (ethereum) is a typical example.

The National University of Singapore and Zhejiang University jointly launched a private blockchain evaluation framework, and published a paper named "BLOCKBENCH: A Framework for Analysis Blockchains" in SIGMOD'2017. BlockBench: A framework for evaluating private chains. As a fair framework for comparing different platforms, it can deeply understand different system design choices. The private chain can be integrated into BLOCKBENCH through simple API functions, and the load is carried out on the basis of real and synthetic smart contracts, but the public chain test is not supported.

The Linux Foundation's Hyperledger, a blockchain technology open source project, announced that it has accepted Huawei's first blockchain benchmark tool "Caliper" as its latest project. Hyperledger (a global cooperation project managed by the Linux Foundation) announced that Caliper has been accepted by the Technical Steering Committee as a Hyperledger project. Caliper was officially launched in May 2017. The goal is to integrate the existing blockchain into a framework, so as to compare according to the performance set by Hyperledger and the standards set by the expansion working group. At present, Caliper can provide analysis support for Hyperledger Fabric, Hylerledger Sawtooth and Hyperledger Iroha, but it does not support other blockchain projects other than Hyperledger.

1) The blockchain system includes many parts, and almost every detail has undergone various design choices between different platforms. This has led to existing blockchain performance testing technical solutions that only support a small number of projects and lack adaptation for many emerging blockchain platforms.

2) The lack of test items, the existing test items are too simple.

3) Existing blockchain performance test solutions are difficult to configure and difficult to get started.

Consensus mechanism: A distributed consensus algorithm for blockchain transactions.

The Byzantine Fault Tolerance (PBFT) algorithm was proposed by Miguel Castro and Barbara Liskov in 1999. The original intention was to solve the problem of achieving consistency in distributed systems, which coincided with the goal of the blockchain consensus mechanism. Its main feature is that the network is highly fault-tolerant. In a network with 3f+1 nodes, the number of failed nodes is f, the network can still operate normally, and the fault tolerance rate is close to 33%. At present, China ChinaLedger Alliance and HyperLedger Alliance are studying and discussing the practical application of PBFT. The algorithm first needs to introduce the concepts of view and replica. The replica contains the master node and the backup node. The primary node (primary) is selected according to a certain formula. After the selection, the other replica nodes become backups. When the master node is valid, it means that it is in the same view. When the master node fails, the backup node needs to initiate a view change process through the timeout mechanism after the backup node detects it, and elect a new master node. The entire algorithm flow is as follows: First, a client initiates a service request to the master node, and the master node distributes the request to all backup nodes. After the backup node is processed, all the feedback to the client is completed. As long as the client receives f+1 The same result returned by the node is the final result of the algorithm. The specific implementation process is divided into three-phase protocol (three-phaseprotocol), namely pre-prepare (pre-prepare), prepare (prepare) and confirm (commit).

Referring to FIGS. 1-2, this application provides a blockchain performance test system, which includes an adaptation layer, a core layer, and a test layer;

The test layer is used to distinguish according to the blockchain consensus mechanism to be tested, and perform performance tests on the blockchain according to the set configuration file.

Optionally, the adaptation layer includes a blockchain native software development kit. Each adapter adapts the interface provided by the test system by using the corresponding blockchain native SDK.

Optionally, the report format is HTML format.

Step 3. Analyze the returned test data and generate a report.

After the blockchain under test is adapted through the adaptation layer of the test system, install the necessary programs to prepare for the test, select the blockchain test scenario to be tested, and run the selected blockchain test scenario, each test scenario It is defined by a configuration file and contains the configuration and test parameters of the underlying blockchain network. After the test is completed, return the test data, analyze the returned test data and generate a technical report to better assist developers in developing the blockchain system.

The specific process is as follows:

Optionally, the step 1 includes the following steps:

103. According to the selection of the test scenario, integrate the blockchain to be tested into the test system through adaptation. Here, the blockchain framework to be tested is abstracted into a consensus layer, an execution layer, and an application layer.

The consensus layer is the foundation of the blockchain. The role of the consensus layer is to make all nodes in the blockchain consistent with the content of the blockchain. The performance of blockchains with different consensus mechanisms varies greatly.

Specifically, after analyzing the blockchain to be tested, it is abstracted into a consensus layer, an execution layer, and an application layer. Because different blockchains have different consensus mechanisms, the performance of the blockchain will be different. Aiming at the different consensus mechanisms at the consensus layer, the consensus mechanisms include: proof-of-work mechanism, proof-of-stake mechanism, proof-of-stake authorization mechanism, Pool verification pool, and practical Byzantine PBFT; to select test scenarios, the test scenarios here include at least: proof-of-work Test scenarios, proof of stake mechanism test scenarios, share authorization proof mechanism test scenarios, Pool verification pool test scenarios, and practical Byzantine PBFT test scenarios. After the consensus mechanism of the blockchain to be tested is differentiated, then through the adaptation layer of the test system, the The blockchain to be tested is integrated into the test system, and each adapter adapts the interface provided by the test system by using the corresponding blockchain native SDK (software development kit).

This application analyzes and stratifies the blockchain to be tested, performs separate tests on the blockchains under test with different consensus mechanisms, and compares the performance of blockchains under test with similar consensus mechanisms.

Optionally, the step 2 includes the following steps:

201. Build a distributed test environment in the test system;

205. Start the test client according to the configuration file of the test system; start a loop test according to the configuration file of the test system to execute the test, and the task will be generated and distributed to the client according to the defined workload; the client will be created after startup Test nodes, call smart contracts, and perform corresponding tests; test the actual performance of the blockchain by creating specific scenarios such as creating accounts, querying transactions, and transaction requests; testing the scalability of the blockchain by adjusting the number of participating nodes and transactions Ability and stability;

At the execution layer, a smart contract is a non-tamperable program stored in the blockchain, which can automatically execute some pre-defined rules and terms in response to the received information. After the contract is released, its operation and maintenance are handed over to the miners of the entire network to reach a consensus. The contract developers define the rules for contract operation. After the rules are released, they are open and transparent and cannot be tampered with. The role of the execution layer is to execute smart contracts quickly and accurately in the operating environment.

Application layer. Many applications are being combined with blockchain technology. The application layer runs on the execution layer and builds various decentralized applications through smart contracts.

The next step is to read and compile the smart contract in the execution layer through the configuration file, and at the same time start the monitor through the resource monitoring module of the core layer in the test system, and the transaction per second (TPS) and transaction delay of the blockchain to be tested , Resource utilization and other information for monitoring, and then start the test client to call the smart contract. The test client here is the test layer in the test system. After distinguishing according to the consensus mechanism of the blockchain to be tested, it is treated according to the set configuration file To test the performance of the blockchain, the test here can choose the test parameters, select according to the required data, there will be many test clients started at the same time, after the test, create a node containing the performance statistics , And then return the performance statistics node that stores the blockchain to be tested for analysis.

Optionally, the step 3 includes the following steps:

301. Analyze statistical data and summarize the data;

302. Establish a table for the summarized data and generate a report.

Analyze the statistical data of all clients in each round of testing, and then aggregate these data through the core layer of the test system to generate a report. There are many forms of reports, HTML format is one of them.

The blockchain performance test system involved in this application tests the currently supported blockchain platforms including: Ethereum (Ethereum), Parity, Quorum, Hyperledger Fabric (Super Ledger), Hyperledger Sawtooth, Hyperledger Iroha, Hyperledger Burrow, Hyperledger Indy, etc., Since the adaptation interface is reserved, it is also easy to adapt the system to the blockchain platform as needed.

Although the present application has been described above with reference to specific embodiments, those skilled in the art should understand that, within the principle and scope of the present disclosure, many modifications can be made to the configuration and details disclosed in the present application. The protection scope of the present application is determined by the appended claims, and the claims are intended to cover all modifications included in the literal meaning or scope of the technical features of the claims.

Claims

A blockchain performance testing system, characterized in that the system includes an adaptation layer, a core layer and a test layer;

The adaptation layer is used to adapt the blockchain to the test system and integrate the blockchain to be tested into the test system;

The core layer is used to deliver the adapted blockchain to the test unit, monitor and analyze the performance of the test layer, and integrate the results of the test;

The test layer is used to distinguish according to the blockchain consensus mechanism to be tested, and perform performance tests on the blockchain according to the set configuration file.
The blockchain performance testing system of claim 1, wherein the adaptation layer includes a blockchain native software development kit.
The blockchain performance testing system according to claim 1, wherein the core layer includes a blockchain calling module, a resource monitoring module, a performance analysis module, and a report generation module;

The blockchain calling module is used to compile and deploy smart contracts, call smart contracts, and query the status from the ledger;

The resource monitoring module is used to start and stop the monitor and obtain the resource consumption status of the blockchain;

The performance analysis module is used to read predefined performance statistics and print test results, and record key indicators;

The report generation module is used to summarize statistical results and generate a report.
The blockchain performance test system according to claim 3, wherein the resources in the resource consumption state include CPU, memory, and network input and output.
The blockchain performance testing system according to claim 3, wherein: the predefined performance statistics include the number of transactions per second, transaction delay, and resource utilization; the key indicators include the creation time and commit time of the transaction And transaction results.
The blockchain performance testing system according to claim 3, wherein the report format is an HTML format.
A blockchain performance testing method, characterized in that the method includes the following steps:

Step 1. Adapt the blockchain to be tested to the test system and select the blockchain test scenario to be tested;

Step 2. Run the blockchain test scenario. After the test is completed, return the test data;

Step 3. Analyze the returned test data and generate a report.
The blockchain performance test method according to claim 7, wherein the step 1 includes the following steps:

101. Analyze the blockchain to be tested and abstract its framework into three layers;

102. After the layering of the blockchain to be tested, different test scenarios are selected for different consensus mechanisms;

103. According to the selection of the test scenario, integrate the blockchain to be tested into the test system through adaptation.
The blockchain performance test method according to claim 7, wherein the step 2 includes the following steps:

201. Build a distributed test environment in the test system;

202. After the blockchain under test enters the distributed test environment, a blockchain configuration file is used to create and initialize the blockchain under test;

203. Read and compile the smart contract according to the method specified in the configuration file;

204. Deploy the compiled smart contract to the test system and start the monitor object to monitor the key performance indicators of the blockchain to be tested;

205. Start a test client according to the configuration file of the test system;

206. After the test is completed, create a node containing performance statistics;

207. Return the performance statistics node that stores the blockchain to be tested for analysis.
The blockchain performance test method according to claim 7, wherein the step 3 includes the following steps:

301. Analyze statistical data and summarize the data;

302. Establish a table for the summarized data and generate a report.