CN110381150B

CN110381150B - Data processing method and device on block chain, electronic equipment and storage medium

Info

Publication number: CN110381150B
Application number: CN201910669874.9A
Authority: CN
Inventors: 陈法山; 刘朝伟; 彭顺求; 赖春晖
Original assignee: Industrial and Commercial Bank of China Ltd ICBC
Current assignee: Industrial and Commercial Bank of China Ltd ICBC
Priority date: 2019-07-24
Filing date: 2019-07-24
Publication date: 2021-12-17
Anticipated expiration: 2039-07-24
Also published as: CN110381150A

Abstract

The invention provides a data processing method and device on a block chain, electronic equipment and a storage medium, wherein the method comprises the following steps: reading data in a data queue, wherein the data queue comprises batch job data and/or online job data; responding to the data as batch job data, and executing batch jobs corresponding to the batch job data; suspending execution of the batch jobs according to the job interruption information of the batch jobs, and reading online job data in the data queue; executing the online operation corresponding to the online operation data; and responding to the completion of the online operation execution, and continuing to execute the batch operation. By the method and the device, batch operation can be processed on the block chain, and data processing efficiency is improved.

Description

Data processing method and device on block chain, electronic equipment and storage medium

Technical Field

The present invention relates to the field of information processing, and in particular, to a method and an apparatus for processing data on a block chain, an electronic device, and a storage medium.

Background

The existing form of the block chain at the beginning of birth is a public chain, and the data characteristics of the public chain are as follows: the data belongs to a large number of all end users, each user monopolizes a small amount of data on a chain, generally only has the right to modify own data, and does not have the right to modify other data. The primary business mode of the public chain is the initiative by the user to initiate a transfer of funds between two or a small number of accounts. Therefore, in technical design, the conventional blockchain system does not need to provide periodically triggered operations for modifying a large amount of data.

With the expansion of the application range of the block chain technology, a federation chain form taking a company organization and the like as participants is generated, the number of nodes of the federation chain is limited, and the nodes need to be able to join through a permitted party. The data characteristics of the federation chain are: the data belongs to a few participating institutions, each of which has a large amount of data on the chain. The services implemented on the federation chain derive from the traditional services of the participating enterprises, and the enterprises' services often involve operations of automatically triggering and modifying a large amount of data. The conventional business of financial institutions such as banks can be divided into two types, namely, online business (online transaction) and batch business, wherein the online business is initiated by a user and is used for modifying a small amount of data, such as transferring accounts and inquiring balance, and the batch business is automatically triggered by a system and is used for modifying a large amount of data, such as daily terminal information.

At present, the existing blockchain system lacks support for batch services in technical design, so that the batch services cannot be realized inside the blockchain system at present, and one batch service needs to be divided into a plurality of online requests in a traditional application system outside the blockchain so as to be indirectly realized. Fig. 1 is a schematic diagram illustrating the execution of batch operations in a conventional blockchain system, and as shown in fig. 1, this implementation requires an application server to request a large number of data records from a blockchain, obtain data to be processed in batch, and then send an online operation request execution processing logic to the blockchain server for each piece of data.

This method has the following problems:

(1) a large amount of data needs to be transferred between the blockchain and the application. Transferring large data between the blockchain and the application system is still a problem, and if the amount of service data is large, the performance is not feasible.

(2) A large number of online job requests need to be sent to the blockchain, network resources are consumed greatly, and batch job processing time is long.

Disclosure of Invention

In view of the above, the present invention provides a method, an apparatus, an electronic device and a storage medium for processing data on a block chain, so as to solve at least one of the above-mentioned problems.

According to a first aspect of the present invention, there is provided a method of data processing on a blockchain, the method comprising: reading data in a data queue, wherein the data queue comprises batch job data and/or online job data; responding to the data as batch job data, and executing batch jobs corresponding to the batch job data; suspending execution of the batch jobs according to the job interruption information of the batch jobs, and reading online job data in the data queue; executing the online operation corresponding to the online operation data; and responding to the completion of the online operation execution, and continuing to execute the batch operation.

According to a second aspect of the present invention, there is provided a data processing apparatus on a blockchain, the apparatus comprising: the data reading unit is used for reading data in a data queue, and the data queue comprises batch job data and/or online job data; the batch job execution unit is used for responding to the fact that the data are batch job data and executing batch jobs corresponding to the batch job data; a job suspending unit configured to suspend execution of the batch job according to job interruption information of the batch job; the data reading unit is also used for reading online operation data in the data queue; the online operation execution unit is used for executing the online operation corresponding to the online operation data; the batch job execution unit is further used for responding to the completion of the online job execution and continuing to execute the batch job.

According to a third aspect of the present invention, there is provided an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the data processing method when executing the program.

According to a fourth aspect of the invention, the invention provides a computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the data processing method as described above.

According to the technical scheme, the batch job data and the online job data are added into the data queue, when the batch job data are read, the corresponding batch jobs are executed, the job interruption information is set for the batch job data to suspend the execution of the batch jobs, so that the online jobs in the data queue can be executed, and after the online jobs are executed, the batch jobs are continuously executed, so that the batch jobs are processed on the block chain, the functions of the block chain are expanded, and the data processing efficiency is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a diagram illustrating the execution of a batch operation in a conventional blockchain system;

FIG. 2 is a flow chart of a method of data processing on a blockchain according to an embodiment of the present invention;

FIG. 3 is a block diagram of a data processing apparatus on a blockchain according to an embodiment of the present invention;

FIG. 4 is a block diagram illustrating a detailed structure of a data processing apparatus on a blockchain according to an embodiment of the present invention;

fig. 5 is a block diagram of the structure of the to-be-added batch job determination unit 16 according to the embodiment of the present invention;

FIG. 6 is a block diagram of an example architecture of a data processing apparatus on a blockchain according to an embodiment of the present invention;

FIG. 7 is a flow chart of a block link point to online job request process according to an embodiment of the present invention;

FIG. 8 is a flow diagram of a batch job scheduling module generating a batch job according to an embodiment of the present invention;

FIG. 9 is a flowchart of an execution module executing a batch job and an online job according to an embodiment of the present invention;

FIG. 10 is a schematic diagram of an electronic device according to an embodiment of the invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

When the current blockchain system executes batch operation, the application server needs to request a large number of data records from the blockchain first to obtain data needing batch processing, and then sends an online operation request execution processing logic to the blockchain server for each piece of data, so that a large number of data are transferred between the blockchain and the application when the batch operation is executed. Since the transfer of large data between the blockchain and the application system is still a problem, if the traffic data volume is large, it will result in performance impracticable. Moreover, sending a large number of on-line job requests to the blockchain consumes a large amount of network resources, and also results in a long processing time for the batch tasks. In actual operation, the execution of the smart contract is serial, a large number of online requests generated by the batch task are queued with normal online jobs, and the TPS (Transactions Per Second) and response speed of the normal online jobs are affected. The large number of on-line requests generated by the batch operation can be regenerated into a large number of new blocks, which are less significant for the service analysis, but mixed with the blocks of the on-line operation, the difficulty of analyzing the block chain data, such as counting the total amount of the operation and the operation amount per second, is increased.

Based on this, the embodiments of the present invention provide a data processing scheme on a blockchain to solve the above-mentioned problems.

An embodiment of the present invention provides a method for processing data on a block chain, where fig. 2 is a flowchart of the method, and as shown in fig. 2, the method includes:

step 21, reading data in a data queue, wherein the data queue comprises batch job data and/or online job data;

step 22, responding to the data being batch job data, executing the batch job corresponding to the batch job data;

step 23, suspending execution of the batch jobs according to the job interruption information of the batch jobs, and reading online job data in the data queue;

step 24, executing the online operation corresponding to the online operation data;

and 25, responding to the completion of the execution of the online operation, and continuing to execute the batch operation.

Embodiments of the present invention add batch job data and online job data to a data queue, executing the corresponding batch job when the batch job data is read, setting job interrupt information for the batch job data to suspend the execution of the batch job, thereby, the online operation in the data queue can be executed, after the online operation is completed, the batch operation is continuously executed, compared with the prior art, the method and the device have the advantages that a large amount of data does not need to be transmitted between the block chain and the application before the batch operation is executed, the application does not need to send a large amount of network requests to the block chain, and the online operation can still be executed concurrently in the batch operation execution process, the influence on the online operation is small, and meanwhile, only one record is generated by processing one batch operation, so that the block chain data analysis is less negative. By the embodiment of the invention, the function of the block chain is expanded, and the data processing efficiency is improved.

In practical operation, the completion of the online job execution in step 25 refers to: and completing the execution of the online operation corresponding to all the online operation data in the data queue.

In the batch job execution process, the number of times of modifying the corresponding job data in the batch job execution process may be recorded.

Preferably, a counter may be set for the batch job according to a predetermined rule, and the counter is used for recording the number of times of modifying corresponding job data in the process of executing the batch job.

That is, a counter is set for each batch job being executed to record the number of times the job data is modified during execution of the batch job. Therefore, the batch job execution operation of each node on the block chain is consistent, and the consistency of data maintenance of each node can be ensured. After the batch operation is completed, the corresponding counter value may be cleared.

In one embodiment, when an online job request is received from a client, whether the online job request is legal is judged according to a preset verification rule. For example, the online job request is judged according to the format, contract existence, function, parameter quantity and the like of the online job request, and if the judgment result is that the online job request is illegal, error information is returned to the client; and if the judgment result is that the online operation request is legal, performing consensus operation on the online operation request, namely sending the online operation information to other nodes in the block chain so as to ensure the consistency of the data of each node on the block chain.

Then, judging whether the batch jobs currently executed exist or not, and adding the online job request into a data queue when the judgment result is that the batch jobs currently executed do not exist; when the judgment result is that the executed batch jobs exist, continuously judging whether the executed batch jobs have set job interruption information or not; if the job interruption information is set, adding the online job request into a data queue, and waiting for batch job interruption to be executed; if the job interrupt information is not set, setting the job interrupt information for the batch jobs being executed on the node according to the counter values corresponding to the batch jobs being executed of all nodes on the block chain, and setting the job interrupt information for the batch jobs being executed on the node by taking the maximum counter value. For example, the maximum counter value (e.g., 5) may be increased by a variable (e.g., 6) as interrupt information, i.e., when the number of times the executing batch job of the node modifies data reaches 6, the job is suspended.

In actual operation, the job operation of each node on the blockchain is consistent in content and sequence, and sometimes occurs in time sequence. For example, at the node 1, the batch jobs 1, the online jobs 1 and 2, and the batch job 1, and the node 2 are executed in sequence, however, due to reasons such as fluency of the network to which each node belongs, the node 1 has currently executed the online job 2, and the node 2 has currently executed only the online job 1, so when setting the interruption information for the batch job of the current node, the corresponding counter maximum value, that is, the batch job with the fastest execution degree may be selected with reference to the execution degrees of the batch jobs on all nodes, so as to set the interruption information for the batch job of the current node. In this way, all nodes may be suspended at a particular execution location of the batch job to execute the online job.

After the job interrupt information is set, the online job request can be added into the data queue, and the online job can be executed when the batch job is suspended.

In one embodiment, batch job configuration information may be pre-stored, the configuration information including at least one of: the job ID (identifier), the smart contract ID, the function, the parameter, the job scheduling policy, the job execution validity period, the job flow number, and the job last scheduling time may specifically be as shown in table 1.

Job ID

Intelligent contract ID

Function(s)

Parameter(s)

Scheduling policy

Period of validity

Last scheduling time

001

cace28243e……5

dayend

[]

00X (once per day)

Permanent

2019-5-20 0:0:0

002

db0d7ca0a4……9

monthend

[]

001 x (once per month)

2020-12-31

2019-5-1 0:0:0

003

e13623492e……b

gen_sum

[]

00X 0 (once per week)

Permanent

2019-5-19 0:0:0

TABLE 1

And, batch job scheduling information may be stored in advance, the scheduling information recording historical scheduling information of the job. The scheduling information includes at least one of: the job scheduling ID, job scheduling time, and job current status are specifically shown in table 2:

scheduling ID	Job ID	Scheduling time	Status of state
				0000001	001	2019-5-1 0:0:0	Complete the process
0000002	001	2019-5-2 0:0:0	Complete the process
				……
0000195	001	2019-6-1 0:00:00	Has already started, has not finished
				0000196	002	2019-6-1 0:00:00	Not started

TABLE 2

In one embodiment, the batch job configuration information and the scheduling information may be periodically read, a to-be-added batch job that needs to be executed and has not been added to the data queue may be determined, and then the to-be-added batch job consensus operation may be added to the data queue to perform steps 21-25. And meanwhile, updating the batch job configuration information and the scheduling information.

The embodiment of the present invention further provides a data processing apparatus on a block chain, which is preferably used to implement the above method. Fig. 3 is a block diagram of the apparatus, and as shown in fig. 3, the apparatus includes: data reading unit 1, batch job execution unit 2, job suspension unit 3 and online job execution unit 4, wherein:

the data reading unit 1 is used for reading data in a data queue, and the data queue comprises batch job data and/or online job data;

the batch job execution unit 2 is used for responding to the fact that the data are batch job data, and executing batch jobs corresponding to the batch job data;

a job suspending unit 3 configured to suspend execution of the batch job according to job interruption information of the batch job;

at the moment, the data reading unit reads the online operation data in the data queue;

an online job execution unit 4, configured to execute an online job corresponding to the online job data, specifically, complete execution of an online job corresponding to all online job data in the data queue;

in response to completion of the online job execution, the batch job execution unit 2 continues to execute the batch job.

Compared with the prior art, the embodiment of the invention has the advantages that the batch job data and the online job data are added into the data queue, when the data reading unit 1 reads the batch job data, the batch job execution unit 2 executes the corresponding batch job, the job interruption information is set for the batch job data so that the job pause unit 3 pauses the execution of the batch job, the online job execution unit 4 can execute the online job in the data queue, and after the online job is executed, the batch job execution unit 2 continues to execute the batch job, so that the batch job is processed on the block chain, the influence on the online operation is small, and meanwhile, only one record is generated by processing one batch operation, so that the block chain data analysis is less negative. By the embodiment of the invention, the function of the block chain is expanded, and the data processing efficiency is improved.

In practice, as shown in fig. 4, the above apparatus may further include one or more of the following units: a modified number recording unit 5, a counter setting unit 6, an online work request receiving unit 7, a verification unit 8, an online work consensus unit 9, an online work request adding unit 10, a work execution judgment unit 11, an interruption information judgment unit 12, an interruption information setting unit 13, a configuration information reading unit 14, a batch work consensus unit 15, a to-be-added batch work determination unit 16, and an updating unit 17, wherein:

and the modification time recording unit 5 is used for recording the times of modifying the corresponding job data in the batch job execution process.

And the counter setting unit 6 is used for setting a counter for the batch operation according to a preset rule, and the counter is used for recording the number of times of modifying corresponding operation data in the batch operation executing process.

An online job request receiving unit 7, configured to receive an online job request from a client.

And the verification unit 8 is used for judging whether the online operation request is legal or not according to a preset verification rule.

And the online work consensus unit 9 is used for responding to the online work request being legal and performing consensus operation on the online work request.

And the online work request adding unit 10 is used for adding the online work request into the data queue.

A job execution judgment unit 11 for judging whether there is a batch job currently being executed. In response to the determination result of the job execution determination unit being non-existent, the online job request adding unit 10 adds the online job request to the data queue.

An interruption information judgment unit 12 configured to judge whether or not job interruption information has been set for the batch job being executed, in response to the judgment result of the job execution judgment unit being that there is a batch job being executed. In response to the determination result of the interruption information determining unit being that job interruption information has been set, the online job request adding unit 10 adds the online job request to the data queue.

And an interruption information setting unit 13, configured to, in response to a determination result of the interruption information determining unit being that no job interruption information is set, set job interruption information for the currently executed batch job of the current node according to the counter values of the currently executed batch jobs to which all nodes on the block chain belong. Preferably, the job interrupt information is set for the executing batch job of the current node according to the maximum counter value of all nodes on the blockchain.

A configuration information reading unit 14, configured to periodically read batch job configuration information, where the configuration information includes at least one of: the job ID, the intelligent contract ID, the job scheduling policy, the job execution validity period, and the job last scheduling time may be specifically referred to table 1 above;

and the batch job consensus unit 15 is used for performing consensus operation on the batch jobs to be added, which need to be executed and are not added into the data queue according to the configuration information.

A to-be-added batch job determining unit 16, configured to determine the to-be-added batch job, as shown in fig. 5, where the to-be-added batch job determining unit includes: the system comprises a scheduling information reading module 161 and a to-be-added batch job determining module 162, wherein the scheduling information reading module 161 is used for reading batch job scheduling information, and the scheduling information comprises at least one of the following: the job scheduling ID, the job scheduling time, and the current state of the job may be specifically referred to in table 2; and the to-be-added batch job determining module 162 is configured to determine the to-be-added batch job according to the configuration information and the scheduling information.

An updating unit 17, configured to update the configuration information and the scheduling information.

For specific execution processes of the units and the modules, reference may be made to the description in the foregoing method embodiments, and details are not described here again.

In practical operation, the units and the modules may be arranged singly or in combination, and the invention is not limited thereto.

For a better understanding of embodiments of the present invention, an example is given below. In this example, as with all typical blockchain systems, the system includes a plurality of blockchain nodes, each belonging to a different participating enterprise of the federation chain. The nodes agree on the operation to be executed through a consensus protocol. The consensus is a series of operations for transmitting information between nodes through a computer network communication protocol, and the aim of the consensus is to ensure that all nodes in a blockchain system receive complete and consistent information, so that consistent operations can be executed, and the consistency of data maintained by each node is further ensured.

The business logic provided by the blockchain system is realized in the form of an intelligent contract, namely a computer program code, and a user triggers a business (namely, operation) function by calling the intelligent contract, wherein key information such as an intelligent contract ID, a function, a parameter and the like is required to be provided when the intelligent contract is called.

The data processing apparatus according to the embodiment of the present invention is described in detail below by taking a node on a blockchain as an example.

Fig. 6 is a block diagram of an exemplary structure of a data processing apparatus on a blockchain according to an embodiment of the present invention, as shown in fig. 6, the apparatus including: the system comprises an online job receiving module 101 (or called online transaction receiving module), a batch job configuration module 102, a batch job scheduling module 103, a consensus module 104, a job queue 105 (or called transaction queue), an execution module 106, a data storage module 107, a job counter 108, and an interrupter 109, which are described in detail below.

The online job receiving module 101 is responsible for receiving an online job request sent by a client, after receiving the job request of the client, the online job receiving module performs validity check on the job request, and if the job is illegal, such as a format error, a contract does not exist, a function does not exist, and the number of parameters is not equal, error information is returned to the client; if the job is legal, the job is sent to the consensus module.

The batch job configuration module 102 stores configuration information and scheduling records of batch jobs, where the configuration information of a job includes a job ID, an intelligent contract ID, a function, a parameter, a scheduling policy, a validity period, a serial number, a last execution time, and the like, and may specifically refer to table 1 above. The scheduling record of the job is a historical scheduling record of the job, and includes a scheduling ID, a job ID, a scheduling time, and the like, which can be specifically referred to table 2 above.

The batch job scheduling module 103 periodically accesses the batch job configuration module to obtain the batch job configuration information, and if the batch job is in the scheduled time window, accesses the scheduling record of the job. If the record of the job at the current scheduling time does not exist, the fact that the job is not scheduled at the current scheduling time indicates that the batch scheduling task should be added newly. Specifically, the scheduling module records the batch job into the job scheduling record of the batch job configuration module 102, where the scheduling ID of the batch job is the largest ID in the scheduling records plus 1; then, modifying the job configuration information of the batch job configuration module 102, and modifying the last scheduling time of the job to the current time; and finally, sending information such as the scheduling ID, the intelligent contract ID, the function, the parameter and the like to a consensus module so as to inform all nodes in the block chain network to start executing batch jobs.

The consensus module 104 is responsible for interacting with other blockchain nodes, ensuring that information of batch jobs and online jobs is sent to all nodes, or received from other nodes. If agreement is achieved between the nodes, the batch job and the on-line job are added to the job queue.

The job queue 105, corresponding to the data queue, is responsible for buffering batch jobs and online jobs to be executed, and table 3 shows data information in the queue, as shown in the following table. The execution module obtains information required for executing the batch jobs and the online jobs from the job queue.

Contract ID	Function(s)	Parameter(s)	Type (B)
				e1c88f2c9c……a	transfer	[2c578e4……2,a1f32be7……e,20]	On-line
db0d7ca0a4……9	monthend	[]	Batch size
				e1c88f2c9c……a	transfer	[30ecc34……0,2f58bd9d……f,10]	On-line

TABLE 3

And the execution module 106 executes the intelligent contracts corresponding to the batch jobs and the online jobs according to the intelligent contract ID, the function and the parameters of the jobs, and updates the data storage.

The data storage module 107 stores service data maintained by batch jobs and online jobs, such as information of account numbers and balances.

The job counter 108 stores the number of times of modification to the service data during the execution of the current batch job. In the process of executing the batch job, the execution module increments the counter by 1 each time the data storage module is accessed to write the service data, and the data structure of the counter may specifically refer to table 4 below. And when the batch operation is finished, resetting the counter.

Job ID

Counter with a memory

TABLE 4

The interrupter 109 stores the location (i.e., the interrupt point) where execution is to be paused during the execution of the batch job, and the data structure of the interrupter may be seen in table 5 below. In the process of executing the batch operation, after the execution module accesses the data storage module each time to write the service data, the execution module not only increases the counter by 1, but also reads the interrupt point, if the interrupt point is set, the value of the counter is compared with the value of the interrupt point, if the interrupt point is matched with the value of the interrupt point, the execution module suspends the execution of the batch operation, and then executes the subsequent online operation, and after the execution of the online operation is finished, the execution of the batch operation is resumed.

Job ID

Break point

TABLE 5

Fig. 7 is a flowchart of processing a block link point to online operation request according to an embodiment of the present invention, as shown in fig. 7, the flowchart includes:

step S101, the client side initiates an online operation request, and the online operation request information comprises information such as ID, functions and parameters of the intelligent contract.

Step S102, the online job receiving module receives the online job request, and conducts preliminary check on the request, such as whether the intelligent contract ID exists, whether the parameter quantity and the type are correct, and the like, if the request has errors, the online job receiving module refuses the job and returns error information to the client; if the check passes, the job information is sent to the consensus module.

And step S103, the consensus module reads the counter and judges whether batch jobs are executed or not, if no batch jobs are executed, the step S104 is carried out, and if not, the step 106 is carried out.

And step S104, the consensus module broadcasts the operation information to other nodes to complete consensus.

In step S105, the consensus module adds the job information to the job queue.

And step S106, continuously reading the value of the interruption point, and judging whether the batch operation has set the interruption point or not. If the interruption point has been set, go back to step S104, otherwise go to step S07.

Step S107, the value of the counter is broadcasted to other nodes together with the operation information for consensus. And when the consensus modules of other nodes receive the consensus requests, reading the values of the counters of the other nodes and broadcasting the values to the other nodes.

Step S108, in the consensus process, the consensus module of each node obtains the counter values and the operation information of all the nodes, the maximum value of the counter is taken out, and a variable is added to the value to serve as an interruption point.

Step S109: the consensus module sets an interrupt point while adding the job to the job queue.

FIG. 8 is a flowchart of a batch job scheduling module generating a batch job according to an embodiment of the present invention, and as shown in FIG. 8, the flowchart includes:

step S201, the batch job scheduling module periodically accesses and reads the batch job configuration module, and obtains and checks each piece of valid job configuration information.

Step S202, if the operation is in the execution window, generating the batch operation to be executed, and submitting the batch operation to the consensus module.

In step S203, the consensus module reads the job queue, checks whether the batch job already exists, if so, proceeds to step 204, otherwise, proceeds to step 205.

In step S204, if the job already exists, it indicates that other nodes have initiated the job, and thus no other operation is required.

Step S205, if the job does not exist, it indicates that the node is the first node to trigger the batch job, and the consensus module broadcasts the job information to other nodes for consensus.

Step S206: after the consensus passes, the consensus module of each node adds the job to the job queue.

FIG. 9 is a flowchart of an execution module executing a batch job and an online job, as shown in FIG. 9, the flowchart including:

in step S301, the execution module reads the job queue and fetches the first record.

Step S302, judging whether the record is an online operation or a batch operation, if the record is the online operation, performing step S303, and if not, performing step S304;

step S303, executing the complete online operation, and accessing the data storage module to modify the corresponding service data.

And step S304, executing the batch job step until the business data needs to be modified.

Step S305, the execution module accesses the data storage module and modifies the service data.

In step S306, the execution module increments the counter by 1.

Step S307, the execution module reads the interrupt point, determines whether the interrupt point has been set, if the interrupt point is zero, it indicates that the interrupt point has not been set, returns to step S304, executes the next step of the job, and if the interrupt point is non-zero, it indicates that the interrupt point has been set, and then proceeds to step S308.

Step S308, the execution module compares the counter with the interrupt point, if the two values are not matched, the execution module indicates that the interrupt point is not executed, the step S303 is returned, and the next step of the operation is executed. If the two values match, it indicates that the execution has reached the interruption point, and step S309 is performed.

In step S309, the execution module suspends the execution of the batch job, fetches the first online job in the queue, and executes the online job.

Step S310, repeating step S309 until the online operation of the operation queue is completed, clearing the interruption point, resuming the execution of the batch operation, returning to step S303, and executing the next step of the batch operation.

In step S311, when the batch job is executed, the counter is cleared.

In actual operation, for a batch job, the interruption point can be set for multiple times, so that the online job from the client can be better executed, the waiting of the online job client is reduced, and the experience of the user is improved.

As can be seen from the foregoing description, the blockchain system supporting batch job processing functions and the method for processing batch jobs on the system according to the embodiments of the present invention can implement batch processing in supporting traditional enterprise services on blockchains, especially federation chains. In the running process of batch operation, the system can still concurrently process on-line operation, and the batch operation is automatically triggered according to configuration.

FIG. 10 is a schematic diagram of an electronic device according to an embodiment of the invention. The electronic device shown in fig. 10 is a general-purpose data processing apparatus comprising a general-purpose computer hardware structure including at least a processor 1001 and a memory 1002. The processor 1001 and the memory 1002 are connected by a bus 1003. The memory 1002 is adapted to store one or more instructions or programs that are executable by the processor 1001. The one or more instructions or programs are executed by the processor 1001 to implement the steps of the above-described method for processing data on a blockchain.

The processor 1001 may be an independent microprocessor or a set of one or more microprocessors. Thus, the processor 1001 implements the processing of data and the control of other devices by executing commands stored in the memory 1002 to thereby execute the method flows of the embodiments of the present invention as described above. The bus 1003 connects the above components together, and also connects the above components to a display controller 1004 and a display device and an input/output (I/O) device 1005. Input/output (I/O) devices 1005 may be a mouse, keyboard, modem, network interface, touch input device, motion sensing input device, printer, and other devices known in the art. Typically, input/output (I/O) devices 1005 are connected to the system through an input/output (I/O) controller 1006.

The memory 1002 may store, among other things, software components such as an operating system, communication modules, interaction modules, and application programs. Each of the modules and applications described above corresponds to a set of executable program instructions that perform one or more functions and methods described in embodiments of the invention.

Embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the steps of the above method for processing data on a blockchain.

In summary, the embodiments of the present invention provide a blockchain system supporting batch processing function and a solution for processing batch jobs on the system, so as to support batch processing in traditional enterprise business on blockchains, especially federation chains. And the system can still concurrently process the online operation in the running process of the batch operation, and the batch task is automatically triggered according to the configuration. Compared with the prior art, the data processing scheme provided by the embodiment of the invention has the advantages that before the batch tasks are executed, a large amount of data does not need to be transmitted between the block chain and the application, when the batch tasks are executed, the application does not need to send a large amount of network requests to the block chain, the system can still concurrently execute online operation in the batch operation execution process, the influence on the online service is minimized, in addition, only one record is generated by one batch processing, and the negative influence on the analysis of the block chain data is minimized. Therefore, the embodiment of the invention expands the function of the block chain, improves the service processing efficiency and reduces the difficulty of service development.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the hardware + program class embodiment, since it is substantially similar to the method embodiment, the description is simple, and the relevant points can be referred to the partial description of the method embodiment.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

Although the present application provides method steps as described in an embodiment or flowchart, additional or fewer steps may be included based on conventional or non-inventive efforts. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. When an actual apparatus or client product executes, it may execute sequentially or in parallel (e.g., in the context of parallel processors or multi-threaded processing) according to the embodiments or methods shown in the figures.

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

The terms "comprises," "comprising," or any other variation thereof, in the embodiments of this specification are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, article, or apparatus that comprises the recited elements is not excluded.

For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. Of course, in implementing the embodiments of the present description, the functions of each module may be implemented in one or more software and/or hardware, or a module implementing the same function may be implemented by a combination of multiple sub-modules or sub-units, and the like. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may therefore be considered as a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

As will be appreciated by one skilled in the art, embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.

The embodiments of this specification may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The described embodiments may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of an embodiment of the specification. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.

The above description is only an example of the embodiments of the present disclosure, and is not intended to limit the embodiments of the present disclosure. Various modifications and variations to the embodiments described herein will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present specification should be included in the scope of the claims of the embodiments of the present specification.

Claims

1. A method of data processing on a blockchain, the method comprising:

reading data in a data queue, wherein the data queue comprises batch job data and online job data;

responding to the data as batch job data, and executing batch jobs corresponding to the batch job data;

suspending execution of the batch jobs according to the job interruption information of the batch jobs, and reading online job data in the data queue;

executing the online operation corresponding to the online operation data;

and responding to the completion of the online operation execution, and continuing to execute the batch operation.

2. The method of claim 1, wherein said performing the on-line operation comprises:

and completing the execution of the online operation corresponding to all the online operation data in the data queue.

3. The method of data processing on a blockchain of claim 1, further comprising:

and recording the times of modifying the corresponding job data in the batch job execution process.

4. The method of data processing on a blockchain of claim 3, further comprising:

and setting a counter for the batch operation according to a preset rule, wherein the counter is used for recording the number of times of modifying corresponding operation data in the batch operation executing process.

5. The method of data processing on a blockchain of claim 1, further comprising:

receiving an online operation request from a client, and judging whether the online operation request is legal or not according to a preset verification rule;

responding to the online operation request to be legal, and performing consensus operation on the online operation request;

and adding the online operation request into the data queue.

6. The method of claim 5, wherein prior to adding the online job request to the data queue, the method further comprises:

judging whether the batch operation currently executed exists or not;

and responding to the judgment result that the online operation request does not exist, and adding the online operation request into the data queue.

7. The method of data processing on a blockchain of claim 6, further comprising:

in response to the judgment result that the executed batch jobs exist, judging whether the executed batch jobs have set job interruption information;

and responding to the judgment result that the operation interruption information is set, and adding the online operation request into the data queue.

8. The method of data processing on a blockchain of claim 7, further comprising:

and in response to the judgment result that the job interruption information is not set, setting the job interruption information for the currently executed batch job of the current node according to the counter values of the currently executed batch jobs to which all nodes on the block chain belong, wherein the counter is used for recording the number of times that the corresponding job data is modified by the currently executed batch job.

9. The method of claim 8, wherein setting the job interrupt information for the executing batch job of the current node according to the counter value of the executing batch job to which all nodes on the blockchain belong comprises:

and setting operation interruption information for the executing batch operation of the current node according to the maximum counter values of all nodes on the block chain.

10. The method of data processing on a blockchain of claim 1, further comprising:

periodically reading batch job configuration information, wherein the configuration information comprises at least one of the following: the method comprises the steps of obtaining a job identifier ID, an intelligent contract ID, a required function, required parameters, a job scheduling strategy, a job execution validity period and a job final scheduling time;

and performing consensus operation on batch jobs to be added, which need to be executed and are not added into the data queue, according to the configuration information, and adding the batch jobs to be added into the data queue.

11. The method of claim 10, wherein the batch job to be added is determined by:

reading batch job scheduling information, wherein the scheduling information comprises at least one of the following: job scheduling ID, job scheduling time and job current state;

and determining the batch jobs to be added according to the configuration information and the scheduling information.

12. The method of data processing on a blockchain of claim 11, further comprising:

and updating the configuration information and the scheduling information.

13. An apparatus for data processing on a blockchain, the apparatus comprising:

the data reading unit is used for reading data in a data queue, and the data queue comprises batch job data and online job data;

the batch job execution unit is used for responding to the fact that the data are batch job data and executing batch jobs corresponding to the batch job data;

a job suspending unit configured to suspend execution of the batch job according to job interruption information of the batch job;

the data reading unit is also used for reading online operation data in the data queue;

the online operation execution unit is used for executing the online operation corresponding to the online operation data;

the batch job execution unit is further used for responding to the completion of the online job execution and continuing to execute the batch job.

14. The apparatus of claim 13, wherein the on-line job execution unit is further configured to:

15. The apparatus for data processing on a blockchain of claim 13, wherein the apparatus further comprises:

and the modification frequency recording unit is used for recording the frequency of modifying the corresponding job data in the batch job execution process.

16. The apparatus for data processing on a blockchain of claim 15, wherein the apparatus further comprises:

and the counter setting unit is used for setting a counter for the batch operation according to a preset rule, and the counter is used for recording the number of times of modifying the corresponding operation data in the batch operation executing process.

17. The apparatus for data processing on a blockchain of claim 13, wherein the apparatus further comprises:

the online job request receiving unit is used for receiving an online job request from the client;

the verification unit is used for judging whether the online operation request is legal or not according to a preset verification rule;

the online operation consensus unit is used for responding to the online operation request to be legal and carrying out consensus operation on the online operation request;

and the online work request adding unit is used for adding the online work request into the data queue.

18. The apparatus for data processing on a blockchain of claim 17, wherein the apparatus further comprises:

a job execution judgment unit for judging whether there is a batch job currently being executed;

the online operation request adding unit is specifically configured to: and adding the online job request into the data queue in response to the judgment result of the job execution judgment unit not existing.

19. The apparatus for data processing on a blockchain of claim 18, wherein the apparatus further comprises:

an interruption information judgment unit configured to judge whether or not job interruption information has been set for the batch job being executed, in response to a judgment result of the job execution judgment unit being that there is a batch job being executed;

the online operation request adding unit is specifically configured to: and adding the online job request into the data queue in response to the judgment result of the interruption information judgment unit that the job interruption information is set.

20. The apparatus for data processing on a blockchain of claim 19, wherein the apparatus further comprises:

and the interrupt information setting unit is used for responding to the judgment result of the interrupt information judgment unit that the job interrupt information is not set, and setting the job interrupt information for the batch job being executed of the current node according to the counter value of the batch job being executed of all nodes on the block chain, wherein the counter is used for recording the number of times of the batch job being executed for the corresponding job data.

21. The apparatus of claim 20, wherein the interrupt information setting unit is specifically configured to:

and setting operation interruption information for the executing batch operation of the current node according to the maximum counter value of the executing batch operation of all nodes on the block chain.

22. The apparatus for data processing on a blockchain of claim 13, wherein the apparatus further comprises:

the configuration information reading unit is used for periodically reading batch job configuration information, and the configuration information comprises at least one of the following: the method comprises the following steps of (1) job ID, intelligent contract ID, job scheduling strategy, job execution validity period and job final scheduling time;

the batch job consensus unit is used for performing consensus operation on the batch jobs to be added which need to be executed and are not added into the data queue according to the configuration information;

and the batch job adding unit is used for adding the batch jobs to be added into the data queue.

23. The apparatus for data processing on a blockchain of claim 22, wherein the apparatus further comprises:

a to-be-added batch job determination unit for determining the to-be-added batch job,

the to-be-added batch job determination unit includes:

the scheduling information reading module is used for reading batch job scheduling information, and the scheduling information comprises at least one of the following: job scheduling ID, job scheduling time and job current state;

and the to-be-added batch job determining module is used for determining the to-be-added batch job according to the configuration information and the scheduling information.

24. The apparatus for data processing on a blockchain of claim 23, wherein the apparatus further comprises:

and the updating unit is used for updating the configuration information and the scheduling information.

25. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method for data processing on a blockchain according to any one of claims 1 to 12 are implemented when the program is executed by the processor.

26. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method for data processing on a blockchain according to any one of claims 1 to 12.