CN111930760A - Transaction flow distributed batch processing method and system - Google Patents

Abstract

The invention discloses a transaction flow distributed batch processing method and a system thereof, comprising a scheduling module, a transaction flow module, a clearing module and a profit sharing calculation module, wherein the scheduling module, the transaction flow, the clearing module and the profit sharing calculation module are connected in sequence , the scheduling module is used to trigger the clearing module to execute clearing transactions, the transaction flow will be synchronized to the transaction storage database in the clearing module through the database of the transaction flow module in real time, and various types of order consumption data will be found at regular intervals. And through cleaning, conversion and filtering, after successful liquidation, it is stored in the database of the liquidation module for the profit distribution calculation module to extract and calculate the profit distribution fee. The present invention can support the increase in data volume in the business dimension and liquidation, and does not affect the execution of batches in other dimensions. Time, can support rapid data migration and facilitate various business processing.

Description

A transaction flow distributed batch processing method and system thereof

technical field

The invention relates to the technical field of network business transactions, in particular to a transaction flow distributed batch processing method and system.

Background technique

In the prior art, the system uses the business (such as customer) dimension to sort and calculate, this method will make when the transaction volume of a certain business dimension is unevenly distributed, the transaction volume of individual business grows rapidly, and the data is too large, This kind of business will affect the sorting calculation time of all customers, causing the system to execute slowly, thus affecting the running time of the entire system.

SUMMARY OF THE INVENTION

In order to overcome the deficiencies of the prior art, the present invention provides a distributed batch processing method and system for the transaction flow. When the transaction volume of a single business increases, the present invention solves the problem that the increase in the data volume of the business dimension affects the batch running of other dimensions. execution time.

The technical solution adopted by the present invention to solve the technical problem is: a transaction flow distributed batch processing method, and its improvement lies in that the method comprises the following steps:

S1: The scheduling module regularly imports the transaction flow in the transaction flow module into the clearing module;

S2: the database of the transaction flow module synchronizes the transaction flow with the transaction storage database of the clearing module;

S3: The clearing module finds out the transaction data, filters it through the cleaning conversion filter, and converts it into clearing flow with clearing batch numbers to form clearing batch number data. The table format is stored in the database of the clearing module.

S4: The scheduling module extracts the liquidation batch table from the database of the liquidation module to the profit distribution calculation module to calculate the profit distribution fee.

As an improvement of the above technical solution, in the above step S2, the steps of cleaning, converting, filtering and storing in the database of the clearing module include:

S21: The clearing module inquires about the successful transaction flow;

S22: Import the successful transaction flow into the clearing module;

S23: After the clearing module converts the successful clearing flow into the required fields by associating with other transaction flow tables, a clearing flow with a clearing batch number or a clearing batch with an empty clearing batch number is formed;

S24: Update the clearing batch number information of each transaction to form a clearing batch number table, which is stored in the database of the clearing module through the scheduling module.

As an improvement of the above technical solution, the transaction flow includes data of consumption, order, payment, refund or other transaction types.

As an improvement of the above technical solution, the liquidation batch table includes liquidation date, batch number, group number, liquidation status and number of transactions.

A distributed batch processing system for transaction flow, which is improved in that it includes a scheduling module, a transaction flow module, a clearing module and a profit sharing calculation module, wherein the scheduling module, the transaction flow, the clearing module and the profit sharing calculation module are connected in sequence, The scheduling module is used to trigger the clearing module to execute the clearing transaction, and the transaction flow will be synchronized to the transaction storage database in the clearing module through the database of the transaction flow module in real time. Through cleaning, conversion and filtering, after successful liquidation, it is stored in the database of the liquidation module for the profit-sharing calculation module to extract and calculate the profit-sharing fee.

As an improvement of the above technical solution, the steps of cleaning, converting, filtering and storing in the database of the clearing module include:

S21: The clearing module inquires about the successful transaction flow;

S22: Import the successful transaction flow into the clearing module;

S23: After the clearing module converts the successful clearing flow into the required fields by associating with other transaction flow tables, a clearing flow with a clearing batch number or a clearing batch with an empty clearing batch number is formed;

S24: Update the clearing batch number information of each transaction to form a clearing batch number table, and store it in the database of the clearing module.

As an improvement of the above technical solution, the transaction flow cleared by the clearing module includes data of consumption, order or other transaction types.

As an improvement of the above technical solution, the liquidation batch table includes liquidation date, batch number, group number, liquidation status and number of transactions.

A computer device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, the improvement is that the processor implements any one of the methods described above when executing the computer program .

A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for executing any one of the methods described above.

The beneficial effects of the present invention are: the present invention can support the increase and clearing of the data volume in the business dimension, without affecting the execution time of batch running in other dimensions, can support rapid data migration, and facilitate various business processing.

Description of drawings

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

FIG. 1 is a system flow diagram of the present invention.

Figure 2 is a system flow diagram of the present invention.

Figure 3 is a flow chart of a clearing transaction of the present invention.

Detailed ways

The concept, specific structure and technical effects of the present invention will be clearly and completely described below with reference to the embodiments and accompanying drawings, so as to fully understand the purpose, characteristics and effects of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative efforts are all within the scope of The scope of protection of the present invention. In addition, all the coupling/connection relationships involved in the patent do not mean that the components are directly connected, but refer to a better coupling structure by adding or reducing coupling accessories according to the specific implementation. Various technical features in the present invention can be combined interactively on the premise of not contradicting each other.

1 and 2, the present invention discloses a transaction flow distributed batch processing method, the method includes the following steps:

S1: The scheduling module 1 regularly imports the transaction flow in the transaction flow module 2 into the clearing module 3;

S2: the database of the transaction flow module 2 synchronizes the transaction flow with the transaction storage database of the clearing module 3;

S3: The clearing module 3 finds out the transaction data, filters it through the cleaning conversion filter, and converts it into a clearing flow with a clearing batch number to form clearing batch number data. The secondary table format is stored in the database of the clearing module 3.

S4: The scheduling module 1 extracts the liquidation batch table from the database of the liquidation module 3 to the profit distribution calculation module 4 to calculate the distribution fee.

In the above-mentioned step S2, the steps of cleaning, converting, filtering and storing in the database of the clearing module 3 include:

S21: The clearing module 3 inquires about the successful transaction flow;

S22: Import the successful transaction flow into the clearing module 3;

S23: After the clearing module 3 converts the successful clearing flow into the required fields by associating with other transaction flow tables, a clearing flow with a clearing batch number or a clearing batch with an empty clearing batch number is formed;

S24: Update the clearing batch number information of each transaction to form a clearing batch number table, which is stored in the database of the clearing module 3 through the scheduling module 1.

In the above embodiment, referring to FIG. 3 , the transaction flow will be synchronized to the transaction storage database of the clearing module 3 through the database of the transaction flow module 2 in real time to form a clearing transaction table, which will be triggered by the scheduling module 1 Execute, find out a batch of data of the clearing transaction flow of different types of order consumption, etc. at regular intervals, and query the successful transaction flow through cleaning, conversion and filtering, and convert it into the required fields by associating other transaction flow tables. into a clearing stream and stored in the database.

Further, the transaction flow includes data of consumption, order, payment, refund or other transaction types, and supports multiple types of clearing transactions.

Still further, the liquidation batch table includes liquidation date, batch number, group number, liquidation status and number of transactions. The clearing date, batch number, group number, clearing status and number of transactions are fully integrated, so that the transaction flow data is fully transparent, and it is convenient to clear the flow transaction data.

When the scheduling module 1 of the present invention regularly extracts data from the transaction flow module 2 to the clearing module 3, a clearing batch number field will be attached to each clearing flow. Set a clearing batch number for N pen. The group number is determined by the number of machine nodes. The number of machine nodes X is obtained from the configuration file and can be dynamically modified, and the number of execution X group numbers is allocated according to the number of X nodes of the machine. Assuming that the number of nodes is X, It is from 1 to X. Each group number is assigned N transactions, and according to the clearing date, batch number, group number, clearing status, and number of transactions, it is saved as a clearing batch table. When the scheduling module 1 executes the clearing run batch task , as shown in Figure 1, the scheduling module 1 will send execution commands to multiple servers in the profit-sharing calculation module 4 according to the number of machine nodes X allocated by the liquidation batch table, and each server will follow the commands sent by the scheduling module 1. Select a machine, and the server will obtain the transaction flow related to this group number to clear the calculation flow, so as to achieve the purpose of multi-service distributed batch processing.

A transaction flow distributed batch processing system, comprising a scheduling module 1, a transaction flow module 2, a clearing module 3 and a profit distribution calculation module 4, the scheduling module 1, the transaction flow module 2, the clearing module 3 and the profit distribution calculation module 4 Connected in sequence, the scheduling module 1 is used to trigger the clearing module 3 to execute the clearing transaction, and the transaction flow will be synchronized to the transaction storage database in the clearing module 3 through the database of the transaction flow module 2 in real time. The consumption data of various types of orders are collected and filtered through cleaning, transformation and filtering. After successful liquidation, they are stored in the database of the liquidation module 3 for the profit distribution calculation module 4 to extract and calculate the profit distribution fee. When the data volume of the business dimension increases, the running of other dimensions is reduced. Batch execution time, supports rapid data migration, and facilitates various business processing.

The steps to clean, transform, filter and store in the clearing module's database include:

S21: The clearing module 3 inquires about the successful transaction flow;

S22: Import the successful transaction flow into the clearing module 3;

S23: After the clearing module 3 converts the successful clearing flow into the required fields by associating with other transaction flow tables, a clearing flow with a clearing batch number or a clearing batch with an empty clearing batch number is formed;

S24 : Update the clearing batch number information of each transaction to form a clearing batch number table, and store it in the database of the clearing module 3 .

In the above embodiment, as shown in FIG. 3 , the transaction flow will be synchronized to the transaction storage database of the clearing module 3 through the database in real time to form a clearing transaction table. The clearing transaction table will be triggered and executed by the scheduling module 1. After a period of time, find out a batch of data of the clearing transaction flow of different types of order consumption, etc., through the cleaning conversion filter, query the successful status that needs to be imported into the clearing flow, and convert it into the required fields by associating other transaction flow tables, It is converted into clearing flow and stored in the database of clearing module 3.

Further, the transaction flow includes data of consumption, order, payment, refund or other transaction types. Multiple clearing transaction types are supported.

Still further, the liquidation batch table includes liquidation date, batch number, group number, liquidation status and number of transactions. The clearing date, batch number, group number, clearing status and number of transactions are fully integrated, so that the transaction flow data is fully transparent, and it is convenient to clear the flow transaction data.

When the scheduling module 1 of the present invention regularly extracts data from the transaction flow module 2 to the clearing module 3, a clearing batch number field will be attached to each clearing flow. Set a clearing batch number for N pen. The group number is determined by the number of machine nodes. The number of machine nodes X is obtained from the configuration file, which can be dynamically modified, and the number of X group numbers is allocated and executed according to the number of X nodes of the machine, assuming that the number of nodes is X , that is, 1 to X, each group number is allocated N transactions, and according to the clearing date, batch number, group number, clearing status, and number of transactions, it is saved as a clearing batch table, and the scheduling module 1 executes the clearing and running batch tasks. At the same time, the scheduling module 1 will send execution commands to multiple servers in the profit-sharing calculation module 4 according to the number of machine nodes X allocated by the liquidation batch table, and each server will select a machine according to the command sent by the scheduling module. The server will obtain the transaction flow related to this group number to clear the calculation flow, so as to achieve the purpose of multi-service distributed batch processing.

The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements any one of the methods described above when the processor executes the computer program .

The present invention also provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program for executing any one of the methods described above.

The beneficial effects of the present invention are: the present invention can support the increase and clearing of the data volume in the business dimension, without affecting the execution time of batch running in other dimensions, can support rapid data migration, and facilitate various business processing.

The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments, and those skilled in the art can also make various equivalent deformations or replacements on the premise that does not violate the spirit of the present invention , these equivalent modifications or substitutions are all included within the scope defined by the claims of the present application.

Claims (10)

1. a transaction flow distributed batch processing method, is characterized in that, described method comprises the following steps: S1: The scheduling module regularly imports the transaction flow in the transaction flow module into the clearing module; S2: the database of the transaction flow module synchronizes the transaction flow with the transaction storage database of the clearing module; S3: The clearing module finds out the transaction data, filters it through the cleaning conversion filter, and converts it into clearing flow with clearing batch numbers to form clearing batch number data. The table format is stored in the database of the clearing module. S4: The scheduling module extracts the liquidation batch table from the database of the liquidation module to the profit distribution calculation module to calculate the profit distribution fee. 2. A transaction flow distributed batch processing method according to claim 1, characterized in that, in the above step S2, the steps of cleaning, converting, filtering and storing in the database of the clearing module include: S21: The clearing module inquires about the successful transaction flow; S22: Import the successful transaction flow into the clearing module; S23: After the clearing module converts the successful clearing flow into the required fields by associating with other transaction flow tables, a clearing flow with a clearing batch number or a clearing batch with an empty clearing batch number is formed; S24: Update the clearing batch number information of each transaction to form a clearing batch number table, which is stored in the database of the clearing module through the scheduling module. 3 . The distributed batch processing method for transaction flow according to claim 1 , wherein the transaction flow includes data of consumption, order, payment, refund or other transaction types. 4 . 4 . The distributed batch processing method for transaction flow according to claim 1 , wherein the liquidation batch table includes liquidation date, batch number, group number, liquidation status and number of transactions. 5 . 5. A transaction flow distributed batch processing system, characterized in that it comprises a scheduling module, a transaction flow module, a clearing module and a profit sharing calculation module, wherein the scheduling module, the transaction flow, the clearing module and the profit sharing calculation module are connected in sequence, The scheduling module is used to trigger the clearing module to execute the clearing transaction, and the transaction flow will be synchronized to the transaction storage database in the clearing module through the database of the transaction flow module in real time. Through cleaning, conversion and filtering, after successful liquidation, it is stored in the database of the liquidation module for the profit-sharing calculation module to extract and calculate the profit-sharing fee. 6. A transaction flow distributed batch processing system according to claim 5, wherein the step of cleaning, converting, filtering and storing in the database of the clearing module comprises: S21: The clearing module inquires about the successful transaction flow; S22: Import the successful transaction flow into the clearing module; S23: After the clearing module converts the successful clearing flow into the required fields by associating with other transaction flow tables, a clearing flow with a clearing batch number or a clearing batch with an empty clearing batch number is formed; S24: Update the clearing batch number information of each transaction to form a clearing batch number table, and store it in the database of the clearing module. 7 . The distributed batch processing system for transaction flow according to claim 5 , wherein the transaction flow includes data of consumption, order, payment, refund or other transaction types. 8 . 8 . The distributed batch processing system for transaction flow according to claim 5 , wherein the liquidation batch table includes liquidation date, batch number, group number, liquidation status and number of transactions. 9 . 9. A computer device comprising a memory, a processor and a computer program stored on the memory and running on the processor, wherein the processor implements any of claims 1 to 4 when the processor executes the computer program method described in item. 10. A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for executing the method of any one of claims 1 to 4.

Images