CN110175069B - Distributed transaction processing system and method based on broadcast channel - Google Patents
Distributed transaction processing system and method based on broadcast channel Download PDFInfo
- Publication number
- CN110175069B CN110175069B CN201910419528.5A CN201910419528A CN110175069B CN 110175069 B CN110175069 B CN 110175069B CN 201910419528 A CN201910419528 A CN 201910419528A CN 110175069 B CN110175069 B CN 110175069B
- Authority
- CN
- China
- Prior art keywords
- data
- main server
- node
- data node
- central
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 69
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000004891 communication Methods 0.000 claims abstract description 37
- 230000005540 biological transmission Effects 0.000 claims abstract description 36
- 238000003672 processing method Methods 0.000 claims abstract description 10
- 230000008569 process Effects 0.000 claims description 40
- 230000002085 persistent effect Effects 0.000 claims description 10
- 238000012790 confirmation Methods 0.000 claims description 3
- 230000002045 lasting effect Effects 0.000 claims description 3
- 230000004044 response Effects 0.000 abstract description 9
- 238000005516 engineering process Methods 0.000 abstract description 3
- 230000002688 persistence Effects 0.000 description 7
- 238000013461 design Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 239000002253 acid Substances 0.000 description 2
- 230000002457 bidirectional effect Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000002955 isolation Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/466—Transaction processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/542—Event management; Broadcasting; Multicasting; Notifications
Abstract
The invention provides a distributed transaction processing system based on a broadcast channel, which comprises a main server side, a central data node, a plurality of data nodes and a plurality of terminals, wherein the central data node is connected with the main server side; the main server is connected with the central data node network and is connected with a plurality of data node networks; the terminal is in communication connection with the data node; the central data node performs data transmission through a broadcast channel, and establishes broadcast communication connection with the main server side and the data node respectively. A distributed transaction processing method based on a broadcast channel is realized by combining the system. The distributed transaction processing system and the method based on the broadcast channel, provided by the invention, use the traditional database technology to ensure the data consistency of the system, ensure that the response time of the system is not greatly related to the scale growth of the system, and only ensure the communication delay between nodes, so that the availability of the system is excellent; and a consistent algorithm based on the message is not used, so that information output and transmission among nodes are reduced, and the processing efficiency of the message is effectively improved.
Description
Technical Field
The invention relates to the technical field of computers and communication, in particular to a distributed transaction processing system based on a broadcast channel and a distributed transaction processing method based on the broadcast channel.
Background
In a distributed system, to ensure that data is not lost, the data is often stored in multiple copies. However, some anomalies will cause data to appear inconsistent, and therefore, consistency is a very important part of a distributed system. The problem of data consistency in a distributed system is also referred to as a distributed transaction problem. For distributed transactions, experts have proposed a variety of algorithms that address the problem of distributed transactions. Many of the existing algorithms for distributed transaction processing implement data consistency based on the manner in which messages are passed between distributed systems. Such as a two-phase commit (2 PC), a three-phase commit (3 PC), a Paxos algorithm, a Raft algorithm, or a ZAB algorithm. Such algorithms may be integrated into a distributed system, implementing complex distributed computations.
Distributed transaction processing algorithms based on message passing are very sensitive to the quality and delay of message passing, so that the algorithms are used in a data center, and the network communication quality and bandwidth between nodes are guaranteed in the environment of a data center. However, in most cases, the distributed computing that needs to be processed often needs to participate together by nodes that are also very geographically dispersed, in this case, if the distributed transaction processing algorithm of the entire distributed system is based on the algorithm of the message, the entire processing flow is very long, and a large amount of computing and bandwidth resources need to be consumed to achieve synchronization, and the expansibility of the system is poor, so that great challenges can be encountered in terms of execution efficiency and usability of the system.
In this case, a solution using divide-and-conquer has been proposed, such as: in these systems, there is a need for data that can be layered or chunked into smaller systems in the problem to be handled, but if some of the data in the system is system-wide unique, all of the subsystems will use the data, the system layering or chunking process will result in the corruption of the consistency (ACID) of such data. The algorithm may have to revert to using distributed transaction processing based on messaging between distributed systems in a wide area network environment, failing to address the technical problem.
Note that: ACID, an abbreviation for four basic elements that refer to the correct execution of a database transaction. Comprising: atomicity (atom), consistency (Consistency), isolation (Isolation), durability (Durability).
Disclosure of Invention
The invention provides a distributed transaction processing system based on a broadcast channel, which aims to overcome the technical defects that the existing algorithm based on the message has long processing flow and consumes a large amount of calculation and bandwidth resources.
The invention also provides a distributed transaction processing method based on the broadcast channel, which has low implementation complexity.
In order to solve the technical problems, the technical scheme of the invention is as follows:
The distributed transaction processing system based on the broadcast channel comprises a main server end, a central data node, a plurality of data nodes and a plurality of terminals; wherein:
the main server end is in communication connection with the central data node network and is also in communication connection with the plurality of data node networks;
the terminal is in communication connection with the data node, and one data node is in communication connection with one or more terminals;
and the central data node performs data transmission through a broadcast channel, and establishes broadcast communication connection with the main server side and the data node respectively.
In the scheme, a user sends a data update request to a data node through a terminal, the data node transmits the data update request to a main server through a network, and the main server transmits a notification center data node through the network to modify the state of a data item corresponding to the data update request, and transmits the modified state to the main server and the data node through a broadcast signal; the data node locks the state of the corresponding data item and feeds back the state update information to the main server side; the main server side processes the data updating request, transmits the processing result to the central data node, and the central data node transmits the processing result to the data node through a broadcast signal, and the data node updates the data; and the user acquires a data updating result from the data node through the terminal.
The main server side comprises a main server, a main database and a main server side broadcast signal receiver; wherein:
the main database and the main server side broadcast signal receiver are all in communication connection with the main server;
the main server establishes network communication connection with the central data node and a plurality of data nodes;
the main server side broadcast signal receiver is used for receiving the broadcast signal sent by the central data node.
In the above scheme, the main server is configured to implement network connection between the main server and each data node, and between the main server and the central data node, and process related data update requests; the main database is a system persistence database and is a standard of system data copy.
The central data node comprises a central server, a central database and a broadcast signal transmitter; wherein:
the central database and the broadcast signal transmitter are all in communication connection with the central server;
the central server is in network communication connection with the main server side;
the broadcast signal transmitter transmits broadcast signals for the main server side and a plurality of data nodes;
the central database is used for storing data on the central data node.
The data node comprises a node server, a node database and a broadcast signal receiver; wherein:
the node database and the broadcast signal receiver are all in communication connection with the node server;
the node server is in network communication connection with the main server end;
the broadcast signal receiver is used for receiving the broadcast signal sent by the central data node;
the node database is used for storing data on the data nodes.
A distributed transaction processing method based on a broadcast channel, comprising the steps of:
s1: generating a partial sequence set P of nodes according to the system i ，i∈[0,n]Wherein P is 0 As a main server, the other data nodes as a system; setting a central data node C of a system, wherein the central data node C has a broadcast transmitting function;
s2: setting the transmission time of a system broadcast channel as T, initializing the system, and obtaining the maximum transmission of the system through statisticsTime T max:i-0 And a system minimum transmission time T min:i-0 Wherein:
s3: data node P i Sending out data updating request, and sending the request to the main server P through network transmission 0 ；
S4: main server side P 0 After receiving the update request, notifying the central data node C; the central data node C modifies the state of the corresponding data item and sends the modified state information out through a broadcast channel;
S5: main server side P 0 Data node P i Receiving the message of the central data node C, and a main server P 0 Judging an effective data updating request and modifying the corresponding data item state;
s6: main server side P 0 Processing the effective data update request according to the receiving sequence, and storing the processing result for the data item in a lasting way through two steps:
main server side P 0 Updating the processing result in the main database and notifying the central data node C, and transmitting the updating information of the corresponding data item by the central data node C through a broadcast channel;
main server side P 0 The update information is received to carry out update confirmation of the main database, and the persistent storage of the data item is completed;
s7: data node P i And updating the corresponding data item information by receiving the update information, and submitting a new data update request.
In the scheme, the data node P i The bidirectional network transmission delay to the central data node C is used as the basis for generating the partial sequence set by the system; the broadcast channel transmission time T is defined byThe value of the broadcast channel selected is determined by: t=electromagnetic signal propagation time over the air+modem time.
In the above scheme, all data nodes P in the system are realized through the broadcast channel i And the time precision is sufficient for the precedence distinction of distributed objects. If there are identical time stamps of two update requests, then pass through the data node P again i Is considered to define the update request precedence order of the same timestamp.
The step S3 specifically includes: when the data node P i When a data item with the upper status of "readable" issues a data update request, the data node P i Transmitting the update request of the data item to the main server P through network transmission 0 The request information carries a time stamp and the version number V of the current data item i 。
In the above scheme, the data node P i Network transmission of data update request to primary server side P through node server 0 On the primary server.
The step S4 specifically includes the following steps:
s41: main server side P 0 Receiving data node P i The update request of (2) notifying the request information to the central data node C through network transmission;
s42: the central data node C modifies the corresponding data item state from readable to locking, and sends the modified state information out through a broadcast channel, and the current time stamp is t 0 。
In the above scheme, the main server side P 0 The data updating request network is transmitted to a central server of the central data node C through a main server, the central server modifies the corresponding data item state from readable to locking, the modified state information is sent out through a broadcast signal transmitter, and the central database is synchronously updated.
The step S5 specifically includes the following steps:
s51: through t 0 After +T time, main suitServer side P 0 Data node P i The method comprises the steps that state update information of a central data node C is received, and the states of corresponding data items on a main database and a node database are modified into 'locking' by the original 'readable';
s52: in the process of the present treatment, t 0 After time +T, the update request sent by the data item with the state of locking is regarded as invalid; in version number V i In the update request of (1), at the latest, pass through T again max:i-0 Reach the main server P 0 Exceeding the time reaches the main server side P 0 Is also regarded as invalid, i.e. version number V i Time tag t 0 +T+T max:i-0 The post data update request and the version number are V i But at t 0 +T+T max:i-0 Before reaching the main server P 0 Are all requested by the main server P 0 A data update request deemed invalid;
s53: after rejecting invalid data update request, the main server P 0 An effective update request is obtained.
In the above scheme, the data node P i The broadcast signal receiver receives the state updating information of the central data node C, modifies the original readable data item into locking data item through the state of the corresponding data item on the node database by the node server, and synchronously updates the node database; main server side P 0 The main server side broadcast signal receiver receives the state updating information of the central data node C, and the main server modifies the original readable data item state of the corresponding data item on the main database into locking state and synchronously updates the main database; and meanwhile, judging the validity of the update request at the main server.
The step S6 specifically includes the following steps:
s61: main server side P 0 Processing the effective data update request according to the receiving sequence, implementing parallel processing of different data items, and returning the processing result to the corresponding data node P through network transmission i ；
S62: main server side P 0 Updating the main database and setting a new data item version number V i+1 ；
S63: notifying the update information to the central data node C, and transmitting the update information of the corresponding data item by the central data node C through a broadcast channel;
s64: set up the main server end P 0 From t 0 +T+T max:i-0 The time spent from starting to process the update request to the time spent by the central data node C receiving the update information and transmitting the update information to the broadcast channel is g, namely the main server side P 0 The moment of receiving the update information is t 0 +2T+T max:i-0 And +g, at the moment, updating and confirming the main database, and finishing the persistent storage of the data item.
In the above scheme, for an effective data update request, the primary server side P 0 Processing the data update requests on the main server according to the receiving sequence, and returning the processing results to the corresponding data nodes P through network transmission i A node server on the network; and for invalid data updating requests, directly returning corresponding error codes to the nodes.
The step S7 specifically includes: data node P i At t 0 +2T+T max:i-0 The update information is received at +g moment, the state of the corresponding data item is changed from locking to readable, and the version number of the data item is updated to V i+1 The method comprises the steps of carrying out a first treatment on the surface of the From t 0 +2T+T max:i-0 The +g-initiated data update request is valid, corresponding to a data item update with version number V i+1 。
In the above scheme, the data node P i Receiving the processing result through the broadcast signal receiver, and updating the processing result to the entry in the information on the original node database by the node server, wherein the entry on the node database is not required to be processed; for data node P with failed update i The state of the data item is checked again before a new update request is submitted.
In the above scheme, the error processing method further comprises the following steps: main server P in system 0 The central data node C and the broadcast channel are considered to be robust, can be clustered or multi-clustered The machine backup mode does not generate single-point faults; data node P i Also a computer cluster, the failure rate is small, so that the main server side P is not needed to be considered 0 Results are sent but data node P i No received condition; if data node P after sending update request i If no data update is received within a certain time, the data node P is marked i The fault, then the user is scheduled, and the user jumps to the data node P j De-access system, j not equal to i, j E [0, n]The system is still available to the user.
In the scheme, the operation application of the system to various databases is to process the states of the data items, namely a readable state and a locking state; when the broadcasting data state changes, adopting an interrupt mode, and processing by using a corresponding interrupt service routine; main server side P 0 The traditional mode is used for interaction with the database, so that the consistency of data is ensured.
In the scheme, the minimum data update interval of the system is set, namely the response time is T rsp The following steps are:
the response time of the system is independent of the distributed scale, and the usability of the system is guaranteed.
In the scheme, the broadcasting data type is introduced, so that time waiting is not required to be consumed for synchronization of the distributed variable operation, and the broadcasting data type adopts an asynchronous mode, so that the local application efficiency is not influenced, and the design of a distributed system is greatly simplified; distributed data node P i In the processing process, the node database does not need to be subjected to persistence operation, and all persistence operations are performed by the main server side P 0 The load of the system is reduced, and the operation efficiency of the system is improved; at the main server side P 0 In the method, the data operation of the value update type can be performed at T rsp Merging operations on the same data in time, only one persistent write is needed, subtractionFewer operations are performed on database IO, and the part can be arranged at the main server side P 0 An intermediate subsystem is divided for processing.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that:
the distributed transaction processing system based on the broadcast channel provided by the invention uses the traditional database technology to ensure the data consistency of the system, the response time of the system and the scale growth relation of the system are not great, and the communication delay among nodes is only required to be ensured, so that the usability of the system is excellent; the invention also provides a distributed transaction processing method based on the broadcast channel, which does not use a consistent algorithm based on the message, reduces information output and transmission among nodes, and effectively improves the processing efficiency of the message.
Drawings
FIG. 1 is a schematic diagram of the structural connection of the system of the present invention;
FIG. 2 is a schematic flow chart of the method of the present invention;
wherein: 1. a main server side; 11. a main server; 12. a master database; 13. a main server side broadcast signal receiver; 2. a central data node; 21. a central server; 22. a central database; 23. a broadcast signal transmitter; 3. a data node; 31. a node server; 32. a node database; 33. a broadcast signal receiver; 4. and (5) a terminal.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the present patent;
for the purpose of better illustrating the embodiments, certain elements of the drawings may be omitted, enlarged or reduced and do not represent the actual product dimensions;
it will be appreciated by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The technical scheme of the invention is further described below with reference to the accompanying drawings and examples.
Example 1
As shown in fig. 1, the distributed transaction processing system based on a broadcast channel includes a main server side 1, a central data node 2, a plurality of data nodes 3 and a plurality of terminals 4; wherein:
the main server 1 is in network communication connection with the central data node 2 and is also in network communication connection with the plurality of data nodes 3;
The terminals 4 are in communication connection with the data nodes 3, and one data node 3 is in communication connection with one or more terminals 4;
and the central data node 2 performs data transmission through a broadcast channel, and establishes broadcast communication connection with the main server side 1 and the data node 3 respectively.
In the specific implementation process, a user sends a data update request to a data node 3 through a terminal 4, the data node 3 transmits the data update request to a main server side 1 through a network, the main server side 1 notifies a central data node 2 to modify the state of a data item corresponding to the data update request through network transmission, and the modified state is transmitted to the main server side 1 and the data node 3 through a broadcast signal; the data node 3 locks the state of the corresponding data item and feeds back the state update information to the main server side 1; the main server 1 processes the data updating request, transmits the processing result to the central data node 2, the central data node 2 transmits the processing result to the data node 3 through a broadcast signal, and the data node 3 updates the data; the user obtains the data update result on the data node 3 via the terminal 4.
More specifically, the main server 1 includes a main server 11, a main database 12, and a main server broadcast signal receiver 13; wherein:
The main database 12 and the main server side broadcast signal receiver 13 are both in communication connection with the main server 11;
the main server 11 establishes network communication connection with the central data node 2 and the plurality of data nodes 3;
the main server-side broadcast signal receiver 13 is configured to receive a broadcast signal sent by the central data node 2.
In a specific implementation process, the main server 11 is configured to implement network connection between the main server 1 and each data node 3, and between the main server and the central data node 2, and process related data update requests; the primary database 12 is a system persistent database, which is the standard for copies of system data.
More specifically, the central data node 2 includes a central server 21, a central database 22, and a broadcast signal transmitter 23; wherein:
the central database 22 and the broadcast signal transmitter 23 are both in communication connection with the central server 21;
the central server 21 is in network communication connection with the main server side 1;
the broadcast signal transmitter 23 transmits broadcast signals to the main server 1 and the plurality of data nodes 3;
the central database 22 is used to store data on the central data node 2.
More specifically, the data node 3 includes a node server 31, a node database 32, and a broadcast signal receiver 33; wherein:
the node database 32 and the broadcast signal receiver 33 are both in communication connection with the node server 31;
the node server 31 is in network communication connection with the main server 1;
the broadcast signal receiver 33 is configured to receive a broadcast signal sent by the central data node 2;
the node database 32 is used to store data on the data nodes 3.
Example 2
As shown in fig. 2, based on the system described in embodiment 1, there is provided a distributed transaction processing method based on a broadcast channel, including the steps of:
s1: generating a partial sequence set P of nodes according to the system i ，i∈[0,n]Wherein P is 0 As a main server, the other data nodes as a system; setting a central data node C of a system, wherein the central data node C has a broadcast transmitting function;
s2: setting the transmission time of a system broadcast channel as T, initializing the system, and counting to obtain a systemMaximum transmission time T max:i-0 And a system minimum transmission time T min:i-0 Wherein:
s3: data node P i Sending out data updating request, and sending the request to the main server P through network transmission 0 ；
S4: main server side P 0 After receiving the update request, notifying the central data node C; the central data node C modifies the state of the corresponding data item and sends the modified state information out through a broadcast channel;
s5: main server side P 0 Data node P i Receiving the message of the central data node C, and a main server P 0 Judging an effective data updating request and modifying the corresponding data item state;
s6: main server side P 0 Processing the effective data update request according to the receiving sequence, and storing the processing result for the data item in a lasting way through two steps:
main server side P 0 Updating the processing result in the main database and notifying the central data node C, and transmitting the updating information of the corresponding data item by the central data node C through a broadcast channel;
main server side P 0 The update information is received to carry out update confirmation of the main database, and the persistent storage of the data item is completed;
s7: data node P i And updating the corresponding data item information by receiving the update information, and submitting a new data update request.
In the implementation process, the data node P i The bidirectional network transmission delay to the central data node C is used as the basis for generating the partial sequence set by the system; the broadcast message The channel transmission time T is determined by the broadcast channel selected by the system and has the value: t=electromagnetic signal propagation time over the air+modem time.
In the implementation process, all data nodes P in the system are realized through a broadcast channel i And the time precision is sufficient for the precedence distinction of distributed objects. If there are identical time stamps of two update requests, then pass through the data node P again i Is considered to define the update request precedence order of the same timestamp.
More specifically, the step S3 specifically includes: when the data node P i When a data item with the upper status of "readable" issues a data update request, the data node P i Transmitting the update request of the data item to the main server P through network transmission 0 The request information carries a time stamp and the version number V of the current data item i 。
In the implementation process, the data node P i Network transmission of data update request to primary server side P through node server 31 0 On the primary server 11.
More specifically, the step S4 specifically includes the following steps:
s41: main server side P 0 Receiving data node P i The update request of (2) notifying the request information to the central data node C through network transmission;
S42: the central data node C modifies the corresponding data item state from readable to locking, and sends the modified state information out through a broadcast channel, and the current time stamp is t 0 。
In the implementation process, the main server side P 0 The data update request is transmitted to the central server 21 of the central data node C through the main server 11, the status of the corresponding data item is modified from "readable" to "locked" by the central server 21, and the modified status information is transmitted through the broadcast signal transmitter 23, and the central database 22 performs synchronous update.
More specifically, the step S5 specifically includes the steps of:
s51: through t 0 After +T time, the main server P 0 Data node P i The method comprises the steps that state update information of a central data node C is received, and the states of corresponding data items on a main database and a node database are modified into 'locking' by the original 'readable';
s52: in the process of the present treatment, t 0 After time +T, the update request sent by the data item with the state of locking is regarded as invalid; in version number V i In the update request of (1), at the latest, pass through T again max:i-0 Reach the main server P 0 Exceeding the time reaches the main server side P 0 Is also regarded as invalid, i.e. version number V i Time tag t 0 +T+T max:i-0 The post data update request and the version number are V i But at t 0 +T+T max:i-0 Before reaching the main server P 0 Are all requested by the main server P 0 A data update request deemed invalid;
s53: after rejecting invalid data update request, the main server P 0 An effective update request is obtained.
In the implementation process, the data node P i The broadcast signal receiver 33 receives the status update information of the central data node C, and modifies the original "readable" into "locked" by the node server 31 for the status of the corresponding data item on the node database 32, and synchronously updates the node database 32; main server side P 0 The main server side broadcast signal receiver 13 receives the state update information of the central data node C, and modifies the original readable data into locking data through the main server 11 on the state of the corresponding data item on the main database 12, and synchronously updates the main database 12; at the same time, the master server 11 determines the validity of the update request.
11. More specifically, the step S6 specifically includes the following steps:
s61: main server side P 0 Processing the effective data update request according to the receiving sequence, implementing parallel processing of different data items, and passing the processing result through the network The network transmission returns the corresponding data node P i ；
S62: main server side P 0 Updating the main database and setting a new data item version number V i+1 ；
S63: notifying the update information to the central data node C, and transmitting the update information of the corresponding data item by the central data node C through a broadcast channel;
s64: set up the main server end P 0 From t 0 +T+T max:i-0 The time spent from starting to process the update request to the time spent by the central data node C receiving the update information and transmitting the update information to the broadcast channel is g, namely the main server side P 0 The moment of receiving the update information is t 0 +2T+T max:i-0 And +g, at the moment, updating and confirming the main database, and finishing the persistent storage of the data item.
In the implementation process, for effective data update request, the main server side P 0 Processing the data update requests in the receiving order on the main server 11 and returning the processing results to the corresponding data nodes P through network transmission i A node server 31 on the upper part; and for invalid data updating requests, directly returning corresponding error codes to the nodes.
More specifically, the step S7 specifically includes:
data node P i At t 0 +2T+T max:i-0 The update information is received at +g moment, the state of the corresponding data item is changed from locking to readable, and the version number of the data item is updated to V i+1 The method comprises the steps of carrying out a first treatment on the surface of the From t 0 +2T+T max:i-0 The +g-initiated data update request is valid, corresponding to a data item update with version number V i+1 。
In the implementation process, the data node P i Receiving the processing result by the broadcast signal receiver 33, and updating the processing result by the node server 31 to the original entry in the information on the node database 32, wherein the entry not on the node database 32 is not processed; for data node P with failed update i The state of the data item is checked again before a new update request is submitted.
In the implementation process, the method further comprises an error processing process: main server P in system 0 The central data node C and the broadcast channel are considered to be robust, a cluster mode or a multi-machine backup mode can be adopted, and single-point faults cannot be generated; data node P i Also a computer cluster, the failure rate is small, so that the main server side P is not needed to be considered 0 Results are sent but data node P i No received condition; if data node P after sending update request i If no data update is received within a certain time, the data node P is marked i The fault, then the user is scheduled, and the user jumps to the data node P j De-access system, j not equal to i, j E [0, n]The system is still available to the user.
In the specific implementation process, the operation application of various databases in the system is to process the states of the data items, namely a readable state and a locking state; when the broadcasting data state changes, adopting an interrupt mode, and processing by using a corresponding interrupt service routine; main server side P 0 The traditional mode is used for interaction with the database, so that the consistency of data is ensured.
In the implementation process, the minimum data update interval of the system, namely the response time is set as T rsp The following steps are:
the response time of the system is independent of the distributed scale, and the usability of the system is guaranteed.
In particular implementations, in the operational interval of each commit, the primary server side P is responsible for including the commit of the primary database 12 0 The processing speed of the system is slowed down by the increase of the data volume, namely g is increased, thereby increasing the response time T of the system rsp . There are many more mature techniques available to improve the efficiency of this part of the execution [1 ]][2]。
In the specific implementation process, the method comprises the following steps ofThe broadcast data type is input without consuming time waiting for the synchronization of the distributed variable operation, and the broadcast data type adopts an asynchronous mode, so that the local application efficiency is not influenced, and the design of a distributed system is greatly simplified; distributed data node P i During the processing, no persistence operation is required to be performed on the node database 32, and all persistence operations are performed by the main server side P 0 The load of the system is reduced, and the operation efficiency of the system is improved; at the main server side P 0 In the method, the data operation of the value update type can be performed at T rsp Merging the same data operation in time, only needing one time of persistent writing, reducing the operation on database IO, and the part can be also at the main server side P 0 An intermediate subsystem is divided for processing.
Example 3
More specifically, for data items to be updated by the distributed system, the method introduces a broadcast variable type whose state is: { Board cast-read, board cast-lock }, represent "readable" and "locked" states, the update version number is Board cast-version, the update version number of the data item will change with the number of times the data update is submitted; the broadcast type attributes are stored in the main server P 0 On other data nodes P i The updated status of the corresponding data item is obtained via the broadcast channel.
In the implementation process, the data node P i The process flow in the method is specifically as follows:
assuming that Data is globally consistent broadcast type Data of interest to the system, value change is the local application's desire for Data changes, readchange is the local application's resulting Data changes. Since the board cast-update is mainly updated through a broadcast channel, the update of the state and value of Data is described by the following algorithm.
Algorithm 1: data node P i Pseudo code segment of the process flow of broadcasting channel update data:
algorithm 2: data node P i Receiving the main server P 0 The processing flow of the returned asynchronous message processing result is as follows:
data node P i Is based on value change and readchange, not on value of broadcast type Data.
In the specific implementation process, by the two-section pseudo code example, the broadcasting data type is introduced without consuming time waiting for the synchronization of the distributed variable operation, and the broadcasting data type adopts an asynchronous mode, so that the local application efficiency is not influenced, and the design of a distributed system is greatly simplified; distributed data node P i During the processing, no persistence operation is required to be performed on the node database 32, and all persistence operations are performed by the main server side P 0 The load of the system is reduced, and the operation efficiency of the system is improved; at the main server side P 0 In the method, the data operation of the value update type can be performed at T rsp Merging the same data operation in time, only needing one time of persistent writing, reducing the operation on database IO, and the part can be also at the main server side P 0 An intermediate subsystem is divided for processing.
Example 4
More specifically, the t=electromagnetic signal propagation time over the air+modem time; if the system selects a geostationary satellite, T is approximately 0.27 seconds; if a low orbit satellite is selected, T is very small, roughly 1/36-1/6 of that of a geostationary satellite; if a high-altitude balloon or a high-altitude unmanned aerial vehicle is used as the signal medium, the value is smaller.
In the course of the specific implementation process, the method comprises,and->Depending on the network environment between the nodes of the distributed system, the system may dynamically detect these values during operation.
Example 4
The system and the method are applied to the national railway ticket selling system to realize the update transaction of the train residual ticket data in the national range. All railway ticketing systems in the whole country serve people in the whole country through websites, people in the whole country can visit websites to subscribe tickets, and the rule of ticket booking is first come first.
In a specific implementation process, a plurality of mirror servers are built nationwide according to the construction mode of the system. Assuming that each cell in the country has an edge computing center, mirror image sites can be deployed in the edge computing center of each cell to serve residents in the cell; each mirror site initially has a consistent database in which ticket sales data for each train across the country is maintained.
In the specific implementation process, after the system is initialized by the method of the invention, a certain service node is selected as P 0 Meanwhile, a server C in the traffic department can transmit data to a certain data frequency point on the Zhongxing No. 9. All nodes in the system are provided with broadcast signal receivers for acquiring data in the data frequency point on the Zhongxing No. 9; after system initialization, response time T of system is confirmed rsp 0.8 seconds; through the broadcasting channel, the ticketing condition data of each train nationally of each node in the system can be directly updated.
In the implementation process, a user a in a certain cell in the system has a user U a Logging onto national railway ticketing system website mirror image in the district, the website mirror image running at node P a ，U a When the web site browses the remaining tickets of the L train, 100 more tickets are left, at the time t a ，U a An L train ticket was ordered on the mirrored web site, at which point the web site displays "ticket order in progress. Since the mirror site is running at P a ，P a Has enough resource guarantee U a Is of the body of (2)The experience is that the user experiences the processing speed of the distributed system.
Meanwhile, b, c, d, e, in the system, z cells also have users U b ,U c ,U d ,U e ,...,U z At time t b ,t c ,t d ,t e ,...,t z They are at node P b ,P c ,P d ,P e ,...,P z Each person on the mirror image website subscribes 10 tickets of the L trains; t is t a To t z In (2), let t be z Maximum, and t z <(t a +0.8)，t a Minimum, all version numbers of the sent requests are 0; node P a At t a The remaining ticket request information of the updated L train is sent to P 0 The update value is-1. P (P) b To P z For P 0 The issued update request is update to-10. P (P) 0 At receiving node P a Is t a +0.1 seconds, corresponding to t in example 2 0 Thereafter, the server C is notified to send a data status update message for the remaining tickets of the L-train, assuming that all nodes receive the broadcast channel status update for a time t a +0.4 seconds, corresponding to t in example 2 0 +T, at T a -t a Time stamped t within +0.5 seconds a To t j ，t j Later submitted ticket booking requests are deemed invalid and therefore are indicated at U k To U (U) z After they submitted a ticket request to order 10L trains, "in ticket order execution.+ -." is started to be displayed, but less than 0.5 seconds, "the remaining tickets of the train that you select are updated, please check.+ -.".
In the specific implementation process, P 0 At t a In the time before +0.5, t a +0.5 corresponds to t in example 2 0 +T+T max:i-0 Receiving P a To P z An update request with version number 0 is issued, where only P b To P j Is valid, assuming they are all at t a P is reached before +0.5 0 。P 0 After processing these requests, the remaining tickets of the L train were 9. P (P) 0 Submitting the result to dataLibrary, data version is updated to 1 and the result is transmitted to the broadcasting channel of the star 9 through the server C; at the same time, P 0 Pair P through point-to-point network a To P z Is responded to by the request of (a).
In the specific implementation process, at t a Time +0.8, t a +0.5 corresponds to t in example 2 0 +2T+T max:i-0 +g, all nodes receive the broadcast channel update. U (U) k To U (U) z The user interface of the system can browse new residual ticket information, only 9 tickets are needed, and the number of tickets to be booked is readjusted to 1 ticket per person; suppose U k To U (U) z While the ticket booking request is submitted again, it is almost impossible in a practical system, but for the purpose of illustrating the handling of this extreme case by the system, it is assumed here that the simultaneous condition is established. Due to U k To U (U) z The submitted request requires 16 tickets, but the system has only 9 tickets, so only partial users' requests can be satisfied. P (P) 0 After receiving the update requests, the method is based on P k To P z Ordering in the system, defining an order of execution of the requests; assuming that the order of k-z is followed, U k To U (U) s Can obtain 1 ticket for everyone, U t To U (U) z If no ticket is ordered, the remaining tickets of the L train in the system are 0.
In the specific implementation process, the system and the method are applied to national railway ticketing systems, and the expandability of the system and the high efficiency and fairness in the processing of things are reflected.
In the specific implementation process, the system uses the traditional database technology, so that the data consistency of the system is ensured, the response time of the system and the scale growth relation of the system are not great, and the communication delay among nodes is ensured, so that the usability of the system is excellent; the invention also provides a distributed transaction processing method based on the broadcast channel, which does not use a consistent algorithm based on the message, reduces information output and transmission among nodes, and effectively improves the processing efficiency of the message.
It is to be understood that the above examples of the present invention are provided by way of illustration only and not by way of limitation of the embodiments of the present invention. Other variations or modifications of the above teachings will be apparent to those of ordinary skill in the art. It is not necessary here nor is it exhaustive of all embodiments. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the invention are desired to be protected by the following claims.
Reference is made to:
[1] xu optimization and implementation of layered coherence protocol under high load environment [ D ].2017.12
[2]CORBETT J C,DEAN J,EPSTEIN M,et al.Spanner:Google's globally-distributed database[C]/USENIX Conference on Operating Systems Design and Implementation,October 8-10,2012,Hollywood,USA.Berkeley:USENIX Association,2012:251-264.
Claims (7)
1. A method for distributed transaction processing based on broadcast channels, comprising the steps of:
s1: generating a partial sequence set P of nodes according to the system i ，i∈[0,n]Wherein P is 0 As a main server, the other data nodes as a system; setting a central data node C of a system, wherein the central data node C has a broadcast transmitting function;
s2: setting the transmission time of a system broadcast channel as T, initializing the system, and counting to obtain the maximum transmission time T of the system max:i-0 And a system minimum transmission time T min:i-0 Wherein:
s3: data node P i Sending out data updating request, and sending the request to the main server P through network transmission 0 ；
S4: main serviceEnd P 0 After receiving the update request, notifying the central data node C; the central data node C modifies the state of the corresponding data item and sends the modified state information out through a broadcast channel;
s5: main server side P 0 Data node P i Receiving the message of the central data node C, and a main server P 0 Judging an effective data updating request and modifying the corresponding data item state;
s6: main server side P 0 Processing the effective data update request according to the receiving sequence, and storing the processing result for the data item in a lasting way through two steps:
main server side P 0 Updating the processing result in the main database and notifying the central data node C, and transmitting the updating information of the corresponding data item by the central data node C through a broadcast channel;
main server side P 0 The update information is received to carry out update confirmation of the main database, and the persistent storage of the data item is completed;
s7: data node P i And updating the corresponding data item information by receiving the update information, and submitting a new data update request.
2. The method for distributed transaction processing based on broadcast channel according to claim 1, wherein the step S3 is specifically: when the data node P i When a data item with the upper status of "readable" issues a data update request, the data node P i Transmitting the update request of the data item to the main server P through network transmission 0 The request information carries a time stamp and the version number V of the current data item i 。
3. The method for distributed transaction processing based on broadcast channel according to claim 2, wherein said step S4 specifically comprises the steps of:
S41: main server side P 0 Receiving data node P i Is transmitted via a network to notify the central data node of the request informationC；
S42: the central data node C modifies the corresponding data item state from readable to locking, and sends the modified state information out through a broadcast channel, and the current time stamp is t 0 。
4. A distributed transaction processing method based on broadcast channels according to claim 3, characterized in that said step S5 comprises the steps of:
s51: through t 0 After +T time, the main server P 0 Data node P i The method comprises the steps that state update information of a central data node C is received, and the states of corresponding data items on a main database and a node database are modified into 'locking' by the original 'readable';
s52: in the process of the present treatment, t 0 After time +T, the update request sent by the data item with the state of locking is regarded as invalid; in version number V i In the update request of (1), at the latest, pass through T again max:i-0 Reach the main server P 0 Exceeding the time reaches the main server side P 0 Is also regarded as invalid, i.e. version number V i Time tag t 0 +T+T max:i-0 The post data update request and the version number are V i But at t 0 +T+T max:i-0 Before reaching the main server P 0 Are all requested by the main server P 0 A data update request deemed invalid;
s53: after rejecting invalid data update request, the main server P 0 An effective update request is obtained.
5. The method for distributed transaction processing based on broadcast channel according to claim 4, wherein said step S6 specifically comprises the steps of:
s61: main server side P 0 Processing the effective data update request according to the receiving sequence, implementing parallel processing of different data items, and returning the processing result to the corresponding data node P through network transmission i ；
S62: main server side P 0 Updating the main database and setting a new data item version number V i+1 ；
S63: notifying the update information to the central data node C, and transmitting the update information of the corresponding data item by the central data node C through a broadcast channel;
s64: set up the main server end P 0 From t 0 +T+T max:i-0 The time spent from starting to process the update request to the time spent by the central data node C receiving the update information and transmitting the update information to the broadcast channel is g, namely the main server side P 0 The moment of receiving the update information is t 0 +2T+T max:i-0 And +g, at the moment, updating and confirming the main database, and finishing the persistent storage of the data item.
6. The method for distributed transaction processing based on broadcast channel according to claim 5, wherein the step S7 is specifically:
Data node P i At t 0 +2T+T max:i-0 The update information is received at +g moment, the state of the corresponding data item is changed from locking to readable, and the version number of the data item is updated to V i+1 The method comprises the steps of carrying out a first treatment on the surface of the From t 0 +2T+T max:i-0 The +g-initiated data update request is valid, corresponding to a data item update with version number V i+1 。
7. A broadcast channel distributed transaction processing system based on the broadcast channel based distributed transaction processing method of any of claims 1-6, characterized by: the system comprises a main server (1), a central data node (2), a plurality of data nodes (3) and a plurality of terminals (4); wherein:
the main service one data node (3) is in communication connection with one or more terminals (4);
the central data node (2) performs data transmission through a broadcast channel, and establishes broadcast communication connection with the main server end (1) and the data node (3) at the same time;
the main server (1) comprises a main server (11), a main database (12) and a main server broadcast signal receiver (13); wherein:
the main database (12) and the main server side broadcast signal receiver (13) are all in communication connection with the main server (11);
the main server (11) establishes network communication connection with the central data node (2) and the plurality of data nodes (3);
The main server side broadcast signal receiver (13) is used for receiving the broadcast signal sent by the central data node (2);
the central data node (2) comprises a central server (21), a central database (22) and a broadcast signal transmitter (23); wherein:
the central database (22) and the broadcast signal transmitter (23) are both in communication connection with the central server (21);
the central server (21) is in network communication connection with the main server (1);
the broadcast signal transmitter (23) transmits broadcast signals to the main server (1) and the plurality of data nodes (3);
the central database (22) is used for storing data on the central data node (2);
the data node (3) comprises a node server (31), a node database (32) and a broadcast signal receiver (33); wherein:
the node database (32) and the broadcast signal receiver (33) are both in communication connection with the node server (31);
the node server (31) is in network communication connection with the main server (1);
the broadcast signal receiver (33) is configured to receive a broadcast signal sent by the central data node (2);
the node database (32) is used for storing data on the data nodes (3).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910419528.5A CN110175069B (en) | 2019-05-20 | 2019-05-20 | Distributed transaction processing system and method based on broadcast channel |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910419528.5A CN110175069B (en) | 2019-05-20 | 2019-05-20 | Distributed transaction processing system and method based on broadcast channel |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110175069A CN110175069A (en) | 2019-08-27 |
| CN110175069B true CN110175069B (en) | 2023-11-14 |
Family
ID=67691724
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910419528.5A Active CN110175069B (en) | 2019-05-20 | 2019-05-20 | Distributed transaction processing system and method based on broadcast channel |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110175069B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6073169A (en) * | 1997-04-08 | 2000-06-06 | Abb Power T&D Company Inc. | Automatic meter reading system employing common broadcast command channel |
| CN102201010A (en) * | 2011-06-23 | 2011-09-28 | 清华大学 | Distributed database system without sharing structure and realizing method thereof |
| CN104618221A (en) * | 2015-01-06 | 2015-05-13 | 厦门雅迅网络股份有限公司 | Decentralized message service system |
| US9471593B1 (en) * | 2010-12-30 | 2016-10-18 | Emc Corporation | Method and apparatus for ensuring consistent ordering of writes in a distributed database storage system |
| CN106844399A (en) * | 2015-12-07 | 2017-06-13 | 中兴通讯股份有限公司 | Distributed data base system and its adaptive approach |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8041832B2 (en) * | 2005-05-05 | 2011-10-18 | Verizon Services Corp. | Network data distribution system and method |
| GB2484086A (en) * | 2010-09-28 | 2012-04-04 | Metaswitch Networks Ltd | Reliability and performance modes in a distributed storage system |
-
2019
- 2019-05-20 CN CN201910419528.5A patent/CN110175069B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6073169A (en) * | 1997-04-08 | 2000-06-06 | Abb Power T&D Company Inc. | Automatic meter reading system employing common broadcast command channel |
| US9471593B1 (en) * | 2010-12-30 | 2016-10-18 | Emc Corporation | Method and apparatus for ensuring consistent ordering of writes in a distributed database storage system |
| CN102201010A (en) * | 2011-06-23 | 2011-09-28 | 清华大学 | Distributed database system without sharing structure and realizing method thereof |
| CN104618221A (en) * | 2015-01-06 | 2015-05-13 | 厦门雅迅网络股份有限公司 | Decentralized message service system |
| CN106844399A (en) * | 2015-12-07 | 2017-06-13 | 中兴通讯股份有限公司 | Distributed data base system and its adaptive approach |
| WO2017097059A1 (en) * | 2015-12-07 | 2017-06-15 | 中兴通讯股份有限公司 | Distributed database system and self-adaptation method therefor |
Non-Patent Citations (3)
| Title |
|---|
| A Novel Structure and Access Mechanism for Mobile Data Broadcast in Digital Ecosystems;Agustinus Borgy Waluyo等;《IEEE Transactions on Industrial Electronics》;全文 * |
| Implementation of distributed XA transactions in MyCat based on table broadcasting mechanism;Yingying Wu等;《2018 IEEE/ACM 5th International Conference on Big Data Computing Applications and Technologies(BDCAT)》);全文 * |
| 移动数据库中基于数据广播缓存一致性的研究;袁华华等;《电脑知识与技术》;20071223(第24期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110175069A (en) | 2019-08-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11693877B2 (en) | Cross-ontology multi-master replication | |
| US10764369B2 (en) | Data storage method and server applicable to distributed server cluster | |
| CN112069265B (en) | Synchronization method of configuration data, business data system, computer system and medium | |
| US7693882B2 (en) | Replicating data across the nodes in a cluster environment | |
| EP2948875B1 (en) | Method and system for using a recursive event listener on a node in hierarchical data structure | |
| CN114363407B (en) | Message service method and device, readable storage medium and electronic equipment | |
| CN106777311B (en) | Flight space state caching method and system | |
| CN109783151B (en) | Method and device for rule change | |
| CN112148798A (en) | Data processing method and device applied to distributed system | |
| CN109862102B (en) | Distributed data multi-copy concurrency control system, server and method | |
| CN114050904B (en) | Consensus system and method based on two-level leader node fragmentation structure | |
| CN105959349A (en) | Distributed service end operation system and method | |
| CN108390919B (en) | Message synchronization system and method for high-reliability dual-computer hot standby | |
| CN110175069B (en) | Distributed transaction processing system and method based on broadcast channel | |
| CN109560903A (en) | A kind of vehicle-mounted command communications system of complete disaster tolerance | |
| CN107465725B (en) | Heterogeneous long transaction processing system and method based on client information control system | |
| US20090083379A1 (en) | Enabling connections for use with a network | |
| EP3427157A1 (en) | Cross-regional data transmission | |
| CN114595051B (en) | Method and system for enhancing cloud computing cooperative data processing and electronic equipment | |
| CN101329670B (en) | Method and system for keep consistency of data under copy database environment | |
| US9253124B2 (en) | Techniques for sending and relaying information over broadcast and non-broadcast communications media | |
| CN115967611A (en) | Cross-domain switching processing method, device, equipment and storage medium | |
| CN111600958A (en) | Service discovery system, service data management method, server, and storage medium | |
| CN112118118B (en) | Method and device for broadcasting instruction notification message based on inverse entropy algorithm | |
| WO2024021746A1 (en) | Data processing method, communication system, and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |