CN103164219B - The distributing real time system system of polymorphic type copy is used in decentralization framework - Google Patents
The distributing real time system system of polymorphic type copy is used in decentralization framework Download PDFInfo
- Publication number
- CN103164219B CN103164219B CN201310005857.8A CN201310005857A CN103164219B CN 103164219 B CN103164219 B CN 103164219B CN 201310005857 A CN201310005857 A CN 201310005857A CN 103164219 B CN103164219 B CN 103164219B
- Authority
- CN
- China
- Prior art keywords
- submodule
- copy
- write
- transaction
- subtransaction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 230000004044 response Effects 0.000 claims description 29
- 238000000034 method Methods 0.000 claims description 22
- 230000008569 process Effects 0.000 claims description 20
- 230000006870 function Effects 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 5
- 238000007689 inspection Methods 0.000 claims description 3
- 230000008439 repair process Effects 0.000 claims description 3
- 238000007726 management method Methods 0.000 description 6
- 230000007246 mechanism Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses the distributing real time system system using polymorphic type copy in a kind of decentralization framework, comprise transaction interface module, transaction model and affairs memory module, transaction interface module comprises external interface submodule and transaction preprocess submodule, and transaction model comprises polymorphic type copy submodule, read transaction model, replica group transaction status submodule, read request distribution submodule, submodule repaired by copy, MVCC reading submodule, issued transaction submodule is write in local, affairs Paxos copy consistency submodule is write in local, local is write affairs and is submitted submodule to, global write issued transaction submodule, main subtransaction Paxos copy consistency submodule, secondary subtransaction Paxos copy consistency submodule and global write affairs submit submodule to, the present invention can solve exist in existing system for read-write environment limit to very much, read-write availability cannot be configured voluntarily according to the demand of application, and global transaction depends on and locks, the problem that cost is higher.
Description
Technical field
The invention belongs to technical field of distributed memory, more specifically, relate to the distributing real time system system using polymorphic type copy in a kind of decentralization framework.
Background technology
Along with the development of Internet technology, the data in internet rise with mysterious speed, and how Storage and Processing data on a large scale like this become the research direction of large data age now.Decentralization NoSQL is a kind of mass data storage system, has following feature: high readwrite performance, without Single Point of Faliure, and high availability, enhanced scalability.Such as Cassandra system, it uses towards arranging the memory model of race to obtain high readwrite performance, uses decentralization framework to avoid Single Point of Faliure and to obtain high availability, uses consistance Hash to obtain enhanced scalability.
Distributing real time system system in existing decentralization framework have following these: Megastore system is the system that a Google completes on the basis of Bigtable.This system employs special data model EntityGroups, then uses extra system module Coordinator to also have replica server to ensure consistance.Its submission algorithm is the variant of Paxos algorithm, for safeguarding the consistance of synchronization counterpart between multiple data center.But the method employs fixing read-write successfully number, cannot carry out the adjustment of availability for different application, its global transaction uses two sections of expensive submissions, can cause obstruction.Scalaris is the distributed transaction system on a chord# ring, uses symmetrical replication policy, uses the Paxos atomic commitment agreement improved, needs three phases just can complete affairs, cannot carry out the adjustment of availability for different application.The prototype system also having an academic research to write, have studied the distributed transaction under P-Ring environment, proposes a kind of MVCC algorithm---LSTP.Article is paid attention to read busy environment, and read-only affairs can not stop and block, but is not suitable for writing busy environment, and this systematic difference scene is comparatively limited to.
In sum, existing distributing real time system system has the following disadvantages: 1, for read-write environment limit to very much, cannot according to application demand configure read-write availability voluntarily.2, global transaction depends on and locks, and cost is higher.
Summary of the invention
For the defect of prior art, the object of the present invention is to provide the distributing real time system system using polymorphic type copy in a kind of decentralization framework, be intended to solve in existing system exist for read-write environment limit to very much, read-write availability cannot be configured voluntarily according to the demand of application, and global transaction depends on and locks, the problem that cost is higher.
For achieving the above object, the invention provides the distributing real time system system using polymorphic type copy in a kind of decentralization framework, comprise transaction interface module, transaction model and affairs memory module, transaction interface module comprises external interface submodule and transaction preprocess submodule, transaction model comprises polymorphic type copy submodule, read transaction model, replica group transaction status submodule, read request distribution submodule, submodule repaired by copy, MVCC reading submodule, issued transaction submodule is write in local, affairs Paxos copy consistency submodule is write in local, local is write affairs and is submitted submodule to, global write issued transaction submodule, main subtransaction Paxos copy consistency submodule, secondary subtransaction Paxos copy consistency submodule and global write affairs submit submodule to, external interface submodule is for receiving the transactions requests from client, and this transactions requests is sent to transaction preprocess submodule, transaction preprocess submodule is for judging that transactions requests is read transactions requests or writes transactions requests, if read transactions requests, then will read transactions requests to be sent to and to read issued transaction submodule, if write transactions requests, then judging that this writes transactions requests is further that transactions requests or global write transactions requests are write in local, if transactions requests is write in local, then this local is write transactions requests be sent to local write issued transaction submodule, if global write transactions requests, then this global write transactions requests is sent to global write issued transaction submodule, read issued transaction submodule and read the address of hard copy corresponding to instruction and the response quantity of needs for each obtaining from polymorphic type copy submodule that this reads to comprise in transactions requests, and will transactions requests be read, the address of hard copy and the response quantity of needs are sent to replica group transaction status submodule, and control the whole timeout retry reading transactions requests treatment scheme, replica group transaction status submodule is used for the affairs executing state read according to the address of hard copy in corresponding replica group, instruction corresponding each maximum submission state daily record numbering and maximum submission state log timestamp is read to obtain, read request distribution submodule judges to read reading instruction and whether can perform in this locality in transactions requests, if passable, then instruction will be read, maximum submission state daily record numbering and maximum submission state log timestamp are sent to local copy and repair submodule, otherwise will instruction be read, maximum submission state daily record numbering and maximum submission state log timestamp send to reads any one hard copy corresponding to instruction, the maximum submission state daily record numbering place that submodule is used for obtain in the node updates at its place to replica group transaction status submodule repaired by copy, MVCC reading submodule reads instruction for basis and maximum submission state log timestamp reads data from affairs memory module, and data are returned to read transaction model, read transaction model also for data are sent to external interface submodule, external interface submodule is also for being sent to client by data, issued transaction submodule writes witness copy corresponding to the write command that comprises in transactions requests and the address of hard copy and the hard copy of needs response quantity for obtaining this local from polymorphic type copy submodule is write in local, and transactions requests is write in local, the response quantity of the address of witness copy and hard copy and the hard copy of needs is sent to local and writes affairs Paxos copy consistency submodule, and control the timeout retry that transactions requests treatment scheme is write in whole local, affairs Paxos copy consistency submodule is write for arranging daily record value consistent on witness copy in witness replica group in local, this daily record value is sent to local and writes affairs submission submodule, and for journal entry add local transaction tag, local is write affairs and is submitted to submodule for daily record value being committed to corresponding hard copy according to the response quantity of the address of hard copy and the hard copy of needs, and successful for process result is returned to local write issued transaction submodule, issued transaction submodule is write also for successful for process result is sent to external interface submodule in local, external interface submodule is also for being sent to client by successful for process result, global write transaction management submodule is used for the response quantity obtaining witness copy corresponding to each write command of comprising this global write transactions requests and the address of hard copy and the hard copy of needs from polymorphic type copy submodule, and by global write transactions requests, the response quantity of the witness copy that each write command is corresponding and the address of hard copy and the hard copy of needs is sent to main subtransaction Paxos copy consistency submodule, and control the timeout retry of whole global write transactions requests treatment scheme, the daily record value that main subtransaction Paxos copy consistency submodule is consistent on witness copy in the witness replica group arranging main subtransaction, the positional information of this daily record value and main subtransaction is sent to secondary subtransaction submission submodule, and add that global transaction marks for this journal entry, the daily record value that secondary subtransaction Paxos copy consistency submodule is consistent on witness copy in the witness replica group all secondary subtransactions being arranged to this secondary subtransaction, journal entry for all secondary subtransactions adds positional information and the global transaction mark of main subtransaction, and the daily record value of the daily record value of main subtransaction and all secondary subtransactions is sent to main subtransaction submission submodule, global write affairs submit to submodule to be used for, according to the response number of the address of the hard copy of main subtransaction and the hard copy of needs, the daily record value of main subtransaction is committed to corresponding hard copy, and successful for process result is returned to global write transaction management submodule, by the daily record value of this secondary subtransaction, corresponding hard copy is committed to according to the response number of the address of the hard copy of this secondary subtransaction and the hard copy of needs to each secondary subtransaction processing successfully, global write transaction management submodule is also for being sent to external interface submodule by successful for process result, external interface submodule is also for being sent to client by successful for process result.
Transaction preprocess submodule judges its type by the OPER field read in transactions requests, this field is that transactions requests is read in expression, transactions requests is write for representing, consistance hash function is utilized to carry out computing to the key writing each write operation comprised in transactions requests, and judge according to operation result the type writing transactions requests, if same node is all pointed in the key computing of all write operations, then this writes transactions requests is that transactions requests is write in local, otherwise is global write transactions requests.
Copy is repaired submodule and all journal entries being less than this numbering is all obtained homogeneity value and judges whether to need to submit to, if the transaction types of this journal entry record is local affairs, then when journal entry reaches consistent in witness copy, namely think and can submit to, otherwise just submit blank operation to; If the transaction types of this journal entry record is global transaction, then except whether needs inspection reaches unanimously in witness copy, also to check whether the main subtransaction of storage is submitted to, when main subtransaction is submitted to, just think and can submit to, otherwise just submitting blank operation to, finally all affairs needing to submit to being performed to completing state.
It is use Paxos algorithm that affairs Paxos copy consistency submodule is write in local, attempt reaching the consistent of daily record value on the same iog location of each witness copy, this daily record value is that this time locally writing the operation of write command in transactions requests adds timestamp, and this timestamp is greater than the maximum submission state log timestamp that this Paxos algorithm performs this witness replica group front.
Global write affairs comprise two or more local and write affairs, all can be coupled with global transaction mark, one of them can be designated as main subtransaction, and used by as submission point, other local are write affairs and are designated as secondary subtransaction, the positional information of main subtransaction will be recorded, for copy reparation.
Main subtransaction Paxos copy consistency submodule uses Paxos algorithm, attempt reaching the consistent of daily record value on the same iog location of each witness copy of main subtransaction, this daily record value is that the operation of write command in this main subtransaction adds timestamp, and this timestamp is greater than the maximum submission state log timestamp that this Paxos algorithm performs this witness replica group front.
Secondary subtransaction Paxos copy consistency submodule uses Paxos algorithm, to each secondary subtransaction, attempt reaching the consistent of daily record value on the same iog location of each witness copy of secondary subtransaction, this daily record value for this reason in secondary subtransaction the operation of write command add timestamp, this timestamp be greater than this Paxos algorithm perform before the maximum submission state log timestamp of this witness replica group.
By the above technical scheme that the present invention conceives, compared with prior art, the present invention has following beneficial effect:
(1) the read-write node be separated
Owing to have employed polymorphic type copy submodule, affairs Paxos copy consistency submodule is write in local, local is write affairs and submitted submodule, main subtransaction Paxos copy consistency submodule, secondary subtransaction Paxos copy consistency submodule and global write affairs submission submodule to, therefore the process of daily record is separated with the processing node of data, improves configurability
(2) the read-write availability level of configurable distributed transaction
Owing to have employed polymorphic type copy submodule, the hard copy quantity needed when therefore can set read-write, like this ensureing conforming while, can also regulate the availability level of read-write.
(3) without the global transaction of lock
Submit submodule to owing to have employed main subtransaction Paxos copy consistency submodule, secondary subtransaction Paxos copy consistency submodule and global write affairs, main subtransaction therefore can be used as submission point, avoid use and lock.
(4) extensibility is strong
Owing to have employed consistance Hash in decentralization framework, because herein is provided very strong extensibility.When data scale increases, can be extending transversely conveniently by increase node, only needing, a token value is set and just can add whole server cluster voluntarily.Overall performance increases along with the increase near-linear of scale.
(5) high reliability
Copy mechanism in the system reliability of data, same number is according to can be kept on multiple node simultaneously, when node failure, data can not be lost.By configuration, copy mechanism can also be risen to data center's rank, the disaster tolerance of disaster rank is provided.
Accompanying drawing explanation
Fig. 1 is the distributing real time system system interconnect topological diagram based on polymorphic type copy in decentralization framework.
Fig. 2 is the distributing real time system system architecture figure based on polymorphic type copy in decentralization framework.
Embodiment
In order to make object of the present invention, technical scheme and advantage clearly understand, below in conjunction with drawings and Examples, the present invention is further elaborated.Should be appreciated that specific embodiment described herein only in order to explain the present invention, be not intended to limit the present invention.
As shown in Figure 1, use the distributing real time system system of polymorphic type copy to be applied in comprise client and multiple distributed system being arranged in node A, B and C of server end in decentralization framework of the present invention, be specifically arranged in node.Client is used for customer in response request, and carries out alternately with node A, B and C, to submit affairs to and to obtain result.Node A, B, C are for storing data, and affairs etc. are read and write in response, and use express network to be interconnected between node.Distributed system adopts distributed hashtable as bottom, the data of process key-value pair (Key-Value) form, key is mapped as a token (Token) by consistance hash function by each node, and determine the memory node of this key-value pair according to this token, and each node is responsible for the key-value pair storing certain limit.
As shown in Figure 2, the distributing real time system system of polymorphic type copy is used to comprise transaction interface module 1, transaction model 2 and affairs memory module 3 in decentralization framework of the present invention.
Transaction interface module 1 comprises external interface submodule 101 and transaction preprocess submodule 102.
Transaction model 2 comprises polymorphic type copy submodule 201, read transaction model 202, replica group transaction status submodule 203, read request distribution submodule 204, submodule 205 repaired by copy, MVCC reading submodule 206, issued transaction submodule 207 is write in local, affairs Paxos copy consistency submodule 208 is write in local, local is write affairs and is submitted submodule 209 to, global write issued transaction submodule 210, main subtransaction Paxos copy consistency submodule 211, secondary subtransaction Paxos copy consistency submodule 212 and global write affairs submit submodule 213 to.
This transactions requests for receiving the transactions requests from client, and is sent to transaction preprocess submodule 102 by external interface submodule 101.
Transaction preprocess submodule 102 is for judging that transactions requests is read transactions requests or writes transactions requests, if read transactions requests, then will read transactions requests to be sent to and to read issued transaction submodule 204, if write transactions requests, then judging that this writes transactions requests is further that transactions requests or global write transactions requests are write in local, if transactions requests is write in local, then this local is write transactions requests be sent to local write issued transaction submodule 207, if global write transactions requests, then this global write transactions requests is sent to global write issued transaction submodule 210.Specifically, be judge its type by the OPER field read in transactions requests, this field is that transactions requests is read in 0 expression, is that transactions requests is write in 1 expression; Consistance hash function is utilized to carry out computing to the key writing each write operation comprised in transactions requests, and judge according to operation result the type writing transactions requests, if same node is all pointed in the key computing of all write operations, then this writes transactions requests is that transactions requests is write in local, otherwise is global write transactions requests.
Read issued transaction submodule 202 and read the address of hard copy corresponding to instruction and the response amount R (R is positive integer) of needs for each obtaining from polymorphic type copy submodule 201 that this reads to comprise in transactions requests, and be sent to replica group transaction status submodule 203 by reading transactions requests, the address of hard copy and the response quantity of needs, and control the whole timeout retry reading transactions requests treatment scheme.
Replica group transaction status submodule 203, for reading the affairs executing state in corresponding replica group according to the address of hard copy, reads instruction corresponding each maximum submission state daily record numbering and maximum submission state log timestamp to obtain; Specifically, the consistance of each node log is safeguarded in a replica group, each of daily record has the numbering and timestamp that constantly increase progressively, and have various states: waiting state, submit state to and complete state, each node can record and known maximumly complete state daily record numbering, maximum submission state daily record numbering and maximum submission state log timestamp, the result of acquisition be at least R successfully respond in maximal value.
Read request distribution submodule 204 judges to read reading instruction and whether can perform in this locality in transactions requests, if passable, then be sent to local copy repair submodule 205 by reading instruction, maximum submission state daily record numbering and maximum submission state log timestamp, otherwise send to read any one hard copy corresponding to instruction by reading instruction, maximum submission state daily record numbering and maximum submission state log timestamp.
The maximum submission state daily record numbering place of submodule 205 for obtaining in the node updates at its place to replica group transaction status submodule 203 repaired by copy; Specifically, the all journal entries being less than this numbering are all obtained homogeneity value and judges whether to need to submit to, if the transaction types of this journal entry record is local affairs, then when journal entry reaches consistent in witness copy, namely think and can submit to, otherwise just submit blank operation to; If the transaction types of this journal entry record is global transaction, then except whether needs inspection reaches unanimously in witness copy, also to check whether the main subtransaction of storage is submitted to, when main subtransaction is submitted to, just think and can submit to, otherwise just submitting blank operation to, finally all affairs needing to submit to being performed to completing state.
Data for according to reading instruction and maximum submission state log timestamp reads data from affairs memory module 3, and return to and read transaction model 202 by MVCC reading submodule 206.
Read transaction model 202 also for data are sent to external interface submodule 101.
External interface submodule 101 is also for being sent to client by data.
Issued transaction submodule 207 writes witness copy corresponding to the write command that comprises in transactions requests and the address of hard copy and the hard copy of needs response quantity W (wherein W is positive integer) for obtaining this local from polymorphic type copy submodule 201 is write in local, and transactions requests is write in local, the response quantity of witness copy and the address of hard copy and the hard copy of needs is sent to local and writes affairs Paxos copy consistency submodule 208, and control the timeout retry that transactions requests treatment scheme is write in whole local.
Affairs Paxos copy consistency submodule 208 is write for arranging daily record value consistent on witness copy in witness replica group in local, this daily record value is sent to local and writes affairs and submit submodule 209 to, and adds locally transaction tag for journal entry; Specifically, use Paxos algorithm, attempt reaching the consistent of daily record value on the same iog location of each witness copy, this daily record value is that this time locally writing the operation of write command in transactions requests adds timestamp, and this timestamp is greater than the maximum submission state log timestamp that this Paxos algorithm performs this witness replica group front.
Local is write affairs and is submitted to submodule 209 for daily record value being committed to corresponding hard copy according to the response quantity W of the address of hard copy and the hard copy of needs, and successful for process result is returned to and locally write issued transaction submodule 207.
Issued transaction submodule 207 is write also for successful for process result is sent to external interface submodule 101 in local.
External interface submodule 101 is also for being sent to client by successful for process result.
Global write transaction management submodule 212 is for obtaining the response quantity V (wherein V is positive integer) of witness copy corresponding to each write command of comprising in this global write transactions requests and the address of hard copy and the hard copy of needs from polymorphic type copy submodule 201, and the response quantity of witness copy corresponding to global write transactions requests, each write command and the address of hard copy and the hard copy of needs is sent to main subtransaction Paxos copy consistency submodule 211, and control the timeout retry of whole global write transactions requests treatment scheme.Specifically, global write affairs comprise two or more local and write affairs, all can be coupled with global transaction mark, one of them can be designated as main subtransaction, and used by as submission point, other local are write affairs and are designated as secondary subtransaction, will record the positional information of main subtransaction, for copy reparation.
The daily record value that main subtransaction Paxos copy consistency submodule 211 is consistent on witness copy in the witness replica group arranging main subtransaction, the positional information of this daily record value and main subtransaction is sent to secondary subtransaction and submits submodule to, and add that global transaction marks for this journal entry; Specifically, use Paxos algorithm, attempt reaching the consistent of daily record value on the same iog location of each witness copy of main subtransaction, this daily record value is that the operation of write command in this main subtransaction adds timestamp, and this timestamp is greater than the maximum submission state log timestamp that this Paxos algorithm performs this witness replica group front.
The daily record value that secondary subtransaction Paxos copy consistency submodule 212 is consistent on witness copy in the witness replica group all secondary subtransactions being arranged to this secondary subtransaction, journal entry for all secondary subtransactions adds positional information and the global transaction mark of main subtransaction, and the daily record value of the daily record value of main subtransaction and all secondary subtransactions is sent to main subtransaction submission submodule; Specifically, use Paxos algorithm, to each secondary subtransaction, attempt reaching the consistent of daily record value on the same iog location of each witness copy of secondary subtransaction, this daily record value for this reason in secondary subtransaction the operation of write command add timestamp, this timestamp be greater than this Paxos algorithm perform before the maximum submission state log timestamp of this witness replica group.
Global write affairs submit to submodule 213, for the response number V of the address of the hard copy according to main subtransaction and the hard copy of needs, the daily record value of main subtransaction is committed to corresponding hard copy, and successful for process result is returned to global write transaction management submodule 212, by the daily record value of this secondary subtransaction, corresponding hard copy is committed to according to the response number V of the address of the hard copy of this secondary subtransaction and the hard copy of needs to each secondary subtransaction processing successfully.
Global write transaction management submodule 212 is also for being sent to external interface submodule 101 by successful for process result.
External interface submodule 101 is also for being sent to client by successful for process result.
Example:
In order to verify feasibility and the validity of the inventive method, configuration-system under true environment, test using the distributed transaction of polymorphic type copy in decentralization framework.
Server basic hardware of the present invention and software merit rating as shown in table 1:
Table 1
Distributing real time system in polymorphic type copy and decentralization framework has been carried out effective combination by the present invention.It uses decentralization framework, provides very strong extensibility, and its copy mechanism improves reliability, the restorability of data, and higher availability of data, provides the distributed transaction function of strong consistency.This system uses polymorphic type copy to be separated the read-write physical node of distributed transaction, effectively reduce node failure to reading affairs and the impact writing transactional availability, for the distributed transaction important in inhibiting for the suitable read-write availability of different application scene configuration, there is larger application potential.
Those skilled in the art will readily understand; the foregoing is only preferred embodiment of the present invention; not in order to limit the present invention, all any amendments done within the spirit and principles in the present invention, equivalent replacement and improvement etc., all should be included within protection scope of the present invention.
Claims (7)
1. in a decentralization framework, use the distributing real time system system of polymorphic type copy, comprise transaction interface module, transaction model and affairs memory module, transaction interface module comprises external interface submodule and transaction preprocess submodule, transaction model comprises polymorphic type copy submodule, read issued transaction submodule, replica group transaction status submodule, read request distribution submodule, submodule repaired by copy, MVCC reading submodule, issued transaction submodule is write in local, affairs Paxos copy consistency submodule is write in local, local is write affairs and is submitted submodule to, global write issued transaction submodule, main subtransaction Paxos copy consistency submodule, secondary subtransaction Paxos copy consistency submodule and global write affairs submit submodule to, it is characterized in that,
This transactions requests for receiving the transactions requests from client, and is sent to transaction preprocess submodule by external interface submodule;
Transaction preprocess submodule is for judging that transactions requests is read transactions requests or writes transactions requests, if read transactions requests, then will read transactions requests to be sent to and to read issued transaction submodule, if write transactions requests, then judging that this writes transactions requests is further that transactions requests or global write transactions requests are write in local, if transactions requests is write in local, then this local is write transactions requests be sent to local write issued transaction submodule, if global write transactions requests, then this global write transactions requests is sent to global write issued transaction submodule;
Read issued transaction submodule and read the address of hard copy corresponding to instruction and the response quantity of needs for each obtaining from polymorphic type copy submodule that this reads to comprise in transactions requests, and be sent to replica group transaction status submodule by reading transactions requests, the address of hard copy and the response quantity of needs, and control the whole timeout retry reading transactions requests treatment scheme;
Replica group transaction status submodule is used for the affairs executing state read according to the address of hard copy in corresponding replica group, reads instruction corresponding each maximum submission state daily record numbering and maximum submission state log timestamp to obtain;
Read request distribution submodule judges to read reading instruction and whether can perform in this locality in transactions requests, if passable, then be sent to local copy repair submodule by reading instruction, maximum submission state daily record numbering and maximum submission state log timestamp, otherwise send to read any one hard copy corresponding to instruction by reading instruction, maximum submission state daily record numbering and maximum submission state log timestamp;
The maximum submission state daily record numbering place that submodule is used for obtain in the node updates at its place to replica group transaction status submodule repaired by copy;
MVCC reading submodule is used for reading data according to reading instruction and maximum submission state log timestamp from affairs memory module, and data is returned to and read issued transaction submodule;
Read issued transaction submodule also for data are sent to external interface submodule;
External interface submodule is also for being sent to client by data;
Issued transaction submodule writes witness copy corresponding to the write command that comprises in transactions requests and the address of hard copy and the hard copy of needs response quantity for obtaining this local from polymorphic type copy submodule is write in local, and transactions requests is write in local, the response quantity of witness copy and the address of hard copy and the hard copy of needs is sent to local and writes affairs Paxos copy consistency submodule, and control the timeout retry that transactions requests treatment scheme is write in whole local;
Affairs Paxos copy consistency submodule is write for arranging daily record value consistent on witness copy in witness replica group in local, this daily record value is sent to local and writes affairs and submit submodule to, and adds locally transaction tag for journal entry;
Local is write affairs and is submitted to submodule for daily record value being committed to corresponding hard copy according to the response quantity of the address of hard copy and the hard copy of needs, and successful for process result is returned to and locally write issued transaction submodule;
Issued transaction submodule is write also for successful for process result is sent to external interface submodule in local;
External interface submodule is also for being sent to client by successful for process result;
Global write issued transaction submodule is used for the response quantity obtaining witness copy corresponding to each write command of comprising this global write transactions requests and the address of hard copy and the hard copy of needs from polymorphic type copy submodule, and the response quantity of witness copy corresponding to global write transactions requests, each write command and the address of hard copy and the hard copy of needs is sent to main subtransaction Paxos copy consistency submodule, and control the timeout retry of whole global write transactions requests treatment scheme;
The daily record value that main subtransaction Paxos copy consistency submodule is consistent on witness copy in the witness replica group arranging main subtransaction, the positional information of this daily record value and main subtransaction is sent to secondary subtransaction and submits submodule to, and add that global transaction marks for this journal entry;
The daily record value that secondary subtransaction Paxos copy consistency submodule is consistent on witness copy in the witness replica group all secondary subtransactions being arranged to this secondary subtransaction, journal entry for all secondary subtransactions adds positional information and the global transaction mark of main subtransaction, and the daily record value of the daily record value of main subtransaction and all secondary subtransactions is sent to main subtransaction submission submodule;
Global write affairs submit to submodule to be used for, according to the response number of the address of the hard copy of main subtransaction and the hard copy of needs, the daily record value of main subtransaction is committed to corresponding hard copy, and successful for process result is returned to global write issued transaction submodule, by the daily record value of this secondary subtransaction, corresponding hard copy is committed to according to the response number of the address of the hard copy of this secondary subtransaction and the hard copy of needs to each secondary subtransaction processing successfully;
Global write issued transaction submodule is also for being sent to external interface submodule by successful for process result;
External interface submodule is also for being sent to client by successful for process result.
2. distributing real time system system according to claim 1, it is characterized in that, transaction preprocess submodule judges its type by the OPER field read in transactions requests, this field is that transactions requests is read in expression, transactions requests is write for representing, consistance hash function is utilized to carry out computing to the key writing each write operation comprised in transactions requests, and judge according to operation result the type writing transactions requests, if same node is all pointed in the key computing of all write operations, then this writes transactions requests is that transactions requests is write in local, otherwise is global write transactions requests.
3. distributing real time system system according to claim 1, it is characterized in that, copy is repaired submodule and all journal entries being less than this numbering is all obtained homogeneity value and judges whether to need to submit to, if the transaction types of this journal entry record is local affairs, then when journal entry reaches consistent in witness copy, namely think and can submit to, otherwise just submit blank operation to; If the transaction types of this journal entry record is global transaction, then except whether needs inspection reaches unanimously in witness copy, also to check whether the main subtransaction of storage is submitted to, when main subtransaction is submitted to, just think and can submit to, otherwise just submitting blank operation to, finally all affairs needing to submit to being performed to completing state.
4. distributing real time system system according to claim 1, it is characterized in that, it is use Paxos algorithm that affairs Paxos copy consistency submodule is write in local, attempt reaching the consistent of daily record value on the same iog location of each witness copy, this daily record value is that this time locally writing the operation of write command in transactions requests adds timestamp, and this timestamp is greater than the maximum submission state log timestamp that this Paxos algorithm performs this witness replica group front.
5. distributing real time system system according to claim 1, it is characterized in that, global write affairs comprise two or more local and write affairs, all can be coupled with global transaction mark, one of them can be designated as main subtransaction, and is used by as submission point, and other local are write affairs and are designated as secondary subtransaction, the positional information of main subtransaction will be recorded, for copy reparation.
6. distributing real time system system according to claim 1, it is characterized in that, main subtransaction Paxos copy consistency submodule uses Paxos algorithm, attempt reaching the consistent of daily record value on the same iog location of each witness copy of main subtransaction, this daily record value is that the operation of write command in this main subtransaction adds timestamp, and this timestamp is greater than the maximum submission state log timestamp that this Paxos algorithm performs this witness replica group front.
7. distributing real time system system according to claim 1, it is characterized in that, secondary subtransaction Paxos copy consistency submodule uses Paxos algorithm, to each secondary subtransaction, attempt reaching the consistent of daily record value on the same iog location of each witness copy of secondary subtransaction, this daily record value for this reason in secondary subtransaction the operation of write command add timestamp, this timestamp be greater than this Paxos algorithm perform before the maximum submission state log timestamp of this witness replica group.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310005857.8A CN103164219B (en) | 2013-01-08 | 2013-01-08 | The distributing real time system system of polymorphic type copy is used in decentralization framework |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310005857.8A CN103164219B (en) | 2013-01-08 | 2013-01-08 | The distributing real time system system of polymorphic type copy is used in decentralization framework |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103164219A CN103164219A (en) | 2013-06-19 |
| CN103164219B true CN103164219B (en) | 2015-09-23 |
Family
ID=48587340
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310005857.8A Expired - Fee Related CN103164219B (en) | 2013-01-08 | 2013-01-08 | The distributing real time system system of polymorphic type copy is used in decentralization framework |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103164219B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106156174A (en) * | 2015-04-16 | 2016-11-23 | 中国移动通信集团山西有限公司 | The system and method that a kind of db transaction processes |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103530362B (en) * | 2013-10-12 | 2017-01-04 | 清华大学 | A kind of computer data reading/writing method for many copies distributed system |
| CN104699527A (en) * | 2013-12-10 | 2015-06-10 | 杭州海康威视系统技术有限公司 | Critical resource management method and device in cloud storage system |
| CN105208096A (en) * | 2015-08-24 | 2015-12-30 | 用友网络科技股份有限公司 | Distributed cache system and method |
| US11157517B2 (en) * | 2016-04-18 | 2021-10-26 | Amazon Technologies, Inc. | Versioned hierarchical data structures in a distributed data store |
| CN106021277A (en) * | 2016-04-27 | 2016-10-12 | 湖南蚁坊软件有限公司 | State-based method for implementation of lock-less distributed ACID consistency |
| US10671639B1 (en) | 2017-03-30 | 2020-06-02 | Amazon Technologies, Inc. | Selectively replicating changes to hierarchial data structures |
| US10860550B1 (en) | 2017-03-30 | 2020-12-08 | Amazon Technologies, Inc. | Versioning schemas for hierarchical data structures |
| CN108322459B (en) * | 2018-01-31 | 2020-10-16 | 北京信息科技大学 | EPaxos-based decentralized network domain name service method and system |
| CN109783578B (en) * | 2019-01-09 | 2022-10-21 | 腾讯科技（深圳）有限公司 | Data reading method and device, electronic equipment and storage medium |
| CN109902127B (en) * | 2019-03-07 | 2020-12-25 | 腾讯科技（深圳）有限公司 | Historical state data processing method and device, computer equipment and storage medium |
| CN112995262B (en) * | 2019-12-18 | 2022-06-10 | 中国移动通信集团浙江有限公司 | Distributed transaction submission method, system and computing equipment |
| CN114661816B (en) * | 2020-12-24 | 2023-03-24 | 金篆信科有限责任公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN115357600B (en) * | 2022-10-21 | 2023-02-03 | 鹏城实验室 | Data consensus processing method, system, device, equipment and readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521330A (en) * | 2011-12-07 | 2012-06-27 | 华中科技大学 | Mirror distributed storage method under desktop virtual environment |
| CN102831156A (en) * | 2012-06-29 | 2012-12-19 | 浙江大学 | Distributed transaction processing method on cloud computing platform |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6671686B2 (en) * | 2000-11-02 | 2003-12-30 | Guy Pardon | Decentralized, distributed internet data management |
-
2013
- 2013-01-08 CN CN201310005857.8A patent/CN103164219B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521330A (en) * | 2011-12-07 | 2012-06-27 | 华中科技大学 | Mirror distributed storage method under desktop virtual environment |
| CN102831156A (en) * | 2012-06-29 | 2012-12-19 | 浙江大学 | Distributed transaction processing method on cloud computing platform |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106156174A (en) * | 2015-04-16 | 2016-11-23 | 中国移动通信集团山西有限公司 | The system and method that a kind of db transaction processes |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103164219A (en) | 2013-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103164219B (en) | The distributing real time system system of polymorphic type copy is used in decentralization framework | |
| US11507480B2 (en) | Locality based quorums | |
| TWI732463B (en) | Block chain state data recovery method and device, and electronic equipment | |
| CN108804112B (en) | Block chain settlement processing method and system | |
| US11360863B2 (en) | Key-value store on persistent memory | |
| CN111338766B (en) | Transaction processing method and device, computer equipment and storage medium | |
| US10956444B2 (en) | Block chain state data synchronization method, apparatus, and electronic device | |
| CN105393243B (en) | Transaction sequencing | |
| US9779128B2 (en) | System and method for massively parallel processing database | |
| CN105359099B (en) | Index update pipeline | |
| US8301600B1 (en) | Failover recovery in a distributed data store | |
| CN108073656A (en) | A kind of method of data synchronization and relevant device | |
| JP6097880B2 (en) | Byzantine fault tolerance data replication method and system | |
| CN103268318A (en) | Distributed key value database system with strong consistency and read-write method thereof | |
| US10089320B2 (en) | Method and apparatus for maintaining data consistency in an in-place-update file system with data deduplication | |
| KR20150129839A (en) | System-wide checkpoint avoidance for distributed database systems | |
| US20180165343A1 (en) | Quorum-based replication of data records | |
| US10324905B1 (en) | Proactive state change acceptability verification in journal-based storage systems | |
| CN105512939A (en) | Foreign exchange transaction-related data storage and query method, declaration method and system | |
| US20120284244A1 (en) | Transaction processing device, transaction processing method and transaction processing program | |
| US11880356B1 (en) | Multi-processor transaction-based validation architecture that compares indicia associated with matching transaction tags | |
| US10467223B1 (en) | Mixed-mode method for combining active/active and validation architectures | |
| CN104978336A (en) | Unstructured data storage system based on Hadoop distributed computing platform | |
| CN102693312B (en) | Flexible transaction management method in key-value store data storage | |
| CN108140054A (en) | For the technology for using the application revocation of SQL patches collection or change collection and reforming |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20150923 |