CN106503257B - Distributed transaction server method and system based on binlog compensation mechanism - Google Patents
Distributed transaction server method and system based on binlog compensation mechanism Download PDFInfo
- Publication number
- CN106503257B CN106503257B CN201611005838.5A CN201611005838A CN106503257B CN 106503257 B CN106503257 B CN 106503257B CN 201611005838 A CN201611005838 A CN 201611005838A CN 106503257 B CN106503257 B CN 106503257B
- Authority
- CN
- China
- Prior art keywords
- binlog
- distributed transaction
- data
- event
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention relates to a kind of Distributed transaction server methods based on binlog compensation mechanism, comprising the following steps: S1: will be in the binlog event of the operation write-in MySql of distributed transaction；S2: the binlog event of distributed transaction is obtained, and the binlog event is parsed, the binlog data after being parsed；S3: according to the binlog data after the parsing, judge whether the distributed transaction submits success；If submission is unsuccessful, binlog event corresponding with the binlog data after the parsing is taken out, according to the Sql sentence of the corresponding binlog event construction Contrary compensation and is executed；Otherwise it completes.The invention further relates to a kind of Distributed transaction server systems based on binlog compensation mechanism.
Description
Technical field
The present invention relates to technical field of the computer network more particularly to a kind of distributed things based on binlog compensation mechanism
Business method of servicing and system.
Background technique
OLTP (On-Line Transaction Processing, Transaction Processing) is mainly to execute basic, day
Normal issued transaction, for example, data-base recording increasing, delete, change, looking into, it is higher to requirement of real-time.Distributed transaction is as single
The sequence of operations or be completely executed or do not execute fully that logical unit of work executes, with atomicity, unanimously
Property, isolation, persistence.Distributed transaction is related to multiple databases for being distributed in different places, but must to the operation of database
It must all be submitted or rollback.As long as any database operation failure, all databases for participating in affairs require rollback.Thing
Business compensation, which refers to, does reverse operating to a distributed transaction, for example a distributed transaction is inserted into n data to a table, then
Corresponding n data is deleted when Contrary compensation.Binlog (binary log, binary log) is that MySql executes change
The journal file of generation, is stored in disk in the form of binary, can be used for data recovery and master-slave synchronisation.
The demand in terms of transaction consistency can be all faced in OLTP system regions, under many business scenarios, it is most typical
Scene such as transferred account service, user A transfer accounts to user B, and A account first is withholdd, and then B account is stored in the corresponding amount of money, this two step
Operation or all success or all failure.Either party successful, another party fails, and all data can be caused inconsistent.
If what is involved is the same databases for above-mentioned two step, the thing carried by relevant database is only needed
Businesslike demand can be realized in business administrative mechanism.However Large-Scale Interconnected net platform is often by a series of distributed systems now
It constitutes, therefore a business processing may need to call multiple " services " and operate multiple databases and could complete, therefore only
Relying only on the included affair mechanism of relevant database can not meet our demand.For this case, present industry
The solution of proposition generally has two-phase commitment protocol (Two Phase Commit Protocol, 2PC agreement) and affairs to mend
Repay mechanism.
Two-phase commitment protocol can guarantee the strong consistency of data, it is to coordinate all distributed atomic transactions to participate in
Person, and determine the distributed algorithm of submission or rollback.In two-phase commitment protocol, system generally comprise two class machines (or
Node): one kind is coordinator, only one in a usual system, another kind of is affairs participant, is generally comprised multiple.It cares for
Name Si Yi, it is respectively request stage and presentation stage that two-phase commitment protocol is made of two stages.In request stage, association
Tune person notifies all affairs participants to prepare to submit affairs or cancel affairs, subsequently into decision process, all participations
The decision of person informing coordinator oneself: agree to submit affairs or cancel affairs.In presentation stage, coordinator can be according to all ginsengs
It determines to submit affairs or cancellation with the decision of person, when all participants are ready to submit affairs, thing
Business coordinator can just issue the notice for submitting affairs to all participants, otherwise, all participants be notified to cancel affairs.
Affairs compensation mechanism refers to that all database atomic transactions in a distributed transaction, being related to are all necessary
There are a corresponding reverse compensation services.When distributing real time system fails to intermediate state, to processed database
Atomic transaction does Contrary compensation operation, reaches access database finally consistent requirement, such as: first database is submitted successfully
Afterwards, it when accessing second failed database, needs in service code, successful data is submitted to first database, do anti-
To compensating operation.
For Two-phase commitment, which has some disadvantages:
Synchronous obstruction: node is in blocked state when waiting for the arrival of news, other processes need to access public in node
It then needs that blocking process is waited to discharge the resource when resource.
Single Point of Faliure: if failure has occurred in coordinator, participant will be unable to complete affairs and block down always.
Especially in second stage, coordinator breaks down, then all participants are also all in the state of locking transaction resource, and
Transaction operation can not be continued to complete.
Data are inconsistent: in presentation stage, if coordinator occurs after sending submission notice to a part of participant
Failure, then notified participant can submit affairs, and not notified participant can not execute affairs submission, this meeting
Cause the data in entire distributed system inconsistent.
Similarly there are some problems in existing affairs compensation mechanism, such as to will cause size of code huge, and coupling is high.And
And have very much limitation, particularly due to many business can not simply realize rollback.If there are many serial service, rollback
Cost it is so high, development efficiency is low, especially under complex scene.
Summary of the invention
In view of this, the present invention provides a kind of Distributed transaction server method and system based on binlog compensation mechanism,
It can be protected according to the binlog analytic construction Contrary compensation Sql sentence of MySql database, Lai Jinhang db transaction Contrary compensation
Hinder data consistency, reduce the size of code of business side, and the degree of coupling is low, and the submission and compensation of distributed transaction are all non-resistances
Plug.
To achieve the above object, according to an aspect of the invention, there is provided a kind of point based on binlog compensation mechanism
Cloth Transaction Service method.
It is of the invention based on the Distributed transaction server method of binlog compensation mechanism the following steps are included: S1: will be distributed
In the binlog event of the operation write-in MySql of formula affairs；S2: the binlog event of distributed transaction is obtained, and to described
Binlog event is parsed, the binlog data after being parsed；S3: according to the binlog data after the parsing, judgement
Whether the distributed transaction submits success；If submission is unsuccessful, take out corresponding with the binlog data after the parsing
Binlog event according to the Sql sentence of the corresponding binlog event construction Contrary compensation and is executed；Otherwise it completes.
Optionally, before step S1, the binlog write-in functions of the MySql are opened, and by the MySql's
Binlog format is set as ROW mode.
Optionally, before step S3, the time-out time of the distributed transaction is set；If can not submit in step s3
Function then judges whether to receive compensation notice；If receiving compensation notice, the binlog data after taking out the parsing are corresponding
Binlog event according to the Sql sentence of the corresponding binlog event construction Contrary compensation and is executed；If not receiving
Compensation notice, then judge whether to reach the time-out time；If reaching the time-out time, after taking out the parsing
The corresponding binlog event of binlog data according to the Sql sentence of the corresponding binlog event construction Contrary compensation and adds
To execute；Otherwise according to the binlog data after the parsing, judge whether the distributed transaction submits success.
Optionally, the binlog-row-image under the ROW mode is set as full.
Optionally, step S1 further include: create the transaction table and mode field of database, the transaction table is for recording institute
State the relevant information of database and the operation.
Optionally, the relevant information includes but is not limited to: major key id, the database of unique identification respective record
The information for all databases that the id of the distributed transaction of participation, the distributed transaction are related to records the database submission
The timestamp of the distributed transaction.
Optionally, after the Sql sentence of Contrary compensation runs succeeded, all records in the transaction table are deleted, and will
Mode field is set as completing；After the distributed transaction is submitted successfully, mode field is set as completing.
Optionally, after the binlog data after being parsed in step s 2, the binlog data after parsing are sealed
Dress processing, and the binlog data of encapsulation are stored in Redis or Memcache.
Optionally, with database participate in distributed transaction id and database unique identification be key, with it is packaged,
All operations involved in one database affairs are value, by the binlog data storage of encapsulation to Redis or Memcache
In.
According to another aspect of the present invention, a kind of Distributed transaction server system based on binlog compensation mechanism is provided
System.
Distributed transaction server system based on binlog compensation mechanism of the invention includes: writing module, for that will divide
In the binlog event of the operation write-in MySql of cloth affairs；Parsing module, for obtaining the binlog thing of distributed transaction
Part, and the binlog event is parsed, the binlog data after being parsed；Judgement and operation module are used for basis
Binlog data after the parsing, judge whether the distributed transaction submits success；If submission is unsuccessful, the judgement
The corresponding binlog event of binlog data after being also used to take out the parsing with operation module, according to described corresponding
The Sql sentence of binlog event construction Contrary compensation is simultaneously executed；Otherwise it completes.
It optionally, further include setup module, for opening the binlog write-in functions of the MySql, and will be described
The binlog format of MySql is set as ROW mode.
Optionally, the judgement and operation module are also used to be arranged the time-out time of the distributed transaction；If described
Submit unsuccessful in judgement and operation module, then the judgement and operation module judge whether to receive compensation notice；If receiving benefit
Notice is repaid, then the corresponding binlog event of binlog data after the judgement takes out the parsing with operation module, according to institute
It states the Sql sentence of corresponding binlog event construction Contrary compensation and is executed；It is described to sentence if not receiving compensation notice
Disconnected and operation module judges whether to reach the time-out time；If reaching the time-out time, the judgement and operation module
The corresponding binlog event of binlog data after taking out the parsing is mended according to the corresponding binlog event construction is reversed
The Sql sentence repaid simultaneously is executed；Otherwise according to the binlog data after the parsing, judgement is described to divide for judgement and operation module
Whether cloth affairs submit success.
Optionally, the setup module is also used to set full for the binlog-row-image under the ROW mode.
It optionally, further include creation module, for creating the transaction table and mode field of database, the transaction table is used for
Record the relevant information of the database and the operation.
Optionally, the relevant information includes but is not limited to: major key id, the database of unique identification respective record
The information for all databases that the id of the distributed transaction of participation, the distributed transaction are related to records the database submission
The timestamp of the distributed transaction.
Optionally, the judgement and operation module are also used to delete all records in the transaction table, and by state
Field is set as completing.
Optionally, the parsing module is also used to the binlog data after parsing being packaged processing, and by encapsulation
Binlog data are stored in Redis or Memcache.
Optionally, the id and database unique identification for the distributed transaction that the parsing module is also used to participate in database
For key, the binlog data of encapsulation are stored for value with all operations involved in a packaged, database affairs
Into Redis or Memcache.
Optionally, the database that the distributed transaction is related to is connect by server with ZooKeeper.
According to the technique and scheme of the present invention, by the binlog event of the operation write-in MySql of distributed transaction, and according to solution
Binlog data after analysis judge whether distributed transaction submits success, if submission is unsuccessful, after taking out and parsing
The corresponding binlog event of binlog data constructs the Sql sentence of Contrary compensation according to corresponding binlog event and is held
Row, and then ensured the consistency of distributed transaction.Wherein in Contrary compensation, if receiving compensation notice, compensation is being received
Start to do Contrary compensation operation to corresponding db transaction after notice.Compensation notice is not being received, it can be according to setting
Time-out time judges whether to compensate.The dual judgment mechanism of compensation notice and time-out time, more can effectively ensure data
Consistency.
Detailed description of the invention
Attached drawing for a better understanding of the present invention, does not constitute an undue limitation on the present invention.Wherein:
Fig. 1 is the main step of the Distributed transaction server method according to an embodiment of the present invention based on binlog compensation mechanism
Rapid schematic diagram；
Fig. 2 is the process of the Distributed transaction server method according to an embodiment of the present invention based on binlog compensation mechanism
Figure；
Fig. 3 is the main mould of the Distributed transaction server system according to an embodiment of the present invention based on binlog compensation mechanism
The schematic diagram of block.
Specific embodiment
Below in conjunction with attached drawing, an exemplary embodiment of the present invention will be described, including the various of the embodiment of the present invention
Details should think them only exemplary to help understanding.Therefore, those of ordinary skill in the art should recognize
It arrives, it can be with various changes and modifications are made to the embodiments described herein, without departing from scope and spirit of the present invention.Together
Sample, for clarity and conciseness, descriptions of well-known functions and structures are omitted from the following description.
Fig. 1 is the main step of the Distributed transaction server method according to an embodiment of the present invention based on binlog compensation mechanism
Rapid schematic diagram.
As shown in Figure 1, the Distributed transaction server method based on binlog compensation mechanism of the embodiment of the present invention is main
Step includes:
S1: will be in the binlog event of the operation write-in MySql of distributed transaction.
Before by the binlog event of distributed transaction write-in MySql, need to do corresponding preparation, such as: it opens
The binlog write-in functions of MySql are opened, and set ROW mode for the binlog format of MySql.Under ROW mode
Binlog log content can most clearly record the details of every data line modification, facilitate reversed according to log content construction
The Sql sentence of compensation.Further, full is set by the binlog-row-image under ROW mode, in this mode,
Each row of data updates the record of front and back in the detailed record sheet of meeting in binlog log.
S2: the binlog event of distributed transaction is obtained, and binlog event is parsed, after being parsed
Binlog data.
In step s 2, the binlog event that can constantly get database will parse after the parsing of binlog event
Rear binlog data according to db transaction are that unit is packaged, and with the xa_id (distributed transaction that database participates in
Id) and database unique identification is key, is deposited with all operations involved by a packaged, database affairs for value
It stores up in Redis or Memcache, to be done used in Contrary compensation later.
S3: according to the binlog data after parsing, judge whether distributed transaction submits success；If submission is unsuccessful,
Binlog event corresponding with the binlog data after parsing is taken out, Contrary compensation is constructed according to corresponding binlog event
Sql sentence is simultaneously executed；Otherwise it completes.
In order to record the operating status of distributed transaction and the relevant information of database and operation, step S1 further include:
Create the transaction table (transaction table) and mode field (status field) of database, wherein transaction table is for recording number
According to the relevant information in library and operation, mode field then is used to describe the state of the record of transaction table.
The database of transaction table record and the relevant information of operation include: the major key id of unique identification respective record, data
The information for all databases that id, the distributed transaction for the distributed transaction that library participates in are related to, database of record are submitted distributed
The timestamp of affairs is denoted as major key id, xa_id, db_set, timestamp respectively.There can be a thing in each database
Business table, which is the key that judge whether the db transaction of Distributed transaction server needs to compensate.For in a distribution
Each database being related in affairs can all be inserted into an identical record in the transaction table in the library, such as: one point
Cloth affairs need while operating DB1, DB2 and DB3, and the ID of the distributed transaction is 64e04a9fe21e, in the distribution thing
The timestamp that first db transaction in business is submitted is 1467343573281, then in the transaction table of DB1, DB2 and DB3
It will be inserted into a same record (major key id is perhaps different), as shown in the table:
| 123 | 64e04a9fe21e | DB1、DB2、DB3 | 1467343573281 |
The foundation for functioning as Distributed transaction server compensating transaction after judging affairs time-out of above-mentioned table.
Other than establishing transaction table in each database, a mode field need to be also added in traffic table, is used for
The state of this record is described.It is entire distributed although some db transactions are submitted in a distributed transaction
Affairs identify this intermediate state, can mention in all db transactions also in an intermediate state, the mode field
It hands over successfully or whole compensates when completing, update the mode field state, make record end intermediate state.
Fig. 2 is the process of the Distributed transaction server method according to an embodiment of the present invention based on binlog compensation mechanism
Figure.
As shown in Fig. 2, the process of the Distributed transaction server method based on binlog compensation mechanism of the embodiment of the present invention
Figure are as follows:
S10: it after starting a distributed transaction, in a code block, as the affairs of general data library, successively mentions
Hand over each db transaction.
S20: one db transaction of every submission can all be inserted into a record in the transaction table of correspondence database, record
Data it is as indicated above, and the state of business datum is set as unfinished.It, can be voluntarily when other transactional queries are to the record
How decision handles the record of the intermediate state, such as modifies or delete the record of the intermediate state.Meanwhile into database
After insertion, deletion or more new data, corresponding record can be recorded in detail binlog event and transaction table by MySql
In, the information recorded is sufficient to be restored to the state before updating.
S30: the binlog event of database constantly is got, and binlog event is parsed, after being parsed
Binlog event.It is packaged for unit according to db transaction after being parsed to binlog event, and only with xa_id and database
One is identified as key, with all operations involved by a packaged, database affairs be value be cached to Redis or
In Memcache, to be done used in Contrary compensation later.
S40: according to the binlog data after parsing, judge whether distributed transaction submits success；If submission is unsuccessful,
S50 is carried out, S80 is otherwise carried out.
S50: judge whether to receive compensation notice, if receiving compensation notice, carry out S60, otherwise carry out S9.If one
Some db transaction in a distributed transaction submits failure, then needs to issue compensation notice to system, can be bright in the notice
Really point out the xa_id for the distributed transaction that needs compensate, and the id for the database for needing to compensate.
S60: after the notice for receiving the transmission of business side, according to the combination of xa_id and database unique identification as key
The corresponding binlog event of binlog data after taking out parsing in Redis or Memcache, constructs according to binlog event
The Sql sentence of Contrary compensation is simultaneously executed, and deletes the record after running succeeded from Redis.
S70: it will need to do after the data-base recording compensation compensated finishes, delete the record in corresponding transaction table.
S80: mode field is set as completing.
S90: judging whether to reach time-out time, if reaching time-out time, carries out S60, otherwise carries out S40.In business
Side prepares to break down when issuing compensation notice, then notice can not issue compensation notice, at this moment can record according in transaction table
Timestamp timestamp and setting time-out time to determine whether needing to compensate.So time-out need to be arranged before S90
Time.After Distributed transaction server waits until that time-out time reaches, the xa_id according to recorded in transaction table and db_set are gone
Whether the binlog record that db transaction involved in the distributed transaction is inquired in Redis is completely recorded, if not having,
Judge that the distributed transaction successfully needs to do submitted db transaction within time-out time without submitting completely
Contrary compensation.The presence of overtime compensation mechanism, which to break down when business side prepares and issues compensation notice, will not influence to mend
The progress repaid.
Distributed transaction server method disclosed by the invention based on binlog compensation mechanism is submitted in distributed transaction
When, all db transactions involved in this distributed transaction, which are not required to block as Two-phase commitment mode, waits mutual the
One stage submitted as a result, the db transaction in the present invention can return immediately after once submitting, and the confirmation operation of affairs exists
It completes on backstage.In addition, the code of user need not be blocked in if any one db transaction of distributed transaction fails
This can just be turned with waiting all compensating operations to carry out other operations after terminating again after distributed transaction submission
And go to handle other logic, all compensating operations are completed on backstage, and user is not influenced.So disclosed by the invention
Based on the Distributed transaction server method of binlog compensation mechanism for the submission of distributed transaction and compensation all right and wrong of affairs
Obstruction.
Distributed transaction server method disclosed by the invention based on binlog compensation mechanism is High Availabitity, distributed thing
Business service system be in the form of cluster existing for, node break down when the work of malfunctioning node can be moved to rapidly
Other nodes.That is for distributed transaction, node village host node and backup node in this method, main-standby nodes are logical
Crossing ZooKeeper is to realize that High Availabitity can be immediately switched to backup node and continue offer service once host node is unavailable.
Fig. 3 is the main mould of the Distributed transaction server system according to an embodiment of the present invention based on binlog compensation mechanism
The schematic diagram of block.
As shown in figure 3, the Distributed transaction server system 3 based on binlog compensation mechanism of the embodiment of the present invention is mainly wrapped
Include writing module 31, parsing module 32 and judgement and operation module 32.Writing module 31 is used for the operation of distributed transaction
It is written in the binlog event of MySql；Parsing module 32 is used to obtain the binlog event of distributed transaction, and to binlog thing
Part is parsed, the binlog data after being parsed；Judgement and operation module 33 are used for according to the binlog data after parsing,
Judge whether distributed transaction submits success；If submission is unsuccessful, judgement and operation module are also used to take out after parsing
The corresponding binlog event of binlog data constructs the Sql sentence of Contrary compensation according to corresponding binlog event and is held
Row；Otherwise it completes.
The Distributed transaction server system based on binlog compensation mechanism of the embodiment of the present invention further includes setup module, is used
In the binlog write-in functions of unlatching MySql, and ROW mode is set by the binlog format of MySql.Setup module is also used
In setting full for the binlog-row-image under ROW mode.In addition, judgement and operation module are also used to be arranged described point
The time-out time of cloth affairs.Judgement and operation module in submit it is unsuccessful, then judgement and operation module judge whether to receive
Compensation notice；If receiving compensation notice, binlog thing corresponding with the binlog data after operation module taking-up parsing is judged
Part constructs the Sql sentence of Contrary compensation according to corresponding binlog event and is executed；If not receiving compensation notice,
Judgement and operation module judge whether to reach time-out time；If reaching time-out time, judgement and operation module are taken out after parsing
The corresponding binlog event of binlog data, according to corresponding binlog event construct Contrary compensation Sql sentence and be subject to
It executes；Otherwise judgement and operation module judge whether distributed transaction submits success according to the binlog data after parsing.
The Distributed transaction server system based on binlog compensation mechanism of the embodiment of the present invention further includes creation module, is used
In the transaction table and mode field of creation database, transaction table is used for the relevant information of database of record and operation.Wherein, related
Information includes but is not limited to: the id for the distributed transaction that major key id, the database of unique identification respective record participate in, distribution
The information for all databases that affairs are related to, database of record submit the timestamp of distributed transaction.Judgement and operation module are also
It is set as completing for deleting all records in transaction table, and by mode field.
Parsing module is also used to the binlog data after parsing being packaged processing, and the binlog data of encapsulation are deposited
Storage is in Redis or Memcache.The id and database unique identification for the distributed transaction that parsing module is participated in database be
Key is arrived the binlog data storage of encapsulation for value with all operations involved in a packaged, database affairs
In Redis or Memcache, to judge whether db transaction needs to compensate.
The database that distributed transaction is related to is connect by server with ZooKeeper, the embodiment of the present invention based on
The Distributed transaction server system of binlog compensation mechanism is a distributed system, after wherein some nodes break down,
There can be other nodes to take over the work of the node rapidly, i.e. system has backup node, and main-standby nodes are real by ZooKeeper
Existing High Availabitity can be immediately switched to backup node and continue offer service once host node is unavailable.
Above-mentioned specific embodiment, does not constitute a limitation on the scope of protection of the present invention.Those skilled in the art should be bright
It is white, design requirement and other factors are depended on, various modifications, combination, sub-portfolio and substitution can occur.It is any
Made modifications, equivalent substitutions and improvements etc. within the spirit and principles in the present invention, should be included in the scope of the present invention
Within.
Claims (19)
1. a kind of Distributed transaction server method based on binlog compensation mechanism, which comprises the following steps:
S1: will be in the binlog event of the operation write-in MySql of distributed transaction；
S2: the binlog event of distributed transaction is obtained, and the binlog event is parsed, after being parsed
Binlog data；
S3: according to the binlog data after the parsing, judge whether the distributed transaction submits success；If can not submit
Function then takes out binlog event corresponding with the binlog data after the parsing, according to the corresponding binlog event structure
It makes the Sql sentence of Contrary compensation and is executed；Otherwise it completes.
2. the method according to claim 1, wherein the method also includes: before step S 1, open institute
The binlog write-in functions of MySql are stated, and set ROW mode for the binlog format of the MySql.
3. the method according to claim 1, wherein the method also includes: before step S3, described in setting
The time-out time of distributed transaction；
If submitting in step s3 unsuccessful, judge whether to receive compensation notice；
If receiving compensation notice, the corresponding binlog event of binlog data after taking out the parsing, according to the correspondence
Binlog event construction Contrary compensation Sql sentence and executed；
If not receiving compensation notice, judge whether to reach the time-out time；If reaching the time-out time, institute is taken out
The corresponding binlog event of binlog data after stating parsing, according to the corresponding binlog event construction Contrary compensation
Sql sentence is simultaneously executed；Otherwise according to the binlog data after the parsing, judge whether the distributed transaction is submitted into
Function.
4. according to the method described in claim 2, it is characterized in that, the binlog-row-image under the ROW mode is arranged
For full.
5. -4 described in any item methods according to claim 1, which is characterized in that step S1 further include: create database
Transaction table and mode field, the transaction table are used to record the relevant information of the database and the operation.
6. according to the method described in claim 5, it is characterized in that, the relevant information includes but is not limited to: unique identification
All data that id, the distributed transaction for the distributed transaction that major key id, the database of respective record participate in are related to
The information in library records the timestamp that the database submits the distributed transaction.
7. according to the method described in claim 5, it is characterized in that, deleting institute after the Sql sentence of Contrary compensation runs succeeded
All records in transaction table are stated, and mode field is set as completing；
After the distributed transaction is submitted successfully, mode field is set as completing.
8. the method according to claim 1, wherein the method also includes: after being parsed in step s 2
Binlog data after, the binlog data after parsing are packaged processing, and the binlog data of encapsulation are stored in
In Redis or Memcache.
9. according to the method described in claim 8, it is characterized in that, the id and database of the distributed transaction participated in database
Unique identification is key, with all operations is value involved in a packaged, database affairs, by the binlog of encapsulation
Data are stored into Redis or Memcache.
10. a kind of Distributed transaction server system based on binlog compensation mechanism characterized by comprising
Writing module, for operating for distributed transaction to be written in the binlog event of MySql；
Parsing module is parsed for obtaining the binlog event of distributed transaction, and to the binlog event, is solved
Binlog data after analysis；
Judgement and operation module, for judging whether the distributed transaction is submitted according to the binlog data after the parsing
Success；If submission is unsuccessful, the binlog data after the judgement is also used to take out the parsing with operation module are corresponding
Binlog event according to the Sql sentence of the corresponding binlog event construction Contrary compensation and is executed；Otherwise it completes.
11. system according to claim 10, which is characterized in that further include setup module, for opening the MySql's
Binlog write-in functions, and ROW mode is set by the binlog format of the MySql.
12. system according to claim 10, which is characterized in that the judgement and operation module are also used to be arranged described point
The time-out time of cloth affairs；
If submitting in the judgement and operation module unsuccessful, it is logical that the judgement and operation module judge whether to receive compensation
Know；
If receiving compensation notice, the judgement take out the parsing with operation module after the corresponding binlog of binlog data
Event according to the Sql sentence of the corresponding binlog event construction Contrary compensation and is executed；
If not receiving compensation notice, the judgement and operation module judge whether to reach the time-out time；If reaching institute
State time-out time, then the corresponding binlog event of binlog data after the judgement takes out the parsing with operation module, root
The Sql sentence of Contrary compensation is constructed according to the corresponding binlog event and is executed；Otherwise judgement and operation module according to
Binlog data after the parsing, judge whether the distributed transaction submits success.
13. system according to claim 11, which is characterized in that the setup module is also used to will be under the ROW mode
Binlog-row-image be set as full.
14. described in any item systems of 0-13 according to claim 1, which is characterized in that further include creation module, for creating
The transaction table and mode field of database, the transaction table are used to record the relevant information of the database and the operation.
15. system according to claim 14, which is characterized in that the relevant information includes but is not limited to: Wei Yibiao
Know the major key id of respective record, all numbers that id, the distributed transaction of the distributed transaction that the database participates in are related to
According to the information in library, record the timestamp that the database submits the distributed transaction.
16. system according to claim 14, which is characterized in that the judgement and operation module are also used to delete the thing
All records being engaged in table, and mode field is set as completing.
17. system according to claim 10, which is characterized in that the parsing module be also used to parse after binlog
Data are packaged processing, and the binlog data of encapsulation are stored in Redis or Memcache.
18. system according to claim 10, which is characterized in that the parsing module is also used to point participated in database
The id and database unique identification of cloth affairs are key, are with all operations involved in a packaged, database affairs
Value stores the binlog data of encapsulation into Redis or Memcache.
19. described in any item systems of 0-13 and 15-18 according to claim 1, which is characterized in that the distributed transaction relates to
And database connect with ZooKeeper by server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611005838.5A CN106503257B (en) | 2016-11-15 | 2016-11-15 | Distributed transaction server method and system based on binlog compensation mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611005838.5A CN106503257B (en) | 2016-11-15 | 2016-11-15 | Distributed transaction server method and system based on binlog compensation mechanism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106503257A CN106503257A (en) | 2017-03-15 |
| CN106503257B true CN106503257B (en) | 2019-09-20 |
Family
ID=58324455
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611005838.5A Active CN106503257B (en) | 2016-11-15 | 2016-11-15 | Distributed transaction server method and system based on binlog compensation mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106503257B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107194674A (en) * | 2017-07-06 | 2017-09-22 | 山东浪潮云服务信息科技有限公司 | A kind of intelligent distribution scheduling system based on item business rule |
| CN108989413A (en) * | 2018-07-06 | 2018-12-11 | 深圳市牛鼎丰科技有限公司 | Abnormal traffic compensation method, device, computer equipment and storage medium |
| CN111008202A (en) * | 2018-10-08 | 2020-04-14 | 北京京东尚科信息技术有限公司 | Distributed transaction processing method and framework |
| CN109871388A (en) * | 2019-02-19 | 2019-06-11 | 北京字节跳动网络技术有限公司 | Data cache method, device, whole electronic equipment and storage medium |
| CN111913972A (en) * | 2019-05-10 | 2020-11-10 | 阿里巴巴集团控股有限公司 | Data processing method, device and equipment |
| CN112822091B (en) * | 2019-11-18 | 2023-05-30 | 北京京东尚科信息技术有限公司 | Message processing method and device |
| CN111813791B (en) * | 2020-06-17 | 2024-05-21 | 上海万物新生环保科技集团有限公司 | Distributed transaction compensation method and equipment |
| CN112084267B (en) * | 2020-07-29 | 2024-06-07 | 北京思特奇信息技术股份有限公司 | Method for solving global broadcasting of distributed database |
| CN112597246B (en) * | 2020-12-25 | 2024-04-12 | 中国农业银行股份有限公司 | Method, device and system for maintaining data consistency |
| CN112990767B (en) * | 2021-04-20 | 2021-08-20 | 上海领健信息技术有限公司 | Vertical consumption medical SaaS production data calculation method, system, terminal and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102203779A (en) * | 2011-05-03 | 2011-09-28 | 华为技术有限公司 | Method for updating data and control apparatus thereof |
| CN103221949A (en) * | 2010-07-27 | 2013-07-24 | 甲骨文国际公司 | MYSQL database heterogeneous log based replication |
| CN103345470A (en) * | 2013-05-31 | 2013-10-09 | 深圳市沃信科技有限公司 | Database disaster tolerance method, database disaster tolerance system and server |
| CN104657382A (en) * | 2013-11-21 | 2015-05-27 | 阿里巴巴集团控股有限公司 | Method and device for detecting consistency of data of MySQL master and slave servers |
| CN106033437A (en) * | 2015-03-13 | 2016-10-19 | 阿里巴巴集团控股有限公司 | Method and system for processing distributed transaction |
-
2016
- 2016-11-15 CN CN201611005838.5A patent/CN106503257B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103221949A (en) * | 2010-07-27 | 2013-07-24 | 甲骨文国际公司 | MYSQL database heterogeneous log based replication |
| CN102203779A (en) * | 2011-05-03 | 2011-09-28 | 华为技术有限公司 | Method for updating data and control apparatus thereof |
| CN103345470A (en) * | 2013-05-31 | 2013-10-09 | 深圳市沃信科技有限公司 | Database disaster tolerance method, database disaster tolerance system and server |
| CN104657382A (en) * | 2013-11-21 | 2015-05-27 | 阿里巴巴集团控股有限公司 | Method and device for detecting consistency of data of MySQL master and slave servers |
| CN106033437A (en) * | 2015-03-13 | 2016-10-19 | 阿里巴巴集团控股有限公司 | Method and system for processing distributed transaction |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106503257A (en) | 2017-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106503257B (en) | Distributed transaction server method and system based on binlog compensation mechanism | |
| US11899684B2 (en) | System and method for maintaining a master replica for reads and writes in a data store | |
| US11894972B2 (en) | System and method for data replication using a single master failover protocol | |
| US8892509B2 (en) | Systems and methods for a distributed in-memory database | |
| US20220019575A1 (en) | System And Method For Augmenting Database Applications With Blockchain Technology | |
| CN103782574B (en) | Idempotence for database transactions | |
| EP1840766B1 (en) | Systems and methods for a distributed in-memory database and distributed cache | |
| CN104793988B (en) | The implementation method and device of integration across database distributed transaction | |
| US8719225B1 (en) | System and method for log conflict detection and resolution in a data store | |
| US9489434B1 (en) | System and method for replication log branching avoidance using post-failover rejoin | |
| US20070239791A1 (en) | Systems and methods for a distributed cache | |
| US20020059279A1 (en) | Apparatus and method for database synchronization in a duplex system | |
| US20150120658A1 (en) | System and method for splitting a replicated data partition | |
| US8332358B2 (en) | Traversal-free rapid data transfer | |
| CN110196856B (en) | Distributed data reading method and device | |
| CN106776121B (en) | Data disaster recovery device, system and method | |
| CN106445644A (en) | Distributed transaction processing method and device based on improved one-phase commit | |
| CN102317913B (en) | A kind of transaction recovery method and transaction recovery device | |
| WO2011120452A2 (en) | Method for updating data and control apparatus thereof | |
| CN112925614A (en) | Distributed transaction processing method, device, medium and equipment | |
| CN110427427B (en) | Method for realizing global transaction distributed processing through pin bridging | |
| CN113905054B (en) | RDMA (remote direct memory access) -based Kudu cluster data synchronization method, device and system | |
| US9201685B2 (en) | Transactional cache versioning and storage in a distributed data grid | |
| CN105723365B (en) | Method for optimum indexing, main database node and subscriber database node | |
| CN109408201A (en) | Office management method based on distributed data base |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 101111 Beijing City branch of Daxing District economic and Technological Development Zone, eleven Street 18 floor 2 block C room 221 Patentee after: JINGDONG DIGITAL TECHNOLOGY HOLDINGS Co.,Ltd. Address before: 101111 Beijing City, Daxing District Beijing City, Beijing economic and Technological Development Zone Branch eleven Street 18 floor 2 block C room 221 Patentee before: BEIJING JD FINANCE TECHNOLOGY HOLDING Co.,Ltd. |
|
| CP03 | Change of name, title or address | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 101111 Beijing City branch of Daxing District economic and Technological Development Zone, eleven Street 18 floor 2 block C room 221 Patentee after: Jingdong Digital Technology Holding Co., Ltd Address before: 101111 Beijing City branch of Daxing District economic and Technological Development Zone, eleven Street 18 floor 2 block C room 221 Patentee before: JINGDONG DIGITAL TECHNOLOGY HOLDINGS Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 101111 Room 221, 2nd Floor, Block C, 18 Kechuang 11th Street, Daxing Economic and Technological Development Zone, Beijing Patentee after: Jingdong Technology Holding Co.,Ltd. Address before: 101111 Room 221, 2nd Floor, Block C, 18 Kechuang 11th Street, Daxing Economic and Technological Development Zone, Beijing Patentee before: Jingdong Digital Technology Holding Co., Ltd |
|
| CP01 | Change in the name or title of a patent holder |