WO2021012868A1 - Transaction rollback method and apparatus, database, system, and computer storage medium - Google Patents
Transaction rollback method and apparatus, database, system, and computer storage medium Download PDFInfo
- Publication number
- WO2021012868A1 WO2021012868A1 PCT/CN2020/098046 CN2020098046W WO2021012868A1 WO 2021012868 A1 WO2021012868 A1 WO 2021012868A1 CN 2020098046 W CN2020098046 W CN 2020098046W WO 2021012868 A1 WO2021012868 A1 WO 2021012868A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- transaction
- log
- rollback
- native
- transaction rollback
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Definitions
- the present disclosure relates to but is not limited to the technical field of transaction rollback.
- Typical scenarios are distributed transaction consistency recovery and misoperation recovery.
- the distributed transaction consistency recovery scenario requires Each data node rolls back to a consistent state, involving multiple data nodes performing transaction rollbacks at the same time, which has high performance requirements. Misoperation recovery is often faced by database operation and maintenance personnel, and it is also a function that mature database products should have.
- Real-time generation of transaction rollback logs during transaction execution is a pessimistic strategy, which has a greater impact on transaction execution performance, and it is difficult to rollback batch transactions at the same time, which has certain limitations.
- this disclosure proposes a fast database rollback method based on transaction logs to improve the rollback performance and further expand the application scenarios of transaction logs.
- the embodiment of the present disclosure provides a transaction rollback method, which includes: when a transaction rollback request is received, parsing a native transaction log file within a preset time range; determining the transaction to be rolled back, A transaction rollback log is generated according to the native transaction log corresponding to the transaction to be rolled back; the transaction rollback log is assembled in reverse order to generate a transaction rollback log file, and the transaction rollback log file is executed.
- the embodiment of the present disclosure also provides a transaction rollback device, the transaction rollback device includes a native transaction rollback control module and a transaction rollback log execution module; the native transaction rollback control module is configured to receive a transaction When a rollback request is made, the native transaction log file within the preset time range is parsed; the transaction to be rolled back is determined, and the transaction rollback log is generated according to the native transaction log corresponding to the transaction to be rolled back; the transaction is returned in reverse order
- the roll log assembly generates a transaction rollback log file; the transaction rollback log execution module is configured to execute the transaction rollback log file.
- the embodiment of the present disclosure also provides a database, which includes the above-mentioned transaction rollback device.
- the embodiment of the present disclosure also provides a system, the system includes the above-mentioned database, and the system includes a transaction rollback system based on a MySQL database, or a data recovery system based on a MariaDB distributed cluster.
- the embodiment of the present disclosure also provides a computer storage medium, the computer storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to realize the transaction response as described above. Steps of rolling method.
- FIG. 1 is a schematic diagram of the basic flow of a transaction rollback method provided by Embodiment 1 of the present disclosure
- FIG. 2 is a schematic diagram of a basic process for generating a native transaction log file according to Embodiment 1 of the present disclosure
- Embodiment 3 is a schematic structural diagram of a feasible transaction rollback device provided by Embodiment 3 of the disclosure.
- FIG. 4 is a schematic structural diagram of another feasible transaction rollback device provided in Embodiment 3 of the disclosure.
- FIG. 5 is a schematic diagram of a transaction rollback system based on a MySQL database provided by the fourth embodiment of the disclosure.
- FIG. 6 is a schematic diagram of a data recovery system based on MariaDB distributed cluster provided in the fourth embodiment of the disclosure.
- Fig. 1 is a schematic diagram of the basic flow of the transaction rollback method provided in this embodiment. As shown in Fig. 1, the method may include steps S101 to S103.
- step S101 when a transaction rollback request is received, the native transaction log file within a preset time range is parsed.
- the method may further include at least steps S201 to S205, as shown in FIG. 2 .
- step S201 the data change of each transaction and the correlation between each transaction are recorded.
- a global transaction identifier is assigned to each transaction, and the global transaction identifier includes a database instance identifier and a global sequence identifier.
- the database instance ID refers to the database instance on which the current transaction is executed
- the global sequence ID refers to the ID of the current transaction in a certain database.
- the global sequence ID starts from 1 and increases in sequence.
- step S203 the same batch number is assigned to each transaction that is not related.
- the same batch number can be assigned to each transaction that is not related, so that the subsequently generated transaction rollback log retains the batch number of each transaction in the native transaction log. That is, the concurrency relationship of each transaction execution in the native transaction log is retained.
- step S204 a native transaction log is generated.
- the generated native transaction log includes not only the data changes before and after the data operation, but also the global transaction identifier and batch number.
- step S205 the native transaction log is assembled to generate a native transaction log file.
- the native transaction log corresponds to a single transaction, so the native transaction log can be assembled in reverse order to generate a transaction rollback log file for storage.
- the native transaction log file includes a binary log binlog file.
- step S102 the transaction to be rolled back is determined, and a transaction rollback log is generated according to the native transaction log corresponding to the transaction to be rolled back.
- the log directory is filtered according to the preset time range to obtain the native transaction log files to be parsed, and then these native transaction log files are parsed in parallel, and based on the global transaction Identify, locate the transaction to be rolled back, and determine the transaction to be rolled back.
- determining the transaction to be rolled back, and then generating the transaction rollback log according to the native transaction log corresponding to the transaction to be rolled back includes: performing reverse operations on the native transaction log corresponding to the transaction to be rolled back, and performing reverse operations.
- the protocol format of the transaction rollback log generated after the operation remains the same as the protocol format of the native transaction log.
- the native transaction log records the table data before and after the execution of each statement operation. Inserting and deleting records requires modifying the operation type, while updating records requires exchanging the table data before and after changes. Therefore, the generated transaction rollback log is consistent with the protocol format of the native transaction log.
- the reverse operation includes at least one of a reverse order operation statement and a reverse order execution statement.
- the protocol format of the transaction rollback log generated after the reverse operation remains the same as the protocol format of the native transaction log, it also includes:
- the transaction rollback log is assigned the same global transaction identifier as the transaction corresponding to the native transaction log.
- the global transaction identifier of the transaction to be rolled back is set to identify that the current transaction is a transaction to be rolled back, and the global transaction identifier of the transaction to be rolled back should correspond to the global transaction identifier of the original transaction, or both exist one by one The mapping relationship.
- step S103 the transaction rollback log is assembled in reverse order to generate a transaction rollback log file, and the transaction rollback log file is executed.
- the transaction rollback log corresponds to a single transaction to be rolled back, so the generated transaction rollback log can be assembled in reverse order according to the global sequence identifier to complete the reverse order of the transactions, and generate a transaction rollback log file.
- executing the transaction rollback log file includes: parsing the batch number corresponding to each transaction in the transaction rollback log file, executing transactions with the same batch number in parallel, and performing transactions with different batch numbers. Transactions are executed serially. In this way, the parallel playback of the transaction to be rolled back is realized, and the performance of the transaction rollback is improved to a certain extent.
- the transaction rollback method analyzes the native transaction log file within a preset time range when a transaction rollback request is received to determine the transaction to be rolled back, and then according to the transaction corresponding to the transaction to be rolled back
- the native transaction log generates the transaction rollback log.
- the transaction rollback log is assembled in reverse order to generate the transaction rollback log file, and the transaction rollback log file is executed; it solves the problem that the transaction rollback method in the prior art cannot meet the high demand for multiple transactions. Performance requirements, low rollback performance issues.
- the transaction rollback method provided by the embodiment of the present disclosure not only reduces the re-analysis of standard sentence files by the database, but also makes full use of the transaction concurrency information of the rollback log, which effectively improves the performance of transaction rollback, especially for batch transactions.
- the rollback effect is better.
- the embodiments of the present disclosure take the MySQL database as an example, and the process of specifying a transaction set to implement fast rollback is as follows.
- Step 1 After receiving the transaction rollback request, retrieve the binlog log according to the specified transaction set to obtain the binlog file to be processed.
- Step 2 Sort the binlog files in reverse order according to the serial numbers of the binlog files from large to small to form a binlog file reverse queue, and process binlog files with large serial numbers first.
- Step 3 Create a binlog file parsing thread group, binlog transaction rollback processing thread pool, and binlog rollback log file output thread group.
- Step 4 Each file output thread sets up a labeled task list. Rollback transactions generated in different binlog files are modulo the binlog file serial number and then delivered to the output thread task list corresponding to different labels.
- Step 5 The parsing threads are selected from the binlog file queue in turn, and the binlog files are sequentially parsed from the beginning. If the parsed transaction exists in the specified transaction set to be rolled back, the transaction log is packaged and placed in the rollback task list.
- Step 6 After the binlog file is parsed, flip the task list to achieve the purpose of reverse transaction processing, and then trigger the task list to perform transaction log rollback processing on the binlog file that has been parsed.
- Step 7 The binlog transaction rollback processing thread pool obtains the task node to be rolled back from the task list, and performs rollback processing on the binlog transaction log.
- the rollback transaction retains the global sequence identifier of the native transaction.
- Step 8 Binlog rollback log file output.
- the thread group receives task nodes from the thread task list in real time, and then writes the transaction rollback log into the corresponding transaction rollback log file according to the binlog protocol.
- Step 9 According to the binlog file reverse queue, retrieve whether the first binlog file has completed the rollback processing and output, then push the transaction rollback log execution module to execute the transaction rollback log file.
- Step 10 After the transaction rollback log execution module receives the transaction rollback log file, it parses the transactions sequentially, and executes the transaction rollback log file playback according to the strategy of paralleling the same batch of transactions and serializing different batches of transactions.
- Step 11 After all the transaction rollback log files are played back, the task of the specified transaction set rollback ends.
- databases that use the binlog logging protocol such as MariaDB can refer to this process to complete the transaction fast rollback, which will not be repeated here.
- the transaction rollback log is consistent with the native transaction log protocol, and transactions can be executed in parallel to operate reverse transactions and forward transactions.
- an embodiment of the present disclosure provides a transaction rollback device, as shown in FIG. 3.
- the transaction rollback device includes a native transaction rollback control module 302 and a transaction rollback log execution module 303.
- the native transaction rollback control module 302 is configured to parse native transaction log files within a preset time range when a transaction rollback request is received; determine the transaction to be rolled back, and based on the transaction corresponding to the transaction to be rolled back The native transaction log generates a transaction rollback log; the transaction rollback log is assembled in reverse order to generate a transaction rollback log file.
- the transaction rollback log execution module 303 is configured to execute the transaction rollback log file.
- the transaction rollback device also includes a native transaction log collection module 301, where the native transaction log collection module 301 is configured to record changes in transaction data and transactions between transactions.
- the relevance of each transaction; the global transaction identifier is assigned to each transaction, which includes the database instance identity identifier and the global sequence identifier; the same batch number is assigned to each transaction that does not have relevance; the native transaction log is generated; the native transaction log is assembled Generate native transaction log files.
- the database instance identity identifier in this embodiment refers to which database instance the current transaction is executed on
- the global sequence identifier refers to the identifier of the current transaction in a certain database. For example, the global sequence identifier starts from 1, and then from increase.
- the same batch number can be assigned to each transaction that does not have correlation, so that the subsequently generated transaction rollback log retains the original transaction log.
- Batch number that is, keep the concurrency relationship of each transaction execution in the native transaction log.
- the native transaction log generated in this embodiment includes not only the data changes before and after the data operation, but also the global transaction identifier and batch number.
- the native transaction log in this embodiment corresponds to a single transaction, so the native transaction log can be assembled in reverse order to generate a transaction rollback log file for easy storage.
- the native transaction log file includes a binary log binlog file.
- the native transaction rollback control module 302 performs filtering in the log directory according to the preset time range, obtains the native transaction log files to be parsed, and then parses these native transactions in parallel Log file, and locate the transaction to be rolled back according to the global transaction identifier, and determine the transaction to be rolled back.
- the native transaction rollback control module 302 performs reverse operations on the native transaction log corresponding to the transaction to be rolled back, and the protocol format of the transaction rollback log generated after the reverse operation remains the same as the protocol format of the native transaction log. the same.
- the native transaction log in this embodiment records table data before and after the execution of each statement operation, inserting and deleting records requires modifying the operation type, and updating records requires exchanging table data before and after changes. Therefore, the generated rollback log is consistent with the protocol format of the native transaction log.
- the reverse operation includes at least one of a reverse order operation statement and a reverse order execution statement.
- the protocol format of the transaction rollback log generated by the native transaction rollback control module 302 after performing the reverse operation remains the same as the protocol format of the native transaction log, and the transaction rollback log is allocated with the native transaction The log corresponds to the same global transaction ID.
- the native transaction rollback control module 302 sets the global transaction identifier of the transaction to be rolled back to identify that the current transaction is a transaction to be rolled back, and the global transaction identifier of the transaction to be rolled back should correspond to the global transaction identifier of the native transaction , Or there is a one-to-one mapping relationship between the two.
- the transaction rollback log in this embodiment corresponds to a single transaction to be rolled back, so the generated transaction rollback log can be assembled in reverse order according to the global sequence identifier to complete the reverse order of the transactions, and generate a transaction rollback log file .
- the transaction rollback log execution module 303 parses the batch number corresponding to each transaction in the transaction rollback log file, executes transactions with the same batch number in parallel, and performs transaction strings with different batch numbers. Line execution. In this way, the parallel playback of the transaction to be rolled back is realized, and the performance of the transaction rollback is improved to a certain extent.
- the transaction rollback device includes a native transaction rollback control module and a transaction log rollback.
- the native transaction rollback control module receives a transaction rollback request, it responds to native transaction log files within a preset time range. Perform analysis to determine the transaction to be rolled back, and then generate a transaction rollback log based on the native transaction log corresponding to the transaction to be rolled back. Further, the transaction rollback log is assembled in reverse order to generate a transaction rollback log file, which is executed by the transaction rollback log.
- the module executes the transaction rollback log file; it solves the problem that the transaction rollback method in the prior art cannot meet the high-performance requirements for multiple transactions and the rollback performance is low.
- the transaction rollback device not only reduces the re-analysis of the standard sentence file by the database, but also makes full use of the transaction concurrency information of the rollback log, which effectively improves the performance of transaction rollback, especially for batch transactions.
- the rollback effect is better.
- an embodiment of the present disclosure provides a system, which includes the above-mentioned third embodiment.
- Transaction rollback device In order to solve the problem that the transaction rollback method in the prior art cannot meet the high-performance requirements for multiple transactions and the low rollback performance, an embodiment of the present disclosure provides a system, which includes the above-mentioned third embodiment.
- Transaction rollback device In order to solve the problem that the transaction rollback method in the prior art cannot meet the high-performance requirements for multiple transactions and the low rollback performance, an embodiment of the present disclosure provides a system, which includes the above-mentioned third embodiment.
- this embodiment uses two more specific systems as examples for illustration.
- the MySQL database is widely used, and the applications based on the MySQL database are even more diverse. No matter its form is a single database node or a database cluster, in the process of providing data services, it must have the ability to quickly roll back data transactions, that is, quick undo The ability to batch transactions.
- the MySQL database itself uses the binlog protocol to record transaction logs, in which the transaction data table records changes and transaction execution information is relatively complete.
- a transaction rollback device is deployed in the system, where the rollback transaction device does not require a native transaction log collection module, and only needs to adapt the native transaction rollback control module 302 based on the binlog file And the transaction rollback log execution module 303.
- the native transaction rollback control module 302 when the native transaction rollback control module 302 receives the transaction rollback request, it analyzes the binlog file, locates the transaction to be rolled back, converts it to generate a transaction rollback log, and then assembles it in reverse order to generate a transaction rollback log file.
- the transaction rollback log execution module 303 executes the transaction rollback log file, connects to the MySQL instance, and applies the rollback binary log Undo binlog file to achieve rapid transaction rollback.
- the distributed system uses MariaDB as the basic data node to establish a data cluster.
- An unavoidable problem in the distributed system is the consistent recovery of cluster data.
- the data shards in each cluster are inconsistent, the data needs to be restored to the cluster consistent Location, that is, to restore each MariaDB data node to the specified transaction set state, and redundant transactions need to be rolled back quickly.
- MariaDB database is a branch of MySQL, and its transaction log protocol is consistent with MySQL, and it is also a binlog protocol.
- a transaction rollback device needs to be deployed on each data node, which is responsible for the transaction rollback of the corresponding node.
- the deployed transaction rollback There is no need for a native transaction log collection module in the rollover device. It only needs to adapt the native transaction rollback control module 302 and transaction rollback log execution module 303 based on the binlog file; at the same time, global data recovery management needs to be deployed in the system
- the module sends data recovery tasks to the native transaction rollback control module 302, and counts the recovery results of each data node.
- the native transaction rollback control module 302 when the native transaction rollback control module 302 receives the transaction rollback request, it analyzes the binlog file, locates the transaction to be rolled back, converts it to generate a transaction rollback log, and then assembles it in reverse order to generate a transaction rollback log file.
- the transaction rollback log execution module 303 executes the transaction rollback log file, connects to the MySQL instance, and applies the rollback log to achieve rapid transaction rollback.
- the transaction rollback log is consistent with the native transaction log protocol, and transactions can be executed in parallel to operate reverse transactions and forward transactions.
- the embodiments of the present disclosure also provide a computer storage medium.
- the computer storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement transactions as in the first and second embodiments above.
- the steps of the rollback method are described below.
- the computer storage medium includes volatile or non-volatile, removable or non-removable implemented in any method or technology for storing information (such as computer readable instructions, data structures, computer program modules, or other data) In addition to the medium.
- Computer storage media include but are not limited to RAM (Random Access Memory), ROM (Read-Only Memory, read-only memory), EEPROM (Electrically Erasable Programmable read only memory), Flash memory or other storage technology, CD-ROM (Compact Disc Read-Only Memory), digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, or Any other medium used to store desired information and that can be accessed by a computer.
- the functional modules/units in the system, and the device can be implemented as software (which can be implemented by program code executable by a computing device) , Firmware, hardware and their appropriate combination.
- the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may consist of several physical components. The components are executed cooperatively.
- Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit .
- the computer-readable medium may include computer storage Medium (or non-transitory medium) and communication medium (or temporary medium).
- computer storage medium includes volatile and non-volatile memory implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data).
- flexible, removable and non-removable media are examples of flexible, removable and non-removable media.
- communication media usually contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as carrier waves or other transmission mechanisms, and may include any information delivery media . Therefore, the present disclosure is not limited to any specific hardware and software combination.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims (13)
- 一种事务回滚方法,包括:A transaction rollback method, including:在接收到事务回滚请求时,对预设时间范围内的原生事务日志文件进行解析;When a transaction rollback request is received, the native transaction log file within the preset time range is parsed;确定待回滚事务,并根据所述待回滚事务对应的原生事务日志生成事务回滚日志;Determine the transaction to be rolled back, and generate a transaction rollback log according to the native transaction log corresponding to the transaction to be rolled back;倒序将所述事务回滚日志组装生成事务回滚日志文件,执行所述事务回滚日志文件。Reverse the assembly of the transaction rollback log to generate a transaction rollback log file, and execute the transaction rollback log file.
- 如权利要求1所述的事务回滚方法,其中,所述在接收到事务回滚请求时,对预设时间范围内的原生事务日志文件进行解析之前,所述方法还包括:3. The transaction rollback method according to claim 1, wherein when a transaction rollback request is received, before parsing native transaction log files within a preset time range, the method further comprises:记录各事务数据变化以及各事务之间的关联性;Record the data changes of each transaction and the correlation between each transaction;为所述各事务分配全局事务标识,所述全局事务标识包括数据库实例身份标识和全局序列标识;Allocating a global transaction identifier for each transaction, where the global transaction identifier includes a database instance identifier and a global sequence identifier;为不存在关联性的各事务分配相同的批次号;Assign the same batch number to each transaction that is not related;生成原生事务日志;Generate native transaction log;将所述原生事务日志组装生成原生事务日志文件。Assemble the native transaction log to generate a native transaction log file.
- 如权利要求2所述的事务回滚方法,其中,所述原生事务日志文件包括二进制日志binlog文件。2. The transaction rollback method of claim 2, wherein the native transaction log file comprises a binary log binlog file.
- 如权利要求2或3所述的事务回滚方法,其中,所述确定待回滚事务,并根据所述待回滚事务对应的原生事务日志生成事务回滚日志,包括:The transaction rollback method of claim 2 or 3, wherein the determining the transaction to be rolled back and generating the transaction rollback log according to the native transaction log corresponding to the transaction to be rolled back comprises:对所述待回滚事务对应的原生事务日志进行逆操作,进行逆操作后生成的事务回滚日志的协议格式保持与所述原生事务日志的协议格式相同。A reverse operation is performed on the native transaction log corresponding to the transaction to be rolled back, and the protocol format of the transaction rollback log generated after the reverse operation remains the same as the protocol format of the native transaction log.
- 如权利要求4所述的事务回滚方法,其中,所述逆操作包括:逆序操作语句和逆序执行语句中的至少一种。The transaction rollback method of claim 4, wherein the reverse operation comprises at least one of operating a statement in reverse order and executing a statement in reverse order.
- 如权利要求4所述的事务回滚方法,其中,所述对所述待回滚事务对应的原生事务日志进行逆操作,进行逆操作后生成的事务回滚日志的协议格式保持与所述原生事务日志的协议格式相同之后,所述方法还包括:The transaction rollback method of claim 4, wherein the reverse operation is performed on the native transaction log corresponding to the transaction to be rolled back, and the protocol format of the transaction rollback log generated after the reverse operation remains the same as that of the native transaction log. After the protocol format of the transaction log is the same, the method further includes:为事务回滚日志分配与所述原生事务日志对应的事务相同的全局事务标识。The transaction rollback log is assigned the same global transaction identifier as the transaction corresponding to the native transaction log.
- 如权利要求2或3所述的事务回滚方法,其中,所述执行所述事务回滚日志文件,包括:The transaction rollback method according to claim 2 or 3, wherein said executing said transaction rollback log file comprises:解析所述事务回滚日志文件中各事务对应的批次号,对批次号相同的事务并行执行,对批次号不相同的事务串行执行。The batch number corresponding to each transaction in the transaction rollback log file is analyzed, and the transactions with the same batch number are executed in parallel, and the transactions with different batch numbers are executed serially.
- 一种事务回滚装置,包括原生事务回滚控制模块及事务回滚日志执行模块;A transaction rollback device includes a native transaction rollback control module and a transaction rollback log execution module;所述原生事务回滚控制模块,配置为在接收到事务回滚请求时,对预设时间范围内的原生事务日志文件进行解析;确定待回滚事务,并根据所述待回滚事务对应的原生事务日志生成事务回滚日志;倒序将所述事务回滚日志组装生成事务回滚日志文件;The native transaction rollback control module is configured to, upon receiving a transaction rollback request, parse the native transaction log file within a preset time range; determine the transaction to be rolled back, and based on the transaction corresponding to the transaction to be rolled back The native transaction log generates a transaction rollback log; the transaction rollback log is assembled in reverse order to generate a transaction rollback log file;所述事务回滚日志执行模块,配置为执行所述事务回滚日志文件。The transaction rollback log execution module is configured to execute the transaction rollback log file.
- 如权利要求8所述的事务回滚装置,还包括原生事务日志采集模块;The transaction rollback device according to claim 8, further comprising a native transaction log collection module;所述原生事务日志采集模块,配置为记录各事务数据变化以及各事务之间的关联性;为所述各事务分配全局事务标识,所述全局事务标识包括数据库实例身份标识和全局序列标识;为不存在关联性的各事务分配相同的批次号;生成原生事务日志;将所述原生事务日志组装生成原生事务日志文件。The native transaction log collection module is configured to record data changes in each transaction and the correlation between each transaction; to assign a global transaction identifier to each transaction, and the global transaction identifier includes a database instance identifier and a global sequence identifier; All transactions that are not related are assigned the same batch number; a native transaction log is generated; the native transaction log is assembled to generate a native transaction log file.
- 如权利要求8所述的事务回滚装置,其中,所述事务回滚日志执行模块,还配置为解析所述事务回滚日志文件中各事务对应的批次号,对批次号相同的事务并行执行,对批次号不相同的事务串行执行。The transaction rollback device of claim 8, wherein the transaction rollback log execution module is further configured to parse the batch number corresponding to each transaction in the transaction rollback log file, and perform transactions with the same batch number. Parallel execution, serial execution of transactions with different batch numbers.
- 一种数据库,其中,所述数据库包括如权利要求8-10任一项所述的事务回滚装置。A database, wherein the database comprises the transaction rollback device according to any one of claims 8-10.
- 一种系统,包括如权利要求11所述的数据库,所述系统包括基于MySQL数据库的事务回滚系统,或,基于MariaDB分布式集群的数据恢复系统。A system, comprising the database according to claim 11, the system comprising a transaction rollback system based on MySQL database, or a data recovery system based on MariaDB distributed cluster.
- 一种计算机存储介质,其上存储有一个或者多个程序,所述一个或者多个程序可被一个或者多个处理器执行,以实现如权利要求1-7任一项所述的事务回滚方法的步骤。A computer storage medium having one or more programs stored thereon, and the one or more programs can be executed by one or more processors to implement the transaction rollback according to any one of claims 1-7 Method steps.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910662619.1 | 2019-07-22 | ||
CN201910662619.1A CN112256656B (en) | 2019-07-22 | 2019-07-22 | Transaction rollback method and device, database, system and computer storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021012868A1 true WO2021012868A1 (en) | 2021-01-28 |
Family
ID=74192444
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/098046 WO2021012868A1 (en) | 2019-07-22 | 2020-06-24 | Transaction rollback method and apparatus, database, system, and computer storage medium |
PCT/CN2020/100246 WO2021012932A1 (en) | 2019-07-22 | 2020-07-03 | Transaction rollback method and device, database, system, and computer storage medium |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/100246 WO2021012932A1 (en) | 2019-07-22 | 2020-07-03 | Transaction rollback method and device, database, system, and computer storage medium |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN112256656B (en) |
WO (2) | WO2021012868A1 (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112817798A (en) * | 2021-02-26 | 2021-05-18 | 北京车和家信息技术有限公司 | Data recovery method, device, medium and electronic equipment |
CN112988775B (en) * | 2021-04-15 | 2021-08-10 | 南京吉拉福网络科技有限公司 | Method, computing device and storage medium for processing batch transactions |
CN113254425B (en) * | 2021-06-24 | 2022-01-11 | 阿里云计算有限公司 | Method, apparatus, system, program and storage medium for database transaction retention |
CN113468196B (en) * | 2021-07-19 | 2024-04-05 | 京东科技控股股份有限公司 | Method, apparatus, system, server and medium for processing data |
CN113485998A (en) * | 2021-07-27 | 2021-10-08 | 中国银行股份有限公司 | Redis transaction processing method and device, electronic device and storage medium |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103312549A (en) * | 2013-06-26 | 2013-09-18 | 华为技术有限公司 | Transaction management method, device and system |
CN106547801A (en) * | 2015-09-23 | 2017-03-29 | 北京奇虎科技有限公司 | Database data flash back method and apparatus |
CN106682140A (en) * | 2016-12-20 | 2017-05-17 | 华北计算技术研究所(中国电子科技集团公司第十五研究所) | Multi-system user incremental synchronization method based on timestamps and mapping strategies |
CN107329966A (en) * | 2017-05-11 | 2017-11-07 | 昆仑智汇数据科技(北京)有限公司 | Machine data storage method and system |
US20190026193A1 (en) * | 2016-03-22 | 2019-01-24 | Alibaba Group Holding Limited | Method and apparatus for executing data recovery operation |
Family Cites Families (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7289992B2 (en) * | 2003-05-01 | 2007-10-30 | International Business Machines Corporation | Method, system, and program for lock and transaction management |
US8145686B2 (en) * | 2005-05-06 | 2012-03-27 | Microsoft Corporation | Maintenance of link level consistency between database and file system |
US20070288526A1 (en) * | 2006-06-08 | 2007-12-13 | Emc Corporation | Method and apparatus for processing a database replica |
CN101360309B (en) * | 2007-08-03 | 2011-06-08 | 华为技术有限公司 | Execution method, apparatus and system for service open command |
CN101464890B (en) * | 2008-12-30 | 2012-05-23 | 中兴通讯股份有限公司 | Mixed log generation method and resolution method |
CN102760161A (en) * | 2012-06-12 | 2012-10-31 | 天津神舟通用数据技术有限公司 | Log organization structure clustered based on transaction aggregation and method for realizing corresponding recovery protocol thereof |
US10089009B2 (en) * | 2013-04-30 | 2018-10-02 | Inuron | Method for layered storage of enterprise data |
CN104750755B (en) * | 2013-12-31 | 2018-06-15 | 阿里巴巴集团控股有限公司 | A kind of data covering method and system after database active-standby switch |
CN104866403B (en) * | 2015-05-13 | 2016-09-21 | 上海钢富电子商务有限公司 | Distributed task scheduling processing system and method |
US10303785B2 (en) * | 2015-08-10 | 2019-05-28 | International Business Machines Corporation | Optimizing online schema processing for busy database objects |
CN106557514A (en) * | 2015-09-29 | 2017-04-05 | 中兴通讯股份有限公司 | A kind of distributed transaction rollback method and device |
SE544816C2 (en) * | 2015-11-25 | 2022-11-29 | Teamifier Inc | Apparatuses for graphically representing a reconfigured portion of a directed acyclic graph as a hierarchical tree structure |
CN107241370A (en) * | 2016-03-29 | 2017-10-10 | 西门子公司 | The rollback method and system of resource operation under IaaS cloud environment |
CN106503264A (en) * | 2016-11-29 | 2017-03-15 | 中国银行股份有限公司 | On-line transaction log recording method and equipment based on oracle database |
CN110019050A (en) * | 2017-10-19 | 2019-07-16 | 中兴通讯股份有限公司 | A kind of data rewind method, system, equipment and computer readable storage medium |
CN109408294A (en) * | 2018-11-13 | 2019-03-01 | 郑州云海信息技术有限公司 | A kind of snapshot rollback method, device, equipment and storage medium |
-
2019
- 2019-07-22 CN CN201910662619.1A patent/CN112256656B/en active Active
-
2020
- 2020-06-24 WO PCT/CN2020/098046 patent/WO2021012868A1/en active Application Filing
- 2020-07-03 WO PCT/CN2020/100246 patent/WO2021012932A1/en active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103312549A (en) * | 2013-06-26 | 2013-09-18 | 华为技术有限公司 | Transaction management method, device and system |
CN106547801A (en) * | 2015-09-23 | 2017-03-29 | 北京奇虎科技有限公司 | Database data flash back method and apparatus |
US20190026193A1 (en) * | 2016-03-22 | 2019-01-24 | Alibaba Group Holding Limited | Method and apparatus for executing data recovery operation |
CN106682140A (en) * | 2016-12-20 | 2017-05-17 | 华北计算技术研究所(中国电子科技集团公司第十五研究所) | Multi-system user incremental synchronization method based on timestamps and mapping strategies |
CN107329966A (en) * | 2017-05-11 | 2017-11-07 | 昆仑智汇数据科技(北京)有限公司 | Machine data storage method and system |
Also Published As
Publication number | Publication date |
---|---|
CN112256656B (en) | 2023-04-18 |
CN112256656A (en) | 2021-01-22 |
WO2021012932A1 (en) | 2021-01-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2021012868A1 (en) | Transaction rollback method and apparatus, database, system, and computer storage medium | |
US11314701B2 (en) | Resharding method and system for a distributed storage system | |
CN110442560B (en) | Log replay method, device, server and storage medium | |
CN108241555B (en) | Backup and recovery method and device of distributed database and server | |
CN107391628B (en) | Data synchronization method and device | |
KR101476018B1 (en) | Buffer disk in flashcopy cascade | |
CN106933703B (en) | Database data backup method and device and electronic equipment | |
US8904225B2 (en) | Stream data processing failure recovery method and device | |
US10176049B2 (en) | Logging process in a data storage system | |
US20130117241A1 (en) | Shadow Paging Based Log Segment Directory | |
CN107665219B (en) | Log management method and device | |
CN105701190A (en) | Data synchronizing method and device | |
US10642530B2 (en) | Global occupancy aggregator for global garbage collection scheduling | |
CN112416654A (en) | Database log replay method, device, equipment and storage medium | |
CN112464044B (en) | File data block change information monitoring and management system and method thereof | |
US9020905B2 (en) | Synchronizing database and non-database resources without a commit coordinator | |
WO2021082925A1 (en) | Transaction processing method and apparatus | |
CN103207916A (en) | Metadata processing method and device | |
WO2017067397A1 (en) | Data recovery method and device | |
US20140149697A1 (en) | Memory Pre-Allocation For Cleanup and Rollback Operations | |
CN111125047B (en) | Cold and hot data catalog identification method and device | |
CN111026764B (en) | Data storage method and device, electronic product and storage medium | |
CN111541747A (en) | Data check point setting method and device | |
CN112559457A (en) | Data access method and device | |
CN109857523B (en) | Method and device for realizing high availability of database |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20844372 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20844372 Country of ref document: EP Kind code of ref document: A1 |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20844372 Country of ref document: EP Kind code of ref document: A1 |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 210922) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20844372 Country of ref document: EP Kind code of ref document: A1 |