CN112883045A - Database transaction splitting execution method and device - Google Patents
Database transaction splitting execution method and device Download PDFInfo
- Publication number
- CN112883045A CN112883045A CN202110347194.2A CN202110347194A CN112883045A CN 112883045 A CN112883045 A CN 112883045A CN 202110347194 A CN202110347194 A CN 202110347194A CN 112883045 A CN112883045 A CN 112883045A
- Authority
- CN
- China
- Prior art keywords
- transaction
- sub
- database
- splitting
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000012545 processing Methods 0.000 claims abstract description 21
- 238000004590 computer program Methods 0.000 claims description 16
- 238000003860 storage Methods 0.000 claims description 16
- 238000002955 isolation Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 12
- 230000000903 blocking effect Effects 0.000 description 6
- 238000013467 fragmentation Methods 0.000 description 6
- 238000006062 fragmentation reaction Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000002360 preparation method Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1873—Versioning file systems, temporal file systems, e.g. file system supporting different historic versions of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2308—Concurrency control
- G06F16/2336—Pessimistic concurrency control approaches, e.g. locking or multiple versions without time stamps
- G06F16/2343—Locking methods, e.g. distributed locking or locking implementation details
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a database transaction splitting and executing method and device, belongs to the technical field of cloud computing, and can be applied to the financial field and other fields. The database transaction splitting execution method comprises the following steps: splitting the data transaction according to the type of the data transaction to obtain each sub-transaction; comparing the basic version of the row to be modified of each sub-transaction with the corresponding current version of the database, and applying an exclusive lock to the row to be modified according to the comparison result; each sub-transaction is executed to modify the corresponding database data, and the exclusive lock is released after each sub-transaction is successfully executed. The invention can optimize and execute the database transaction in a splitting mode, thereby reducing the processing pressure of the database.
Description
Technical Field
The invention relates to the technical field of cloud computing, in particular to a method and a device for splitting and executing database transactions.
Background
At present, the business volume of large-scale banks is gradually increased along with the strategic layout of platforms under IOE, database localization and host business. Although the pressure can be reduced by binning the table, the database is still very stressful for processing a single large transaction. Large transactions can cause too much locked data, generate a large amount of blocking and lock overtime, have long rollback time and slow primary and secondary synchronization, cause large pressure on IO, CPU, memory cache and primary and secondary synchronization performance of a database server, and even cause the unavailability of the database in a short time because the primary and secondary databases are not synchronized timely and reselected.
Disclosure of Invention
The embodiment of the invention mainly aims to provide a method and a device for splitting and executing database transactions, which are used for performing optimized execution on the database transactions in a splitting mode and reducing the processing pressure of a database.
In order to achieve the above object, an embodiment of the present invention provides a database transaction splitting execution method, including:
splitting the data transaction according to the type of the data transaction to obtain each sub-transaction;
comparing the basic version of the row to be modified of each sub-transaction with the corresponding current version of the database, and applying an exclusive lock to the row to be modified according to the comparison result;
each sub-transaction is executed to modify the corresponding database data, and the exclusive lock is released after each sub-transaction is successfully executed.
An embodiment of the present invention further provides a database transaction splitting execution device, including:
the splitting module is used for splitting the data transaction according to the type of the data transaction to obtain each sub-transaction;
the exclusive lock module is used for comparing the basic version of the row to be modified of each sub-transaction with the corresponding current version of the database and applying an exclusive lock to the row to be modified according to the comparison result;
and the sub-transaction execution module is used for executing each sub-transaction to modify the corresponding database data and releasing the exclusive lock after each sub-transaction is successfully executed.
The embodiment of the invention also provides computer equipment which comprises a memory, a processor and a computer program which is stored on the memory and runs on the processor, wherein the steps of the database transaction splitting execution method are realized when the processor executes the computer program.
The embodiment of the present invention further provides a computer readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the database transaction splitting execution method are implemented.
The method and the device for splitting and executing the database transaction split of the embodiment of the invention split the data transaction into the sub-transactions according to the type of the data transaction, compare the basic version of the row to be modified of each sub-transaction with the corresponding current version of the database, apply the exclusive lock to the row to be modified according to the comparison result, execute each sub-transaction and release the exclusive lock after each sub-transaction is successfully executed, and can optimize and execute the database transaction in a splitting mode to reduce the processing pressure of the database.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
FIG. 1 is a flow chart of a method for split execution of a database transaction in an embodiment of the invention;
FIG. 2 is a flow chart of a method for split execution of a database transaction in another embodiment of the invention;
FIG. 3 is a schematic diagram of a table splitting decision in an embodiment of the present invention;
FIG. 4 is a flow chart of current limiting in an embodiment of the present invention;
FIG. 5 is a data version visibility principle diagram;
FIG. 6 is a diagrammatic illustration of the preparation phase prior to execution of a sub-transaction in an embodiment of the present invention;
FIG. 7 is a diagrammatic illustration of a commit phase in which a sub-transaction is performed in an embodiment of the present invention;
FIG. 8 is a schematic diagram of a rollback phase in an embodiment of the present invention;
FIG. 9 is a block diagram of a database transaction split execution apparatus according to an embodiment of the present invention;
FIG. 10 is a block diagram of a database transaction split execution apparatus according to another embodiment of the present invention;
fig. 11 is a block diagram of a computer device in the embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As will be appreciated by one skilled in the art, embodiments of the present invention may be embodied as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.
In view of the very high processing pressure of the database on a single large transaction, the embodiment of the invention provides a database transaction splitting execution method, which can perform optimized execution on the database transaction in a splitting manner, so as to reduce the processing pressure of the database. The split sub-transactions are executed by the transaction manager in a coordinated mode in the current limiting context, and the submission, rollback, isolation, timeout waiting and isolation and data consistency of the large transactions of each sub-transaction are guaranteed in the process.
FIG. 1 is a flow chart of a method for split execution of a database transaction according to an embodiment of the present invention. FIG. 2 is a flow chart of a method for performing database transaction splitting in another embodiment of the invention. As shown in fig. 1 and fig. 2, the database transaction split execution method includes:
s101: and splitting the data transaction according to the type of the data transaction to obtain each sub-transaction.
In one embodiment, S101 includes:
when the data transaction is a file, splitting the file into subfiles according to a preset slice line number, and generating sub-transactions according to the subfiles.
Combining with the general database use specification in the industry, the line number Row (DML operation) of a transaction change should be less than 10 thousands, so that the preset slice line number is 10 thousands, a file can be split by a splitter according to 10 thousands of lines and one subfile, each imported sub-transaction is generated according to each subfile, then the file interacts with a resource manager, the resource manager is requested to provide a global unique ID (thead _ ID), a transaction number is recorded in a sub-transaction object, and each sub-transaction is stored in a Blocking cache Queue Array Blocking Queue.
Where the resource manager provides a globally unique ID (current thread ID) through the linked database. Taking MySQL database as an example, MySQL stores a global variable Thread _ id _ counter, and every time a connection is newly created, Thread _ id _ counter is assigned to the Thread variable of the new connection. The size is 2^32-1, the global uniqueness is ensured, dirty reading problems can not occur even if the 2^32-1 is fully written and recycled, and the unique array thread _ ids, insert _ unique () realized by the MySQL bottom layer can ensure the following:
new_id＝thread_id_counter++；
}while(！thread_ids.insert_unique(new_id).second)。
FIG. 3 is a table splitting decision diagram according to an embodiment of the present invention. As shown in fig. 3, when the data transaction is a table, the table is divided into field intervals according to the metadata of the table, each field interval is divided into fields according to the number of lines of each field interval and the number of preset slicing lines, and each sub-transaction is generated according to each field.
When the method is implemented specifically, the resolver internally realizes an SQL (structured query language) syntax parser, parses SQL semantics and interacts with the resource manager, and requests the resource manager to provide metadata (including column names, indexes, main keys, data distribution conditions and the like) of the table. When the DML (Data management Language) operation for the Data transaction is Update, an incoming fragmentation policy may be received through a fragmentation policy interface provided externally, and the table is split according to the fragmentation policy. When the DML operation on the data transaction is Insert or Delete, not only can the incoming fragmentation policy be received through the externally provided fragmentation policy interface to split the table according to the fragmentation policy, but also the table can be sliced according to the metadata (for example, a main key, a unique index, and the like) of the table, the table is split into field intervals, each field interval is split into fields according to the line number of each field interval and the preset slicing line number, that is, each field interval is split by one field of 10 ten thousand lines; and finally, interacting with a resource manager, requesting the resource manager to provide a global unique ID (the _ ID), recording the transaction number into a sub-transaction object, and storing each sub-transaction into a Blocking cache Queue (current-limiting Queue).
The invention optimizes the large affairs in an automatic splitting mode. Large transactions (data transactions) are divided into files and tables: when the data transaction is a file, the file can be split into subfiles, and then each sub-transaction is generated according to each subfile and is imported; when the data transaction is a table (that is, the large transaction is a single large transaction executed by non-nested SQL), the metadata of the table can be obtained after the SQL is semantically identified, and the data transaction is split into a plurality of sub-transactions according to the main key, the unique index, the predicate condition, and the like. Splitting a large transaction into multiple sub-transactions can be done in a split-up manner, so that the purpose of optimizing the large transaction is achieved, and meanwhile, the freedom of optimizing transaction slices and the freedom of accepting trade-off between system availability and data consistency can be provided for a user.
Fig. 4 is a flow chart of current limiting in an embodiment of the present invention. As shown in fig. 4, storing each sub-transaction in the Blocking buffer Queue (throttling Queue) includes: and storing each sub-transaction into a blocking buffer queue according to the preset flow rate (the limited number of the sub-transactions) initialized by the current limiter, and enabling a user to configure whether the current limiting scene needs the Segment data consistency of the sub-transactions according to the parameters.
When segment data consistency is not required and the current limit queue is not empty, sub-transactions are taken out of the current limit queue one by the transaction manager to execute S102. Meanwhile, the transaction manager can poll the interval to scan whether the data in the current-limiting queue has space, and if the data in the current-limiting queue has space, the sub-transaction is put into the current-limiting queue.
When segmented data consistency is required and the current limiting queue is not empty, all sub-transactions in the current limiting queue are taken out through the transaction manager to execute S102 for isolation operation, and consistency and isolation of all sub-transactions obtained by splitting the data transaction are guaranteed through S102. And when the sub-transactions are executed completely, putting all sub-transactions obtained by splitting the next data transaction into a current-limiting queue until all sub-transactions are executed completely. When the preset flow rate is less than the number of all sub-transactions corresponding to one data transaction, no flow restrictor is adopted for realizing segmented data consistency.
Therefore, the current limiter can meet the consistency of large transaction data through user-defined requirements, including: strong consistency (scenario without current limiter), weak consistency (scenario with current limiter but satisfying segment consistency), and final consistency (scenario with current limiter but not satisfying segment consistency). The method not only provides current limiting, but also considers data consistency and isolation of the large transaction, and the user can select the degree of data consistency of the large transaction.
S102: and comparing the basic version of the row to be modified of each sub-transaction with the corresponding current version of the database, and applying an exclusive lock to the row to be modified according to the comparison result.
In one embodiment, applying an exclusive lock to the row to be modified based on the comparison comprises:
and when the basic version of the row to be modified of each sub-transaction is consistent with the current version of the corresponding database, applying an exclusive lock to the row to be modified, otherwise, performing rollback processing on each sub-transaction.
Fig. 5 is a data version visibility principle diagram. Referring to fig. 5, the isolation level of the database baselines for different business scenarios is different. When the isolation level is repeatable reads, the query only acknowledges data that has completed committing before the transaction starts. Taking MySQL as an example, isolation is implemented with MVCC version number control, and each data version has its own row _ trx _ id (base version). Therefore, before the transaction manager starts the sub-transaction work, the resource manager queries the row _ trx _ id of the row to be modified of each sub-transaction, then checks whether the current versions of the corresponding databases are consistent when the sub-transactions perform DML operation, if not, indicates that other submitted data are modified after the main transaction is started, and at this time, all the sub-transactions corresponding to the data transaction need to be rolled back, and then re-initiates the operation of querying the row _ trx _ id.
When the isolation level is read commit, the query only acknowledges data that has been committed to completion before the statement started. The Update operation (Update) is a current read that only reads the latest version that has been committed, and therefore isolation of data transactions is achieved without additional processing.
FIG. 6 is a diagrammatic illustration of the preparation phase prior to execution of a sub-transaction in an embodiment of the present invention. As shown in fig. 6, when the base version of the row to be modified of each sub-transaction is consistent with the corresponding current version of the database, the transaction manager sends the sub-transaction to be executed to the sub-transaction executor, and inquires whether the sub-transaction executor is ready to commit the transaction. The child transaction executor feeds back to the transaction manager after applying an exclusive lock to the line to be modified.
S103: each sub-transaction is executed to modify the corresponding database data, and the exclusive lock is released after each sub-transaction is successfully executed.
FIG. 7 is a diagrammatic representation of the commit phase of executing a sub-transaction in an embodiment of the present invention. As shown in fig. 7, when the transaction manager receives the successful execution instruction (yes) fed back by all the sub-transaction executors, a commit instruction is issued to the sub-transaction executors, and each sub-transaction executor releases the exclusive lock. When all exclusive lock releases are successful, each sub-transaction executor sends an execution success instruction to the transaction manager (YES). When the transaction manager receives the execution success instructions returned by all the sub-transaction executors, the transaction is closed and the log is recorded.
The execution subject of the database transaction split execution method shown in fig. 1 may be a computer. As can be seen from the process shown in fig. 1, the database transaction splitting execution method according to the embodiment of the present invention first splits a data transaction into sub-transactions according to the type of the data transaction, then compares the basic version of the row to be modified of each sub-transaction with the corresponding current version of the database, applies an exclusive lock to the row to be modified according to the comparison result, and finally executes each sub-transaction and releases the exclusive lock after each sub-transaction is successfully executed.
In one embodiment, the method further comprises: and when one of the sub-transactions fails to execute, performing rollback processing on each sub-transaction according to the failure state and releasing the exclusive lock after the rollback processing.
Fig. 8 is a schematic diagram of a rollback phase in an embodiment of the present invention. As shown in FIG. 8, when one of the sub-transaction managers fails to commit, it is confirmed whether a retry is required according to the state machine reaction. If the state machine reacts as having insufficient authority or there is a definite data error problem, no retry is needed to avoid wasting overhead. If the state machine reflects a network connection timeout, a reasonable number of retries is set to retry the sub-transaction. If the retry fails, the transaction manager returns NO and initiates a RollBack (RollBack) operation on all the sub-transactions corresponding to the data transaction. And when the sub-transaction executor returns False (error) during rollback, retrying is stopped, the state of the large transaction is recorded as an abnormal state, the field is recorded, the large transaction is stopped, and the transaction is closed-loop.
In summary, in order to solve the problem of database processing bottleneck caused by single-machine large transaction, the invention provides a database transaction splitting and executing method, which can swiftly and effectively split a large transaction (data transaction) into a plurality of sub-transactions, ensure the global uniqueness of the transaction by using a database thread ID, skillfully realize data isolation by using a version control method, and finally realize consistency and complete current limiting and degradation by using a current limiter, and has the following beneficial effects:
1. the transaction characteristics of the large transaction are completely guaranteed, creativity is combined with the coordination of the sub-transactions, and the data consistency and isolation of the large transaction are realized.
2. The method is light and agile to realize the splitting of large transactions.
3. A slicing active strategy is provided, and a space for transaction slicing efficiency optimization exists.
4. The current limiter and the linkage operation thereof are provided, the data consistency and the isolation of the large transaction are considered, a user can decide the consistency of the large transaction data by himself or can adjust the free choice of the system availability and the data consistency through the interface of the current limiting queue.
Based on the same inventive concept, the embodiment of the invention also provides a database transaction splitting and executing device, and as the principle of solving the problems of the device is similar to the database transaction splitting and executing method, the implementation of the device can refer to the implementation of the method, and repeated parts are not described again.
Fig. 9 is a block diagram of a database transaction split execution device in an embodiment of the present invention. Fig. 10 is a block diagram of a database transaction split execution apparatus according to another embodiment of the present invention. As shown in fig. 9-10, the database transaction split execution device includes:
the splitting module is used for splitting the data transaction according to the type of the data transaction to obtain each sub-transaction;
the exclusive lock module is used for comparing the basic version of the row to be modified of each sub-transaction with the corresponding current version of the database and applying an exclusive lock to the row to be modified according to the comparison result;
and the sub-transaction execution module is used for executing each sub-transaction to modify the corresponding database data and releasing the exclusive lock after each sub-transaction is successfully executed.
In one embodiment, the splitting module comprises:
the file splitting module is used for splitting the file into subfiles according to a preset slice line number and generating sub-transactions according to the subfiles when the data transaction is the file;
and the table splitting module is used for splitting the table into field intervals according to the metadata of the table when the data transaction is the table, splitting the field intervals into fields according to the line number of the field intervals and the preset slicing line number, and generating sub-transactions according to the fields.
In one embodiment, the exclusive lock module is specifically configured to:
and when the basic version of the row to be modified of each sub-transaction is consistent with the current version of the corresponding database, applying an exclusive lock to the row to be modified, otherwise, performing rollback processing on each sub-transaction.
In one embodiment, the method further comprises the following steps:
and the rollback module is used for performing rollback processing on each sub-transaction according to the failure state and releasing the exclusive lock after the rollback processing when one of the sub-transactions fails to execute.
As shown in fig. 10, in practical applications, the database transaction splitting execution apparatus includes: the system comprises a splitter, a resource manager, a transaction manager and a plurality of sub-transaction executors; the transaction manager includes a sub-transaction flow restrictor.
The splitter includes a splitting module to identify data transactions, split sub-transactions, and assign globally unique IDs to sub-transactions. Splitting a large transaction into multiple sub-transactions requires the following characteristics to be satisfied: large transactions are teardown, children can meet the basic characteristics of the transaction, and the split must tightly fit the database development specification.
The resource manager is used for realizing proxy for the database connection pool, acquiring metadata of the database, metadata of the table and the global uniqueness ID of the sub-transaction, and providing database connection service.
The transaction manager comprises an exclusive lock module and a rollback module and is responsible for the global management of the sub-transactions, including the life cycle management of the sub-transactions, the allocation coordination of resources and the like.
The current limiter provides an interface for a user to realize a current limiting queue by himself, and the freedom of the user for flexibly accepting or rejecting data consistency or system availability is met. The user can use the method in a scene without strong consistency of data to relieve system pressure and degrade the consistency of data from strong consistency to final consistency. The purpose of the current limiter is on the one hand to guarantee the response time and availability of large transaction components at a certain speed and on the other hand to reduce the pressure on the whole system by clipping peaks. Therefore, in the distributed scene of a large transaction component, if a current limiter is needed, data consistency needs to be sacrificed, and the data consistency is degraded from strong consistency to final consistency, and the sacrifice degree can be defined by an interface for realizing the current limiting queue by a user.
The current limiter is coupled to the transaction manager, which relies on the current limiter. The transaction manager implements multiple simultaneous sub-transactions according to the flow rate policy configured by the flow restrictor. The current limiter provides parameter configuration to distinguish whether the current limiting scene needs the Segment data consistency of the sub-transaction, and the specific flow is as follows:
1. the flow restrictor initializes and informs the transaction manager of the flow rate (defined number of sub-transactions).
2. The transaction manager realizes that a plurality of sub-transactions simultaneously perform transaction operation according to the flow rate, and writes the sub-transactions into the current limiting queue of the current limiter.
3. The transaction manager reads the sub-transactions of the current limiting queue to perform the transaction operation of the sub-transactions, and a user can distinguish whether the current limiting scene needs the Segment data consistency of the sub-transactions by parameter configuration.
The sub-transaction executor comprises a sub-transaction execution module which is responsible for executing tasks of sub-transactions and realizes functions of pre-submission, submission and rollback.
To sum up, the database transaction splitting execution device according to the embodiment of the present invention first splits a data transaction into sub-transactions according to the type of the data transaction, then compares the basic version of the row to be modified of each sub-transaction with the corresponding current version of the database, applies an exclusive lock to the row to be modified according to the comparison result, and finally executes each sub-transaction and releases the exclusive lock after each sub-transaction is successfully executed.
The embodiment of the present invention further provides a specific implementation manner of a computer device, which can implement all the steps in the database transaction splitting execution method in the foregoing embodiment. Fig. 11 is a block diagram of a computer device in an embodiment of the present invention, and referring to fig. 11, the computer device specifically includes the following contents:
a processor (processor)1101 and a memory (memory) 1102.
The processor 1101 is configured to call a computer program in the memory 1102, and the processor executes the computer program to implement all the steps in the database transaction splitting execution method in the foregoing embodiments, for example, when executing the computer program, the processor implements the following steps:
splitting the data transaction according to the type of the data transaction to obtain each sub-transaction;
comparing the basic version of the row to be modified of each sub-transaction with the corresponding current version of the database, and applying an exclusive lock to the row to be modified according to the comparison result;
each sub-transaction is executed to modify the corresponding database data, and the exclusive lock is released after each sub-transaction is successfully executed.
To sum up, the computer device according to the embodiment of the present invention first splits the data transaction into sub-transactions according to the type of the data transaction, then compares the basic version of the row to be modified of each sub-transaction with the corresponding current version of the database, applies an exclusive lock to the row to be modified according to the comparison result, finally executes each sub-transaction and releases the exclusive lock after each sub-transaction is successfully executed, and can perform optimized execution on the database transaction in a split manner, thereby reducing the processing pressure of the database.
An embodiment of the present invention further provides a computer-readable storage medium capable of implementing all the steps in the database transaction splitting execution method in the foregoing embodiment, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements all the steps of the database transaction splitting execution method in the foregoing embodiment, for example, when the processor executes the computer program, the processor implements the following steps:
splitting the data transaction according to the type of the data transaction to obtain each sub-transaction;
comparing the basic version of the row to be modified of each sub-transaction with the corresponding current version of the database, and applying an exclusive lock to the row to be modified according to the comparison result;
each sub-transaction is executed to modify the corresponding database data, and the exclusive lock is released after each sub-transaction is successfully executed.
To sum up, the computer-readable storage medium according to the embodiment of the present invention first splits a data transaction into sub-transactions according to the type of the data transaction, then compares the basic version of the row to be modified of each sub-transaction with the corresponding current version of the database, applies an exclusive lock to the row to be modified according to the comparison result, and finally executes each sub-transaction and releases the exclusive lock after each sub-transaction is successfully executed.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Those of skill in the art will further appreciate that the various illustrative logical blocks, units, and steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate the interchangeability of hardware and software, various illustrative components, elements, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design requirements of the system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.
The various illustrative logical blocks, or elements, or devices described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor, an Application Specific Integrated Circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. For example, a storage medium may be coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC, which may be located in a user terminal. In the alternative, the processor and the storage medium may reside in different components in a user terminal.
In one or more exemplary designs, the functions described above in connection with the embodiments of the invention may be implemented in hardware, software, firmware, or any combination of the three. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media that facilitate transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a general purpose or special purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store program code in the form of instructions or data structures and which can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Additionally, any connection is properly termed a computer-readable medium, and, thus, is included if the software is transmitted from a website, server, or other remote source via a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wirelessly, e.g., infrared, radio, and microwave. Such discs (disk) and disks (disc) include compact disks, laser disks, optical disks, DVDs, floppy disks and blu-ray disks where disks usually reproduce data magnetically, while disks usually reproduce data optically with lasers. Combinations of the above may also be included in the computer-readable medium.
Claims (10)
1. A database transaction split execution method is characterized by comprising the following steps:
splitting the data transaction according to the type of the data transaction to obtain each sub-transaction;
comparing the basic version of the row to be modified of each sub-transaction with the corresponding current version of the database, and applying an exclusive lock to the row to be modified according to the comparison result;
each sub-transaction is executed to modify the corresponding database data, and the exclusive lock is released after each sub-transaction is successfully executed.
2. The database transaction splitting execution method of claim 1, wherein splitting the data transaction according to the type of the data transaction to obtain each sub-transaction comprises:
when the data transaction is a file, splitting the file into subfiles according to a preset slice line number, and generating sub-transactions according to the subfiles;
when the data transaction is a table, splitting the table into field intervals according to metadata of the table, splitting the field intervals into fields according to the line number of the field intervals and the preset slicing line number, and generating sub-transactions according to the fields.
3. The database transaction split execution method of claim 1, wherein applying an exclusive lock to the row to be modified according to the comparison result comprises:
and when the basic version of the row to be modified of each sub-transaction is consistent with the current version of the corresponding database, applying an exclusive lock to the row to be modified, otherwise, performing rollback processing on each sub-transaction.
4. The database transaction split execution method according to claim 1, further comprising:
and when one of the sub-transactions fails to execute, performing rollback processing on each sub-transaction according to the failure state and releasing the exclusive lock after the rollback processing.
5. A database transaction split execution apparatus, comprising:
the splitting module is used for splitting the data transaction according to the type of the data transaction to obtain each sub-transaction;
the exclusive lock module is used for comparing the basic version of the row to be modified of each sub-transaction with the corresponding current version of the database and applying an exclusive lock to the row to be modified according to the comparison result;
and the sub-transaction execution module is used for executing each sub-transaction to modify the corresponding database data, and releasing the exclusive lock after each sub-transaction is successfully executed.
6. The database transaction split execution apparatus of claim 5, wherein the splitting module comprises:
the file splitting module is used for splitting the file into subfiles according to a preset slice line number and generating sub-transactions according to the subfiles when the data transaction is the file;
and the table splitting module is used for splitting the table into field intervals according to the metadata of the table when the data transaction is the table, splitting the field intervals into fields according to the line number of the field intervals and the preset slicing line number, and generating sub-transactions according to the fields.
7. The database transaction split execution apparatus of claim 5, wherein the exclusive lock module is specifically configured to:
and when the basic version of the row to be modified of each sub-transaction is consistent with the current version of the corresponding database, applying an exclusive lock to the row to be modified, otherwise, performing rollback processing on each sub-transaction.
8. The database transaction split execution apparatus according to claim 5, further comprising:
and the rollback module is used for performing rollback processing on each sub-transaction according to the failure state and releasing the exclusive lock after the rollback processing when one of the sub-transactions fails to execute.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and running on the processor, wherein the steps of the database transaction splitting execution method according to any of claims 1 to 4 are implemented when the computer program is executed by the processor.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the database transaction split execution method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110347194.2A CN112883045B (en) | 2021-03-31 | 2021-03-31 | Database transaction splitting execution method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110347194.2A CN112883045B (en) | 2021-03-31 | 2021-03-31 | Database transaction splitting execution method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112883045A true CN112883045A (en) | 2021-06-01 |
| CN112883045B CN112883045B (en) | 2024-05-17 |
Family
ID=76040787
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110347194.2A Active CN112883045B (en) | 2021-03-31 | 2021-03-31 | Database transaction splitting execution method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112883045B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115563086A (en) * | 2022-11-10 | 2023-01-03 | 云和恩墨(北京)信息技术有限公司 | Data modification method and device for database |
| CN117076147A (en) * | 2023-10-13 | 2023-11-17 | 支付宝(杭州)信息技术有限公司 | Deadlock detection method, device, equipment and storage medium |
| CN117667319A (en) * | 2024-02-02 | 2024-03-08 | 建信金融科技有限责任公司 | Transaction processing method and device |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102508834A (en) * | 2011-09-22 | 2012-06-20 | 用友软件股份有限公司 | Message-based event application device and message-based event application method |
| US20130110766A1 (en) * | 2009-10-13 | 2013-05-02 | Open Text Software Gmbh | Method for performing transactions on data and a transactional database |
| US20130290243A1 (en) * | 2012-04-26 | 2013-10-31 | Cloudtree, Inc. | Method and system for transaction representation in append-only datastores |
| CN103514028A (en) * | 2012-06-14 | 2014-01-15 | 北京新媒传信科技有限公司 | Method and device for processing distributed transactions |
| CN104317850A (en) * | 2014-10-14 | 2015-01-28 | 北京国双科技有限公司 | Data processing method and device |
| CN106445644A (en) * | 2016-08-30 | 2017-02-22 | 中国民生银行股份有限公司 | Distributed transaction processing method and device based on improved one-phase commit |
| CN108205464A (en) * | 2016-12-20 | 2018-06-26 | 阿里巴巴集团控股有限公司 | A kind of processing method of database deadlocks, device and Database Systems |
| CN108363806A (en) * | 2018-03-01 | 2018-08-03 | 上海达梦数据库有限公司 | Multi-version concurrency control method, device, server and the storage medium of database |
| CN110196856A (en) * | 2019-05-09 | 2019-09-03 | 腾讯科技（深圳）有限公司 | A kind of distributed data read method and device |
| CN110941446A (en) * | 2019-11-06 | 2020-03-31 | 苏宁云计算有限公司 | Version release method and device based on multi-environment offline task |
| CN111259071A (en) * | 2020-01-04 | 2020-06-09 | 浙江科技学院 | Concurrent access control method in distributed database system |
-
2021
- 2021-03-31 CN CN202110347194.2A patent/CN112883045B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130110766A1 (en) * | 2009-10-13 | 2013-05-02 | Open Text Software Gmbh | Method for performing transactions on data and a transactional database |
| CN102508834A (en) * | 2011-09-22 | 2012-06-20 | 用友软件股份有限公司 | Message-based event application device and message-based event application method |
| US20130290243A1 (en) * | 2012-04-26 | 2013-10-31 | Cloudtree, Inc. | Method and system for transaction representation in append-only datastores |
| CN103514028A (en) * | 2012-06-14 | 2014-01-15 | 北京新媒传信科技有限公司 | Method and device for processing distributed transactions |
| CN104317850A (en) * | 2014-10-14 | 2015-01-28 | 北京国双科技有限公司 | Data processing method and device |
| CN106445644A (en) * | 2016-08-30 | 2017-02-22 | 中国民生银行股份有限公司 | Distributed transaction processing method and device based on improved one-phase commit |
| CN108205464A (en) * | 2016-12-20 | 2018-06-26 | 阿里巴巴集团控股有限公司 | A kind of processing method of database deadlocks, device and Database Systems |
| CN108363806A (en) * | 2018-03-01 | 2018-08-03 | 上海达梦数据库有限公司 | Multi-version concurrency control method, device, server and the storage medium of database |
| CN110196856A (en) * | 2019-05-09 | 2019-09-03 | 腾讯科技（深圳）有限公司 | A kind of distributed data read method and device |
| CN110941446A (en) * | 2019-11-06 | 2020-03-31 | 苏宁云计算有限公司 | Version release method and device based on multi-environment offline task |
| CN111259071A (en) * | 2020-01-04 | 2020-06-09 | 浙江科技学院 | Concurrent access control method in distributed database system |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115563086A (en) * | 2022-11-10 | 2023-01-03 | 云和恩墨(北京)信息技术有限公司 | Data modification method and device for database |
| CN117076147A (en) * | 2023-10-13 | 2023-11-17 | 支付宝(杭州)信息技术有限公司 | Deadlock detection method, device, equipment and storage medium |
| CN117076147B (en) * | 2023-10-13 | 2024-04-16 | 支付宝(杭州)信息技术有限公司 | Deadlock detection method, device, equipment and storage medium |
| CN117667319A (en) * | 2024-02-02 | 2024-03-08 | 建信金融科技有限责任公司 | Transaction processing method and device |
| CN117667319B (en) * | 2024-02-02 | 2024-05-03 | 建信金融科技有限责任公司 | Transaction processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112883045B (en) | 2024-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112883045A (en) | Database transaction splitting execution method and device | |
| US10977227B2 (en) | Dynamic snapshot isolation protocol selection | |
| US20120284228A1 (en) | User-Defined Parallelization in Transactional Replication of In-Memory Database | |
| US20150326673A1 (en) | Transactional and non-transactional data for maintaining session state | |
| US9990224B2 (en) | Relaxing transaction serializability with statement-based data replication | |
| US20150205850A1 (en) | Eager replication of uncommitted transactions | |
| US11928089B2 (en) | Data processing method and device for distributed database, storage medium, and electronic device | |
| US11899648B2 (en) | Concurrency control for transactions in database systems | |
| US10191663B1 (en) | Using data store accelerator intermediary nodes and write control settings to identify write propagation nodes | |
| US10678775B2 (en) | Determining integrity of database workload transactions | |
| US9785695B2 (en) | Providing multiple concurrent transactions on a single database schema using a single concurrent transaction database infrastructure | |
| EP4170509A1 (en) | Method for playing back log on data node, data node, and system | |
| CN113032421A (en) | MongoDB-based distributed transaction processing system and method | |
| US11921708B1 (en) | Distributed execution of transactional queries | |
| US11960369B2 (en) | Efficient creation of a secondary database system | |
| US20230325404A1 (en) | Query execution on a replicated database with most recent data | |
| US11855910B2 (en) | Configuration management of cloud resources for multiple providers and frameworks | |
| CN114003203B (en) | Maintenance method, device and equipment for activity counting variable and readable medium | |
| CN109857523B (en) | Method and device for realizing high availability of database | |
| CN112835887A (en) | Database management method, database management device, computing equipment and storage medium | |
| WO2024109415A1 (en) | Database redistribution method and system, and device cluster and storage medium | |
| US11995102B2 (en) | Bidirectional data replication with wait for data function | |
| US12007990B1 (en) | Deferred constraints support in distributed database systems | |
| US20240176775A1 (en) | Datastore workload isolation | |
| US20240134880A1 (en) | Data visibility for nested transactions in distributed systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |