CN112631741A - Transaction processing method, device and storage medium - Google Patents
Transaction processing method, device and storage medium Download PDFInfo
- Publication number
- CN112631741A CN112631741A CN202011449010.5A CN202011449010A CN112631741A CN 112631741 A CN112631741 A CN 112631741A CN 202011449010 A CN202011449010 A CN 202011449010A CN 112631741 A CN112631741 A CN 112631741A
- Authority
- CN
- China
- Prior art keywords
- cache
- data
- database
- transaction
- processing method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 42
- 238000000034 method Methods 0.000 claims abstract description 27
- 238000012217 deletion Methods 0.000 claims description 14
- 230000037430 deletion Effects 0.000 claims description 14
- 230000002159 abnormal effect Effects 0.000 claims description 12
- 230000004048 modification Effects 0.000 claims description 10
- 238000012986 modification Methods 0.000 claims description 10
- 230000002085 persistent effect Effects 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 5
- 238000012544 monitoring process Methods 0.000 claims description 4
- 238000005096 rolling process Methods 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims 1
- 238000012545 processing Methods 0.000 abstract description 24
- 230000008569 process Effects 0.000 abstract description 15
- 230000006870 function Effects 0.000 description 7
- 230000007246 mechanism Effects 0.000 description 6
- 238000013461 design Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000005856 abnormality Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000000977 initiatory effect Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/466—Transaction processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2477—Temporal data queries
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Computer Security & Cryptography (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a transaction processing method, transaction processing equipment and a storage medium. In the invention, in the process of normally executing the operation on the database in the transaction request, the operation aiming at the cache is correspondingly generated according to the operation on the database executed between the beginning and the end of the transaction, the generated operation aiming at the cache is recorded, and after all the operations on the database in the transaction request are completed, the cache is synchronously processed according to the recorded operation aiming at the cache.
Description
Technical Field
The embodiment of the invention relates to the technical field of data storage, in particular to a transaction processing method, transaction processing equipment and a storage medium.
Background
With the great growth of the mobile internet, various internet technologies are in endless, but a bank which can never be bypassed no matter how long the various technical frameworks are, namely the database performance. At present, in order to have a multiple increase in performance, a cache design is introduced, for example, data (hereinafter referred to as business data) supporting business system applications is all stored in a database, and the business data which is not large in data volume and is often used is stored as cache data in a cache, such as a local cache (e.g., a java heap memory cache) and a distributed cache (e.g., a Remote directory Server (Redis)), so that the performance is greatly improved under the condition of considering the cost.
However, since the buffered data temporarily stored in the buffer is only a copy, in order to keep the normal operation of the service, it is necessary to ensure the consistency between the buffered data stored in the buffer and the service data stored in the database. However, in the current data caching scheme, the transaction relationship between the service data and the cached data is not considered at all, so that dirty data or lost data in the cache, that is, the cached data stored in the cache is inconsistent with the service data stored in the database, often occurs due to some operation exception in the transaction or a bug of the service logic code itself.
Disclosure of Invention
Embodiments of the present invention provide a transaction processing method, a transaction processing device, and a storage medium, which are used to ensure consistency between cache data stored in a cache and service data stored in a database.
To solve the above technical problem, an embodiment of the present invention provides a transaction processing method, including the following steps: receiving a transaction request, and normally executing the operation on the database in the transaction request; correspondingly generating cache operation according to the database operation executed between the beginning and the end of the transaction, and recording the cache operation; and after all the operations on the database are completed, executing the operations on the cache in the record.
An embodiment of the present invention further provides a transaction processing device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a transaction processing method as described above.
Embodiments of the present invention also provide a computer-readable storage medium storing a computer program which, when executed by a processor, implements a transaction processing method as described above.
According to the transaction processing method, the transaction processing device and the storage medium provided by the embodiment of the invention, in the process of normally executing the operation on the database in the transaction request, the operation aiming at the cache is correspondingly generated according to the operation on the database executed between the beginning and the end of the transaction, the generated operation aiming at the cache is recorded, and after all the operations on the database in the transaction request are completed, the cache is synchronously processed according to the recorded operation aiming at the cache.
In addition, in the process of normally executing the operation on the database in the transaction request, the method further comprises the following steps: monitoring whether the operation on the database is abnormal or not; and if the exception occurs, rolling back the transaction. According to the embodiment of the invention, by introducing the rollback mechanism, when the operation of the database is abnormal, the state before the transaction request is received is recovered by performing transaction rollback, namely directly finishing the operation, so that the problem of data inconsistency between the database and the cache due to the abnormality is avoided.
In addition, in the process of executing the operation on the cache in the record, the method further includes: monitoring whether the operation on the cache is abnormal or not; and if the exception occurs, performing the transaction. According to the embodiment of the invention, by introducing the rollback mechanism, when the operation on the cache is abnormal, the database and the cache are restored to the state before receiving the transaction request by performing transaction rollback, namely directly finishing the operation, so that the problem of data inconsistency in the database and the cache due to the abnormality is avoided.
In addition, during the normal execution of the operation on the database in the transaction request, the method further includes: if the operation on the database is a query operation; responding to the query operation based on the cache. According to the embodiment of the invention, when the query operation aiming at the database is performed, the query operation on the data is converted into the query operation on the cache, so that the quick response to the transaction request is realized, and the user experience is further improved.
In addition, the cache comprises a temporary cache area; the responding to the query operation based on the cache comprises: determining whether the data of the query operation request is contained in the temporary cache region; if yes, directly responding to the query request based on the data in the temporary cache region; and if not, acquiring the data of the query operation request from the database and caching the data into the temporary cache region. In the embodiment of the invention, the query operation is performed on the temporary cache area in the cache.
In addition, the cache comprises a temporary cache region and a permanent cache region; the correspondingly generating the cache operation according to the database operation executed between the beginning and the end of the transaction, and recording the cache operation, includes: when the operation on the database is newly added data, the newly added data operation on the permanent cache is correspondingly generated; when the operation on the database is data deletion, data deletion operations on the permanent cache and the temporary cache are correspondingly generated; and when the operation on the database is data modification, correspondingly generating a data deletion operation on the temporary cache and a data modification operation on the permanent cache. According to the embodiment of the invention, the cache is divided into the temporary buffer area and the permanent buffer area, so that when the operation on the cache is correspondingly generated, the operation on the temporary buffer area and/or the permanent buffer area can be correspondingly generated according to different operations carried out on the database, and the consistency of the data recorded in the database and the cache after the submitted transaction is finished is ensured.
In addition, when the operation of deleting data from the temporary cache area is performed, the method includes: and searching a data record corresponding to the storage information from the permanent cache region based on the storage information of the data to be deleted, determining the corresponding data to be deleted in the temporary cache region based on the data record, and deleting the data. The embodiment of the invention provides a specific processing mode for deleting data aiming at a temporary cache region.
In addition, the temporary cache region is used for responding to the query operation in the transaction request, and if the data of the query operation request is not contained in the temporary cache region, the data of the query operation request is obtained from the database and cached. The embodiment of the invention provides a specific processing mode of a temporary cache region aiming at query operation.
In addition, the data cached in the permanent cache region is data directly acquired from the database based on a preset caching range.
In addition, a cache range is set for the cache in advance, before the cache operation is correspondingly generated according to the operation on the database executed between the beginning and the end of the transaction, whether the data corresponding to the operation on the database is in the cache range is determined, and if yes, a record on the cache is correspondingly generated; if not, no record is generated for the cache.
In addition, the buffer range contains all records in the database or part of records in the database.
In addition, in an embodiment of the present invention, the transaction processing method further includes: after the step of executing the operation on the cache in the record is completed, committing the transaction to end.
Drawings
One or more embodiments are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings in which elements having the same reference numeral designations represent like elements and in which the figures are not to scale unless specifically stated.
Fig. 1 is a detailed flowchart of a transaction processing method according to a first embodiment of the present invention;
FIG. 2 is a detailed flow chart of a transaction processing method according to a second embodiment of the invention;
FIG. 3 is an interaction diagram of a transaction processing method according to a second embodiment of the present invention;
FIG. 4 is a detailed flowchart of a transaction processing method according to a third embodiment of the present invention;
fig. 5 is a schematic structural diagram of a transaction processing device according to a fourth embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, it will be appreciated by those of ordinary skill in the art that in various embodiments of the invention, numerous technical details are set forth in order to provide a better understanding of the present application. However, the technical solution claimed in the present application can be implemented without these technical details and various changes and modifications based on the following embodiments. The following embodiments are divided for convenience of description, and should not constitute any limitation to the specific implementation manner of the present invention, and the embodiments may be mutually incorporated and referred to without contradiction.
The first embodiment relates to a transaction processing method, which is applied to a transaction processing device, and in practical application, the transaction processing device is, for example, a client or a server with a storage space.
Regarding the specific choice of the client or the server, those skilled in the art can select the client or the server as needed, and the embodiment does not limit this.
The following describes implementation details of the transaction processing method of the present embodiment, and the following description is provided only for easy understanding and is not necessary for implementing the present embodiment.
The specific flow of this embodiment is shown in fig. 1, and specifically includes the following steps:
Specifically, a transaction generally refers to what is to be done or what is to be done. In computer terminology, refers to a program execution unit that accesses and possibly updates various data items in a database. A transaction is typically caused by the execution of a user program written in a high level database manipulation language or programming language (e.g., SQL, C + +, or Java), and is tagged with specific tags and then annotated with a programming language that is appropriate for the current framework or programming language, i.e., in a particular application, a transaction is made up of the totality of operations performed between the beginning of the transaction and the end of the transaction.
Taking the adopted software development framework as a Spring framework as an example, a Transaction is usually labeled by using a "@ Transaction" tag, a beginning of the Transaction is identified by a call of a "begin Transaction" statement or function, and an end Transaction is identified by a call of an "end Transaction" statement or function.
In addition, in this embodiment, all the transactions are transactions at a service level, that is, service operations composed of multiple database related operations are bound. Therefore, the transaction request essentially carries information corresponding to the operations related to the plurality of databases.
It can be understood that the information corresponding to the operations related to the databases may be, in practical applications, query operations, delete operations, add operations, and modify operations on the databases.
For example, for any received transaction request, the following may be included: firstly, modifying data 'abc' with the field of name in the table A into 'bcd'; deleting afg data with the field of name in the table A; ③ insert the contents of the data "11" with the field "age" into the table B.
Correspondingly, the operations needed to be carried out on the database according to the service request comprise data modification operation determined on the basis of the first step, data deletion operation determined on the basis of the second step and data addition operation determined on the basis of the third step.
It should be understood that the above examples are only examples for better understanding of the technical solution of the present embodiment, and are not to be taken as the only limitation to the present embodiment.
In addition, it is worth mentioning that the data acquisition speed is faster because the data is queried in the cache, that is, the data is acquired by querying the cache, compared with the data acquisition speed of the database. Therefore, in the process of normally executing the operation on the database in the transaction request, whether the query operation on the database exists in the currently received transaction request can be monitored.
Correspondingly, if the query operation for the database exists, the query operation for the database is directly converted into the query operation for the cache, namely, the query operation is responded based on the cache.
Specifically, in order to avoid data redundancy, it is ensured that the data recorded in the cache can be automatically emptied from the data recorded in the cache at the last working stage after the device is restarted every time.
That is, when responding to the query operation based on the cache, specifically by determining whether the data requested by the query operation is contained in the temporary cache region.
Correspondingly, if the transaction request contains the data, the query request is directly responded based on the data in the temporary buffer area, namely, the data requested by the query request is taken out of the temporary buffer area and fed back to the user side sending the transaction request.
Otherwise, if the temporary buffer area does not contain the query operation, the query operation on the buffer is converted into the query operation on the database again, the data of the query operation request is obtained from the database, and the query operation request is fed back based on the queried data.
Further, in order to ensure that the response can be made quickly when the device executing the transaction processing method receives the query operation on the same data again in the work cycle, that is, in the interval from the startup state to the shutdown state, after the data of the query operation request is obtained from the database, the obtained data may be cached in the temporary buffer.
In order to better understand the case that the operation is a query operation in the transaction request, this embodiment provides a specific implementation manner for querying the cache according to the query operation, which is specifically as follows:
assume that the data stored in the database in the a table is as follows:
database A table
TABLE 1
The data recorded in the temporary buffer is recorded in a key-value pair, i.e., a key-value. Regarding keys, this embodiment provides a specific format, specifically: table name, field name-field value.
For example, the key generated by the information with the field name id and the field value 123 in the table a is specifically: and A, id-123.
Correspondingly, based on a:: id-123, the corresponding value searched from the temporary buffer area is specifically the other record corresponding to the id-123 in the a table, namely, name-abc, and age-11.
For better understanding, the format of the data recorded in the temporary buffer is given by taking the field names id and name in the a table and the field values corresponding to these fields as examples, and a specific record stored in the temporary buffer is given, that is, only data of "id 123, name abc, age 11", and "id 234name bcd, age 12" are recorded in the record table in the temporary buffer.
Record table in temporary buffer area
| key | value |
| A::id-123 | name＝abc，age＝11 |
| A::id-234 | name＝bcd，age＝12 |
| A::name-abc | id＝123，age＝11 |
| A::name-bcd | id＝234，age＝12 |
| A::age-11 | id＝123，name＝abc |
| A::age-12 | id＝234，name＝bcd |
TABLE 2
Correspondingly, if the query operation in the service request is to query all data with id 123 in the a table, the query operation carries the query information, and the generated query index, namely key, is a:: id-123.
Correspondingly, traversing the record table in the temporary buffer area based on the generated key, and if the same key exists in the temporary buffer area, taking out the value corresponding to the key inquired in the record table, namely taking out the name abc and the age 11 as the inquired data, and feeding back the inquired data to the user side initiating the service request.
If the query operation in the service request is to query all data of the name-cdf in the a table, the query operation carries the query information, and the generated query index, namely the key is the a:: name-cdf.
Correspondingly, traversing the record table in the temporary buffer area based on the generated key, and if the record table does not have the same key, converting the query operation on the temporary buffer area into the query operation on the database again, and then responding to the query request based on the database.
Specifically, the name cdf is used as a query condition, the lookup is performed in the a table, data of id 456 and age 13 which have a corresponding relationship with the name cdf can be found, and then the data of id 456 and age 13 queried from the database is taken out as queried data and fed back to the user side initiating the service request.
It should be understood that the above examples are only examples for better understanding of the technical solution of the present embodiment, and are not to be taken as the only limitation to the present embodiment.
Regarding the format of the key recorded in the temporary buffer, in practical applications, the format is not limited to the one given in the embodiment, as long as the key can represent the association relationship among the table name, the field name, and the field value.
And 102, correspondingly generating a cache operation according to the database operation executed between the beginning and the end of the transaction, and recording the cache operation.
As can be seen from the above description, in practical applications, a transaction start annotation and a transaction end annotation are pre-established for a transaction involved in a transaction processing process based on an adopted framework or programming language. Therefore, when a transaction request is received, the operation interval for the database in the normal execution transaction request can be determined according to the transaction start and the transaction end, and the operation for the cache is correspondingly generated according to the operation for the database executed in the operation interval.
In addition, as can be seen from the above description, the transaction of the business layer in this embodiment usually binds the related operations of multiple databases. In practical applications, the cache introduced in the data storage technology, such as a local cache (e.g., java heap memory cache) and a distributed cache (e.g., Redis), is also a database in essence, except that the access speed of such a database is faster than that of the SQL database and the Oracle database in general. Therefore, for the same operation in the transaction, the operation for the database such as SQL, Oracle and the like can be generated, and the operation for the cache can also be generated.
For convenience of explanation, the present embodiment takes the cache as Redis, and refers to data stored in the Redis as cache data, i.e., replica data, while the database may be SQL, Oracle, or other database, and refers to data stored in the database as business data, i.e., source data.
Still taking the example given in step 101 as an example, when a data operation that needs to be modified or deleted on the database needs to be performed, the data in the cache needs to be deleted correspondingly, so as to ensure that the cached data is consistent with the corresponding data in the database.
In practical applications, there may be a case where the database is inconsistent with the data in the temporary cache at a certain time. For example, in the a table of the database, name is abc, and the name is abcd. At this time, if it is determined whether a record exists in the temporary buffer based on the modified name:: name-abcd in the a table, the key of the record table in the generated query temporary buffer is "a:: name-abcd" instead of "a:: name-abc", and therefore, the record corresponding to "a:: name-abc" in the buffer cannot be deleted based on "a:: name-abcd", the key of "a:: name-abc" and the value corresponding to the key are recorded in the temporary buffer, which causes garbage data to appear in the temporary buffer. In order to clear the old data cached in the temporary buffer area, the cache in this embodiment may be divided into a temporary buffer area and a permanent buffer area.
The data cached in the temporary cache region can be used for quickly responding to a service request, namely a data query request, and meanwhile, the data is also acquired based on the service request and is the same as a general caching mechanism; the permanent cache region may be configured to store preconfigured cache data, that is, a developer may preset a cache range according to actual needs to determine data records to be cached, for example, the cache range may include all data records in the table a, or data records corresponding to id in the table B as some specific value, or data corresponding to a partial field in the table C, when the system is started, the permanent cache region may obtain corresponding data records from the database according to the set cache range, and perform caching, and when data in the temporary cache region needs to be deleted, the corresponding data records may be obtained from records in the permanent cache region, and a corresponding key is generated, so as to delete data in the temporary cache region and ensure that data in the temporary cache region can be completely removed. It should be noted that the set buffer ranges are also applicable to the temporary buffer area, that is, only the data included in the buffer range is buffered in the temporary buffer area.
That is, in this embodiment, the data cached in the permanent cache region is data directly acquired from the database based on a preset cache range. The temporary cache region only responds to the data query operation request in the cache range, so that the cache data in the temporary cache region are all recorded in the permanent cache region, and when the data in the temporary cache region needs to be deleted, the corresponding data can be acquired from the permanent cache region to generate the key value of the data in the temporary cache region.
Correspondingly, before the cache operation is correspondingly generated according to the database operation executed between the beginning and the end of the transaction, it may be determined whether the data corresponding to the database operation is within the cache range, and if so, a record of the cache is correspondingly generated; if not, no record is generated for the cache.
In practical applications, the cache range may include all records in the database, or may include a part of records in the database.
Based on the above description, when the cache includes the temporary buffer area and the permanent buffer area, the above-mentioned generating the operation on the cache according to the operation on the database executed between the beginning of the transaction and the end of the transaction, and recording the operation on the cache specifically includes:
and when the operation on the database is newly added data, correspondingly generating newly added data operation on the permanent cache region, thereby ensuring that the data in the permanent cache region is consistent with the corresponding data in the database.
When the operation on the database is data deletion, data deletion operations on the permanent cache area and the temporary cache area are correspondingly generated;
and when the operation on the database is data modification, correspondingly generating a data deletion operation on the temporary cache region and a data modification operation on the permanent cache region.
Correspondingly, when the operation of deleting the data in the temporary buffer area is executed, specifically, based on the storage information of the data to be deleted, the data record corresponding to the storage information is searched from the permanent buffer area, and the data to be deleted corresponding to the temporary buffer area is determined based on the data record and is deleted.
Further, in order to ensure that the cache data stored in the cache is data with a high access frequency, that is, hot data, when the transaction queried in the database is processed to the temporary cache region, the hot value of the data may be determined first, and when it is determined that the data meets a preset requirement, for example, the hot value is higher than a certain threshold, the transaction is processed to the temporary cache region.
Further, in practical application, a timeout time can be set for the cache data stored in the temporary cache region, and when the set timeout time is exceeded, the cache data in the temporary cache region is automatically deleted, so that the redundancy of the garbage data is prevented.
In this embodiment, after all the operations on the database are completed, the operations on the cache may be executed before the data is submitted into the database, and after the cache operations are successfully completed, the database is subjected to the warehousing operations, and the submission transaction is completed, so that the transaction processing is completed, thereby reducing the number of times of interaction with the database.
It is noted that in other embodiments of the present invention, after all the operations on the database are completed, the data warehousing operation may be directly completed, the transaction is committed to end, and then the operation on the cache is executed, thereby ensuring fast response to the transaction processing request.
In addition to query operations, operations on data typically include delete data operations, add data operations, and modify data operations. The query operation can be responded based on the cache or directly responded based on the database, the deletion or modification of the cache data is not involved, the cache data does not need to be recorded, and other operations need to be recorded.
In the transaction processing method provided in this embodiment, in order to prevent the temporary buffer area from having the above-mentioned problem of garbage data, when the data deletion operation is performed on the temporary buffer area, the data record in the permanent buffer area is used, which is specifically as follows:
(1) and extracting information to be deleted corresponding to the data deleting operation from the recorded data deleting operation.
It is to be understood that, here, the operation of extracting the to-be-deleted information corresponding to the data deletion operation from the service request is similar to the operation of extracting the to-be-queried information corresponding to the query operation from the service request when the operation in step 102 is a query operation, that is, in actual application, the record form is, for example, all data whose id is 123 in the deletion table a, which is not described herein again.
Correspondingly, the information to be deleted specifically includes a table name, a field name and a field value.
(2) And generating an index of a permanent cache region in the cache according to the table name, the field value and a preset permanent cache identifier.
For convenience of explanation, the index applied to the persistent cache area is referred to as a first index, and the first index needs to embody an association relationship between a table name, a field value, and a preset persistent cache identifier. The permanent cache identifier is used to distinguish the permanent cache region from the temporary cache region in the cache, that is, the data containing the permanent cache identifier in the key value is the data stored in the permanent cache region.
In one implementation, the format of the first index may be: table name-permanent cache identification field name-field value.
Still taking the field value as "123", the belonging field name as "id", and the belonging table name as "a", as examples, when the preset permanent cache identifier is "fv", the first index generated according to the above format is "a-fv:: id-123". It is noted that the corresponding key value is generated only for the unique identifier (e.g., id) of the data record in the persistent cache, and all other data records corresponding to the unique identifier are saved in the value.
For better understanding of the contents of the records in the persistent cache, assuming that the caching range of the persistent cache is the entire contents of the database table a, the corresponding key-value record table is as follows:
record list of permanent buffer zone
| key | value |
| A-fv::id-123 | name＝abc，age＝11 |
| A-fv::id-234 | name＝bcd，age＝12 |
| A-fv::id-456 | name＝cdf，age＝13 |
TABLE 3
It should be understood that the above examples are only examples for better understanding of the technical solution of the present embodiment, and are not to be taken as the only limitation to the present embodiment.
After the first index is generated, the corresponding data in the permanent cache region may be queried based on the first index, for example, based on key: "a-fv:: id-123", the table may be queried to obtain value ═ name ═ abc, and age ═ 11 ", and then a second index may be correspondingly generated according to the queried value, where the second index is used to delete the corresponding data in the temporary cache region, and the format may be: for example, when value is "name" abc and age "11", a second index may be generated: "A:: name-abc", "A:: age-11" and "A:: id-123".
(3) And deleting the data in the temporary buffer area according to the second index.
And obtaining a plurality of corresponding second indexes based on the steps, and deleting the corresponding data in the temporary cache region by using the second indexes as key values. Taking table 2 as an example, the second index generated according to the first index includes: and the second indexes are used as key values respectively to delete the data in the table 2, and then the deleted data are the 1 st, 3 rd and 5 th data in the table 2.
Therefore, all relevant data in the temporary cache region can be guaranteed to be cleaned up.
It should be noted that, when the recorded caching operations include a data deleting operation for the temporary cache area and a data deleting or modifying operation for the permanent cache area, the data deleting operation for the temporary cache area needs to be executed first.
In the transaction processing method provided in this embodiment, the operations of adding, deleting, and modifying data to the permanent cache region may be directly performed based on the corresponding table name, field name, and field value to generate a key value corresponding to the permanent cache region, and then the related operations are directly performed.
Furthermore, it can be understood that in practical applications, the transaction process for a transaction request is to end the commit transaction as the end node. Therefore, after the operation on the cache in the record is completed, the transaction needs to be submitted to end.
That is, after step 103, a step of committing the end of the transaction may also be included.
In this embodiment, after all operations on the cache are completed, the database may be submitted and put in storage, and the transaction processing is completed after the transaction is submitted. Therefore, the database can be guaranteed to be put into storage uniformly after the cache is subjected to consistency processing, the interaction times with the database can be reduced, and the situation that the database needs to be accessed for many times and service resources are occupied when processing abnormity occurs in the process is prevented.
It should be understood that the above examples are only examples for better understanding of the technical solution of the present embodiment, and are not to be taken as the only limitation to the present embodiment.
Therefore, in the transaction processing method provided in this embodiment, in the process of normally executing the operation on the database in the transaction request, the operation on the cache is correspondingly generated according to the operation on the database executed between the start of the transaction and the end of the transaction, the generated operation on the cache is recorded, and after all the operations on the database in the transaction request are completed, the cache is synchronized according to the recorded operation on the cache.
A second embodiment of the present invention relates to a transaction processing method. The second embodiment is further improved on the basis of the first embodiment, and the main improvements are as follows: a queue mechanism is introduced, a plurality of operations can be inserted into a queue in sequence in a multithreading and multitasking processing stage, and then the operations are taken out and executed in sequence when restarting is needed, so that the problem of thread blocking caused by accumulation of a large number of operations is solved.
As shown in fig. 2, the transaction processing method according to the second embodiment includes the following steps:
It is to be understood that step 201 of this embodiment is substantially the same as step 101 of the first embodiment, and will not be described herein again.
At step 202, an empty operation queue is generated.
Specifically, in the transaction processing method provided in this embodiment, the operations performed on the database and the cache are performed in units of transactions. Therefore, the generated operation queue may be only for the current transaction, and the generated operation queue is specifically a sequential operation queue.
Further, in practical applications, in order to distinguish operations corresponding to different transactions received under multithreading and multitasking, before generating an empty operation queue, a transaction identifier, that is, identification information capable of identifying transaction uniqueness, may be allocated to each transaction based on a preset rule; then, an empty operation queue corresponding to the transaction is generated according to the transaction identification for identifying the uniqueness of the transaction, so that the binding between the transaction and the operation composing the transaction is realized.
Specifically, since the query operation does not change the service data stored in the database and the cache data stored in the cache, it is also necessary to first determine whether the operation on the database in the transaction request is the query operation in the process of sequentially inserting the operations on the cache into the operation queue.
Correspondingly, if yes, the query operation on the database is directly converted into the query operation on the cache, and then the query operation is responded based on the cache, namely when the query operation is performed on the database in the transaction request, the query search on the cache is not required to be inserted into the operation queue, but the query operation is directly responded by the cache, so that the transaction request can be quickly responded, and further the user experience is improved.
For querying the cache according to the query operation, and making feedback for the transaction request after the query operation is completed, see the description in the first embodiment for details, which is not described herein again.
And step 204, after all the operations on the database are completed, taking out the operations on the cache from the operation queue in sequence and executing the operations.
It should be readily apparent that step 204 of this embodiment is substantially the same as step 103 of the first embodiment, and the main differences are:
after all operations on the database are completed, the operations on the cache are taken out from the operation queue in sequence, specifically, the operations on the cache are taken out from the operation queue in a first-in first-out order, and then according to a specific type of the taken-out operations, for example, a data deleting operation, the operations on the cache are performed according to the scenario 1 given in step 103 of the first embodiment, and if the operations are not data deleting operations, that is, data adding or modifying operations, the operations on the cache are performed according to the scenario 2 given in step 103 of the first embodiment.
For specific operation processing performed on the cache, see the detailed description of scenario 1 and scenario 2 given in step 103 in the first embodiment, which is not described herein again.
For convenience of explanation, the present embodiment takes an Aspect Oriented Programming (AOP) as an example, and the whole processing flow of the transaction processing is described with reference to fig. 3:
as shown in fig. 3, first, when an operation starts, that is, a service request is received, a transaction identifier capable of identifying uniqueness is allocated to a transaction based on a preset rule; then, generating an empty operation queue corresponding to the transaction according to the transaction identifier; then, identifying a transaction start annotation based on a function or statement before (AOP) in the AOP for identifying the start of the transaction, identifying a transaction start end annotation based on a function or statement After (AOP) in the AOP for identifying the end of the transaction, and further, at the start of the transaction, in a processing stage between the start of the transaction and the end of the transaction, namely a function stage in fig. 3, generating operations on the cache according to the operations on the database executed between the start of the transaction and the end of the transaction, and adding the generated operations on the cache into an operation queue in order, for example, N new data operations, N modified data operations, and N deleted data operations (N is a positive integer) are sequentially inserted into the operation queue, and at the same time, the operations on the database in the transaction request are normally executed in the stage, and after all the operations on the database are monitored to be completed, and suspending the operation of ending the submitted transaction, firstly taking out the operations to the cache from the operation queue in sequence and executing the operations, restarting the operation of ending the submitted transaction after finishing all the operations to the cache in the operation queue, ending the submitted transaction, finishing the request of the transaction and ending the operation.
In addition, it is worth mentioning that, in practical applications, in order to ensure that after each operation on the database is completed, the corresponding operation on the cache may be started, an operation queue may be created for the database according to the transaction identifier, and the operations on the database may be sequentially inserted into the operation queue for the database.
Moreover, it is easy to find that step 205 of this embodiment is substantially the same as step 104 of the first embodiment, and the details are not repeated here.
Therefore, in the transaction processing method provided in this embodiment, the operation queue is introduced, the operations performed on the cache are added to the operation queue, and the corresponding operations are subsequently taken out from the operation queue in order when needed and executed, so that the problem of thread blocking caused by accumulation of a large number of operations is greatly alleviated, peak pressure is eliminated, and the transaction processing scheme is more robust and stable.
Furthermore, the transaction identifier capable of identifying the uniqueness of the transaction is distributed to the transaction, and then the operation queue is generated according to the transaction identifier with the uniqueness, so that the binding between the transaction and the operation is realized, and the corresponding operation in different transactions can be well identified.
A third embodiment of the present invention relates to a transaction processing method. The third embodiment is further improved on the basis of the first or second embodiment, and the main improvement is as follows: and introducing a rollback mechanism, specifically, when an exception occurs in the normal execution process of the operation on the database in the transaction request and the normal execution process of the operation on the database in the transaction request, directly performing transaction rollback to recover to the state before receiving the transaction request.
For convenience of explanation, this embodiment is based on the second embodiment, as shown in fig. 4.
In particular, the exception may occur during the operation of the database and may also occur during the operation of the cache. Therefore, in order to ensure the consistency of the data stored in the database and the data cached in the cache, before the transaction is submitted, whether the operation on the database is abnormal or the operation on the cache is abnormal, the transaction rollback is carried out.
It can be understood that when the operation on the database is monitored to be abnormal, the transaction rollback performed at this time is mainly for the database because the processing on the cache is not performed at the current stage.
Specifically, in this embodiment, the operation that has been performed on the database in the current transaction request is recovered, that is, the state of the database is recovered to a position before the transaction starts.
Further, if the operation on the cache is already generated when the operation on the database is abnormal, the operation on the cache that has already been generated needs to be deleted, so that the phenomenon that the operation on the cache is still executed when the operation on the database is abnormal is avoided, and the consistency of the data recorded in the database and the cache is further ensured.
Further, if operations to the cache have already been generated when an exception occurs to an operation to the database, and these operations to the cache are added to the operation queue, the operations to the cache added to the operation queue also need to be emptied.
Furthermore, it can be understood that when an exception occurs to the operation on the cache, since all operations on the database have been completed at this stage, the transaction rollback performed at this time needs to be performed not only on the cache but also on the database, i.e., both the database and the cache need to be restored to the state before the transaction started.
Therefore, in the transaction processing method provided in this embodiment, by introducing a rollback mechanism, if an operation on the database is abnormal, the operation that has been performed on the database in the current transaction request is recovered by performing transaction rollback, that is, the state of the database is recovered to a point before the transaction starts; if an exception occurs in the operation process of the cache, the cache and the database are simultaneously subjected to transaction rollback, and the database and the cache are both restored to the state before the beginning of the transaction, so that the problem of data inconsistency in the database and the cache caused by the exception is solved.
It should be understood that the above steps of the various methods are divided for clarity, and the implementation may be combined into one step or split into a plurality of steps, and all that includes the same logical relationship is within the protection scope of the present patent; it is within the scope of the patent to add insignificant modifications to the algorithms or processes or to introduce insignificant design changes to the core design without changing the algorithms or processes.
A fourth embodiment of the present invention relates to a transaction device, as shown in fig. 5, comprising at least one processor 501; and, a memory 502 communicatively coupled to the at least one processor 501; the memory 502 stores instructions executable by the at least one processor 501, and the instructions are executed by the at least one processor 501, so that the at least one processor 501 can execute the transaction processing method described in the first, second, or third embodiment.
The memory 502 and the processor 501 are coupled by a bus, which may comprise any number of interconnected buses and bridges that couple one or more of the various circuits of the processor 501 and the memory 502 together. The bus may also connect various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other devices over a transmission medium. The data processed by the processor 501 is transmitted over a wireless medium via an antenna, which further receives the data and transmits the data to the processor 501.
The processor 501 is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And memory 502 may be used to store data used by processor 501 in performing operations.
A fifth embodiment of the present invention relates to a computer-readable storage medium storing a computer program. The computer program, when executed by a processor, implements the transaction processing method embodiments described above.
That is, as those skilled in the art can understand, all or part of the steps in the method of the embodiments described above may be implemented by a program to instruct related hardware, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific embodiments for practicing the invention, and that various changes in form and details may be made therein without departing from the spirit and scope of the invention in practice.
Claims (14)
1. A transaction processing method, comprising:
receiving a transaction request, and normally executing the operation on the database in the transaction request;
correspondingly generating cache operation according to the database operation executed between the beginning and the end of the transaction, and recording the cache operation;
and after all the operations on the database are completed, executing the operations on the cache in the record.
2. The transaction processing method according to claim 1, wherein during the normal execution of the operation on the database in the transaction request, the method further comprises:
monitoring whether the operation on the database is abnormal or not;
and if the exception occurs, rolling back the transaction.
3. A transaction processing method according to claim 1, wherein during said performing said operation on the cache in said record, said method further comprises:
monitoring whether the operation on the cache is abnormal or not;
and if the exception occurs, rolling back the transaction.
4. The transaction processing method according to claim 1, wherein during the normal execution of the operation on the database in the transaction request, the method further comprises:
if the operation on the database is a query operation;
responding to the query operation based on the cache.
5. The transaction processing method of claim 4, wherein the cache comprises a temporary cache area;
the responding to the query operation based on the cache comprises:
determining whether the data of the query operation request is contained in the temporary cache region;
if yes, directly responding to the query request based on the data in the temporary cache region;
and if not, acquiring the data of the query operation request from the database and caching the data into the temporary cache region.
6. The transaction processing method according to claim 1, wherein the cache comprises a temporary cache area and a permanent cache area;
the correspondingly generating the operation on the cache according to the operation on the database executed between the beginning and the end of the transaction, and recording the operation on the cache, includes:
when the operation on the database is newly added data, the newly added data operation on the permanent cache area is correspondingly generated;
when the operation on the database is data deletion, data deletion operations on the permanent cache area and the temporary cache area are correspondingly generated;
and when the operation on the database is data modification, correspondingly generating a data deletion operation on the temporary cache region and a data modification operation on the permanent cache region.
7. The transaction processing method according to claim 6, when performing the delete data operation on the temporary buffer, comprising:
and searching a data record corresponding to the storage information from the permanent cache region based on the storage information of the data to be deleted, determining the corresponding data to be deleted in the temporary cache region based on the data record, and deleting the data.
8. The transaction processing method according to claim 6, wherein the temporary cache region is configured to respond to the query operation in the transaction request, and if the temporary cache region does not contain the data of the query operation request, the data of the query operation request is obtained from the database and cached.
9. The transaction processing method according to claim 6, wherein the data cached in the persistent cache region is data directly obtained from the database based on a preset caching range.
10. The transaction processing method of claim 1, wherein the method further comprises:
setting a cache range for the cache in advance, determining whether data corresponding to the operation on the database is in the cache range or not before the cache operation is correspondingly generated according to the operation on the database executed between the beginning and the end of the transaction, and if so, correspondingly generating a record on the cache; if not, no record is generated for the cache.
11. A transaction processing method according to claim 9 or 10, wherein the buffer contains all records in the database or part of records in the database.
12. The transaction processing method of claim 1, wherein the method further comprises: after the step of executing the operation on the cache in the record is completed, committing the transaction to end.
13. A transaction device, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the transaction processing method of any of claims 1 to 12.
14. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the transaction processing method of any one of claims 1 to 12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011449010.5A CN112631741A (en) | 2020-12-09 | 2020-12-09 | Transaction processing method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011449010.5A CN112631741A (en) | 2020-12-09 | 2020-12-09 | Transaction processing method, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112631741A true CN112631741A (en) | 2021-04-09 |
Family
ID=75309686
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011449010.5A Pending CN112631741A (en) | 2020-12-09 | 2020-12-09 | Transaction processing method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112631741A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7822727B1 (en) * | 2004-07-02 | 2010-10-26 | Borland Software Corporation | System and methodology for performing read-only transactions in a shared cache |
| CN106354851A (en) * | 2016-08-31 | 2017-01-25 | 广州市乐商软件科技有限公司 | Data-caching method and device |
| CN106446037A (en) * | 2016-08-31 | 2017-02-22 | 南威软件股份有限公司 | Method for realizing consistency of Redis and MYSQL data based on distributed lock |
| CN108958955A (en) * | 2018-07-06 | 2018-12-07 | 美利车（北京）网络技术有限公司 | A kind of transaction methods and device based on message-driven |
| CN109947801A (en) * | 2019-02-25 | 2019-06-28 | 交通银行股份有限公司 | Database in phase system, method and device |
| CN110008224A (en) * | 2019-03-08 | 2019-07-12 | 阿里巴巴集团控股有限公司 | A kind of db transaction processing method and processing device |
-
2020
- 2020-12-09 CN CN202011449010.5A patent/CN112631741A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7822727B1 (en) * | 2004-07-02 | 2010-10-26 | Borland Software Corporation | System and methodology for performing read-only transactions in a shared cache |
| CN106354851A (en) * | 2016-08-31 | 2017-01-25 | 广州市乐商软件科技有限公司 | Data-caching method and device |
| CN106446037A (en) * | 2016-08-31 | 2017-02-22 | 南威软件股份有限公司 | Method for realizing consistency of Redis and MYSQL data based on distributed lock |
| CN108958955A (en) * | 2018-07-06 | 2018-12-07 | 美利车（北京）网络技术有限公司 | A kind of transaction methods and device based on message-driven |
| CN109947801A (en) * | 2019-02-25 | 2019-06-28 | 交通银行股份有限公司 | Database in phase system, method and device |
| CN110008224A (en) * | 2019-03-08 | 2019-07-12 | 阿里巴巴集团控股有限公司 | A kind of db transaction processing method and processing device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11429641B2 (en) | Copying data changes to a target database | |
| EP3111347B1 (en) | Efficient methods and systems for consistent read in record-based multi-version concurrency control | |
| US8874515B2 (en) | Low level object version tracking using non-volatile memory write generations | |
| US6567928B1 (en) | Method and apparatus for efficiently recovering from a failure in a database that includes unlogged objects | |
| US7376674B2 (en) | Storage of multiple pre-modification short duration copies of database information in short term memory | |
| US20190129894A1 (en) | Database Transaction Processing Method, Client, and Server | |
| US11132350B2 (en) | Replicable differential store data structure | |
| EP0501180A2 (en) | Dynamic, finite versioning for concurrent transaction and query processing | |
| CN108509462B (en) | Method and device for synchronizing activity transaction table | |
| US20070118547A1 (en) | Efficient index versioning in multi-version databases | |
| US20130132349A1 (en) | Tenant separation within a database instance | |
| US7769792B1 (en) | Low overhead thread synchronization system and method for garbage collecting stale data in a document repository without interrupting concurrent querying | |
| EP1806668B1 (en) | System and method for managing log information for transaction | |
| US8793288B2 (en) | Online access to database snapshots | |
| CN110209528B (en) | Data backup method, device, server and storage medium | |
| CN106446044B (en) | Storage space recovery method and device | |
| US20050125458A1 (en) | Chronological data record access | |
| JP6293709B2 (en) | Storage system and storage system program | |
| CN110196788B (en) | Data reading method, device and system and storage medium | |
| US8521789B2 (en) | Undrop objects and dependent objects in a database system | |
| US11593352B2 (en) | Cloud-native object storage for page-based relational database | |
| EP4002148A1 (en) | Cloud-native object storage for page-based relational database | |
| CN112631741A (en) | Transaction processing method, device and storage medium | |
| CN114691307A (en) | Transaction processing method and computer system | |
| CN114595224A (en) | Data storage method and device and data query method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |