CN117056313A - Memory database transaction management method and device, electronic equipment and medium - Google Patents

Memory database transaction management method and device, electronic equipment and medium Download PDF

Info

Publication number
CN117056313A
CN117056313A CN202311063933.0A CN202311063933A CN117056313A CN 117056313 A CN117056313 A CN 117056313A CN 202311063933 A CN202311063933 A CN 202311063933A CN 117056313 A CN117056313 A CN 117056313A
Authority
CN
China
Prior art keywords
transaction
version data
read
data information
write
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311063933.0A
Other languages
Chinese (zh)
Inventor
陈奕林
周信静
张椿旭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Zhiyu Technology Co ltd
Original Assignee
Zhejiang Zhiyu Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Zhiyu Technology Co ltd filed Critical Zhejiang Zhiyu Technology Co ltd
Priority to CN202311063933.0A priority Critical patent/CN117056313A/en
Publication of CN117056313A publication Critical patent/CN117056313A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/219—Managing data history or versioning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2308—Concurrency control
    • G06F16/2315—Optimistic concurrency control
    • G06F16/2329—Optimistic concurrency control using versioning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2365—Ensuring data consistency and integrity

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application relates to the field of memory database management, in particular to a memory database transaction management method, a device, electronic equipment and a medium, wherein the method comprises the steps of acquiring a transaction identifier of a read-only transaction and a plurality of version data information corresponding to a data row when detecting that the read-only transaction requests to access data, wherein each version data information comprises a version data row, a version transaction identifier, reading times, a transaction access interval and log persistence parameters; determining target version data information from the plurality of version data information based on a transaction identifier of the read-only transaction and transaction access intervals corresponding to the plurality of version data information, wherein the target version data information is version data information appointed to be accessed by the read-only transaction; a visible version data line is determined based on the log persistence parameter of the target version data information. The method and the device can ensure the correctness of the transaction in the Dolphin DB memory database.

Description

Memory database transaction management method and device, electronic equipment and medium
Technical Field
The present application relates to the field of memory database management, and in particular, to a method, an apparatus, an electronic device, and a medium for managing a memory database transaction.
Background
The Dolphin DB is a high-performance distributed memory database, aims to process large-scale data analysis and real-time data processing tasks, and adopts a two-stage lock protocol and multi-version concurrency control mechanism to ensure the atomicity and durability of transactions in the memory database. The two-stage lock protocol requires that before the log of the read-write transaction falls on a disk, the write lock of the read-write transaction cannot be released, so that the read lock cannot be timely acquired by the concurrent read-only transaction having data intersection with the read-write transaction.
In the related technology, the read-write lock of the read-write transaction is released before the log is dropped, so that the concurrent read-only transaction can acquire the read-lock, and the locking time of the data line is reduced to improve the concurrency performance of the memory database. The read-write transaction immediately releases the lock after the data modification is completed, if the concurrent read-only transaction of the read-write transaction needs to read the data modified by the read-write transaction and the concurrent read-only transaction is completed to be submitted at the moment, but the modification operation of the read-write transaction on the data is lost and is not stored in a persistent storage medium (hard disk), the read-write transaction is a transaction needing rollback, and further the read-only transaction reads the modified data needing rollback, which violates the isolation of the transaction and further influences the correctness of the transaction. Therefore, how to ensure the correctness of the transaction in the dolphin db memory database is a urgent problem to be solved.
Disclosure of Invention
In order to ensure the correctness of the transaction in the Dolphin DB memory database, the application provides a memory database transaction management method, a device, electronic equipment and a medium.
In a first aspect, the present application provides a method for managing memory database transactions, which adopts the following technical scheme:
a memory database transaction management method, comprising:
when a read-only transaction request access data line is detected, acquiring a transaction identifier of the read-only transaction and a plurality of version data information corresponding to the data line, wherein each version data information comprises a version data line, a version transaction identifier, reading times, a transaction access interval and a log persistence parameter, the transaction access interval is a transaction identifier interval of the read-only transaction capable of reading the version data line, and the log persistence parameter is used for representing whether a modification operation log of a read-write transaction creating the version data line is written into a persistent storage medium;
determining target version data information from the plurality of version data information based on a transaction identifier of the read-only transaction and transaction access intervals corresponding to the plurality of version data information, wherein the target version data information designates accessed version data information for the read-only transaction;
And determining visible version data lines based on the log persistence parameters of the target version data information.
By adopting the technical scheme, the target version data information which is required to be read by the read-only transaction is determined from the plurality of version data information according to the transaction identifier of the read-only transaction; and judging whether the modification operation log of the read-write transaction is written into the persistent storage medium according to the log persistence parameter of the target version data information, wherein the corresponding version data row is regarded as visible only when the modification operation log of the read-write transaction is written into the persistent storage medium, namely, the version data row is determined. This ensures the correctness of the transaction and avoids reading data that has been deleted or not yet committed.
In one possible implementation manner, the determining the visible version data line based on the log persistence parameter of the target version data line includes:
after a modification operation log of a read-write transaction corresponding to the target version data information is written into a persistent storage medium, modifying a log persistence parameter in the version data information into a preset visible value;
when the log persistence parameter of the target version data information is the preset visible value, determining the version data line in the target version data information as a visible version data line, and modifying the reading times in the target version data information;
When the log persistence parameter of the target version data information is not the preset visible value, adding the read-write transaction corresponding to the target version data information into a dependent transaction set of the read-only transaction, determining visible version data lines based on the dependent transaction set, and modifying the reading times in the target version data information.
By adopting the technical scheme, if the modification operation log of the read-write transaction corresponding to the target version data information is written into the persistent storage medium, the log persistence parameter is correspondingly modified to be a preset visible value; when the log persistence parameter is a preset visible value, the modified version data row in the target version data information is determined to be the visible version data row, so that the read data of the read-only transaction is correct. When the log persistence parameter of the target version data information is not a preset visible value, the read-write transaction corresponding to the target version data information is added into the dependent transaction set of the read-only transaction, so that a dependent relationship can be established, and the read-only transaction can be ensured to read the data submitted and updated by the dependent transaction. By recording the number of reads, the access condition of each version data line can be tracked for subsequent optimization and statistical analysis.
In one possible implementation manner, the adding the read-write transaction corresponding to the target version data information to the dependent transaction set of the read-only transaction includes:
determining a transaction identifier of a read-write transaction corresponding to the target version data information based on the transaction access interval of the target version data information;
and adding the read-write transaction corresponding to the target version data information into a dependent transaction set of the read-only transaction based on the transaction identifier of the read-write transaction corresponding to the target version data information.
By adopting the technical scheme, when the target version data information is created, the lower limit of the transaction access interval is the transaction identifier of the read-write transaction for creating the target version data information, so that the transaction identifier of the read-write transaction for creating the target version data information can be determined through the transaction access interval of the target version data information, and then the read-write transaction is added into the dependent transaction set of the read-write transaction according to the transaction identifier of the read-write transaction, so that the read-write transaction for creating the target version data information can be determined more quickly.
In one possible implementation manner, before acquiring the version data information corresponding to the data line, the method further includes:
When the read-write transaction request is detected to modify the data row, based on the read-write transaction, version data information of the data row is created, wherein the version data information comprises a version data row, a version transaction identifier, reading times, a transaction access interval and a log persistence parameter, the initial value of the version transaction identifier is a transaction identifier of the read-write transaction for creating the version data information, and the initial value of the log persistence parameter is a preset invisible value.
By adopting the technical scheme, when the read-write transaction request data line is detected to be modified, version data information of the data line is created according to the read-write data line, and the version data information comprises the version data line to be modified, the version transaction identifier, the reading times, the transaction access interval and the log persistence parameter. By setting the version transaction identifier to the transaction identifier of the read-write transaction that created the version data information, the source transaction for each version data information can be accurately tracked. By setting the initial value of the log persistence parameter to a preset invisible value, it can be ensured that the newly created version data is invisible to other transactions; only when the modification operation log of the read-write transaction is written into the persistent storage medium, namely, when the log persistence parameter of the version data information is modified to a preset visible value, other transactions can read the version data, so that the consistency of the transactions is ensured. Meanwhile, when the read-only transaction needs to access the version data, the data of the version can be determined to be safely read through the log persistence parameter, and the query data in the global transaction set is not needed, so that the expenditure of a database management system can be reduced.
In one possible implementation, after creating version data information of the data line, the method further includes:
determining that the read-write transaction is in a write lock binding state;
after the read-write transaction completes the modification operation on the data of the data line, determining that the read-write transaction is in a write lock release state, and modifying a version transaction identifier in version data information corresponding to the read-write transaction into a preset acquisition lock value;
writing a modification operation log of the read-write transaction to a persistent storage medium, the modification operation log comprising modification operations of the read-write transaction to the data line.
By adopting the technical scheme, after the version data information corresponding to the data line is created, the read-write transaction needs to acquire the write lock of the data line, namely, the read-write transaction is determined to be in a write lock binding state, so that other read-only transactions needing to access the version data line can not read the data line which is being modified, and the consistency of the data is ensured; after the read-write transaction completes the modification operation on the data line, the read-write transaction releases the write lock of the data line, namely, the read-write transaction is determined to be in a write lock release state, and meanwhile, the version transaction identifier corresponding to the read-write transaction is modified to be a preset acquisition lock value, and the modification can be used as a signal to inform other read-only transactions that the read lock of the version data line can be started to be read; and then writing the modification operation log of the read-write transaction into the persistent storage medium to ensure that the modification operation of the read-write transaction is persisted. The consistency and concurrency of the concurrent read-only transactions are ensured by the determination of the write lock binding and release states and the modification of the version transaction identification.
In one possible implementation manner, after determining the target version data information from the plurality of version data information based on the transaction identifier of the read-only transaction and the transaction access interval corresponding to each of the plurality of version data information, the method further includes:
a version transaction identifier of the target version data information is obtained;
and when the version transaction identifier of the target version data information is a preset acquisition lock value, determining that the read-only transaction is in a read lock binding state.
By adopting the technical scheme, whether the version data line in the target version data information is modified is judged through the version transaction identification of the target version data information, and if the version data line is modified (the version transaction identification is a preset acquisition lock value), a corresponding read lock can be acquired from a read-only transaction accessing the version data, namely, the read-only transaction is in a read lock binding state. The read lock binding state means that read-only transactions can concurrently read the target version data information without collision with write transactions. Thus, the concurrency performance can be improved, a plurality of read-only transactions are allowed to read the target version data information at the same time, and the throughput of the system is increased.
In a second aspect, the present application provides a memory database transaction management device, which adopts the following technical scheme:
a memory database transaction management device, comprising:
the system comprises a transaction information and version data information acquisition module, a data storage module and a data storage module, wherein the transaction information and version data information acquisition module is used for acquiring a transaction identifier of a read-only transaction and a plurality of version data information corresponding to the data row when detecting that the read-only transaction requests to access the data row, each version data information comprises a version data row, a version transaction identifier, reading times, a transaction access interval and a log persistence parameter, the transaction access interval is a transaction identifier interval of the read-only transaction capable of reading the version data row, and the log persistence parameter is used for representing whether a modification operation log of a read-write transaction creating the version data row is written into a persistent storage medium;
the target version data information determining module is used for determining target version data information from the plurality of version data information based on the transaction identifier of the read-only transaction and the transaction access interval corresponding to the plurality of version data information, wherein the target version data information is version data information appointed for access by the read-only transaction;
And the visible version data row determining module is used for determining visible version data rows based on the log persistence parameters of the target version data information.
By adopting the technical scheme, the target version data information which is required to be read by the read-only transaction is determined from the plurality of version data information according to the transaction identifier of the read-only transaction; and judging whether the modification operation log of the read-write transaction is written into the persistent storage medium according to the log persistence parameter of the target version data information, wherein the corresponding version data row is regarded as visible only when the modification operation log of the read-write transaction is written into the persistent storage medium, namely, the version data row is determined. This ensures the correctness of the transaction and avoids reading data that has been deleted or not yet committed.
In one possible implementation manner, the visible version data line determining module is specifically configured to, when determining the visible version data line based on the log persistence parameter of the target version data line:
after a modification operation log of a read-write transaction corresponding to the target version data information is written into a persistent storage medium, modifying a log persistence parameter in the version data information into a preset visible value;
When the log persistence parameter of the target version data information is the preset visible value, determining the version data line in the target version data information as a visible version data line, and modifying the reading times in the target version data information;
when the log persistence parameter of the target version data information is not the preset visible value, adding the read-write transaction corresponding to the target version data information into a dependent transaction set of the read-only transaction, determining visible version data lines based on the dependent transaction set, and modifying the reading times in the target version data information.
In one possible implementation manner, the visible version data line determining module is specifically configured to, when adding a read-write transaction corresponding to the target version data information to a dependent transaction set of the read-only transaction:
determining a transaction identifier of a read-write transaction corresponding to the target version data information based on the transaction access interval of the target version data information;
and adding the read-write transaction corresponding to the target version data information into a dependent transaction set of the read-only transaction based on the transaction identifier of the read-write transaction corresponding to the target version data information.
In one possible implementation manner, an in-memory database transaction management device further includes:
the version data information creation module is used for creating version data information of the data row based on the read-write transaction when the read-write transaction request is detected to modify the data row, wherein the version data information comprises a version data row, a version transaction identifier, the number of reading times, a transaction access interval and a log persistence parameter, the initial value of the version transaction identifier is a transaction identifier of the read-write transaction for creating the version data information, and the initial value of the log persistence parameter is a preset invisible value.
In one possible implementation manner, an in-memory database transaction management device further includes:
the write lock binding state determining module is used for determining that the read-write transaction is a write lock binding state;
the write lock release state determining module is used for determining that the read-write transaction is in a write lock release state after the read-write transaction completes the modification operation on the data of the data row, and modifying the version transaction identifier in the version data information corresponding to the read-write transaction into a preset acquisition lock value;
and the log drop module is used for writing a modification operation log of the read-write transaction into the persistent storage medium, wherein the modification operation log comprises modification operation of the read-write transaction on the data line.
In one possible implementation manner, an in-memory database transaction management device further includes:
the version transaction identifier acquisition module is used for acquiring the version transaction identifier of the target version data information;
and the read lock binding state determining module is used for determining that the read-only transaction is in a read lock binding state when the version transaction identifier of the target version data information is a preset acquisition lock value.
In a third aspect, the present application provides an electronic device, which adopts the following technical scheme:
an electronic device, the electronic device comprising:
at least one processor;
a memory;
at least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, the at least one application configured to: executing the memory database transaction management method.
In a fourth aspect, the present application provides a computer readable storage medium, which adopts the following technical scheme:
a computer-readable storage medium, comprising: a computer program capable of being loaded by a processor and executing the memory database transaction management method described above is stored.
In summary, the present application includes at least one of the following beneficial technical effects:
1. Determining target version data information to be read by the read-only transaction from a plurality of version data information according to the transaction identifier of the read-only transaction; and judging whether the modification operation log of the read-write transaction is written into the persistent storage medium according to the log persistence parameter of the target version data information, wherein the corresponding version data row is regarded as visible only when the modification operation log of the read-write transaction is written into the persistent storage medium, namely, the version data row is determined. This ensures the correctness of the transaction and avoids reading data that has been deleted or not yet committed.
2. If the modification operation log of the read-write transaction corresponding to the target version data information is written into the persistent storage medium, correspondingly modifying the log persistence parameter of the modification operation log into a preset visible value; when the log persistence parameter is a preset visible value, the modified version data row in the target version data information is determined to be the visible version data row, so that the read data of the read-only transaction is correct. When the log persistence parameter of the target version data information is not a preset visible value, the read-write transaction corresponding to the target version data information is added into the dependent transaction set of the read-only transaction, so that a dependent relationship can be established, and the read-only transaction can be ensured to read the data submitted and updated by the dependent transaction. By recording the number of reads, the access condition of each version data line can be tracked for subsequent optimization and statistical analysis.
3. When detecting that a read-write transaction requests to modify a data row, creating version data information of the data row according to the read-write data row, wherein the version data information comprises the version data row needing modification, a version transaction identifier, the reading times, a transaction access interval and log persistence parameters. By setting the version transaction identifier to the transaction identifier of the read-write transaction that created the version data information, the source transaction for each version data information can be accurately tracked. By setting the initial value of the log persistence parameter to a preset invisible value, it can be ensured that the newly created version data is invisible to other transactions; only when the modification operation log of the read-write transaction is written into the persistent storage medium, namely, when the log persistence parameter of the version data information is modified to a preset visible value, other transactions can read the version data, so that the consistency of the transactions is ensured. Meanwhile, when the read-only transaction needs to access the version data, the data of the version can be determined to be safely read through the log persistence parameter, and the query data in the global transaction set is not needed, so that the expenditure of a database management system can be reduced.
Drawings
FIG. 1 is a flow chart of a memory database transaction management method according to an embodiment of the application;
FIG. 2 is a timeline diagram of a read-only transaction acquiring a read lock in an embodiment of the present application;
FIG. 3 is a schematic diagram of a memory database transaction management device according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The application is described in further detail below in connection with fig. 1-4.
Modifications of the embodiments which do not creatively contribute to the application may be made by those skilled in the art after reading the present specification, but are protected by patent laws only within the scope of the present application.
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
In addition, the term "and/or" herein is merely an association relationship describing an association object, and means that three relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone. In this context, unless otherwise specified, the term "/" generally indicates that the associated object is an "or" relationship.
The embodiment of the application provides a memory database transaction management method, which is executed by electronic equipment, and referring to fig. 1, the method comprises the steps of S101-S103, wherein:
step S101, when a read-only transaction request access data line is detected, a transaction identifier of the read-only transaction and a plurality of version data information corresponding to the data line are obtained, each version data information comprises a version data line, a version transaction identifier, reading times, a transaction access interval and a log persistence parameter, the transaction access interval is a transaction identifier interval of the read-only transaction capable of reading the version data line, and the log persistence parameter is used for representing whether a modification operation log of a read-write transaction creating the version data line is written into a persistent storage medium.
For embodiments of the present application, each transaction may be automatically assigned a unique transaction identifier in an in-memory database system so that the system can identify and track the execution and status of the transaction. At the same time, the memory database system records and associates the transaction identifier of the transaction, which may be a number, string or other type of identifier, which may be generated in different ways, depending on the requirements and design of the system. The manner in which the transaction identifier is generated may include: increment a counter: the database system may use an incremented counter to generate the transaction identifier, with the counter value incremented each time a new transaction is created, and the value taken as the identifier of the new transaction. Timestamp: the current time stamp is used as a transaction identifier, typically a string that converts the current time to a particular format or the time stamp is stored using a particular data type. Globally Unique Identifier (GUID): using the globally unique identifier as the transaction identifier, the GUID is a 128-bit unique identifier, typically generated using algorithms and system-specific information. The generation mode and the expression form of the transaction identifier are not particularly limited in the embodiment of the application, and only the corresponding transaction can be accurately represented.
Further, when the read-only transaction request access to the data line is detected, a transaction identifier of the read-only transaction is acquired, and meanwhile, a plurality of version data information corresponding to the data line is acquired according to the data line which is required to be accessed by the read-only transaction, wherein the version data information is version data information created by different read-write transactions for the data line before the read-only transaction request access to the data line is detected. Each version data information of the data line is associated with a transaction identifier of a read-write transaction for creating the version data information, an initial value of the version transaction identifier is the transaction identifier of the read-write transaction for creating the version data information, and when the read-write transaction completes the read-write operation, the corresponding version identifier is changed into a certain preset identification value, and the preset identification value is used for distinguishing the transaction identifier of the read-write transaction.
The user can know the number of read-only transactions currently reading the version data line according to the value corresponding to the number of read-out times, for example, the initial value of the number of read-out times is 0, and when a certain read-out transaction reads the version data, the number of read-out times is increased by 1; the corresponding number of reads is decremented by 1 when the read-only transaction completes a read.
The transaction access interval is used to characterize a transaction identifier interval of a read-only transaction that can read the version data line, wherein the lower threshold of the transaction access interval can be a transaction identifier of a read-write transaction that creates the version data information, i.e., an initial value of a version transaction identification. The upper threshold of the transaction access interval can be determined according to the requirement of a user and can also be determined according to the use condition of the next read-write transaction. The creation of a corresponding transaction is illustrated with a transaction identifier as a timestamp: the lower bound of the transaction access interval is the timestamp of the read-write transaction that created this version, while the upper bound of the transaction access interval may be the start timestamp of the next read-write transaction minus 1. Assuming that there are two transactions T1 and T2, T1 starts at a time stamp of 1000 and T2 starts at a time stamp of 2000; when T1 modifies a certain data line, a new version is created, the lower limit of the transaction access interval of the version is 1000, but the upper limit is set to 1999 (the start timestamp of T2 minus 1), so that the data modified by T1 is invisible to T2, and the transaction access intervals of the data information of the versions are not overlapped.
The log persistence parameters correspond to a set of numbers or characters, e.g., 0 and 1, false and true, etc. When the operation log of the read-write transaction is written into the persistent storage medium, the log persistence parameter is modified from an initial value to another non-initial value in the corresponding array or character set, for example, the initial value of the log persistence parameter is false, and when the operation data of the read-write transaction is written into the persistent storage medium, the operation data is true.
Step S102, determining target version data information from a plurality of version data information based on a transaction identifier of a read-only transaction and transaction access intervals corresponding to the plurality of version data information, wherein the target version data information is version data information appointed to be accessed by the read-only transaction;
for the embodiment of the application, when the read-only transaction starts to execute, the version of the data line to be read is specified through the transaction identifier, so that the target version data information can be determined from a plurality of version data information by judging the relation between the transaction identifier of the read-only transaction and the transaction access interval of each version data information, wherein the transaction identifier of the read-only transaction is in the transaction access interval of the target version data information, and the transaction access intervals of the version data information are mutually exclusive.
Step S103, determining visible version data lines based on the log persistence parameters of the target version data information.
For the embodiment of the application, the log persistence parameter of the current target version data information is determined, if the log persistence parameter is a certain preset value, the preset value indicates that the data modification operation corresponding to the target version data information is written into the persistent storage medium, and the modified version data line in the target version data information is determined to be the visible version data line which can be read by the read-only transaction.
Determining target version data information to be read by the read-only transaction from a plurality of version data information according to the transaction identifier of the read-only transaction; and judging whether the modification operation log of the read-write transaction is written into the persistent storage medium according to the log persistence parameter of the target version data information, wherein the corresponding version data row is regarded as visible only when the modification operation log of the read-write transaction is written into the persistent storage medium, namely, the version data row is determined. This ensures the correctness of the transaction and avoids reading data that has been deleted or not yet committed.
Further, based on the log persistence parameter of the target version data line, a visible version data line is determined, including step S1031 (not shown in the figure) -step S1033 (not shown in the figure), wherein:
step S1031, after the modification operation log of the read-write transaction corresponding to the target version data information is written into the persistent storage medium, modifying the log persistence parameter in the version data information to a preset visible value.
Specifically, after the operation log of the read-write transaction corresponding to the target version data information is written into the persistent storage medium (log landing), the log persistence parameter in the target version data information needs to be modified from an initial value to a preset visible value. The preset visible value of the log persistence parameter is different from the initial value, and whether the corresponding read-write transaction finishes log landing or not can be known through reading the log persistence parameter of the version data information. For example, when the read-write transaction a creates the version data information X, the log persistence parameter is an initial value false, and after the operation log of the read-write transaction a finishes log landing, the log persistence parameter is modified to a preset visible value true.
Further, step S1031 may be performed before any of steps S101 to S103, step S1031 may be performed after any of steps S101 to S103, step S1031 may be performed simultaneously with any of steps S101 to S103, and the log persistence parameter in the version data information may be modified to a preset visible value as long as the modification operation log of the read/write transaction corresponding to the version data information has been written into the persistent storage medium. Step S1032 and step S1033 need to be performed after step S102, and if step S1031 is performed before determining the target version data information, step S1032 is continued; if step S1031 has not been performed before the determination of the target version data information, S1033 is continued.
S1032, when the log persistence parameter of the target version data information is a preset visible value, determining the version data line in the target version data information as a visible version data line, and modifying the reading times in the target version data information;
step S1033, when the log persistence parameter of the target version data information is not a preset visible value, adding the read-write transaction corresponding to the target version data information into a dependent transaction set of the read-only transaction, determining visible version data lines based on the dependent transaction set, and modifying the reading times in the target version data information.
Specifically, when the log persistence parameter of the target version data information is a preset visible value, determining a version data behavior visible version data row corresponding to the target version data information. When the log persistence parameter of the target version data information is not a preset visible value, in order to ensure that the read-only transaction can read the correct version data, a read-write transaction corresponding to the target version data information needs to be added into the self-dependent transaction set of the read-only transaction. And then, by a dependency tracking method, monitoring and analyzing the transactions in the dependency transaction set until the log persistence parameter of each dependent read-write transaction in the dependency transaction set becomes a preset visible value, so as to obtain a visible version data row. After determining the visible version data row corresponding to the read-only transaction, the reading times in the target version data information to which the visible version data row belongs need to be modified, for example, the value of the current reading times is +1.
Further, adding the read-write transaction corresponding to the target version data information to the dependent transaction set of the read-only transaction includes a step SA1 (not shown in the figure) -a step SA2 (not shown in the figure), wherein:
Step SA1, determining a transaction identifier of a read-write transaction corresponding to target version data information based on a transaction access interval of the target version data information;
and step SA2, adding the read-write transaction corresponding to the target version data information into a dependent transaction set of the read-write transaction based on the transaction identifier of the read-write transaction corresponding to the target version data information.
Specifically, when creating the transaction access interval of the target version data information, the lower limit of the transaction access interval is the transaction identifier of the read-write transaction that creates the target version data information. The transaction identifier of a read-write transaction that requires dependency tracking can thus be determined by the transaction access interval. And then adding the read-write transaction corresponding to the target version data information into a dependent transaction set of the read-only transaction, so as to realize dependency tracking.
Further, before the plurality of version data information corresponding to the data line is obtained, step S01 is further included, when the read-write transaction request for modifying the data line is detected, version data information of the data line is created based on the read-write transaction, the version data information includes a version data line, a version transaction identifier, a reading number, a transaction access interval and a log persistence parameter, an initial value of the version transaction identifier is a transaction identifier of the read-write transaction for creating the version data information, and an initial value of the log persistence parameter is a preset invisible value.
For the embodiment of the application, the Dolphin DB memory database adopts a concurrency control protocol for managing the concurrency access of the transaction in the database system, and the concurrency execution and isolation of the transaction are realized by using a multi-version concurrency control mechanism and a two-stage locking protocol. When a read-write transaction needs to modify a data line, a write lock corresponding to the data line is requested to be acquired. If it is detected that the read-write transaction requests to modify the data line, before the read-write transaction starts to modify the data line, version data information of the data line is created according to the read-write transaction, and the version data information contains data of the data line, namely version data line, which needs to be modified by the read-write transaction. The version data information also includes a version transaction identification, a number of read-only transactions currently reading the version data line, a transaction identifier interval (transaction access interval) of the read-only transactions that can read the version data line, and a log persistence parameter for characterizing whether operations of the read-write transactions complete a log landing. The initial value of the version transaction identifier is a transaction identifier of a read-write transaction for creating the version data information; the initial value of the log persistence parameter is a preset invisible value, and the preset invisible value is different from the preset visible value and is used for representing different log landing states.
Further, after creating version data information of the data line, step S02 (not shown in the figure) -step S04 (not shown in the figure) is further included, wherein:
step S02, determining that the read-write transaction is in a write lock binding state;
and S03, after the read-write transaction completes the modification operation of the data line data, determining that the read-write transaction is in a write lock release state, and modifying the version transaction identification in the version data information corresponding to the read-write transaction into a preset acquisition lock value.
For the embodiment of the application, before starting to modify a data line, a read-write transaction may need to acquire a write Lock (Exclusive Lock) to protect the modified data line, i.e., determine that the read-write transaction is in a write Lock binding state. Each transaction will get a view (snapshot) at the beginning that represents the consistent state of the database at the beginning of the transaction. When a read-write transaction is to modify a data line, it can determine whether the data line can be modified according to the information of the view, if the view of the read-write transaction does not conflict with the versions of other concurrent read-write transactions, the read-write transaction can generate a new version and acquire a write lock of the data line for writing. After a read-write transaction successfully obtains a write lock for a data line, it may perform modification operations on the data line, such as updating, inserting, and deleting operations. Once a read-write transaction completes a modify operation on a data line, it needs to release the held write lock, i.e., modify the state of the read-write transaction to a write lock release state.
Further, after the read-write transaction corresponding to the version data information finishes the modification operation to release the write lock, that is, when the read-write transaction corresponding to the version data information is in a write lock release state, the version transaction identifier in the version data information needs to be modified into a preset acquired lock value. The version transaction identifier can be used for judging whether the read-write transaction corresponding to the version data information to which the version transaction identifier belongs completes the operation or not, the preset acquisition lock value is different from the transaction identifier of the read-write transaction, and when the version transaction identifier is the preset acquisition lock value (the state of the read-write transaction is the write lock release state), other read-only transactions can acquire the read lock of the data line. For example, if the transaction identifier of the transaction is a timestamp when the transaction identifier is created, the preset acquisition lock value is 0, and the transaction identifier of the read-write transaction a is 100, the initial value of the version transaction identifier is 100, and after the read-write transaction a finishes modifying operation to release the write lock, the initial value of the version transaction identifier is modified to 0.
Step S04, writing a modification operation log of the read-write transaction into the persistent storage medium, wherein the modification operation log comprises modification operation of the read-write transaction on the data line.
For the embodiment of the application, in order to persist the operation of the read-write transaction and ensure the ACID attribute (atomicity, consistency, isolation and durability), the operation record of the read-write transaction is generally written into a persistent storage medium, such as a disk or a log file, so that the modification operation of the data line can be recovered even if the system fault occurs. Writing operation records of read-write transactions to persistent storage is a waiting log-drop process involving disk I/O operations in a "waiting log-drop" phase, and this phase typically requires a "fsync" call in order to guarantee absolute security of the data. "fsync" is a synchronous operation in a file system to ensure that data in a buffer or memory can be persisted to a persistent storage medium (disk), while the overhead of "fsync" is very large, typically taking several milliseconds for a mechanical hard disk, even hundreds of microseconds for a solid state hard disk. In order to improve the throughput of the transaction, the application needs to release the write lock before the log of the read-write transaction is dropped, as shown in fig. 2, and the concurrent read-only transaction can normally work when the log of the previous read-write transaction waits for the drop.
Further, after determining the target version data information from the plurality of version data information based on the transaction access interval corresponding to the transaction identifier of the read-only transaction and the plurality of version data information, the method further includes a step SB1 (not shown in the figure) -a step SB2 (not shown in the figure), wherein:
step SB1, obtaining version transaction identification of target version data information;
step SB2, when the version transaction identification of the target version data information is the preset acquisition lock value, determining that the read-only transaction is in a read lock binding state.
For the embodiment of the application, after the target version data information is determined from a plurality of version data information, the version transaction identification of the target version data information is acquired. When the version transaction identification of the target version data information is a preset acquisition lock value, the version transaction identification means that other transactions can read the version data line, and the read-only transaction is determined to be in a read lock binding state.
And judging whether the version data line in the target version data information is modified or not through the version transaction identification of the target version data information, and if the version data line in the target version data information is modified (the version transaction identification is a preset acquisition lock value), acquiring a corresponding read lock from a read-only transaction accessing the version data, namely, the read-only transaction is in a read lock binding state. The read lock binding state means that read-only transactions can concurrently read the target version data information without collision with write transactions. Thus, the concurrency performance can be improved, a plurality of read-only transactions are allowed to read the target version data information at the same time, and the throughput of the system is increased.
The above embodiments describe a method for managing memory database transactions from the perspective of a method flow, and the following embodiments describe an apparatus for managing memory database transactions from the perspective of a virtual module or a virtual unit, which are specifically described in the following embodiments.
The embodiment of the present application provides a device for managing a memory database transaction, as shown in fig. 3, the device for managing a memory database transaction may specifically include a transaction information and version data information acquisition module 201, a target version data information determination module 202, and a visible version data row determination module 203, where:
the transaction information and version data information obtaining module 201 is configured to obtain, when a read-only transaction request access data line is detected, a transaction identifier of the read-only transaction and a plurality of version data information corresponding to the data line, where each version data information includes a version data line, a version transaction identifier, a reading number, a transaction access interval, and a log persistence parameter, the transaction access interval is a transaction identifier interval of the read-only transaction that can read the version data line, and the log persistence parameter is used to characterize whether a modification operation log of a read-write transaction for creating the version data line is written in a persistent storage medium;
The target version data information determining module 202 is configured to determine target version data information from the plurality of version data information based on the transaction identifier of the read-only transaction and the transaction access interval corresponding to each of the plurality of version data information, where the target version data information is version data information that is appointed to be accessed by the read-only transaction;
the visible version data line determining module 203 is configured to determine a visible version data line based on the log persistence parameter of the target version data information.
By adopting the technical scheme, the target version data information which is required to be read by the read-only transaction is determined from the plurality of version data information according to the transaction identifier of the read-only transaction; and judging whether the modification operation log of the read-write transaction is written into the persistent storage medium according to the log persistence parameter of the target version data information, wherein the corresponding version data row is regarded as visible only when the modification operation log of the read-write transaction is written into the persistent storage medium, namely, the version data row is determined. This ensures the correctness of the transaction and avoids reading data that has been deleted or not yet committed.
In one possible implementation, the visible version data line determining module 203 is specifically configured to, when determining the visible version data line based on the log persistence parameter of the target version data line:
After the modification operation log of the read-write transaction corresponding to the target version data information is written into the persistent storage medium, modifying the log persistence parameter in the version data information into a preset visible value;
when the log persistence parameter of the target version data information is a preset visible value, determining the version data row in the target version data information as a visible version data row, and modifying the reading times in the target version data information;
when the log persistence parameter of the target version data information is not a preset visible value, adding the read-write transaction corresponding to the target version data information into a dependent transaction set of the read-only transaction, determining visible version data lines based on the dependent transaction set, and modifying the reading times in the target version data information.
In one possible implementation manner, when the visible version data line determining module 203 adds the read-write transaction corresponding to the target version data information to the dependent transaction set of the read-only transaction, the visible version data line determining module is specifically configured to:
determining a transaction identifier of a read-write transaction corresponding to the target version data information based on the transaction access interval of the target version data information;
and adding the read-write transaction corresponding to the target version data information into the dependent transaction set of the read-only transaction based on the transaction identifier of the read-write transaction corresponding to the target version data information.
In one possible implementation manner, an in-memory database transaction management device further includes:
the version data information creation module is used for creating version data information of the data line based on the read-write transaction when detecting that the read-write transaction requests to modify the data line, wherein the version data information comprises a version data line, a version transaction identifier, the reading times, a transaction access interval and a log persistence parameter, the initial value of the version transaction identifier is a transaction identifier of the read-write transaction for creating the version data information, and the initial value of the log persistence parameter is a preset invisible value.
In one possible implementation manner, an in-memory database transaction management device further includes:
the write lock binding state determining module is used for determining that the read-write transaction is in a write lock binding state;
the write lock release state determining module is used for determining that the read-write transaction is in a write lock release state after the read-write transaction completes the modification operation on the data of the data row, and modifying the version transaction identifier in the version data information corresponding to the read-write transaction into a preset acquisition lock value;
the log landing module is used for writing a modification operation log of the read-write transaction into the persistent storage medium, wherein the modification operation log comprises modification operation of the read-write transaction on the data line.
In one possible implementation manner, an in-memory database transaction management device further includes:
the version transaction identifier acquisition module is used for acquiring the version transaction identifier of the target version data information;
and the read lock binding state determining module is used for determining that the read-only transaction is in a read lock binding state when the version transaction identifier of the target version data information is a preset acquisition lock value.
In an embodiment of the present application, as shown in fig. 4, an electronic device 300 shown in fig. 4 includes: a processor 301 and a memory 303. Wherein the processor 301 is coupled to the memory 303, such as via a bus 302. Optionally, the electronic device 300 may also include a transceiver 304. It should be noted that, in practical applications, the transceiver 304 is not limited to one, and the structure of the electronic device 300 is not limited to the embodiment of the present application.
The processor 301 may be a CPU (Central Processing Unit ), general purpose processor, DSP (Digital Signal Processor, data signal processor), ASIC (Application Specific Integrated Circuit ), FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic device, transistor logic device, hardware components, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules and circuits described in connection with this disclosure. Processor 301 may also be a combination that implements computing functionality, e.g., comprising one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.
Bus 302 may include a path to transfer information between the components. Bus 302 may be a PCI (Peripheral Component Interconnect, peripheral component interconnect Standard) bus or an EISA (Extended Industry Standard Architecture ) bus, or the like. Bus 302 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in fig. 4, but not only one bus or one type of bus.
The Memory 303 may be, but is not limited to, a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory ) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory ), a CD-ROM (Compact Disc Read Only Memory, compact disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.
The memory 303 is used for storing application program codes for executing the inventive arrangements and is controlled to be executed by the processor 301. The processor 301 is configured to execute the application code stored in the memory 303 to implement what is shown in the foregoing method embodiments.
Among them, electronic devices include, but are not limited to: mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and stationary terminals such as digital TVs, desktop computers, and the like. But may also be a server or the like. The electronic device shown in fig. 4 is only an example and should not be construed as limiting the functionality and scope of use of the embodiments of the application.
Embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, which when run on a computer, causes the computer to perform the corresponding method embodiments described above.
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.
The foregoing is only a partial embodiment of the present application, and it should be noted that it will be apparent to those skilled in the art that modifications and adaptations can be made without departing from the principles of the present application, and such modifications and adaptations are intended to be comprehended within the scope of the present application.

Claims (9)

1. A memory database transaction management method, comprising:
when a read-only transaction request access data line is detected, acquiring a transaction identifier of the read-only transaction and a plurality of version data information corresponding to the data line, wherein each version data information comprises a version data line, a version transaction identifier, reading times, a transaction access interval and a log persistence parameter, the transaction access interval is a transaction identifier interval of the read-only transaction capable of reading the version data line, and the log persistence parameter is used for representing whether a modification operation log of a read-write transaction creating the version data line is written into a persistent storage medium;
determining target version data information from the plurality of version data information based on a transaction identifier of the read-only transaction and transaction access intervals corresponding to the plurality of version data information, wherein the target version data information designates accessed version data information for the read-only transaction;
And determining visible version data lines based on the log persistence parameters of the target version data information.
2. The method of claim 1, wherein determining visible version data lines based on log persistence parameters of the target version data lines comprises:
after a modification operation log of a read-write transaction corresponding to the target version data information is written into a persistent storage medium, modifying a log persistence parameter in the version data information into a preset visible value;
when the log persistence parameter of the target version data information is the preset visible value, determining the version data line in the target version data information as a visible version data line, and modifying the reading times in the target version data information;
when the log persistence parameter of the target version data information is not the preset visible value, adding the read-write transaction corresponding to the target version data information into a dependent transaction set of the read-only transaction, determining visible version data lines based on the dependent transaction set, and modifying the reading times in the target version data information.
3. The method for managing memory database transactions according to claim 2, wherein adding the read-write transaction corresponding to the target version data information to the dependent transaction set of the read-only transaction includes:
determining a transaction identifier of a read-write transaction corresponding to the target version data information based on the transaction access interval of the target version data information;
and adding the read-write transaction corresponding to the target version data information into a dependent transaction set of the read-only transaction based on the transaction identifier of the read-write transaction corresponding to the target version data information.
4. The method for managing memory database transactions according to claim 1, further comprising, before obtaining the version data information corresponding to the data line:
when the read-write transaction request is detected to modify the data row, based on the read-write transaction, version data information of the data row is created, wherein the version data information comprises a version data row, a version transaction identifier, reading times, a transaction access interval and a log persistence parameter, the initial value of the version transaction identifier is a transaction identifier of the read-write transaction for creating the version data information, and the initial value of the log persistence parameter is a preset invisible value.
5. The method of claim 4, further comprising, after creating version data information for the data line:
determining that the read-write transaction is in a write lock binding state;
after the read-write transaction completes the modification operation on the data of the data line, determining that the read-write transaction is in a write lock release state, and modifying a version transaction identifier in version data information corresponding to the read-write transaction into a preset acquisition lock value;
writing a modification operation log of the read-write transaction to a persistent storage medium, the modification operation log comprising modification operations of the read-write transaction to the data line.
6. The method of claim 5, further comprising, after determining the target version data information from the plurality of version data information based on the transaction identifier of the read-only transaction and the transaction access interval corresponding to each of the plurality of version data information:
a version transaction identifier of the target version data information is obtained;
and when the version transaction identifier of the target version data information is a preset acquisition lock value, determining that the read-only transaction is in a read lock binding state.
7. A memory database transaction management device, comprising:
the system comprises a transaction information and version data information acquisition module, a data storage module and a data storage module, wherein the transaction information and version data information acquisition module is used for acquiring a transaction identifier of a read-only transaction and a plurality of version data information corresponding to the data row when detecting that the read-only transaction requests to access the data row, each version data information comprises a version data row, a version transaction identifier, reading times, a transaction access interval and a log persistence parameter, the transaction access interval is a transaction identifier interval of the read-only transaction capable of reading the version data row, and the log persistence parameter is used for representing whether a modification operation log of a read-write transaction creating the version data row is written into a persistent storage medium;
the target version data information determining module is used for determining target version data information from the plurality of version data information based on the transaction identifier of the read-only transaction and the transaction access interval corresponding to the plurality of version data information, wherein the target version data information is version data information appointed for access by the read-only transaction;
and the visible version data row determining module is used for determining visible version data rows based on the log persistence parameters of the target version data information.
8. An electronic device, comprising:
at least one processor;
a memory;
at least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, the at least one application configured to: performing the in-memory database transaction management method of any one of claims 1-6.
9. A computer-readable storage medium, comprising: a computer program stored with instructions capable of being loaded by a processor and executing the method of memory database transaction management as claimed in any one of claims 1 to 6.
CN202311063933.0A 2023-08-22 2023-08-22 Memory database transaction management method and device, electronic equipment and medium Pending CN117056313A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311063933.0A CN117056313A (en) 2023-08-22 2023-08-22 Memory database transaction management method and device, electronic equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311063933.0A CN117056313A (en) 2023-08-22 2023-08-22 Memory database transaction management method and device, electronic equipment and medium

Publications (1)

Publication Number Publication Date
CN117056313A true CN117056313A (en) 2023-11-14

Family

ID=88669057

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311063933.0A Pending CN117056313A (en) 2023-08-22 2023-08-22 Memory database transaction management method and device, electronic equipment and medium

Country Status (1)

Country Link
CN (1) CN117056313A (en)

Similar Documents

Publication Publication Date Title
US7032229B1 (en) Automatic tracking of user progress in a software application
US8161128B2 (en) Sharing of data across disjoint clusters
CN109471851B (en) Data processing method, device, server and storage medium
US9542235B2 (en) Process-safe read/write locks
CN110865888A (en) Resource loading method and device, server and storage medium
JP5435741B2 (en) Using mold-fixability to facilitate conflict management
US20120059997A1 (en) Apparatus and method for detecting data race
CN109800062B (en) Distributed database transaction processing system
CN116467975B (en) Data processing method, device, electronic equipment and storage medium
CN117112522A (en) Concurrent process log management method, device, equipment and storage medium
US20090210617A1 (en) Multi-level volume table of contents
US7657664B2 (en) Method and system for tracking device driver requests
CN117056313A (en) Memory database transaction management method and device, electronic equipment and medium
US20060149696A1 (en) Method and systems for controlling access to a data object by means of locks
CN115629822A (en) Concurrent transaction processing method and system based on multi-core processor
CN115454570A (en) Disaster recovery method, virtual machine system, device, and storage medium
CN109791541B (en) Log serial number generation method and device and readable storage medium
CN112632211A (en) Semantic information processing method and equipment for mobile robot
CN110750569A (en) Data extraction method, device, equipment and storage medium
CN115237875B (en) Log data processing method, device, equipment and storage medium
US10776344B2 (en) Index management in a multi-process environment
CN112364040B (en) Data checking method, device, medium and electronic equipment
US20230107071A1 (en) Asynchronous transaction conflict resolution
CN115617464A (en) Data concurrent access control method and device, electronic equipment and storage medium
CN116774906A (en) Disk data deleting method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination