WO2022135471A1 - 多版本并发控制和日志清除方法、节点、设备和介质 - Google Patents

多版本并发控制和日志清除方法、节点、设备和介质 Download PDF

Info

Publication number
WO2022135471A1
WO2022135471A1 PCT/CN2021/140468 CN2021140468W WO2022135471A1 WO 2022135471 A1 WO2022135471 A1 WO 2022135471A1 CN 2021140468 W CN2021140468 W CN 2021140468W WO 2022135471 A1 WO2022135471 A1 WO 2022135471A1
Authority
WO
WIPO (PCT)
Prior art keywords
gtid
global
node
statement
computing node
Prior art date
Application number
PCT/CN2021/140468
Other languages
English (en)
French (fr)
Inventor
任济双
徐文锋
郭龙波
刘志文
付裕
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2022135471A1 publication Critical patent/WO2022135471A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/219—Managing data history or versioning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2308—Concurrency control
    • G06F16/2315—Optimistic concurrency control
    • G06F16/2329—Optimistic concurrency control using versioning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Definitions

  • the present application relates to the field of communication technologies, and in particular to a method, node, device and medium for multi-version concurrency control and log clearing.
  • Multiversion concurrency control refers to processing the memory read by transactions when accessing the database concurrently, avoiding mutual blocking of reads and writes, and finally realizing that the content read by different transactions at the same time may be different. Function.
  • MVCC is mainly implemented on a single machine.
  • multi-version concurrency control is realized by combining the two-phase commit method.
  • an update statement is issued to multiple shards, some shard operation instructions are executed successfully, and the other shards are still being executed.
  • the database receives another query statement, then there are Some shards return the result before commit, and some return the result after commit, then the returned result is unreliable and needs to be discarded; the possible approach is to repeat the query continuously until multiple shards return the operation command execution success As a result, the results of the query will be returned to the client. This approach seriously affects the throughput performance in high concurrency scenarios.
  • an embodiment of the present application provides a multi-version concurrency control method, comprising: a computing node sending a first statement to a database node, wherein the first statement carries a global list of active things; the database node obtains the the data snapshot corresponding to the first statement, and determine the return result based on the data snapshot and the global active thing list; the database node sends the return result to the computing node.
  • an embodiment of the present application provides a method for clearing logs, including: a computing node sends a global transaction identifier (GTID) and a clearing strategy to a database node; the database node, based on the clearing strategy, will The undo log corresponding to the GTID of the GTID mentioned above is cleared.
  • GTID global transaction identifier
  • an embodiment of the present application provides a computing node, including: a first sending module configured to send a first statement to a database node, wherein the first statement carries a global active transaction list; a first receiving module , is configured to receive a return result returned by the database, wherein the returned result is obtained by the database node of the data snapshot corresponding to the first statement, and is determined based on the data snapshot and the global active transaction list.
  • an embodiment of the present application provides a database node, including: a second receiving module configured to receive a first statement sent by a computing node, wherein the first statement carries a global list of active things; the returned result is determined a module, configured to obtain a data snapshot corresponding to the first statement, and determine a return result based on the data snapshot and the global active thing list; a second sending module, configured to send the return result to the calculate node.
  • an embodiment of the present application provides a computing node, including: a third sending module configured to send a GTID and a clearing policy to a database node, where the clearing policy is used to instruct the database node to send a GTID smaller than the GTID to the database node The corresponding undo log is cleared.
  • an embodiment of the present application provides a database node, including: a third receiving module, configured to receive a GTID and a clearing policy sent by a computing node; a log clearing module, configured to The undo log corresponding to the GTID of the GTID is cleared.
  • embodiments of the present application provide an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are processed by the one or more programs The processor executes, so that the one or more processors implement the method according to any one of the embodiments of the present application.
  • an embodiment of the present application provides a storage medium, where the storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of the embodiments of the present application is implemented.
  • FIG. 1 is a flowchart of a multi-version concurrency control method provided by an embodiment of the present application
  • FIG. 2 is a flowchart of a method for clearing a log provided by an embodiment of the present application
  • FIG. 3 is a schematic structural diagram of a distributed database with a single PROXY, multiple GTMs, and multiple cluster structures provided by the present embodiment
  • FIG. 4 is a schematic structural diagram of a multi-version concurrency control device provided by an embodiment of the present application.
  • a multi-version concurrency control method is provided. As shown in FIG. 1 , the multi-version concurrency control method provided by this embodiment mainly includes steps S11 , S12 and S13 .
  • the computing node sends a first statement to the database node, where the first statement carries a global active thing list.
  • the database node acquires a data snapshot corresponding to the first statement, and determines a return result based on the data snapshot and the global active transaction list.
  • the database node sends the returned result to the computing node.
  • the first statement may be an SQL query statement carrying a list of globally active things, or may be other statements carrying a global active thing, which is not limited in this embodiment.
  • the currently active global transaction identifier GTID is stored in the global active transaction list.
  • the computing node PROXY when the computing node PROXY sends a query statement, it sends the global active transaction list to the DB (database, database) together, and makes judgments at the DB layer, thereby realizing the multi-version concurrency control of the distributed database and reducing duplication. query, which improves the high-throughput performance of multi-concurrency scenarios.
  • the database node determines the return result based on the data snapshot and the global active transaction list, including determining a global transaction identifier GTID in the data snapshot; using the GTID as a keyword in the global transaction Query the list of active things; determine the returned result based on the query result.
  • the database node obtains a data snapshot based on other statements in the first statement except for the global active transaction list, for example: determining and isolation level based on the SQL query statement, and then querying and obtaining the GTID in the data snapshot to query
  • the obtained GTID is the keyword to query in the global active transaction list.
  • the database node determines the return result based on the query result, including: if the GTID is queried in the global active thing list, rollback is performed based on the undo log log, and the rollback result is determined as the return result; If the GTID is not found in the global active transaction list, the data snapshot is determined as the returned result.
  • the GTID is queried in the global active transaction list, it means that there are other database nodes in the entire storage engine that are executing the transaction corresponding to the GTID, that is, the transaction corresponding to the GTID is active, then based on the undo log The log is rolled back, and the rollback result is determined as the return result.
  • the GTID is not queried in the global active transaction list, it means that no other database node in the entire storage engine is executing the transaction corresponding to the GTID, and the data snapshot is determined as the returned result.
  • the database node DB determines whether the returned result needs to be rolled back according to whether the transaction is active, thus solving the problem of inaccurate results returned by multiple aggregate functions in the distributed database and the performance of repeated queries caused by incorrect return results. The problem.
  • the method before the computing node sends the first statement to the database node, the method further includes: after the computing node receives the second statement, sending a batch processing sequence request to the global transaction management node, where the batch processing sequence request is used for requesting a global active transaction list; the global transaction management node sends the global active transaction list to the computing node based on a batch processing sequence request; the computing node adds the global active transaction list to the second statement to obtain the first statement .
  • the second statement refers to the SQL statement sent by the application and received by the computing node, which does not carry the global active transaction list.
  • the second statement forms the first statement after carrying the global list of active things.
  • the global transaction management node when the second statement is a query statement, the global transaction management node returns the global active transaction list to the computing node.
  • the global transaction management node when the second statement is a new transaction request statement, the global transaction management node generates a new GTID and returns it to the computing node. It should be noted that only one transaction applies for one GTID, and one transaction can contain multiple statements.
  • the second statement is a statement included in the current transaction, and the second statement may be one statement or multiple statements, which is not limited in this implementation.
  • the global transaction management node when the second statement is not related to the GTID, the global transaction management node directly sends the queried global active transaction list to the computing node.
  • the global transaction management node updates the global active transaction list based on the GTID, and then sends the updated global active transaction list to the computing node.
  • sending a batch processing sequence request to the global transaction management node includes: after the computing node receives the second statement, parsing the second statement, The obtained DDL (data manipulation language, data definition language) and DML (data definition language, data manipulation language), for DDL and DML, send batch processing sequence requests to the global transaction management node of the corresponding cluster through the merge thread.
  • DDL data manipulation language, data definition language
  • DML data definition language, data manipulation language
  • the computing node PROXY will cluster and merge a large number of GTID processing requirements in a short period of time, and send them to GTM (Global Traffic Manager, global traffic management) for batch processing, which reduces the need for multiple processing threads to independently send GTID processing requests The resulting message redundancy and duplication of processing.
  • GTM Global Traffic Manager, global traffic management
  • the GTM is a global transaction management node
  • each GTM cluster serves one or more business clusters
  • the GTM cluster can be expanded according to performance or high availability requirements
  • each GTM host creates an independent thread for each business cluster, Ensure the isolation and efficiency of each cluster and database during transaction processing, so as to realize the isolation of transactions at the cluster level and the uniqueness within the cluster.
  • Single-function, multi-threaded concurrent processing greatly improves the efficiency of global transaction processing and ensures the high performance of the distributed database system.
  • each transaction must go to the GTM to apply for a GTID, and then combine it into the first statement and send it to the database node;
  • the active transaction list was not included before. In this embodiment, the active transaction list is added to form the first statement.
  • the multi-version concurrency control method, storage engine, device, and storage medium include: a computing node sends a first statement to a database node, where the first statement carries a global list of active things; and the database node obtains the first statement corresponding to the The data snapshot, and the return result is determined based on the data snapshot and the global active transaction list, and the database node sends the returned result to the computing node.
  • the technical solution of this embodiment sends the global active transaction list to the database node together, and judges at the database node. , thereby realizing multi-version concurrency control of distributed databases, reducing repeated queries, and improving high-throughput performance in multi-concurrency scenarios.
  • a log clearing method is provided. As shown in FIG. 2 , the log clearing method provided by this embodiment mainly includes steps S21 and S22.
  • the computing node sends the GTID and the clearing policy to the database node;
  • the database node clears the undo log log corresponding to the GTID smaller than the GTID based on the clearing strategy.
  • it further includes: the computing node sends a GTID application request to the global transaction management node; the global transaction management node generates a new GTID based on the GTID application request, and establishes the new GTID in a preset mapping table Correspondence to the smallest active GTID.
  • the computing node before the computing node sends the minimum GTID and the clearing policy to the database node, it further includes: the computing node sends a GTID release request to the global transaction management node; the global transaction management node deletes the GTID release request The GTID corresponding to the global active transaction list is requested; the global transaction management node obtains the GTID corresponding to the minimum active GTID in the preset mapping table, and sends it to the computing node.
  • the mapping relationship between the applied GTID and the smallest active GTID at this time needs to be recorded, so as to find the smallest active GTID returned to the computing node when the GTM releases the GTID.
  • the computing node PROXY controls the database node DB to carry out the purge through a unique purge strategy, and the database node DB clears the data copies smaller than the minimum GTID in the undo log according to the minimum global GTID.
  • the most important thing in the de-redundancy design of the undo log is to prevent the data from being unable to be rolled back due to the self-purging of the DB.
  • a method for controlling multi-version concurrency in a distributed database is provided, which is mainly used to implement a high-throughput implementation method, which mainly includes:
  • Multiple processing threads within PROXY analyze the SQL statements from the application after they receive them. For all DDL and DML involving GTIDs, batch processing sequence requests are sent to the GTM of the corresponding cluster through the merged thread. Using merged threads to send batched sequential requests can reduce message redundancy and performance degradation caused by multiple requests.
  • each GTM service includes one or more business clusters, which are stored in the metadata table in the form of metadata.
  • PROXY starts, the metadata information is obtained, and the message is pushed to PROXY when the GTM service cluster changes. Make sure PROXY knows which business clusters each GTM cluster serves.
  • the main GTM merges the processing results of multiple threads and sends it to the standby GTM for data synchronization. After the main GTM receives the response that the standby GTM has successfully synchronized, the main GTM returns the processing result to PROXY.
  • Multiple persistent threads within the GTM cluster use shared memory to regularly perform full backups of globally active GTIDs to ensure that data integrity is not lost.
  • the multi-copy architecture of a GTM cluster with one master and multiple slaves ensures that when the master GTM goes down, the slave GTM has complete global active GTID information and can take over the master's GTID function.
  • the GTM When the GTM receives the applied GTID, it stores the smallest global active GTID in the current active GTID list.
  • the GTM When the GTM receives the released GTID, it queries the current minimum active GTID corresponding to the global minimum active GTID stored at the time of its application, and returns it to PROXY.
  • PROXY executes a query or aggregate function, it sends the global list of active transactions to the DB.
  • the above method can better solve the multi-version concurrency control in the distributed database, improve the throughput performance, and especially solve the problem of inconsistent results of multiple queries of aggregate functions.
  • this technology can improve the throughput performance of high-concurrency scenarios and improve the functions of distributed databases.
  • PROXY processes the thread parsing statement, sorts the messages, sends them to the GTM of the corresponding cluster, and queries the global active transaction list at this time.
  • the main GTM parses the query request message and returns the global active transaction list of the corresponding cluster to PROXY; the global active transaction list stores the GTIDs of all current active transactions.
  • the Proxy reorganizes the SQL statement and sends the query statement and the global active transaction list to the DB;
  • the DB node first obtains a data snapshot according to the SQL statement and isolation level (such as RC); then queries whether the GTID in the data snapshot is in the active transaction list; if it is in the active transaction list, it combines the undo log to roll back the data. The result is returned to PROXY; if it is not in the global active transaction list, the retrieved data snapshot is returned to PROXY;
  • isolation level such as RC
  • PROXY integrates the returned results on different shards and returns it to the final result of the application.
  • a purge strategy is provided.
  • DB After DB receives the query or aggregation function sent by PROXY, it obtains a data snapshot and compares it with the global active transaction list. If the GTID of the queried transaction is not in the active transaction list, it returns the currently obtained snapshot result to PROXY; if the current transaction If the GTID is still in the global active transaction list, the rolled back result will be returned to PROXY in combination with the undo log.
  • PROXY After PROXY sends the release GTID to GTM, it obtains the minimum GTID at this time, sends it to the DB, executes the purge policy, and clears the undo log before the minimum GTID.
  • the master GTM When the master GTM receives the GTID request from PROXY, it generates a new GTID, stores the newly generated GTID and the smallest global active transaction in the mapping table, synchronizes it to the slave GTM, and finally returns the newly generated GTID to PROXY. ;
  • the master GTM When the master GTM receives a request to release a certain GTID, it synchronously deletes the GTID in the global active transaction list of the slave machine, takes the GTID in the mapping table corresponding to the smallest GTID in the global active transaction list at this time, and returns it to PROXY for processing. undo log purge;
  • PROXY sends a purge policy to DB, and sends the GTID in the mapping table returned by GTM to DB at the same time, and DB clears the undo log before this GTID and returns the execution result to PROXY;
  • PROXY After PROXY receives the replies from all DB nodes, it completes the purge.
  • FIG. 3 is a schematic structural diagram of a distributed database with a single PROXY, multiple GTMs, and multiple clusters provided in this embodiment; as shown in FIG. 3, a distributed database with a single PROXY, multiple GTMs, and multiple clusters includes: application nodes, The computing node PROXY, the database node DB, and the global transaction management node GTM.
  • the global transaction management node GTM is mainly used to manage global transactions in the entire distributed database system, including assigning GTIDs to each transaction at the beginning of the transaction, managing the global active GTID list, and releasing GTIDs after the transaction ends.
  • GTM is generally a multi-copy high-availability cluster architecture composed of one master and multiple slaves, which ensures that when the master is down, there are multiple available slaves that hold complete sequence copies to take over the master and maintain the normal use of database functions.
  • a GTM cluster corresponds to one or more business clusters, which ensures the high performance of transaction processing on the cluster. At the same time, multiple GTM clusters ensure the isolation of transaction processing between clusters and better disaster tolerance.
  • the computing node PROXY is mainly used to parse the SQL sent by the application, and initiate application, release, and query GTID processing requests to the GTM. After receiving the processing result of the GTID, the modified SQL statement is sent to the database node for execution.
  • PROXY is bound to the cluster by connecting an instance, and establishes a link with the GTM host that processes the GTID of the corresponding cluster.
  • the related information of GTM and the cluster is stored in the metadata. When the metadata is changed, it is pushed to the PROXY node, and the full metadata information is obtained when the PROXY is started.
  • the business cluster is composed of multiple data security groups and is a complete storage unit externally. Provides services such as high availability, high reliability, and consistency of data copies.
  • a GTM cluster consists of multiple GTMs, providing GTID services for one or more business clusters, ensuring GTID-related features, and ensuring high availability and reliability of GTMs.
  • the database node DB is mainly used to execute the SQL statement sent by the computing node and return the response.
  • a multi-version concurrency control method in a distributed database mainly includes:
  • PROXY Processing thread 1 and processing thread 2 in PROXY parse the SQL statement and classify the message to merge the processing request.
  • 3PROXY sends a processing sequence request to the GTM host of the corresponding cluster through the merge thread.
  • the master GTM merges the messages, and initiates data synchronization to the slave GTM if it is an application or release request for GTID.
  • the main GTM distributes the processing result through the merge thread, and sends it to each processing thread in PROXY.
  • 8PROXY sends the SQL statement to be executed after the change to each DB node.
  • 9DB executes the SQL statement and returns the execution result to PROXY.
  • a computing node includes: a first sending module configured to send a first statement to a database node, wherein the first statement carries a global active transaction list; the first The receiving module is configured to receive the return result returned by the database, wherein the return result is obtained by the database node of the data snapshot corresponding to the first statement, and based on the data snapshot and the global active thing list Sure.
  • the computing node provided in this embodiment includes: a first sending module configured to send a first statement to a database node, wherein the first statement carries a global active transaction list; a first receiving module configured to receive all The returned result is returned by the database, wherein the returned result is obtained by the database node of the data snapshot corresponding to the first statement, and determined based on the data snapshot and the global active thing list.
  • the technical solution of this embodiment sends the global active transaction list to the database nodes together, and makes judgments at the database nodes, thereby realizing the multi-version concurrency control of the distributed database, reducing repeated queries, and improving the high-throughput performance of the multi-concurrency scenario. .
  • the method before the computing node sends the first statement to the database node, the method further includes: after the computing node receives the second statement, sending a batch processing sequence request to the global transaction management node, where the batch processing sequence request is used for A global active thing list is requested; the computing node receives the global active thing list, adds the global active thing list to the second statement, and obtains the first statement.
  • sending a batch processing sequence request to the global transaction management node includes: after the computing node receives the second statement, parsing the second statement, DDL and DML are obtained, and for DDL and DML, batch processing sequence requests are sent to the global transaction management node of the corresponding cluster through the merge thread.
  • a database node comprising:
  • the second receiving module is configured to receive the first statement sent by the computing node, wherein the first statement carries a global list of active things;
  • a return result determining module is configured to obtain a data snapshot corresponding to the first statement, and determine a return result based on the data snapshot and the global active thing list;
  • the second sending module is configured to send the returned result to the computing node.
  • determining the return result based on the data snapshot and the global active transaction list includes: determining a global transaction identifier GTID in the data snapshot; using the GTID as a keyword in the global active transaction list Make a query; determine the returned result based on the query result.
  • determining the return result based on the query result includes: if the GTID is queried in the global active thing list, performing rollback based on the undo log log, and determining the rollback result as the return result; If the GTID is not found in the global active transaction list, the data snapshot is determined as the returned result.
  • the computing node and database node provided in this embodiment can execute the multi-version concurrency control method provided by any embodiment of the present application, and have corresponding functional modules and beneficial effects for executing the method.
  • the computing node and database node provided in this embodiment can execute the multi-version concurrency control method provided by any embodiment of the present application, and have corresponding functional modules and beneficial effects for executing the method.
  • a computing node comprising: a third sending module configured to send a GTID and a clearing policy to a database node, where the clearing policy is used to instruct the database node to send the corresponding GTID smaller than the GTID to the database node. undo log log is cleared.
  • the computing node is further configured to send a GTID application request to the global transaction management node; the GTID application request is used by the global transaction management node to generate a new GTID and establish a new GTID in a preset mapping table Correspondence to the smallest active GTID.
  • the computing node is further configured to send a GTID release request to the global transaction management node; the GTID release request is used by the global transaction management node to delete the GTID corresponding to the GTID release request in the global active transaction list, and The GTID corresponding to the minimum active GTID in the preset mapping table is obtained and sent to the computing node.
  • a database node comprising:
  • the third receiving module is configured to receive the GTID and the clearing policy sent by the computing node;
  • the log clearing module is configured to clear the undo log log corresponding to the GTID smaller than the GTID based on the clearing strategy.
  • the computing node and database node provided in this embodiment can execute the log clearing method provided by any embodiment of the present application, and have corresponding functional modules and beneficial effects for executing the method.
  • the log clearing method provided by any embodiment of this application.
  • FIG. 4 is a schematic structural diagram of a participant selection device provided by an embodiment of the present application.
  • the device includes a processor 41 , a memory 42 , and an input device 43, an output device 44 and a communication device 45; the number of processors 41 in the device can be one or more, and one processor 41 is taken as an example in FIG. 4; the processor 41, the memory 42, the input device 43 and the output device in the device
  • the devices 44 may be connected by a bus or in other ways, and the connection by a bus is taken as an example in FIG. 5 .
  • the memory 42 can be used to store software programs, computer-executable programs, and modules.
  • the processor 41 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 42, ie, implements any method provided by the embodiments of the present application.
  • the memory 42 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the device, and the like. Additionally, memory 42 may include high speed random access memory, and may also include nonvolatile memory, such as at least one magnetic disk storage device, flash memory device, or other nonvolatile solid state storage device. In some instances, memory 42 may further include memory located remotely from processor 41, which may be connected to the device through a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
  • the input device 43 may be used to receive input numerical or character information, and to generate key signal input related to user settings and function control of the device.
  • the output device 44 may include a display device such as a display screen.
  • the communication device 45 may include a receiver and a transmitter.
  • the communication device 45 is configured to transmit and receive information according to the control of the processor 41 .
  • the embodiments of the present application further provide a storage medium containing computer-executable instructions, where the computer-executable instructions are used to execute a multi-version concurrency control method when executed by a computer processor, include:
  • the computing node sends a first statement to the database node, wherein the first statement carries a global list of active things;
  • the database node sends the returned result to the computing node.
  • a storage medium containing computer-executable instructions provided by the embodiments of the present application, the computer-executable instructions of which are not limited to the above-mentioned method operations, and can also execute the multi-version concurrency control method provided by any embodiment of the present application. related operations in .
  • the embodiment of the present application also provides a storage medium containing computer-executable instructions, the computer-executable instructions being used to execute a log clearing method when executed by a computer processor, including:
  • the computing node sends the GTID and clearing policy to the database node;
  • the database node clears the undo log log corresponding to the GTID smaller than the GTID based on the clearing strategy.
  • a storage medium containing computer-executable instructions provided by an embodiment of the present application, the computer-executable instructions of which are not limited to the above-mentioned method operations, and can also execute the log clearing method provided by any embodiment of the present application. related operations.
  • the present application can be implemented by software and necessary general-purpose hardware, and of course can also be implemented by hardware, but in many cases, the former is a better implementation.
  • the technical solutions of the present application can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products can be stored in a computer-readable storage medium, such as a floppy disk of a computer , read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), flash memory (FLASH), hard disk or CD, etc., including several instructions to make a computer device (which can be a personal computer) , server, or network device, etc.) to execute the methods described in the various embodiments of this application.
  • a computer-readable storage medium such as a floppy disk of a computer , read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), flash memory (FLASH), hard disk or CD, etc.
  • user terminal encompasses any suitable type of wireless user equipment such as a mobile telephone, portable data processing device, portable web browser or vehicle mounted mobile station.
  • the various embodiments of the present application may be implemented in hardware or special purpose circuits, software, logic, or any combination thereof.
  • some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor or other computing device, although the application is not limited thereto.
  • Embodiments of the present application may be implemented by the execution of computer program instructions by a data processor of a mobile device, eg in a processor entity, or by hardware, or by a combination of software and hardware.
  • the computer program instructions may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code written in any combination of one or more programming languages or object code.
  • ISA instruction set architecture
  • the block diagrams of any logic flow in the figures of this application may represent program steps, or may represent interconnected logic circuits, modules and functions, or may represent a combination of program steps and logic circuits, modules and functions.
  • Computer programs can be stored on memory.
  • the memory may be of any type suitable for the local technical environment and may be implemented using any suitable data storage technology such as, but not limited to, read only memory (ROM), random access memory (RAM), optical memory devices and systems (Digital Versatile Discs). DVD or CD disc) etc.
  • Computer-readable media may include non-transitory storage media.
  • the data processor may be of any type suitable for the local technical environment, such as, but not limited to, a general purpose computer, special purpose computer, microprocessor, digital signal processor (DSP), application specific integrated circuit (ASIC), programmable logic device (FGPA) and processors based on multi-core processor architectures.
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FGPA programmable logic device

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

提供了一种多版本并发控制和日志清除方法、节点、设备和介质，方法包括：计算节点向数据库节点发送第一语句，其中，第一语句中携带全局活跃事物列表（S11）；数据库节点获取第一语句对应的数据快照，并基于数据快照和全局活跃事物列表确定返回结果（S12）；数据库节点将返回结果发送至计算节点（S13）。

Description

多版本并发控制和日志清除方法、节点、设备和介质
交叉引用
本申请基于申请号为“202011552901.3”、申请日为2020年12月24日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此以引入方式并入本申请。
技术领域
本申请涉及通信技术领域，具体涉及一种多版本并发控制和日志清除方法、节点、设备和介质。
背景技术
随着社会的发展，技术的进步，传统单机数据库逐渐被新兴的分布式数据库代替。大数据量、频繁的交互，给分布式数据库的实现带来了很多问题。
多版本并发控制(Multiversion concurrency control，MVCC)是指在并发访问数据库时，对事务读取到的内存进行处理，避免读写相互阻塞，最终实现不同事务在同一时间读取到的内容可能不同的功能。
然而，目前MVCC主要在单机上得以实现，在分布式数据库中，结合两阶段提交方法实现多版本并发控制。然而，在两阶段提交方法中，一条更新语句下发到多个分片，其中部分分片操作指令执行成功，另一部分分片还在执行中，此时数据库又收到一个查询语句，那么有的分片返回commit前的结果，有的返回commit之后的结果，那么返回的结果是不可信的，需要丢弃；可能采用的做法是不断的重复查询，直到多个分片都返回操作指令执行成功结果，才会把查询的结果返回给客户。这种做法严重影响了高并发场景下的吞吐性能。
发明内容
第一方面，本申请实施例提供一种多版本并发控制方法，包括:计算节点向数据库节点发送第一语句，其中，所述第一语句中携带全局活跃事物列表；所述数据库节点获取所述第一语句对应的数据快照，并基于所述数据快照和所述全局活跃事物列表确定返回结果；所述数据库节点将所述返回结果发送至所述计算节点。
第二方面，本申请实施例提供一种日志清除方法，包括：计算节点将全局事务标志(Global Transaction Identifier，GTID)和清除策略发送至数据库节点；所述数据库节点基于所述清除策略将小于所述GTID的GTID对应的undo log日志清除。
第三方面，本申请实施例提供一种计算节点，包括：第一发送模块，被配置为向数据库节点发送第一语句，其中，所述第一语句中携带全局活跃事物列表；第一接收模块，被配置为接收所述数据库返回的返回结果，其中，所述返回结果由所述数据库节点获取所述第一语句对应的数据快照，并基于所述数据快照和所述全局活跃事物列表确定。
第四方面，本申请实施例提供一种数据库节点，包括：第二接收模块，被配置为接收计算节点发送的第一语句，其中，所述第一语句中携带全局活跃事物列表；返回结果确定模块，被配置为获取所述第一语句对应的数据快照，并基于所述数据快照和所述全局活跃事物列表确定返回结果；第二发送模块，被配置为将所述返回结果发送至所述计算节点。
第五方面，本申请实施例提供一种计算节点，包括：第三发送模块，被配置为将GTID和清除策略发送至数据库节点，所述清除策略用于指示数据库节点将小于所述GTID的GTID对应的undo log日志清除。
第六方面，本申请实施例提供一种数据库节点，包括：第三接收模块，被配置为接收计算节点发送的GTID和清除策略；日志清除模块，被配置为基于所述清除策略将小于所述GTID的GTID对应的undo log日志清除。
第七方面，本申请实施例提供一种电子设备，包括：一个或多个处理器；存储器，用于存储一个或多个程序；当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如本申请实施例提供的任一项所 述的方法。
第八方面，本申请实施例提供一种存储介质，所述存储介质存储有计算机程序，所述计算机程序被处理器执行时实现如本申请实施例提供的任一项所述的方法。
关于本申请的以上实施例和其他方面以及其实现方式，在附图说明、具体实施方式和权利要求中提供更多说明。
附图说明
图1是本申请实施例提供的一种多版本并发控制方法的流程图；
图2是本申请实施例提供的一种日志清除方法的流程图；
图3是本实施例提供的单PROXY，多GTM，多集群结构的分布式数据库的结构示意图；
图4是本申请实施例提供的一种多版本并发控制设备的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚明白，下文中将结合附图对本申请的实施例进行详细说明。需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互任意组合。
在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行。并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
在一个实施例中，提供一种多版本并发控制方法，如图1所示，本实施例提供的多版本并发控制方法主要包括步骤S11、S12、S13。
S11、计算节点向数据库节点发送第一语句，其中，所述第一语句中携带全局活跃事物列表。
S12、所述数据库节点获取所述第一语句对应的数据快照，并基于所述数据快照和所述全局活跃事物列表确定返回结果。
S13、所述数据库节点将所述返回结果发送至所述计算节点。
所述第一语句可以是携带全局活跃事物列表的SQL查询语句，也可以是携带全局活跃事物的其他的语句，本实施例中，不进行限定。所述全局活跃事物列表中存储当前正在活跃的全局事务标志GTID。
本实施例中，计算节点PROXY发送查询语句时，将全局活跃事务列表一起下发给DB(database，数据库)，在DB层进行判断，从而实现了分布式数据库的多版本并发控制，减少了重复查询，提高了多并发场景的高吞吐性能。
在一个实施例中，所述数据库节点基于所述数据快照和所述全局活跃事物列表确定返回结果，包括确定所述数据快照中的全局事务标志GTID；将所述GTID为关键字在所述全局活跃事物列表进行查询；基于查询结果确定返回结果。
在本实施例中，数据库节点基于第一语句中的除全局活跃事物列表外的其他语句，例如：基于SQL查询语句确定和隔离级别获取数据快照，然后在数据快照中查询并获取GTID，以查询到的GTID为关键字在全局活跃事物列表进行查询。
具体的，所述数据库节点基于查询结果确定返回结果，包括：如果在所述全局活跃事物列表查询到所述GTID，则基于undo log日志进行回滚，将回滚结果确定为返回结果；如果在所述全局活跃事物列表未查询到所述GTID，则将所述数据快照确定为返回结果。
在实施例中，如果在全局活跃事物列表查询到所述GTID，则表示整个存储引擎中有其他的数据库节点正在执行该GTID对应的事务，即该GTID对应的事务是活跃的，则基于undo log日志进行回滚，将回滚结果确定为返回结果。
如果在所述全局活跃事物列表未查询到GTID，则表示整个存储引擎中没有其他的数据库节点在执行该GTID对应的事务，则将数据快照确定为返回结果。
本实施例中，数据库节点DB根据事务是否活跃，决定返回结果是否需要回滚，从而解决了分布式数据库中多次使用聚合函数返回结果不准确的问题以及返回结果不对导致的不断重复查询影响性能的问题。
在一个实施例中，计算节点向数据库节点发送第一语句之前还包括：所述计算节点接收到第二语句后，向全局事物管理节点发送批量处理序列请求，所 述批量处理序列请求用于请求全局活跃事物列表；所述全局事物管理节点基于批量处理序列请求将全局活跃事物列表发送至计算节点；所述计算节点将所述全局活跃事物列表添加至所述第二语句中，得到第一语句。
在本实施例中，第二语句是指计算节点接收到的应用发送的SQL语句，其不携带全局活跃事物列表。第二语句携带全局活跃事物列表后形成第一语句。
在一个例子中，第二语句是查询语句时，全局事物管理节点将全局活跃事务列表返回给计算节点。
在一个例子中，第二语句是新事物请求语句时，全局事物管理节点产生新的GTID，并返回给计算节点。需要说明的是，一个事务才申请一个GTID，而一个事务可以包含多条语句。其中，第二语句是当前事务包含的语句，所述第二语句可以是一条语句，也可以是多条语句，本实施中并不进行限定。
在一个例子中，在所述第二语句是不与GTID相关的语句时，全局事物管理节点直接将查询到的全局活跃事物列表发送至计算节点。
在所述第二语句是与GTID相关的语句时，全局事物管理节点基于GTID更新全局活跃事物列表，然后将更新后的的全局活跃事物列表发送至计算节点。
在一个实施例中，所述计算节点接收到第二语句后，向全局事物管理节点发送批量处理序列请求，包括：所述计算节点接收到第二语句后，对所述第二语句进行解析，得到的DDL(data manipulation language，数据定义语言)和DML(data definition language，数据操控语言)，对于DDL和DML，通过合并线程向相应集群的全局事物管理节点发送批量处理序列请求。
在本实施例中，计算节点PROXY会对短时间内大量的GTID处理需求进行分集群合并，发送至GTM(Global Traffic Manager，全局流量管理)进行批量处理，减少了多处理线程独立发送GTID处理请求造成的消息冗余和重复处理。
本实施例中，GTM为全局事务管理节点，每个GTM集群服务于一个或多个业务集群，根据性能或高可用需要可以进行GTM集群扩充，每个GTM主机为每个业务集群创建独立线程，保证各集群、各数据库在事务处理时的隔离性、高效性，从而实现事务在集群层面的隔离和集群内部的唯一。单一功能、多线程并发处理使得全局事务处理效率的大幅提升，保证分布式数据库系统的高性 能。
需要说明的是，第二语句中不携带GTID信息的，计算节点收到客户发来的DDL或DML后，每个事务都要去GTM申请GTID，然后组合成第一语句下发到数据库节点；对于查询语句和聚合函数，以前是不带活跃事务列表的，本实施例中把活跃事务列表加上组成第一语句。
本实施例提供的多版本并发控制方法、存储引擎、设备和存储介质，包括：计算节点向数据库节点发送第一语句，其中，第一语句中携带全局活跃事物列表；数据库节点获取第一语句对应的数据快照，并基于数据快照和全局活跃事物列表确定返回结果，数据库节点将返回结果发送至计算节点，本实施例的技术方案将全局活跃事务列表一起下发给数据库节点，在数据库节点进行判断，从而实现了分布式数据库的多版本并发控制，减少了重复查询，提高了多并发场景的高吞吐性能。
在一个实施例中，提供一种日志清除方法，如图2所示，本实施例提供的日志清除方法主要包括步骤S21、S22。
S21、计算节点将GTID和清除策略发送至数据库节点；
S22、所述数据库节点基于所述清除策略将小于所述GTID的GTID对应的undo log日志清除。
在一个实施例中，还包括：所述计算节点向全局事物管理节点发送GTID申请请求；所述全局事物管理节点基于所述GTID申请请求生成新GTID，并在预设映射表汇中建立新GTID与最小活跃GTID的对应关系。
在一个实施例中，所述计算节点将所述最小GTID和清除策略发送至数据库节点，之前还包括：所述计算节点向全局事物管理节点发送GTID释放请求；所述全局事物管理节点删除GTID释放请求在全局活跃事物列表中对应的GTID；所述全局事物管理节点获取最小活跃GTID在预设映射表中对应的GTID，并发送至所述计算节点。
在本实施例中，在计算节点申请GTID的时候，需要记录申请的GTID与此时最小的活跃GTID之间的映射关系，是为了GTM释放GTID的时候，找到返回给计算节点的最小活跃GTID。
本实施例中，计算节点PROXY通过特有的purge策略，控制数据库节点DB进行purge，数据库节点DB根据最小全局GTID，清除undo log中小于最小GTID的数据副本，保证多版本并发控制的同时，实现了undo log的去冗余设计，最重要的是防止DB自行purge导致数据无法完成回滚。
在一个实施例中，提供一种分布式数据库中多版本并发控制方法，主要用于实现高吞吐量实现方法，主要包括：
PROXY内部多个处理线程收到来自应用的SQL语句后进行分析，对于所有涉及GTID的DDL和DML，统一通过合并线程向相应集群的GTM发送批量处理序列请求。采用合并线程发送批量处理序列请求，可以减少多次请求导致的消息冗余和性能下降。
其中，每个GTM服务包括1个或者多个业务集群，以元数据形式保存在元数据表中，PROXY启动时获取元数据信息，GTM服务集群变更时消息推送到PROXY。确保PROXY知晓每个GTM集群服务于哪些业务集群。
主GTM将多个线程的处理结果合并以后发往备GTM，进行数据同步，主GTM接收到备GTM同步成功的响应后，主GTM向PROXY返回处理结果。
GTM集群内部多个持久化线程使用共享内存，定时对全局活跃的GTID进行全量备份，保证数据完整不丢失。GTM集群一主机多从机的多副本架构，保证当主机GTM宕机时，从机GTM上有完整的全局活跃GTID信息，能够接替主机处理GTID功能。
GTM收到申请GTID时，存储当前活跃GTID列表中的最小全局活跃GTID。
GTM收到释放GTID时，查询当前最小活跃GTID对应其申请时存储的全局最小活跃GTID，将其返回给PROXY。
PROXY执行查询或者聚合函数时，将全局活跃事务列表一起发送给DB。
采用上述方法，与现有的技术相比，能够较好地解决分布式数据库中多版本并发控制，提高吞吐性能，尤其是解决了聚合函数多次查询结果不一致的问题。鉴于多版本并发控制是数据库中非常重要且基本的功能之一，该技术能够提高高并发场景的吞吐性能实现方法，完善了分布式数据库的功能。
具体的，多版本并发控制实施方法的具体步骤描述：
1)应用向PROXY发送查询处理的SQL语句。
2)PROXY处理线程解析语句，将消息分类整理，发送到对应集群的GTM，查询此时全局活跃事务列表。
3)主GTM解析查询请求消息，将对应集群的全局活跃事务列表返回给PROXY；全局活跃事务列表里存放的是当前所有活跃事务的GTID。
4)Proxy重新组织SQL语句，将查询语句和全局活跃事务列表一起发送到DB；
5)DB节点首先根据SQL语句和隔离级别(例如RC)获取数据快照；然后查询数据快照中的GTID是否在活跃事务列表中；如果在活跃事务列表中，则结合undo log将数据回滚后的结果返回给PROXY；如果不在全局活跃事务列表中，则将取出的数据快照返回给PROXY；
6)PROXY将不同分片上的返回结果进行整合处理后返回给应用最终结果。
在一个实施例中，提供一种Purge策略。
DB收到PROXY发送的查询或者聚合函数后，获取数据快照后与全局活跃事务列表进行比较，如果所查询事务GTID不在活跃事务列表中，则将当前获取到的快照结果返回给PROXY；如果当前事务GTID仍在全局活跃事务列表中，则结合undo log将回滚后的结果返回给PROXY。
PROXY向GTM发送释放GTID后，得到此时最小的GTID，将之发送给DB，执行purge策略，清除最小GTID之前的undo log日志。
具体的，Purge策略实施方法具体步骤描述：
1)当主GTM收到PROXY申请GTID请求后，产生新的GTID，将新产生的GTID和此时最小的全局活跃事务一起存储到映射表中，同步到从GTM，最后返回给PROXY新产生的GTID；
2)当主GTM收到释放某个GTID请求后，同步删除从机全局活跃事务列表中的此GTID，取此时全局活跃事务列表中的最小GTID所对应的映射表中的GTID，返回给PROXY进行undo log的purge；
3)PROXY向DB下发purge策略，将GTM返回的映射表中的GTID同时下发到DB，DB将此GTID以前的undo log清除，向PROXY返回执行结果；
4)PROXY收到所有DB节点的应答后，完成此次purge。
在一个实施例中，以单PROXY，多GTM，多集群结构的分布式数据库为例，具体叙述多版本并发控制的处理步骤。图3是本实施例提供的单PROXY，多GTM，多集群结构的分布式数据库的结构示意图；如图3所示，单PROXY，多GTM，多集群结构的分布式数据库，包括：应用节点，计算节点PROXY，数据库节点DB，全局事务管理节点GTM。
全局事务管理节点GTM主要用于，对整个分布式数据库系统中的全局事务进行管理，包括事务开始时给每个事务分配GTID，管理全局活跃GTID列表，事务结束后释放GTID等。
GTM一般为一主机多从机组成的多副本高可用集群架构，保证主机宕机时有多个持有完整序列副本的可用从机能够接替主机，维持数据库功能的正常使用。一个GTM集群对应一个或多个业务集群，保证集群上事务处理的高性能，同时多GTM集群保证了集群间事务处理的隔离性以及更好的容灾能力。
计算节点PROXY主要用于，对应用发来的SQL进行解析，向GTM发起申请、释放、查询GTID处理请求。在收到GTID的处理结果后将修改后的SQL语句下发至数据库节点执行。PROXY通过连接实例与集群绑定，并与处理对应集群GTID的GTM主机建立链接，GTM和集群相关信息在元数据中存储，元数据变更时推动到PROXY节点，PROXY启动时获取全量元数据信息。
业务集群由多个数据安全组组成，对外是一个完整的存储单位。提供数据副本的高可用、高可靠、一致性等服务。
GTM集群由多个GTM组成，为一个或多个业务集群提供GTID服务，保证GTID相关特性，并确保GTM高可用、高可靠。
数据库节点DB主要用于，执行计算节点发来的SQL语句并返回响应。
如图3所示，一种分布式数据库中多版本并发控制方法主要包括：
①应用向PROXY下发SQL语句。
②PROXY中的处理线程1和处理线程2解析SQL语句并将消息分类进行合并处理请求。
③PROXY通过合并线程向对应集群的GTM主机发送处理序列请求。
④主GTM对消息进行合并处理，如果是申请、释放GTID请求则向从GTM发起数据同步。
⑤从GTM数据更新成功后向主GTM返回同步结果响应。
⑥主GTM向PROXY返回处理结果。
⑦主GTM通过合并线程分发处理结果，并发送至PROXY中的各处理线程。
⑧PROXY将变更后待执行的SQL语句发送给各DB节点。
⑨DB执行SQL语句并向PROXY返回执行结果。
⑩PROXY向应用返回处理结果。
在一个实施例中，提供一种计算节点，所述计算节点包括：第一发送模块，被配置为向数据库节点发送第一语句，其中，所述第一语句中携带全局活跃事物列表；第一接收模块，被配置为接收所述数据库返回的返回结果，其中，所述返回结果由所述数据库节点获取所述第一语句对应的数据快照，并基于所述数据快照和所述全局活跃事物列表确定。
本实施例提供的计算节点，包括：第一发送模块，被配置为向数据库节点发送第一语句，其中，所述第一语句中携带全局活跃事物列表；第一接收模块，被配置为接收所述数据库返回的返回结果，其中，所述返回结果由所述数据库节点获取所述第一语句对应的数据快照，并基于所述数据快照和所述全局活跃事物列表确定。本实施例的技术方案将全局活跃事务列表一起下发给数据库节点，在数据库节点进行判断，从而实现了分布式数据库的多版本并发控制，减少了重复查询，提高了多并发场景的高吞吐性能。
在一个实施例中，计算节点向数据库节点发送第一语句之前，还包括：所述计算节点接收到第二语句后，向全局事物管理节点发送批量处理序列请求，所述批量处理序列请求用于请求全局活跃事物列表；所述计算节点接收所述全局活跃事物列表，将所述全局活跃事物列表添加至所述第二语句中，得到第一语句。
在一个实施例中，所述计算节点接收到第二语句后，向全局事物管理节点发送批量处理序列请求，包括：所述计算节点接收到第二语句后，对所述第二 语句进行解析，得到DDL和DML，对于DDL和DML，通过合并线程向相应集群的全局事物管理节点发送批量处理序列请求。
在一个实施例中，提供一种一种数据库节点，包括：
第二接收模块，被配置为接收计算节点发送的第一语句，其中，所述第一语句中携带全局活跃事物列表；
返回结果确定模块，被配置为获取所述第一语句对应的数据快照，并基于所述数据快照和所述全局活跃事物列表确定返回结果；
第二发送模块，被配置为将所述返回结果发送至所述计算节点。
在一个实施例中，基于所述数据快照和所述全局活跃事物列表确定返回结果，包括：确定所述数据快照中的全局事务标志GTID；将所述GTID为关键字在所述全局活跃事物列表进行查询；基于查询结果确定返回结果。
在一个实施例中，基于查询结果确定返回结果，包括：如果在所述全局活跃事物列表查询到所述GTID，则基于undo log日志进行回滚，将回滚结果确定为返回结果；如果在所述全局活跃事物列表未查询到所述GTID，则将所述数据快照确定为返回结果。
本实施例中提供的计算节点和数据库节点可执行本申请任意实施例所提供的多版本并发控制方法，具备执行该方法相应的功能模块和有益效果。未在本实施例中详尽描述的技术细节，可参见本申请任意实施例所提供的多版本并发控制方法。
在一个实施例中，提供一种计算节点，包括：第三发送模块，被配置为将GTID和清除策略发送至数据库节点，所述清除策略用于指示数据库节点将小于所述GTID的GTID对应的undo log日志清除。
在一个实施方式中，计算节点，还用于向全局事物管理节点发送GTID申请请求；所述GTID申请请求用于所述全局事物管理节点生成新GTID，并在预设映射表汇中建立新GTID与最小活跃GTID的对应关系。
在一个实施方式中，计算节点，还用于向全局事物管理节点发送GTID释放请求；所述GTID释放请求用于所述全局事物管理节点删除GTID释放请求 在全局活跃事物列表中对应的GTID，并获取最小活跃GTID在预设映射表中对应的GTID发送至所述计算节点。
在一个实施例中，提供一种数据库节点，包括：
第三接收模块，被配置为接收计算节点发送的GTID和清除策略；
日志清除模块，被配置为基于所述清除策略将小于所述GTID的GTID对应的undo log日志清除。
本实施例中提供的计算节点和数据库节点可执行本申请任意实施例所提供的日志清除方法，具备执行该方法相应的功能模块和有益效果。未在本实施例中详尽描述的技术细节，可参见本申请任意实施例所提供的日志清除方法。
本申请实施例还提供一种参与者选择设备，图4是本申请实施例提供的一种参与者选择设备的结构示意图，如图4所示，该设备包括处理器41、存储器42、输入装置43、输出装置44和通信装置45；设备中处理器41的数量可以是一个或多个，图4中以一个处理器41为例；设备中的处理器41、存储器42、输入装置43和输出装置44可以通过总线或其他方式连接，图5中以通过总线连接为例。
存储器42作为一种计算机可读存储介质，可用于存储软件程序、计算机可执行程序以及模块。处理器41通过运行存储在存储器42中的软件程序、指令以及模块，从而执行设备的各种功能应用以及数据处理，即实现本申请实施例提供的任一方法。
存储器42可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序；存储数据区可存储根据设备的使用所创建的数据等。此外，存储器42可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实例中，存储器42可进一步包括相对于处理器41远程设置的存储器，这些远程存储器可以通过网络连接至设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
输入装置43可用于接收输入的数字或字符信息，以及产生与设备的用户设置以及功能控制有关的键信号输入。输出装置44可包括显示屏等显示设备。
通信装置45可以包括接收器和发送器。通信装置45设置为根据处理器41的控制进行信息收发通信。
在一个示例性的实施方式中，本申请实施例还提供一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行一种多版本并发控制方法，包括:
计算节点向数据库节点发送第一语句，其中，所述第一语句中携带全局活跃事物列表；
所述数据库节点获取所述第一语句对应的数据快照，并基于所述数据快照和所述全局活跃事物列表确定返回结果；
所述数据库节点将所述返回结果发送至所述计算节点。
当然，本申请实施例所提供的一种包含计算机可执行指令的存储介质,其计算机可执行指令不限于如上所述的方法操作,还可以执行本申请任意实施例所提供的多版本并发控制方法中的相关操作。
在一个示例性的实施方式中，本申请实施例还提供一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行一种日志清除方法，包括:
计算节点将GTID和清除策略发送至数据库节点；
所述数据库节点基于所述清除策略将小于所述GTID的GTID对应的undo log日志清除。
当然，本申请实施例所提供的一种包含计算机可执行指令的存储介质,其计算机可执行指令不限于如上所述的方法操作,还可以执行本申请任意实施例所提供的日志清除方法中的相关操作。
通过以上关于实施方式的描述，所属领域的技术人员可以清楚地了解到，本申请可借助软件及必需的通用硬件来实现，当然也可以通过硬件实现，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如计算机的软盘、只读存储器(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory,RAM)、 闪存(FLASH)、硬盘或光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述的方法。
以上所述，仅为本申请的示例性实施例而已，并非用于限定本申请的保护范围。
本领域内的技术人员应明白，术语用户终端涵盖任何适合类型的无线用户设备，例如移动电话、便携数据处理装置、便携网络浏览器或车载移动台。
一般来说，本申请的多种实施例可以在硬件或专用电路、软件、逻辑或其任何组合中实现。例如，一些方面可以被实现在硬件中，而其它方面可以被实现在可以被控制器、微处理器或其它计算装置执行的固件或软件中，尽管本申请不限于此。
值得注意的是，上述节点的实施例中，所包括的各个单元和模块只是按照功能逻辑进行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，各功能单元的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。
本申请的实施例可以通过移动装置的数据处理器执行计算机程序指令来实现，例如在处理器实体中，或者通过硬件，或者通过软件和硬件的组合。计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码。
本申请附图中的任何逻辑流程的框图可以表示程序步骤，或者可以表示相互连接的逻辑电路、模块和功能，或者可以表示程序步骤与逻辑电路、模块和功能的组合。计算机程序可以存储在存储器上。存储器可以具有任何适合于本地技术环境的类型并且可以使用任何适合的数据存储技术实现，例如但不限于只读存储器(ROM)、随机访问存储器(RAM)、光存储器装置和系统(数码多功能光碟DVD或CD光盘)等。计算机可读介质可以包括非瞬时性存储介质。数据处理器可以是任何适合于本地技术环境的类型，例如但不限于通用计算机、专用计算机、微处理器、数字信号处理器(DSP)、专用集成电路(ASIC)、可编程逻辑器件(FGPA)以及基于多核处理器架构的处理器。
通过示范性和非限制性的示例，上文已提供了对本申请的示范实施例的详细描述。但结合附图和权利要求来考虑，对以上实施例的多种修改和调整对本领域技术人员来说是显而易见的，但不偏离本申请的范围。因此，本申请的恰当范围将根据权利要求确定。

Claims (14)

  1. 一种多版本并发控制方法，包括:
    计算节点向数据库节点发送第一语句，其中，所述第一语句中携带全局活跃事物列表；
    所述数据库节点获取所述第一语句对应的数据快照，并基于所述数据快照和所述全局活跃事物列表确定返回结果；
    所述数据库节点将所述返回结果发送至所述计算节点。
  2. 根据权利要求1所述的方法，其中，所述数据库节点基于所述数据快照和所述全局活跃事物列表确定返回结果，包括
    确定所述数据快照中的全局事务标志GTID；
    将所述GTID为关键字在所述全局活跃事物列表进行查询；
    基于查询结果确定返回结果。
  3. 根据权利要求2所述的方法，其中，所述数据库节点基于查询结果确定返回结果，包括：
    如果在所述全局活跃事物列表查询到所述GTID，则基于undo log日志进行回滚，将回滚结果确定为返回结果；
    如果在所述全局活跃事物列表未查询到所述GTID，则将所述数据快照确定为返回结果。
  4. 根据权利要求1至3中任一项所述的方法，其中，计算节点向数据库节点发送第一语句之前还包括：
    所述计算节点接收到第二语句后，向全局事物管理节点发送批量处理序列请求，所述批量处理序列请求用于请求全局活跃事物列表；
    所述全局事物管理节点基于批量处理序列请求将全局活跃事物列表发送至计算节点；
    所述计算节点将所述全局活跃事物列表添加至所述第二语句中，得到第一语句。
  5. 根据权利要求4所述的方法，其中，所述计算节点接收到第二语句后，向全局事物管理节点发送批量处理序列请求，包括：
    所述计算节点接收到第二语句后，对所述第二语句进行解析，得到数据定 义语言DDL和数据操控语言DML，
    对于DDL和DML，通过合并线程向相应集群的全局事物管理节点发送批量处理序列请求。
  6. 一种日志清除方法，其中，包括：
    计算节点将全局事务标志GTID和清除策略发送至数据库节点；
    所述数据库节点基于所述清除策略将小于所述GTID的GTID对应的undolog日志清除。
  7. 根据权利要求6所述的方法，其中，还包括：
    所述计算节点向全局事物管理节点发送GTID申请请求；
    所述全局事物管理节点基于所述GTID申请请求生成新GTID，并在预设映射表汇中建立新GTID与最小活跃GTID的对应关系。
  8. 根据权利要求7所述的方法，其中，所述计算节点将所述最小GTID和清除策略发送至数据库节点，之前还包括：
    所述计算节点向全局事物管理节点发送GTID释放请求；
    所述全局事物管理节点删除GTID释放请求在全局活跃事物列表中对应的GTID；
    所述全局事物管理节点获取最小活跃GTID在预设映射表中对应的GTID，并发送至所述计算节点。
  9. 一种计算节点，包括：
    第一发送模块，被配置为向数据库节点发送第一语句，其中，所述第一语句中携带全局活跃事物列表；
    第一接收模块，被配置为接收所述数据库返回的返回结果，其中，所述返回结果由所述数据库节点获取所述第一语句对应的数据快照，并基于所述数据快照和所述全局活跃事物列表确定。
  10. 一种数据库节点，包括：
    第二接收模块，被配置为接收计算节点发送的第一语句，其中，所述第一语句中携带全局活跃事物列表；
    返回结果确定模块，被配置为获取所述第一语句对应的数据快照，并基于所述数据快照和所述全局活跃事物列表确定返回结果；
    第二发送模块，被配置为将所述返回结果发送至所述计算节点。
  11. 一种计算节点，包括：
    第三发送模块，被配置为将GTID和清除策略发送至数据库节点，所述清除策略用于指示数据库节点将小于所述GTID的GTID对应的undo log日志清除。
  12. 一种数据库节点，包括：
    第三接收模块，被配置为接收计算节点发送的GTID和清除策略；
    日志清除模块，被配置为基于所述清除策略将小于所述GTID的GTID对应的undo log日志清除。
  13. 一种电子设备，包括：
    一个或多个处理器；
    存储器，用于存储一个或多个程序；
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现任一项所述的方法。
  14. 一种存储介质，所述存储介质存储有计算机程序，所述计算机程序被处理器执行时实现任一项所述的方法。
PCT/CN2021/140468 2020-12-24 2021-12-22 多版本并发控制和日志清除方法、节点、设备和介质 WO2022135471A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011552901.3 2020-12-24
CN202011552901.3A CN114661690A (zh) 2020-12-24 2020-12-24 多版本并发控制和日志清除方法、节点、设备和介质

Publications (1)

Publication Number Publication Date
WO2022135471A1 true WO2022135471A1 (zh) 2022-06-30

Family

ID=82025402

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/140468 WO2022135471A1 (zh) 2020-12-24 2021-12-22 多版本并发控制和日志清除方法、节点、设备和介质

Country Status (2)

Country Link
CN (1) CN114661690A (zh)
WO (1) WO2022135471A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116737744A (zh) * 2023-08-14 2023-09-12 金篆信科有限责任公司 数据库的控制系统、方法、计算机设备及存储介质
CN117131060A (zh) * 2023-07-26 2023-11-28 泽拓科技(深圳)有限责任公司 分布式数据库并发控制方法、系统、计算机设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108090056A (zh) * 2016-11-21 2018-05-29 中兴通讯股份有限公司 数据查询方法、装置及系统
CN109739935A (zh) * 2019-01-09 2019-05-10 腾讯科技（深圳）有限公司 数据读取方法、装置、电子设备以及存储介质
US20190235974A1 (en) * 2018-01-29 2019-08-01 Guy Pardon Transaction processing system, recovery subsystem and method for operating a recovery subsystem
CN110196856A (zh) * 2019-05-09 2019-09-03 腾讯科技（深圳）有限公司 一种分布式数据读取方法及装置
CN111143389A (zh) * 2019-12-27 2020-05-12 腾讯科技（深圳）有限公司 事务执行方法、装置、计算机设备及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108090056A (zh) * 2016-11-21 2018-05-29 中兴通讯股份有限公司 数据查询方法、装置及系统
US20190235974A1 (en) * 2018-01-29 2019-08-01 Guy Pardon Transaction processing system, recovery subsystem and method for operating a recovery subsystem
CN109739935A (zh) * 2019-01-09 2019-05-10 腾讯科技（深圳）有限公司 数据读取方法、装置、电子设备以及存储介质
CN110196856A (zh) * 2019-05-09 2019-09-03 腾讯科技（深圳）有限公司 一种分布式数据读取方法及装置
CN111143389A (zh) * 2019-12-27 2020-05-12 腾讯科技（深圳）有限公司 事务执行方法、装置、计算机设备及存储介质

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117131060A (zh) * 2023-07-26 2023-11-28 泽拓科技(深圳)有限责任公司 分布式数据库并发控制方法、系统、计算机设备
CN116737744A (zh) * 2023-08-14 2023-09-12 金篆信科有限责任公司 数据库的控制系统、方法、计算机设备及存储介质
CN116737744B (zh) * 2023-08-14 2023-11-24 金篆信科有限责任公司 数据库的控制系统、方法、计算机设备及存储介质

Also Published As

Publication number Publication date
CN114661690A (zh) 2022-06-24

Similar Documents

Publication Publication Date Title
EP3968175B1 (en) Data replication method and apparatus, and computer device and storage medium
EP4064067B1 (en) Automatic query offloading to a standby database
CN110502507B (zh) 一种分布式数据库的管理系统、方法、设备和存储介质
US7653668B1 (en) Fault tolerant multi-stage data replication with relaxed coherency guarantees
JP6404907B2 (ja) 効率的な読み取り用レプリカ
CN109710388B (zh) 数据读取方法、装置、电子设备以及存储介质
US11822540B2 (en) Data read method and apparatus, computer device, and storage medium
CN105493474B (zh) 用于支持用于同步分布式数据网格中的数据的分区级别日志的系统及方法
WO2022135471A1 (zh) 多版本并发控制和日志清除方法、节点、设备和介质
CN112084258A (zh) 一种数据同步方法和装置
CN111159252A (zh) 事务执行方法、装置、计算机设备及存储介质
CN107870954B (zh) 一种分布式数据库的扩展方法及装置
JP7549137B2 (ja) トランザクション処理方法、システム、装置、機器、及びプログラム
WO2020025049A1 (zh) 数据同步的方法、装置、数据库主机及存储介质
CN112307119A (zh) 数据同步方法、装置、设备及存储介质
EP4307137A1 (en) Transaction processing method, distributed database system, cluster, and medium
US20230185795A1 (en) Method and system for processing database transactions in a distributed online transaction processing (oltp) database
US12111817B2 (en) Log execution method and apparatus, computer device and storage medium
CN108090056B (zh) 数据查询方法、装置及系统
WO2020119709A1 (zh) 数据合并的实现方法、装置、系统及存储介质
WO2024109253A1 (zh) 一种数据备份方法、系统和设备
WO2024051027A1 (zh) 一种大数据的数据配置方法和系统
KR100492167B1 (ko) 비공유 구조의 데이터베이스 클러스터 서버 시스템과온라인 확장 방법
CN114218236A (zh) 一种数据库集群元数据管理方法
US11914571B1 (en) Optimistic concurrency for a multi-writer database

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21909466

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 161123)

122 Ep: pct application non-entry in european phase

Ref document number: 21909466

Country of ref document: EP

Kind code of ref document: A1