WO2023065868A1 - 事务执行方法、装置、计算设备及存储介质 - Google Patents

事务执行方法、装置、计算设备及存储介质 Download PDF

Info

Publication number
WO2023065868A1
WO2023065868A1 PCT/CN2022/117451 CN2022117451W WO2023065868A1 WO 2023065868 A1 WO2023065868 A1 WO 2023065868A1 CN 2022117451 W CN2022117451 W CN 2022117451W WO 2023065868 A1 WO2023065868 A1 WO 2023065868A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
ddl
transaction
block
data block
Prior art date
Application number
PCT/CN2022/117451
Other languages
English (en)
French (fr)
Inventor
熊亮春
潘安群
雷海林
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Publication of WO2023065868A1 publication Critical patent/WO2023065868A1/zh
Priority to US18/450,606 priority Critical patent/US20230394027A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2365—Ensuring data consistency and integrity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation
    • G06F16/24534—Query rewriting; Transformation
    • G06F16/24549—Run-time optimisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/242—Query formulation
    • G06F16/2433—Query languages
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/466—Transaction processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
    • G06F9/526—Mutual exclusion algorithms
    • G06F9/528—Mutual exclusion algorithms by using speculative mechanisms

Definitions

  • the present application relates to the technical field of databases, and in particular to a transaction execution method, device, computing device and storage medium.
  • SQL Structured Query Language
  • Structured Query Language is a database query and programming language for accessing data and querying, updating and managing relational database systems.
  • a type of DDL (Data Definition Language, Data Definition Language) statement is involved in the SQL command, and the DDL statement is a statement used to modify the definition of objects (such as tables, indexes, columns, triggers, etc.) in the database.
  • objects such as tables, indexes, columns, triggers, etc.
  • Embodiments of the present application provide a transaction execution method, device, computing device, and storage medium.
  • a transaction execution method includes: in response to the data definition language DDL transaction whose execution is interrupted meets the recovery condition, determining the last data block processed by the DDL transaction before the execution is interrupted; from the last Starting from the next data block of a data block, the DDL transaction is continued to be executed; in response to the completion of processing all data blocks operated on the DDL transaction, the DDL transaction is submitted.
  • a transaction execution device includes: a determination module, configured to determine the last data block processed by the DDL transaction before the execution interruption in response to the data definition language DDL transaction interrupted by execution meeting the recovery condition
  • the execution module is used to continue to execute the DDL transaction starting from the next data block of the last data block;
  • the submission module is used to submit all data blocks in response to the DDL transaction operation. Describe DDL transactions.
  • a computing device in one aspect, includes one or more processors and one or more memories, at least one computer program is stored in the one or more memories, and the at least one computer program is executed by the one or more A plurality of processors are loaded and executed to implement the above-mentioned transaction execution method.
  • a storage medium is provided, and at least one computer program is stored in the storage medium, and the at least one computer program is loaded and executed by a processor to implement the above-mentioned transaction execution method.
  • a computer program product or computer program comprising one or more pieces of program code stored in a computer-readable storage medium.
  • One or more processors of the computing device can read the one or more pieces of program code from the computer-readable storage medium, and the one or more processors execute the one or more pieces of program code, so that the computing device can Execute the transaction execution method above.
  • FIG. 1 is a schematic diagram of an implementation environment of a transaction execution method provided by an embodiment of the present application
  • FIG. 2 is a flow chart of a transaction execution method provided by an embodiment of the present application.
  • FIG. 3 is a flow chart of a transaction execution method provided by an embodiment of the present application.
  • FIG. 4 is a flow chart of a transaction execution method provided by an embodiment of the present application.
  • Fig. 5 is a flow chart of normal execution of online index addition provided by the embodiment of the present application.
  • Fig. 6 is a flow chart of recovery after abnormal interruption of online index addition provided by the embodiment of the present application.
  • FIG. 7 is a schematic diagram of a block split provided by an embodiment of the present application.
  • Fig. 8 is a schematic diagram of a block split provided by the embodiment of the present application.
  • Fig. 9 is a flow chart of normal execution of offline index addition provided by the embodiment of the present application.
  • Fig. 10 is a flow chart of recovery after an abnormal interruption of offline index addition provided by the embodiment of the present application.
  • Fig. 11 is a flow chart of normal parallel execution of online index addition provided by the embodiment of the present application.
  • Fig. 12 is a flow chart of parallel recovery after abnormal interruption of online index addition provided by the embodiment of the present application.
  • Fig. 13 is a schematic structural diagram of a transaction execution device provided by an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of a terminal provided in an embodiment of the present application.
  • FIG. 15 is a schematic structural diagram of a computing device provided by an embodiment of the present application.
  • first and second are used to distinguish the same or similar items with basically the same function and function. It should be understood that “first”, “second” and “nth” There are no logical or timing dependencies, nor are there restrictions on quantity or order of execution.
  • the term "at least one" means one or more, and the meaning of "multiple” means two or more, for example, a plurality of first positions means two or more first positions.
  • Cloud Technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network in a wide area network or a local area network to realize data calculation, storage, processing, and sharing, that is, a business model based on cloud computing
  • Cloud computing technology will become an important support in the field of cloud technology.
  • the background service of the technical network system requires a lot of computing and storage resources, such as video websites, picture websites or more portal websites. With the rapid development and application of the Internet industry, each item may have its own identification mark in the future, which needs to be transmitted to the background system for logical processing. Data of different levels will be processed separately, and all kinds of industry data need to be powerful.
  • the system backing support can be realized through cloud computing.
  • Cloud Storage It is a new concept extended and developed from the concept of cloud computing.
  • Distributed cloud storage system (hereinafter referred to as storage system) refers to the functions of cluster application, grid technology and distributed storage file system.
  • a storage system that integrates a large number of different types of storage devices (storage devices are also called storage nodes) in the network to work together through application software or application interfaces, and jointly provide data storage and service access functions.
  • Database In short, it can be regarded as an electronic file cabinet - a place where electronic files are stored, and supports users to add, query, update, delete and other operations on the data in the file.
  • the so-called “database” is a collection of data that is stored together in a certain way, can be shared with multiple users, has as little redundancy as possible, and is independent of the application program.
  • DDL statement operation refers to the statement used to modify the definition of objects in the database (such as: tables, indexes, columns, triggers, etc.).
  • Online DDL Online DDL is relative to offline DDL.
  • the object will be locked and then the change operation will be performed.
  • the above locking is used to block other business transactions (such as DML transactions) from modifying the data in the object during the execution of the DDL statement, so as to ensure the consistency of the data stored in the object operated by the DDL statement, where DML refers to Data Manipulate Language (data manipulation language).
  • DML Data Manipulate Language
  • business operation business transaction
  • various database vendors have proposed the implementation of online DDL, that is, when making DDL changes to an object, the implementation does not block the user's business or only blocks a small one. period.
  • Thomas Write is a timestamp-based concurrency control mechanism, which is summarized as ignoring obsolete writes. For example: transaction 1 starts to modify data A at T1, and transaction 2 also modifies data A at T2, T1 ⁇ T2. For some reason, transaction 1 and transaction 2 submit applications to the database at the same time. Under the optimistic transaction submission mechanism, The database will select the transaction that submitted the application first to commit, and roll back other conflicting transactions. However, under the Thomas write rule, the database needs to ensure that the commit of transaction 2 will succeed, because the timestamp of transaction 1 is changed under the Thomas write rule. Writes that are considered obsolete need to be rolled back in case of conflict. It should be noted that under the pessimistic transaction model, some data to be modified will be locked, causing other transactions to wait, so there will be no situation where two transactions modify the same data at the same time and submit them at the same time.
  • Tuple usually refers to a row of data records in a data table in a relational database. This data record stores the instantiation information of all columns in the table definition, and is arranged in the order of the column definitions to form a continuous Content, that is to say, this continuous content is called a data record of the data table, that is, Tuple.
  • Data dictionary The place where the database is used to store the definitions of objects in the database.
  • the objects in the database include: database, table space, data table, data column, data index, partition information, user, function, role and so on.
  • Parallelism how many parallel processing tasks will be started in parallel processing to decompose the original tasks that need to be processed; usually in computer processing, it is to divide the full amount of data that needs to be processed into several subtasks, and start the specified number of parallelism Threads obtain subtasks and process them according to the corresponding processing logic. Ideally, parallel execution can achieve performance improvement of multiples of parallelism.
  • Schema a collection of database objects
  • the cluster database system is an extension of the stand-alone database system.
  • the cluster includes multiple computing nodes (that is, computing devices), and each computing node stores multiple data tables in the database.
  • Each data table stores There are one or more data records (that is, data rows), and each data record is composed of a set of fields arranged according to the same position index, that is, data field columns.
  • the database of the computing node is any type of cluster database, including relational At least one of databases or non-relational databases, such as SQL databases, MySQL, NoSQL, NewSQL (generally referring to various new scalable/high-performance databases), etc., the type of database is not specifically limited in this embodiment of the application.
  • the cost of redoing the DDL statement is high for the database system.
  • the success rate of the DDL transaction is low.
  • the utilization rate of database resources is low.
  • the embodiment of the present application provides a transaction execution method, which is applied to various database systems such as traditional databases and distributed databases, and provides a solution for continuing to resume execution at the abnormal interruption point for DDL transactions.
  • the background thread or process can continue to complete the implementation of DDL at the last interrupted position, so that for DDL transactions that operate on relatively large objects (such as tables that store massive data), the success rate of DDL transactions can be improved, and because it can retain Part of the processing results that have been completed before the abnormal interruption improves the overall utilization of database resources.
  • the embodiments of the present application are also applied to a database system based on blockchain technology (hereinafter referred to as "blockchain system").
  • blockchain system is essentially a decentralized
  • the distributed database system uses a consensus algorithm to keep the ledger data recorded by different computing devices on the blockchain consistent, and uses a cryptographic algorithm to ensure the encrypted transmission of ledger data between different computing devices and cannot be tampered with.
  • the script system is used to expand the ledger function.
  • Network routing is used to interconnect different computing devices.
  • the blockchain system includes one or more blockchains.
  • the blockchain is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information for verification. The validity of its information (anti-counterfeiting) and the generation of the next block.
  • computing devices form a peer-to-peer (Peer To Peer, P2P) network.
  • the P2P protocol is an application layer protocol that runs on top of the Transmission Control Protocol (TCP).
  • computing devices have the following functions: 1) routing, the basic functions of computing devices, used to support communication between computing devices; 2) applications, used to deploy in the blockchain, according to the actual business Realize specific business according to the demand, record the data related to the realization function to form ledger data, carry digital signature in the ledger data to indicate the data source, and send the ledger data to other computing devices in the blockchain system for other computing devices to verify the ledger When the data source and integrity are successful, the ledger data is added to the temporary block.
  • the business implemented by the application includes wallet, shared ledger, smart contract, etc.; Once a new block is added to the blockchain, it will not be removed.
  • the block records the ledger data submitted by computing devices in the blockchain system.
  • each block includes the hash value of the transaction records stored in this block (the hash value of this block) and the hash value of the previous block, and each block is connected to form a block through the hash value.
  • the block also includes information such as the time stamp when the block was generated.
  • FIG. 1 is a schematic diagram of an implementation environment of a transaction execution method provided by an embodiment of the present application.
  • the embodiment of the present application is applicable to various database systems such as stand-alone databases, cluster databases, and cloud-native database architectures with separate computing and storage.
  • the following uses a distributed cluster database as an example for illustration.
  • the distributed cluster database system includes an application client 101, a gateway server 102, and a distributed storage cluster 103, and the distributed storage cluster 103 includes one or more computing devices (ie computing nodes).
  • the application client 101 refers to a client installed and running on a user-side terminal capable of initiating a data request.
  • the data request is a DDL request or a DML request, which is not specifically limited in this embodiment of the present application.
  • the types of application clients 101 include: payment applications, social networking applications, audio and video applications, live broadcast applications, shopping applications, food delivery applications, or taxi-hailing applications, etc.
  • the embodiment of the present application does not specifically limit the types of application clients 101 .
  • the terminal on the user side is also referred to as user equipment, terminal equipment, user terminal, mobile terminal, smart terminal, communication device, and the like.
  • the types of terminal devices include: smart phones, tablet computers, laptops, desktop computers, smart speakers, smart watches, vehicle terminals, smart home appliances, smart voice interaction devices, etc., but are not limited thereto.
  • the application client 101 and the gateway server 102 are directly or indirectly connected through wired or wireless communication, which is not limited in this application.
  • the gateway server 102 is used to receive external data requests, and distribute the read and write transactions corresponding to the data requests to the distributed storage cluster 103.
  • the user logs in to the application client 101 on the terminal, and triggers the application client 101 to generate a DDL request
  • the DDL request is to modify the table name of data table A
  • the application client 101 invokes the API (Application Programming Interface, Application Programming Interface) provided by the distributed cluster database system, and sends the DDL request to the gateway server 102
  • the API is the MySQL API (an API provided by a relational database system).
  • the request for description information of a new parking space is a DDL request
  • the request for querying an existing parking space is a DML request.
  • the gateway server 102 and any computing device in the distributed storage cluster 103 are combined on the same physical machine, that is, a certain computing device is allowed to serve as the gateway server 102.
  • the distributed storage cluster 103 includes one or more computing devices.
  • the embodiment of the present application does not specifically limit the number of computing devices in the distributed storage cluster 103.
  • the number of computing devices is m, and m is an integer greater than or equal to 1 .
  • each computing device adopts a master-standby structure (one master and multiple backup clusters), as shown in FIG.
  • each main machine or standby machine is equipped with an agent (Agent) device, and the agent device is physically independent from the main machine or the standby machine, or the agent device is used as an agent module on the main machine or the standby machine to Take computing device 1 as an example.
  • Computing device 1 includes a primary database and agent device (main database+agent, referred to as primary DB+agent), and also includes two backup databases and agent devices (standby database+agent, referred to as backup DB+agent) .
  • the set of database instances of each computing device's master or standby is called a SET (set).
  • SET set
  • the SET of the computing device is only the Assuming that a computing device is a cluster with one master and two backups, then the SET of the computing device is a collection of the master database instance and two backup database instances.
  • the distributed cluster database system composed of the above-mentioned gateway server 102 and distributed storage cluster 103 is regarded as a server that provides data services to user terminals.
  • the server is an independent physical server, or a plurality of A server cluster or distributed system composed of physical servers, or a cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN (Content Delivery Network , content distribution network) and cloud servers for basic cloud computing services such as big data and artificial intelligence platforms.
  • CDN Content Delivery Network , content distribution network
  • the objects of the database organize the data in the object by the underlying file system or storage method, and mapping the data in the object to the file system or storage system generally involves the following situations:
  • the database manages the data records in the data table in the form of a heap table, that is, organizes and stores the data records in the data table with a certain size of page (page) or block (block).
  • data records are stored in data blocks of a fixed size, usually called pages or blocks.
  • a data block is a 2 nk page, and the actual data is also stored in a 2 nk page.
  • the LSM data management mode is the Key-Value (key-value) data management mode.
  • Key-Value pairs key-value pairs
  • block mode In order to improve the efficiency of reading and writing, a certain number of Key-Value pairs (key-value pairs) are usually organized in block mode.
  • region In some databases, in order to do When storage can be expanded horizontally, the definition of region (domain) is introduced.
  • a region is a logical mapping of one or more physical blocks. By adding a logical mapping, the real storage location of the data associated with the region can be shielded. In other words, data of a certain size under the LSM storage method will form the concept of block or region, and a region or block has a certain data range. Therefore, the LSM data management mode is also considered to store data in a block manner.
  • the embodiment of the present application will describe the abnormal interruption recovery execution flow of the transaction execution method.
  • the embodiments of the present application are applicable to stand-alone database systems, cluster database systems, distributed database systems, cloud-native database architectures with separate computing and storage, blockchain systems, etc., and are not specifically limited here.
  • Fig. 2 is a flow chart of a transaction execution method provided by an embodiment of the present application. Referring to Fig. 2, this embodiment is carried out by any computing device in the database system, and this embodiment comprises the following steps:
  • the computing device determines the last data block processed by the DDL transaction before the execution interruption, in response to the data definition language DDL transaction whose execution is interrupted meets the recovery condition.
  • the user logs in to the application client on the terminal, triggering the application client to generate a DDL request, for example, the DDL request is to add an index (Index) to data table A, or the DDL request is to modify the table of data table A name, etc.
  • the embodiment of the present application does not specifically limit the type of the DDL request.
  • the application client After the application client generates the DDL request, it calls the API to send the DDL request to the computing device.
  • the computing device receives any data request from the application client, parses the header field of the data request, and creates a new DDL process or thread for the DDL request when the header field indicates that the data request is a DDL request , or reuse a created DDL process or thread, and execute the DDL transaction associated with the DDL request through the DDL process or thread.
  • the computing device executes the DDL transaction serially, or one or more computing devices in the database system execute the DDL transaction in parallel, for example, multiple DDL threads on a single computing device execute the DDL transaction in parallel, Alternatively, multiple DDL threads on multiple computing devices execute the DDL transaction in parallel, which is not specifically limited in this embodiment of the present application.
  • the execution of the DDL transaction is interrupted due to some reason (for example, process or thread crash, flashback, etc.), after the database system is restarted, if the background DDL execution detector detects To a DDL transaction whose execution is interrupted, if the DDL transaction meets the recovery conditions, since the data block is used as the smallest unit of transaction processing, the last data block processed before the execution interruption of the DDL transaction is determined, and Execute the following step 202 .
  • the DDL execution detector is used to periodically detect whether any DDL transaction in the database is abnormally executed.
  • each DDL transaction is divided into two stages of execution.
  • the definition of the object operated by the DDL transaction is modified in the data dictionary of the data table operated by the DDL transaction. For example, if the DDL transaction is to add An index, then in the first stage, it is necessary to modify the data dictionary of the data table to add the definition of the index, in order to complete the addition of the index structure in the table object, and set the index to the Delete Only (deletion only allowed) state, at this time the table
  • the index definition is added in the structure, but the index cannot be selected by the optimizer, and the newly added index cannot be used in the query to query data; in the second stage, it is necessary to scan the entire table data of the DDL transaction operation to Process each data record involved in the object of the DDL transaction operation.
  • the task of filling the index data needs to be performed synchronously or asynchronously, that is, to scan the entire table data, for each piece of data Record (Tuple) to create the Key value (key value) of the index.
  • the index needs to be set to the Write Only (write only) state, and the transaction that modifies the table data started before the Write Only state, if the transaction is not committed , then after the Write Only state is successfully set, the database engine will reject the transaction submission at the time of commit so that the transaction will be rolled back.
  • a transaction identification (Identification, ID) is also obtained or applied for, and the transaction identification is used to indicate that the index starts from the earliest after the Write Only state is set.
  • the transaction that is, the DDL transaction associated with the DDL request
  • the first phase is regarded as the initialization phase of the DDL transaction
  • the second phase is regarded as the execution phase of the DDL transaction, wherein the transaction identifier increases monotonically with the timestamp.
  • the transaction identifier is allocated based on preset rules, and in the cluster database, a transaction identifier is applied to a device (such as a coordinating node) for generating a global transaction identifier.
  • a device such as a coordinating node
  • the embodiment of the present application does not apply to the transaction identifier
  • the acquisition method is specifically limited.
  • the entire table data is scanned based on the DDL transaction. For example, if the DDL transaction adds an index, the Thomas write rule is used to fill the index Key value of each data record. In addition, after setting the index to the Write Only state, all modification transactions for the data table must also update the corresponding index. However, since the Write Only state index has not yet completed data filling, it is added under the second stage The index cannot be used by the optimizer to query data.
  • the Thomas write rule for user operations (such as DML transactions) that start after the DDL transaction that scans the full table and populates the index and modifies the index, since the timestamp of the DML transaction is newer than the timestamp of the scanned DDL transaction, resulting in
  • the DDL transaction fills the index Key value
  • the DDL filling transaction directly Discard the current Key value and continue the subsequent record scan.
  • the embodiment of the present application is applicable to both online DDL scenarios and offline DDL scenarios.
  • the recovery conditions of DDL transactions are different in different scenarios, which will be described below.
  • the DDL transaction is an online DDL transaction
  • the database engine rolls back the DDL transaction:
  • the first stage needs to modify the data dictionary of the data table to add the definition of the index, and set the index to the Delete Only (deletion only) state. If the DDL transaction has not completed the first stage, then the There is no definition of the index in the data dictionary of the data table. For example, if the index information cannot be queried in the data dictionary of the data table, the DDL transaction cannot be continued from the abnormal interruption point, and the DDL transaction will be rolled back.
  • the above (I) and (II) introduce the two situations that do not meet the recovery conditions in the online DDL scenario, then the rest of the situations that do not belong to the above (I) and (II) meet the recovery conditions, that is to say, the online DDL scenario
  • the following recovery conditions include: the data dictionary of the data table of the DDL transaction operation contains the definition of the object of the DDL transaction operation, and the data block of the DDL transaction operation or at least one copy of the data block is readable.
  • the execution process of the offline DDL transaction is relatively simple.
  • a temporary data table is created after the DDL transaction is executed, and the data records in the original data table are copied to the In the temporary data table, after completing the corresponding processing on the objects in the temporary data table, modify the table name of the original data table, change the table name of the temporary data table to the table name of the original data table, and delete the original data table asynchronously.
  • the recovery conditions in the offline DDL scenario include: both the original data table operated by the DDL transaction and the temporary data table created by the DDL transaction exist. Resume scanning at the point of interruption.
  • the recovery conditions are not met, it means that at least one item in the original data table or the temporary data table does not exist. At this time, it is judged whether to roll back the DDL transaction or continue to advance the DDL transaction. If the original data table exists, but the temporary data table has not been created, roll back the DDL transaction when this situation is met, otherwise, it can be divided into the following three cases (a) to (c) for discussion, when the following conditions are met ( Any one of a) to (c), means that the entire table has been scanned before the execution is interrupted, and there is no need to roll back the DDL transaction, just continue to advance the DDL transaction:
  • the temporary data table is renamed the original data table, and another temporary data table formed after the original data table is renamed also exists, indicating that the scanning of the original data table is completed, and the DDL transaction can be continued, in other words, continue to be deleted from the data dictionary
  • the other temporary data table and asynchronously delete the data in the other temporary data table;
  • the temporary data table is renamed to the original data table.
  • Another temporary data table formed after the original data table is renamed does not exist. Just continue to advance the DDL transaction. In other words, continue to confirm the asynchronous completion and delete the data in the other temporary data table .
  • the process of judging whether a DDL transaction meets the recovery conditions in the online DDL scenario and the offline DDL scenario is introduced respectively.
  • the computing device determines the DDL transaction The last block processed before the execution interrupt. It has been introduced before the embodiment of this application that no matter it is for the heap table data management mode, B+ tree data management mode, or LSM data management mode, the block can be used as the smallest unit for processing DDL transactions. Therefore, for each DDL transaction, add A management data, which is used to record each block that has been processed by the DDL transaction. Whenever a DDL transaction completes the corresponding processing for each data record pair in a block, the processed block will be recorded in the management data . Optionally, in order to save storage space, only the block ID of each block is recorded in the management data, which is also called the block management structure.
  • the computing device acquires the management data of the DDL transaction, and from the management data, obtains the last block processed by the DDL transaction before execution interruption. Since the block is the smallest unit for processing DDL transactions, and management data is maintained for each DDL transaction, each block that has been processed will be recorded in the management data, so that after the execution of the DDL transaction is interrupted, it can be conveniently retrieved from the management data. Locate the abnormal interruption point in the middle, and continue to execute the DDL transaction from the abnormal interruption point, without redoing or overall rollback from the beginning, which greatly improves the success rate of the DDL transaction.
  • the computing device continues to execute the DDL transaction starting from the next data block of the last data block.
  • each block since each block is stored in the data table in sequence, after locating the last block processed before the execution interrupt, it means that all blocks before the last block have been processed. There is no need to repeat the process, just locate the next block of the last block from the data table, and continue to execute the DDL transaction from the next block. For example, the DDL transaction is to add an index and scan in the second phase When block10 of the data table is reached, it is interrupted abnormally. At this time, block10 has not yet completed processing, so block10 will not be recorded in the management data, but blocks1 ⁇ 9 that have been processed are recorded.
  • the computing device continues to execute the DDL transaction starting from the next block10 of block9, which can avoid rolling back the DDL transaction as a whole, and does not need to redo it from block1, which will generate a redundant workload of one block at most ( That is, an interruption occurs when the last data record of block10 is processed, and the redundant workload of half a block is increased on average), which greatly improves the resource utilization rate of the database.
  • the computing device submits the DDL transaction in response to the processing of each data block operated on the DDL transaction.
  • the computing device processes the DDL transaction in units of blocks, that is, reads one block at a time, and then processes each data record in the read block, and the above-mentioned processing is based on the type of the DDL transaction It depends. For example, if a DDL transaction is to add an index, then the Key value of the index needs to be filled for each data record. After processing all data records of this block, read the next block, and perform the same processing on each data record in the next block, and so on, iteratively perform the above operations until there is no block to be processed, It means that all blocks are processed, and the DDL transaction is submitted at this time. For example, the DDL transaction is to add an index. When the DDL transaction is submitted, the index is set to Public (published) state, and the DDL transaction enters Commit (submit) stage, marking the end of the DDL task.
  • the technical solution provided by the embodiment of the present application uses a data block as the smallest unit for processing a DDL transaction.
  • the last data block that has been processed before the execution interruption can be easily located without the need to roll back the DDL transaction as a whole. , but continues to execute the DDL transaction from the next data block, avoiding the redundant workload of redoing the DDL transaction from the beginning, improving the success rate of the DDL transaction indicated by the DDL statement, and improving the resource utilization of the database.
  • Fig. 3 is a flowchart of a transaction execution method provided by an embodiment of the present application. Referring to Fig. 3, this embodiment is executed by any computing device in the database system, and the database system includes: stand-alone database system, cluster database system, distributed database system, cloud-native database architecture with separation of computing and storage, block chain system, etc. , this embodiment includes the following steps:
  • the computing device acquires management data of the DDL transaction in response to the execution of the interrupted DDL transaction meeting the recovery condition, where the management data is used to record each block of the DDL transaction that has been processed.
  • the computing device When the computing device detects that the interrupted DDL transaction meets the recovery condition, it obtains the management data of the DDL transaction. Since the management data is used to record each block that has been processed by the DDL transaction, when the DDL transaction is processing each block When each data record stored in the block has been processed, the block ID (data block identifier) of the block that has been processed is recorded in the management data.
  • the management data which is also called a block management structure, so that the amount of stored data occupied by the management data can be saved.
  • the management data is a dynamic array, which stores the block ID of each block that has been processed by the DDL transaction, and the last element in the dynamic array stores the block ID of the last block that has been processed.
  • management data is only an exemplary description of management data, and the management data is also provided as other data structures such as hash tables, sets, queues, and stacks, which are not specifically limited in this embodiment of the present application.
  • the computing device when it enters the second phase of the DDL transaction, it obtains the transaction ID of the DDL transaction, creates management data for the DDL transaction, and then maintains the management data in real time as the DDL transaction processes the data table.
  • Data that is to say, as the DDL transaction scans the entire table data, and a block in each pair of data tables is scanned, the latest scanned block is recorded in the management data, for example, the block of the latest scanned block
  • the ID is stored into the last element of the dynamic array.
  • the execution of the DDL transaction is interrupted, and the DDL execution detector finds that the DDL transaction meets the recovery condition, and obtains the management data of the DDL transaction from the cache, for example, obtains the dynamic array of the DDL transaction from the cache.
  • the computing device queries to obtain the last block processed by the DDL transaction before execution interruption.
  • each block in the data table is usually processed in ascending order of blocks
  • the computing device when querying, the computing device reads the DDL transaction that has been processed before execution interruption from the management data.
  • the block ID of each block, the block with the largest block ID is determined as the last block processed before the execution interrupt.
  • the management data is a dynamic array
  • since the last element in the dynamic array stores the block ID of the last processed block
  • the computing device when querying, the computing device will store the last block ID in the dynamic array.
  • the block indicated by the block ID stored in an element is determined to be the last block processed before the execution interrupt.
  • a possible implementation of determining the last block of the DDL transaction that is processed before the execution interruption is provided. Since the block is used as the smallest unit for processing DDL transactions, and each DDL transaction is maintained In addition to the management data, each block that has been processed will be recorded in the management data, so that after the execution of the DDL transaction is interrupted, it is convenient to locate the abnormal interruption point from the management data, and continue to execute the DDL transaction from the abnormal interruption point. There is no need to redo or rollback from scratch, which greatly improves the success rate of DDL transactions.
  • the computing device reads the block next to the last block or any block after the next block, and processes each data record stored in the block.
  • each block since each block is stored in the data table in sequence, after locating the last block processed before the execution interrupt, it means that all blocks before the last block have been processed. There is no need to repeat processing, just locate the next block of the last block from the data table, and continue to execute the DDL transaction from the next block of the last block.
  • the DDL transaction is to add an index.
  • block10 of the data table is scanned, it is abnormally interrupted. At this time, block10 has not yet completed processing, so block10 will not be recorded in the management data, but the processed block1 ⁇ 9. It is determined that the last block processed before the abnormal interruption is block9. Therefore, the computing device starts from the next block10 of block9 and continues to execute the DDL transaction, which can avoid rolling back the entire DDL transaction and does not need to start from block1 Redoing will generate a redundant workload of one block at most (that is, an interruption occurs when the last data record of block10 is processed), which greatly improves the resource utilization of the database.
  • the computing device When continuing to execute the DDL transaction from the next block of the last block, any block after the last block (including the next block, or any block after the next block), due to the block Therefore, the computing device needs to read data in units of blocks, that is, the computing device reads the current block and processes each data record stored in the current block.
  • the above processing is based on the type of DDL transaction. For example, a DDL transaction is to add an index, and the Key value of the index needs to be filled for each data record.
  • the processed block means that all blocks have been processed, and the following step 306 is executed.
  • the computing device acquires the data progress of the DDL transaction based on the management data and the total number of data blocks to be processed by the DDL transaction.
  • a function of externally querying the data progress of DDL transactions in real time is provided. Since the block is used as the smallest unit for processing DDL transactions, the number of blocks currently processed by DDL transactions (that is, the number of records recorded in the management data) is compared. number of blocks) and the total number of blocks to be processed by the DDL transaction, that is, the data progress of the DDL transaction.
  • the computing device determines the value obtained by dividing the number of blocks recorded in the management data by the total number of blocks as the data progress of the DDL transaction.
  • the numerical value is converted into a percentage and then determined as the data progress of the DDL transaction, or other linear or nonlinear mapping is performed, which is not specifically limited in this embodiment of the present application.
  • the computing device returns the data progress of the DDL transaction to the device that triggered the progress query instruction.
  • the user inputs a progress query command on the computing device, and the database engine queries and returns the data progress of the DDL transaction from the bottom layer. At this time, the computing device itself visually displays the data progress. If it is a cluster database system, the device where the user enters the progress query command is usually different from the computing device that processes the DDL transaction. At this time, the computing device returns the data progress of the DDL transaction to the device on the user side based on the progress query command, and the device on the user side checks it. The data progress is visualized.
  • the progress bar includes: a bar-shaped progress bar, a fan-shaped progress bar, a ring-shaped progress bar, and the like.
  • the user can enter the following code to display the data progress of the DDL transaction while entering the DDL statement.
  • the DDL transaction Take the DDL transaction as an example to create a new data table:
  • *public* means that the current state of the primary key index is Public, that is, the normal state.
  • the object operated by the DDL transaction will be displayed as the Public state, indicating that the object is currently in the stage where it can provide services to users normally.
  • parallel degree 24* means that the current state of the secondary index idx1 is Write Only, that is, it is in the stage of scanning table data to generate index data during the index creation process, and at the same time, it shows that the data progress of the current index creation is 66% , and process data in parallel with a degree of parallelism of 24.
  • the computing device submits the DDL transaction in response to the processing of each data block operated on the DDL transaction.
  • the above-mentioned step 306 is the same as the above-mentioned step 203, which will not be repeated here.
  • the technical solution provided by the embodiment of the present application uses a data block as the smallest unit for processing a DDL transaction.
  • the last data block that has been processed before the execution interruption can be easily located without the need to roll back the DDL transaction as a whole. , but continues to execute the DDL transaction from the next data block, avoiding the redundant workload of redoing the DDL transaction from the beginning, improving the success rate of the DDL transaction indicated by the DDL statement, and improving the resource utilization of the database.
  • Fig. 4 is a flowchart of a transaction execution method provided by an embodiment of the present application. Referring to Fig. 4, this embodiment is executed by any computing device in the database system, and the database system includes: stand-alone database system, cluster database system, distributed database system, cloud-native database architecture with separation of computing and storage, block chain system, etc. , this embodiment includes the following steps:
  • the computing device obtains management data of the DDL transaction in response to the execution of the interrupted DDL transaction meeting the recovery condition, where the management data is used to record each block of the DDL transaction that has been processed.
  • the above step 401 is the same as the above step 301, and will not be repeated here.
  • the computing device queries to obtain the last block processed by the DDL transaction before execution interruption.
  • the above-mentioned step 402 is the same as the above-mentioned step 302, which will not be repeated here.
  • the computing device Based on the parallelism of the DDL transaction, the computing device performs parallel processing on blocks that have not been processed in the DDL transaction starting from a block next to the last block.
  • the degree of parallelism refers to how many parallel processing tasks will be started in parallel processing to decompose the original tasks that need to be processed.
  • it is to divide the full amount of data that needs to be processed into several subtasks, start a number of threads with a specified number of parallelism, obtain subtasks, and process them according to the corresponding processing logic.
  • parallel execution can achieve parallelism. For example, in the serial processing mode, a single DDL thread scans the entire table data, and in the parallel processing mode, assuming that the parallelism is 10, 10 DDL threads scan the entire table data in parallel.
  • the degree of parallelism is preset by the database engine, for example, a certain default value (for example, the degree of parallelism is 5) is preset, and for example, multiple degrees of parallelism are preset, and each degree of parallelism is established
  • the associated data volume interval forms a mapping relationship between the parallelism and the data volume interval.
  • select the parallelism that has a mapping relationship with the data volume interval where the data volume is located and For example, by default, the degree of parallelism adopted by the DDL transaction before the execution interruption, or the degree of parallelism adopted by the previous DDL transaction is inherited by default.
  • the degree of parallelism is set by the technician after the execution interruption. This application The embodiment does not specifically limit the manner of obtaining the degree of parallelism.
  • the DDL transaction is continued to be executed from the next block of the last block, and the difference from the above step 303 is that the serial processing is converted into parallel processing.
  • the parallel processing mode based on the parallelism of the DDL transaction (different DDL transactions have the same or different parallelism, not limited here), the original single DDL transaction can be divided into multiple DDL sub-transactions, where the DDL sub-transaction The number of transactions is equal to the parallelism of the DDL transaction, and the processes or threads of the parallelism number are used to process the multiple DDL sub-transactions in parallel.
  • each DDL thread or process processes its own DDL sub-transaction in the parallel processing mode, it is the same as the DDL thread or process processing the DDL transaction in the serial processing mode, which will not be repeated here.
  • these processes or threads can be implemented using multi-threaded parallel processing capabilities; in a cluster database, these processes or threads can be distributed on multiple computing devices to achieve the effect of distributed parallel processing , which can relieve the computing pressure of a single node.
  • a DDL transaction when a DDL transaction is divided into multiple DDL sub-transactions based on the degree of parallelism, it can be equally divided according to the total number of blocks to be processed by the DDL transaction, so that tasks allocated to different processes or threads are more load-balanced.
  • the DDL transaction when the DDL transaction is divided into multiple DDL sub-transactions based on the degree of parallelism, since the respective processes or threads of the DDL sub-transactions may not be located on the same computing device, and the computing device where the DDL sub-transaction is located is currently
  • the available computing resources are also different, so it is not necessary to divide them equally according to the total number of blocks that DDL transactions need to process, but to flexibly allocate them according to the currently available computing resources of the computing devices where the DDL sub-transactions are located. For example, the currently available computing resources When the resource is greater than the preset threshold, there are more blocks to be scanned in the allocated DDL sub-transaction.
  • the threshold is set to be a fixed value or a percentage, which is not specifically limited in this embodiment of the present application.
  • a DDL transaction when a DDL transaction is divided into multiple DDL sub-transactions based on the degree of parallelism, taking the distributed scenario as an example, since different blocks may be distributed and stored on different computing devices, it can be based on different The computing device where the block of the block is located divides DDL sub-transactions for each computing device. For example, first determine which computing devices are involved in the object of the DDL transaction operation, and then allocate storage in the computing device for each computing device. A DDL subtransaction that computes data on the device.
  • the computing device acquires the first block set and the second block set of the DDL transaction, and the first block set is used to record each block in the data table that the DDL transaction needs to operate at the beginning of execution , the second data block set is used to record each block in the data table that the DDL transaction needs to operate when the parallel processing is completed.
  • the user business can modify the data (using the Thomas write rule), so in the process of scanning the whole table by the DDL transaction, the block may be split, that is, an original block is inserted into the data record As a result, it is split into multiple blocks. At this time, the new blocks that are split may be missed under parallel scanning, resulting in data inconsistency. Therefore, after the parallel processing is completed, the above step 404 is executed to obtain the first block set and the second block set to solve the inconsistency problem caused by block splitting.
  • the first block set refers to the set formed by collecting all block IDs in the data table to be scanned when the DDL transaction starts parallel scanning in the second phase, which is recorded as blockIdInit and used as the baseline of parallel scanning.
  • the second block set refers to the set formed by re-collecting all block IDs in the data table to be scanned after the parallel scanning of the DDL transaction in the second phase, denoted as blockIdCur, by comparing the first block set and the second block set
  • blockIdCur the parallel scanning of the DDL transaction in the second phase
  • the first set of blocks will be stored once when the parallel scan is started (usually before the execution is interrupted), and when the execution is resumed after the execution is interrupted, once the parallel scan is completed, the first block set is read from the cache , and collect again to obtain the second block set, compare the size of the first block set and the second block set, that is, compare the block IDs (ie, set elements) stored in the first block set and the second block set, if the first If the first block set is consistent with the second block set, it means that there is no block split in this data table during DDL execution, and the following step 405 is executed to directly submit the DDL transaction; if the first block set is inconsistent with the second block set, it means that during DDL execution A block split occurs in this data table, that is, a new block is added in the second block set compared with the first block set, and the computing device determines that the second block set is compared with the newly added block in the first block set. After processing the newly added block, submit the DDL transaction.
  • the above-mentioned newly added block is processed serially, that is, the same operation as step 303 in the previous embodiment is performed, or, the above-mentioned newly added block is processed in parallel, that is, the above step 403 ( It can be switched to another degree of parallelism, or the original degree of parallelism can be used), which is not specifically limited in this embodiment of the present application.
  • the computing device submits the DDL transaction when the first block set is consistent with the second block set.
  • the process of submitting the DDL transaction in the above step 405 is the same as the process of submitting the DDL transaction in the above step 306, and will not be repeated here.
  • operations similar to the above steps 304-305 can also be performed to obtain and return the data progress of the DDL transaction to the device on the user side in the parallel processing mode.
  • the Obtain and return the total data progress of the DDL transaction can also be performed.
  • it also supports obtaining and returning the respective sub-data progress for each DDL sub-transaction, and also supports returning the parallelism of the device on the user side to improve the efficiency of human-computer interaction. .
  • the technical solution provided by the embodiment of the present application uses a data block as the smallest unit for processing a DDL transaction.
  • the last data block that has been processed before the execution interruption can be easily located without the need to roll back the DDL transaction as a whole. , but continues to execute the DDL transaction from the next data block, avoiding the redundant workload of redoing the DDL transaction from the beginning, improving the success rate of the DDL transaction indicated by the DDL statement, and improving the resource utilization of the database.
  • online DDL transactions are divided into two types: one is the DDL transaction that can be completed only by modifying the table definition in the data dictionary. For example, adding a data column to the data table only needs to modify the table definition in the online DDL execution process. That’s it; the other kind not only needs to modify the definition of the table in the data dictionary, but also needs to complete the process of creating the data of the corresponding object based on the data of the table. For example, adding an index to the data table requires modifying the index definition and scanning the entire Table data and use the column information involved in the corresponding index to create the Key information content (that is, the Key value) corresponding to the index.
  • the embodiment of this application is mainly aimed at the above-mentioned latter case to achieve intermittent operation, because the former case itself does not need to scan the entire table data. If an exception occurs, even if it is rolled back, the next re-execution only needs to complete the modification of the table in the data dictionary. The definition is sufficient, and the cost of redoing is not high.
  • Fig. 5 is a flow chart of normal execution of adding an index online provided by the embodiment of the present application. Please refer to Fig. 5. This embodiment is executed by any computing device in the database system, and the DDL transaction is used as an example to add an index online for illustration. .
  • Step 1 Start a DDL task to add an index (Add Index).
  • Step 2 Modify the data dictionary corresponding to the data table, add an index to the table structure, and set the current state of the index to the Delete Only state. In the Delete Only state, only the index can be deleted.
  • Step 3 Submit the transaction and save the new table structure to the data dictionary.
  • Step 4 Start the second phase of DDL execution.
  • Step 5 After obtaining the corresponding permission, change the index status to Write Only status, and obtain a transaction ID at the same time.
  • the step of obtaining permissions is optional to improve the security of DDL transactions, but it is not necessary to obtain permissions before modifying the index status, and it is also supported to directly modify the index status without obtaining permissions to improve the execution efficiency of DDL transactions.
  • the embodiments of the application do not specifically limit this.
  • Step 6 Start the full table scan and complete the following tasks:
  • Step 6.1 use the transaction ID obtained in step 5 as the transaction ID for scanning and establishing secondary index data transactions;
  • Step 6.2 Collect the total number of blocks contained in the table and register them in the DDL task as the total workload value total blocks of the DDL task.
  • step 6 start to scan the full table data, and record the total number of blocks (that is, the total workload total blocks) required to scan the full table data into the DDL task.
  • Step 7 start scanning, and record the above transaction ID in the DDL task.
  • Step 8 Obtain the first block.
  • Step 9 Process each data record in the block, extract the corresponding data field to create the key of the secondary index, and write it into the index structure using the Thomas writing rule. Since the transaction ID is applied when the index status becomes Write Only, according to the Thomas write rule, the previously uncommitted transactions are rolled back; in addition, the above-mentioned transactions that scan and create indexes are more efficient than any subsequent users modifying the table. The transactions are all old, so if there is a conflict between the two, you only need to keep the subsequent user's modification of the corresponding data in the table.
  • Step 10 Determine whether the data records in the block have been processed? If yes, the data records in the block have been processed, go to step 11; if not, the data records in the block have not been processed, return to step 9 to continue the processing of the next data record.
  • Step 11 register in the management data of DDL (such as the Map dynamic array structure) that the block has been scanned, and update the total number of blocks processed block num that has been scanned.
  • DDL such as the Map dynamic array structure
  • step 11 register the current block in the Map structure of DDL, and update the currently processed block number (i.e. processed block num).
  • the data progress of the DDL transaction is equal to processed block num/total blocks.
  • the execution progress of the current DDL transaction is calculated for visual display on the foreground.
  • Step 12 Obtain the next block.
  • Step 13 Determine whether there are still blocks to be processed? If yes, that is, there are still blocks to be processed, return to step 9 to continue processing; if not, that is, there are no blocks to be processed, indicating that the index creation process is over, and go to step 14.
  • Step 14 Modify the index status to Public (indicating that the index is created and can be used normally).
  • Step 15 marking the end of the DDL task.
  • Figure 6 is a flow chart of recovery after an abnormal interruption of online index addition provided by the embodiment of the present application, please refer to Figure 6, this embodiment is executed by any computing device in the database system, taking DDL transaction as an example of online index addition Be explained.
  • Step 1 The background thread finds an abnormally interrupted Add Index task.
  • Step 2 Determine whether the execution can continue? If no, it cannot continue to execute, go to step 3; if yes, it can continue to execute, go to step 4.
  • Step 3 Roll back the DDL transaction and go to step 13.
  • Step 4. Find the last block that has been processed through the management data recorded in the DDL task (such as the Map dynamic array structure).
  • Step 5 Find the next block through the last block found in step 4.
  • the last block (leaf block) can be linked to the next block.
  • the corresponding next region can be found through the region to continue scanning.
  • Step 6 Determine whether to find the next block? If yes, the next block is found, go to step 7; if no, the next block is not found, go to step 12.
  • Step 7 Process each data record in the block, extract the corresponding data field to create the key of the secondary index, and write it into the index structure using the Thomas writing rule. Since the transaction ID is applied when the index status becomes Write Only, according to the Thomas write rule, the previously uncommitted transactions are rolled back; in addition, the above-mentioned transactions that scan and create indexes are more efficient than any subsequent users modifying the table. The transactions are all old, so if there is a conflict between the two, you only need to keep the subsequent user's modification of the corresponding data in the table.
  • Step 8 Determine whether the data records in the block have been processed? If yes, the data records in the block have been processed, go to step 9; if not, the data records in the block have not been processed, return to step 7 to continue the processing of the next data record.
  • Step 9 Register in the management data of DDL (such as the Map dynamic array structure) that the block has been scanned, and update the total number of blocks processed block num that has been scanned.
  • DDL such as the Map dynamic array structure
  • step 9 register the current block into the DDL Map structure, and update the number of currently processed blocks (that is, processed block num).
  • the data progress of the DDL transaction is equal to processed block num/total blocks.
  • the execution progress of the current DDL transaction is calculated for visual display in the foreground.
  • Step 10 obtain the next block.
  • Step 11 Determine whether there are still blocks to be processed? If yes, that is, there are still blocks to be processed, return to step 7 to continue processing; if not, that is, there are no blocks to be processed, indicating that the index creation process is over, and go to step 12.
  • Step 12. Modify the index status to Public (indicating that the index is created and can be used normally).
  • Step 13 marking the end of the DDL task.
  • Scenario 1 The split or merged blocks have been scanned and processed.
  • Figure 7 is a schematic diagram of a block split provided by the embodiment of the present application. Assuming that block n is an interrupt breakpoint, for block m (m ⁇ n) that has been scanned and processed before block n, the above block m is split into block m and block m'. Usually, the reason for block splitting is that a data record is added in block m, or the field of the data record is modified to exceed the size that the block can hold, thus splitting into two blocks. Because the index status is Write Only when scanning starts, all new additions or modifications to records in block m will be reflected in the newly created index, so changes in this part of data will be synchronized to the index. Figure 7 shows block m direction In the case of splitting on the right, if block m splits to the left, it will not have any effect.
  • Block m and block m' are merged into block m.
  • the merge is because block m or block m' is deleted If the data record is deleted or the data record is modified, the space usage or space usage rate in block m or block m' is less than a certain threshold. In order to make better use of the space, the two blocks will be merged, and the merged block will become the above figure.
  • Block m as in the case of splitting above, deleting or modifying data records will also be updated to the index, so the index data is also correct.
  • Scenario 2 The split or merged block is the currently scanned block.
  • the above scenario 2 is also divided into two situations. First, the currently processed data is recorded in the split block m, and second, the currently processed data is recorded in the split block m'.
  • Fig. 8 is a schematic diagram of a block split provided by the embodiment of the present application. It is assumed that the DDL transaction is interrupted when block m is currently being processed, and the task is restarted after the interruption, and block m is split into block m and block m'. At this time, find the previous completed block m-1 of block m, and start rescanning from the next block m of block m-1.
  • Fig. 9 is a flow chart of normal execution of adding an offline index provided by the embodiment of the present application. Please refer to Fig. 9. This embodiment is executed by any computing device in the database system, and the DDL transaction is used as an example for adding an index offline for illustration. .
  • Step 1 Start an offline (non-online) DDL task for adding indexes.
  • Step 2 Create a temporary data table of the table structure after DDL execution, that is, create a newly defined temporary data table.
  • Step 3 Apply for the permission corresponding to the data table (apply for a lock).
  • Step 4 Set the state of the table to No Write (forbidden to write) state.
  • the No Write state is a table state in a special offline DDL scenario.
  • the table definition is set to the No Write state, other nodes in the distributed database are not allowed to perform or submit the modification transaction of the table, even if the table state is in The transaction started before entering the No Write state, and the table state is found to be No Write when it is submitted, then the database engine will also reject the transaction submission.
  • Step 5 Start the full table scan, collect the total number of blocks contained in the table, and register them in the DDL task as the total workload value total blocks of the DDL task.
  • step 5 start to scan the full table data, and record the total number of blocks (that is, the total workload total blocks) required to scan the full table data into the DDL task.
  • Step 6 Obtain the first block.
  • Step 7 Process each data record in the block, combine each data record in the original data table according to the new data table structure into a new data record, and insert the temporary data table created in step 2, that is, according to the new table structure After modification, insert it into the temporary data table created in step 2.
  • Step 8 Determine whether the data records in the block have been processed? If yes, the data records in the block have been processed, go to step 9; if not, the data records in the block have not been processed, return to step 7 to continue the processing of the next data record.
  • Step 9 Register in the management data of DDL (such as the Map dynamic array structure) that the block has been scanned, and update the total number of blocks processed block num that has been scanned.
  • DDL such as the Map dynamic array structure
  • step 9 register the current block into the DDL Map structure, and update the number of currently processed blocks (that is, processed block num).
  • the data progress of the DDL transaction is equal to processed block num/total blocks.
  • the execution progress of the current DDL transaction is calculated for visual display on the foreground.
  • Step 10 obtain the next block.
  • Step 11 Determine whether there are still blocks to be processed? If yes, that is, there are still blocks to be processed, return to step 7 to continue processing; if not, that is, there are no blocks to be processed, indicating that the index creation process is over, and go to step 12.
  • Step 12 Modify the table name of the original data table, change the table name of the temporary data table created in step 2 to the table name of the original data table, and delete the original data table asynchronously.
  • Step 13 marking the end of the DDL task.
  • Figure 10 is a flow chart of recovery after an abnormal interruption of adding an offline index provided by the embodiment of the present application, please refer to Figure 10, this embodiment is executed by any computing device in the database system, taking DDL transactions as an example for adding an index offline Be explained.
  • Step 1 The background thread finds an abnormally interrupted DDL task.
  • Step 2 Determine whether the execution can continue? If yes, both the original data table and the temporary data table exist, you need to resume scanning from the interruption point of the scanning table process, and go to step 4; if no, that is, the above situation is not satisfied, that is, at least one of the original data table or the temporary data table If it does not exist, go to step 3.
  • Step 3 Determine whether to roll back the DDL transaction? If yes, go to step 13, that is, only the original data table exists, and the temporary data table has not been created, and the DDL transaction is rolled back; if not, there are three cases: a) the original data table is renamed successfully, and the temporary data table is also there, Scanning the entire table has ended, continue to advance; b) The temporary data table is renamed the original data table, and the temporary data table after the original data table is renamed also exists, continue to advance, delete the temporary data table from the data dictionary, and delete the temporary data table asynchronously c) The temporary data table is renamed the original data table, the temporary data table after the original data table is renamed does not exist, continue to advance, confirm that the asynchronous completion of the deletion of the data records in the temporary data table meets a)-c) Either way, go to step 12.
  • Step 4 Apply for the corresponding modification permission of the form (apply for a lock). At this time, the state of the table is still No Write.
  • Step 5 Find the last block that has been processed through the management data recorded in the DDL task (such as the Map dynamic array structure).
  • Step 6 Determine whether to find the next block? If yes, the next block is found, go to step 7; if no, the next block is not found, go to step 12.
  • Step 7 Process each data record in the block, combine each data record in the original data table according to the new data table structure into a new data record, and insert the temporary data table created in step 2, that is, according to the new table structure After modification, insert it into the temporary data table created in step 2.
  • Step 8 Determine whether the data records in the block have been processed? If yes, the data records in the block have been processed, go to step 9; if not, the data records in the block have not been processed, return to step 7 to continue the processing of the next data record.
  • Step 9 Register in the management data of DDL (such as the Map dynamic array structure) that the block has been scanned, and update the total number of blocks processed block num that has been scanned.
  • DDL such as the Map dynamic array structure
  • step 9 register the current block into the DDL Map structure, and update the number of currently processed blocks (that is, processed block num).
  • Step 10 obtain the next block.
  • Step 11 Determine whether there are still blocks to be processed? If yes, that is, there are still blocks to be processed, return to step 7 to continue processing; if not, that is, there are no blocks to be processed, indicating that the index creation process is over, and go to step 12.
  • Step 12 Modify the table name of the original data table, change the table name of the temporary data table created in step 2 to the table name of the original data table, and delete the original data table asynchronously.
  • Step 13 marking the end of the DDL task.
  • the first block collection denoted as blockIdInit, refers to the collection of all current block IDs collected when scanning the entire table, and is used as the baseline for parallel scanning.
  • the second block collection refers to the collection of all block IDs after the parallel scanning is completed, which is used to compare with the baseline to check whether there are blocks missed due to parallel scanning.
  • Figure 11 is a flow chart of normal parallel execution of online index addition provided by the embodiment of this application. Please refer to Figure 11. This embodiment is executed by any computing device in the database system, and the DDL transaction is used as an example to illustrate online index addition. .
  • Step 1 Start a DDL task to add an index (Add Index).
  • Step 2 Modify the data dictionary corresponding to the data table, add an index to the table structure, and set the current state of the index to the Delete Only state. In the Delete Only state, only the index can be deleted.
  • Step 3 Submit the transaction and save the new table structure to the data dictionary.
  • Step 4 Start the second phase of DDL execution.
  • Step 5 After obtaining the corresponding permission, change the index status to Write Only status, and obtain a transaction ID at the same time.
  • the step of obtaining permissions is optional to improve the security of DDL transactions, but it is not necessary to obtain permissions before modifying the index status, and it is also supported to directly modify the index status without obtaining permissions to improve the execution efficiency of DDL transactions.
  • the embodiments of the application do not specifically limit this.
  • Step 6 Start scanning the entire table data, and use the transaction ID obtained in step 5 as the transaction ID for the transaction of scanning and building secondary index data.
  • Step 7 Collect the block ID set (the first block set) of the table to be scanned, collect the total number of blocks contained in the table, and register it in the DDL task as the total workload value total blocks of the DDL task.
  • step 7 record the block ID set (the first block set) that needs to be scanned in the full table and the total number of blocks that need to be processed for scanning the full table data (that is, the total workload total blocks) into the DDL task.
  • Step 8 Select an appropriate degree of parallelism, initialize parallel tasks according to the set of block IDs, and start the parallel execution process (steps 9-15 are the main process of parallel task execution).
  • Step 9 start scanning, and record the above transaction ID in the DDL task.
  • Step 10 Obtain the first block.
  • Step 11 Process each data record in the block, extract the corresponding data field to create the key of the secondary index, and write it into the index structure using the Thomas writing rule. Since the transaction ID is applied when the index status becomes Write Only, according to the Thomas write rule, the previously uncommitted transactions are rolled back; in addition, the above-mentioned transactions that scan and create indexes are more efficient than any subsequent users modifying the table. The transactions are all old, so if there is a conflict between the two, you only need to keep the subsequent user's modification of the corresponding data in the table.
  • Step 12 Determine whether the data records in the block have been processed? If yes, that is, the data records in the block are processed, go to step 13; if not, that is, the data records in the block have not been processed, return to step 11 to continue the processing of the next data record.
  • Step 13 register in the management data of DDL (such as the Map dynamic array structure) that the block has been scanned, and update the total number processed block num of the scanned blocks.
  • DDL such as the Map dynamic array structure
  • step 13 register the current block in the Map structure of the DDL, and update the currently processed block number (ie processed block num).
  • the data progress of the DDL transaction is equal to processed block num/total blocks.
  • the execution progress of the current DDL transaction is calculated for visual display on the foreground.
  • Step 14 obtain the next block.
  • Step 15 Determine whether there are still blocks to be processed? If yes, that is, there are still blocks to be processed, return to step 11 to continue processing; if not, that is, there are no blocks to be processed, indicating that the index creation process is over, and go to step 16.
  • Step 16 Determine whether to execute in parallel? If yes, it is in parallel processing mode, go to step 17; if not, it is in serial processing mode, go to step 21.
  • Step 17. Determine that all parallel processing tasks end.
  • Step 18 obtain the block ID set (second block set) of the table again.
  • Step 19 Compare the block ID set obtained in step 18 with the block ID set initialized in step 7. Is there any new (unprocessed due to block split) block? If there is a new block, it means that a split occurred during the execution process, and go to step 20; if there is no new block, it means that all blocks have been processed, and go to step 21.
  • Step 20 Determine whether parallel execution is required? If so, that is, the newly added blocks still need to be executed in parallel (for example, many blocks are newly generated, and parallel execution is still selected), return to step 8, select the degree of parallelism for the set of newly added blocks, initialize parallel tasks, and start parallel execution Process; if not, that is, the newly added block does not need to be executed in parallel, return to step 9, and perform serial processing of the newly added block.
  • This serial execution method does not need to judge whether a new block is added in the process after the end of the serial execution method.
  • Step 21 Modify the index status to Public (indicating that the index is created and can be used normally), and mark the DDL task as completed.
  • Figure 12 is a flow chart of parallel recovery after abnormal interruption of online index addition provided by the embodiment of this application, please refer to Figure 12, this embodiment is executed by any computing device in the database system, taking DDL transaction as an example of online index addition illustrate.
  • Step 1 The background thread finds an abnormally interrupted Add Index task.
  • Step 2 Determine whether the execution can continue? If no, it cannot continue to execute, go to step 3; if yes, it can continue to execute, go to step 4.
  • Step 3 Roll back the DDL transaction and go to step 18.
  • Step 4 Find the last block processed through the management data (such as the Map dynamic array structure) recorded in the DDL task, and search for the next block of the last block.
  • management data such as the Map dynamic array structure
  • Step 5 Select an appropriate degree of parallelism for the remaining blocks, initialize parallel tasks, and start parallel execution.
  • Step 6 Determine whether to find the next block? If yes, the next block is found, go to step 7; if no, the next block is not found, go to step 17.
  • Step 7 Process each data record in the block, extract the corresponding data field to create the key of the secondary index, and write it into the index structure using the Thomas writing rule. Since the transaction ID is applied when the index status becomes Write Only, according to the Thomas write rule, the previously uncommitted transactions are rolled back; in addition, the above-mentioned transactions that scan and create indexes are more efficient than any subsequent users modifying the table. The transactions are all old, so if there is a conflict between the two, you only need to keep the subsequent user's modification of the corresponding data in the table.
  • Step 8 Determine whether the data records in the block have been processed? If yes, the data records in the block have been processed, go to step 9; if not, the data records in the block have not been processed, return to step 7 to continue the processing of the next data record.
  • Step 9 Register in the management data of DDL (such as the Map dynamic array structure) that the block has been scanned, and update the total number of blocks processed block num that has been scanned.
  • DDL such as the Map dynamic array structure
  • step 9 register the current block into the DDL Map structure, and update the number of currently processed blocks (that is, processed block num).
  • the data progress of the DDL transaction is equal to processed block num/total blocks.
  • the execution progress of the current DDL transaction is calculated for visual display on the foreground.
  • Step 10 obtain the next block.
  • Step 11 Determine whether there are still blocks to be processed? If yes, that is, there are still blocks to be processed, return to step 7 to continue processing; if not, that is, there are no blocks to be processed, indicating that the index creation process is over, and go to step 12.
  • Step 12 Determine whether to execute in parallel? If yes, it is in parallel processing mode, go to step 13; if not, it is in serial processing mode, go to step 17.
  • Step 13 Determine that all parallel processing tasks end.
  • Step 14 obtain the block ID set (second block set) of the table again.
  • Step 15 Compare the block ID set obtained in step 14 with the block ID set initialized by the DDL transaction. Is there any new (unprocessed due to block split) block? If there is a new block, it means that a split occurred during the execution process, and go to step 16; if there is no new block, it means that all blocks have been processed, and go to step 17.
  • Step 16 Determine whether parallel execution is required? If so, that is, the newly added blocks still need to be executed in parallel (for example, many new blocks are newly generated, and parallel execution is still selected), return to step 5, select the degree of parallelism for the set of newly added blocks, initialize parallel tasks, and start parallel execution Process; if not, that is, the newly added block does not need to be executed in parallel, return to step 6, and perform serial processing of the newly added block.
  • This serial execution method does not need to judge whether a new block is added in the process after the end of the serial execution method.
  • Step 17. Modify the index status to Public (indicating that the index is created and can be used normally).
  • Step 18 marking the end of the DDL task.
  • the parallel execution process provided by the above two embodiments can greatly increase the scanning speed of large data tables. Compared with the increased detection process, the overall performance of the database system can still be guaranteed to be greatly improved.
  • DDL transactions are processed in units of blocks, it is easy to implement parallel scanning and processing of data, thereby greatly improving the speed of DDL execution. Through reasonable parallel processing of data, DDL statements can be truly completed quickly.
  • Fig. 13 is a schematic structural diagram of a transaction execution device provided by an embodiment of the present application. As shown in Fig. 13, the device includes:
  • a determining module 1301, configured to determine the last data block processed before the execution of the interrupted DDL transaction in the case that the interrupted data definition language DDL transaction meets the recovery condition;
  • An execution module 1302, configured to continue executing the DDL transaction starting from the next data block of the last data block;
  • a commit module 1303, configured to commit the DDL transaction when all data blocks operated on the DDL transaction have been processed.
  • the technical solution provided by the embodiment of the present application uses a data block as the smallest unit for processing a DDL transaction.
  • the last data block that has been processed before the execution interruption can be easily located without the need to roll back the DDL transaction as a whole. , but continues to execute the DDL transaction from the next data block, avoiding the redundant workload of redoing the DDL transaction from the beginning, improving the success rate of the DDL transaction indicated by the DDL statement, and improving the resource utilization of the database.
  • the determination module 1301 includes: an acquisition unit, configured to acquire management data of the DDL transaction, the management data being used to record each data block processed by the DDL transaction; a query unit , used to obtain the last data block from the management data.
  • the management data is a dynamic array
  • the data block identification of each data block that has been processed by the DDL transaction is stored in the dynamic array
  • the query unit is used to: store the last element in the dynamic array
  • the data block indicated by the data block identifier is determined to be the last data block.
  • the data block identifier of the data block is recorded in the management data.
  • the device further includes: a first acquisition module, configured to respond to the progress query instruction of the DDL transaction, based on the management data and the data blocks that the DDL transaction needs to process The total number is to obtain the data progress of the DDL transaction; the sending module is used to return the data progress of the DDL transaction to the device that triggers the progress query instruction.
  • a first acquisition module configured to respond to the progress query instruction of the DDL transaction, based on the management data and the data blocks that the DDL transaction needs to process The total number is to obtain the data progress of the DDL transaction; the sending module is used to return the data progress of the DDL transaction to the device that triggers the progress query instruction.
  • the first obtaining module is configured to: divide the number of data blocks recorded in the management data by the total number of data blocks to determine the data progress of the DDL transaction.
  • the execution module 1302 is configured to: read the next data block or any data block after the next data block, and process each data record stored in the data block.
  • the execution module 1302 is configured to: based on the parallelism degree of the DDL transaction, start from the next data block of the last data block to perform parallel processing on each data block that has not been processed by the DDL transaction.
  • the apparatus further includes: a second acquisition module, configured to acquire the first data block set and the second data block set of the DDL transaction after the parallel processing is completed,
  • the first data block set is used to record each data block in the data table that the DDL transaction needs to operate when it starts to execute, and the second data block set is used to record the data that the DDL transaction needs to operate when the parallel processing is completed each data block in the table;
  • the submission module 1303 is also used for submitting the DDL transaction when the first data block set is consistent with the second data block set;
  • the execution module 1302 is also used for the second data block set When a set of data blocks is inconsistent with the second set of data blocks, it is determined that the second set of data blocks is a newly added data block compared to the first set of data blocks;
  • the submission module 1303 is also used to After the newly added data block is processed, submit the DDL transaction.
  • the recovery condition when the DDL transaction is an online DDL transaction, the recovery condition includes: the data dictionary of the data table of the DDL transaction operation contains the definition of the object of the DDL transaction operation, and the DDL transaction operation The data block or at least one copy of the data block is readable.
  • the recovery condition when the DDL transaction is an offline DDL transaction, includes: both the original data table operated by the DDL transaction and the temporary data table created by the DDL transaction exist.
  • the transaction execution device provided by the above embodiment executes DDL transactions, it only uses the division of the above-mentioned functional modules as an example. In practical applications, the above-mentioned function allocation can be completed by different functional modules according to needs. That is, the internal structure of the computing device is divided into different functional modules to complete all or part of the functions described above.
  • the transaction execution device provided by the above embodiment is of the same concept as the transaction execution method embodiment, and its specific implementation process is detailed in the transaction execution method embodiment, which will not be repeated here.
  • FIG. 14 is a schematic structural diagram of a terminal provided by an embodiment of the present application, and a terminal 1400 is an exemplary illustration of a computing device.
  • the device type of the terminal 1400 includes: a smart phone, a tablet computer, a notebook computer or a desktop computer.
  • the terminal 1400 may also be called user equipment, portable terminal, laptop terminal, desktop terminal and other names.
  • the terminal 1400 includes: a processor 1401 and a memory 1402 .
  • the processor 1401 includes one or more processing cores, such as a 4-core processor, an 8-core processor, and the like.
  • the processor 1401 adopts at least one of DSP (Digital Signal Processing, digital signal processing), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array, programmable logic array) implemented in the form of hardware.
  • the processor 1401 includes a main processor and a coprocessor, and the main processor is a processor for processing data in a wake-up state, also called a CPU (Central Processing Unit, central processing unit);
  • a coprocessor is a low-power processor for processing data in a standby state.
  • the processor 1401 is integrated with a GPU (Graphics Processing Unit, image processor), and the GPU is used for rendering and drawing the content that needs to be displayed on the display screen.
  • memory 1402 includes one or more computer-readable storage media, which are optionally non-transitory.
  • the memory 1402 also includes a high-speed random access memory, and a non-volatile memory, such as one or more magnetic disk storage devices and flash memory storage devices.
  • the non-transitory computer-readable storage medium in the memory 1402 is used to store at least one program code, and the at least one program code is used to be executed by the processor 1401 to implement the various embodiments provided in this application. Transaction execution method.
  • the terminal 1400 may optionally further include: a peripheral device interface 1403 and at least one peripheral device.
  • the processor 1401, the memory 1402, and the peripheral device interface 1403 can be connected through buses or signal lines.
  • Each peripheral device can be connected to the peripheral device interface 1403 through a bus, a signal line or a circuit board.
  • the peripheral equipment includes: a display screen 1405 .
  • the peripheral device interface 1403 may be used to connect at least one peripheral device related to I/O (Input/Output, input/output) to the processor 1401 and the memory 1402 .
  • the processor 1401, memory 1402 and peripheral device interface 1403 are integrated on the same chip or circuit board; in some other embodiments, any one of the processor 1401, memory 1402 and peripheral device interface 1403 or Both are implemented on separate chips or boards.
  • the display screen 1405 is used to display a UI (User Interface, user interface).
  • the UI includes graphics, text, icons, videos and any combination thereof.
  • the display screen 1405 also has the ability to collect touch signals on or above the surface of the display screen 1405 .
  • the touch signal can be input to the processor 1401 as a control signal for processing.
  • the display screen 1405 is also used to provide virtual buttons and/or virtual keyboards, also called soft buttons and/or soft keyboards.
  • the display screen 1405 is made of LCD (Liquid Crystal Display, liquid crystal display), OLED (Organic Light-Emitting Diode, organic light-emitting diode) and other materials.
  • FIG. 14 does not constitute a limitation on the terminal 1400, and may include more or less components than shown in the figure, or combine some components, or adopt a different component arrangement.
  • the computing device 1500 may have relatively large differences due to different configurations or performances.
  • the computing device 1500 includes one or more than one processor (Central Processing Units, CPU) 1501 and one or more memories 1502, wherein at least one computer program is stored in the memory 1502, and the at least one computer program is loaded and executed by the one or more processors 1501 to implement the above-mentioned various embodiments.
  • Transaction execution method is included in the computing device 1500
  • the computing device 1500 also has components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output.
  • the computing device 1500 also includes other components for implementing device functions, which will not be repeated here.
  • a computer-readable storage medium such as a memory including at least one computer program
  • the at least one computer program can be executed by a processor in the terminal to complete the transaction execution method in each of the above embodiments .
  • the computer-readable storage medium includes ROM (Read-Only Memory, read-only memory), RAM (Random-Access Memory, random-access memory), CD-ROM (Compact Disc Read-Only Memory, read-only disc), Magnetic tapes, floppy disks, and optical data storage devices, etc.
  • a computer program product or computer program comprising one or more pieces of program code stored in a computer readable storage medium.
  • One or more processors of the computing device can read the one or more program codes from the computer-readable storage medium, and the one or more processors execute the one or more program codes, so that the computing device can execute to complete The transaction execution method in the foregoing embodiments.
  • the steps for implementing the above embodiments can be completed by hardware, and can also be completed by instructing related hardware through a program.
  • the program is stored in a computer-readable storage medium.
  • the storage medium mentioned above is a read-only memory, a magnetic disk or an optical disk, and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种事务执行方法、装置、计算设备及存储介质，涉及数据库技术领域。方法包括：在执行中断的DDL事务符合恢复条件的情况下，确定(201)所述DDL事务在执行中断前处理完毕的最后一个数据块；从所述最后一个数据块的下一个数据块开始，继续执行(202)所述DDL事务；在对所述DDL事务操作的各个数据块均处理完毕的情况下，提交(203)所述DDL事务。

Description

事务执行方法、装置、计算设备及存储介质
本申请要求于2021年10月19日提交的申请号为202111214946.4、发明名称为“事务执行方法、装置、计算设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及数据库技术领域，特别涉及一种事务执行方法、装置、计算设备及存储介质。
背景技术
随着数据库技术的发展，SQL(Structured Query Language，结构化查询语言)是一种数据库查询和程序设计语言，用于存取数据以及查询、更新和管理关系数据库系统。在SQL命令中涉及一类DDL(Data Definition Language，数据定义语言)语句，DDL语句是用于修改数据库中的对象(如表、索引、列、触发器等)的定义的语句。在对数据库中的某个对象执行DDL语句时，通常需要对该对象加锁，阻塞在该DDL语句执行期间其他业务事务对该对象中数据的修改，以保证DDL语句所操作的对象中所存储数据的一致性。
发明内容
本申请实施例提供了一种事务执行方法、装置、计算设备及存储介质。
一方面，提供了一种事务执行方法，该方法包括：响应于执行中断的数据定义语言DDL事务符合恢复条件，确定所述DDL事务在执行中断前处理完毕的最后一个数据块；从所述最后一个数据块的下一个数据块开始，继续执行所述DDL事务；响应于对所述DDL事务操作的各个数据块均处理完毕，提交所述DDL事务。
一方面，提供了一种事务执行装置，该装置包括：确定模块，用于响应于执行中断的数据定义语言DDL事务符合恢复条件，确定所述DDL事务在执行中断前处理完毕的最后一个数据块；执行模块，用于从所述最后一个数据块的下一个数据块开始，继续执行所述DDL事务；提交模块，用于响应于对所述DDL事务操作的各个数据块均处理完毕，提交所述DDL事务。
一方面，提供了一种计算设备，该计算设备包括一个或多个处理器和一个或多个存储器，该一个或多个存储器中存储有至少一条计算机程序，该至少一条计算机程序由该一个或多个处理器加载并执行以实现如上述事务执行方法。
一方面，提供了一种存储介质，该存储介质中存储有至少一条计算机程序，该至少一条计算机程序由处理器加载并执行以实现如上述事务执行方法。
一方面，提供一种计算机程序产品或计算机程序，所述计算机程序产品或所述计算机程序包括一条或多条程序代码，所述一条或多条程序代码存储在计算机可读存储介质中。计算设备的一个或多个处理器能够从计算机可读存储介质中读取所述一条或多条程序代码，所述一个或多个处理器执行所述一条或多条程序代码，使得计算设备能够执行上述事务执行方法。
附图说明
图1是本申请实施例提供的一种事务执行方法的实施环境示意图；
图2是本申请实施例提供的一种事务执行方法的流程图；
图3是本申请实施例提供的一种事务执行方法的流程图；
图4是本申请实施例提供的一种事务执行方法的流程图；
图5是本申请实施例提供的一种在线增加索引的正常执行流程图；
图6是本申请实施例提供的一种在线增加索引的异常中断后恢复流程图；
图7是本申请实施例提供的一种block分裂的原理性示意图；
图8是本申请实施例提供的一种block分裂的原理性示意图；
图9是本申请实施例提供的一种离线增加索引的正常执行流程图；
图10是本申请实施例提供的一种离线增加索引的异常中断后恢复流程图；
图11是本申请实施例提供的一种在线增加索引的正常并行执行流程图；
图12本申请实施例提供的一种在线增加索引的异常中断后并行恢复流程图；
图13是本申请实施例提供的一种事务执行装置的结构示意图；
图14是本申请实施例提供的一种终端的结构示意图；
图15是本申请实施例提供的一种计算设备的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
本申请中术语“第一”“第二”等字样用于对作用和功能基本相同的相同项或相似项进行区分，应理解，“第一”、“第二”、“第n”之间不具有逻辑或时序上的依赖关系，也不对数量和执行顺序进行限定。
本申请中术语“至少一个”是指一个或多个，“多个”的含义是指两个或两个以上，例如，多个第一位置是指两个或两个以上的第一位置。
在介绍本申请实施例之前，需要引入一些云技术领域内的基本概念。
云技术(Cloud Technology)：是指在广域网或局域网内将硬件、软件、网络等系列资源统一起来，实现数据的计算、储存、处理和共享的一种托管技术，也即是基于云计算商业模式应用的网络技术、信息技术、整合技术、管理平台技术、应用技术等的总称，通过组成资源池，按需所用，灵活便利。云计算技术将变成云技术领域的重要支撑。技术网络系统的后台服务需要大量的计算、存储资源，如视频网站、图片类网站或更多的门户网站。伴随着互联网行业的高度发展和应用，将来每个物品都有可能存在自己的识别标志，都需要传输到后台系统进行逻辑处理，不同程度级别的数据将会分开处理，各类行业数据皆需要强大的系统后盾支撑，均能通过云计算来实现。
云存储(Cloud Storage)：是在云计算概念上延伸和发展出来的一个新的概念，分布式云存储系统(以下简称存储系统)是指通过集群应用、网格技术以及分布存储文件系统等功能，将网络中大量各种不同类型的存储设备(存储设备也称之为存储节点)通过应用软件或应用接口集合起来协同工作，共同对外提供数据存储和业务访问功能的一个存储系统。
数据库(Database)：简而言之能够视为一种电子化的文件柜——存储电子文件的处所，支持用户对文件中的数据进行新增、查询、更新、删除等操作。所谓“数据库”是以一定方式储存在一起、能与多个用户共享、具有尽可能小的冗余度、与应用程序彼此独立的数据集合。
以下，对本申请实施例所涉及的术语进行解释说明。
DDL(Data Definition Language，数据定义语言)语句：即DDL语句操作，指用来修改数据库中的对象(如：表、索引、列、触发器等)的定义的语句。
Online DDL(在线DDL)：在线DDL是相对于离线DDL来说的，通常对数据库中的某个对象执行一个DDL语句时，都会对该对象加上锁，再执行变更操作。上述加锁是用来阻塞在该DDL语句执行期间其他业务事务(如DML事务)对该对象中数据的修改，以保证DDL语句所操作的对象中所存储数据的一致性，其中DML是指Data Manipulate Language(数据操纵语言)。换言之，在DDL语句执行期间，用户发起的业务操作(业务事务)会因为该对象被锁定而阻塞。随着用户对于业务的高可用要求越来越高，各家数据库厂商都提出在线DDL的实现，即在对某个对象做DDL变更的时候，实现对于用户业务不阻塞或者只阻塞一个很小的时间段。
Thomas Write(托马斯写)：在计算机科学领域，特别是数据库领域，托马斯写是一种基于时间戳的并发控制机制，被总结为忽略掉已经过时的写。例如：事务1在T1时刻开始修改数据A，事务2在T2时刻开始也修改数据A，T1<T2，因为某种原因事务1和事务2同时向数据库提出提交申请，在乐观事务提交机制下，数据库会选择先达到的提交申请的事务进行提交，其他冲突事务回滚，但是在托马斯写规则下，则数据库需要确保事务2的提交一定会成功，因为事务1的时间戳在托马斯写规则下被认为是已经过时的写，发生冲突的情况下需要被回滚掉。需要说明的是，悲观事务模型下会因为要修改的某个数据已经被锁定，导致其他事务等待，因此不会出现同时有两个事务同时修改同一数据，并同时提交的情况。
Tuple(数据记录)：通常是指关系型数据库中的数据表中的某一行数据记录，这个数据记录存储了表定义中所有列的实例化信息，并按照列定义的顺序排列，组成一个连续的内容，也即是说，这段连续的内容被称为数据表的一条数据记录，即Tuple。
数据字典：数据库中用来存放数据库中的对象的定义的地方。其中，数据库中的对象包括：数据库、表空间、数据表、数据列、数据索引、分区信息、用户、函数、角色等。
并行度：并行处理中将会启动多少并行处理任务来将原来需要处理的任务进行分解；通常在计算机处理中，就是将原来需要处理的全量数据，划分为若干子任务，启动并行度指定数量的线程，取得子任务，并按照相应的处理逻辑进行处理，理想情况下并行执行能够达到并行度倍数的性能提升。
以下，对传统数据库中DDL语句的离线执行模式进行说明。
对于传统单机数据库系统，通过对元数据加锁的方式来阻塞有冲突的操作的执行，保证Schema(数据库对象的集合)变更的正确执行，即保证DDL语句的正确执行。
对于传统集群数据库系统，集群数据库系统作为单机数据库系统的一个扩展，集群内包括多个计算节点(即计算设备)，每个计算节点的数据库中存储有多个数据表，每个数据表中存储有一条或多条数据记录(即数据行)，每一条数据记录由一组按照相同位置索引排列的字段集组成，即数据字段列，计算节点的数据库为任一类型的集群数据库，包括关系型数据库或者非关系型数据库中至少一项，例如SQL数据库、MySQL、NoSQL、NewSQL(泛指各种新式的可拓展/高性能数据库)等，在本申请实施例中对数据库的类型不作具体限定。
针对上述集群数据库系统，DDL语句的执行模式划分为Share Memory(共享内存)模式和Share Nothing(零共享)模式：
A)Share Memory(共享内存)模式，通过在集群内的一个中央协调节点来控制各个计算节点对于某一个Schema对象的访问控制逻辑；
B)Share Nothing(零共享)模式，通常所有计算节点都有一个控制节点来确定哪些计算节点需要参与DDL语句的执行，然后DDL语句再分发到相应的计算节点执行，在每个计算节点中视为在一个单机数据库系统上执行，因此与单机数据库基本同理。
对于计算存储分离的云原生数据库架构，目前都只实现了单写，与MySQL单实例一致，在实现多写集群方案中，需要在多个写节点同步DDL锁以保证数据在DDL变更前后以及变更过程中的数据一致性。
综上所述，不管是传统单机数据库、传统集群分布式数据库还是计算存储分离的数据库架构，如果在DDL语句执行期间，由于某种故障导致DDL语句的执行线程崩溃，在DDL语句异常中断后，均无法提供一种在中断处继续执行的能力，这是因为要么DDL语句会被回滚掉，要么通过重做日志(Redo Log)回滚到DDL语句开始时的状态后，从头开始对该DDL语句进行重做，针对DDL语句操作的对象所存储的数据量较大(例如存储了海量数据的表)的情况，数据库系统重做该DDL语句的代价较高，换言之，DDL事务的成功率低，数据库资源利用率低。
有鉴于此，本申请实施例提供一种事务执行方法，应用于传统数据库、分布式数据库等各类数据库系统，针对DDL事务提供在异常中断点继续恢复执行的方案，在DDL事务执行发生异常中断后，后台线程或进程能够在上次中断位置继续完成DDL的实现，从而对于操作 比较大的对象(例如存储了海量数据的表)的DDL事务，能够提高DDL事务的成功率，并且由于能够保留异常中断之前已经完成的部分处理结果，使得整体的数据库资源利用率得到了提升。
在一些实施例中，本申请实施例还应用于一种基于区块链技术的数据库系统(以下简称为“区块链系统”)，区块链系统在本质上属于一种去中心化式的分布式数据库系统，采用共识算法保持区块链上不同计算设备所记载的账本数据一致，通过密码算法保证不同计算设备之间账本数据的加密传送以及不可篡改，通过脚本系统来拓展账本功能，通过网络路由来进行不同计算设备之间的相互连接。
在区块链系统中包括一条或多条区块链，区块链是一串使用密码学方法相关联产生的数据块，每一个数据块中包含了一批次网络交易的信息，用于验证其信息的有效性(防伪)和生成下一个区块。
区块链系统中计算设备之间组成点对点(Peer To Peer，P2P)网络，P2P协议是一个运行在传输控制协议(Transmission Control Protocol，TCP)协议之上的应用层协议。在区块链系统中，计算设备具备如下功能：1)路由，计算设备具有的基本功能，用于支持计算设备之间的通信；2)应用，用于部署在区块链中，根据实际业务需求而实现特定业务，记录实现功能相关的数据形成账本数据，在账本数据中携带数字签名以表示数据来源，将账本数据发送至区块链系统中的其他计算设备，供其他计算设备在验证账本数据来源以及完整性成功时，将账本数据添加至临时区块中，其中，应用实现的业务包括钱包、共享账本、智能合约等；3)区块链，包括一系列按照先后的时间顺序相互接续的区块，新区块一旦加入到区块链中就不会再被移除，区块中记录了区块链系统中计算设备提交的账本数据。
在一些实施例中，每个区块中包括本区块存储交易记录的哈希值(本区块的哈希值)以及前一区块的哈希值，各区块通过哈希值连接形成区块链，另，区块中还包括有区块生成时的时间戳等信息。
以下，对本申请实施例的系统架构进行说明。
图1是本申请实施例提供的一种事务执行方法的实施环境示意图。参见图1，本申请实施例适用于单机数据库、集群数据库、计算存储分离的云原生数据库架构等各类数据库系统，下面以分布式集群数据库为例进行说明。分布式集群数据库系统中包括应用客户端101、网关服务器102以及分布式存储集群103，在分布式存储集群103中包括一个或多个计算设备(即计算节点)。
应用客户端101是指用户侧的终端上安装和运行的能够发起数据请求的客户端，该数据请求是DDL请求或者DML请求等，本申请实施例对此不进行具体限定。可选地，应用客户端101的类型包括：支付应用、社交应用、音视频应用、直播应用、购物应用、外卖应用或者打车应用等，本申请实施例不对应用客户端101的类型进行具体限定。在一些实施例中，用户侧的终端也称为用户设备、终端设备、用户终端、移动终端、智能终端、通信设备等。终端的设备类型包括：智能手机、平板电脑、笔记本电脑、台式计算机、智能音箱、智能手表、车载终端、智能家电、智能语音交互设备等，但并不局限于此。
应用客户端101以及网关服务器102通过有线或无线通信方式进行直接或间接地连接，本申请在此不做限制。
网关服务器102用于接收外部的数据请求，并将数据请求对应的读写事务分发至分布式存储集群103，示意性地，用户在终端上登录应用客户端101，触发应用客户端101生成DDL请求，例如，DDL请求为修改数据表A的表名，应用客户端101调用分布式集群数据库系统提供的API(Application Programming Interface，应用程序编程接口)，将该DDL请求发送至网关服务器102，比如，该API是MySQL API(一种关系型数据库系统提供的API)。又例如，在智慧交通场景下，新增停车位的描述信息请求为DDL请求，而查询已有停车位的请求为DML请求。
在一些实施例中，该网关服务器102与分布式存储集群103中的任一个计算设备合并在 同一个物理机上，也即是，让某个计算设备充当网关服务器102。
分布式存储集群103包括一个或多个计算设备，本申请实施例不对分布式存储集群103中计算设备的数量进行具体限定，例如，计算设备的数量为m个，m为大于或等于1的整数。可选地，每个计算设备采用主备结构(一主多备集群)，如图1所示，以计算设备为一主两备集群为例进行示意，每个计算设备中包括一个主机和两个备机，可选地，每个主机或备机都配置有代理(Agent)设备，代理设备与主机或备机是物理独立的，或者，代理设备作为主机或备机上的一个代理模块，以计算设备1为例，计算设备1包括一个主数据库及代理设备(主database+agent，简称主DB+agent)，此外还包括两备数据库及代理设备(备database+agent，简称备DB+agent)。
在一个示例性场景中，每个计算设备的主机或备机的数据库实例集合称为一个SET(集合)，例如，假设某一计算设备为单机设备，那么该计算设备的SET仅为该单机设备的数据库实例，假设某一计算设备为一主两备集群，那么该计算设备的SET为主机数据库实例以及两个备机数据库实例的集合。
在一些实施例中，上述网关服务器102以及分布式存储集群103所构成的分布式集群数据库系统，视为一种向用户终端提供数据服务的服务器，该服务器是独立的物理服务器，或者是多个物理服务器构成的服务器集群或者分布式系统，或者是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、CDN(Content Delivery Network，内容分发网络)以及大数据和人工智能平台等基础云计算服务的云服务器。
以下，对本申请实施例涉及的数据结构进行说明。
为了做到能够让DDL事务实现从中断处恢复执行，需要将DDL事务所修改数据表中的数据划分为多个小的逻辑或者物理单元，并以这些最小单元作为这一数据表中这一数据行中的断点位置。通常情况下，数据库的对象会以底层文件系统或存储方式来组织对象中的数据，将对象中的数据映射到文件系统或者存储系统大致会涉及如下情况：
a)heap(堆)表数据管理模式
heap表数据管理模式下，数据库以heap表方式来管理数据表中的数据记录，即以一定大小的page(页)或者block(块)来组织和存放数据表中的数据记录。换言之，数据记录都是存储在一个一个固定大小的数据块中，通常称为page或者block，例如，数据块是2 n k大小的page，同时也以2 n k大小的page存储实际数据。
b)B+树数据管理模式
选取一个固定大小的block来代表B+树的非叶子结点和叶子结点，实际数据会存放在叶子结点当中，是和上述a)的heap表数据管理模式同理的存储数据方式。
c)LSM(Log-Structured Merge Tree，结构化合并树)数据管理模式
LSM数据管理模式即Key-Value(键-值)数据管理模式，为了提升读写效率，通常会以block模式来组织一定数量的Key-Value对(键值对)，在一些数据库中，为了做到存储能够水平扩展，引入了region(域)的定义，region是一种一个或多个物理block的逻辑映射，通过增加逻辑映射能够屏蔽掉region关联的数据真实存储的位置。换言之，LSM存储方式下一定大小的数据会组成block或者region的概念，region或者block是有一定数据范围的。因此，LSM数据管理模式也被认为是以block方式存储数据的。
通过上面的分析，对于各种数据库中关于表数据存储的实现，均能够在存储数据这个粒度上找到一个block来作为本申请实施例的DDL事务在进行数据处理时的最小单元。
在上述以block作为DDL事务在进行数据处理时的最小单元的基础上，本申请实施例将对事务执行方法的异常中断恢复执行流程进行说明。本申请实施例适用于单机数据库系统、集群数据库系统、分布式数据库系统、计算存储分离的云原生数据库架构、区块链系统等，在此不做具体限定。
图2是本申请实施例提供的一种事务执行方法的流程图。参见图2，该实施例由数据库系统中的任一计算设备执行，该实施例包括下述步骤：
201、计算设备响应于执行中断的数据定义语言DDL事务符合恢复条件，确定该DDL事务在执行中断前处理完毕的最后一个数据块。
在一些实施例中，用户在终端上登录应用客户端，触发应用客户端生成DDL请求，例如，DDL请求为向数据表A增加一个索引(Index)，或者，DDL请求为修改数据表A的表名等，本申请实施例不对DDL请求的类型进行具体限定。应用客户端在生成DDL请求之后，调用API将该DDL请求发送至计算设备。
在一些实施例中，计算设备接收应用客户端的任一数据请求，解析该数据请求的头字段，当该头字段指示该数据请求为DDL请求时，为该DDL请求创建一个新的DDL进程或线程，或者复用已创建的某一DDL进程或线程，并通过该DDL进程或线程执行该DDL请求关联的DDL事务。
在一些实施例中，计算设备串行执行该DDL事务，或者，数据库系统内的一个或多个计算设备并行执行该DDL事务，例如，单个计算设备上的多个DDL线程并行执行该DDL事务，或者，多个计算设备上各自的多个DDL线程并行执行该DDL事务，本申请实施例不对此进行具体限定。
在一些实施例中，在DDL事务的执行过程中，由于某种原因(例如，进程或线程崩溃、闪退等)导致DDL事务执行中断，在数据库系统重启后，如果后台的DDL执行检测器检测到某一个执行中断的DDL事务，在该DDL事务符合恢复条件的情况下，由于以数据块作为事务处理的最小单元，因此确定出该DDL事务在执行中断前处理完毕的最后一个数据块，并执行下述步骤202。其中，该DDL执行检测器用于周期性地检测数据库当前是否有DDL事务发生了执行异常情况。
在一些实施例中，每个DDL事务划分为两阶段执行，在第一阶段中，向DDL事务操作的数据表的数据字典中修改该DDL事务操作的对象的定义，例如，如果DDL事务是增加一个索引，那么第一阶段下需要修改数据表的数据字典以增加索引的定义，以完成索引结构在表对象中的增加，并将索引置为Delete Only(仅允许删除)状态，此时表的结构中增加了索引的定义，但该索引还不能被优化器选中，也不能在查询中使用新增的该索引来查询数据；在第二阶段中，需要扫描DDL事务操作的全表数据，以处理该DDL事务操作的对象涉及的每一条数据记录，例如，如果DDL事务是增加一个索引，那么第二阶段下需要同步或者异步执行填充索引数据的任务，即扫描全表数据，为每条数据记录(Tuple)创建该索引的Key值(键值)，此时需要将索引置为Write Only(仅允许写入)状态，在Write Only状态之前开始的修改表数据的事务，如果该事务没有提交，那么在Write Only状态设置成功之后，在提交时数据库引擎会拒绝该事务提交以使得该事务被回滚掉。
在一些实施例中，在该第二阶段中，将索引设置成Write Only状态的同时，还获取或申请一个事务标识(Identification，ID)，以该事务标识来指示从设置Write Only状态之后最早开始的事务(即DDL请求所关联的DDL事务)，因此，将第一阶段视为DDL事务的初始化阶段，将第二阶段视为DDL事务的执行阶段，其中，事务标识随着时间戳单调递增。可选地，在单机数据库中基于预先设定的规则来分配事务标识，在集群数据库中向一个用于生成全局事务标识的设备(如协调节点)来申请事务标识，本申请实施例不对事务标识的获取方式进行具体限定。
在一些实施例中，在该第二阶段中，基于该DDL事务开始扫描全表数据，例如，如果DDL事务是增加一个索引，采用托马斯写规则来填充每条数据记录的索引Key值。另外，在将索引设置成Write Only状态之后，所有对于该数据表的修改事务，同时也要更新相应的索引，但由于Write Only状态的索引还没有完成数据填充，因此在第二阶段下新增的索引也不能被优化器使用来查询数据。根据托马斯写规则，针对在扫描全表、填充索引的DDL事务之后开始并修改了索引的用户操作(如DML事务)，由于DML事务的时间戳都比进行扫描的DDL事务的时间戳新，导致DDL事务在填充索引Key值的时候，如果发现有冲突(即索引中已经包含某一Key值，说明已经有用户对Key值字段做了修改并完成了索引Key值的同 步)，DDL填充事务直接丢弃当前Key值，继续后续记录扫描即可。
本申请实施例适用于在线DDL场景，也适用于离线DDL场景，在不同场景下DDL事务的恢复条件是不尽相同的，下面进行说明。
针对在线DDL场景，若该DDL事务为在线DDL事务，有两种情况下DDL事务的执行中断没有办法继续执行，即满足下述情况(Ⅰ)或(Ⅱ)时，代表不符合该恢复条件，数据库引擎回滚该DDL事务：
情况(Ⅰ)、DDL事务尚未完成第一阶段，由于第一阶段是指在DDL事务操作的数据表的数据字典中修改该DDL事务操作的对象的定义，尚未完成第一阶段就代表：在DDL事务操作的数据表的数据字典中不包含该DDL事务操作的对象的定义。
例如，DDL事务增加一个索引，第一阶段需要修改数据表的数据字典以增加索引的定义，并将索引置为Delete Only(仅允许删除)状态，如果该DDL事务尚未完成第一阶段，则该数据表的数据字典中没有关于该索引的定义，比如在该数据表的数据字典中查询不到索引的信息，那么无法从异常中断点继续执行DDL事务，该DDL事务会被回滚掉。
情况(Ⅱ)、DDL事务出现硬件问题，导致最后一个block的下一个block以及该下一个block的所有副本都不可用。
换言之，DDL事务在对该最后一个block处理完毕之后，由于某种硬件故障，导致下一个block以及下一个block的所有副本都读不出来，这种原因所引发的异常中断即使在系统重启后也仍然无法恢复。需要说明的是，在保证了block数据的一致性能力的情况下，通常在数据库中每一个block都会存储有多个副本(一主两备、一主多备等)，来确保即使出现硬件磁盘故障，数据也不会丢失，因此情况(Ⅱ)只会在极端情况下发生，即所有机器上的所有副本都不可用，但这种极端情况的发生概率极低，几乎为0。
上述(Ⅰ)和(Ⅱ)介绍了在线DDL场景下不满足恢复条件的两种情况，那么不属于上述(Ⅰ)和(Ⅱ)的其余情况均满足恢复条件，也即是说，在线DDL场景下的恢复条件包括：该DDL事务操作的数据表的数据字典中包含该DDL事务操作的对象的定义，且该DDL事务操作的数据块或该数据块的至少一个副本可读。
针对离线DDL场景，若该DDL事务为离线DDL事务，离线DDL事务的执行流程相对来说比较简单，通常都是在DDL事务执行后创建一个临时数据表，将原数据表中的数据记录复制到临时数据表中，并对临时数据表中的对象完成相应的处理之后，修改原数据表的表名，将临时数据表的表名修改为原数据表的表名，再异步删除原数据表即可。因此，在离线DDL场景下的恢复条件包括：该DDL事务操作的原数据表和该DDL事务创建的临时数据表均存在，此时说明对原数据表中的数据记录尚未扫描完毕，需要从执行中断点恢复扫描。
需要说明的是，在离线DDL场景下，如果不满足恢复条件，说明原数据表或者临时数据表中至少一项不存在，此时视情况判断是回滚DDL事务还是继续推进DDL事务。如果原数据表存在，但临时数据表还没有创建，在满足这种情况时回滚DDL事务，否则，可分为如下三种情况(a)至(c)进行讨论，在满足下述情况(a)至(c)中任一项时，代表执行中断前全表已经扫描完毕，无需回滚DDL事务，只需要继续推进DDL事务即可：
(a)原数据表改名成功，临时数据表存在，说明对原数据表扫描完毕，继续推进DDL事务即可，换言之，继续将临时数据表的表名修改为原数据表的表名，再异步删除原数据表；
(b)临时数据表改名为原数据表，原数据表改名后形成的另一临时数据表也存在，说明对原数据表扫描完毕，继续推进DDL事务即可，换言之，继续从数据字典中删除该另一临时数据表，并异步完成删除该另一临时数据表中的数据；
(c)临时数据表改名为原数据表，原数据表改名后形成的另一临时数据表不存在，继续推进DDL事务即可，换言之，继续确认异步完成删除该另一临时数据表中的数据。
在上述过程中，分别介绍了在线DDL场景和离线DDL场景下，对DDL事务是否符合恢复条件的判定过程，在确认执行中断的DDL事务符合所属场景下的恢复条件时，计算设备确定该DDL事务在执行中断前处理完毕的最后一个block。在本申请实施例之前已经介绍过， 不管是针对heap表数据管理模式、B+树数据管理模式、LSM数据管理模式，都能够以block作为处理DDL事务的最小单元，因此，针对每个DDL事务增加一个管理数据，该管理数据用于记录该DDL事务处理完毕的各个block，每当DDL事务对一个block中的每条数据记录对完成相应的处理，则将处理完毕的block记录到该管理数据中。可选地，为节约存储空间仅在管理数据中记录各个block的block ID，该管理数据亦称为block管理结构。
在一些实施例中，计算设备获取该DDL事务的管理数据，从该管理数据中，查询得到该DDL事务在执行中断前处理完毕的最后一个block。由于以block作为处理DDL事务的最小单元，且对每个DDL事务都维护了管理数据，在管理数据中会记录已处理完毕的各个block，使得DDL事务在执行中断后，能够方便地从管理数据中定位到异常中断点，并从异常中断点开始继续执行DDL事务，而无需从头开始进行重做或者整体回滚，大大提高了DDL事务的成功率。
202、计算设备从该最后一个数据块的下一个数据块开始，继续执行该DDL事务。
在一些实施例中，由于各个block都是按序存放在数据表中的，因此在定位到执行中断前处理完毕的最后一个block之后，说明该最后一个block之前的所有block都是处理完毕的，无需重复处理，只需要从数据表中定位到该最后一个block的下一个block，并从该下一个block开始继续执行DDL事务即可，例如，DDL事务是增加一个索引，在第二阶段中扫描到数据表的block10时异常中断了，此时block10尚未完成处理因此管理数据中不会记录block10，而是记录了已处理完毕的block1～9，确定出异常中断前处理完毕的最后一个block是block9，因此，计算设备从block9的下一个block10开始，继续执行该DDL事务，这样能够避免将DDL事务整体回滚，并且也无需从block1开始进行重做，最多会产生一个block的冗余工作量(即处理到block10的最后一条数据记录时发生中断，平均增加半个block的冗余工作量)，大大提高了数据库的资源利用率。
203、计算设备响应于对该DDL事务操作的各个数据块均处理完毕，提交该DDL事务。
在一些实施例中，计算设备以block为单位来处理该DDL事务，即每次读取一个block，然后对读取的该block中的每条数据记录进行处理，上述处理是依据DDL事务的类型不同而定的，例如，DDL事务是增加一个索引，那么对每条数据记录都需要填充索引的Key值。在对本block的所有数据记录处理完毕后，读取下一个block，并对下一个block中的每条数据记录执行相同的处理，以此类推，迭代执行上述操作，直到不存在需要处理的block，即代表对所有block均处理完毕，此时提交该DDL事务，例如，DDL事务是增加一个索引，在提交DDL事务时，将索引置为Public(发布)状态，并将DDL事务进入Commit(提交)阶段，标记DDL任务结束。
上述所有可选技术方案，能够采用任意结合形成本公开的可选实施例，在此不再一一赘述。
本申请实施例提供的技术方案，以数据块作为处理DDL事务的最小单元，在DDL事务执行中断时，能够方便地定位到执行中断前处理完毕的最后一个数据块，无需整体回滚该DDL事务，而是从下一个数据块开始继续执行该DDL事务，避免了从头开始重做DDL事务的冗余工作量，提高了DDL语句所指示的DDL事务的成功率，提高了数据库的资源利用率。
在上述实施例中，简略介绍了DDL事务在执行中断后，如何定位异常中断点并继续执行DDL事务，由于本申请实施例既支持对DDL事务进行串行处理，也支持对DDL事务进行并行处理，因此，在本申请实施例中，将详细介绍DDL事务的串行处理流程，并在下一实施例中详细介绍DDL事务的并行处理流程。
图3是本申请实施例提供的一种事务执行方法的流程图。参见图3，该实施例由数据库系统中的任一计算设备执行，该数据库系统包括：单机数据库系统、集群数据库系统、分布式数据库系统、计算存储分离的云原生数据库架构、区块链系统等，该实施例包括下述步骤：
301、计算设备响应于执行中断的DDL事务符合恢复条件，获取该DDL事务的管理数据，该管理数据用于记录该DDL事务处理完毕的各个block。
关于在线DDL场景和离线DDL场景下判断该DDL事务是否符合恢复条件的过程，已在上一实施例中介绍过，这里不做赘述。
计算设备在检测到执行中断的DDL事务符合该恢复条件的情况下，获取该DDL事务的管理数据，由于该管理数据用于记录该DDL事务处理完毕的各个block，因此当DDL事务在对每一个block中存储的各条数据记录均处理完毕的情况下，将处理完毕的该block的block ID(数据块标识)记录到该管理数据中。
可选地，仅在管理数据中记录各个block的block ID，该管理数据亦称为block管理结构，从而能够节约管理数据所占用的存储数据量。示意性地，该管理数据为动态数组，该动态数组中存储有该DDL事务处理完毕的各个block的block ID，动态数组中的最后一个元素存放的是最后一个处理完毕的block的block ID。
需要说明的是，动态数组仅仅是管理数据的一种示例性说明，该管理数据还被提供为哈希表、集合、队列、堆栈等其他数据结构，本申请实施例对此不进行具体限定。
在一些实施例中，计算设备在进入DDL事务的第二阶段时，获取该DDL事务的事务ID，对该DDL事务创建管理数据，此后随着DDL事务对数据表的处理过程，实时维护该管理数据，也即是说，随着DDL事务扫描全表数据，每对数据表中的一个block扫描完毕，则将最新扫描完毕的block记录到管理数据中，例如，将最新扫描完毕的block的block ID存储到动态数组的最后一个元素中。某一时刻，DDL事务执行中断，DDL执行检测器发现该DDL事务符合恢复条件，从缓存区中获取该DDL事务的管理数据，例如，从缓存区中获取该DDL事务的动态数组。
302、计算设备从该管理数据中，查询得到该DDL事务在执行中断前处理完毕的最后一个block。
在一些实施例中，由于通常是按照block从小到大的顺序来处理数据表中的各个block，因此在查询时，计算设备从该管理数据中，读取该DDL事务在执行中断前处理完毕的各个block的block ID，将block ID最大的block确定为执行中断前处理完毕的最后一个block。
在一些实施例中，该管理数据为动态数组的情况下，由于动态数组中的最后一个元素存放的是最后一个处理完毕的block的block ID，因此在查询时，计算设备将该动态数组中最后一个元素内存储的block ID所指示的block，确定为执行中断前处理完毕的最后一个block。
在上述步骤301-302中，提供了确定该DDL事务在执行中断前处理完毕的最后一个block的一种可能实施方式，由于以block作为处理DDL事务的最小单元，且对每个DDL事务都维护了管理数据，在管理数据中会记录已处理完毕的各个block，使得DDL事务在执行中断后，能够方便地从管理数据中定位到异常中断点，并从异常中断点开始继续执行DDL事务，而无需从头开始进行重做或者整体回滚，大大提高了DDL事务的成功率。
303、计算设备对该最后一个block的下一个block或该下一个block之后的任一个block，读取该block，处理该block中存储的每条数据记录。
在一些实施例中，由于各个block都是按序存放在数据表中的，因此在定位到执行中断前处理完毕的最后一个block之后，说明该最后一个block之前的所有block都是处理完毕的，无需重复处理，只需要从数据表中定位到该最后一个block的下一个block，并从该最后一个block的下一个block开始，继续执行该DDL事务即可。
例如，DDL事务是增加一个索引，在第二阶段中扫描到数据表的block10时异常中断了，此时block10尚未完成处理因此管理数据中不会记录block10，而是记录了已处理完毕的block1～9，确定出异常中断前处理完毕的最后一个block是block9，因此，计算设备从block9的下一个block10开始，继续执行该DDL事务，这样能够避免将DDL事务整体回滚，并且也无需从block1开始进行重做，最多会产生一个block的冗余工作量(即处理到block10的最后一条数据记录时发生中断)，大大提高了数据库的资源利用率。
在从该最后一个block的下一个block开始继续执行该DDL事务时，对该最后一个block之后的任一个block(包括该下一个block，或者该下一个block之后的任一个block)，由于以 block为单位来处理DDL事务，因此计算设备需要以block为单位读取数据，即，计算设备读取当前block，并处理当前block中存储的每条数据记录，上述处理是依据DDL事务的类型不同而定的，例如，DDL事务是增加一个索引，对每条数据记录都需要填充索引的Key值。
在对当前block的所有数据记录处理完毕后，读取当前block的下一个block，并对下一个block中的每条数据记录执行相同的处理，以此类推，迭代执行上述操作，直到不存在需要处理的block，代表对所有block均处理完毕，执行下述步骤306。
304、计算设备响应于对该DDL事务的进度查询指令，基于该管理数据和该DDL事务所需处理的数据块总数，获取该DDL事务的数据进度。
在本申请实施例中，提供一种对外可实时查询DDL事务的数据进度的功能，由于以block作为处理DDL事务的最小单位，比较DDL事务当前处理完毕的block数量(即，管理数据中记录的block数量)与DDL事务所需处理的block总数，即获取到DDL事务的数据进度。
在一些实施例中，计算设备将该管理数据中记录的block数量除以该block总数所得的数值，确定为该DDL事务的数据进度。可选地，将该数值转换成百分比之后再确定为该DDL事务的数据进度，或者，进行其他的线性或非线性映射，本申请实施例对此不进行具体限定。
305、计算设备向触发该进度查询指令的设备返回该DDL事务的数据进度。
在一些实施例中，如果是单机数据库系统，用户在计算设备上输入进度查询指令，数据库引擎从底层查询并返回DDL事务的数据进度，此时由计算设备自身对该数据进度进行可视化显示。如果是集群数据库系统，用户输入进度查询指令的设备和处理DDL事务的计算设备通常不同，此时计算设备基于进度查询指令向用户侧的设备返回DDL事务的数据进度，由用户侧的设备来对该数据进度进行可视化显示。
可选地，以文本形式直接显示该数据进度的数值，或者，以进度条形式实时更新该数据进度，或者，以动画形式动态显示该数据进度的变化等，本申请实施例不对该数据进度的显示方式进行具体限定。其中，该进度条包括：条形进度条、扇形进度条、环形进度条等。
示意性地，用户可输入如下代码，以在输入DDL语句的同时，附带显示出DDL事务的数据进度，以DDL事务为创建一张新的数据表为例：
Figure PCTCN2022117451-appb-000001
其中，*public*代表主键索引当前状态为Public，即正常状态，通常在DDL事务执行完成后，DDL事务所操作的对象会显示为Public状态，说明该对象当前处于能够正常为用户提供服务的阶段。
其中，*write only 66％，parallel degree 24*代表二级索引idx1当前状态为Write Only，即处于索引创建过程中扫描表数据以生成索引数据的阶段，同时显示当前创建索引的数据进度为66％，并且以24的并行度来并行处理数据。
306、计算设备响应于对该DDL事务操作的各个数据块均处理完毕，提交该DDL事务。
上述步骤306与上述步骤203同理，这里不做赘述。
上述所有可选技术方案，能够采用任意结合形成本公开的可选实施例，在此不再一一赘述。
本申请实施例提供的技术方案，以数据块作为处理DDL事务的最小单元，在DDL事务执行中断时，能够方便地定位到执行中断前处理完毕的最后一个数据块，无需整体回滚该DDL事务，而是从下一个数据块开始继续执行该DDL事务，避免了从头开始重做DDL事务的冗余工作量，提高了DDL语句所指示的DDL事务的成功率，提高了数据库的资源利用率。
在上述实施例中，详细介绍了对执行中断的DDL事务的串行恢复处理流程，由于除了支持串行处理之外，还支持并行处理，因此在本申请实施例中将详细介绍对执行中断的DDL事务的并行恢复处理流程。
图4是本申请实施例提供的一种事务执行方法的流程图。参见图4，该实施例由数据库系统中的任一计算设备执行，该数据库系统包括：单机数据库系统、集群数据库系统、分布式数据库系统、计算存储分离的云原生数据库架构、区块链系统等，该实施例包括下述步骤：
401、计算设备响应于执行中断的DDL事务符合恢复条件，获取该DDL事务的管理数据，该管理数据用于记录该DDL事务处理完毕的各个block。
上述步骤401与上述步骤301同理，这里不做赘述。
402、计算设备从该管理数据中，查询得到该DDL事务在执行中断前处理完毕的最后一个block。
上述步骤402与上述步骤302同理，这里不做赘述。
403、计算设备基于该DDL事务的并行度，从该最后一个block的下一个block开始，对该DDL事务尚未处理的各个block进行并行处理。
并行度是指：并行处理中将会启动多少并行处理任务来将原来需要处理的任务进行分解。通常在计算机处理中，就是将原来需要处理的全量数据，划分为若干子任务，启动并行度指定数量的线程，取得子任务，并按照相应的处理逻辑进行处理，理想情况下并行执行能够达到并行度倍数的性能提升，例如，串行处理模式下，由单个DDL线程扫描全表数据，并行处理模式下，假设并行度为10，则会由10个DDL线程并行扫描全表数据。
在一些实施例中，该并行度是由数据库引擎预先设置的，例如，预先设置某个默认值(如并行度为5)，又例如，预先设置多个并行度，并对每个并行度建立与之关联的数据量区间，形成并行度与数据量区间的映射关系，根据DDL事务所操作的对象涉及的数据量，选择与该数据量所位于的数据量区间具有映射关系的并行度，又例如，默认继承该DDL事务在执行中断之前所采用的并行度，或者继承上一个DDL事务所采用的并行度，又例如，该并行度是在执行中断之后由技术人员进行设定的，本申请实施例不对该并行度的获取方式进行具体限定。
上述步骤403中，从该最后一个block的下一个block开始，继续执行该DDL事务，与上述步骤303不同的是，将串行处理转换为并行处理。在并行处理模式下，基于该DDL事务的并行度(不同的DDL事务具有相同或不同的并行度，这里不做限定)，可将原本单个的DDL事务划分成多个DDL子事务，其中DDL子事务的数量等于该DDL事务的并行度，并分别由并行度数量的进程或线程来并行处理该多个DDL子事务，此外，为避免冗余的工作量，不同的DDL子事务中不存在任何相同的block，即避免在并行处理中对某一block重复扫描多次所带来的冗余工作量。其中，并行处理模式下每个DDL线程或进程处理分配至自身的DDL子事务时，与串行处理模式下DDL线程或进程处理DDL事务的方式同理，这里不做赘述。
需要说明的是，在单机数据库中，这些进程或线程可使用多线程并行处理能力来实现，在集群数据库中，这些进程或线程可分布在多个计算设备上，以达到分布式并行处理的效果，可缓解单个节点的计算压力。
在一些实施例中，在基于并行度将DDL事务划分为多个DDL子事务时，可根据DDL事务所需处理的block总数进行等分，使得不同进程或线程上分配的任务较为负载均衡。
在另一些实施例中，在基于并行度将DDL事务划分为多个DDL子事务时，由于DDL子事务各自的进程或线程可能并非位于同一计算设备上，而DDL子事务所位于的计算设备当前可用的计算资源也不尽相同，因此无需根据DDL事务所需处理的block总数进行等分，而是根据DDL子事务所位于的计算设备当前可用的计算资源进行灵活分配，例如，当前可用的 计算资源大于预设阈值时，分配的DDL子事务中待扫描的block较多，当前可用的计算资源小于或等于预设阈值时，分配的DDL子事务中待扫描的block较少，其中，该预设阈值为固定值，或者为百分比，本申请实施例对此不进行具体限定。
在另一些实施例中，在基于并行度将DDL事务划分为多个DDL子事务时，以分布式场景为例，由于不同的block可能分布式地存储在不同的计算设备上，因此可依据不同的block所位于的计算设备，对每个计算设备有针对性地划分DDL子事务，例如，先确定该DDL事务所操作的对象涉及到哪些计算设备，再针对每个计算设备，分配存储于该计算设备上的数据的DDL子事务。
404、计算设备在并行处理完毕时，获取该DDL事务的第一block集合和第二block集合，该第一block集合用于记录该DDL事务在开始执行时所需操作的数据表中的各个block，该第二数据块集合用于记录该DDL事务在并行处理完毕时所需操作的数据表中的各个block。
由于在线DDL执行过程中，用户业务是能够修改数据的(采用托马斯写规则)，因此在DDL事务扫描全表的过程中，有可能block会产生分裂，即原本的某个block由于插入了数据记录导致分裂成了多个block，此时并行扫描下有可能会漏掉分裂出来的新的block，从而导致数据不一致。因此，在并行处理完毕后，通过执行上述步骤404，获取第一block集合和第二block集合，来解决由于block分裂所导致的不一致问题。
其中，第一block集合是指DDL事务在第二阶段中开始并行扫描时，收集所需扫描的数据表中所有block ID所形成的集合，记作blockIdInit，作为并行扫描的基线。
其中，第二block集合是指DDL事务在第二阶段中并行扫描结束后，重新收集一遍所需扫描的数据表中所有block ID所形成的集合，记作blockIdCur，通过比较第一block集合和第二block集合，只要发生了block分裂现象，那么第二block集合一定会比第一block集合的block ID数量要多，从而能够避免遗漏掉部分分裂产生的block。
在一些实施例中，在开始并行扫描时(通常在执行中断之前)会存储一遍第一block集合，在执行中断之后恢复执行时，一旦并行扫描完毕，则从缓存中读取该第一block集合，并再次收集以获取到第二block集合，比较第一block集合和第二block集合的大小，即比较第一block集合和第二block集合中存储的各个block ID(即集合元素)，如果第一block集合和第二block集合一致，代表DDL执行期间本数据表中没有发生block分裂，执行下述步骤405，直接提交DDL事务；如果第一block集合和第二block集合不一致，代表DDL执行期间本数据表中发生了block分裂，即第二block集合中比第一block集合中新增了block，则计算设备确定该第二block集合相较于该第一block集合中新增的block，在对该新增的block处理完毕后，提交该DDL事务。
在一些实施例中，对上述新增的block采取串行处理，即执行与上一实施例中步骤303同理的操作，或者，对上述新增的block采取并行处理，即返回上述步骤403(可切换至另一并行度，也可采用原本的并行度)，本申请实施例对此不进行具体限定。
405、计算设备在该第一block集合和该第二block集合一致的情况下，提交该DDL事务。
上述步骤405中提交DDL事务的过程与上述步骤306中提交DDL事务的过程同理，这里不做赘述。
需要说明的是，本申请实施例中，也可执行与上述步骤304-305同理的操作，以在并行处理模式下向用户侧的设备获取并返回DDL事务的数据进度，可选地，可获取并返回DDL事务的总数据进度，此外，还支持对每个DDL子事务分别获取并返回各自的子数据进度，并且还支持对用户侧的设备返回设置的并行度，以提高人机交互效率。
上述所有可选技术方案，能够采用任意结合形成本公开的可选实施例，在此不再一一赘述。
本申请实施例提供的技术方案，以数据块作为处理DDL事务的最小单元，在DDL事务执行中断时，能够方便地定位到执行中断前处理完毕的最后一个数据块，无需整体回滚该DDL事务，而是从下一个数据块开始继续执行该DDL事务，避免了从头开始重做DDL事务 的冗余工作量，提高了DDL语句所指示的DDL事务的成功率，提高了数据库的资源利用率。
在上述两个实施例中，详细介绍了在执行中断后，串行恢复处理DDL事务的流程，和并行恢复处理DDL事务的流程，在本申请实施例中，将以DDL事务为新增一个索引为例，详细介绍在线DDL场景下的串行处理流程。
通常在线DDL事务又分为两种：一种是只需要修改数据字典中表的定义就能够完成的DDL事务，例如，在数据表中追加一个数据列，在线DDL执行流程中只需要修改表定义即可；另外一种不但需要修改数据字典中表的定义，同时还需要完成基于表的数据创建对应对象的数据的过程，例如，为数据表增加一个索引，需要修改索引定义，同时需要扫描全表数据并使用对应索引涉及的列信息创建索引对应的Key信息内容(即Key值)。
本申请实施例主要针对上述后一种情况实现中断续做，因为前一种情况本身就不需要扫描全表数据，如果发生异常即便回滚，下次重新执行也只需要完成修改数据字典中表的定义即可，重做的代价也不大。
图5是本申请实施例提供的一种在线增加索引的正常执行流程图，请参考图5，该实施例由数据库系统中的任一计算设备执行，以DDL事务为在线增加索引为例进行说明。
步骤1、开始一个增加索引(Add Index)的DDL任务。
步骤2、修改数据表对应的数据字典，在表结构上增加一个索引，并设置索引当前状态为Delete Only状态，Delete Only状态下只允许对索引进行删除操作。
步骤3、提交事务，保存新的表结构到数据字典当中。
步骤4、开始第二阶段DDL执行。
步骤5、在获得相应权限之后，将索引状态修改为Write Only状态，同时获得一个事务ID。
其中，获取权限的步骤是可选的，以提高DDL事务的安全性，但并非一定要获取权限后才能修改索引状态，也支持不获取权限直接修改索引状态，以提高DDL事务的执行效率，本申请实施例对此不进行具体限定。
其中，将索引修改成Write Only状态之后，后续对于这张表的修改都会同步到索引当中，并且，在设置Write Only状态成功时，如果还有未提交的对于表的修改事务，这些修改事务将会被回滚掉，能够保证创建索引的正确性。
步骤6、开始全表扫描，完成如下任务：
步骤6.1、以步骤5获得的事务ID为扫描和建立二级索引数据事务的事务ID；
步骤6.2、收集表所包含的block总数，并登记到DDL任务当中，用来作为DDL任务的总体工作量值total blocks。
换言之，在步骤6中，开始扫描全表数据，并记录扫描全表数据所需处理的block总数(即总体工作量total blocks)到DDL任务中。
步骤7、开始扫描，并在DDL任务中记录上述事务ID。
步骤8、获取第一个block。
步骤9、处理block中的每一条数据记录，提取对应的数据字段用来创建二级索引的Key，使用托马斯写规则，写入索引结构当中。由于事务ID是在索引状态变为Write Only的时候申请的，因此根据托马斯写规则，之前未提交的事务都被回滚了；此外，上述扫描并创建索引的事务，比后续任何用户对于表修改的事务都要老，因此如果两者之间有冲突的话，则只需要保留后续用户对于表对应数据的修改即可。
步骤10、判断是否处理完block中的数据记录？如果是，即处理完block中的数据记录，转到步骤11；如果否，即尚未处理完block中的数据记录，返回步骤9继续下一条数据记录的处理。
步骤11、在DDL的管理数据(如Map动态数组结构)中登记block已经扫描完毕，并更新已经完成扫描block的总数processed block num。
换言之，在步骤11中，登记当前block到DDL的Map结构中，更新当前已经处理完毕 的block数(即processed block num)。
其中，DDL事务的数据进度等于processed block num/total blocks，在用户使用show create table语句的时候，计算得到当前DDL事务的执行进度，以进行前台可视化显示。
步骤12、获得下一个block。
步骤13、判断是否还有待处理的block？如果是，即仍有待处理的block，返回步骤9继续处理；如果否，即没有待处理的block，说明创建索引过程结束，转到步骤14。
步骤14、修改索引状态为Public(表明索引创建完成，能够被正常使用)。
步骤15、标记DDL任务结束。
需要说明的是，上述步骤6-13中的扫描全表创建二级索引的过程，也能够采取并行实现的方式来加快索引创建的速度，将在后续的实施例中详细描述并行适配的流程，不做赘述。
图6是本申请实施例提供的一种在线增加索引的异常中断后恢复流程图，请参考图6，该实施例由数据库系统中的任一计算设备执行，以DDL事务为在线增加索引为例进行说明。
步骤1、后台线程发现一个异常中断的增加索引(Add Index)任务。
步骤2、判断是否能够继续执行？如果否，即不能继续执行，进入步骤3；如果是，即能够继续执行，进入步骤4。
其中，有两种情况的执行中断没有办法继续：
情况(Ⅰ)、如果该DDL还没有完成第一阶段，则表结构中没有关于该索引的信息，DDL事务无法继续，该DDL事务会被回滚掉。
情况(Ⅱ)、如果出现硬件问题导致block数据读不出来，DDL事务无法继续，该DDL事务会被回滚掉。由于目前部分数据库已经实现了block数据的一致性能力，通常在数据库中每一个block都存在有多个副本，来确保即使出现硬件磁盘故障，数据也不会丢失，所以情况(Ⅱ)只会在极端情况下发生，即所有机器上的所有副本都不可用，这种概率几乎为0。
步骤3、回滚DDL事务，进入步骤13。
步骤4、通过DDL任务中记录的管理数据(如Map动态数组结构)中找到处理完毕的最后一个block。
步骤5、通过步骤4找到的最后一个block，找到下一个block。
可选地，如果是heap表结构，通过文件定位到该最后一个block，再顺序移动到下一个block即可。
可选地，如果是B+树结构，通过最后一个block(叶子block)能够链接到下一个block。
可选地，如果采用region作为数据的逻辑组织方式，则能够通过region找到对应的下一个region继续扫描。
步骤6、判断是否找到下一个block？如果是，即找到了下一个block，进入步骤7；如果否，即没有找到下一个block，进入步骤12。
步骤7、处理block中的每一条数据记录，提取对应的数据字段用来创建二级索引的Key，使用托马斯写规则，写入索引结构当中。由于事务ID是在索引状态变为Write Only的时候申请的，因此根据托马斯写规则，之前未提交的事务都被回滚了；此外，上述扫描并创建索引的事务，比后续任何用户对于表修改的事务都要老，因此如果两者之间有冲突的话，则只需要保留后续用户对于表对应数据的修改即可。
步骤8、判断是否处理完block中的数据记录？如果是，即处理完block中的数据记录，转到步骤9；如果否，即尚未处理完block中的数据记录，返回步骤7继续下一条数据记录的处理。
步骤9、在DDL的管理数据(如Map动态数组结构)中登记block已经扫描完毕，并更新已经完成扫描block的总数processed block num。
换言之，在步骤9中，登记当前block到DDL的Map结构中，更新当前已经处理完毕的block数(即processed block num)。
其中，DDL事务的数据进度等于processed block num/total blocks，在用户使用show create  table语句的时候，计算得到当前DDL事务的执行进度，以进行前台可视化显示。
步骤10、获得下一个block。
步骤11、判断是否还有待处理的block？如果是，即仍有待处理的block，返回步骤7继续处理；如果否，即没有待处理的block，说明创建索引过程结束，转到步骤12。
步骤12、修改索引状态为Public(表明索引创建完成，能够被正常使用)。
步骤13、标记DDL任务结束。
下面针对串行处理模式下，在线DDL事务在执行过程中，如果发生了中断后恢复，且block发生分裂或合并现象进行分析，以验证上述中断后恢复流程的正确性。
场景一、分裂或者合并的block已经扫描处理完毕。
图7是本申请实施例提供的一种block分裂的原理性示意图，假设block n为中断断点，针对block n之前已经扫描处理完毕的block m(m＜n)，上述block m分裂为block m和block m’。通常block分裂的原因是block m中增加了数据记录，或者修改其中数据记录的字段导致超过了block所能够容纳的大小，从而分裂为两个block。因为开始扫描的时候索引状态为Write Only，所有对于block m中记录的新增或者修改都会反映到新建的索引当中，因此这部分数据的变化会同步到索引当中，图7示出了block m向右边分裂的情况，如果block m向左边分裂，也不会有任何影响。
针对合并block，能够看作是分裂block的反向操作，上图7从下往上的变化过程，即block m和block m’合并为block m，一般合并是因为block m或者block m’中删除了数据记录或者修改数据记录，导致block m或者block m’中的空间使用量或空间使用率小于某个阀值，为了更好的利用空间会将这两个block合并，合并后成为上图的block m，和上面分裂的情况一样，删除数据记录或者修改数据记录也会更新到索引当中，因此索引的数据也是正确的。
综上所述，针对场景一，只需要从block n的下一个block继续扫描并完成索引填写即可，因为索引变为Write Only状态之后的所有修改，都会同步到索引当中，从而能够保证数据的一致性。
场景二、分裂或者合并的block，就是当前扫描处理的block。
上述场景二也分为两种情况，其一，当前处理的数据记录在分裂后的block m中，其二，当前处理的数据记录在分裂后的block m’中。
图8是本申请实施例提供的一种block分裂的原理性示意图，假设当前正在处理block m时DDL事务发生中断，在中断之后重启任务，且block m分裂为block m和block m’。此时找到block m的前一个完成的block m-1，并从block m-1的下一个block m开始重新扫描，扫描的数据记录如果之前已经扫描并填充到索引中，那么多次扫描会因为托马斯写规则被丢弃掉，因此会产生最多一个block所容纳的数据记录的冗余工作量，即假设block m原来已经扫描处理到最后一条数据记录了，中断后就相当于重新处理了一遍block m然后丢弃了。
在上述两个实施例中，详细介绍了在线DDL场景下，DDL事务的正常串行处理流程和中断后恢复流程，而在本申请实施例和下一实施例中，将详细介绍离线DDL场景下，DDL事务的正常串行处理流程和中断后恢复流程。
图9是本申请实施例提供的一种离线增加索引的正常执行流程图，请参考图9，该实施例由数据库系统中的任一计算设备执行，以DDL事务为离线增加索引为例进行说明。
步骤1、开始一个离线(非Online)增加索引的DDL任务。
步骤2、创建一个DDL执行后的表结构的临时数据表，即创建一张新定义的临时数据表。
步骤3、申请数据表对应的权限(申请锁)。
需要说明的是，对于分布式数据库，需要申请分布式的锁。
步骤4、将表状态设置为No Write(禁止写)状态。
其中，No Write状态一种特殊的离线DDL场景下的表状态，当表定义设置为No Write状态，在分布式数据库中的其他节点对于表的修改事务都不允许进行或者提交，即使表状态在进入No Write状态之前开始的事务，在提交的时候发现表状态变为了No Write，那么数据 库引擎也会拒绝事务提交。
步骤5、开始全表扫描，并收集表所包含的block总数，并登记到DDL任务当中，用来作为DDL任务的总体工作量值total blocks。
换言之，在步骤5中，开始扫描全表数据，并记录扫描全表数据所需处理的block总数(即总体工作量total blocks)到DDL任务中。
步骤6、获取第一个block。
步骤7、处理block中的每一条数据记录，将原数据表中的每一条数据记录按照新数据表结构组合成为新的数据记录，并插入步骤2创建的临时数据表，即按照新的表结构修改后插入步骤2创建的临时数据表中。
步骤8、判断是否处理完block中的数据记录？如果是，即处理完block中的数据记录，转到步骤9；如果否，即尚未处理完block中的数据记录，返回步骤7继续下一条数据记录的处理。
步骤9、在DDL的管理数据(如Map动态数组结构)中登记block已经扫描完毕，并更新已经完成扫描block的总数processed block num。
换言之，在步骤9中，登记当前block到DDL的Map结构中，更新当前已经处理完毕的block数(即processed block num)。
其中，DDL事务的数据进度等于processed block num/total blocks，在用户使用show create table语句的时候，计算得到当前DDL事务的执行进度，以进行前台可视化显示。
步骤10、获得下一个block。
步骤11、判断是否还有待处理的block？如果是，即仍有待处理的block，返回步骤7继续处理；如果否，即没有待处理的block，说明创建索引过程结束，转到步骤12。
步骤12、修改原数据表的表名，将步骤2创建的临时数据表的表名修改为原数据表的表名，异步删除原数据表。
步骤13、标记DDL任务结束。
需要说明的是，上述步骤5-11中的扫描全表创建二级索引的过程，也能够采取并行实现的方式来加快索引创建的速度，将在后续的实施例中详细描述并行适配的流程，不做赘述。
图10是本申请实施例提供的一种离线增加索引的异常中断后恢复流程图，请参考图10，该实施例由数据库系统中的任一计算设备执行，以DDL事务为离线增加索引为例进行说明。
步骤1、后台线程发现一个异常中断的DDL任务。
步骤2、判断是否能够继续执行？如果是，即原数据表和临时数据表都存在，需要从扫描表过程的中断点恢复扫描，进入步骤4；如果否，即不满足上述情况，即原数据表或临时数据表中至少一项不存在，进入步骤3。
步骤3、判断是否回滚DDL事务？如果是，进入步骤13，即只有原数据表存在，临时数据表还没有创建，回滚DDL事务；如果否，则分为三种情况：a)原数据表改名成功，临时数据表也在，扫描全表已经结束，继续推进；b)临时数据表改名为原数据表，原数据表改名后的临时数据表也存在，继续推进，从数据字典中删除临时数据表，异步完成删除临时数据表中的数据记录；c)临时数据表改名为原数据表，原数据表改名后的临时数据表不存在，继续推进，确认异步完成删除临时数据表中的数据记录，满足a)-c)中任一种则进入步骤12。
步骤4、申请表相应的修改权限(申请锁)，此时表状态还是No Write状态。
步骤5、通过DDL任务中记录的管理数据(如Map动态数组结构)中找到处理完毕的最后一个block。
步骤6、判断是否找到下一个block？如果是，即找到了下一个block，进入步骤7；如果否，即没有找到下一个block，进入步骤12。
步骤7、处理block中的每一条数据记录，将原数据表中的每一条数据记录按照新数据表结构组合成为新的数据记录，并插入步骤2创建的临时数据表，即按照新的表结构修改后插入步骤2创建的临时数据表中。
步骤8、判断是否处理完block中的数据记录？如果是，即处理完block中的数据记录，转到步骤9；如果否，即尚未处理完block中的数据记录，返回步骤7继续下一条数据记录的处理。
步骤9、在DDL的管理数据(如Map动态数组结构)中登记block已经扫描完毕，并更新已经完成扫描block的总数processed block num。
换言之，在步骤9中，登记当前block到DDL的Map结构中，更新当前已经处理完毕的block数(即processed block num)。
步骤10、获得下一个block。
步骤11、判断是否还有待处理的block？如果是，即仍有待处理的block，返回步骤7继续处理；如果否，即没有待处理的block，说明创建索引过程结束，转到步骤12。
步骤12、修改原数据表的表名，将步骤2创建的临时数据表的表名修改为原数据表的表名，异步删除原数据表。
步骤13、标记DDL任务结束。
在上述实施例中，在离线即非Online DDL执行的过程中，通过将表的状态设置为No Write，就保证了在DDL事务完成或者回滚前，数据表对应的block不会发生变化。
上述各个实施例中，以DDL事务为增加索引为例，介绍了在线DDL、离线DDL场景下各自的正常执行流程和异常中断恢复流程，而在本申请实施例中，将介绍上述各类应用场景下，如何与并行处理模式进行适配。由于是以block为单位进行并行处理，因此需要增加用于并行处理的2个数据结构：第一block集合和第二block集合。
第一block集合，记作blockIdInit，是指在开始扫描全表时收集到当前所有的block ID构成的集合，用来作为并行扫描的基线。
第二block集合，记作blockIdCur，是指在并行扫描结束后重新收集一遍所有block ID构成的集合，用来与基线对比，以检查出是否有因为并行扫描漏掉的block。
需要说明的是，只有在线DDL并采用并行处理模式的情况下，才有可能会漏掉block，因为在线DDL执行过程中，用户业务是能够修改数据记录的，因此在扫描全表过程中block会产生分裂，并行扫描因为需要初始化记录block ID，因此会出现可能后续新分裂出来的block ID被漏掉，即新分裂出来的block仍然需要保证被处理完毕，才能够保证数据一致性。对于离线DDL并采用并行处理模式，由于用户业务被阻塞，block在DDL执行过程中不会发生变化，因此并行执行扫描的完成并不会遗漏某些block。另外，只有block分裂才有可能造成遗漏，block合并是不影响的，因为合并的话会将2个block合并为一个，这样block在初始化的时候都在blockIdInit中有记录，因此不会漏掉数据。
图11是本申请实施例提供的一种在线增加索引的正常并行执行流程图，请参考图11该实施例由数据库系统中的任一计算设备执行，以DDL事务为在线增加索引为例进行说明。
步骤1、开始一个增加索引(Add Index)的DDL任务。
步骤2、修改数据表对应的数据字典，在表结构上增加一个索引，并设置索引当前状态为Delete Only状态，Delete Only状态下只允许对索引进行删除操作。
步骤3、提交事务，保存新的表结构到数据字典当中。
步骤4、开始第二阶段DDL执行。
步骤5、在获得相应权限之后，将索引状态修改为Write Only状态，同时获得一个事务ID。
其中，获取权限的步骤是可选的，以提高DDL事务的安全性，但并非一定要获取权限后才能修改索引状态，也支持不获取权限直接修改索引状态，以提高DDL事务的执行效率，本申请实施例对此不进行具体限定。
其中，将索引修改成Write Only状态之后，后续对于这张表的修改都会同步到索引当中，并且，在设置Write Only状态成功时，如果还有未提交的对于表的修改事务，这些修改事务将会被回滚掉，能够保证创建索引的正确性。
步骤6、开始扫描全表数据，并以步骤5获得的事务ID为扫描和建立二级索引数据事务的事务ID。
步骤7、收集需要扫描的表的block ID集合(第一block集合)，收集表中所包含的block总数，并登记到DDL任务当中，用来作为DDL任务的总体工作量值total blocks。
换言之，在步骤7中，记录全表中需要扫描的block ID集合(第一block集合)和扫描全表数据所需处理的block总数(即总体工作量total blocks)到DDL任务中。
步骤8、选择合适的并行度，根据该block ID集合，初始化并行任务，启动并行执行流程(步骤9-15为并行任务的执行主流程)。
步骤9、开始扫描，并在DDL任务中记录上述事务ID。
步骤10、获取第一个block。
步骤11、处理block中的每一条数据记录，提取对应的数据字段用来创建二级索引的Key，使用托马斯写规则，写入索引结构当中。由于事务ID是在索引状态变为Write Only的时候申请的，因此根据托马斯写规则，之前未提交的事务都被回滚了；此外，上述扫描并创建索引的事务，比后续任何用户对于表修改的事务都要老，因此如果两者之间有冲突的话，则只需要保留后续用户对于表对应数据的修改即可。
步骤12、判断是否处理完block中的数据记录？如果是，即处理完block中的数据记录，转到步骤13；如果否，即尚未处理完block中的数据记录，返回步骤11继续下一条数据记录的处理。
步骤13、在DDL的管理数据(如Map动态数组结构)中登记block已经扫描完毕，并更新已经完成扫描block的总数processed block num。
换言之，在步骤13中，登记当前block到DDL的Map结构中，更新当前已经处理完毕的block数(即processed block num)。
其中，DDL事务的数据进度等于processed block num/total blocks，在用户使用show create table语句的时候，计算得到当前DDL事务的执行进度，以进行前台可视化显示。
其中，并行任务修改同一变量实现由数据并发编程变量控制实现。
步骤14、获得下一个block。
步骤15、判断是否还有待处理的block？如果是，即仍有待处理的block，返回步骤11继续处理；如果否，即没有待处理的block，说明创建索引过程结束，转到步骤16。
步骤16、判断是否并行执行？如果是，即处于并行处理模式，进入步骤17；如果否，即处于串行处理模式，进入步骤21。
步骤17、确定所有并行处理任务结束。
步骤18、再次获取表的block ID集合(第二block集合)。
步骤19、将步骤18获取的block ID集合和步骤7初始化的block ID集合比较，是否有新增的(由于block分裂导致未处理的)block？如果有新增的block，说明执行过程中发生了分裂，进入步骤20；如果没有新增的block，说明所有block处理完毕，进入步骤21。
步骤20、判断是否需要并行执行？如果是，即对新增的block仍需要并行执行(例如新生成了很多block，依然选择并行执行)，返回步骤8，对新增的block构成的集合选择并行度，初始化并行任务，启动并行执行流程；如果否，即对新增的block不需要并行执行，返回步骤9，进行串行处理新增的block，这种串行执行方式在结束后不需要再判断过程中是否新增了block。
步骤21、修改索引状态为Public(表明索引创建完成，能够被正常使用)，同时标记DDL任务为已完成。
图12本申请实施例提供的一种在线增加索引的异常中断后并行恢复流程图，请参考图12，该实施例由数据库系统中的任一计算设备执行，以DDL事务为在线增加索引为例说明。
步骤1、后台线程发现一个异常中断的增加索引(Add Index)任务。
步骤2、判断是否能够继续执行？如果否，即不能继续执行，进入步骤3；如果是，即能 够继续执行，进入步骤4。
其中，有两种情况的执行中断没有办法继续：
情况(Ⅰ)、如果该DDL还没有完成第一阶段，则表结构中没有关于该索引的信息，DDL事务无法继续，该DDL事务会被回滚掉。
情况(Ⅱ)、如果出现硬件问题导致block数据读不出来，DDL事务无法继续，该DDL事务会被回滚掉。由于目前部分数据库已经实现了block数据的一致性能力，通常在数据库中每一个block都存在有多个副本，来确保即使出现硬件磁盘故障，数据也不会丢失，所以情况(Ⅱ)只会在极端情况下发生，即所有机器上的所有副本都不可用，这种概率几乎为0。
步骤3、回滚DDL事务，进入步骤18。
步骤4、通过DDL任务中记录的管理数据(如Map动态数组结构)找到处理完毕的最后一个block，并查找最后一个block的下一个block。
步骤5、对剩下的block选择合适的并行度，初始化并行任务，启动并行执行。
步骤6、判断是否找到下一个block？如果是，即找到了下一个block，进入步骤7；如果否，即没有找到下一个block，进入步骤17。
步骤7、处理block中的每一条数据记录，提取对应的数据字段用来创建二级索引的Key，使用托马斯写规则，写入索引结构当中。由于事务ID是在索引状态变为Write Only的时候申请的，因此根据托马斯写规则，之前未提交的事务都被回滚了；此外，上述扫描并创建索引的事务，比后续任何用户对于表修改的事务都要老，因此如果两者之间有冲突的话，则只需要保留后续用户对于表对应数据的修改即可。
步骤8、判断是否处理完block中的数据记录？如果是，即处理完block中的数据记录，转到步骤9；如果否，即尚未处理完block中的数据记录，返回步骤7继续下一条数据记录的处理。
步骤9、在DDL的管理数据(如Map动态数组结构)中登记block已经扫描完毕，并更新已经完成扫描block的总数processed block num。
换言之，在步骤9中，登记当前block到DDL的Map结构中，更新当前已经处理完毕的block数(即processed block num)。
其中，DDL事务的数据进度等于processed block num/total blocks，在用户使用show create table语句的时候，计算得到当前DDL事务的执行进度，以进行前台可视化显示。
步骤10、获得下一个block。
步骤11、判断是否还有待处理的block？如果是，即仍有待处理的block，返回步骤7继续处理；如果否，即没有待处理的block，说明创建索引过程结束，转到步骤12。
步骤12、判断是否并行执行？如果是，即处于并行处理模式，进入步骤13；如果否，即处于串行处理模式，进入步骤17。
步骤13、确定所有并行处理任务结束。
步骤14、再次获取表的block ID集合(第二block集合)。
步骤15、将步骤14获取的block ID集合和DDL事务初始化的block ID集合比较，是否有新增的(由于block分裂导致未处理的)block？如果有新增的block，说明执行过程中发生了分裂，进入步骤16；如果没有新增的block，说明所有block处理完毕，进入步骤17。
步骤16、判断是否需要并行执行？如果是，即对新增的block仍需要并行执行(例如新生成了很多block，依然选择并行执行)，返回步骤5，对新增的block构成的集合选择并行度，初始化并行任务，启动并行执行流程；如果否，即对新增的block不需要并行执行，返回步骤6，进行串行处理新增的block，这种串行执行方式在结束后不需要再判断过程中是否新增了block。
步骤17、修改索引状态为Public(表明索引创建完成，能够被正常使用)。
步骤18、标记DDL任务结束。
通过上述两个实施例，能够看出，在并行扫描数据情况下，增加了一个新增block集合 的检测与处理的过程，即比较第一block集合和第二block集合中是否发现新增block，以避免有分裂产生的新的block被漏掉。需要说明的是，串行执行是不会出现漏扫描的，因为串行的执行不需要提前采集block集合，顺序扫描所有的block，因此不会出现漏掉过程中新分裂的block中有未扫描到的数据。
上述两个实施例所提供的并行执行流程，能够大大的提升数据大表的扫描速度，相对于增加的检测过程来说，数据库系统的整体性能仍然能够保证大幅度提升。
在上述各个实施例中，详细介绍了在各类应用场景下，从DDL事务的数据处理中断点，恢复继续执行的技术方案，针对涉及到需要扫码全表完成数据处理的每条DDL事务，理论上只需要扫码一次全表数据即可完成DDL事务，从而大大提高了DDL事务的成功率；此外，由于能够从中断处继续执行，大大节约DDL执行期间占用的集群计算和存储资源，使得之前计算所使用的资源不会因为DDL事务的中断，最终语句被回滚而被浪费掉，换言之，大大的节约了由于DDL事务失败带来的资源重复使用的浪费，使得分布式数据库资源利用率更高。
进一步地，由于能够很方便地返回DDL事务的数据进度，使得用户能够清楚的知道目前DDL执行的情况，提升了人机交互效率。此外，由于以block为单位来处理DDL事务，很容易实现并行扫描处理数据，从而大大提升了DDL执行的速度，通过合理地并行处理数据，能够真正做到迅速完成DDL语句。
图13是本申请实施例提供的一种事务执行装置的结构示意图，如图13所示，该装置包括：
确定模块1301，用于在执行中断的数据定义语言DDL事务符合恢复条件的情况下，确定该DDL事务在执行中断前处理完毕的最后一个数据块；
执行模块1302，用于从该最后一个数据块的下一个数据块开始，继续执行该DDL事务；
提交模块1303，用于在对该DDL事务操作的各个数据块均处理完毕的情况下，提交该DDL事务。
本申请实施例提供的技术方案，以数据块作为处理DDL事务的最小单元，在DDL事务执行中断时，能够方便地定位到执行中断前处理完毕的最后一个数据块，无需整体回滚该DDL事务，而是从下一个数据块开始继续执行该DDL事务，避免了从头开始重做DDL事务的冗余工作量，提高了DDL语句所指示的DDL事务的成功率，提高了数据库的资源利用率。
在一些实施例中，基于图13的装置组成，该确定模块1301包括：获取单元，用于获取该DDL事务的管理数据，该管理数据用于记录该DDL事务处理完毕的各个数据块；查询单元，用于从该管理数据中，查询得到该最后一个数据块。
在一些实施例中，该管理数据为动态数组，该动态数组中存储有该DDL事务处理完毕的各个数据块的数据块标识；该查询单元用于：将该动态数组中最后一个元素内存储的数据块标识所指示的数据块，确定为该最后一个数据块。
在一些实施例中，在对每一个数据块中存储的各条数据记录均处理完毕的情况下，将该数据块的数据块标识记录在该管理数据中。
在一些实施例中，基于图13的装置组成，该装置还包括：第一获取模块，用于响应于对该DDL事务的进度查询指令，基于该管理数据和该DDL事务所需处理的数据块总数，获取该DDL事务的数据进度；发送模块，用于向触发该进度查询指令的设备返回该DDL事务的数据进度。
在一些实施例中，该第一获取模块用于：将该管理数据中记录的数据块数量除以该数据块总数所得的数值，确定为该DDL事务的数据进度。
在一些实施例中，该执行模块1302用于：对该下一个数据块或该下一个数据块之后的任一个数据块，读取该数据块，处理该数据块中存储的每条数据记录。
在一些实施例中，该执行模块1302用于：基于该DDL事务的并行度，从该最后一个数据块的下一个数据块开始，对该DDL事务尚未处理的各个数据块进行并行处理。
在一些实施例中，基于图13的装置组成，该装置还包括：第二获取模块，用于在并行处 理完毕的情况下，获取该DDL事务的第一数据块集合和第二数据块集合，该第一数据块集合用于记录该DDL事务在开始执行时所需操作的数据表中的各个数据块，该第二数据块集合用于记录该DDL事务在并行处理完毕时所需操作的数据表中的各个数据块；该提交模块1303，还用于在该第一数据块集合和该第二数据块集合一致的情况下，提交该DDL事务；该执行模块1302，还用于在该第一数据块集合和该第二数据块集合不一致的情况下，确定该第二数据块集合相较于该第一数据块集合中新增的数据块；该提交模块1303，还用于在对该新增的数据块处理完毕后，提交该DDL事务。
在一些实施例中，在该DDL事务为在线DDL事务的情况下，该恢复条件包括：该DDL事务操作的数据表的数据字典中包含该DDL事务操作的对象的定义，且该DDL事务操作的数据块或该数据块的至少一个副本可读。
在一些实施例中，在该DDL事务为离线DDL事务的情况下，该恢复条件包括：该DDL事务操作的原数据表和该DDL事务创建的临时数据表均存在。
上述所有可选技术方案，能够采用任意结合形成本公开的可选实施例，在此不再一一赘述。
需要说明的是：上述实施例提供的事务执行装置在执行DDL事务时，仅以上述各功能模块的划分进行举例说明，实际应用中，能够根据需要而将上述功能分配由不同的功能模块完成，即将计算设备的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的事务执行装置与事务执行方法实施例属于同一构思，其具体实现过程详见事务执行方法实施例，这里不再赘述。
图14是本申请实施例提供的一种终端的结构示意图，终端1400是计算设备的一种示例性说明。可选地，该终端1400的设备类型包括：智能手机、平板电脑、笔记本电脑或台式电脑。终端1400还可能被称为用户设备、便携式终端、膝上型终端、台式终端等其他名称。
通常，终端1400包括有：处理器1401和存储器1402。
可选地，处理器1401包括一个或多个处理核心，比如4核心处理器、8核心处理器等。可选地，处理器1401采用DSP(Digital Signal Processing，数字信号处理)、FPGA(Field－Programmable Gate Array，现场可编程门阵列)、PLA(Programmable Logic Array，可编程逻辑阵列)中的至少一种硬件形式来实现。在一些实施例中，处理器1401包括主处理器和协处理器，主处理器是用于对在唤醒状态下的数据进行处理的处理器，也称CPU(Central Processing Unit，中央处理器)；协处理器是用于对在待机状态下的数据进行处理的低功耗处理器。在一些实施例中，处理器1401集成有GPU(Graphics Processing Unit，图像处理器)，GPU用于负责显示屏所需要显示的内容的渲染和绘制。
在一些实施例中，存储器1402包括一个或多个计算机可读存储介质，可选地，该计算机可读存储介质是非暂态的。可选地，存储器1402还包括高速随机存取存储器，以及非易失性存储器，比如一个或多个磁盘存储设备、闪存存储设备。在一些实施例中，存储器1402中的非暂态的计算机可读存储介质用于存储至少一个程序代码，该至少一个程序代码用于被处理器1401所执行以实现本申请中各个实施例提供的事务执行方法。
在一些实施例中，终端1400还可选包括有：外围设备接口1403和至少一个外围设备。处理器1401、存储器1402和外围设备接口1403之间能够通过总线或信号线相连。各个外围设备能够通过总线、信号线或电路板与外围设备接口1403相连。外围设备包括：显示屏1405。
外围设备接口1403可被用于将I/O(Input/Output，输入/输出)相关的至少一个外围设备连接到处理器1401和存储器1402。在一些实施例中，处理器1401、存储器1402和外围设备接口1403被集成在同一芯片或电路板上；在一些其他实施例中，处理器1401、存储器1402和外围设备接口1403中的任意一个或两个在单独的芯片或电路板上实现。
显示屏1405用于显示UI(User Interface，用户界面)。可选地，该UI包括图形、文本、图标、视频及其它们的任意组合。当显示屏1405是触摸显示屏时，显示屏1405还具有采集在显示屏1405的表面或表面上方的触摸信号的能力。该触摸信号能够作为控制信号输入至处 理器1401进行处理。可选地，显示屏1405还用于提供虚拟按钮和/或虚拟键盘，也称软按钮和/或软键盘。可选地，显示屏1405采用LCD(Liquid Crystal Display，液晶显示屏)、OLED(Organic Light-Emitting Diode，有机发光二极管)等材质制备。
本领域技术人员能够理解，图14中示出的结构并不构成对终端1400的限定，能够包括比图示更多或更少的组件，或者组合某些组件，或者采用不同的组件布置。
图15是本申请实施例提供的一种计算设备的结构示意图，该计算设备1500可因配置或性能不同而产生比较大的差异，该计算设备1500包括一个或一个以上处理器(Central Processing Units，CPU)1501和一个或一个以上的存储器1502，其中，该存储器1502中存储有至少一条计算机程序，该至少一条计算机程序由该一个或一个以上处理器1501加载并执行以实现上述各个实施例提供的事务执行方法。可选地，该计算设备1500还具有有线或无线网络接口、键盘以及输入输出接口等部件，以便进行输入输出，该计算设备1500还包括其他用于实现设备功能的部件，在此不做赘述。
在示例性实施例中，还提供了一种计算机可读存储介质，例如包括至少一条计算机程序的存储器，上述至少一条计算机程序可由终端中的处理器执行以完成上述各个实施例中的事务执行方法。例如，该计算机可读存储介质包括ROM(Read-Only Memory，只读存储器)、RAM(Random-Access Memory，随机存取存储器)、CD-ROM(Compact Disc Read-Only Memory，只读光盘)、磁带、软盘和光数据存储设备等。
在示例性实施例中，还提供了一种计算机程序产品或计算机程序，包括一条或多条程序代码，该一条或多条程序代码存储在计算机可读存储介质中。计算设备的一个或多个处理器能够从计算机可读存储介质中读取该一条或多条程序代码，该一个或多个处理器执行该一条或多条程序代码，使得计算设备能够执行以完成上述实施例中的事务执行方法。
本领域普通技术人员能够理解实现上述实施例的全部或部分步骤能够通过硬件来完成，也能够通过程序来指令相关的硬件完成，可选地，该程序存储于一种计算机可读存储介质中，可选地，上述提到的存储介质是只读存储器、磁盘或光盘等。
以上所述仅为本申请的可选实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (15)

  1. 一种事务执行方法，由计算设备执行，所述方法包括：
    在执行中断的数据定义语言DDL事务符合恢复条件的情况下，确定所述DDL事务在执行中断前处理完毕的最后一个数据块；
    从所述最后一个数据块的下一个数据块开始，继续执行所述DDL事务；
    在对所述DDL事务操作的各个数据块均处理完毕的情况下，提交所述DDL事务。
  2. 根据权利要求1所述的方法，所述确定所述DDL事务在执行中断前处理完毕的最后一个数据块包括：
    获取所述DDL事务的管理数据，所述管理数据用于记录所述DDL事务处理完毕的各个数据块；
    从所述管理数据中，查询得到所述最后一个数据块。
  3. 根据权利要求2所述的方法，所述管理数据为动态数组，所述动态数组中存储有所述DDL事务处理完毕的各个数据块的数据块标识；
    所述从所述管理数据中，查询得到所述最后一个数据块包括：
    将所述动态数组中最后一个元素内存储的数据块标识所指示的数据块，确定为所述最后一个数据块。
  4. 根据权利要求2或3所述的方法，在对每一个数据块中存储的各条数据记录均处理完毕的情况下，将所述数据块的数据块标识记录在所述管理数据中。
  5. 根据权利要求2或3所述的方法，所述方法还包括：
    响应于对所述DDL事务的进度查询指令，基于所述管理数据和所述DDL事务所需处理的数据块总数，获取所述DDL事务的数据进度；
    向触发所述进度查询指令的设备返回所述DDL事务的数据进度。
  6. 根据权利要求5所述的方法，所述基于所述管理数据和所述DDL事务所需处理的数据块总数，获取所述DDL事务的数据进度包括：
    将所述管理数据中记录的数据块数量除以所述数据块总数所得的数值，确定为所述DDL事务的数据进度。
  7. 根据权利要求1所述的方法，所述从所述最后一个数据块的下一个数据块开始，继续执行所述DDL事务包括：
    对所述下一个数据块或所述下一个数据块之后的任一个数据块，读取所述数据块，处理所述数据块中存储的每条数据记录。
  8. 根据权利要求1或7所述的方法，所述从所述最后一个数据块的下一个数据块开始，继续执行所述DDL事务包括：
    基于所述DDL事务的并行度，从所述最后一个数据块的下一个数据块开始，对所述DDL事务尚未处理的各个数据块进行并行处理。
  9. 根据权利要求8所述的方法，所述方法还包括：
    在并行处理完毕的情况下，获取所述DDL事务的第一数据块集合和第二数据块集合，所述第一数据块集合用于记录所述DDL事务在开始执行时所需操作的数据表中的各个数据块，所述第二数据块集合用于记录所述DDL事务在并行处理完毕时所需操作的数据表中的各个数据块；
    在所述第一数据块集合和所述第二数据块集合一致的情况下，提交所述DDL事务；
    在所述第一数据块集合和所述第二数据块集合不一致的情况下，确定所述第二数据块集合相较于所述第一数据块集合中新增的数据块，在对所述新增的数据块处理完毕后，提交所述DDL事务。
  10. 根据权利要求1所述的方法，在所述DDL事务为在线DDL事务的情况下，所述恢复 条件包括：所述DDL事务操作的数据表的数据字典中包含所述DDL事务操作的对象的定义，且所述DDL事务操作的数据块或所述数据块的至少一个副本可读。
  11. 根据权利要求1所述的方法，在所述DDL事务为离线DDL事务的情况下，所述恢复条件包括：所述DDL事务操作的原数据表和所述DDL事务创建的临时数据表均存在。
  12. 一种事务执行装置，所述装置包括：
    确定模块，用于在执行中断的数据定义语言DDL事务符合恢复条件的情况下，确定所述DDL事务在执行中断前处理完毕的最后一个数据块；
    执行模块，用于从所述最后一个数据块的下一个数据块开始，继续执行所述DDL事务；
    提交模块，用于在对所述DDL事务操作的各个数据块均处理完毕的情况下，提交所述DDL事务。
  13. 一种计算设备，所述计算设备包括一个或多个处理器和一个或多个存储器，所述一个或多个存储器中存储有至少一条计算机程序，所述至少一条计算机程序由所述一个或多个处理器加载并执行以实现如权利要求1至权利要求11任一项所述的事务执行方法。
  14. 一种存储介质，所述存储介质中存储有至少一条计算机程序，所述至少一条计算机程序由处理器加载并执行以实现如权利要求1至权利要求11任一项所述的事务执行方法。
  15. 一种计算机程序产品，所述计算机程序产品包括至少一条计算机程序，所述至少一条计算机程序由处理器加载并执行以实现如权利要求1至权利要求11任一项所述的事务执行方法。
PCT/CN2022/117451 2021-10-19 2022-09-07 事务执行方法、装置、计算设备及存储介质 WO2023065868A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/450,606 US20230394027A1 (en) 2021-10-19 2023-08-16 Transaction execution method, computing device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111214946.4 2021-10-19
CN202111214946.4A CN115098537B (zh) 2021-10-19 2021-10-19 事务执行方法、装置、计算设备及存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/450,606 Continuation US20230394027A1 (en) 2021-10-19 2023-08-16 Transaction execution method, computing device, and storage medium

Publications (1)

Publication Number Publication Date
WO2023065868A1 true WO2023065868A1 (zh) 2023-04-27

Family

ID=83287679

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/117451 WO2023065868A1 (zh) 2021-10-19 2022-09-07 事务执行方法、装置、计算设备及存储介质

Country Status (3)

Country Link
US (1) US20230394027A1 (zh)
CN (1) CN115098537B (zh)
WO (1) WO2023065868A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115509694B (zh) * 2022-10-08 2024-04-30 北京火山引擎科技有限公司 一种事务处理方法、装置、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100011026A1 (en) * 2008-07-10 2010-01-14 International Business Machines Corporation Method and system for dynamically collecting data for checkpoint tuning and reduce recovery time
CN103092712A (zh) * 2011-11-04 2013-05-08 阿里巴巴集团控股有限公司 一种任务中断恢复方法和设备
CN103677752A (zh) * 2012-09-19 2014-03-26 腾讯科技（深圳）有限公司 基于分布式数据的并发处理方法和系统
CN106294477A (zh) * 2015-06-05 2017-01-04 阿里巴巴集团控股有限公司 一种数据处理方法和装置
CN106682017A (zh) * 2015-11-09 2017-05-17 高德软件有限公司 一种数据库更新方法及装置
US20180322156A1 (en) * 2017-05-08 2018-11-08 Sap Se Atomic processing of compound database transactions that modify a metadata entity

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0944437A (ja) * 1995-08-02 1997-02-14 Canon Inc デバイスドライバ制御方法及び装置及び情報処理装置
CN102752372A (zh) * 2012-06-18 2012-10-24 天津神舟通用数据技术有限公司 一种基于文件的数据库同步方法
CN102968374B (zh) * 2012-11-29 2015-12-09 中国移动（深圳）有限公司 一种数据仓库测试方法
US10769134B2 (en) * 2016-10-28 2020-09-08 Microsoft Technology Licensing, Llc Resumable and online schema transformations
US20190102401A1 (en) * 2017-09-29 2019-04-04 Oracle International Corporation Session state tracking
CN109241175B (zh) * 2018-06-28 2021-06-04 东软集团股份有限公司 数据同步方法、装置、存储介质及电子设备
CN110837535A (zh) * 2018-08-16 2020-02-25 中国移动通信集团江西有限公司 数据同步的方法、装置、设备和介质
US11550514B2 (en) * 2019-07-18 2023-01-10 Pure Storage, Inc. Efficient transfers between tiers of a virtual storage system
CN110647579A (zh) * 2019-08-16 2020-01-03 北京百度网讯科技有限公司 数据同步方法及装置、计算机设备与可读介质
CN110727709A (zh) * 2019-10-10 2020-01-24 北京优炫软件股份有限公司 一种集群数据库系统
CN113254425B (zh) * 2021-06-24 2022-01-11 阿里云计算有限公司 数据库事务保持的方法、设备、系统、程序及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100011026A1 (en) * 2008-07-10 2010-01-14 International Business Machines Corporation Method and system for dynamically collecting data for checkpoint tuning and reduce recovery time
CN103092712A (zh) * 2011-11-04 2013-05-08 阿里巴巴集团控股有限公司 一种任务中断恢复方法和设备
CN103677752A (zh) * 2012-09-19 2014-03-26 腾讯科技（深圳）有限公司 基于分布式数据的并发处理方法和系统
CN106294477A (zh) * 2015-06-05 2017-01-04 阿里巴巴集团控股有限公司 一种数据处理方法和装置
CN106682017A (zh) * 2015-11-09 2017-05-17 高德软件有限公司 一种数据库更新方法及装置
US20180322156A1 (en) * 2017-05-08 2018-11-08 Sap Se Atomic processing of compound database transactions that modify a metadata entity

Also Published As

Publication number Publication date
US20230394027A1 (en) 2023-12-07
CN115098537B (zh) 2023-03-10
CN115098537A (zh) 2022-09-23

Similar Documents

Publication Publication Date Title
US10860612B2 (en) Parallel replication across formats
WO2022161308A1 (zh) 事务处理方法、装置、计算机设备及存储介质
US10185632B2 (en) Data synchronization with minimal table lock duration in asynchronous table replication
Bichsel et al. A simple algorithm for shape from shading
Loesing et al. On the design and scalability of distributed shared-data databases
US11263236B2 (en) Real-time cross-system database replication for hybrid-cloud elastic scaling and high-performance data virtualization
US20170192863A1 (en) System and method of failover recovery
US20130262389A1 (en) Parallel Backup for Distributed Database System Environments
US20220138056A1 (en) Non-Blocking Backup in a Log Replay Node for Tertiary Initialization
US20160210228A1 (en) Asynchronous garbage collection in a distributed database system
US20230418811A1 (en) Transaction processing method and apparatus, computing device, and storage medium
Waqas et al. Transaction management techniques and practices in current cloud computing environments: A survey
US20230098963A1 (en) Object processing method and apparatus, computer device, and storage medium
Agrawal et al. A Taxonomy of Partitioned Replicated Cloud-based Database Systems.
US20230394027A1 (en) Transaction execution method, computing device, and storage medium
Margara et al. A model and survey of distributed data-intensive systems
US10650021B2 (en) Managing data operations in an integrated database system
EP4356257A2 (en) Versioned metadata using virtual databases
US11797523B2 (en) Schema and data modification concurrency in query processing pushdown
Yang From Google file system to omega: a decade of advancement in big data management at Google
Yao et al. Scaling distributed transaction processing and recovery based on dependency logging
Li et al. Research and implementation of a distributed transaction processing middleware
US20230376479A1 (en) Schema and data modification concurrency in query processing pushdown
WO2022242401A1 (zh) 一种数据库系统的事务处理方法、装置、电子设备、计算机可读存储介质及计算机程序产品
US20230195747A1 (en) Performant dropping of snapshots by linking converter streams

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22882483

Country of ref document: EP

Kind code of ref document: A1