WO2023103990A1

WO2023103990A1 - Database transaction snapshot generation method and apparatus, device, and storage medium

Info

Publication number: WO2023103990A1
Application number: PCT/CN2022/136680
Authority: WO
Inventors: 王烨; 宦家佳
Original assignee: 中兴通讯股份有限公司
Priority date: 2021-12-10
Filing date: 2022-12-05
Publication date: 2023-06-15
Also published as: CN116303754A

Abstract

The present application relates to the field of databases, and provides a database transaction snapshot generation method and apparatus, a device, and a storage medium. The database transaction snapshot generation method comprises: alternately acquiring transaction snapshots generated by a first global transaction management component and a second global transaction management component at different moments, respectively; according to a transaction number in an active state in two adjacent transaction snapshots acquired from the first global transaction management component, generating a transaction snapshot corresponding to the first global transaction management component; according to a transaction number in the active state in two adjacent transaction snapshots acquired from the second global transaction management component, generating a transaction snapshot corresponding to the second global transaction management component; and generating a global transaction snapshot according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component.

Description

Database transaction snapshot generation method, device, equipment and storage medium

related application

This application claims the priority of the Chinese patent application with application number 202111509135.7 filed on December 10, 2021.

technical field

The embodiments of the present application relate to the field of databases, and in particular to a method, device, device, and storage medium for generating transaction snapshots of databases.

Background technique

In a distributed database, multiple global transaction managers are usually deployed to solve the problem of high concurrency pressure on each component or node in a load balancing manner. However, while deploying multiple global transaction managers to share the concurrency pressure of the entire system, it also makes Multiple global transaction managers have difficulty producing globally consistent snapshots.

Contents of the invention

The main purpose of the embodiments of the present application is to provide a database transaction snapshot generation method, device, device and storage medium, so as to achieve accurate global consistent snapshot acquisition.

In order to achieve the above purpose, an embodiment of the present application provides a database transaction snapshot generation method, comprising: alternately obtaining the transaction snapshots generated by the first global transaction management component and the second global transaction management component at different times; according to the The transaction number in the active state in the two adjacent transaction snapshots obtained by the first global transaction management component generates a transaction snapshot corresponding to the first global transaction management component; according to the transaction snapshot obtained from the second global transaction management component The transaction number in the active state in two adjacent transaction snapshots generates a transaction snapshot corresponding to the second global transaction management component; according to the transaction snapshot corresponding to the first global transaction management component and the second global transaction The transaction snapshot corresponding to the management component generates a global transaction snapshot.

In order to achieve the above purpose, the embodiment of the present application also proposes a database transaction snapshot generation device, including:

An acquisition module, configured to alternately acquire transaction snapshots generated by the first global transaction management component and the second global transaction management component at different times;

A first generating module, configured to generate a transaction snapshot corresponding to the first global transaction management component according to the transaction number in an active state in two consecutive transaction snapshots obtained from the first global transaction management component;

The second generation module is configured to generate a transaction snapshot corresponding to the second global transaction management component according to the transaction number in the active state in two adjacent transaction snapshots obtained from the second global transaction management component;

The third generating module is configured to generate a global transaction snapshot according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component.

To achieve the above purpose, an embodiment of the present application also provides an electronic device, including: at least one processor; and a memory connected to the at least one processor in communication; wherein, the memory stores information that can be used by the at least one processor An instruction executed by a processor, the instruction is executed by the at least one processor, so that the at least one processor can execute the above-mentioned method for generating a database transaction snapshot.

In order to achieve the above purpose, the embodiment of the present application also proposes a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned method for generating a database transaction snapshot is realized.

A transaction snapshot generation method, device, device and storage medium of a database proposed by the present application obtain transaction snapshots of multiple global transaction management components at different times alternately, and generate a global transaction snapshot according to multiple transaction snapshots, so that the final The obtained global transaction snapshot can correct the transaction status that has changed within the generation time difference of multiple transaction snapshots. The whole process can quickly and accurately obtain globally consistent snapshots.

Description of drawings

Fig. 1a is a schematic diagram of the transaction state in the distributed database system provided by the embodiment of the present application;

Fig. 1b is a schematic diagram of a transaction snapshot in a distributed database system provided by an embodiment of the present application;

Fig. 2 is the flow chart of the transaction snapshot generation method of the database that the embodiment of the present application provides;

FIG. 3 is a schematic structural diagram of a database transaction snapshot generation device provided by an embodiment of the present application;

FIG. 4 is a schematic structural diagram of an electronic device provided in an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the accompanying drawings. However, those of ordinary skill in the art can understand that in each embodiment of the application, many technical details are provided for readers to better understand the application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in this application can also be realized. The division of the following embodiments is for the convenience of description, and should not constitute any limitation to the specific implementation of the present application, and the embodiments can be combined and referred to each other on the premise of no contradiction.

The database transaction snapshot generation method of the present application is aimed at distributed databases. A distributed database is logically a unified whole, and physically stored on different physical nodes. An application can access databases distributed in different geographical locations through a network connection. Distributed databases usually expand multiple computing nodes and storage nodes to handle massive data tasks. During data processing, multiple nodes or components processing tasks are prone to high concurrency pressure problems, and multiple global transaction managers are usually deployed. And use load balancing to reduce the concurrency pressure of each node or component. Deploying multiple global transaction managers brings the problem of difficulty in generating globally consistent snapshots.

The meaning of a global snapshot is the global state at a certain moment, and consistency means that for any event saved in the global snapshot, the events that occurred before this event should be saved in this snapshot. Simply put, the combination of the states of all transactions at the same time is a globally consistent snapshot.

However, the current global snapshot generation method has the following problems: as shown in Figure 1a, if two global transaction managers gtm1 and gtm2 are deployed in a distributed database system, the first global transaction manager (i.e. the first global transaction management component ) gtm1 is responsible for managing transaction 8, transaction 9, transaction 10 and transaction 11, the second global transaction manager gtm2 is responsible for managing transaction 91, transaction 92, transaction 93 and transaction 94, before T0 time, transaction 8, transaction 10 and transaction 91 Transaction 9, Transaction 11, Transaction 92, Transaction 93, and Transaction 94 are active transactions that have not been completed. If gtm1 receives a snapshot generation request at time T0, the snapshot obtained by gtm1 is shown in Figure 1b. Similarly, when gtm2 receives a snapshot generation request at T1, the snapshot obtained by gtm2 is shown in Figure 1b. Merge these two snapshots into a global transactional snapshot of the entire system. However, combined with Figure 1a, it can be seen that due to the time difference between gtm1 and gtm2 to generate snapshots, during the time period from T0 to T1, the transaction status managed by gtm1 has changed, that is, transaction 9 has changed from an active transaction that has never ended to a completed transaction. Ended inactive transaction. In this way, the final obtained global transaction snapshot does not meet the consistency requirements.

In one embodiment, if the global transaction snapshot that does not meet the consistency requirements is used for data backup, it is easy to cause data inconsistency on the data shards. It should be noted that in a distributed database system, multiple operations contained in a transaction are usually distributed across multiple data shards, and a transaction should have four attributes: atomicity, consistency, isolation, and durability. Among them, atomicity means that a transaction is an indivisible unit of work, and all operations included in the transaction are either executed or not executed. Consistency means that the transaction must change the database from one consistent state to another consistent state. However, the data inconsistency problem on the data fragmentation makes the corresponding transaction not meet the transaction attribute requirements.

In addition, through globally consistent snapshots, it is also possible to check and recover from faults in the system, and it is also possible to check whether there is a deadlock problem in the current application. All of these need to be realized on the basis of obtaining accurate global consistent snapshots.

However, this application alternately obtains the transaction snapshots of multiple global transaction management components at different times, and generates a global transaction snapshot according to the multiple transaction snapshots, so that the finally obtained global transaction snapshot can correct the changes that occur within the generation time difference of multiple transaction snapshots. transaction status.

The embodiment of the present application relates to a transaction snapshot generation method of a database, as shown in Figure 2, comprising:

Step 201, alternately acquire transaction snapshots generated by the first global transaction management component and the second global transaction management component at different times.

In this embodiment, step 201 specifically includes: sending a first snapshot generation request to the first global transaction management component, receiving the first transaction snapshot generated at the first moment fed back by the first global transaction management component; The management component sends a second snapshot generation request, receives the second transaction snapshot generated at the second moment fed back by the second global transaction management component; sends the third snapshot generation request to the first global transaction management component, and receives the first global transaction management component Feedback of the third transaction snapshot generated at the third moment; sending a fourth snapshot generation request to the second global transaction management component, and receiving the fourth transaction snapshot generated at the fourth moment fed back by the second global transaction management component. For example, taking two global transaction management components gtm1 and gtm2 as an example, gtm1 generates a transaction snapshot at time T0, gtm2 generates a transaction snapshot at time T1, gtm1 generates another snapshot at time T2, and gtm2 generates another snapshot at time T3. In this way, alternate Get snapshots of transactions at different times. Of course, the method for generating transaction snapshots in this application is not limited to two global transaction management components, but can also target multiple global transaction management components.

The sending interval between the third snapshot generation request and the second snapshot generation request is greater than or equal to the time required for the second snapshot generation request to be sent to the second global transaction management component; the sending interval between the fourth snapshot generation request and the third snapshot generation request The time is greater than or equal to the time required for the third snapshot generation request to be sent to the first global transaction management component. In this way, it is guaranteed that the global transaction manager receives the next transaction snapshot generation request after obtaining the previous snapshot generation request.

In addition, when the first global transaction management component and the second global transaction management component are deployed on the same physical device, the sending interval between the third snapshot generation request and the second snapshot generation request is the same as that between the fourth snapshot generation request and the third snapshot generation request. Snapshot generation requests are sent at equal intervals. In this way, while ensuring accurate acquisition of transaction snapshots, the acquisition efficiency is improved.

Step 202: Generate a transaction snapshot corresponding to the first global transaction management component according to an active transaction number in two consecutive transaction snapshots obtained from the first global transaction management component.

In this embodiment, step 202 specifically includes: performing a logical AND operation on the transaction numbers in the active state in two consecutive transaction snapshots obtained from the first global transaction management component to obtain the transaction number corresponding to the first global transaction management component Transactional snapshots. That is to say, only when the same transaction is in the active state that has not ended in the two snapshots, is it confirmed that the transaction is in the active state that has not ended.

Step 203: Generate a transaction snapshot corresponding to the second global transaction management component according to an active transaction number in two consecutive transaction snapshots obtained from the second global transaction management component.

In this embodiment, step 203 specifically includes: performing a logical AND operation on the transaction numbers in the active state in two consecutive transaction snapshots obtained from the second global transaction management component to obtain the transaction number corresponding to the second global transaction management component Transactional snapshots.

Step 204: Generate a global transaction snapshot according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component.

In this embodiment, step 204 specifically includes: numbering the transaction number in the active state in the transaction snapshot corresponding to the first global transaction management component, and the transaction number in the active state in the transaction snapshot corresponding to the second global transaction management component Merge to get a global transaction snapshot.

In addition, after step 204, it also includes: backing up the data in each data fragment according to the global transaction snapshot. In this embodiment, the data in each data fragment can be backed up according to the status of each transaction in the global transaction snapshot. Those skilled in the art can understand that multiple operations included in a transaction are usually executed on multiple data shards.

A database transaction snapshot generation method proposed by this application, by alternately obtaining transaction snapshots of multiple global transaction management components at different times, generating a global transaction snapshot according to multiple transaction snapshots, so that the final obtained global transaction snapshot can be corrected Transaction states that have changed within the time difference between multiple transaction snapshots. The whole process can quickly and accurately obtain globally consistent snapshots.

In addition, it should be understood that the division of steps in the above methods is only for clarity of description, and may be combined into one step or split into multiple steps during implementation. As long as the same logical relationship is included, all Within the scope of protection of this application; adding insignificant modifications to the process or introducing insignificant designs without changing the core design of the process is within the scope of protection of this application.

The embodiment of the present application relates to a database transaction snapshot generation device, as shown in Figure 3, comprising:

An acquisition module 301, configured to alternately acquire transaction snapshots generated by the first global transaction management component and the second global transaction management component at different times;

The first generation module 302 is configured to generate the transaction snapshot corresponding to the first global transaction management component according to the transaction number in the active state in two adjacent transaction snapshots obtained from the first global transaction management component;

The second generation module 303 is configured to generate a transaction snapshot corresponding to the second global transaction management component according to the transaction number in the active state in two consecutive transaction snapshots obtained from the second global transaction management component;

The third generating module 304 is configured to generate a global transaction snapshot according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component.

This device embodiment is mainly aimed at the description of the database transaction snapshot generation method provided by the method embodiment at the software implementation level, and its implementation also needs to rely on hardware support. For example, the functions of related modules can be deployed on the processor, so that The processor runs to implement corresponding functions, especially, the data generated by the running can be stored in the memory for subsequent inspection and use.

All the modules involved in this embodiment are logical modules, and a logical unit may be a physical unit, or a part of a physical unit, or may be realized by a combination of multiple physical units. In addition, in order to highlight the innovative part of the present application, units that are not closely related to solving the technical problem proposed in the present application are not introduced in this embodiment, but this does not mean that there are no other units in this embodiment.

This embodiment is a device embodiment corresponding to the embodiment of the database transaction snapshot generation method, and this embodiment can be implemented in cooperation with the above-mentioned embodiments. The relevant technical details mentioned in the foregoing embodiments are still valid in this embodiment, and will not be repeated here in order to reduce repetition. Correspondingly, the relevant technical details mentioned in this embodiment can also be applied to the above method embodiments.

The embodiment of the present application relates to an electronic device, as shown in FIG. 4 , including: at least one processor 401; Instructions executed by the at least one processor 401, the instructions are executed by the at least one processor 401, so that the at least one processor 401 can execute the data packet capture method in the foregoing implementation manner.

Wherein, the memory and the processor are connected by a bus, and the bus may include any number of interconnected buses and bridges, and the bus connects one or more processors and various circuits of the memory together. The bus may also connect together various other circuits such as peripherals, voltage regulators, and power management circuits, all of which are well known in the art and therefore will not be further described herein. The bus interface provides an interface between the bus and the transceivers. A transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing means for communicating with various other devices over a transmission medium. The data processed by the processor is transmitted on the wireless medium through the antenna, further, the antenna also receives the data and transmits the data to the processor.

The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. Instead, memory can be used to store data that the processor uses when performing operations.

Embodiments of the present application relate to a computer-readable storage medium storing a computer program. When the computer program is executed by the processor, the above-mentioned database transaction snapshot generation method is realized.

That is, those skilled in the art can understand that all or part of the steps in the method of the above-mentioned embodiments can be completed by instructing related hardware through a program, the program is stored in a storage medium, and includes several instructions to make a device ( It may be a single-chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .

Those of ordinary skill in the art can understand that the above-mentioned embodiments are specific embodiments for realizing the present application, and in practical applications, various changes can be made to it in form and details without departing from the spirit and spirit of the present application. scope.

Claims

A database transaction snapshot generation method, comprising:

Alternately acquiring transaction snapshots generated by the first global transaction management component and the second global transaction management component at different times;

Generate a transaction snapshot corresponding to the first global transaction management component according to the transaction number in the active state in two adjacent transaction snapshots obtained from the first global transaction management component;

Generate a transaction snapshot corresponding to the second global transaction management component according to the transaction number in the active state in two adjacent transaction snapshots obtained from the second global transaction management component;

A global transaction snapshot is generated according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component.
The method for generating a transaction snapshot of a database according to claim 1, wherein said first transaction number is generated in an active state in two consecutive transaction snapshots obtained from said first global transaction management component. A transaction snapshot corresponding to a global transaction management component, including:

performing a logical AND operation on the transaction numbers in the active state in the two adjacent transaction snapshots obtained from the first global transaction management component to obtain the transaction snapshot corresponding to the first global transaction management component;

The generating the transaction snapshot corresponding to the second global transaction management component according to the transaction number in the active state in two adjacent transaction snapshots obtained from the second global transaction management component includes:

A logical AND operation is performed on transaction numbers in an active state in two adjacent transaction snapshots obtained from the second global transaction management component to obtain a transaction snapshot corresponding to the second global transaction management component.
The method for generating a transaction snapshot of a database according to claim 2, wherein the global transaction snapshot is generated according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component, include:

Merging the transaction number in the active state in the transaction snapshot corresponding to the first global transaction management component with the transaction number in the active state in the transaction snapshot corresponding to the second global transaction management component to obtain the global Transactional snapshots.
The method for generating a transaction snapshot of a database according to any one of claims 1 to 3, wherein said alternately obtaining the transaction snapshots generated by the first global transaction management component and the second global transaction management component at different times respectively comprises:

Sending a first snapshot generation request to the first global transaction management component, and receiving the first transaction snapshot generated at the first moment fed back by the first global transaction management component;

Send a second snapshot generation request to the second global transaction management component, and receive a second transaction snapshot generated at a second moment fed back by the second global transaction management component;

Send a third snapshot generation request to the first global transaction management component, and receive a third transaction snapshot generated at a third moment fed back by the first global transaction management component;

Sending a fourth snapshot generation request to the second global transaction management component, and receiving a fourth transaction snapshot generated at a fourth moment fed back by the second global transaction management component.
The transaction snapshot generation method for a database according to claim 4, wherein the interval between sending the third snapshot generation request and the second snapshot generation request is greater than or equal to the second snapshot generation request sent to the second snapshot generation request 2. The time required by the global transaction management component;

The interval between sending the fourth snapshot generation request and the third snapshot generation request is greater than or equal to the time required for the third snapshot generation request to be sent to the first global transaction management component.
The transaction snapshot generation method of a database according to claim 5, wherein, when the first global transaction management component and the second global transaction management component are deployed on the same physical device, the third snapshot generation request An interval between sending the second snapshot generation request and an interval between sending the fourth snapshot generation request and the third snapshot generation request is equal.
The method for generating a transactional snapshot of a database according to any one of claims 1 to 3, wherein, after said generating the global transactional snapshot, further comprising:

The data in each data fragment is backed up according to the global transaction snapshot.
A database transaction snapshot generating device, comprising:

The obtaining module is configured to alternately obtain transaction snapshots generated by the first global transaction management component and the second global transaction management component at different times;

The first generation module is configured to generate the transaction snapshot corresponding to the first global transaction management component according to the transaction number in the active state in two consecutive transaction snapshots obtained from the first global transaction management component;

The second generation module is configured to generate the transaction snapshot corresponding to the second global transaction management component according to the transaction number in the active state in two adjacent transaction snapshots obtained from the second global transaction management component;

The third generation module is configured to generate a global transaction snapshot according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component.
An electronic device comprising:

at least one processor; and,

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, the instructions are executed by the at least one processor, so that the at least one processor can perform the operation described in any one of claims 1 to 7 The transaction snapshot generation method of the database described above.
A computer-readable storage medium storing a computer program, wherein, when the computer program is executed by a processor, the method for generating a database transaction snapshot according to any one of claims 1 to 7 is implemented.