WO2023103990A1 - Procédé et appareil de génération d'instantané de transaction de base de données, dispositif, et support de stockage - Google Patents

Procédé et appareil de génération d'instantané de transaction de base de données, dispositif, et support de stockage Download PDF

Info

Publication number
WO2023103990A1
WO2023103990A1 PCT/CN2022/136680 CN2022136680W WO2023103990A1 WO 2023103990 A1 WO2023103990 A1 WO 2023103990A1 CN 2022136680 W CN2022136680 W CN 2022136680W WO 2023103990 A1 WO2023103990 A1 WO 2023103990A1
Authority
WO
WIPO (PCT)
Prior art keywords
transaction
snapshot
management component
global
transaction management
Prior art date
Application number
PCT/CN2022/136680
Other languages
English (en)
Chinese (zh)
Inventor
王烨
宦家佳
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2023103990A1 publication Critical patent/WO2023103990A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1458—Management of the backup or restore process
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/11—File system administration, e.g. details of archiving or snapshots
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/11—File system administration, e.g. details of archiving or snapshots
    • G06F16/128—Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/219—Managing data history or versioning

Definitions

  • the embodiments of the present application relate to the field of databases, and in particular to a method, device, device, and storage medium for generating transaction snapshots of databases.
  • multiple global transaction managers are usually deployed to solve the problem of high concurrency pressure on each component or node in a load balancing manner.
  • multiple global transaction managers to share the concurrency pressure of the entire system, it also makes Multiple global transaction managers have difficulty producing globally consistent snapshots.
  • the main purpose of the embodiments of the present application is to provide a database transaction snapshot generation method, device, device and storage medium, so as to achieve accurate global consistent snapshot acquisition.
  • an embodiment of the present application provides a database transaction snapshot generation method, comprising: alternately obtaining the transaction snapshots generated by the first global transaction management component and the second global transaction management component at different times; according to the The transaction number in the active state in the two adjacent transaction snapshots obtained by the first global transaction management component generates a transaction snapshot corresponding to the first global transaction management component; according to the transaction snapshot obtained from the second global transaction management component The transaction number in the active state in two adjacent transaction snapshots generates a transaction snapshot corresponding to the second global transaction management component; according to the transaction snapshot corresponding to the first global transaction management component and the second global transaction The transaction snapshot corresponding to the management component generates a global transaction snapshot.
  • a database transaction snapshot generation device including:
  • An acquisition module configured to alternately acquire transaction snapshots generated by the first global transaction management component and the second global transaction management component at different times;
  • a first generating module configured to generate a transaction snapshot corresponding to the first global transaction management component according to the transaction number in an active state in two consecutive transaction snapshots obtained from the first global transaction management component;
  • the second generation module is configured to generate a transaction snapshot corresponding to the second global transaction management component according to the transaction number in the active state in two adjacent transaction snapshots obtained from the second global transaction management component;
  • the third generating module is configured to generate a global transaction snapshot according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component.
  • an embodiment of the present application also provides an electronic device, including: at least one processor; and a memory connected to the at least one processor in communication; wherein, the memory stores information that can be used by the at least one processor An instruction executed by a processor, the instruction is executed by the at least one processor, so that the at least one processor can execute the above-mentioned method for generating a database transaction snapshot.
  • the embodiment of the present application also proposes a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned method for generating a database transaction snapshot is realized.
  • a transaction snapshot generation method, device, device and storage medium of a database proposed by the present application obtain transaction snapshots of multiple global transaction management components at different times alternately, and generate a global transaction snapshot according to multiple transaction snapshots, so that the final The obtained global transaction snapshot can correct the transaction status that has changed within the generation time difference of multiple transaction snapshots.
  • the whole process can quickly and accurately obtain globally consistent snapshots.
  • Fig. 1a is a schematic diagram of the transaction state in the distributed database system provided by the embodiment of the present application.
  • Fig. 1b is a schematic diagram of a transaction snapshot in a distributed database system provided by an embodiment of the present application
  • Fig. 2 is the flow chart of the transaction snapshot generation method of the database that the embodiment of the present application provides;
  • FIG. 3 is a schematic structural diagram of a database transaction snapshot generation device provided by an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of an electronic device provided in an embodiment of the present application.
  • the database transaction snapshot generation method of the present application is aimed at distributed databases.
  • a distributed database is logically a unified whole, and physically stored on different physical nodes.
  • An application can access databases distributed in different geographical locations through a network connection.
  • Distributed databases usually expand multiple computing nodes and storage nodes to handle massive data tasks.
  • multiple nodes or components processing tasks are prone to high concurrency pressure problems, and multiple global transaction managers are usually deployed. And use load balancing to reduce the concurrency pressure of each node or component. Deploying multiple global transaction managers brings the problem of difficulty in generating globally consistent snapshots.
  • the meaning of a global snapshot is the global state at a certain moment, and consistency means that for any event saved in the global snapshot, the events that occurred before this event should be saved in this snapshot. Simply put, the combination of the states of all transactions at the same time is a globally consistent snapshot.
  • the current global snapshot generation method has the following problems: as shown in Figure 1a, if two global transaction managers gtm1 and gtm2 are deployed in a distributed database system, the first global transaction manager (i.e. the first global transaction management component ) gtm1 is responsible for managing transaction 8, transaction 9, transaction 10 and transaction 11, the second global transaction manager gtm2 is responsible for managing transaction 91, transaction 92, transaction 93 and transaction 94, before T0 time, transaction 8, transaction 10 and transaction 91 Transaction 9, Transaction 11, Transaction 92, Transaction 93, and Transaction 94 are active transactions that have not been completed. If gtm1 receives a snapshot generation request at time T0, the snapshot obtained by gtm1 is shown in Figure 1b.
  • atomicity means that a transaction is an indivisible unit of work, and all operations included in the transaction are either executed or not executed.
  • Consistency means that the transaction must change the database from one consistent state to another consistent state.
  • the data inconsistency problem on the data fragmentation makes the corresponding transaction not meet the transaction attribute requirements.
  • this application alternately obtains the transaction snapshots of multiple global transaction management components at different times, and generates a global transaction snapshot according to the multiple transaction snapshots, so that the finally obtained global transaction snapshot can correct the changes that occur within the generation time difference of multiple transaction snapshots. transaction status.
  • the embodiment of the present application relates to a transaction snapshot generation method of a database, as shown in Figure 2, comprising:
  • Step 201 alternately acquire transaction snapshots generated by the first global transaction management component and the second global transaction management component at different times.
  • step 201 specifically includes: sending a first snapshot generation request to the first global transaction management component, receiving the first transaction snapshot generated at the first moment fed back by the first global transaction management component;
  • the management component sends a second snapshot generation request, receives the second transaction snapshot generated at the second moment fed back by the second global transaction management component; sends the third snapshot generation request to the first global transaction management component, and receives the first global transaction management component Feedback of the third transaction snapshot generated at the third moment; sending a fourth snapshot generation request to the second global transaction management component, and receiving the fourth transaction snapshot generated at the fourth moment fed back by the second global transaction management component.
  • gtm1 generates a transaction snapshot at time T0
  • gtm2 generates a transaction snapshot at time T1
  • gtm1 generates another snapshot at time T2
  • gtm2 generates another snapshot at time T3.
  • alternate Get snapshots of transactions at different times are not limited to two global transaction management components, but can also target multiple global transaction management components.
  • the sending interval between the third snapshot generation request and the second snapshot generation request is greater than or equal to the time required for the second snapshot generation request to be sent to the second global transaction management component; the sending interval between the fourth snapshot generation request and the third snapshot generation request The time is greater than or equal to the time required for the third snapshot generation request to be sent to the first global transaction management component. In this way, it is guaranteed that the global transaction manager receives the next transaction snapshot generation request after obtaining the previous snapshot generation request.
  • the sending interval between the third snapshot generation request and the second snapshot generation request is the same as that between the fourth snapshot generation request and the third snapshot generation request. Snapshot generation requests are sent at equal intervals. In this way, while ensuring accurate acquisition of transaction snapshots, the acquisition efficiency is improved.
  • Step 202 Generate a transaction snapshot corresponding to the first global transaction management component according to an active transaction number in two consecutive transaction snapshots obtained from the first global transaction management component.
  • step 202 specifically includes: performing a logical AND operation on the transaction numbers in the active state in two consecutive transaction snapshots obtained from the first global transaction management component to obtain the transaction number corresponding to the first global transaction management component Transactional snapshots. That is to say, only when the same transaction is in the active state that has not ended in the two snapshots, is it confirmed that the transaction is in the active state that has not ended.
  • Step 203 Generate a transaction snapshot corresponding to the second global transaction management component according to an active transaction number in two consecutive transaction snapshots obtained from the second global transaction management component.
  • step 203 specifically includes: performing a logical AND operation on the transaction numbers in the active state in two consecutive transaction snapshots obtained from the second global transaction management component to obtain the transaction number corresponding to the second global transaction management component Transactional snapshots.
  • Step 204 Generate a global transaction snapshot according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component.
  • step 204 specifically includes: numbering the transaction number in the active state in the transaction snapshot corresponding to the first global transaction management component, and the transaction number in the active state in the transaction snapshot corresponding to the second global transaction management component Merge to get a global transaction snapshot.
  • step 204 it also includes: backing up the data in each data fragment according to the global transaction snapshot.
  • the data in each data fragment can be backed up according to the status of each transaction in the global transaction snapshot.
  • a database transaction snapshot generation method proposed by this application by alternately obtaining transaction snapshots of multiple global transaction management components at different times, generating a global transaction snapshot according to multiple transaction snapshots, so that the final obtained global transaction snapshot can be corrected Transaction states that have changed within the time difference between multiple transaction snapshots.
  • the whole process can quickly and accurately obtain globally consistent snapshots.
  • the embodiment of the present application relates to a database transaction snapshot generation device, as shown in Figure 3, comprising:
  • An acquisition module 301 configured to alternately acquire transaction snapshots generated by the first global transaction management component and the second global transaction management component at different times;
  • the first generation module 302 is configured to generate the transaction snapshot corresponding to the first global transaction management component according to the transaction number in the active state in two adjacent transaction snapshots obtained from the first global transaction management component;
  • the second generation module 303 is configured to generate a transaction snapshot corresponding to the second global transaction management component according to the transaction number in the active state in two consecutive transaction snapshots obtained from the second global transaction management component;
  • the third generating module 304 is configured to generate a global transaction snapshot according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component.
  • This device embodiment is mainly aimed at the description of the database transaction snapshot generation method provided by the method embodiment at the software implementation level, and its implementation also needs to rely on hardware support.
  • the functions of related modules can be deployed on the processor, so that The processor runs to implement corresponding functions, especially, the data generated by the running can be stored in the memory for subsequent inspection and use.
  • modules involved in this embodiment are logical modules, and a logical unit may be a physical unit, or a part of a physical unit, or may be realized by a combination of multiple physical units.
  • a logical unit may be a physical unit, or a part of a physical unit, or may be realized by a combination of multiple physical units.
  • units that are not closely related to solving the technical problem proposed in the present application are not introduced in this embodiment, but this does not mean that there are no other units in this embodiment.
  • This embodiment is a device embodiment corresponding to the embodiment of the database transaction snapshot generation method, and this embodiment can be implemented in cooperation with the above-mentioned embodiments.
  • the relevant technical details mentioned in the foregoing embodiments are still valid in this embodiment, and will not be repeated here in order to reduce repetition.
  • the relevant technical details mentioned in this embodiment can also be applied to the above method embodiments.
  • the embodiment of the present application relates to an electronic device, as shown in FIG. 4 , including: at least one processor 401; Instructions executed by the at least one processor 401, the instructions are executed by the at least one processor 401, so that the at least one processor 401 can execute the data packet capture method in the foregoing implementation manner.
  • the memory and the processor are connected by a bus
  • the bus may include any number of interconnected buses and bridges, and the bus connects one or more processors and various circuits of the memory together.
  • the bus may also connect together various other circuits such as peripherals, voltage regulators, and power management circuits, all of which are well known in the art and therefore will not be further described herein.
  • the bus interface provides an interface between the bus and the transceivers.
  • a transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing means for communicating with various other devices over a transmission medium.
  • the data processed by the processor is transmitted on the wireless medium through the antenna, further, the antenna also receives the data and transmits the data to the processor.
  • the processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. Instead, memory can be used to store data that the processor uses when performing operations.
  • Embodiments of the present application relate to a computer-readable storage medium storing a computer program.
  • the computer program is executed by the processor, the above-mentioned database transaction snapshot generation method is realized.
  • a storage medium includes several instructions to make a device ( It may be a single-chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Quality & Reliability (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

La présente demande porte sur le domaine des bases de données, et concerne un procédé et un appareil de génération d'instantané de transaction de base de données, un dispositif, et un support de stockage. Le procédé de génération d'instantané de transaction de base de données consiste : à acquérir en alternance des instantanés de transaction générés par un premier composant de gestion de transaction globale et un second composant de gestion de transaction globale à différents moments, respectivement ; en fonction d'un numéro de transaction dans un état actif dans deux instantanés de transaction adjacents acquis à partir du premier composant de gestion de transaction globale, à générer un instantané de transaction correspondant au premier composant de gestion de transaction globale ; en fonction d'un numéro de transaction dans l'état actif dans deux instantanés de transaction adjacents acquis à partir du second composant de gestion de transaction globale, à générer un instantané de transaction correspondant au second composant de gestion de transaction globale ; et à générer un instantané de transaction globale en fonction de l'instantané de transaction correspondant au premier composant de gestion de transaction globale et de l'instantané de transaction correspondant au second composant de gestion de transaction globale.
PCT/CN2022/136680 2021-12-10 2022-12-05 Procédé et appareil de génération d'instantané de transaction de base de données, dispositif, et support de stockage WO2023103990A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111509135.7 2021-12-10
CN202111509135.7A CN116303754A (zh) 2021-12-10 2021-12-10 数据库的事务快照生成方法、装置、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2023103990A1 true WO2023103990A1 (fr) 2023-06-15

Family

ID=86729618

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/136680 WO2023103990A1 (fr) 2021-12-10 2022-12-05 Procédé et appareil de génération d'instantané de transaction de base de données, dispositif, et support de stockage

Country Status (2)

Country Link
CN (1) CN116303754A (fr)
WO (1) WO2023103990A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130124475A1 (en) * 2011-11-16 2013-05-16 Sap Ag System and Method of Performing Snapshot Isolation in Distributed Databases
CN104885054A (zh) * 2013-03-13 2015-09-02 华为技术有限公司 大规模并行处理数据库中执行事务的系统和方法
CN105684377A (zh) * 2013-10-31 2016-06-15 华为技术有限公司 一种创建支持mpp数据库中的可重读隔离级别的分布式事务管理器的系统和方法
CN107045454A (zh) * 2016-02-06 2017-08-15 华为技术有限公司 跨进程分布式事务控制方法及相关系统
CN112069258A (zh) * 2020-09-08 2020-12-11 北京金山云网络技术有限公司 事务进度的管理方法、装置及分布式数据库
CN113495872A (zh) * 2020-04-08 2021-10-12 北京万里开源软件有限公司 分布式数据库中的事务处理方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130124475A1 (en) * 2011-11-16 2013-05-16 Sap Ag System and Method of Performing Snapshot Isolation in Distributed Databases
CN104885054A (zh) * 2013-03-13 2015-09-02 华为技术有限公司 大规模并行处理数据库中执行事务的系统和方法
CN105684377A (zh) * 2013-10-31 2016-06-15 华为技术有限公司 一种创建支持mpp数据库中的可重读隔离级别的分布式事务管理器的系统和方法
CN107045454A (zh) * 2016-02-06 2017-08-15 华为技术有限公司 跨进程分布式事务控制方法及相关系统
CN113495872A (zh) * 2020-04-08 2021-10-12 北京万里开源软件有限公司 分布式数据库中的事务处理方法及系统
CN112069258A (zh) * 2020-09-08 2020-12-11 北京金山云网络技术有限公司 事务进度的管理方法、装置及分布式数据库

Also Published As

Publication number Publication date
CN116303754A (zh) 2023-06-23

Similar Documents

Publication Publication Date Title
EP2932370B1 (fr) Système et procédé permettant d'effectuer une transaction dans une base de données de traitement massivement parallèle
EP2928160B1 (fr) Idempotence pour des transactions de base de données
EP1019822B1 (fr) Base de donnees de configuration de grappe a disponibilite elevee
US9923967B2 (en) Storage management system for preserving consistency of remote copy data
Sciascia et al. Scalable deferred update replication
US8924346B2 (en) Idempotence for database transactions
EP2738698B1 (fr) Protocole de verrouillage pour tables distribuées et partitionnées
US11726984B2 (en) Data redistribution method and apparatus, and database cluster
WO2017219857A1 (fr) Procédé et dispositif de traitement de données
EP3958126A1 (fr) Système de base de données basé sur des composants de microservice, et procédé associé
CN111753013A (zh) 分布式事务处理方法及装置
CN104793988A (zh) 跨数据库分布式事务的实现方法和装置
WO2015169067A1 (fr) Procédé, dispositif et système pour une reproduction de données poste à poste, et procédé, dispositif et système pour une commutation de nœud maître
CN108989391B (zh) 一种一致性处理的方法及系统
CN111522631A (zh) 分布式事务处理方法、装置、服务器及介质
US20210034605A1 (en) Transaction processing for a database distributed across availability zones
WO2012083697A1 (fr) Procédé et dispositif de reprise de transaction
US20230110826A1 (en) Log execution method and apparatus, computer device and storage medium
CN115562911B (zh) 虚拟机数据备份方法及装置、系统、电子设备、存储介质
CN110427427B (zh) 一种通过引脚桥接实现全局事务分布式处理方法
US7752225B2 (en) Replication and mapping mechanism for recreating memory durations
CN109408201B (zh) 基于分布式数据库的事务管理方法
WO2023103990A1 (fr) Procédé et appareil de génération d'instantané de transaction de base de données, dispositif, et support de stockage
Pankowski Consistency and availability of Data in replicated NoSQL databases
US10747572B2 (en) Computing system including enhanced application performance based on last completed operation sequence value

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22903412

Country of ref document: EP

Kind code of ref document: A1