WO2023124431A1

WO2023124431A1 - Database processing method and related device

Info

Publication number: WO2023124431A1
Application number: PCT/CN2022/126150
Authority: WO
Inventors: 张子杰; 任阳; 李思昊; 张岩峰; 付国; 周慰星
Original assignee: 华为技术有限公司; 东北大学
Priority date: 2021-12-31
Filing date: 2022-10-19
Publication date: 2023-07-06
Also published as: CN116414844A

Abstract

Disclosed in the embodiments of the present application is a database processing method. The method is applied to a database system. The database system comprises a first node and a second node, wherein the first node and the second node are respectively deployed with a first database and a second database, which store the same data. The method comprises: acquiring a first operation set; sending the first operation set to a second node; receiving a second operation set, which is sent by the second node, wherein the first operation set and the second operation set are used for the second node to update a second database; and updating the first database on the basis of the first operation set and the second operation set. Operation sets of transactions are mutually sent between a plurality of nodes in a database system, such that the transactions, which are submitted by other databases, are taken into consideration during the update of databases on the nodes, thereby ensuring the update synchronization between a plurality of databases.

Description

A database processing method and related equipment

This application claims the priority of the Chinese patent application with the application number 202111679433.0 and the title of the invention "a database processing method and related equipment" submitted to the China Patent Office on December 31, 2021, the entire contents of which are incorporated by reference in this application middle.

technical field

The present application relates to the technical field of databases, in particular to a database processing method and related equipment.

Background technique

With the rapid development of the Internet, the software system has changed from the original single application to distributed application. A distributed system will split an application system into multiple services that can be deployed independently, so remote collaboration between services is required to complete transaction operations. In this distributed system environment, different services collaborate remotely through the network Completing a transaction is called a distributed transaction, such as user registration to send points, create an order to reduce inventory, and bank transfer transactions are all distributed transactions. In a distributed system, each node is physically independent from each other and communicates and coordinates through the network. Due to the existence of the transaction mechanism, it can be guaranteed that the data operation on each independent node can satisfy ACID. However, if you want to keep the data in multiple machines in a distributed deployment consistent, then you must ensure that all data write operations on all nodes are executed, or all of them are executed, or none of them are executed. However, when a machine executes a local transaction, it cannot know the execution result of the local transaction in other machines. So he doesn't know whether this transaction should be committed or roolbacked.

At present, the conventional solution is to introduce a "coordinator" component to uniformly schedule the execution of all distributed nodes, which can also be called a two-phase commit method. The idea of the two-phase commit method can be summarized as follows: the coordinator sends preparation information (prepare) to each participant. Each participant notifies the coordinator of the success or failure of the operation, and enters the state of locking transaction resources to wait for the coordinator to reply. The coordinator decides whether each participant performs a commit operation (commit) or a rollback operation (rollback) based on the feedback information of all participants, and replies to each participant with the decision of the coordinator, and each participant then releases transaction resources in response to the decision, thus Ensure that the databases of all participants are consistent.

However, on the one hand, there are three interactions between the coordinator and each participant, and the communication overhead is large. On the other hand, due to the importance of the coordinator, once the coordinator fails. All participants are still in the state of locking transaction resources, and cannot continue to complete transaction operations.

Contents of the invention

Embodiments of the present application provide a database processing method and related equipment. Multiple nodes in the database system send transactions to each other. The update of the database on the node takes into account the transactions submitted by other databases, so as to ensure the synchronization of updates among multiple databases.

The first aspect of the embodiment of the present application provides a database processing method, the method is applied to a database system, the database system includes a first node and a second node, the first node is deployed with the first database, and the second node is deployed with the second database , the first database and the second database store the same data, and the method may be executed by the first node, or may be executed by components of the first node (such as a processor, a chip, or a chip system, etc.). The method includes: the first node obtains a first operation set, and the first operation set is the write operation or read-write operation of the first transaction set in the first database within the target time period; the first node sends the first operation set to the second node ; The first node receives the second operation set sent by the second node, the second operation set is the write operation or read-write operation of the second transaction set in the second database within the target time period, the first operation set and the second operation set Used for the second node to update the second database; the first node to update the first database based on the first operation set and the second operation set.

In the embodiment of the present application, multiple nodes in the database system send transaction operation sets to each other, and the update of the database on the node takes into account the transactions submitted by other databases, so as to ensure the update synchronization among multiple databases. On the one hand, transaction modification is notified between nodes through the transmission operation set, which does not involve multiple interactions between nodes, and the communication overhead is small. On the other hand, compared with the two-phase commit method, it avoids the inoperability of the transaction caused by the coordinator due to a single point of failure.

Optionally, in a possible implementation manner of the first aspect, the above step: the first node acquires the first operation set includes: the first node responds to and executes the first user's operation instruction on the first database; the first During the process of executing the operation instruction, the node records the operation of the first user on the data items in the first database to obtain the first operation set.

In this possible implementation manner, during the process of executing the user's operation instruction on the database, the node records the first user's operation on the data items in the database to obtain an operation set. Therefore, the user's operations on the database can be restored according to the operation set, thereby providing a basis for subsequent database synchronization.

Optionally, in a possible implementation of the first aspect, the above-mentioned operation instruction is a write instruction, and the first operation set is a write operation set; or, the operation instruction is a read and write instruction, and the first operation set is a read and write A collection of operations or a collection of write operations.

In this possible implementation, the operation set corresponding to the write command is the write operation set, and the operation set corresponding to the read and write commands is the read and write operation set or the write operation set, which can be set according to business needs, and is not limited here. The user's operation on the database can be recorded through the operation set, and the replay of the user's operation can be realized.

Optionally, in a possible implementation manner of the first aspect, the above steps further include: the first node acquires the first information of the first transaction set, each transaction in the first transaction set corresponds to a first information, and the first A message includes a first identifier and/or a first timestamp; the first node sends the first message to the second node; the first node receives the second message sent by the second node, and each transaction in the second transaction set corresponds to a The second information, the second information includes the second identification, and/or the second time stamp; the first information and the second information are used for the second node to update the second database; the first node is based on the first operation set and the second operation set Updating the first database includes: the first node updating the first database based on the first operation set, the first information, the second operation set, and the second information.

In this possible implementation, in addition to the operation set of the interactive transaction, the nodes can also interact with the timestamp or identification of the transaction, so that when the database is updated using the first transaction set and the second transaction set, it can be passed Timestamps or identifiers identify which transactions need to be committed, and the order in which they are committed. Conflicting transactions can be quickly determined through the first information and the second information, thereby increasing the speed of subsequent database updates.

Optionally, in a possible implementation manner of the first aspect, the above step: the first node updates the first database based on the first operation set, the first information, the second operation set, and the second information includes: first The node determines the transaction to be committed in the first transaction set and the second transaction set based on the first operation set, the first information, the second operation set and the second information; the first node submits the transaction to be committed in the first database, and obtains Updated first database.

In this possible implementation, when the node uses the first transaction set and the second transaction set to update the first database, it can use the first timestamp or the first identification to determine the transactions to be submitted and the order of submitting the transactions. Conflicting transactions can be quickly determined through the first information and the second information, thereby increasing the speed of subsequent updating of the first database.

Optionally, in a possible implementation of the first aspect, the first information of the first sub-transaction in the above-mentioned first transaction set is the first timestamp, and the second information of the second sub-transaction in the second transaction set The information is a second timestamp, the first sub-transaction and the second sub-transaction include at least one overlapping data item, the first timestamp is smaller than the second timestamp, and the second sub-transaction is a transaction that cannot be committed. Here, a transaction with a larger timestamp is determined as a transaction that cannot be committed. It is understandable that a transaction with a smaller timestamp can also be determined as a transaction that cannot be committed. The specific rules can be set according to actual needs, and there is no limitation here.

In this possible implementation, by determining the transaction with a larger timestamp as a transaction that cannot be committed, this rule can determine which transactions in the first transaction set and the second transaction set can be committed or cannot be committed, and compare all transactions After the end, you can submit the transaction to be submitted, roll back the transaction that cannot be submitted, and then realize the update of the database.

The second aspect of the embodiment of the present application provides a database processing method. The method is applied to a database system. The database system includes a first node and a second node. The first node is deployed with the first database, and the second node is deployed with the second database. , the first database and the second database store the same data, and the method may be executed by the second node, or may be executed by components of the second node (such as a processor, a chip, or a chip system, etc.). The method includes: the second node obtains a second operation set, and the second operation set is a write operation or a read-write operation of a second transaction set in the second database within a target time period; the second node sends the second operation set to the first node ; The second node receives the first operation set sent by the first node, the first operation set is the write operation or read-write operation of the first transaction set in the first database within the target time period, the first operation set and the second operation set The first node updates the second database; the second node updates the second database based on the first operation set and the second operation set.

Optionally, in a possible implementation of the second aspect, the above step: the second node acquires the second operation set includes: the second node responds to and executes the second user's operation instruction on the second database; the second During the process of executing the operation instruction, the node records the second user's operation on the data item in the second database, and obtains the second operation set.

Optionally, in a possible implementation of the second aspect, the above-mentioned operation instruction is a write instruction, and the second operation set is a write operation set; or, the operation instruction is a read and write instruction, and the second operation set is a read and write operation A collection of operations or a collection of write operations.

Optionally, in a possible implementation manner of the second aspect, the above steps further include: the second node acquires second information of a second transaction set, each transaction in the second transaction set corresponds to a second information, and the second The second information includes the second identification, and/or the second timestamp; the second node sends the second information to the first node; the second node receives the first information sent by the first node, and each transaction in the first transaction set corresponds to a The first information, the first information includes the first identification, and/or the first time stamp; the first information and the second information are used for the first node to update the second database; the second node is based on the first operation set and the second operation set Updating the second database includes: updating the second database by the second node based on the first operation set, the first information, the second operation set, and the second information.

Optionally, in a possible implementation manner of the second aspect, the above step: the second node updates the second database based on the first operation set, the first information, the second operation set, and the second information includes: the second The node determines the transaction to be committed in the first transaction set and the second transaction set based on the first operation set, the first information, the second operation set and the second information; the second node submits the transaction to be committed in the second database, and obtains Updated second database.

Optionally, in a possible implementation of the second aspect, the first information of the first sub-transaction in the above-mentioned first transaction set is the first timestamp, and the second information of the second sub-transaction in the second transaction set The information is a second timestamp, the first sub-transaction and the second sub-transaction include at least one overlapping data item, the first timestamp is smaller than the second timestamp, and the second sub-transaction is a transaction that cannot be committed.

The third aspect of the embodiment of the present application provides a first node, the first node is applied to a database system, the database system further includes a second node, the first node is deployed with a first database, and the second node is deployed with a second database, The first database and the second database store the same data, and the first node includes: an acquisition unit configured to acquire a first operation set, where the first operation set is a write operation of a first transaction set in the first database within a target time period Or read and write operations; the sending unit is used to send the first operation set to the second node; the receiving unit is used to receive the second operation set sent by the second node, and the second operation set is in the second database within the target time period The write operation or read-write operation of the second transaction set, the first operation set and the second operation set are used for the second node to update the second database; the update unit is used for updating the first database based on the first operation set and the second operation set .

Optionally, in a possible implementation manner of the third aspect, the above-mentioned obtaining unit is specifically configured to respond to and execute an operation instruction of the first user on the first database; the obtaining unit is specifically configured to During the process, the first user's operations on the data items in the first database are recorded to obtain a first operation set.

Optionally, in a possible implementation of the third aspect, the above-mentioned operation instruction is a write instruction, and the first operation set is a write operation set; or, the operation instruction is a read and write instruction, and the first operation set is a read and write operation set A collection of operations or a collection of write operations.

Optionally, in a possible implementation manner of the third aspect, the above-mentioned acquisition unit is further configured to acquire first information of a first transaction set, each transaction in the first transaction set corresponds to a first information, and the first A piece of information includes a first identifier and/or a first time stamp; a sending unit is also used to send the first information to the second node; a receiving unit is also used to receive the second information sent by the second node, and the second transaction set Each transaction corresponds to a second information, and the second information includes a second identifier and/or a second timestamp; the first information and the second information are used for the second node to update the second database; the update unit is specifically used for The first set of operations, the first information, the second set of operations, and the second information update the first database.

Optionally, in a possible implementation manner of the third aspect, the above update unit is specifically configured to determine the first transaction set and the second transaction set based on the first operation set, the first information, the second operation set, and the second information. Two transactions to be submitted in the transaction set; an updating unit, specifically configured to submit the transactions to be submitted in the first database to obtain an updated first database.

Optionally, in a possible implementation of the third aspect, the first information of the first sub-transaction in the above-mentioned first transaction set is the first timestamp, and the second information of the second sub-transaction in the second transaction set The information is a second timestamp, the first sub-transaction and the second sub-transaction include at least one overlapping data item, the first timestamp is smaller than the second timestamp, and the second sub-transaction is a transaction that cannot be committed.

The fourth aspect of the embodiment of the present application provides a second node, the second node is applied to a database system, the database system further includes a first node, the first node is deployed with a first database, and the second node is deployed with a second database, The first database and the second database store the same data, and the second node includes: an acquisition unit, configured to acquire a second operation set, where the second operation set is a write operation of a second transaction set in the second database within a target time period Or read and write operations; the sending unit is used to send the second operation set to the first node; the receiving unit is used to receive the first operation set sent by the first node, the first operation set is in the first database within the target time period The write operation or read-write operation of the first transaction set, the first operation set and the second operation set are used for the first node to update the first database; the update unit is used for updating the second database based on the first operation set and the second operation set .

Optionally, in a possible implementation manner of the fourth aspect, the above-mentioned acquiring unit is specifically configured to respond to and execute an operation instruction of the second user on the second database; the acquiring unit is specifically configured to During the process, the second user's operations on the data items in the second database are recorded to obtain a second operation set.

Optionally, in a possible implementation of the fourth aspect, the above-mentioned operation instruction is a write instruction, and the second operation set is a write operation set; or, the operation instruction is a read and write instruction, and the second operation set is a read and write operation A collection of operations or a collection of write operations.

Optionally, in a possible implementation manner of the fourth aspect, the above-mentioned acquiring unit is further configured to acquire second information of a second transaction set, each transaction in the second transaction set corresponds to a second information, and the first The second information includes the second identification, and/or the second timestamp; the sending unit is also used to send the second information to the first node; the receiving unit is also used to receive the first information sent by the first node, the first transaction set Each transaction corresponds to a first information, the first information includes the first identification, and/or the first timestamp; the first information and the second information are used for the first node to update the first database; the update unit is specifically used for The first set of operations, the first information, the second set of operations, and the second information update the second database.

Optionally, in a possible implementation manner of the fourth aspect, the above update unit is specifically configured to determine the first transaction set and the second transaction set based on the first operation set, the first information, the second operation set, and the second information. Transactions to be submitted in the transaction set; an updating unit, specifically configured to submit the transactions to be submitted in the second database to obtain an updated second database.

Optionally, in a possible implementation of the fourth aspect, the first information of the first sub-transaction in the above-mentioned first transaction set is the first timestamp, and the second information of the second sub-transaction in the second transaction set The information is a second timestamp, the first sub-transaction and the second sub-transaction include at least one overlapping data item, the first timestamp is smaller than the second timestamp, and the second sub-transaction is a transaction that cannot be committed.

A fifth aspect of the embodiments of the present application provides a first node, where the first node is configured to execute the method in the foregoing first aspect or any possible implementation manner of the first aspect.

A sixth aspect of the embodiments of the present application provides a second node, where the second node is configured to execute the method in the foregoing second aspect or any possible implementation manner of the second aspect.

The seventh aspect of the embodiment of the present application provides a database system, including the first node of the aforementioned third aspect or the fifth aspect, and/or the second node of the aforementioned fourth aspect or the sixth aspect.

The eighth aspect of the embodiment of the present application provides a chip, the chip includes a processor and a data interface, wherein the processor reads the instructions stored in the memory through the data interface to execute any one of the first aspect or the first aspect. or to execute the method in the second aspect or any possible implementation manner of the second aspect. In the specific implementation process, the chip can be based on a central processing unit (CPU), a microcontroller (micro controller unit, MCU), a microprocessor (micro processing unit, MPU), a digital signal processor (digital signal processing, DSP), system on chip (SoC), application-specific integrated circuit (ASIC), field programmable gate array (field programmable gate array, FPGA) or programmable logic device (programmable logic device , PLD) in the form of realization.

The ninth aspect of the embodiment of the present application provides a computer-readable medium, the computer-readable medium stores program codes, and when the computer program codes run on the computer, the computer executes the above-mentioned first aspect or any of the first aspects. A possible execution method; causing a computer to execute the above second aspect or any possible execution method of the second aspect. These computer-readable storages include, but are not limited to, one or more of the following: read-only memory (read-only memory, ROM), programmable ROM (programmable ROM, PROM), erasable PROM (erasable PROM, EPROM), Flash memory, electrical EPROM (electrically EPROM, EEPROM) and hard drive (hard drive).

The tenth aspect of the embodiments of the present application provides a computer program product. When the computer program product is executed on a computer, it causes the computer to execute the method in the aforementioned first aspect or any possible implementation of the first aspect; or causes the computer to execute The aforementioned second aspect or the method in any possible implementation of the second aspect.

Wherein, the technical effects brought about by the second aspect to the tenth aspect or any one of the possible implementations can refer to the first aspect or the technical effects brought about by different possible implementations of the first aspect, and will not be repeated here.

It can be seen from the above technical solutions that the embodiment of the present application has the following advantages: the first node obtains the first operation set related to the first transaction set in the first database, and receives the second transaction in the second database sent by the second node. Collect the related second operation set, and update the first database based on the first operation set and the second operation set, so that the update of the first database takes into account the transactions submitted by other databases, thereby ensuring the update synchronization between multiple databases. On the one hand, transaction modification is notified between nodes through the transmission operation set, which does not involve multiple interactions between nodes, and the communication overhead is small. On the other hand, compared with the two-phase commit method, it avoids the inoperability of the transaction caused by the coordinator due to a single point of failure.

Description of drawings

FIG. 1 is a schematic diagram of a database system architecture provided by an embodiment of the present invention;

FIG. 2 is a schematic flow diagram of a database processing method provided in an embodiment of the present application;

FIG. 3 is an example diagram of the first database and the second database provided by the embodiment of the present application;

FIG. 4 is a schematic flow diagram of updating the first database and the second database provided by the embodiment of the present application;

FIG. 5 is a schematic structural diagram of the first node provided by the embodiment of the present application;

FIG. 6 is a schematic structural diagram of a second node provided by an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a communication device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

In addition, in the embodiments of the present application, words such as "exemplary" and "for example" are used as examples, illustrations or explanations. Any embodiment or design described herein as "example" is not to be construed as preferred or advantageous over other embodiments or designs. Rather, the use of the word example is intended to present concepts in a concrete manner.

In this application, "at least one" means one or more, and "multiple" means two or more. "And/or" describes the association relationship of associated objects, indicating that there can be three types of relationships, for example, A and/or B, which can mean: including the existence of A alone, the existence of A and B, and the existence of B alone, where A , B can be singular or plural. The character "/" generally indicates that the contextual objects are an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one item (piece) of a, b, or c can represent: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, c can be single or multiple .

For ease of understanding, the relevant terms and concepts mainly involved in the embodiments of the present application are firstly introduced below.

1. Database

A database, which can also be called a data management system, is an electronic filing system that stores data in a structured manner. The main storage structure in a database is a table.

2. Database system

The database system can be understood as a distributed database system, and the database system includes multiple nodes, and databases are respectively deployed on the multiple nodes. In one case, some of the multiple nodes can process read and write requests, and the other part of the nodes can only process read requests. In another case, multiple nodes can handle write requests, and the overall resource utilization in this case is high. Wherein, a node among multiple nodes refers to a computer or a software instance in a network involving multiple computers or a distributed system.

3. Database management system (DBMS)

The database management system is a large-scale computer software management system designed for object database management. The database processing method provided in the embodiment of the present application can be applied to MySQL, Oracle, open Gauss (openGauss), etc., and is not specifically limited here.

4. Database transactions (or transactions for short)

A transaction is a logical unit in the execution process of a database management system, consisting of a limited sequence of database operations. An example is a transfer from a bank: a complete transaction involves debiting an amount from one account and adding an amount to another.

After the transaction is successfully executed, a "commit" is required. After the commit is successful, the modification will take effect and cannot be lost. If the transaction execution fails (abort), it needs to be "rolled back" (rollback), and its modification will not take effect.

Transactions have the following four characteristics:

Atomicity: All operations in a transaction are indivisible as a whole, either all succeed or all fail.

Consistency: The execution result of the transaction must bring the database from one consistent state to another consistent state. The consistent state refers to: 1. The state of the system satisfies the data integrity constraints (master code, referential integrity, check constraints, etc.). 2. The state of the system reflects the real state of the real world that the database should describe. For example, the sum of the two accounts before and after the transfer should remain unchanged.

Isolation: Concurrently executing transactions do not affect each other and have the same impact on the database as they do serially. For example, if multiple users transfer funds to an account at the same time, the result of the final account should be the same as the result of their transfers in sequence.

Durability: Once a transaction is committed, its updates to the database are durable. No transaction or system failure will result in data loss.

5. Transaction collection

The transaction set mentioned in the embodiment of the present application includes one transaction or multiple transactions, and one transaction includes one operation or multiple operations. Furthermore, a transaction corresponds to a timestamp or an identifier.

6. Optimistic concurrency control (OCC)

OCC is a concurrency control method applied to database management systems. OCC assumes that multiple transactions can be completed in parallel without interfering with each other. At runtime, transactions use data resources without acquiring locks on those resources. Before committing, each transaction verifies that no other transaction has modified the data it has read or modified. If the check reveals conflicting modifications, the transaction will need to be rolled back.

7. Two-phase submission

Two-phase commit (two-phase commit) refers to an algorithm designed in the field of computer networks and databases to make all nodes based on the distributed system architecture maintain consistency when committing transactions. Often, two-phase commit is also referred to as a protocol. In a distributed system, although each node can know the success or failure of its own operation, it cannot know the success or failure of other nodes' operations. When a transaction spans multiple nodes, in order to maintain the four characteristics of the transaction, it is necessary to introduce a component as a coordinator to uniformly control the operation results of all nodes (called participants) and finally instruct these nodes whether to perform the operation results The actual commit (such as writing the updated data to disk, etc.). Therefore, the algorithm idea of the two-phase commit can be summarized as: the coordinator sends preparation information (prepare) to each participant. Each participant notifies the coordinator of the success or failure of the operation, and enters the state of locking transaction resources to wait for the coordinator to reply. The coordinator decides whether each participant performs a commit operation (commit) or a rollback operation (rollback) based on the feedback information of all participants, and replies to each participant with the decision of the coordinator, and each participant then releases transaction resources in response to the decision, thus Ensure that the databases of all participants are consistent.

In order to solve the above problems, the embodiment of the present application provides a database processing method, through the operation set of sending transactions between multiple nodes in the database system, so that the update of the database on the node takes into account the transactions submitted by other databases, thereby ensuring Synchronization of updates between multiple databases.

Figure 1 shows a schematic diagram of a database system. The database system may include: A node 101 , B node 102 , C node 103 and D node 104 . Wherein, A node 101 is deployed with an A database, and a user of the A database is an A user. The B node 102 is deployed with a B database, and a user of the B database is a B user. The C database is deployed on the C node 103, and the user of the C database is a C user. The D node 104 is deployed with a D database, and the user of the D database is a D user. Wherein, each database stores the same data. The A node 101 , the B node 102 , the C node 103 and the D node 104 communicate with each other.

The above-mentioned nodes can transmit the operation sets of their own nodes (also called local operation sets) to each other, and update the database of their own nodes according to the operation sets of transaction sets in other databases and the operation sets of local transaction sets received (also can be called the local database).

The operation set in the embodiment of the present application refers to the data items modified by the node during the process of responding to user operation instructions (such as write instructions, read and write instructions, etc.), or the modified data items and the read data items. The set of operations may also be referred to as a write set or a read-write set. The receiving node can update the local database through the operation set sent by the sending node, so that the updated local database is consistent with the data stored in the database of the sending node, thereby ensuring the synchronization between the databases.

In a possible implementation manner, each of the above nodes can provide read and write services for users. Of course, there may also be nodes that only provide read services to users or forwarding nodes among the above nodes, or it can be understood that the method provided in this embodiment of the application can be applied to multi-master replication scenarios or master-slave replication scenarios. Do limited.

Optionally, in order to ensure the synchronization of the databases, each node, after sending the operation set to other nodes, receives a message of successful reception sent by other nodes, and then uses the local operation set and the operation set of other nodes to update the database. In addition, it should be noted that, in the scenario where multiple nodes interact with each other to operate a set, the four nodes shown in FIG. 1 are taken as an example. If node A sends A operation set to node B, node C and node D respectively, but only receives the successful reception message sent by node B and node C, but does not receive the successful reception message sent by node D. Node A can also use the local operation set and the operation set of other nodes to update the A database. In other words, most of the nodes in the database system have successfully received the operation set and updated the local database, which can ensure that the databases of most of the nodes in the database system are synchronized. Of course, the best situation is: the sending node updates the local database after receiving the successful reception message sent by the receiving node, so as to ensure that the data stored in the databases of each node in the database system is consistent.

Taking user A as an example, generally speaking, user A sends read and write requests to the nearest node (such as node A) for processing. In the case of write requests, wait for node A to complete the processing and synchronize to other nodes, and combine the operation sets of other nodes (such as transaction read and write sets) for conflict detection and conflict processing, and then return the results to the client (you can also It is understood that the result is returned after the database synchronization is successful).

Exemplarily, the interaction between node A and node B among the above-mentioned nodes is described as an example. Node A can receive user A's operation instruction for A database, and node A records user A's read in the process of executing the operation instruction. Get or modify the data item to get the A operation set of A node. After the A node obtains the A operation set, it can send the A operation set to the B node. The A node can also receive the B operation set sent by the B node (that is, the data items read and modified by the B node during the execution of the B user's operation instruction on the B database). Further, the A node can update the A database based on the A operation set and the B operation set. Node B can update the B database based on the A operation set and the B operation set. And then realize the consistency between A database and B database. Of course, an interaction cycle can also be set, that is, in each interaction cycle, the two nodes will exchange their own data sets, thereby realizing the synchronization between the A database and the B database.

The operation set in the embodiment of the present application refers to the integration of the user's operations in the process of operating the database. For example: the operation set is a read-write set or a write set, etc., which are not limited here.

In addition, any two users among the A user, the B user, the C user, and the D user in the embodiment of the present application may be the same user or different users, which are not specifically limited here.

In the embodiment of the present application, only four nodes and four users are taken as an example for schematic illustration. In practical applications, the database system in the embodiment of the present application may include more or fewer nodes and users, and the embodiment of the present application does not limit the number of nodes and users.

A node in this embodiment of the present application refers to a computer or a software instance in a network involving multiple computers or a distributed system. For example: the node can be a server or a terminal device. The terminal device is a device with wireless/wired transceiver functions, which can be deployed on land, including indoor or outdoor, handheld, wearable or vehicle-mounted; it can also be deployed on water ( Such as ships, etc.); can also be deployed in the air (such as aircraft, balloons and satellites, etc.). The terminal device can be a mobile phone, a tablet computer (pad), a computer with a wireless transceiver function, a virtual reality (virtual reality, VR) terminal device, an augmented reality (augmented reality, AR) terminal device, an industrial control (industrial control), wireless terminals in vehicle, wireless terminals in self driving, wireless terminals in remote medical, wireless terminals in smart grid, transportation safety ), wireless terminals in a smart city, wireless terminals in a smart home, wearable terminal devices, etc. The embodiments of the present application do not limit the application scenarios. Terminal equipment may sometimes be called terminal, user equipment (UE), access terminal equipment, vehicle-mounted terminal, industrial control terminal, UE unit, UE station, mobile station, mobile station, remote station, remote terminal equipment, mobile device, UE proxy or UE device, etc. Terminal equipment can also be fixed or mobile. In addition, the terminal device may also be a chip system for implementing UE functions.

The database processing method in the embodiment of the present application will be described below in conjunction with the interaction flow between any two nodes in the database system shown in FIG. 1 . Referring to FIG. 2 , an embodiment of the database processing method provided by the embodiment of the present application includes steps 201 to 206 . Wherein, the first node and the second node may be any two nodes in the aforementioned FIG. 1 , which are not specifically limited here. In addition, the method may be executed by a communication device (such as a server or a terminal device), or may be executed by components of the communication device (such as a processor, a chip, or a chip system, etc.). It can be understood that the communication device may be a chip or a unit with a sending and receiving function or a physical device. In addition, the database processing method provided in the embodiment of the present application may be applied to MySQL, Oracle, openGauss (openGauss), etc., which are not specifically limited here.

Step 201, the first node acquires a first operation set.

The first node and the second node in the embodiment of the present application can be any two nodes in the database system, the first node is deployed with the first database, the second node is deployed with the second database, and the first database and the second database Store the same data.

In the embodiment of the present application, an operation set may be used to record the execution process of the database transaction. No matter how complicated the execution process of a database transaction is, the execution process of the transaction can be replayed by using the operation set, and the same execution result can be produced. Furthermore, multiple nodes can realize database synchronization by sending local operation sets to each other.

The operation set (including the first operation set and the second operation set) in the embodiment of the present application may include one or more transactions, and one transaction may correspond to one or more operations (eg, read, write, etc.). Wherein, the write operation may include adding, deleting, modifying and other operations.

In addition, it should be noted that the first transaction set may be committable transactions in the first database (it can also be understood that when the number of transactions in the first transaction set is multiple, there is no conflict between multiple transactions), The second transaction set may be committable transactions in the second database (it can also be understood that when the number of transactions in the second transaction set is multiple, the multiple transactions do not conflict). Alternatively, the first transaction set is all transactions in the first database, and the second transaction set is all transactions in the second database.

In the embodiment of the present application, the way for the first node to obtain the first operation set is only based on the operation instructions of the first user. It can be understood that other devices can also send the first operation set to obtain the first operation set. Specifically, There is no limit. Wherein, the first operation set is the write operation or read-write operation of the first transaction set in the first database within the target time period. In addition, the number of transactions in the first transaction set may be one or more, and the number of write operations or read-write operations in the first operation set is not limited. The target time period may be set according to actual needs, which is not specifically limited here.

The aforementioned target time period may refer to a time period. The target time period in this case can be understood as the time period between the time of the last synchronization and the target time. Optionally, the first node may obtain the first operation set in each preset period, and periodically send the operation set to the second node. The number of cycles is not limited here. For example, if one period is 10 milliseconds, the target time period may be 10 milliseconds, that is, the first node will obtain the first operation set every 10 milliseconds, and send the first operation set to the second node once.

It can be understood that the target time period may also refer to the time period from the creation time of the first database to the target time, that is, the first operation set is the write operation or the first transaction set in the first database before the target time. read and write operations. In this case, it can be understood that, in order to ensure data synchronization between the first database of the first node and the database of the second node at the target time, all the first operation sets before the target time may be sent at one time.

Optionally, the first database may provide write services or read-write services for the first user, that is, the first user may perform a series of operations on entries in the first database in the first node, for example: write (for example: add, delete, modify), read (e.g. query), etc. During the process of executing the series of operations, the first node records the operations of the first user on the data items in the first database to obtain a first operation set.

Exemplarily, the first node receives the first user's read and write instructions for the first database, the first node responds to the read and write instructions, performs read and write operations on entries in the first database, and executes the read and write operations During the process, the data items read and modified by the first user are recorded to obtain the first read-write set. That is, in this example, the first operation set is the read and write operation set or the write operation set of the first transaction set, which is set according to actual needs and is not limited here.

Exemplarily, the first node receives a first user's write instruction to the first database, and the first node responds to the write instruction to perform a write operation on an entry in the first database, and during the process of performing the write operation, records The data item modified by the first user obtains the first write set. That is, in this example, the first operation set is the write operation set of the first transaction set.

Exemplarily, in order to better understand the subsequent synchronization process of the first database and the second database, the data shown in Table 1 in FIG. 3 is stored in the first database and the second database as an example for exemplary description below. The first transaction set executed by the first user in the first database includes: transaction 1 and transaction 2, transaction 1: delete "Sun Wu". Transaction 2: Change the "delivery address" of "Li Liu" to "Room 801, Hotel F, E Street, Nanshan District, Shenzhen, Guangdong Province". Then the first operation set of the first transaction set includes: delete operation and modification operation.

For the convenience of subsequent description, at least one item of the first time stamp, the first identifier, or the first cycle identifier of each transaction in the first transaction set mentioned later may be referred to as first information. Wherein, a transaction in the first transaction set corresponds to a first timestamp or a first identifier. That is, the first timestamp and the first identifier can be used to locate a unique transaction in the first transaction set. Further, the first time stamp or the first identifier may also be used to sort the multiple transactions in the first transaction set, so as to determine the transaction to be committed or the order in which multiple transactions are committed.

Optionally, in the process of obtaining the first operation set of the first transaction set, the first node may also record the end time (which may be called the first timestamp) or the first identifier of the last operation of each transaction, which The first timestamp can be the local timestamp of the first node at the end of the last operation, it can also be the logical time in the database system, or it can be the timestamp assigned by a specific central node, etc., which will not be done here limited. The first identifier may be a globally unique auto-incremented transaction identifier of the database system.

Further, if multiple nodes in the database system periodically send local operation sets to each other, the first node may also obtain the period identifier of each transaction. In the process of updating the database, if there are operation sets that cross cycles in multiple operation sets, for example, one of the multiple operation sets is in the first cycle and the other is in the second cycle, then you can use The cycle number determines the order of the transactions corresponding to the commit operation set, thereby ensuring the synchronization between multiple databases. Certainly, if there are operation sets across periods in the multiple operation sets, there are also operation sets with the same period. In the process of updating the database, the order of the transactions corresponding to the small-scale operation set can be determined first according to the period identifier, and then the order of submitting the transactions can be determined according to the timestamp or the identifier.

Exemplarily, if it is applied to an openGauss scenario, and the four nodes shown in FIG. 1 are located in different areas, all transactions submitted by users in the area where each node is located can be processed by the node. Specifically, after the first database connector receives the user's operation request, and after the user's operation request passes the authentication, it transmits the structured query language (structured query language, SQL) statement of the operation request to the SQL parser, Components such as the optimizer perform lexical analysis, syntax analysis, and query optimization on SQL statements. First identify the keywords and identifiers supported by the system from the query statement, then define the grammar rules according to the SQL language standard, and use the words generated in the lexical analysis to match the grammar rules. If an SQL statement can match a grammar rule, then Generate the corresponding abstract syntax tree. Finally, the query optimizer optimizes SQL through query rewriting, uses existing statement features and relational algebra operations to generate more efficient equivalent statements, and generates the final query plan based on cost estimation. There is a preset period in advance, and the first node can perform conflict detection and processing (it can also be understood as updating the database) after collecting all the operations of all nodes in the database system in the current period in each period. For example, the first information includes a first period identifier and a first time stamp. And the preset cycle is 50 milliseconds (ms), that is, a new synchronization cycle is generated every 50 ms. It should be noted that the successful execution of the transaction by the first node cannot be written into the first database immediately, and the transaction corresponding to the operation set sent by other nodes in the current cycle needs to be merged and verified, and the database should be synchronized. When two transactions in the same cycle conflict, concurrency control is performed according to the principle of "write first wins". Specifically, when two transactions try to modify the same data item, the transaction to be committed and the transaction to be rolled back are determined according to the timestamps of the two transactions (relevant descriptions are provided in subsequent step 205 and will not be expanded here). It can be understood that the operation of the subsequent second node is similar to that of the first node, and will not be repeated hereafter.

Step 202, the first node sends the first operation set to the second node. Correspondingly, the second node receives the first operation set sent by the first node.

After the first node acquires the first operation set, it may send the first operation set to the second node. Correspondingly, the second node receives the first operation set sent by the first node.

It can be understood that the number of times the first node sends the first operation set to the second node may be one or more times, that is, it may be sent in full or in batches, which is not limited here.

Optionally, if in the foregoing step 201, the first node also obtains at least one of the first timestamp, the first identifier, or the first period identifier, the first node may also send the first timestamp, At least one of the first identifier or the first periodic identifier. Correspondingly, the second node receives at least one of the first timestamp, the first identifier, or the first period identifier sent by the first node.

Exemplarily, continuing the above example applied to the openGauss scenario, when the end of a cycle is reached, the first node will send the first set of operations to other nodes in the database system, and eventually each node will have all transactions in the cycle The collection is operated, and concurrency control can be performed according to the same rules (for example: "write first wins"), so as to ensure that the data of all nodes is consistent. It can be understood that the operation of the subsequent second node is similar to that of the first node, and will not be repeated hereafter.

Step 203, the second node acquires a second operation set.

In this embodiment of the application, the way the second node acquires the second operation set is similar to the way the first node acquires the first operation set. It can be obtained by means of operating a set, etc., and the details are not limited here. Wherein, the second operation set is the write operation or read-write operation of the second transaction set in the second database within the target time period. In addition, the number of transactions in the second transaction set may be one or more, and the number of write operations or read-write operations in the second operation set is not limited. The target time period may be set according to actual needs, which is not specifically limited here.

The target time period in this step is similar to the description of the target time period in step 201 above. Optionally, the second node may obtain the second operation set in each preset period, and periodically send the operation set to the first node. The number of cycles is not limited here. For example, if one period is 10 milliseconds, the target time period may be 10 milliseconds, that is, the second node will obtain the second operation set every 10 milliseconds, and send the second operation set to the first node once. Wherein, for the target time period, reference may be made to the previous description, which will not be repeated here.

Optionally, the second database is similar to the first database, and can provide write services or read-write services for the second user, that is, the second user can perform a series of operations on the entries of the second database in the second node, for example: Add, delete, modify, query, etc. During the process of executing the series of operations, the second node records the second user's operations on the data items in the second database to obtain a second operation set.

Exemplarily, the second node receives the second user's read and write instructions for the second database, and the second node responds to the read and write instructions, performs read and write operations on entries in the second database, and executes the read and write operations During the process, the data items read and modified by the second user are recorded to obtain the second read-write set. That is, in this example, the second operation set is a set of read and write operations or a set of write operations of the second transaction set, which is specifically set according to actual needs and is not limited here.

Exemplarily, the second node receives a second user's write instruction for the second database, and the second node responds to the write instruction to perform a write operation on the entry in the second database, and during the process of performing the write operation, records The data item modified by the second user obtains the second write set. That is, in this example, the second operation set is a set of write operations of the second transaction set.

Exemplarily, continuing the example of Table 1 in FIG. 3 for the second database, it is assumed that the second transaction set executed by the second user in the second database includes: transaction 3 . Transaction 3: Add "Sunday", "15789012345", "No. W Primary School, Luohu District, Shenzhen City, Guangdong Province". Then the second operation set includes: adding operation.

Similar to the foregoing first information, for the convenience of subsequent descriptions, at least one of the second timestamp, the second identifier, or the second period identifier mentioned later may be referred to as second information. Wherein, one transaction in the second transaction set corresponds to a second timestamp or a second identifier. That is, the second timestamp and the second identifier can be used to locate a unique transaction. Further, the second time stamp or the second identifier can also be used to sort multiple transactions, so as to determine the transaction to be committed or the order in which multiple transactions are committed.

Optionally, during the process of obtaining the second operation set of the second transaction set, the second node may also record the end time (which may be called the second timestamp) or the second identifier of the last operation of each transaction, which The second timestamp can be the local timestamp of the second node at the end of the last operation, it can also be the logical time in the database system, or it can be the timestamp assigned by a specific central node, etc., which will not be done here. limited. The second identifier may be a globally unique auto-incremented transaction identifier of the database system.

Further, if multiple nodes in the database system periodically send local operation sets to each other, the second node may also obtain the period identifier of the transaction. In the process of updating the database, if there are operation sets that cross cycles in multiple operation sets, for example, one of the multiple operation sets is in the first cycle and the other is in the second cycle, then you can use The cycle number determines the order of the transactions corresponding to the commit operation set, thereby ensuring the synchronization between multiple databases. Certainly, if there are operation sets across periods in the multiple operation sets, there are also operation sets with the same period. In the process of updating the database, the order of the transactions corresponding to the small-scale operation set can be determined first according to the period identifier, and then the order of submitting the transactions can be determined according to the timestamp or the identifier of the transaction.

Step 204, the second node sends the second operation set to the first node. Correspondingly, the first node receives the second operation set sent by the second node.

After the second node obtains the second operation set, it may send the second operation set to the first node. Correspondingly, the first node receives the second operation set sent by the second node.

Optionally, if in the foregoing step 203, the second node also obtains at least one of the second timestamp, the second identifier, or the second periodic identifier, the second node may also send the second timestamp, At least one of the second identification or the second periodic identification. Correspondingly, the first node receives at least one of the second timestamp, the second identifier, or the second period identifier sent by the second node.

Step 205, the first node updates the first database based on the first operation set and the second operation set.

After the first node acquires the first operation set and the second operation set, the first node may update the first database based on the first operation set and the second operation set. Specifically, the first node may first determine the transaction to be submitted in the first database based on the first operation set and the second operation set, and submit the transaction to be submitted in the first database. In other words, the first node can determine the operations performed in the second database in the second node based on the second operation set, and then can execute the operations in the second operation set, so that the update of the first database has considered the modification of the second database , so as to ensure data synchronization between databases.

In a possible implementation manner, if the first transaction set is committable transactions in the first database, the second transaction set is committable transactions in the second database. Then when the first node updates the first database, it only needs to determine whether there is a transaction conflict between the first transaction set and the second transaction set, because between the transactions in the first transaction set or between the transactions in the second transaction set Whether there is a conflict between them has been judged locally at the respective nodes. The determination of whether transactions conflict in the embodiment of the present application can be understood as determining whether each transaction is a transaction to be committed or a transaction to be rolled back.

In another possible implementation manner, if the first transaction set is the transactions owned by the first database, the second transaction set is the transactions owned by the second database. Then, when the first node subsequently updates the first database, in addition to determining whether there is a transaction conflict between the first transaction set and the second transaction set, it is also necessary to determine whether there is a conflict between multiple transactions in the first transaction set, and whether the second transaction Whether there are conflicts between multiple transactions in the collection.

Optionally, the first node may merge the first operation set with the second operation set, and determine which operations in the first operation set conflict with the second operation set, determine the transaction to be committed and the transaction to be rolled back, And then promote the consistency point of the database.

Optionally, the first node obtains at least one of the first time stamp, the first identifier, or the first cycle identifier (or called first information) of each transaction in the first transaction set, and receives the information sent by the second node. At least one of the second timestamp, the second identifier, or the second periodic identifier (or referred to as second information) of the. The first node may determine the transaction to be committed in the first transaction set and the second transaction set based on the first operation set, the first information, the second operation set, and the second information, and submit the transaction to be committed in the first database , to get the updated first database.

Further, if there is no conflict between the transactions in the first transaction set and the second transaction set, then the transactions in the first transaction set and the second transaction set can be used as transactions to be committed. If there is a conflict between the transactions in the first transaction set and the second transaction set (these conflicting transactions can be called conflicting transactions), the transaction to be committed and the transaction to be rolled back in the conflicting transaction can be determined according to preset rules , and commit the transaction to be committed, and roll back the transaction to be rolled back. The preset rule can be set according to actual needs, which is not specifically limited here. For example: the default rule is that the transaction with the smaller timestamp in the conflicting transactions is the transaction to be committed, or the transaction with the larger timestamp is the transaction to be committed.

It can be understood that, for the case where the first transaction set includes a plurality of first sub-transactions and the second transaction set includes a plurality of second sub-transactions, judging whether the plurality of first sub-transactions conflict with the plurality of second sub-transactions Assuming that there is a target sub-transaction in multiple first sub-transactions, the target sub-transaction needs to be used for conflict detection with all other remaining transactions, and the target can only be determined when it does not conflict with all other transactions Subtransactions are transactions to be committed.

Exemplarily, for the case where the first transaction set includes multiple first sub-transactions, and the second transaction set includes multiple second sub-transactions, if judging whether a sub-transaction 11 conflicts with a sub-transaction 12, and the sub-transaction The timestamp of 11 is greater than the timestamp of subtransaction 12. Then the sub-transaction 11 with a larger timestamp is a transaction to be rolled back, and the sub-transaction 12 with a smaller timestamp is not rolled back, but the sub-transaction 12 is not necessarily a transaction to be committed. Because, whether the subtransaction 12 is a transaction to be committed can only be determined after the subtransaction 12 has been checked for conflicts with other transactions. After all transactions are compared, the transactions that are determined not to be rolled back can be committed. It can be understood that the identifications of subtransaction 11 and subtransaction 12 (ie 11 and 12 ) are just examples.

It should be noted that the method in the embodiment of the present application can be applied to the OCC scenario, and the "commit" in the commit transaction in this scenario is equivalent to actually writing the data updated by the write operation into the database.

Exemplarily, the first information of the first sub-transaction in the first transaction set is the first timestamp, and the second information of the second sub-transaction in the second transaction set is the second timestamp. The first sub-transaction and the second sub-transaction include at least one overlapping data item, and the first timestamp of the first sub-transaction is less than the second timestamp of the second sub-transaction, then it can be determined that the first sub-transaction does not need to be rolled back transaction, the second sub-transaction is a rollback transaction. After comparing the timestamps of the first sub-transaction with the timestamps of all other transactions, it is determined that the transaction that does not need to be rolled back is the transaction to be committed. Of course, it may also be that a transaction with a larger timestamp is a transaction that does not need to be rolled back, and a transaction with a smaller timestamp is a transaction to be rolled back.

In addition, the first node may also determine a commit order of transactions to be committed according to the first information and the second information. Specifically, if multiple nodes in the database system periodically send local operation sets to each other, this step may be periodic. In addition, if there are operation sets that cross cycles in multiple operation sets, for example, one operation set in the multiple operation sets is in the first cycle, and the other operation set is in the second cycle, then you can determine the commit operation through the cycle number The collection corresponds to the order of transactions, thereby ensuring synchronization between multiple databases. Certainly, if there are operation sets across periods in the multiple operation sets, there are also operation sets with the same period. In the process of updating the database, the order of the transactions corresponding to the small-scale operation set can be determined first according to the period identifier, and then the order of submitting the transactions can be determined according to the timestamp or the identifier of the transaction.

Optionally, after updating the first database, the first node may record a log or update a snapshot.

Exemplarily, continuing the previous example, the first transaction set executed by the first user in the first database includes: transaction 1 and transaction 2, transaction 1: delete "Sun Wu". Transaction 2: Change the "delivery address" of "Li Liu" to "Room 801, Hotel F, E Street, Nanshan District, Shenzhen, Guangdong Province". The first operation set of the first transaction set includes: delete operation and modification operation. The second transaction set executed by the second user in the second database includes: transaction 3 . Transaction 3: Add "Sunday", "15789012345", "No. W Primary School, Luohu District, Shenzhen City, Guangdong Province". Then the second operation set includes: adding operation. Then, the process of updating the first database by the first node based on the first operation set and the second operation set is shown in FIG. 4 , and the updated first database is shown in Table 2 in FIG. 4 .

Exemplarily, continuing the above-mentioned example applied to the openGauss scenario, the first node parses out the relevant data of the transaction after receiving the operation set of the transaction sent by other nodes in the current period, and uses the same rule to implement concurrency control (or It is understood as determining the transaction to be committed and the transaction to be rolled back). Commit the transaction to be committed and roll back the transaction to be rolled back. It can be understood that the operation of the subsequent second node is similar to that of the first node, and will not be repeated hereafter.

Step 206, the second node updates the second database based on the first operation set and the second operation set.

After the second node acquires the first operation set and the second operation set, the second node may update the second database based on the first operation set and the second operation set. Specifically, the second node may first determine the transaction to be submitted in the second database based on the first operation set and the second operation set, and submit the transaction to be submitted in the second database. In other words, the second node can determine the operations performed in the first database in the first node based on the first operation set, and then can execute the operations in the first operation set, so that the update of the second database has considered the modification of the first database , so as to ensure data synchronization between databases.

Optionally, the second node acquires at least one of the second time stamp, the second identifier, or the second periodic identifier (or called second information) of each transaction in the second transaction set, and receives the information sent by the first node. At least one of the first timestamp, the first identification, or the second period identification (or referred to as the first information) of . The second node may determine the transaction to be committed in the first transaction set and the second transaction set based on the first operation set, the first information, the second operation set, and the second information, and submit the transaction to be committed in the second database , to obtain the updated second database.

For descriptions about overlapping data items in the transactions in the first transaction set and the second transaction set, determining the transaction to be committed and the transaction to be rolled back, you can refer to the description in the aforementioned step 205 , which will not be repeated here.

Optionally, after updating the second database, the second node may record a log or update a snapshot.

Exemplarily, continuing the previous example, the first transaction set includes: transaction 1 and transaction 2, transaction 1: delete "Sun Wu". Transaction 2: Change the "delivery address" of "Li Liu" to "Room 801, Hotel F, E Street, Nanshan District, Shenzhen, Guangdong Province". The first operation set of the first transaction set includes: delete operation and modification operation. The second set of operations includes: increment processing. The second transaction includes: adding "Sunday", "15789012345", "No. W Primary School, Luohu District, Shenzhen City, Guangdong Province". Then, the flow of updating the second database by the second node based on the first operation set and the second operation set is shown in FIG. 4 , and the updated second database is shown in Table 2 in FIG. 4 .

It can be understood that, the embodiment of the present application does not limit the time sequence between the above steps. For example: step 203 may be before step 202 or step 201 . Another example: step 206 may be after step 205, etc., which is not limited here.

In addition, steps 201 to 206 in this step may be executed periodically, and the number of executions of steps 201 to 206 in this embodiment of the present application may be one time or multiple times, which is not specifically limited here. Of course, if it is executed periodically, the transactions in the new cycle cannot be committed before the synchronization of the previous cycle is completed.

In the embodiment of the present application, the first node obtains the first operation set related to the first transaction set in the first database, receives the second operation set related to the second transaction set in the second database sent by the second node, and based on The first operation set and the second operation set update the first database, so that the update of the first database takes into account the transactions submitted by other databases, thereby ensuring the update synchronization between multiple databases. On the one hand, transaction modification is notified between nodes through the transmission operation set, which does not involve multiple interactions between nodes, and the communication overhead is small. On the other hand, compared with the two-phase commit method, it avoids the inoperability of the transaction caused by the coordinator due to a single point of failure.

The database processing method in the embodiment of the application is described above, and the first node and the second node in the embodiment of the application are described below. The first node and the second node are applied to the database system, and the first node is deployed with the first A database, the second node is deployed with a second database, and the first database and the second database store the same data. Please refer to Figure 5, an embodiment of the first node in the embodiment of the present application includes:

An acquisition unit 501, configured to acquire a first operation set, where the first operation set is a write operation or a read-write operation of a first transaction set in the first database within a target time period;

a sending unit 502, configured to send the first operation set to the second node;

The receiving unit 503 is configured to receive the second operation set sent by the second node, the second operation set is the write operation or read-write operation of the second transaction set in the second database within the target time period, the first operation set and the second The set of operations is used by the second node to update the second database;

An updating unit 504, configured to update the first database based on the first operation set and the second operation set.

In this embodiment, the operations performed by each unit in the first node are similar to those described in the foregoing embodiments shown in FIG. 1 to FIG. 4 , and will not be repeated here.

In this embodiment, multiple nodes in the database system send transaction operation sets to each other, and database updates on nodes take into account transactions submitted by other databases, thereby ensuring synchronization of updates between multiple databases. On the one hand, transaction modification is notified between nodes through the transmission operation set, which does not involve multiple interactions between nodes, and the communication overhead is small. On the other hand, compared with the two-phase commit method, it avoids the inoperability of the transaction caused by the coordinator due to a single point of failure.

Please refer to Figure 6, an embodiment of the second node in the embodiment of the present application includes:

An acquisition unit 601, configured to acquire a second operation set, where the second operation set is a write operation or a read-write operation of a second transaction set in the second database within a target time period;

a sending unit 602, configured to send the second operation set to the first node;

The receiving unit 603 is configured to receive the first operation set sent by the first node, the first operation set is the write operation or read-write operation of the first transaction set in the first database within the target time period, the first operation set and the second The operation set is used for the first node to update the first database;

An updating unit 604, configured to update the second database based on the first operation set and the second operation set.

In this embodiment, the operations performed by each unit in the second node are similar to those described in the foregoing embodiments shown in FIG. 1 to FIG. 4 , and will not be repeated here.

Referring to FIG. 7 , it is a schematic structural diagram of a communication device provided in this application. The communication device may be the first node or the second node in the foregoing embodiments shown in FIG. 1 to FIG. 4 . The communication device may include a processor 701 , a memory 702 and a communication interface 703 . The processor 701, the memory 702 and the communication interface 703 are interconnected by wires. Wherein, program instructions and data are stored in the memory 702 .

In a possible implementation manner, the memory 702 stores program instructions and data corresponding to the steps executed by the first node in the foregoing implementations corresponding to FIG. 1 to FIG. 4 . The processor 701 is configured to execute the steps performed by the first node shown in any one of the embodiments shown in FIGS. 1 to 4 . The communication interface 703 may be used to execute steps related to acquiring, sending, and receiving performed by the first node in any of the embodiments shown in FIGS. 1 to 4 . For example, the communication interface 703 may be used to send a first operation set and first information, and receive a second operation set and second information.

In another possible implementation manner, the memory 702 stores program instructions and data corresponding to the steps executed by the second node in the aforementioned implementation manners corresponding to FIG. 1 to FIG. 4 . The processor 701 is configured to execute the steps executed by the second node shown in any one of the embodiments shown in FIGS. 1 to 4 . The communication interface 703 may be used to execute steps related to acquiring, sending, and receiving performed by the second node in any of the embodiments shown in FIGS. 1 to 4 . For example, the communication interface 703 may be used to send the second operation set and the second information, and receive the first operation set and the first information.

It can be understood that the communication device may include more or less components than those shown in FIG. 7 , and this application is only illustrative and not limiting.

In addition, an embodiment of the present application provides a database system, and the database system includes the first node as shown in FIG. 5 and/or the second node as shown in FIG. 6 . Or the database system includes the communication device as shown in FIG. 7 above.

In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated. to another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units may be fully or partially realized by software, hardware, firmware or any combination thereof.

When the integrated units are implemented using software, they may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, all or part of the processes or functions according to the embodiments of the present invention will be generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (solid state disk, SSD)), etc.

The terms "first", "second" and the like in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, and this is merely a description of the manner in which objects with the same attribute are described in the embodiments of the present application. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, product, or apparatus comprising a series of elements is not necessarily limited to those elements, but may include elements not expressly included. Other elements listed explicitly or inherent to the process, method, product, or apparatus.

Claims

A database processing method, characterized in that the method is applied to a database system, the database system includes a first node and a second node, the first node is deployed with a first database, and the second node is deployed with a second node Two databases, the first database and the second database store the same data, and the method includes:

The first node acquires a first operation set, where the first operation set is a write operation or a read-write operation of a first transaction set in the first database within a target time period;

the first node sends the first set of operations to the second node;

The first node receives a second operation set sent by the second node, the second operation set is a write operation or a read-write operation of a second transaction set in the second database within the target time period, The first operation set and the second operation set are used by the second node to update the second database;

The first node updates the first database based on the first set of operations and the second set of operations.
The method according to claim 1, wherein said first node acquiring a first set of operations comprises:

The first node responds to and executes an operation instruction of the first user on the first database;

The first node records operations performed by the first user on data items in the first database during execution of the operation instructions to obtain the first operation set.
The method according to claim 2, wherein the operation instruction is a write instruction, and the first operation set is a write operation set; or, the operation instruction is a read and write instruction, and the first operation set is A collection of read and write operations or a collection of write operations.
The method according to any one of claims 1 to 3, wherein the method further comprises:

The first node obtains first information of the first transaction set, each transaction in the first transaction set corresponds to a first information, and the first information includes a first identifier, and/or a first timestamp ;

the first node sends the first information to the second node;

The first node receives the second information sent by the second node, each transaction in the second transaction set corresponds to a second information, and the second information includes a second identifier and/or a second timestamp ; The first information and the second information are used by the second node to update the second database;

The first node updating the first database based on the first operation set and the second operation set includes:

The first node updates the first database based on the first set of operations, the first information, the second set of operations, and the second information.
The method according to claim 4, wherein the first node updates the first database based on the first set of operations, the first information, the second set of operations, and the second information ,include:

The first node determines the transactions to be committed in the first transaction set and the second transaction set based on the first operation set, the first information, the second operation set, and the second information ;

The first node submits the transaction to be submitted in the first database to obtain an updated first database.
The method according to claim 5, wherein the first information of the first sub-transaction in the first transaction set is the first timestamp, and the information of the second sub-transaction in the second transaction set The second information is the second timestamp, the first sub-transaction and the second sub-transaction include at least one overlapping data item, the first timestamp is smaller than the second timestamp, the The second subtransaction is a transaction that cannot be committed.
A first node, wherein the first node is applied to a database system, and the database system further includes a second node, the first node is deployed with a first database, and the second node is deployed with a second A database, the first database and the second database store the same data, and the first node includes:

An acquisition unit, configured to acquire a first operation set, where the first operation set is a write operation or a read-write operation of a first transaction set in the first database within a target time period;

a sending unit, configured to send the first operation set to the second node;

a receiving unit, configured to receive a second operation set sent by the second node, the second operation set is a write operation or a read-write operation of a second transaction set in the second database within the target time period, The first operation set and the second operation set are used by the second node to update the second database;

An updating unit, configured to update the first database based on the first operation set and the second operation set.
The first node according to claim 7, wherein the acquiring unit is specifically configured to respond to and execute an operation instruction of the first user on the first database;

The obtaining unit is specifically configured to record operations performed by the first user on data items in the first database during execution of the operation instructions, to obtain the first operation set.
The first node according to claim 8, wherein the operation instruction is a write instruction, and the first operation set is a write operation set; or, the operation instruction is a read and write instruction, and the first operation A collection is a collection of read and write operations or a collection of write operations.
The first node according to any one of claims 7 to 9, wherein the obtaining unit is further configured to obtain the first information of the first transaction set, each of the first transaction set The transaction corresponds to a first piece of information, and the first piece of information includes a first identifier and/or a first time stamp;

The sending unit is further configured to send the first information to the second node;

The receiving unit is further configured to receive second information sent by the second node, each transaction in the second transaction set corresponds to a second information, the second information includes a second identifier, and/or the second Two timestamps; the first information and the second information are used by the second node to update the second database;

The updating unit is specifically configured to update the first database based on the first operation set, the first information, the second operation set, and the second information.
The first node according to claim 10, wherein the updating unit is specifically configured to determine based on the first operation set, the first information, the second operation set, and the second information Transactions to be committed in the first transaction set and the second transaction set;

The update unit is specifically configured to submit the transaction to be submitted in the first database to obtain an updated first database.
The first node according to claim 11, wherein the first information of the first sub-transaction in the first transaction set is the first timestamp, and the second sub-transaction in the second transaction set The second information of the transaction is the second timestamp, the first sub-transaction and the second sub-transaction include at least one overlapping data item, the first timestamp is smaller than the second timestamp, The second subtransaction is a transaction that cannot be committed.
A first node, characterized by comprising: a processor, the processor is coupled with a memory, and the memory is used to store a program or an instruction, and when the program or instruction is executed by the processor, the The first node executes the method according to any one of claims 1-6.
A computer storage medium, characterized in that it includes computer instructions, and when the computer instructions are run on an electronic device, the electronic device is made to execute the method according to any one of claims 1 to 6.
A computer program product, characterized in that, when the computer program product is run on a computer, the computer is made to execute the method according to any one of claims 1 to 6.