CN109783578B

CN109783578B - Data reading method and device, electronic equipment and storage medium

Info

Publication number: CN109783578B
Application number: CN201910021181.9A
Authority: CN
Inventors: 李海翔; 卢卫; 杜小勇; 潘安群
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd; Tencent Cloud Computing Beijing Co Ltd
Priority date: 2019-01-09
Filing date: 2019-01-09
Publication date: 2022-10-21
Anticipated expiration: 2039-01-09
Also published as: CN109783578A

Abstract

The invention discloses a data reading method and device, electronic equipment and a storage medium, and belongs to the technical field of databases. The method provided by the embodiment of the invention determines the global write transaction which possibly influences the transaction consistency by finding the global write transaction which is in an active state when the current global read transaction occurs, namely, finds the common point with the transaction consistency based on the MVCC among a plurality of node devices, and based on the reading of the global write transaction, the read time is pushed back to a time which is more delayed relative to the current time, so that the read time can be regarded as a delayed read process.

Description

Data reading method and device, electronic equipment and storage medium

Technical Field

The present invention relates to the field of database technologies, and in particular, to a data reading method and apparatus, an electronic device, and a storage medium.

Background

Many of the existing distributed database systems can support write operations across nodes, that is, for a certain write operation, a write process may be involved for a plurality of node devices in the distributed database system, and thus, a problem of consistency of read data may occur, for example: when the cross-node write operation is implemented, assuming that there are two node devices, a preparation commit stage is passed, a transaction can be Committed, a first node device commits completely, and a second node device does not commit yet, at this time, a new global Read operation is performed by the Distributed database system, data submitted by the first node device is Read, but data is not Read because the second node device does not complete data commit yet, which is called a Distributed Read Committed-Committing exception (DRCC for short), and thus current data reading cannot guarantee that the Read data is in a consistent state.

Disclosure of Invention

The invention provides a data reading method, a data reading device, electronic equipment and a storage medium, which can solve the problem of data reading consistency. The technical scheme is as follows:

in one aspect, a data reading method is provided, and the method includes:

sending a first indication message to a plurality of node devices corresponding to a global read transaction, wherein the first indication message is used for indicating the node devices to stop submitting a global write transaction;

receiving a related transaction list of the plurality of node devices, wherein the related transaction list of each node device comprises related global write transactions which are in progress on the node device and a transaction state of each related global write transaction, and each related global write transaction corresponds to at least two node devices in the plurality of node devices;

determining a first global write transaction group according to the relevant transaction lists of the plurality of node devices, wherein the transaction states of first global write transactions on corresponding node devices, which are included in the first global write transaction group, comprise an executing state and a ready-to-commit state;

sending the first global write transaction group to the plurality of node devices;

receiving first target data returned by the plurality of node devices, wherein the first target data comprises data acquired based on the global read transaction and the first global write transaction group.

In one aspect, a data reading method is provided, and is applied to a node device, where the method includes:

according to a received first indication message, stopping the commit operation of the global write transaction, wherein the first indication message is used for indicating a plurality of node devices corresponding to the global read transaction to stop the commit operation of the global write transaction;

obtaining a related transaction list, where the related transaction list includes related global write transactions that are in progress on the node devices and a transaction state of each related global write transaction, and each related global write transaction corresponds to at least two node devices in the plurality of node devices;

sending the related transaction list;

receiving a first global write transaction group, wherein transaction states of first global write transactions on corresponding node equipment, which are included in the first global write transaction group, comprise an executing state and a ready-to-commit state;

and outputting first target data according to the first global write transaction group and the global read transaction, wherein the first target data comprises data acquired based on the global read transaction and the first global write transaction group.

In one aspect, a data reading method is provided, and the method includes:

sending a second indication message to a plurality of node devices corresponding to a global read transaction, wherein the second indication message is used for indicating the node devices to return a relevant transaction list of the global read transaction;

determining a first global write transaction group and a second global write transaction group according to the relevant transaction list of the plurality of node devices, wherein the transaction states of the first global write transactions on the corresponding node devices, which are included in the first global write transaction group, comprise an executing state and a ready-to-commit state, and the transaction states of the second global write transactions on the corresponding node devices, which are included in the second global write transaction group, are all executing states;

when the first global write affairs in the first global write affair group are all in a submitted state on the corresponding node equipment, the first global write affair group and the second global write affair group are sent to the plurality of node equipment;

receiving second target data returned by the plurality of node devices, wherein the second target data comprises data obtained based on the global read transaction, the first global write transaction group and the second global write transaction group.

receiving a second indication message, wherein the second indication message is used for indicating a plurality of node devices of the global read transaction to return a related transaction list;

acquiring a related transaction list, wherein the related transaction list of each node device comprises related global write transactions in progress on the node device and a transaction state of each related global write transaction, and each related global write transaction corresponds to at least two node devices in the plurality of node devices;

sending the related transaction list;

receiving a first global write transaction group and a second global write transaction group, wherein the transaction states of first global write transactions on corresponding node equipment, which are included in the first global write transaction group, comprise an executing state and a ready-to-commit state, and the transaction states of second global write transactions on corresponding node equipment, which are included in the second global write transaction group, are all the executing state;

and outputting second target data according to the first global write transaction group, the second global write transaction group and the global read transaction, wherein the second target data comprises data acquired based on the global read transaction, the first global write transaction group and the second global write transaction group.

In one aspect, there is provided a data reading apparatus, the apparatus comprising:

a sending module, configured to send a first indication message to multiple node devices corresponding to a global read transaction, where the first indication message is used to indicate the multiple node devices to stop performing a commit operation of a global write transaction;

a receiving module, configured to receive a related transaction list of the multiple node devices, where the related transaction list of each node device includes related global write transactions that are currently performed on the node device and a transaction state of each related global write transaction, and each related global write transaction corresponds to at least two node devices in the multiple node devices;

a determining module, configured to determine a first global write transaction group according to the relevant transaction lists of the multiple node devices, where transaction states of a first global write transaction included in the first global write transaction group on a corresponding node device include an executing state and a ready-to-commit state;

the sending module is further configured to send the first global write transaction group to the plurality of node devices;

the receiving module is further configured to receive first target data returned by the multiple node devices, where the first target data includes data obtained based on the global read transaction and the first global write transaction group.

In one aspect, there is provided a data reading apparatus, the apparatus including:

the system comprises a stopping module and a sending module, wherein the stopping module is used for stopping the submitting operation of the global write transaction according to a received first indication message, and the first indication message is used for indicating a plurality of node devices corresponding to the global read transaction to stop the submitting operation of the global write transaction;

an obtaining module, configured to obtain a related transaction list, where the related transaction list includes related global write transactions that are in progress on the node device and a transaction state of each related global write transaction, and each related global write transaction corresponds to at least two node devices in the plurality of node devices;

a sending module, configured to send the relevant transaction list;

the receiving module is used for receiving a first global write transaction group, and the transaction states of first global write transactions on corresponding node equipment, which are included in the first global write transaction group, comprise an executing state and a ready-to-commit state;

and the output module is used for outputting first target data according to the first global write transaction group and the global read transaction when a first global write transaction in the first global write transaction group is in a committed state, wherein the first target data comprises data acquired based on the global read transaction and the first global write transaction group.

a sending module, configured to send a second indication message to multiple node devices corresponding to a global read transaction, where the second indication message is used to indicate the multiple node devices to return a relevant transaction list of the global read transaction;

a determining module, configured to determine a first global write transaction group and a second global write transaction group according to the relevant transaction list of the multiple node devices, where transaction states of a first global write transaction included in the first global write transaction group on a corresponding node device include an executing state and a ready-to-commit state, and transaction states of a second global write transaction included in the second global write transaction group on the corresponding node device are all executing states;

the sending module is further configured to send the first global write transaction group and the second global write transaction group to the plurality of node devices when the first global write transaction in the first global write transaction group is in a committed state on the corresponding node device;

the receiving module is further configured to receive second target data returned by the multiple node devices, where the second target data includes data obtained based on the global read transaction, the first global write transaction group, and the second global write transaction group.

a receiving module, configured to receive a second indication message, where the second indication message is used to indicate that multiple node devices of a global read transaction return a relevant transaction list;

an obtaining module, configured to obtain a related transaction list, where the related transaction list of each node device includes related global write transactions that are in progress on the node device and a transaction state of each related global write transaction, and each related global write transaction corresponds to at least two node devices in the multiple node devices;

a sending module, configured to send the relevant transaction list;

the receiving module is further configured to receive a first global write transaction group and a second global write transaction group, where transaction states of a first global write transaction included in the first global write transaction group on a corresponding node device include an executing state and a ready-to-commit state, and transaction states of a second global write transaction included in the second global write transaction group on the corresponding node device are both the executing state;

and the output module is used for outputting second target data according to the first global write transaction group, the second global write transaction group and the global read transaction when a first global write transaction in the first global write transaction group is in a committed state, wherein the second target data comprises data acquired based on the global read transaction, the first global write transaction group and the second global write transaction group.

In one aspect, an electronic device is provided and includes a processor and a memory, where at least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor to implement the operations performed by the data reading method.

In one aspect, a computer-readable storage medium is provided, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement the operations performed by the data reading method as described above.

The method provided by the embodiment of the invention determines the global write transaction which possibly influences the transaction consistency by finding the global write transaction which is in an active state when the current global read transaction occurs, namely, finds the common point with the transaction consistency based on the MVCC among a plurality of node devices, and based on the reading of the global write transaction, the read time is pushed back to a time which is more delayed relative to the current time, so that the read time can be regarded as a delayed read process.

Drawings

Fig. 1 is a schematic diagram of an implementation environment of a data reading method according to an embodiment of the present invention;

FIG. 2 is a flow chart of a data reading method according to an embodiment of the present invention;

FIGS. 3 and 4 are comparative diagrams of transaction identifications, respectively;

FIG. 5 is a flow chart of a data reading method according to an embodiment of the present invention;

FIG. 6 provides a diagram of signaling interactions between node devices acting as hosts and slaves in a database system;

FIG. 7 is a schematic structural diagram of a data reading apparatus according to an embodiment of the present invention;

FIG. 8 is a schematic structural diagram of a data reading apparatus according to an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of a data reading apparatus according to an embodiment of the present invention;

FIG. 10 is a schematic structural diagram of a data reading apparatus according to an embodiment of the present invention;

fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

To make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Embodiments of the present invention relate to a database that stores a plurality of data tables, each data table may be used to store tuples, which may have one or more versions. The database may be any type of database based on MVCC (Multi-Version concurrent Control). In the embodiment of the present invention, the type of the database is not particularly limited. It should be noted that the data in the database may include three states based on the state attribute: the data state identification method comprises a current state, a transition state and a history state, wherein the three states are collectively called a 'data full state', for short, full state data, and different state attributes in the full state data can be used for identifying the state of the data in the life cycle track of the data.

Current State (Current State): the latest version of the tuple's data is the data at the current stage. The state of the data in the current phase is referred to as the current state.

Transition State (Transitional State): the data in the transition state, which is not the latest version or the history state version of the tuple, is called half-decay data in the process of transition from the current state to the history state.

Historical state (Historical state): a state of a tuple in history whose value is the old value, not the current value. The state of the data in the history phase is referred to as the history state. The historical state of a tuple can be multiple, and the process of state transition of data is reflected. Data in a history state can only be read and cannot be modified or deleted.

It should be noted that, under the MVCC mechanism, all the three states of the data exist, and under the non-MVCC mechanism, the data may exist only in the history state and the current state. Under MVCC or lockout concurrent access control mechanisms, the new value of the data after the transaction commits is in the current state. Taking the MVCC mechanism as an example, the state of the data generated by the transaction before the smallest transaction in the current active transaction list is in a history state. Under the lock-out concurrency access control mechanism, after the transaction commits, the value of the data before the commit is changed to the value of the history state, that is, the old value of the tuple is in the history state. The read version has active transactions (non-latest relevant transactions) in use, and the latest relevant transactions modify the values of the tuples, the latest values of the tuples are in a current state, the read values are in a historical state relative to the current state, and the data state of the tuples is between the current state and the historical state, so the tuple is called a transition state.

For example, under the MVCC mechanism, the balance of the account a in the User table is changed from 10-element recharging to 20-element, then 15-element consumption is changed to 5-element consumption, at this time, the financial B institution reads data and checks the transaction all the time, after a, 20-element recharging is changed to 25-element consumption, 25-element consumption is the current state data, 5-element consumption is the transition state, and the remaining two values 20 and 10 are the states existing in history, and are both history state data.

Fig. 1 is a schematic diagram of an implementation environment of a data reading method according to an embodiment of the present invention. Referring to fig. 1, an implementation environment provided in fig. 1 may be a distributed database system, which may include a gateway server, a global transaction identifier generation cluster, and a distributed storage cluster, where the distributed storage cluster may include a plurality of node devices, and the distributed storage cluster may adopt a master-slave structure. In some embodiments, the gateway server may be merged with any node device in the distributed storage cluster on the same physical machine, that is, a node device participating in a read operation is allowed to act as the gateway server.

The gateway server is used for receiving the read-write request, and generating a unique global transaction identifier for a cluster application from the global transaction identifier for the read transaction or the write transaction corresponding to the read-write request based on whether the read transaction or the write transaction corresponding to the read-write request is a global transaction, so as to ensure the consistency of data read-write in the whole distributed database system.

The global transaction identifier generation cluster is configured to generate a global transaction identifier, namely gxid, to identify a global transaction, which may refer to a transaction involving multiple node devices, for example, a global read transaction may involve reading data stored on multiple node devices, and for example, a global write transaction may involve writing data on multiple node devices. The generation of the global transaction identifier is realized in a cluster form, so that single-point failure can be prevented. When a global transaction occurs, a gateway server can apply a globally unique identification value for the global transaction identification generation cluster. In some embodiments, the global transaction identifier generation cluster may be physically independent, or may be merged with a distributed storage cluster (e.g., zooKeeper) to provide a global transaction identifier generation service for each gateway server. Fig. 1 is an architecture diagram providing a lightweight distributed transaction, which is a kind of distributed database system.

Fig. 2 is a flowchart of a data reading method according to an embodiment of the present invention. Referring to fig. 2, the method includes:

201. when the read transaction involves cross-node operation, the gateway server determines the read transaction as a global read transaction, and the gateway server sends a generation request to the global transaction identifier generation cluster.

When the gateway server receives any operation statement (such as SQL statement), the gateway server serving as a high-level computing layer of the database can analyze the operation statement, and when the operation statement of the read transaction carries a specified keyword, the gateway server determines that the read transaction relates to cross-node operation. For example, the specified keyword may be "GLOBAL" to indicate that the read object of the operation statement includes all data in the database system, that is, covers all node devices in the database system, and if the operation statement includes "GLOBAL", step 201 is executed.

Of course, in some embodiments, it may also be determined whether the data to be read is on one node device according to an operation statement of the read transaction, and if it is determined that the data to be read is not on the same node device, it is determined that the read transaction involves a cross-node operation. Specifically, it is determined whether data to be read is stored in two or more node devices according to a range of the data to be read in an operation statement of the read transaction and metadata in the range, and when it is determined that the data is stored in the two or more node devices, the read transaction involves a cross-node operation. Since the metadata is recorded with the current storage device of the data. For example, the gateway server may determine, according to the metadata, an accessed node device for each operation statement (SQL (Structured Query Language)) of each transaction, record the determined node device, and perform step 201 when it is counted that the number of different node devices accessed is greater than or equal to 2. In the above embodiment, this determination manner based on metadata may be applied to a SELECT statement of a single statement.

The above process of determining whether to involve cross-node operations can be summarized as identification based on a specified key and automatic identification by the gateway server, if there are cross-node operations (global transaction identity), then a global transaction. If the operation statement (for example, SQL statement) only relates to a single node device, and belongs to a local transaction, also called local transaction, a global transaction identifier does not need to be applied, and only a local transaction identifier is allocated to the transaction.

Taking the example of specifying the keyword as "GLOBAL", the operation statement may take the following form:

BEGIN GLOBAL; the GLOBAL transaction identifier gxid is applied to the GLOBAL gxid generation cluster

SELECT…

…

END；

202. And after receiving the generation request, the global transaction identifier generation cluster generates a global transaction identifier for the global read transaction and sends the global transaction identifier to the gateway server.

In this embodiment of the present invention, the global transaction identifier generated by the global transaction identifier generation cluster may be a numerical type.

203. The gateway server takes the global transaction identification as the transaction identification of the global read transaction.

For tuples, if MVCC technology is supported, a gxid may be added to the tuple in the case that the xid field originally exists, and the xid is renamed to ixid, and the above difference in format can be seen in fig. 3 and fig. 4. The transaction identification of the local transaction and the global transaction identification may both be monotonically increasing.

For example, if a transaction T needs to write two node devices to perform a transfer operation, the operation statement may take the following form:

BEGIN GLOBAL; v/apply for a global gxid, assumed to be 20

UPDATE user _ account SET my _ wallet = my _ wallet-10where key =100; // node 1, local ixid 18

UPDATE user _ account SET my _ wallet = my _ wallet + 10wheere key =900; // node 2, local ixid 22

COMMIT；

The key on the node device 1 is a tuple of 100, and the transaction identifier on the tuple is a two-tuple: { gxid, ixid } = {20, 18}.

The key on the node device 2 is a tuple of 900, and the transaction identifier on the tuple is a two-tuple: { gxid, ixid } = {20, 22}.

Through the transaction identification, whether the data from different node devices are the data of the same global transaction operation or not can be identified, namely whether the data belong to the same transaction or not. If the next transaction of the node apparatus 1 is a global transaction, the transaction identifier is { gxid, ixid } = {21, 19}; and the next transaction is a local transaction, the transaction identifier is { gxid, ixid } = {0, 20}; and the next transaction is a global transaction, the transaction is identified as { gxid, lxid } = {22, 21}, and so on.

204. And the gateway server sends a first indication message to a plurality of node devices corresponding to the global read transaction according to the transaction identifier of the global read transaction, wherein the first indication message is used for indicating the plurality of node devices to stop submitting the global write transaction.

In order to avoid the continuation of transactions which may cause data inconsistency and continue on each node device involved in the global read transaction, the gateway server may decompose the global read transaction, and first require all the involved nodes to stop the global commit operation of other transactions.

For the distributed database system, each node device may have active global write transactions, and if the active global write transactions are committed, a new data inconsistency may occur, so that the gateway server may indicate that the active global write transactions on the respective node devices can no longer be changed from the executing state to the ready-to-commit state. Of course, the local transaction may continue to commit. Wherein the executing state indicates that the transaction phase identified in the distributed database system has not entered a prepare-to-commit phase. The ready-to-commit state indicates that the transaction has entered the commit phase.

It should be noted that the gateway server may set a timeout mechanism, for example, if any node device in the plurality of node devices times out and does not return a relevant transaction list, which indicates that the node device may have a network problem or be down, the gateway server may send the first indication message to the node device or the plurality of node devices again, so as to ensure that each node device can stop the commit operation of the global write transaction. Of course, in order to save signaling and avoid resource waste, a sending threshold may be set for the retransmission operation, for example, the retry sending operation is set to 3 times, and when the sending threshold is reached, and a node device still does not return the relevant transaction list, the global read transaction is rolled back. Rolling back a global read transaction refers to restoring the database system to a state where the global read transaction starts, for example, the database system sets all data that has been read or output based on the global read transaction to be invalid.

The first indication message may include a transaction identifier of a global read transaction and node identifiers of a plurality of node devices, for example, the global read transaction may be denoted by T-current, and the node devices referred to by T-current include N1, N3, N5, N9. The node identifier may adopt a similar style of { N1, N3, N5, N9} to indicate that the global read transaction operation involves the node devices.

In the above embodiments, the node device involved in stopping committing and transferring the global read transaction is executed through one indication message, so as to achieve the purpose of saving signaling, and in some embodiments, the message for executing the committing stopping and the message for transferring the global read transaction may be two messages, which are sent in two pieces, for example, the gateway server first sends a commit stopping message to the plurality of node devices, so that each node device stops the commit operation of the global write transaction, and after receiving the stop responses of the plurality of node devices, the gateway server sends a message for transferring the node device involved in the global read transaction to the plurality of node devices, so that each node device can return the relevant transaction list based on the node device involved in the global read transaction.

Certainly, in any message sending process of the gateway server, a timeout mechanism can be set to avoid affecting the normal operation of the database system.

205. And each node device in the plurality of node devices stops the commit operation of the global write transaction according to the received first indication message, wherein the first indication message is used for indicating the plurality of node devices corresponding to the global read transaction to stop the commit operation of the global write transaction.

For any node device, after receiving the first indication message, identifying, according to the transaction identifier of each active write transaction, whether the active write transaction is a global write transaction, for determining that the active write transaction on the node device (a write transaction in an executing state or a ready-to-commit state) is a global write transaction, and stopping a commit operation of the global write transaction when any active write transaction is a global write transaction, where stopping the commit operation of the global write transaction means that, for any global write transaction, the global write transaction is prohibited from being converted from the executing state to the ready-to-commit state.

In some embodiments, the commit operation of the global write transaction may also be implemented by using a resource lock, for example, when the node device receives the first indication message, the resource corresponding to the active global write transaction on the node device may be locked, so that the locked resource cannot be written, and after the relevant transaction list is obtained, the resource may be released. In one example, the resource lock involved in the lock may be mutex.

In some embodiments, if the first indication message is sent in two, the node device may stop the commit operation after receiving the stop commit message, and return a stop response to the gateway server, so as to inform the gateway server that the gateway server itself has stopped performing the commit operation, so as to trigger the gateway server to further send a message for indicating to acquire the relevant transaction list, so as to trigger the acquisition process.

206. Each node device in the plurality of node devices obtains a related transaction list, where the related transaction list includes related global write transactions that are in progress on the node device and a transaction status of each related global write transaction, and each related global write transaction corresponds to at least two node devices in the plurality of node devices.

The node device may provide the gateway server with the relevant global write transactions that are ongoing on the node device and the transaction status of each relevant global write transaction. The related transaction means: other node devices involved in global write transactions must be encompassed by node devices involved in global read transactions. For example, if a global write transaction T9 writes node devices N1, N3, it is determined that the contents are contained and need to be fed back to the gateway server; and a global write transaction T10 writes node devices N3, N7, and if N7 is not in the node range involved in the global read transaction, then node device N7 is not required to respond proxy even if it is not included. The transactions in the related transaction list must at least relate to at least two node devices corresponding to the global read transaction, for example, the global read transaction relates to { N1, N3, N5, N9}, and then the related global write transaction of the global read transaction needs to relate to at least two of the node devices.

In this embodiment of the present invention, optionally, the obtaining the relevant transaction list may include: traversing an ongoing global write transaction on the node device; and when the node equipment corresponding to any global write transaction comprises at least two node equipment in the plurality of node equipment, adding the write transaction to the relevant transaction list.

The related transaction list may be carried in a reply returned by the node device to the gateway server. Certainly, in order to enable each node device to know which write transactions on the node device may affect the consistency of the global read transaction, each global write transaction also needs to issue the node device related to the transaction to each node device related to the transaction to know. For example, for the global write transaction T9, N1 knows that its global write transaction only involves N1, N3, and the two node devices are in the range of the node devices involved in the global read transaction, so the information of the global write transaction T9 is used as the reply content. For example, the global write transaction T10, N3 knows that its global write transaction only involves N1, N7, and N7 is not in the range of the node devices involved in the global read transaction, so that the information of the global write transaction T10 is not used as the reply content.

Further, if the node device involved in the global write transaction is not a subset of the plurality of node devices involved in the global read transaction, the global write transaction is also not a relevant global write transaction of the global read transaction, and information of the global write transaction may not be used as the reply content.

Optionally, the list of related transactions returned by the node device to the gateway server may be: node identification of the node device, transaction identification of each global write transaction, and transaction status, for example:

list of related transactions for N1: { N1, { gxid 2-prepare commit status, gxid 5-execute status, gxid 9-execute status }

List of related transactions for N3: { N3, { gxid 2-executing state, gxid 5-executing state, gxid 9-ready-commit state }

List of related transactions for N5: { N5, { gxid 2-ready-commit state, gxid 5-executing state }

List of related transactions for N9: { N9, { gxid 5-executing state }

In some embodiments, if the first indication message is sent twice, the node device may stop the commit operation after receiving the stop commit message, and return a stop response to the gateway server, so as to notify the gateway server that the gateway server itself has stopped performing the commit operation. And the node device receives a message for transferring the node devices involved in the global read transaction, so that each node device can return a relevant transaction list based on the node devices involved in the global read transaction.

207. Each node device of the plurality of node devices transmits the list of related transactions.

In step 207, since each node device has performed the process of prohibiting committing the global write transaction based on the first indication message, each node device has sent the relevant transaction list to the gateway server, that is, the commit operation of the global write transaction can be resumed, that is, for the node N1, after sending the relevant transaction list, the commit operation of the global write transaction can be continued, and the executing state is allowed to be changed into the ready-to-commit state, even into the Committed state (Committed).

208. And after receiving the relevant transaction lists of the plurality of node devices, the gateway server determines a first global write transaction group according to the relevant transaction lists of the plurality of node devices.

The transaction states of the first global write transactions included in the first global write transaction group on the corresponding node device include an executing state and a ready-to-commit state, that is, for any first global write transaction, the transaction state of the first global write transaction on the corresponding node device is both the executing state and the ready-to-commit state. If the transactions in the first global write transaction group are subsequently submitted, new inconsistency can be caused, so that the transactions of the type can be recorded into a group, the transactions in the first global write transaction group are all transactions which can generate inconsistency, the submission of the transactions can be subsequently controlled to be completed, and then the versions of the tuples corresponding to the transactions are classified in a reading range, so that the situation of data inconsistency is avoided.

In an embodiment of the present invention, the step 208 may include: and adding any global write transaction to the first global write transaction group as a first global write transaction when the transaction state of the global write transaction on the corresponding node device comprises an executing state and a ready-to-commit state from the related transaction list of the plurality of node devices.

In a possible implementation manner, when the transaction states of any global write transaction on the corresponding node device include an executing state and a ready-to-commit state, the gateway server may notify the node device corresponding to the global write transaction to resume the commit operation on the global write transaction, and when the global write transaction is in a committed state on the node device, add the global write transaction as the first global write transaction to the first global write transaction group. The above-described process of controlling commit may be performed for each global write transaction in the list of related transactions, resulting in a respective first global write transaction.

In another possible embodiment, after determining each first global write transaction in the first global write transaction group, the gateway server may notify the node device corresponding to each first global write transaction to resume the commit operation on the global write transaction, and when each first global write transaction is in a committed state on the node device, the following sending process of step 209 may be performed.

For the node device, when receiving a recovery commit message sent by the gateway server, the node device may resume the commit operation of the global write transaction, and when the global write transaction is in a committed state, the node device may return a committed response to the gateway server, so that the gateway server knows the committed state. Of course, for the node device, it is also not necessary to notify the recovery by the gateway server, but the commit operation may be recovered after the relevant transaction list is obtained and sent, and when the node device performs data reading, it is only necessary that all the global write transactions are already in the committed state.

For the transactions except for the first global write transaction group, the transaction may also be notified to reply the commit operation of the global write transaction, or the node device may recover the commit operation by itself, which is not limited in the embodiment of the present invention.

In order to express the transaction status on the node device more specifically, refer to table 1 below, where table 1 shows a table of the case where the node device executes a global transaction.

TABLE 1

	gxid2 transactions	gxid5 transactions	gxid9 transactions	gxid10 transactions
					N1	Prepare to commit state	Executing state	Executing state
N3	Executing state	Executing state	Prepare to commit state	Executing state
					N5	Prepare to commit state	Executing state
N9		Executing state
					N7				Prepare commit state

Based on the example of table 1 above, it can be seen that for a global write transaction on a node device, there may be four cases:

in case 1, the sub-transactions of the global write transaction are in a ready-to-commit state on each node device;

in case 2, the sub-transactions of the global write transaction are in an executing state on each node device, which indicates that the distributed read inconsistency cannot be caused when the transaction does not enter a commit stage at the moment;

in case 3, at least one of the sub-transactions of the global write transaction is in an executing state and one of the sub-transactions of the global write transaction is in a ready-to-commit state on each node device;

in case 4, the sub-transactions of the global write transaction are not in a ready-to-commit state and an executing state on the respective node devices, i.e., no global write transaction is executing. Since this case has been excluded by the acquisition of the relevant transaction list, it is not provided as the reply content to the gateway server.

Based on the above cases 1 to 3, the gateway server may traverse each relevant global write transaction, such as gxid2, gxid5, and gxid9 in table 1 above, filter out the global write transaction corresponding to the case 2, and do no processing.

The first global transaction group may be a transaction identifier set, and may be regarded as a snapshot (PostgreSQL becomes SanpShot, and MySQL/inodb is called ReadView) is constructed, the snapshot is equivalent to registering all currently active global write transactions to a list, and subsequently, based on the list and the tuple visibility judgment, it may be determined whether the read tuple is a version written with respect to a transaction whose transaction has already been committed, and if so, it is visible, so that the read consistency of the single machine is ensured.

209. The gateway server sends the first global write transaction group to the plurality of node devices.

Since the actual reading is completed by the node device, the gateway server needs to send the first global write transaction group to the node device, so as to perform visibility judgment by the node device, thereby implementing the reading of data.

210. After each node device in the plurality of node devices receives the first global write transaction group, a first tuple in a target tuple of the global read transaction is output according to the transaction snapshot of the global read transaction, and the first tuple is a tuple visible to the global read transaction.

When each node device finishes executing all the global write transactions including the ready-to-commit status, that is, for each node device, each first global write transaction in the first global write transaction group is in a committed status on the node device, the node device may determine to execute a subsequent data visibility determination process, so as to perform actual data reading.

For the global read transaction, whether multiple versions of a tuple are visible or not can be determined by the transaction snapshot of the global read transaction, for example, whether the tuple is visible or not can be determined according to the creation time of the transaction snapshot of the global read transaction and the creation time, deletion time or commit time of the tuple. Specifically, whether the tuple related to the embodiment of the present invention is visible means whether the tuple can be read by the transaction at the time corresponding to the transaction snapshot of the global read transaction. Each tuple read from the data table may read the life cycle information of the tuple, that is, the creation time, deletion time, commit time, and other information of the tuple, taking visibility determination based on the history time, that is, the creation time of the transaction snapshot as an example:

(I): when a certain version of the tuple is generated for the insert operation, the tuple is determined to be visible when the creation time and the commit time are both before the start time of the creation time of the transaction snapshot.

(II): when a certain version of the tuple is generated for a delete operation, the tuple is determined to be visible when both the delete time and the commit time are before the creation time of the transaction snapshot.

(III): when a version of the tuple is generated for an update operation, the tuple is determined to be visible when both the creation time and the commit time are before the creation time of the transaction snapshot.

211. And if the node equipment determines that the target tuple of the global read transaction comprises a second tuple and the transaction submitting the second tuple is any one of the first global write transactions, outputting the second tuple, wherein the second tuple is a tuple invisible to the global read transaction.

In the visibility judgment of step 210, when a certain version of any tuple is invisible to the global read transaction, that is, the visibility condition is not satisfied, it may be further judged whether the transaction submitting the version is the first global write transaction in the first global write transaction group, and if so, the tuple of the version is also acquired and returned to the gateway server.

In the above process, the first global write transaction has already finished committing when the node device starts reading, so that the part of data does not have a state of being partially committed and partially uncommitted, and reading inconsistency is not caused, and because the data related to the global write transaction which is in the executing state in the case 2 is excluded in the reading process of the node device, the above-mentioned distributed read semi-committed abnormal phenomenon of the related data of some transactions does not occur, and the consistency of data reading is ensured.

For example, if any tuple, although not visible to the global read transaction, is committed by gxid2, and gxid2 is in the first global write transaction group, then the tuple is output.

The foregoing steps 210 to 211 are outputting first target data according to the first global write transaction group and the global read transaction, where the first target data includes data acquired based on the global read transaction and the first global write transaction group.

212. And each node device in the plurality of node devices sends the acquired first target data to the gateway server.

213. The gateway server outputs the received first target data.

For each node device, visible data may be obtained according to the received transaction in the first global write transaction group and the global read transaction, so as to output data for the global read transaction. In a possible implementation manner, the gateway server may output the acquired data when receiving the first target data returned by any node device, and in another possible implementation manner, the gateway server may further output the acquired data after receiving the first target data returned by all node devices, which is not limited in the embodiment of the present invention.

The method provided by the embodiment of the invention determines the global write transaction which possibly influences the transaction consistency by finding the global write transaction which is in an active state when the current global read transaction occurs, namely, finds the common point with the transaction consistency based on the MVCC among a plurality of node devices, and based on the reading of the global write transaction, the read time is pushed back to a time which is more delayed relative to the current time, so that the read time can be regarded as a delayed read process. In the process of delayed reading, the global write transactions which may cause data inconsistency are analyzed through the active global write transactions on each node device involved in the global read transactions, so that a transaction group is formed by some global write transactions which may cause data inconsistency.

In the above embodiment, in order to improve the concurrency and avoid blocking that may be caused by inhibiting commit, in some embodiments, the inhibiting of commit may not be performed, but some transactions that may cause data inconsistency and transactions that may not cause data inconsistency are determined by determining the first global write transaction group and the second global write transaction group, and then tuples are acquired based on the first global write transaction group and the second global write transaction group, see fig. 5, which may include the following steps:

501. when the read transaction involves cross-node operation, the gateway server determines the read transaction as a global read transaction, and the gateway server sends a generation request to the global transaction identifier generation cluster.

502. After receiving the generation request, the global transaction identifier generation cluster generates a global transaction identifier for the global read transaction, and sends the global transaction identifier to the gateway server.

503. The gateway server takes the global transaction identification and the local transaction identification of the read transaction as the transaction identification of the global read transaction.

The above steps 501 to 503 are similar to the steps 201 to 203, and are not described herein again.

504. And the gateway server sends a second indication message to the plurality of node devices corresponding to the global read transaction, wherein the second indication message is used for indicating the plurality of node devices to return a relevant transaction list of the global read transaction.

The second indication message may include a transaction identification of the global read transaction and node identifications of the plurality of node devices. The relevant transaction list of each node device contains relevant global write transactions ongoing on the node device and a transaction status of each relevant global write transaction, each relevant global write transaction corresponding to at least two node devices of the plurality of node devices.

505. Each node device in the plurality of node devices acquires a related transaction list according to the received second indication message, wherein the related transaction list comprises related global write transactions which are in progress on the node device and the transaction state of each related global write transaction, and each related global write transaction corresponds to at least two node devices in the plurality of node devices.

Step 505 is similar to step 206 and will not be described herein.

506. Each node device of the plurality of node devices transmits the list of related transactions.

507. And after receiving the relevant transaction lists of the plurality of node devices, the gateway server determines a first global write transaction group and a second global write transaction group according to the relevant transaction lists of the plurality of node devices.

The transaction states of the first global write transactions on the corresponding node devices included in the first global write transaction group include an executing state and a ready-to-commit state, and the transaction states of the second global write transactions on the corresponding node devices included in the second global write transaction group are all executing states.

Optionally, this step 507 may include the following steps 507A to 507B:

507A: and adding any transaction to the first global write transaction group as a first global write transaction when the transaction state of the transaction on the corresponding node comprises an executing state and a ready-to-commit state from the related transaction list of the plurality of node devices.

The process is similar to the above step 208, and will not be described herein.

507B: and when the transaction states of any transaction on the corresponding node are all in-execution states from the related transaction lists of the plurality of node devices, adding the transaction as a second global write transaction to the second global write transaction group.

If the transactions in the second global write transaction group are subsequently submitted, new inconsistency may be caused, and therefore, a group of transactions of this type may also be recorded, the transactions in the second global write transaction group are all transactions that may generate inconsistency, and the versions of tuples corresponding to these transactions may be subsequently excluded from reading, so as to avoid the situation of data inconsistency. The second global write transaction group may be considered a tuple invisible set. For example, based on the example provided in table 1 above, the transactions involved in case 2, i.e., belonging to this second global write transaction group.

It should be noted that, the steps 507A and 507B may be performed in parallel or performed according to any sequence, which is not limited in the embodiment of the present invention.

508. The gateway server sends the first global write transaction group and the second global write transaction group to the plurality of node devices.

509. And the node equipment outputs second target data according to the first global write transaction group, the second global write transaction group and the global read transaction, wherein the second target data comprises data acquired based on the global read transaction, the first global write transaction group and the second global write transaction group.

When the node device completes execution of all the global write transactions including the ready-to-commit state, that is, the first global write transaction is already in the committed state, it may be determined to execute a subsequent data visibility determination process, so as to perform actual data reading. The fact that the global write transaction is in the committed state means that the transaction states of the global write transaction on all the node devices are all the committed states.

Specifically, this step 509 includes: outputting a first tuple in a target tuple of the global read transaction according to the transaction snapshot of the global read transaction, wherein the first tuple is a tuple visible to the global read transaction; if the target tuple of the global read transaction comprises a second tuple and the transaction submitting the second tuple is any one of the first global write transactions, outputting the second tuple, wherein the second tuple is a tuple invisible to the global read transaction; and if the target tuple of the global read transaction comprises a second tuple and the transaction submitting the second tuple is any second global write transaction, ignoring the second tuple.

The above data acquisition process is mainly determined by combining MVCC visibility judgment and two transaction groups with different purposes, and for tuple versions which do not satisfy the visibility condition, if the tuple versions are data generated by a certain transaction in the first global write transaction group, the tuple versions are visible. But not if it is data generated by a transaction in the second global write transaction group.

In the above process, the first global write transaction has already completed committing when the node device starts reading, so that the part of data does not have a state in which a part of data has been committed and a part of data has not been committed, and thus no inconsistency in reading is caused, and since there is no control in the entire flow that commits the transaction that is in the executing state on each node device in case 2, it is necessary to exclude the part of transaction, that is, the relevant data generated by the second global write transaction, in the reading process of the node device, and therefore, based on the data reading process of the first global write transaction group and the data exclusion process of the second global write transaction group, it is possible to exclude the data that affects the data consistency while ensuring that the output data has consistency.

In the above steps 501 to 509, the transaction submission will not be prohibited, and the transaction submission will not be prohibited subsequently, which not only reduces the number of signaling interactions, but also improves the concurrency.

510. And the node equipment sends the acquired second target data to the gateway server.

511. The gateway server outputs the received second target data.

For each node device, visible data may be obtained according to the received transaction in the first global write transaction group and the global read transaction, so as to output data for the global read transaction. In a possible implementation manner, the gateway server may output the acquired data when receiving the second target data returned by any node device, and in another possible implementation manner, the gateway server may further output the acquired data after receiving the second target data returned by all node devices, which is not limited in the embodiment of the present invention.

Certainly, in any message sending process of the gateway server, a timeout mechanism can be set to avoid affecting the normal operation of the database system. In some embodiments, for a Global read transaction involving only read-only transactions, if any information sending process is timed out, the rolling back may not be performed, for example, for a read-only transaction, the gateway server may obtain a GTID (Global transaction identifier) value from the beginning, and then distribute the GTID value to each node device, and each node device reconstructs the first Global transaction group to perform the read-only operation. The mechanism avoids the rollback operation of the read-only transaction, can improve the transaction throughput of the system, and is particularly more effective to the application scene of frequent reading in the HTAP system.

The method provided by the embodiment of the invention can also support the concurrent execution of a plurality of global read transactions. For example, when the time interval between the global read transactions is smaller than a preset time interval (e.g., within 1 second, which may be set as a parameter), and the read node devices are the same or contain a relationship (e.g., the node range read by the previous global read transaction contains the read node device range of the next global read transaction), the next global read transaction is allowed to reuse at least one of the first global read transaction group and the second global read transaction group of the previous global read transaction, so as to further improve the overall performance.

Further, the system architecture according to the embodiment of the present invention provides a lightweight decentralized transaction processing architecture for a distributed database system, where the lightweight characteristic is that a global transaction identifier is used to generate a cluster, and the cluster has a single function, can generate global transaction identifiers in bulk in a memory, and is very efficient. The performance advantage is particularly significant compared to a global transaction manager that implements global transaction management, conflicting access control, and MVCC mechanisms. The decentralization is characterized in that the processing of the transaction in the system depends on each node device, but not on a global transaction manager, so that a single-point, complex and time-consuming global transaction manager does not exist in the architecture, and the decentralization on the basis of complete functions is realized.

Further, the embodiment of the present invention can also ensure that the external consistency of the distributed database system is achieved, that is, the events occurring in the distributed database system need to be obtained in the order of occurrence of the events, and the original order of the events can still be reflected when the data is referred to in the database. The global transaction identifier generation cluster provides a globally unique and monotonically increasing logical time identifier for the TDSQL-like system, so that external consistency can be effectively ensured. I.e., a new transaction following the global read transaction, whose data remains invisible to the global read transaction even though the transaction committed prior to the global read transaction. Moreover, the global transaction identifier generation cluster is logically a single point, and both the master copy and the slave copy can provide services physically, so that the probability that the global transaction identifier generation cluster becomes a performance bottleneck is reduced.

The embodiment of the invention is suitable for any transactional database system supporting cross-node global write operation, such as distributed databases (SQL, noSQL, newSQL, relational and non-relational), MVCC-based relational databases, MVCC-based non-relational databases, distributed big data processing and other systems. Further, the embodiment of the present invention is particularly suitable for constructing a distributed HTAP database, especially a massive temporal distributed HTAP (Transactional/Analytical Processing) database. The embodiment of the invention reduces the load of the architecture of the distributed transaction and analysis type database on the transaction processing mechanism level, so that the transaction processing mechanism is simple and efficient. And the single-machine database system of the MVCC is used as a base line of each node and is matched with a decentralized and lightweight transaction processing mechanism, so that the limitation on concurrent query is less, and query operation is autonomous among the nodes, thereby being very suitable for an analysis system. In summary, the embodiments of the present invention provide a possibility for high performance of a distributed hybrid (transactional, analytical) database, and the technical advantage of improving the performance of the entire system is obvious in the architecture level of the distributed database.

For a database system for realizing the MVCC technology by a single machine, different implementation modes of the database system have certain influence on the embodiment of the invention. For example, the concurrent access control technology for data realizes a serializable scenario, for a database that depends on TO (timestamp) + MVCC, such as PostgreSQL, serializability is guaranteed by using an SSI (spring struts ibatis) technology, and since the SSI technology is also an MVCC technology in nature, the data reading method provided by the embodiment of the present invention can be applied no matter which isolation level is under. Of course, the isolation level set on each node device by transactions across nodes needs to be consistent. For a database relying on a blocking-based technology, such as MySQL/InnoDB, MVCC is used to implement RR (Repeatable Read, isolation level) and RC (Read _ Committed, read Committed isolation level). Of course, the isolation level set on each node by transactions across the nodes also needs to be consistent. The embodiment of the invention is also applicable to the database adopting the SI isolation mode.

It should be noted that the data reading method may be applied to a backup technology, and when a global backup is performed, objects to be read are all node devices in a cluster, in this case, the data reading method may suspend all executing write transactions to commit (the time is theoretically shorter relative to a global transaction manager). But does not affect the starting and running of new transactions and the distributed write transaction of which all the sub-transactions are in the executing state phase. For non-global backup, the influence on a database system is small due to the limited number of involved nodes; the method provided by the embodiment of the invention eliminates the single-point bottleneck of the global transaction from the architecture, can fundamentally improve the overall transaction throughput of the system to a large extent, has very small performance loss caused by suspending the submission operation of the partial global write transaction, cannot influence the overall performance, and cannot cause mutual blocking of read-write operation.

In some embodiments, the node devices may keep heartbeat, and once any read node device fails to respond to a read, the heartbeat may be reported to the gateway server, and the gateway server is responsible for releasing a prohibition such as prohibition of write resumption and processing an event such as failure of a write transaction, such as transaction rollback). If some node devices have completed the sub-transaction, the user may be notified that the returned data is invalid, and of course, for the centralized result returning mechanism, when any node device is in a situation, the result is not output to the user, so that the notification to the user is not needed.

In some embodiments, each node cluster may have a different primary/secondary architecture, and when a global read transaction is executed, the global read transaction may be performed based on the different primary/secondary architectures, where a preferential read target of the global read transaction is a node device storing a primary copy. Based on different main/standby architectures, the method provided by the embodiment of the present invention may have different applications:

the first way is to rely on master-slave logic replication technology to achieve high reliability, such as MySQL, which is a logic replication using binglog. For the first mode, because the write transaction first occurs on the node device of the primary copy, and the secondary copy is only used for backup and read-only services, the read method provided by the embodiment of the present invention is used for performing read operation on the host (i.e., the node device used for storing the primary copy), so that the transaction consistency of global read can be ensured. Taking the master-slave replication technique of MySQL as an example, due to its master-slave replication mechanism, it may cause the SQL statement of the user to be re-executed on the backup machine, that is, the node device for storing the slave replica), and since the transaction identifier is automatically allocated by each database engine, it may cause the transaction identifiers on the tuples on the master replica and the slave replica to be different, for this reason, a certain signaling interaction may be performed between the host and the backup machine, for example, fig. 6 provides a signaling interaction diagram between the node devices serving as the host and the backup machine in the database system, and as shown in fig. 6, the method may further include: when a master-slave logic copying technology is adopted in a database of a master-slave structure, when a standby machine receives an operation instruction of a global transaction transmitted by a host, assigning a transaction identifier of an execution transaction on the standby machine according to the global transaction identifier of the global transaction. The global transaction may be a global read transaction or a global write transaction, which is not limited in the embodiments of the present invention. For example, when the host generates a bindlog, for each transaction, { gxid, ixid } of the global transaction may be passed to the standby, which, upon receiving { gxid, ixid } and executing the SQL statement in the bindlog, assigns a value to each transaction's gxid and ixid with the received { gxid, ixid }. A comparison table of MySQL Binlog format improvements is provided, as in table 2 below.

TABLE 2

By the method, the problem of global read consistency under a logic replication framework is solved, and the problem of global read data consistency under any cross-node transaction (such as main-main, main-standby and standby combination) can be solved. For example, the data read by the global read transaction is all originated from the standby system, so that the host in the distributed database system is not influenced, and the overall performance is high.

The second way is a master-slave physical replication technique to achieve high reliability, such as physical replication using the REDO Log (Log), for example, the stream replication technique of PostgreSQL is a physical replication technique relying on the REDO Log. For the second mode, namely the physical replication mode, because the tuple is a physical-level replication, the transaction identifier of the tuple is not lost, but the multiple versions of data of different databases are organized differently, so that the recovery process may be different. Two typical ways are as follows:

class PostgreSQL mode: the characteristic of this kind of mode is that the multi-version data is stored in the PAGE (PAGE), and when the REDO mode is used for recovery, the data is recovered to the standby machine, so that the history data read "dirty" can be successfully read, and the application of the data reading method of the embodiment of the present invention will not be affected.

MySQL like/InnodB mode: the characteristic of this type of mode is that the multi-version data is stored in the rollback section of the memory. The method includes that recovery is performed based on an RODO Log, information lacking a rollback section is recovered in a standby machine at the same time, and therefore the information of the rollback section can be synchronized to the standby machine (for example, the rollback section information is recorded into an REDO Log, and memory recovery is performed according to the recorded rollback section information during recovery).

The third way is to realize data consistency of logical or physical multiple copies by means of a distributed consistency protocol, so as to realize high reliability, for example, a high-reliability system constructed by distributed consistency protocols such as Paxos, raft and the like. For the third manner, a logical manner and a physical manner may be distinguished, and specific reference may be made to the processing of the first manner and the processing of the second manner.

Fig. 7 is a schematic structural diagram of a data reading apparatus according to an embodiment of the present invention, and referring to fig. 4, the apparatus includes:

a sending module 701, configured to send a first indication message to multiple node devices corresponding to a global read transaction, where the first indication message is used to indicate the multiple node devices to stop performing a commit operation of a global write transaction;

a receiving module 702, configured to receive a related transaction list of the plurality of node devices, where the related transaction list of each node device includes related global write transactions that are ongoing on the node device and a transaction status of each related global write transaction, and each related global write transaction corresponds to at least two node devices in the plurality of node devices;

a determining module 703, configured to determine a first global write transaction group according to the relevant transaction list of the multiple node devices, where transaction states of a first global write transaction included in the first global write transaction group on a corresponding node device include an executing state and a ready-to-commit state;

the sending module 701 is further configured to send the first global write transaction group to the plurality of node devices;

the receiving module 702 is further configured to receive first target data returned by the multiple node devices, where the first target data includes data obtained based on the global read transaction and the first global write transaction group.

In one possible embodiment, the determining module is configured to:

and when the transaction state of any transaction on the corresponding node device comprises an executing state and a ready-to-commit state from the related transaction list of the plurality of node devices, adding the transaction to the first global write transaction group as a first global write transaction.

In one possible embodiment, the apparatus further comprises:

and the global transaction identifier acquisition module is used for determining any read transaction as a global read transaction and acquiring a global transaction identifier for the global read transaction when the read transaction involves cross-node operation.

In a possible implementation manner, the global transaction identifier obtaining module is configured to send a generation request to a global transaction identifier generation cluster; and receiving the global transaction identifier returned by the global transaction identifier generation cluster, and taking the global transaction identifier as the transaction identifier of the global read transaction.

In a possible implementation manner, when an operation statement of the read transaction carries a specified key, determining that the read transaction relates to a cross-node operation;

and determining whether the data to be read is stored in more than two node devices according to the range of the data to be read in the operation statement of the read transaction and the metadata in the range, and determining that the read transaction relates to cross-node operation when the data to be read is determined to be stored in the more than two node devices.

In one possible embodiment, the apparatus further comprises:

when a master-slave logic copying technology is adopted in a database of a master-slave structure, when a standby machine receives an operation instruction of a global transaction transmitted by a host, assigning a transaction identifier of an execution transaction on the standby machine according to the global transaction identifier of the global transaction;

when a master-slave physical replication technology is adopted in the database of the master-slave structure, the current state data stored in the standby machine and the transition state data in the rollback section information of the standby machine are read in the process of reading based on the global read transaction.

In one possible embodiment, the apparatus further comprises:

a rollback module, configured to roll back the global read transaction when a response of at least one node device of the plurality of node devices is not received within a target duration.

All the above optional technical solutions may be combined arbitrarily to form optional embodiments of the present disclosure, and are not described in detail herein.

It should be noted that: in the data reading apparatus provided in the foregoing embodiment, only the division of the functional modules is illustrated when data is read, and in practical applications, the functions may be distributed by different functional modules as needed, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the functions described above. In addition, the data reading apparatus and the data reading method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments and are not described herein again.

Fig. 8 is a schematic structural diagram of a data reading apparatus provided in an embodiment of the present invention, and referring to fig. 8, the apparatus includes:

a stopping module 801, configured to stop performing a commit operation of a global write transaction according to a received first indication message, where the first indication message is used to indicate a plurality of node devices corresponding to a global read transaction to stop performing the commit operation of the global write transaction;

an obtaining module 802, configured to obtain a related transaction list, where the related transaction list includes related global write transactions that are in progress on the node device and a transaction state of each related global write transaction, and each related global write transaction corresponds to at least two node devices in the plurality of node devices;

a sending module 803, configured to send the relevant transaction list;

a receiving module 804, configured to receive a first global write transaction group, where transaction states of first global write transactions included in the first global write transaction group on corresponding node devices include an executing state and a ready-to-commit state;

an output module 805, configured to, when a first global write transaction in the first global write transaction group is in a committed state, output first target data according to the first global write transaction group and the global read transaction, where the first target data includes data obtained based on the global read transaction and the first global write transaction group.

In one possible implementation, the obtaining module is configured to: traversing an ongoing global write transaction on the node device; when the node device corresponding to any global write transaction comprises at least two node devices in the plurality of node devices, adding the write transaction to the relevant transaction list.

In a possible implementation manner, the output module is configured to output a first tuple corresponding to the global read transaction according to a transaction snapshot of the global read transaction, where the first tuple is a visible version of a target tuple;

if the target tuple includes a second tuple and the transaction that committed the second tuple is any one of the first global write transactions, outputting the second tuple, the second tuple being an invisible version of the target tuple.

It should be noted that: in the data reading apparatus provided in the foregoing embodiment, only the division of the functional modules is illustrated when data is read, and in practical applications, the functions may be distributed by different functional modules as needed, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the functions described above. In addition, the data reading apparatus and the data reading method provided in the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments, and are not described herein again.

Fig. 9 is a schematic structural diagram of a data reading apparatus according to an embodiment of the present invention, and referring to fig. 9, the apparatus includes:

a sending module 901, configured to send a second indication message to multiple node devices corresponding to a global read transaction, where the second indication message is used to indicate the multiple node devices to return a relevant transaction list of the global read transaction;

a receiving module 902, configured to receive a related transaction list of the multiple node devices, where the related transaction list of each node device includes related global write transactions that are ongoing on the node device and a transaction status of each related global write transaction, and each related global write transaction corresponds to at least two node devices in the multiple node devices;

a determining module 903, configured to determine, according to the relevant transaction list of the multiple node devices, a first global write transaction group and a second global write transaction group, where transaction states of a first global write transaction included in the first global write transaction group on the corresponding node device include an executing state and a ready-to-commit state, and transaction states of a second global write transaction included in the second global write transaction group on the corresponding node device are all executing states;

the sending module 901 is further configured to send, to the multiple node devices, the first global write transaction group and the second global write transaction group when the first global write transaction in the first global write transaction group is in a committed state on the corresponding node device;

the receiving module 902 is further configured to receive second target data returned by the multiple node devices, where the second target data includes data obtained based on the global read transaction, the first global write transaction group, and the second global write transaction group.

In one possible embodiment of the method according to the invention,

the determining module is configured to add, from the relevant transaction list of the multiple node devices, the global write transaction as a first global write transaction to the first global write transaction group if the transaction state of any global write transaction on the corresponding node device includes an executing state and a ready-to-commit state; and if the transaction states of any global write transaction on the corresponding node equipment are all in-execution states from the related transaction lists of the plurality of node equipment, adding the global write transaction as a second global write transaction to the second global write transaction group.

It should be noted that: in the data reading apparatus provided in the above embodiment, only the division of the functional modules is illustrated when data is read, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the functions described above. In addition, the data reading apparatus and the data reading method provided in the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments, and are not described herein again.

Fig. 10 is a schematic structural diagram of a data reading apparatus according to an embodiment of the present invention, and referring to fig. 10, the apparatus includes:

a receiving module 1001, configured to receive a second indication message, where the second indication message is used to indicate that multiple node devices of a global read transaction return a relevant transaction list;

an obtaining module 1002, configured to obtain a related transaction list, where the related transaction list of each node device includes related global write transactions that are ongoing on the node device and a transaction state of each related global write transaction, and each related global write transaction corresponds to at least two node devices in the plurality of node devices;

a sending module 1003, configured to send the relevant transaction list;

the receiving module 1001 is further configured to receive a first global write transaction group and a second global write transaction group, where transaction states of a first global write transaction included in the first global write transaction group on a corresponding node device include an executing state and a ready-to-commit state, and transaction states of a second global write transaction included in the second global write transaction group on the corresponding node device are both the executing state;

an output module 1004, configured to output, when a first global write transaction in the first global write transaction group is in a committed state, second target data according to the first global write transaction group, the second global write transaction group, and the global read transaction, where the second target data includes data obtained based on the global read transaction, the first global write transaction group, and the second global write transaction group.

In a possible implementation manner, the output module is configured to output a first tuple corresponding to the global read transaction according to a transaction snapshot of the global read transaction, where the first tuple is a visible version of a target tuple; if the target tuple comprises a second tuple and the transaction submitting the second tuple is any one of the first global write transactions, outputting the second tuple, wherein the second tuple is an invisible version of the target tuple; if the target tuple includes a second tuple and the transaction that committed the second tuple is any second global write transaction, ignoring the second tuple.

Fig. 11 is a schematic structural diagram of an electronic device 1100 according to an embodiment of the present invention, where the electronic device 1100 may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 1101 and one or more memories 1102, where the memory 1102 stores at least one instruction, and the at least one instruction is loaded and executed by the processors 1101 to implement the data reading method provided by each method embodiment. Certainly, the electronic device may further have a wired or wireless network interface, a keyboard, an input/output interface, and other components to facilitate input and output, and the electronic device may further include other components for implementing functions of the device, which are not described herein again. The gateway server and the node device involved in the data reading process can adopt the hardware structure of the electronic device.

An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium is applied to a server, and at least one instruction, at least one program, a code set, or a set of instructions is stored in the computer-readable storage medium, where the instruction, the program, the code set, or the set of instructions is loaded and executed by a processor to implement the operations performed by a gateway server or a node device in the data reading method according to the foregoing embodiments.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims

1. A method of reading data, the method comprising:

receiving a related transaction list of the plurality of node devices, the related transaction list of each node device including related global write transactions that are in progress on the node device and a transaction status of each related global write transaction, each related global write transaction corresponding to at least two node devices of the plurality of node devices;

determining a first global write transaction group according to the relevant transaction list of the plurality of node devices, wherein the transaction states of first global write transactions on corresponding node devices, which are included in the first global write transaction group, comprise an executing state and a ready-to-commit state;

receiving first target data returned by the plurality of node devices, wherein the first target data comprises a first tuple and a second tuple in target tuples of the global read transaction output by the plurality of node devices according to the transaction snapshots of the global read transaction, the first tuple is a tuple visible for the global read transaction, the second tuple is an invisible tuple for the global read transaction, and a submitted transaction corresponding to the second tuple is any first global write transaction.

2. The method of claim 1, wherein determining a first global write transaction group from the list of related transactions for the plurality of node devices comprises:

and adding any transaction to the first global write transaction group as a first global write transaction when the transaction state of the transaction on the corresponding node device comprises an executing state and a ready-to-commit state from the related transaction list of the plurality of node devices.

3. The method of claim 1, further comprising:

when any read transaction relates to cross-node operation, determining the read transaction as a global read transaction, and sending a generation request to a global transaction identifier generation cluster;

receiving the global transaction identifier returned by the global transaction identifier generation cluster, and using the global transaction identifier as the transaction identifier of the global read transaction.

4. The method of claim 1, further comprising:

when a master-slave physical replication technology is adopted in the database of the master-slave structure, reading the current state data stored in the standby machine and the transition state data in the rolling section information of the standby machine in the process of reading based on the global reading transaction.

5. The method of claim 1, further comprising:

rolling back the global read transaction when a reply of at least one of the plurality of node devices is not received within a target duration.

6. A data reading method is applied to node equipment, and the method comprises the following steps:

sending the relevant transaction list;

when a first global write transaction in the first global write transaction group is in a committed state, outputting first target data according to the first global write transaction group and the global read transaction, where the first target data includes a first tuple and a second tuple in target tuples of the global read transaction output by the node devices according to the transaction snapshots of the global read transaction, the first tuple is a tuple visible for the global read transaction, the second tuple is a tuple invisible for the global read transaction, and a committed transaction corresponding to the second tuple is any first global write transaction.

7. The method of claim 6, wherein obtaining the list of related transactions comprises:

traversing an ongoing global write transaction on the node device;

and when the node equipment corresponding to any global write transaction comprises at least two node equipment in the plurality of node equipment, adding the write transaction to the related transaction list.

8. The method of claim 6, wherein outputting first target data based on the first global write transaction group and the global read transaction comprises:

outputting a first tuple in a target tuple of the global read transaction according to the transaction snapshot of the global read transaction;

and if the target tuple of the global read transaction comprises a second tuple and the transaction submitting the second tuple is any one of the first global write transactions, outputting the second tuple.

9. A method of reading data, the method comprising:

sending a second indication message to a plurality of node devices corresponding to a global read transaction, wherein the second indication message is used for indicating the plurality of node devices to return a relevant transaction list of the global read transaction;

when the first global write affairs in the first global write affair group are all in a committed state on the corresponding node equipment, the first global write affair group and the second global write affair group are sent to the plurality of node equipment;

receiving second target data returned by the plurality of node devices, where the second target data includes a first tuple and a second tuple in target tuples of the global read transaction output by the plurality of node devices according to the transaction snapshot of the global read transaction, the first tuple is a tuple visible to the global read transaction, the second tuple is an invisible tuple to the global read transaction, and a committed transaction corresponding to the second tuple is any first global write transaction.

10. The method of claim 9, wherein determining the first global write transaction group and the second global write transaction group from the list of related transactions for the plurality of node devices comprises:

if the transaction state of any global write transaction on the corresponding node equipment comprises an executing state and a ready-to-commit state, adding the global write transaction as a first global write transaction to the first global write transaction group from the relevant transaction list of the plurality of node equipment;

and if the transaction states of any global write transaction on the corresponding node equipment are all in-execution states from the related transaction lists of the plurality of node equipment, adding the global write transaction as a second global write transaction to the second global write transaction group.

11. A data reading method is applied to node equipment, and the method comprises the following steps:

acquiring a related transaction list, wherein the related transaction list of each node device comprises related global write transactions which are in progress on the node device and a transaction state of each related global write transaction, and each related global write transaction corresponds to at least two node devices in the plurality of node devices;

sending the related transaction list;

when a first global write transaction in the first global write transaction group is in a committed state, outputting second target data according to the first global write transaction group, the second global write transaction group and the global read transaction, wherein the second target data comprises a first tuple and a second tuple in target tuples of the global read transaction output by the node devices according to transaction snapshots of the global read transaction, the first tuple is a tuple visible for the global read transaction, the second tuple is an tuple invisible for the global read transaction, and a committed transaction corresponding to the second tuple is any first global write transaction.

12. The method of claim 11, wherein outputting second target data based on the first global write transaction group, the second global write transaction group, and the global read transaction comprises:

if the target tuple of the global read transaction comprises a second tuple and the transaction submitting the second tuple is any one of the first global write transactions, outputting the second tuple;

and if the target tuple of the global read transaction comprises a second tuple and the transaction submitting the second tuple is any second global write transaction, ignoring the second tuple.

13. A data reading apparatus, characterized in that the apparatus comprises:

a receiving module, configured to receive a related transaction list of the plurality of node devices, where the related transaction list of each node device includes related global write transactions that are ongoing on the node device and a transaction state of each related global write transaction, and each related global write transaction corresponds to at least two node devices in the plurality of node devices;

a determining module, configured to determine a first global write transaction group according to the relevant transaction list of the multiple node devices, where transaction states of a first global write transaction included in the first global write transaction group on a corresponding node device include an executing state and a ready-to-commit state;

the receiving module is further configured to receive first target data returned by the plurality of node devices, where the first target data includes a first tuple and a second tuple in target tuples of the global read transaction output by the plurality of node devices according to the transaction snapshot of the global read transaction, the first tuple is a tuple visible to the global read transaction, the second tuple is a tuple invisible to the global read transaction, and a committed transaction corresponding to the second tuple is any first global write transaction.

14. An electronic device, comprising a processor and a memory, wherein at least one instruction is stored in the memory, and is loaded and executed by the processor to implement the operations performed by the data reading method as claimed in any one of claims 1 to 12.

15. A computer-readable storage medium having stored therein at least one instruction which is loaded and executed by a processor to perform operations performed by a data reading method as provided in any one of claims 1 to 12.