CN111291062B

CN111291062B - Data synchronous writing method and device, computer equipment and storage medium

Info

Publication number: CN111291062B
Application number: CN202010070579.4A
Authority: CN
Inventors: 李建华; 王鹏程
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-01-21
Filing date: 2020-01-21
Publication date: 2023-01-10
Anticipated expiration: 2040-01-21
Also published as: CN111291062A

Abstract

The application relates to a data synchronous writing method, a data synchronous writing device, computer equipment and a storage medium. The method relates to the technical field of network data storage, and comprises the following steps: any node in the storage system receives a first write request sent by a client; the method comprises the steps of sending a first synchronization request containing data information of a first node to a second node, receiving first indication information sent by the second node, writing first data into a first data record on the first node when the data information of each storage node in a storage system meets a consistency condition, and sending a first write-in notice to the second node. Because the write-in operation is the direct operation on the data records and the consistency judgment is carried out on different nodes before the write-in operation is carried out, the accuracy and consistency of data write-in among different nodes can be ensured, the write-in operation of different data records can be simultaneously executed on a plurality of nodes, and the data write-in efficiency of the storage system is improved.

Description

Data synchronous writing method and device, computer equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of network data storage, in particular to a data synchronous writing method and device, computer equipment and a storage medium.

Background

With the continuous development of network and computer technologies, a storage system often uses multiple nodes to store the same data, and the consistency of data among different nodes becomes more and more important.

In order to achieve consistency of data in different nodes, in the related art, a data-specific partitioning rule in a node is divided into a plurality of regions, the regions included in different nodes are the same, and only one region in the same region in different nodes can provide a write service to the outside.

However, although the scheme shown in the related art supports the simultaneous execution of write operations on multiple nodes, it cannot achieve the purpose of writing data in the same area at different nodes, which affects the data writing efficiency of the storage system.

Disclosure of Invention

The embodiment of the application provides a data synchronous writing method, a data synchronous writing device, computer equipment and a storage medium, which can realize the concurrent execution of the writing operation of different data records on a plurality of nodes at the same time, and the technical scheme is as follows:

in one aspect, a data synchronous writing method is provided, where the method is used in a storage system, where the storage system includes at least two storage nodes, where the at least two storage nodes are used to store the same data record, and the method is performed by a first node, where the first node is any one of the at least two storage nodes; the method comprises the following steps:

receiving a first write-in request sent by a client, wherein the first write-in request is used for requesting to write a first data record; the first data record is any data record stored in the storage system;

sending a first synchronization request containing data information of the first node to a second node, where the data information of the first node includes global snapshot information of the first node and dependent version information of the first data record in the first node, the global snapshot information is used to indicate a write operation completed by a corresponding storage node, and the dependent version information is used to indicate a write operation completed by the corresponding data record at the last time; the first synchronization request is used for indicating that first data is written in the first data record on the second node; the second node comprises each node of the at least two storage nodes except the first node; the first data is data corresponding to the first write request;

receiving first indication information sent by the second node, wherein the first indication information is used for indicating whether the data information of the first node and the data information of the second node are synchronous on the first data record;

and in response to the data information of the at least two storage nodes meeting a consistency condition, writing the first data in the first data record on the first node, and sending a first write notification to the second node, wherein the first write notification is used for notifying that the first data is written in the first data record on the second node.

In another aspect, an apparatus for synchronously writing data is provided, where the apparatus is used in a storage system, where the storage system includes at least two storage nodes, where the at least two storage nodes are used to store the same data record, and the apparatus is used in a first node, where the first node is any one of the at least two storage nodes; the device comprises:

the first receiving module is used for receiving a first write request sent by a client, wherein the first write request is used for requesting to write a first data record; the first data record is any data record stored in the storage system;

a first sending module, configured to send a first synchronization request including data information of the first node to a second node, where the data information of the first node includes global snapshot information of the first node and dependency version information of the first data record in the first node, the global snapshot information is used to indicate a write operation completed by a corresponding storage node, and the dependency version information is used to indicate a write operation completed by a corresponding data record for the last time; the first synchronization request is used for indicating that first data is written in the first data record on the second node; the second node comprises each node except the first node in the at least two storage nodes; the first data is data corresponding to the first write request;

a second receiving module, configured to receive first indication information sent by the second node, where the first indication information is used to indicate whether data information of the first node and data information of the second node are synchronized on the first data record;

a first writing module, configured to, in response to that data information of the at least two storage nodes meets a consistency condition, write the first data in the first data record on the first node, and send a first writing notification to the second node, where the first writing notification is used to notify that the first data is written in the first data record on the second node.

Optionally, the first indication information includes indication information sent by each of the n nodes; the indication information includes:

and the consistency identification of the corresponding node is used for indicating whether the write operation indicated by the data information of the corresponding node is synchronous with the write operation indicated by the data information of the first node.

Optionally, the apparatus further includes:

a returning module, configured to send a rollback notification to the second node in response to that the data information of the at least two storage nodes does not satisfy the consistency condition, where the rollback notification is used to instruct the second node to rollback to a state before receiving the first synchronization request.

Optionally, the data information includes a write operation identifier of a write operation that is completed last time by a corresponding first data record;

the device further comprises:

and the distribution module is used for distributing a globally unique write operation identifier in the storage system for the write operation corresponding to the first write request before the first sending module sends the first synchronization request containing the data information of the first node to the second node.

Optionally, the write operation identifier includes an originating node identifier and an operation sequence number; the initiating node identification is the identification of a storage node initiating the corresponding write operation; the operation sequence number is a sequence number which is increased with the initiating sequence of the write operation on the storage node corresponding to the initiating node identification.

Optionally, the first sending module is configured to, in response to completion or failure of sending a previous synchronization request in the first node, execute the step of sending the first synchronization request including the data information of the first node to the second node;

and the identification of the initiating node of the previous synchronization request is the same as the identification of the initiating node of the write operation corresponding to the first write request.

Optionally, the apparatus further comprises:

a third receiving module, configured to receive a second synchronization request that includes data information of a third node, where the second synchronization request is used to instruct to write second data in a second data record on the first node; the third node is any one of the at least two storage nodes except the first node; the second data record is any data record stored in the storage system;

a second sending module, configured to send second indication information to the third node, where the second indication information is used to indicate whether data information of the third node and data information of the first node are synchronized on the second data record;

a second writing module, configured to write the second data in the second data record on the first node in response to receiving a second write notification returned by the third node; and the second write-in notification is sent by the third node when the third node determines that the data information of the at least two storage nodes meets the consistency condition according to the second indication information.

Optionally, the apparatus further comprises:

a first obtaining module, configured to obtain dependent version information of the second data record in the third node, where the dependent version information is included in the data information of the third node;

a second obtaining module, configured to obtain record snapshot information of the second data record in the first node in response to that the dependency version information of the second data record in the third node exists in the global snapshot information of the first node, where the record snapshot information is used to indicate a write operation completed in a corresponding data record;

a second determining module, configured to determine, in response to the record snapshot information of the second data record in the first node being a subset of the global snapshot information of the third node, the data information of the first node to be synchronized with the data information of the third node on the second data record.

Optionally, the apparatus further includes:

a third determining module, configured to determine the data information of the first node in response to the dependent version information of the second data record in the third node not existing in the global snapshot information of the first node or the record snapshot information of the second data record in the first node is not a subset of the global snapshot information of the third node, the data information of the first node not being synchronized with the data information of the third node on the second data record

Optionally, the apparatus further includes:

a third sending module, configured to send, to the third node, second indication information in response to a third synchronization request sent by the third node ending execution;

wherein a receiving time of the third synchronization request is before a receiving time of the second synchronization request, and the third synchronization request is used to instruct to write third data in the second data record on the first node. Optionally, the apparatus further includes:

and the deleting module is used for responding to the successful synchronization of the write operation corresponding to the second write request to the at least two storage nodes, and deleting the write operation identifier of the write operation corresponding to the second write request from the record snapshot information of the data record in the first node.

In another aspect, a computer device is provided, which includes a processor and a memory, wherein at least one instruction, at least one program, a set of codes, or a set of instructions is stored in the memory, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the data synchronous writing method as described above.

In another aspect, a computer-readable storage medium is provided, in which at least one instruction, at least one program, a set of codes, or a set of instructions is stored, which is loaded and executed by a processor to implement the data synchronous writing method as described above.

The technical scheme provided by the application can comprise the following beneficial effects:

after receiving a first write-in request for performing write operation on a first data record sent by a client, any node in the storage system sends a first synchronization request containing data information of the first node to a second node, receives first indication information which is sent by the second node and used for indicating whether the data information of the first node and the data information of the second node are synchronized, and when the data information of each storage node in the storage system meets a consistency condition, writes first data in the first data record on the first node and sends a first write-in notification to the second node. Because the write-in operation is the direct operation on the data records and the consistency judgment is carried out on different nodes before the write-in operation is carried out, the accuracy and consistency of data write-in among different nodes can be ensured, the write-in operation of different data records can be simultaneously carried out on a plurality of nodes, and the data write-in efficiency of the storage system is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

FIG. 1 illustrates a schematic diagram of a storage system shown in an exemplary embodiment of the present application;

FIG. 2 illustrates a schematic diagram of a storage system including data regions according to an embodiment of the present application;

FIG. 3 is a flow chart illustrating a method of synchronized writing of data in accordance with an exemplary embodiment;

FIG. 4 is a flow chart illustrating a method of synchronous writing of data in accordance with an exemplary embodiment;

FIG. 5 illustrates a corresponding diagram of a write operation identification shown in an exemplary embodiment of the present application;

FIG. 6 illustrates a record snapshot information diagram according to an exemplary embodiment of the present application;

FIG. 7 is a diagram illustrating that data information of an information node does not satisfy a consistency condition according to an exemplary embodiment of the present application;

FIG. 8 is a schematic diagram illustrating the synchronous writing of data in an exemplary real-time embodiment of the present application;

FIG. 9 is a diagram illustrating synchronization of data between nodes according to an exemplary embodiment of the present application;

FIG. 10 is a block diagram of a data synchronous writing device according to an exemplary embodiment of the present application;

FIG. 11 is a block diagram illustrating the structure of a computer device according to an example embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the application, as detailed in the appended claims.

The embodiment of the application provides a data synchronous writing method, which can realize the concurrent execution of the writing operation of different data records on a plurality of nodes. For ease of understanding, the terms referred to in this application are explained below.

1) Storage system (memory system)

The storage system is a system in which a computer is composed of various storage devices for storing programs and data, a control unit, a device (hardware) for managing information scheduling, and an algorithm (software). The main memory of the computer can not meet the requirements of high access speed, large storage capacity and low cost at the same time, and a multi-level hierarchical memory with the speed from slow to fast and the capacity from large to small is required in the computer to form a storage system with acceptable performance by using an optimal control scheduling algorithm and reasonable cost.

In this application, a storage system includes at least two storage nodes, each storage node may store the same data record characteristics or a data area obtained by dividing data records in the storage node according to a predetermined rule, please refer to fig. 1, which shows a schematic diagram of a storage system according to an exemplary embodiment of the present application, as shown in fig. 1, the storage system 110 includes three nodes, which are respectively a node a, a node B, and a node C, each node separately maintains a set of data records, and data records in each node are the same. In the related art, in each node shown in fig. 1, data is divided into a plurality of areas according to a specific partition rule, please refer to fig. 2, which shows a schematic diagram of a storage system including a data area shown in an embodiment of the present application, as shown in fig. 2, the storage system 210 includes three nodes, namely, a node a, a node B, and a node C, the node a includes an area 1, an area 2, and an area 3, each area includes a plurality of data records, the node B and the node C include the same area 1, the same area 2, and the same area 3 as the node a, the same area on different nodes form an area copy group, only one area in each area copy group can provide a write service to the outside, for example, the area a can only provide a write service to the outside through the node a, the area 2 can only provide a write service to the outside through the node B, the area 3 intelligently provides a write service to the outside through the node C, for the area 1, because the node a, the node B, and the node C all include the area 1, when data is to be written into the area 1, it is necessary to provide a write service to the outside through the corresponding copy group through the corresponding area 1, and then the other areas, the same data area a corresponding to perform a parallel write operation, and then the other nodes can perform a parallel data write operation, and the other data, which supports the same data area.

2) Write service

The write service operates similar to the data update class in an operations database.

3) Data snapshot

A data snapshot, also called a database snapshot, refers to a condition of a database at a specific time, the data snapshot provides a read-only, static view at the time of creating the snapshot, and the database snapshot does not change once it is created.

Alternatively, the database snapshot always exists until deleted by the database proprietor.

FIG. 3 is a flowchart illustrating a method for synchronized writing of data for a storage system including at least two storage nodes configured to store the same data record, the method being performed by a first node that is any one of the at least two storage nodes, according to an exemplary embodiment; for example, the first node may be any of the nodes shown in fig. 1. As shown in fig. 1, the method may include the steps of:

step 310, receiving a first write request sent by a client, where the first write request is used to request a write operation on a first data record; the first data record is any data record stored in the storage system.

The first write request may be a write operation requesting to change the first data, a write operation adding new content to the first data, a write operation decreasing content in the first data, or the like.

Optionally, the first data record may be any one data record stored in the storage system, or may be any plurality of data records stored in the storage system, that is, the same write request may be used to perform a write operation on one data record in the storage system, or may be used to perform a write operation on a plurality of data records in the storage system.

Step 320, sending a first synchronization request including data information of the first node to the second node, where the data information includes global snapshot information of the first node and dependent version information of the first data record in the first node, the global snapshot information is used to indicate a write operation completed by a corresponding storage node, and the dependent version information is used to indicate a write operation completed last time by the first data record; the first synchronization request is used for indicating that the first data is written in the first data record on the second node; the second node comprises at least two storage nodes, each node except the first node; the first data is data corresponding to the first write request.

When a first node receives a first write request sent by a client, executing a pre-write operation, where the pre-write operation may be used to trigger the first node to send, to all nodes (second nodes) except the first node in a storage system, version-dependent information including global snapshot information of the first node and a first data record in the first node, where under a normal condition, under a condition that data of each node is consistent, the global snapshot information of each node is consistent with the version-dependent information of the first data record; however, in a possible case, due to network limitation or equipment failure, data synchronization between nodes may fail, so that there is a case where the global snapshot information between nodes and the dependent version information of the first data record are inconsistent at a certain time. After receiving the first synchronization request sent by the first node, the second node responds to the first synchronization request and sends first indication information to the first node.

Step 330, receiving first indication information sent by the second node, where the first indication information is used to indicate whether the data information of the first node and the data information of the second node are synchronized on the first data record.

The second node refers to all other nodes in the storage system except the first node that receives the first write request sent by the client, and taking the storage system shown in fig. 1 as an example, assuming that in the storage system, node a receives the first write request sent by the client, node a is the first node, and relatively, node B and node C are the second nodes; if the node B receives the first write request sent by the client, the node B is the first node, and relatively speaking, the node a and the node C are the second nodes.

For a first synchronization request containing data information of a first node sent by the first node, each node in the second nodes responds to the first synchronization request and sends first indication information, wherein the first indication information is used for indicating whether the data information of the first node is synchronized with the data information of each node in the second nodes, and the first indication information is sent to the node sending the first indication information. For example, in the storage system shown in fig. 1, after receiving the first write request, the node a sends a first synchronization request to the node B and the node C, both the node B and the node C send the first indication information in response to the first synchronization request, which is represented by that the node B sends the indication information indicating whether the data information in the node B is synchronized with the indication information in the node a to the node a, and the node C sends the indication information indicating whether the data information in the node C is synchronized with the indication information in the node a to the node a, so that the node a can obtain the indication information of all nodes in the storage system, and thus determine whether the data information of at least two storage nodes in the storage system satisfies the consistency condition. Step 340, in response to that the data information of the at least two storage nodes meets the consistency condition, writing first data in the first data record on the first node, and sending a first write notification to the second node, where the first write notification is used to notify that the first data is written in the first data record on the second node.

And the first node judges whether the data information of at least two storage nodes contained in the storage system meets the consistency condition according to the first indication information obtained in the step, and when the data information of at least two storage nodes contained in the storage system meets the consistency condition, the first node writes first data in a first data record on the first node. And meanwhile, sending a first write-in notification indicating that the second node writes the first data in the corresponding first data record to the second node, and correspondingly, for other nodes in the storage system, receiving the first write-in notification sent by the first node by the other nodes, and writing the first data in the first data record corresponding to each node of the second node according to the first write-in notification, so as to realize the synchronous write-in of the data in the storage system.

To sum up, in the data synchronous writing method provided in the embodiment of the present application, after receiving a first write request for performing a write operation on a first data record sent by a client, any node in a storage system sends a first synchronization request including data information of the first node to a second node, receives first indication information sent by the second node and used for indicating whether the data information of the first node and the data information of the second node are synchronized, and when the data information of each storage node in the storage system satisfies a consistency condition, writes first data in the first data record on the first node, and sends a first write notification to the second node. Because the write-in operation is the direct operation on the data records and the consistency judgment is carried out on different nodes before the write-in operation is carried out, the accuracy and consistency of data write-in among different nodes can be ensured, the write-in operation of different data records can be simultaneously carried out on a plurality of nodes, and the data write-in efficiency of the storage system is improved.

In the scheme shown in fig. 3, data writing in each node in the storage system is performed concurrently, that is, each node receives a synchronization request sent by another node and sends a synchronization request to another node at the same time. This process may be as shown in the embodiment shown in fig. 4.

FIG. 4 is a flowchart illustrating a method for synchronized writing of data for a storage system including at least two storage nodes configured to store the same data record, the method being performed by a first node that is any one of the at least two storage nodes, according to an exemplary embodiment; for example, the first node may be any of the nodes shown in fig. 1. As shown in fig. 1, the method may include the steps of:

step 410, receiving a first write request sent by a client, where the first write request is used to request to perform a write operation on a first data record; the first data record is any data record stored in the storage system.

In this embodiment of the present application, when a client needs to write data into a first data record in a storage system, the client may send the first write request to any storage node in the storage system.

Optionally, in this embodiment of the present application, the storage system may include an access node, where the access node is connected to the at least two storage nodes, and the client may send the first write request to the first node through the access node.

For example, when a client needs to write data into a first data record in the storage system, the client may send a first write request to an access node, where the access node selects one storage node from the storage nodes as the first node according to information such as load conditions of the storage nodes in the storage system (for example, processing resource utilization rates of the storage nodes), operating states of the storage nodes, and processing capabilities of the storage nodes, and distributes the first write request to the selected first node.

Optionally, the access node may be an access gateway device or a routing device of a storage system, and the implementation form of the access node is not limited in this embodiment of the application.

Step 420, assigning a globally unique write operation identifier in the storage system to the write operation corresponding to the first write request.

In order to ensure that all write operations can be completely and repeatedly synchronized to other nodes, a globally unique write operation identifier in the storage system is allocated to each write operation in the storage system, and the first write request has a globally unique write operation identifier corresponding to the first write request.

Optionally, the write operation identifier includes an originating node identifier and an operation sequence number; the initiating node identification is the identification of the storage node which initiates the corresponding write operation; the operation sequence number is a sequence number which is increased along with the initiating sequence of the write operation on the storage node corresponding to the initiating node identification.

Optionally, the write operation identifier corresponding to each write request may be allocated by the storage node that receives the write request and initiates the write operation.

For example, referring to fig. 5, which shows a corresponding schematic diagram of the write operation identifiers shown in an exemplary embodiment of the present application, as shown in fig. 5, in a storage system 510, for a write request initiated by a client through a node a, a write operation identifier composed of a corresponding node identifier and an operation sequence number corresponds to each write request, for example, a first write request initiated on the node a corresponds to a write operation identifier "A1", where a indicates that the write request is initiated on the node a, and 1 indicates that the write request is a first write request initiated in the node a, and similarly, for a second write request initiated on the node a, corresponds to a write operation identifier "A2", and the write operation identifiers in subsequent nodes a are incremented according to the rule, where the write operation identifiers initiated on the node a by the "A1", "A2", "A3", "A4", and the like are sequentially allocated by the node a, and the write operation identifiers corresponding to the write operation initiated on the node B may be sequentially allocated by the node a and the "B1", "B2", "B3", 823 ", and the like, and the write operation identifiers corresponding to the write operations initiated on the node B may be sequentially allocated by the" B1"," B2 ". The write operation identifiers corresponding to the write operations initiated on the node C can be sequentially represented as "C1", "C2" \8230, and the like, and the write operation identifiers initiated on the node C by the "C1", "C2", and the like are sequentially allocated by the node C.

Step 430, generating data information of the first node according to the last completed write operation of the first node.

Optionally, the data information may include a write operation identifier corresponding to a write operation that is completed by the first node according to the write operation request or the synchronization request last time.

Optionally, the data information of the first node includes global snapshot information of the first node and dependency version information of the first data record in the first node, where the global snapshot information is used to indicate a write operation completed by a corresponding storage node, and the dependency version information is used to indicate a write operation completed last time by the first data record.

In the embodiment of the present application, in addition to receiving a write request and synchronously writing to other storage nodes, any one storage node in a storage system may also receive synchronization initiated by the other storage nodes according to the write request. That is to say, the write operation that is completed last time on the first node may be a write operation that is initiated on the first node and writes to other storage nodes synchronously, or a write operation that is initiated on other nodes and writes to storage nodes including the first node synchronously.

For example, referring to fig. 6, which illustrates a schematic diagram of record snapshot information according to an exemplary embodiment of the present application, as shown in fig. 6, in a storage system 610, for a node a, record snapshot information of a data record numbered R1 is write operation A1 and write operation A2, record snapshot information of a data record numbered R2 is write operation A2, record snapshot information of a data record numbered R3 is write operation A1, record snapshot information of a data record numbered R4 is write operation A2 and write operation B1, record snapshot information of a data record numbered R5 is write operation C1 \8230303030303030, where a write operation that is completed most recently in the node a is write operation identified as B1 performed on the data record numbered R4. When a node a receives a latest first write operation request sent by a client, before a write operation is not performed according to the latest first write operation request, data information of a current first node is generated, where the data information includes global snapshot information of the first node and dependent version information of a first data record in the first node, where the write operation that is completed last time in the node a is a write operation identified as B1 performed on a data record with a number of R4, the global snapshot information of the first node is (A1, A2, B1, C1), and if the latest first write operation request sent by the received client is a write operation performed on a data record with a number of R3, that is, the data record with a number of R3 is the first data record, the dependent version information of the first data record is A1; for the node B, the content indicated by the record snapshot information is that, in the write operations completed in the node B, the record snapshot information of the data record with the number R1 is the write operations A1 and A2, the record snapshot information of the data record with the number R2 is the write operations A2 and B2, the record snapshot information of the data record with the number R3 is the write operation A1, the record snapshot information of the data record with the number R4 is the write operations A2 and B1, and the record snapshot information of the data record with the number R5 is the write operation C1 \8230, wherein the write operation completed most recently in the node B is the write operation performed on the data record with the number R2 and is identified as B2, and the corresponding global snapshot information is (A1, A2, B1, B2, C1).

That is to say, for each node, after a certain node in the storage system receives a first write request sent by a client, each node in the storage nodes generates recorded data information and global snapshot information corresponding to the node, and for a first node acted by a first write request sent by the client, the first node acted by the first write request generates version-dependent version information according to the last write operation of a first data record in the first node acted by the first write request.

Step 440 sends a first synchronization request to the second node containing the data information of the first node. The first synchronization request is used for indicating that the first data is written in the first data record on the second node; the second node comprises at least two storage nodes, each node except the first node; the first data is data corresponding to the first write request.

Optionally, in response to completion or failure of sending a previous synchronization request in the first node, the first node performs a step of sending a first synchronization request including data information of the first node to the second node;

In this embodiment of the present application, since any storage node in the storage system may receive a write request for any data record, in this process, two or more write requests that are concurrent for the same data record may occur on the same storage node, and at this time, if write operations corresponding to the two or more write requests are processed simultaneously, a situation that the sequence of multiple write operations for the same data record on each storage node is inconsistent may occur, so that data is not synchronized between each storage node.

For example, assuming that two write requests corresponding to the data record R1 are concurrently issued on the node a, the two write requests respectively correspond to the write operation A1 and the write operation A2, if the node a simultaneously initiates synchronization to the write operation A1 and the write operation A2 to other storage nodes (taking the node B as an example), the following situation may occur: on the node a, the processing speed of the write operation A1 is fast, the processing speed of the write operation A2 is slow, the node a completes the write operation A1 on the data record R1 first, and then completes the write operation A2 on the data record R1, while on the node B, the processing speed of the write operation A2 may be fast, the processing speed of the write operation A1 is slow, the node B completes the write operation A2 on the data record R1 first, and then completes the write operation A1 on the data record R1, which may cause inconsistency between the data records R1 stored on the node a and the node B.

In order to solve the above problem, in this embodiment of the present application, for multiple write requests for the same data record received by a first node, the first node may initiate synchronization to other nodes in sequence according to the sequence of operation sequence numbers. That is to say, in the embodiment of the present application, for multiple write requests for the same data record that are concurrently sent to the same storage node, when the storage node performs write synchronization, the storage node sequentially synchronizes, according to the sequence of the operation sequence numbers, write operations corresponding to the multiple write requests for the same data record to other storage nodes.

That is, on the first node, if the action objects of the write operations corresponding to the previous write operation and the first write request are the same data record, the first node will perform a new round of data write synchronization only when the previous write operation is completed or fails. For example, for the node a shown in fig. 5, if the write operation identifier corresponding to the previous write operation in the node a is "A2", and the write operation identifier corresponding to the write operation corresponding to the first write request is "A3", it indicates that the write operation corresponding to the first write request and the previous write operation are both initiated on the node a, and the write operation corresponding to the first write request is immediately before the previous write operation, and if the write operation A2 and the write operation A3 both act on the same data record, for example, the write operation A2 and the write operation A3 both act on the data record R1, the first node performs the step of sending the first synchronization request including the data information of the first node to the second node after the previous write operation is performed, regardless of whether the previous write operation is successfully performed, so as to satisfy the consistency of the data records of the respective nodes.

And for multiple write requests received by the first node for different data records, the first node may initiate synchronization to other nodes concurrently. That is to say, in the embodiment of the present application, for multiple concurrent write requests for different data records on the same storage node, when the storage node performs write synchronization, the storage node may synchronize write operations corresponding to the multiple write requests for different data records to other storage nodes.

That is, on the first node, if the action objects of the write operations corresponding to the previous write operation and the first write request are different data records, the write operations corresponding to the previous write operation and the first write request may be synchronized concurrently. For example, for the node a shown in fig. 5, if the write operation identifier corresponding to the previous write operation in the node a is "A2" and the write operation identifier corresponding to the write operation corresponding to the first write request is "A3", it indicates that the write operation corresponding to the first write request and the previous write operation are both initiated on the node a and the write operation corresponding to the first write request is immediately before the previous write operation, and if the write operation A2 and the write operation A3 act on different data records, for example, the write operation A2 acts on the data record R2 and the write operation A3 acts on the data record R3, the first node may perform a step of sending a first synchronization request including data information of the first node to the second node regardless of whether the previous write operation is performed, so as to improve the synchronization efficiency of the data records.

Optionally, the second node includes n nodes, where n is an integer greater than or equal to 2. Optionally, in the embodiment of the present application, the value of n may be an even number.

Optionally, the data information includes a write operation identifier of a write operation that the corresponding storage node has completed last time.

Step 450, receiving first indication information sent by the second node, where the first indication information is used to indicate whether the data information of the first node and the data information of the second node are synchronized.

The indication information includes a consistency identifier of the corresponding node, that is, the other nodes generate the consistency identifier after performing synchronization judgment on the data information of the first node, the own data information and the recorded snapshot information, included in the first synchronization information sent by the first node, so as to indicate whether the data information of the first node and the own data information are synchronized.

For example, in the storage system shown in fig. 6, taking node a as a first node as an example, when node a receives a first write request based on a data record R2 sent by a client, the first node (node a) sends a first synchronization request including global snapshot information of the first node and dependent version information of a first data record (data record R2) in the first node, that is, a first synchronization request including global snapshot information (A1, A2, B1, C1) and dependent version information A2, to a second node (taking node B as an example), and after receiving the first synchronization request, node B performs a determination on whether to synchronize with its own data information and record snapshot information according to the received first synchronization request, where the determination process may be implemented as follows:

1) And obtaining the dependent version information of the first data record in the first node, which is contained in the data information of the first node.

2) And responding to the fact that the dependent version information of the first data record in the first node exists in the global snapshot information of the second node, and obtaining record snapshot information of the first data record in the second node.

3) In response to the record snapshot information of the first data record of the second node being a subset of the global snapshot information of the first node, determining the data information of the first node to be synchronized with the data information of the second node on the first data information.

Optionally, in response to that the dependency version information corresponding to the first data record in the first node does not exist in the global snapshot information of the second node, or that the record snapshot information corresponding to the first data record of the second node is not a subset of the global snapshot information of the first node, it is determined that the data information of the first node and the data information of the second node are not synchronized on the first data information.

For example, in the node a (first node) and the node B (second node) shown in fig. 6, the dependency version information corresponding to the first data record (data record R2) of the node a is A2, the global snapshot information of the node a is (A1, A2, B1, C1), the record snapshot information corresponding to the first data record (data record R2) of the node B is (A2, B2), and the global snapshot information of the node B is (A1, A2, B1, B2, C1). Since the dependency version information A2 corresponding to the first data record of the node a exists in the global snapshot information (A1, A2, B1, B2, C1) of the node B, the synchronous check of the global snapshot information of the first node and the record snapshot information corresponding to the first data record of the second node is performed, and since B2 in the record snapshot information (A2, B2) corresponding to the first data record of the node B is not a subset of the global snapshot information of the node a, it can be determined that the data information of the node a is not synchronous with the data information of the node B on the data record R2, and the modification operation conflicts.

In the storage system shown in fig. 6, taking node a as the first node as an example, when node a receives the first write request based on the data record R3 from the client, the first synchronization request sent by node a to node B includes the global snapshot information (A1, A2, B1, C1) and the dependent version information A1 of node a, after node B receives the first synchronization request, node B determines that the dependent version information A1 of the data record R3 of node a exists in the global snapshot information (A1, A2, B1, B2, C1) of node B, and the record snapshot information A1 corresponding to the data record R3 of node B is a subset of the global snapshot information (A1, A2, B1, C1) of node a, so that it can be determined that the data information of node a is synchronized with the data information of node B on the data record R3.

Step 460, in response to the occupation ratio of the consistency node in the n nodes exceeding the occupation ratio threshold, determining that the data information of at least two storage nodes meets the consistency condition.

The consistency node is a storage node which is synchronous with the corresponding data information and the data information of the first node. The synchronization of the data information corresponding to the two nodes may mean that the writing operation identifiers indicated by the data information corresponding to the two nodes are the same.

Optionally, the ratio threshold is greater than or equal to 50%, and the ratio threshold is less than or equal to 100%. For example, when the percentage threshold is 50%, the data information of at least two storage nodes meets the consistency condition, which means that the data records of more than half of the storage nodes in all of the at least two storage nodes are required to be synchronized.

For example, for the storage system shown in fig. 1, if the node a is the first node that sends the first synchronization request, the occupancy of the consistency node is determined by the data information in the node B and the node C, if the data information of the node B and the node C is not synchronized with the data information of the node a, the occupancy of the consistency node is 0, if the data information of any one of the node B and the node C is synchronized with the data information of the node a, the occupancy of the consistency node is 50%, and if the data information of the node B and the node C is synchronized with the data information of the node a, the occupancy of the consistency node is 100%. If the percentage threshold is set to 50%, the data information of at least one of the node B and the node C is synchronous with the data information of the node A, and the data information of the storage nodes in the storage system can be determined to meet the consistency condition; if the percentage threshold is set to 100%, it can be determined that the data information of the storage nodes in the storage system satisfies the consistency condition only when the data information of both the node B and the node C is synchronized with the data information of the node a.

Step 470, in response to that the data information of the at least two storage nodes meets the consistency condition, writing the first data in the first data record on the first node, and sending a first write notification to the second node, where the first write notification is used to notify that the first data is written in the first data record on the second node.

Meanwhile, for the second node, in response to the fact that the data information of the at least two storage nodes meets the consistency condition, a first write notification sent by the first node is received, and according to the first write notification, each node in the second node writes the first data in the corresponding first data record according to the first write notification.

By the scheme, the synchronous write operation sequence executed in other nodes is also carried out according to the execution sequence of the write operation in the first node.

Corresponding to a source node initiating a synchronization request, for write operations concurrently executed in the same node, if the same record is modified, the write operations may be executed serially in a manner of locking or the like on the source node, for example, in a possible case, two consecutive write operations are both initiated by a node a, and both the two write operations act on the same data record, then a previous write operation A3 is initiated on the node a, after the write operation A3 is completed synchronously in each storage node, the node a initiates a write operation A4, so that each storage node writes the write operation A4 synchronously, and other storage nodes sequentially synchronize the write operations A3 and A4 according to a numbering sequence.

Or, in a possible case, two consecutive write operations are both initiated by the node a, but the two write operations act on different data records, then the node a may initiate the write operation A3 and the write operation A4 at the same time, and accordingly, the write operations of other storage nodes in different data records may be executed concurrently, so as to improve the write efficiency of the data records.

Optionally, in response to that the data information of the at least two storage nodes does not satisfy the consistency condition, a rollback notification is sent to the second node, where the rollback notification is used to instruct the second node to roll back to a state before the first synchronization request is received.

Optionally, after the second node receives the first synchronization request issued by the first node, a pre-write operation corresponding to the first synchronization request may be performed at the second node, in a subsequent consistency condition judgment process, if the data information of the at least two storage nodes in the storage system satisfies the consistency condition, the write operation in the pre-write operation is performed, and if the data information of the at least two storage nodes in the storage system does not satisfy the consistency condition, a rollback notification is sent to the second node to indicate that the second node rolls back to a state before receiving the first synchronization request, that is, the state before receiving the first synchronization request is not performed, that is, the content of the pre-write operation performed in the second node is deleted.

In the case that the data information of the storage node does not satisfy the consistency condition, refer to fig. 7, which shows a schematic diagram that the data information of the information node related to the embodiment of the present application does not satisfy the consistency condition, as shown in fig. 7, record snapshot information of the node a shows that the last operation in the node a at the current time is that a first write operation B1 initiated by the node B is performed on a data record numbered as R2, while data display information of the corresponding node B shows that the last operation in the node B at the current time is that a second write operation B3 initiated by the node B is performed on a data record numbered as R2, and if the data information of other nodes in the storage system that satisfy the duty threshold is not synchronized with the data information in the node a, it may be determined that the data information of at least two storage nodes included in the storage system does not satisfy the consistency condition, thereby triggering the first node to return a write failure response to the client.

In addition, the first node may also receive a second synchronization request sent by another node, and execute, in the first node, a write operation corresponding to the second synchronization request, where the execution process may be as follows:

1) Receiving a second synchronization request containing data information of a third node, wherein the second synchronization request is used for indicating that second data is written in a second data record on the first node; the third node is any one of the at least two storage nodes except the first node; the second data record is any data record stored in the storage system.

2) And acquiring the dependent version information of the second data record in the third node, which is contained in the data information of the third node.

3) And in response to the dependency version information of the second data record in the third node existing in the global snapshot information of the first node, obtaining record snapshot information of the second data record in the first node, wherein the record snapshot information is used for indicating write operation completed in the corresponding data record.

4) In response to the record snapshot information of the second data record in the first node being a subset of the global snapshot information of the third node, determining the data information of the first node to be synchronized with the data information of the third node on the second data record.

5) And sending second indication information to the third node, wherein the second indication information is used for indicating whether the data information of the third node and the data information of the first node are synchronous on the second data record.

6) In response to receiving a second writing notification returned by the third node, writing second data in a second data record on the first node; and the second write-in notification is sent when the third node determines that the data information of the at least two storage nodes meets the consistency condition according to the second indication information. Optionally, in response to the end of execution of a third synchronization request sent by the third node, sending second indication information to the third node, and determining whether the data information of at least two storage nodes meets a consistency condition; that is to say, after the third synchronization request sent by the third node is executed, the second synchronization request is started to be synchronously written, that is, whether the data information of the first node and the third node is synchronized on the second data record is judged, and the indication information is fed back to the third node. And the receiving time of the third synchronization request is before the receiving time of the second synchronization request, and the third synchronization request is used for indicating that third data is written in the second data record of the first node.

The first node may also receive second synchronization requests sent by other nodes, and the implementation process of executing the write operation corresponding to the second synchronization request in the first node may refer to the implementation process of receiving, by the second node, the first synchronization request sent by the first node and executing, in the second node, the write operation corresponding to the first synchronization request, which is not described herein again.

Optionally, in response to that the write operation corresponding to the second write request is successfully synchronized to the at least two storage nodes, the write operation identifier of the write operation corresponding to the second write request is deleted from the record snapshot information of the data record in the first node.

Because the snapshot information is recorded for recording the historical write operation in the corresponding data record, when it is determined that the write operation corresponding to the write request is successfully synchronized in all the storage nodes in the storage system, that is, it indicates that the historical write operations of the storage nodes in the storage system in the data record before the write operation are the same, when the next write request comes, the determined synchronized historical write operation still needs to be determined again. Therefore, it is necessary to delete the determined synchronized record snapshot information, that is, in response to the successful synchronization of the write operation corresponding to the write request to other storage nodes in the storage system, the write operation identifier of the write operation corresponding to the write request may be deleted from the record snapshot information of the data record in the source node.

For example, as shown in the record snapshot information shown in fig. 6, in the data record R2, the write operation A2 in the node a is already synchronized to all storage nodes, that is, the write operation A2 may be deleted from the record snapshot information of the data records R2 of all storage nodes, and the write operation B2 in the node B is not synchronized to all storage nodes, and needs to be retained, so that the data record information corresponding to the data record R2 of the node a after the write operation identifier of the synchronized write operation is deleted is null, and the data record information corresponding to the data record R2 of the node B after the write operation identifier of the synchronized write operation is deleted is B2.

Referring to fig. 8, which shows a schematic diagram of data synchronous writing illustrated in an exemplary real-time example of the present application, as shown in fig. 8, different clients may perform write operations on the same or different data records in different nodes, for example, a client 1 performs a write operation T1 in a node a to perform a write operation on data records corresponding to numbers R1 and R3; the client 2 executes a write operation T2 in the node B to write data records corresponding to the numbers R2 and R5; the client 3 executes write operation T3 in the node C so as to record write operation on the data corresponding to the number R4; the three write operations T1, T2, and T3 may be executed simultaneously, the write operation T1 initiated at the node a may be synchronized to the node B and the node C, similarly, the write operation T2 initiated at the node B may also be synchronized to the node a and the node C, the write operation T3 initiated at the node C may also be synchronized to the node a and the node B, and finally, the three write operations T1, T2, and T3 may be executed at all the nodes.

To ensure data consistency, multiple write operations initiated from the same node may be synchronized to other nodes according to an execution sequence in the node initiating the multiple write operations, for example, please refer to fig. 9, which shows a schematic diagram of data synchronization between nodes according to an exemplary embodiment of the present application. As shown in fig. 9, for a plurality of write operations A1, A2, A3, and A4 based on the same data record from node a, in order to ensure data consistency, synchronization needs to be performed in the order of execution in node a, that is, in the order of A2, A3, and A4 during data synchronization.

For example, taking the node a initiating the write operation A3 as an example, the node a receives a write request of the client for instructing to perform the write operation A3 based on the data record R1 in the node a, after receiving the write request, the node a sends a first synchronization request including current data information of the node a to the node B and the node C, and the first synchronization request includes a consistency identifier for instructing whether the write operation indicated by the data information of the node B is synchronous with the write operation indicated by the data information of the node a or not, and after receiving the first synchronization request sent by the node a, the node B and the node C may send consistency identifiers for instructing whether the write operation indicated by the data information of the node B is synchronous with the write operation indicated by the data information of the node a to the node a and the node C, and similarly, the node C may send consistency identifiers for instructing whether the write operation indicated by the data information of the node C is synchronous with the write operation indicated by the data information of the node a to the node a and the node B, that the data information of the node a and the node C do not satisfy the storage system consistency judgment condition that the storage system is not satisfied, and that the storage system fails to perform the storage system, that the storage system is not satisfied.

If after the write operation A3 is successfully executed in the node of the storage system, the node a initiates a write operation A4 based on the data record R1, the node a receives a write request from the client to indicate that the write operation A4 is performed in the node a, the node a sends a first synchronization request including current data information of the node a (the current data information of the node a is data information after the write operation A3 is executed) to the node B and the node C after receiving the write request, and the first synchronization request includes a write operation A4 to indicate that the write operation A4 is executed on the node B and the node C, which is similar to a process in which the node a initiates the write operation A3 in the storage system and is not described herein again.

To sum up, in the data synchronous writing method provided in the embodiment of the present application, after receiving a first write request for performing a write operation on a first data record sent by a client, any node in a storage system sends a first synchronization request including data information of the first node to a second node, receives first indication information sent by the second node and used for indicating whether the data information of the first node and the data information of the second node are synchronized, and when the data information of each storage node in the storage system satisfies a consistency condition, writes first data in the first data record on the first node, and sends a first write notification to the second node. Because the write-in operation is the direct operation on the data record and the consistency judgment is carried out on different nodes before the write-in operation is carried out, the accuracy and the consistency of data write-in among different nodes can be ensured, the write-in operation of different data records can be simultaneously executed on a plurality of nodes, and the data write-in efficiency of the storage system is improved.

Fig. 10 is a block diagram of a data synchronous writing apparatus provided in an exemplary embodiment of the present application, where the apparatus is used in a storage system, where the storage system includes at least two storage nodes, the at least two storage nodes are used for storing the same data record, and the apparatus is applied in a first node, where the first node is any one of the at least two storage nodes; the apparatus may include:

a first receiving module 1010, configured to receive a first write request sent by a client, where the first write request is used to request to perform a write operation on a first data record; the first data record is any data record stored in the storage system;

a first sending module 1020, configured to send a first synchronization request including data information of a first node to a second node, where the data information of the first node includes global snapshot information of the first node and dependent version information of a first data record in the first node, the global snapshot information is used to indicate a write operation completed by a corresponding storage node, and the dependent version information is used to indicate a write operation completed by the corresponding data record for the last time; the first synchronization request is used for indicating that the first data is written in the first data record on the second node; the second node comprises at least two storage nodes, wherein each node except the first node; the first data is data corresponding to the first write request;

a second receiving module 1030, configured to receive first indication information sent by a second node, where the first indication information is used to indicate whether data information of the first node and data information of the second node are synchronized on a first data record;

the first writing module 1040 is configured to, in response to that the data information of the at least two storage nodes meets the consistency condition, write first data in the first data record on the first node, and send a first write notification to the second node, where the first write notification is used to notify that the first data is written in the first data record on the second node.

Optionally, the second node includes n nodes, where n is an integer greater than or equal to 2;

the device also comprises:

the first determining module is used for responding to the fact that the occupation ratio of a consistency node in the n nodes exceeds the occupation ratio threshold value, and determining that the data information of at least two storage nodes meets the consistency condition;

the consistency node is corresponding data information, and is a storage node synchronized with the data information of the first node on the first data record.

and the consistency identification is used for indicating whether the write operation indicated by the data information of the corresponding node is synchronous with the write operation indicated by the data information of the first node.

Optionally, the apparatus further includes:

and the return module is used for responding to the condition that the data information of the at least two storage nodes does not meet the consistency condition, and sending a rollback notification to the second node, wherein the rollback notification is used for indicating the second node to roll back to a state before the first synchronization request is received.

the device also includes:

the allocating module is configured to allocate a globally unique write operation identifier in the storage system to a write operation corresponding to the first write request before the first sending module 1010 sends the first synchronization request including the data information of the first node to the second node.

Optionally, the write operation identifier includes an originating node identifier and an operation sequence number; the initiating node identification is the identification of the storage node which initiates the corresponding write operation; the operation sequence number is a sequence number which is increased along with the initiation sequence of the write operation on the storage node corresponding to the initiation node identifier.

Optionally, the first sending module 1010 is configured to, in response to completion or failure of sending a previous synchronization request in the first node, execute a step of sending a first synchronization request including data information of the first node to the second node;

and the identification of the initiating node of the previous synchronous request is the same as the identification of the initiating node of the write operation corresponding to the first write request.

Optionally, the apparatus further comprises:

a third receiving module, configured to receive a second synchronization request including data information of a third node, where the second synchronization request is used to instruct to write second data in a second data record on the first node; the third node is any one of the at least two storage nodes except the first node; the second data record is any data record stored in the storage system;

the second sending module is used for sending second indication information to the third node, wherein the second indication information is used for indicating whether the data information of the third node and the data information of the first node are synchronous on a second data record;

the second writing module is used for writing second data in a second data record on the first node in response to receiving a second writing notification returned by the third node; and the second writing notification is sent when the third node determines that the data information of the at least two storage nodes meets the consistency condition according to the second indication information.

Optionally, the apparatus further comprises:

the first obtaining module is used for obtaining the dependent version information of the second data record in the third node, which is contained in the data information of the third node;

a second obtaining module, configured to obtain record snapshot information of a second data record in a first node in response to that dependency version information of the second data record in a third node exists in global snapshot information of the first node, where the record snapshot information is used to indicate write operation completed in the corresponding data record;

and the second determining module is used for determining the data information of the first node to be synchronous with the data information of the third node on the second data record in response to the record snapshot information of the second data record in the first node being the subset of the global snapshot information of the third node.

Optionally, the apparatus further includes:

and a third determining module, configured to determine that the dependency version information of the second data record in the third node does not exist in the global snapshot information of the first node, or that the record snapshot information of the second data record in the first node is not a subset of the global snapshot information of the third node, and that the data information of the first node is not synchronized with the data information of the third node on the second data record.

Optionally, the apparatus further includes:

wherein the receiving time of the third synchronization request is before the receiving time of the second synchronization request, and the third synchronization request is used for indicating that the third data is written in the second data record on the first node.

Optionally, the apparatus further includes:

and the deleting module is used for responding to the successful synchronization of the write operation corresponding to the second write request to at least two storage nodes, and deleting the write operation identifier of the write operation corresponding to the second write request from the record snapshot information of the data record in the first node.

To sum up, the data synchronization writing device provided in the embodiment of the present application is applied to a storage system, so that after receiving a first write request that a client sends a write operation to a first data record, any node in the storage system sends a first synchronization request including data information of the first node to a second node, receives first indication information that is sent by the second node and used for indicating whether the data information of the first node and the data information of the second node are synchronized, and when the data information of each storage node in the storage system satisfies a consistency condition, writes first data in the first data record on the first node, and sends a first write notification to the second node. Because the write-in operation is the direct operation on the data records and the consistency judgment is carried out on different nodes before the write-in operation is carried out, the accuracy and consistency of data write-in among different nodes can be ensured, the write-in operation of different data records can be simultaneously carried out on a plurality of nodes, and the data write-in efficiency of the storage system is improved.

FIG. 11 is a block diagram illustrating the architecture of a computer device 1100 in accordance with an exemplary embodiment. The computer device may be implemented as a server in the above-mentioned aspects of the present application. The computer device 1100 includes a Central Processing Unit (CPU) 1101, a system Memory 1104 including a Random Access Memory (RAM) 1102 and a Read-Only Memory (ROM) 1103, and a system bus 1105 connecting the system Memory 1104 and the CPU 1101. The computer device 1100 also includes a basic Input/Output system (I/O system) 1106, which facilitates transfer of information between devices within the computer, and a mass storage device 1107 for storing an operating system 1113, application programs 1114, and other program modules 1115.

The basic input/output system 1106 includes a display 1108 for displaying information and an input device 1109 such as a mouse, keyboard, etc. for user input of information. Wherein the display 1108 and input device 1109 are connected to the central processing unit 1101 through an input output controller 1110 connected to the system bus 1105. The basic input/output system 1106 may also include an input/output controller 1110 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input-output controller 1110 also provides output to a display screen, a printer, or other type of output device.

The mass storage device 1107 is connected to the central processing unit 1101 through a mass storage controller (not shown) that is connected to the system bus 1105. The mass storage device 1107 and its associated computer-readable media provide non-volatile storage for the computer device 1100. That is, the mass storage device 1107 may include a computer-readable medium (not shown) such as a hard disk or Compact disk-only memory (CD-ROM) drive.

Without loss of generality, the computer-readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, erasable Programmable Read-Only Memory (EPROM), electrically Erasable Programmable Read-Only Memory (EEPROM), flash Memory or other solid state Memory technology, CD-ROM, digital Versatile Disks (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer storage media is not limited to the foregoing. The system memory 1104 and mass storage device 1107 described above may be collectively referred to as memory.

The computer device 1100 may also operate as a remote computer connected to a network via a network, such as the internet, in accordance with various embodiments of the present disclosure. That is, the computer device 1100 may connect to the network 1112 through the network interface unit 1111 connected to the system bus 1105, or may connect to other types of networks or remote computer systems (not shown) using the network interface unit 1111.

The memory further includes at least one instruction, at least one program, a code set, or a set of instructions, where the at least one instruction, at least one program, a code set, or a set of instructions is stored in the memory, and the central processor 1101 implements all or part of the steps performed by the nodes in the storage system in the data synchronous writing method shown in the foregoing embodiments by executing the at least one instruction, at least one program, a code set, or a set of instructions.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It will be understood that the present application is not limited to the precise arrangements that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A data synchronous writing method is characterized in that the method is used for a storage system, the storage system comprises an access node and at least two storage nodes, the access node is connected with the at least two storage nodes, the at least two storage nodes are used for storing the same data record, the method is executed by a first node, and the first node is a node determined by the access node according to the load condition, the working state and the processing capacity of each storage node in the storage system; the method comprises the following steps:

synchronously sending a first synchronization request containing data information of a first node to each storage node in a second node, wherein the second node comprises all storage nodes except the first node in the storage system, the data information of the first node comprises global snapshot information of the first node and dependent version information of the first data record in the first node, the global snapshot information is used for indicating write operation completed by the corresponding storage node, and the dependent version information is used for indicating write operation completed by the corresponding data record at the latest time; the first synchronization request is used to instruct writing of first data in the first data record on the second node; the first data is data corresponding to the first write request;

receiving first indication information sent by each storage node in the second node, where the first indication information is used to indicate whether the data information of the first node and the data information and record snapshot information of each storage node in the second node satisfy a synchronization condition, where the synchronization condition refers to that dependent version information of the first data record in the first node exists in the global snapshot information of the second node, and the record snapshot information of the first data record in the second node is a subset of the global snapshot information of the first node, and the record snapshot information is used to indicate a write operation completed in a corresponding data record;

in response to the data information of the at least two storage nodes meeting a consistency condition, writing the first data in the first data record on the first node, and simultaneously sending a first write notification to the second node, wherein the first write notification is used for notifying that the first data is written in the first data record on the second node;

receiving a second synchronization request containing data information of a third node, wherein the second synchronization request is used for indicating that second data is written in a second data record on the first node; the third node is any node in the storage system other than the first node; the second data record is any data record stored in the storage system;

sending second indication information to the third node, wherein the second indication information is used for indicating whether the data information of the third node and the data information of the first node are synchronous on the second data record;

in response to receiving a second write notification returned by the third node, writing the second data in the second data record on the first node; and the second write-in notification is sent when the third node determines that the data information of the at least two storage nodes meets the consistency condition according to the second indication information.

2. The method of claim 1, wherein the second node comprises n nodes, n being an integer greater than or equal to 2;

before the writing the first data in the first data record on the first node, further comprising:

in response to the occupation ratio of a consistency node in the n nodes exceeding an occupation ratio threshold, determining that the data information of the at least two storage nodes meets the consistency condition;

3. The method of claim 2, wherein the first indication information comprises indication information sent by each of the n nodes; the indication information includes:

and the consistency identification of the corresponding node is used for indicating whether the write operation indicated by the data information of the corresponding node is synchronous with the write operation indicated by the data information of the first node on the first data record.

4. The method of claim 1, further comprising:

in response to the data information of the at least two storage nodes not meeting the consistency condition, sending a rollback notification to the second node, the rollback notification being used to instruct the second node to rollback to a state before receiving the first synchronization request.

5. The method according to claim 1, wherein the data information includes a write operation identifier of a write operation that was last completed by a corresponding first data record;

before the sending the first synchronization request containing the data information of the first node to the second node, the method further includes:

and distributing globally unique write operation identification in the storage system for the write operation corresponding to the first write request.

6. The method of claim 5, wherein the write operation identifier comprises an originating node identifier and an operation sequence number; the initiating node identification is the identification of the storage node which initiates the corresponding write operation; the operation sequence number is a sequence number which is increased with the initiating sequence of the write operation on the storage node corresponding to the initiating node identification.

7. The method of claim 6, wherein sending a first synchronization request containing data information of the first node to a second node comprises:

in response to completion or failure of a previous synchronization request transmission in the first node, performing the step of transmitting a first synchronization request including data information of the first node to a second node;

8. The method of claim 1, wherein before sending the second indication information to the third node, further comprising:

acquiring the dependent version information of the second data record in the third node, wherein the dependent version information is contained in the data information of the third node;

responding to the dependency version information of the second data record in the third node existing in the global snapshot information of the first node, and acquiring record snapshot information of the second data record in the first node;

determining data information of the first node to be synchronized with data information of the third node on the second data record in response to record snapshot information of the second data record in the first node being a subset of global snapshot information of the third node.

9. The method of claim 8, further comprising:

determining that the data information of the first node is not synchronized with the data information of the third node on the second data record in response to the dependent version information of the second data record in the third node not being present in the global snapshot information of the first node or the record snapshot information of the second data record in the first node not being a subset of the global snapshot information of the third node.

10. The method of claim 8, wherein before sending the second indication information to the third node, further comprising:

responding to the end of execution of a third synchronization request sent by the third node, and sending second indication information to the third node;

wherein a receiving time of the third synchronization request is before a receiving time of the second synchronization request, and the third synchronization request is used to instruct to write third data in the second data record on the first node.

11. The method of claim 8, further comprising:

and responding to the successful synchronization of the write operation corresponding to the second write request to the at least two storage nodes, and deleting the write operation identifier of the write operation corresponding to the second write request from the record snapshot information of the data record in the first node.

12. A data synchronous writing device is characterized in that the device is used for a storage system, the storage system comprises an access node and at least two storage nodes, the access node is connected with the at least two storage nodes, the at least two storage nodes are used for storing the same data record, the device is applied to a first node, and the first node is a node determined by the access node according to the load condition, the working state and the processing capacity of each storage node in the storage system; the device comprises:

a first sending module, configured to synchronously send a first synchronization request including data information of a first node to each storage node in a second node, where the second node includes all storage nodes except the first node in the storage system, the data information of the first node includes global snapshot information of the first node and dependent version information of the first data record in the first node, the global snapshot information is used to indicate a write operation completed by a corresponding storage node, and the dependent version information is used to indicate a write operation completed by the corresponding data record at the latest time; the first synchronization request is used for indicating that first data is written in the first data record on the second node; the first data is data corresponding to the first write request;

a second receiving module, configured to receive first indication information sent by each storage node in the second node, where the first indication information is used to indicate whether data information of the first node and data information and record snapshot information of each storage node in the second node satisfy a synchronization condition, where the synchronization condition indicates that dependent version information of the first data record in the first node exists in global snapshot information of the second node, and the record snapshot information of the first data record in the second node is a subset of the global snapshot information of the first node, and the record snapshot information is used to indicate a write operation completed in a corresponding data record;

a first writing module, configured to, in response to that data information of the at least two storage nodes meets a consistency condition, write the first data in the first data record on the first node, and simultaneously send a first writing notification to the second node, where the first writing notification is used to notify that the first data is written in the first data record on the second node;

a third receiving module, configured to receive a second synchronization request including data information of a third node, where the second synchronization request is used to instruct to write second data in a second data record on the first node; the third node is any node in the storage system other than the first node; the second data record is any data record stored in the storage system;

a second writing module, configured to write the second data in the second data record on the first node in response to receiving a second write notification returned by the third node; and the second write-in notification is sent when the third node determines that the data information of the at least two storage nodes meets the consistency condition according to the second indication information.

13. A computer device comprising a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the data synchronous writing method according to any one of claims 1 to 11.

14. A computer-readable storage medium, wherein at least one program is stored in the storage medium, and the at least one program is loaded and executed by a processor to implement the data synchronous writing method according to any one of claims 1 to 11.