CN110737716A

CN110737716A - data writing method and device

Info

Publication number: CN110737716A
Application number: CN201810706826.8A
Authority: CN
Inventors: 戴冠群
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2018-07-02
Filing date: 2018-07-02
Publication date: 2020-01-31
Anticipated expiration: 2038-07-02
Also published as: CN110737716B

Abstract

The method comprises the steps that a main node stores N sub-logs of a log to the local, the log is generated according to atomic write requests, the atomic write requests comprise N sub-writes, information of the sub-writes is recorded in the sub-logs, N is a positive integer larger than 0, the main node backs up the N sub-logs to a standby node, and the main node marks the logs as effective under the condition that the N sub-logs are all stored in the standby node.

Description

data writing method and device

Technical Field

The present application relates to the field of storage, and more particularly, to a method and apparatus for data writes.

Background

In the storage system, in order to prevent data loss caused by hardware or storage media failure of an operating system, locally stored data can be backed up to a standby node, a file system is a software mechanism for managing and storing file information in the operating system and is mainly responsible for establishing files, storing, reading and dumping files, controlling access of the files and the like for a user, in the process of providing a storage service for data backup, the file system mainly prevents data from being caused or damaged by power failure or system crash through atomic operation (namely, groups of write requests need to be guaranteed to be completely successful or completely failed), for example, Network Attached Storage (NAS) can be types of efficient and stable file systems for storage equipment.

In the prior art, in the process of data backup, in order to ensure atomic operation among a plurality of nodes in a file system, a semantic lock needs to be added in the file system, and in the process of data backup, group write request atomic operation is ensured through the semantic lock.

Therefore, in the data backup process, the flow complexity and the system overhead can be reduced while ensuring the atomic operation of the write request.

Disclosure of Invention

The data writing methods and devices provided by the application can ensure the atomicity of group writing requests through a log mechanism in the data backup process, and avoid the realization of the atomicity through a file system semantic lock in the prior art, thereby reducing the process complexity and the system overhead.

, the method for writing data includes that a main node stores N sub-logs of a log locally, the main node backs up the N sub-logs to a standby node, and the main node marks the log as valid when the main node determines that all the N sub-logs are stored in the standby node.

To ensure that the group of write requests all succeed or all fail (that is, the group of write requests are or series of operations that cannot be interrupted), the group of write requests may be referred to as atomic writes, which may also be referred to as having atomicity.

It should be appreciated that consecutive sub-writes of the atomic writes may generate logs (logs), and N sub-writes of the atomic writes may generate N sub-logs (sublogs), wherein the sub-logs may record information of the sub-writes (that is, sub-writes may correspond to sub-logs), and the recorded information may include, but is not limited to, the address of deposit (start address), the data written, the length of the data written, etc.

In the embodiment of the application, the atomicity of the group write request is ensured by a log mechanism in the data backup process, and the atomicity realized by a file system semantic lock in the prior art is avoided, so that the process complexity and the system overhead can be reduced.

With reference to , in some implementations of , the method further includes the master node deleting the N stored sub-logs in the event that the N sub-logs in the standby node fail to be stored.

In embodiments, if the peer fails to perform operations on the peer due to resources or various reasons, the present application embodiment may employ a rollback mechanism, which may roll back logs stored by the peer, and may guarantee atomicity of the entire system data ( group write requests either all successful or all failed).

In the embodiment of the application, the atomicity of the write request between the nodes is guaranteed through a rollback mechanism, the phenomenon that the atomicity of the write request between the nodes is guaranteed by means of retry in the prior art is avoided, and the serviceability of a system is improved.

With reference to the aspect, in some implementations of the aspect, the N sub-logs carry the same log identification number ID, and the master node stores the N sub-logs in order in a log storage container of the master node according to the ID.

In the embodiment of the application, in the data backup process, a standby node is used as disaster recovery backup of a main node, and the data sequence written by the standby node needs to be written according to the sequence of the main node, that is, the sequence of the data written by the standby node and the data written by the main node needs to be kept , as examples, in the embodiment of the application, identification numbers (identities, IDs) can be generated for every logs, multiple superbogs on the logs can also have the same log ID., the standby node can hang the superbogs in corresponding chunks according to the log IDs, and the superbogs in a superbog linked list in the chunks can be orderly arranged according to the log IDs.

With reference to , in some implementations of , the master node receives a notification message indicating that all of the N sub-logs are stored in the standby node, and marks the log as valid.

In a second aspect, data writing methods are provided, and the method includes that a standby node stores N sub-logs of a log backed up by a main node into the standby node, the standby node marks the log as valid when all the N sub-logs are stored into the standby node, and the standby node informs the main node that all the N sub-logs are stored into the standby node.

It should be understood that the log is generated from atomic write requests, the atomic write requests including N sub-writes, the sub-logs recording information of the sub-writes, N being a positive integer greater than 0.

With reference to the second aspect, in some implementation manners of the second aspect, the N sub-logs carry the same log identification number ID, and the standby node sequentially stores the N sub-logs into a log storage container of the standby node according to the ID.

With reference to the second aspect, in some implementation manners of the second aspect, the standby node sends a notification message to the master node, where the notification message is used to indicate that all of the N sub-logs are stored in the log storage container of the standby node.

In a third aspect, an data writing device is provided, where the device includes a storage module configured to store N sub-logs of a log locally, a sending module configured to backup the N sub-logs to a standby node, and a processing module configured to mark the log as valid when it is determined that all of the N sub-logs are stored in the standby node.

It should be understood that the log is generated from atomic write requests, the atomic write requests include N sub-writes, each of the N sub-logs records information for each sub-write, and N is a positive integer greater than 0.

The data writing device provided by the embodiment of the application can ensure the atomicity of group writing requests through a log mechanism in the data backup process, and avoids the realization of the atomicity through a file system semantic lock in the prior art, so that the process complexity and the system overhead can be reduced.

With reference to the third aspect, in certain implementations of the third aspect, the apparatus further includes: and the second processing module is used for deleting the stored N sub-logs under the condition that the N sub-logs in the standby node fail to be stored.

With reference to the third aspect, in some implementation manners of the third aspect, the N sub-logs carry the same log identification number ID, and the storage module is specifically configured to: and the main node orderly stores the N sub-logs into a log storage container of the main node according to the ID.

With reference to the third aspect, in some implementation manners of the third aspect, the processing module is specifically configured to receive a notification message, where the notification message is used to indicate that all of the N sub-logs are stored in the standby node, and mark the log as valid.

For example, computer instructions stored in the cache can be called by the processor to execute the steps required to be executed by each module (the storage module, the sending module, the -th processing module, and the second processing module).

The fourth aspect provides data writing devices, which include a storage module configured to store N sub-logs of a log backed up by a master node in a standby node, a processing module configured to mark the log as valid when all the N sub-logs are stored in the standby node, and a sending module configured to notify the master node that all the N sub-logs are stored in the standby node by the standby node.

With reference to the fourth aspect, in some implementation manners of the fourth aspect, the N sub-logs carry the same log identification number ID, and the storage module is specifically configured to: and orderly storing the N sub-logs into a log storage container of the standby node according to the ID.

With reference to the fourth aspect, in some implementations of the fourth aspect, the sending module is specifically configured to: and sending a notification message to the master node, wherein the notification message is used for indicating that all the N sub-logs are stored in a log storage container of the standby node.

It should be noted that, in the entity apparatus shown in the fourth aspect, the processor may implement the steps executed by the respective modules by calling the computer program in the memory. For example, computer instructions stored in the cache may be called by the processor to perform steps required by the various modules (storage module, processing module, sending module).

In a fifth aspect, kinds of master nodes are provided, which comprises an input/output interface, a processor and a memory, wherein the processor is configured to control the input/output interface to send and receive information, the memory is configured to store a computer program, and the processor is configured to call and execute the computer program from the memory, so that the master node executes the method described in any possible implementation manners of the or .

When implemented in hardware, the processor may be a logic circuit, an integrated circuit, or the like, and when implemented in software, the processor may be general-purpose processors implemented by reading software code stored in a memory that may be integrated within the processor, located external to the processor, or stand-alone.

In a sixth aspect, there are standby nodes, comprising an input/output interface, a processor and a memory, wherein the processor is configured to control the input/output interface to send and receive information, and the memory is configured to store a computer program, and the processor is configured to call and execute the computer program from the memory, so that the master node executes the method described in any possible implementation manners of the second aspect or the second aspect.

In a seventh aspect, computer program products are provided, the computer program products comprising computer program code to, when run on a computer, cause the computer to perform the method of the above aspects.

In an eighth aspect, computer readable media are provided, the computer readable media storing program code which, when run on a computer, causes the computer to perform the method in the above aspects.

In a ninth aspect, there are provided systems comprising the aforementioned master node and backup node.

Drawings

Fig. 1 is a schematic block diagram of a scenario in which an embodiment of the present application is applicable.

Fig. 2 is a schematic flow chart of data writing methods provided by embodiments of the present application.

Fig. 3 is a schematic structural diagram of an data writing system provided in an embodiment of the present application.

FIG. 4 is a schematic block diagram of a data writing system provided by another embodiment of the present application.

FIG. 5 is a schematic block diagram of a data writing system provided by another embodiment of the present application.

FIG. 6 is a schematic flow chart of a data writing method according to another embodiment of the present application.

FIG. 7 is a schematic block diagram of a data writing system provided by another embodiment of the present application.

Fig. 8 is a schematic block diagram of data writing devices 800 provided in the embodiments of the present application.

Fig. 9 is a schematic block diagram of data writing apparatuses 900 according to an embodiment of the present application.

Fig. 10 is a schematic block diagram of host nodes 1000 provided in this embodiment of the present application.

Fig. 11 is a schematic block diagram of standby nodes 1100 according to an embodiment of the present disclosure.

Detailed Description

The technical solution in the present application will be described below with reference to the accompanying drawings.

In a distributed storage system, system availability is , which is an important indicator, and it is necessary to ensure that system availability is not affected when a machine fails.to ensure system availability, data needs to be backed up (i.e., multiple copies of stored data need to be kept ), and when copies fail due to a machine failure, other copies can still be serviced.

FIG. 1 is a schematic block diagram of a scenario in which an embodiment of the present application is applicable, the scenario shown in FIG. 1 may include at least two nodes generally refers to a source node that may be duplicated as a master node 110, and the destination nodes ( or more) that may be duplicated may be referred to as standby nodes 120.

As an example, master node 110 may write data locally after receiving a write request for it, and may backup the written data to standby node 120. master node 110 may also be referred to as a primary copy.

The standby node 120, as a disaster recovery backup based on disk data, enables the standby node 120 to take over applications and application data quickly when the host performs scheduled maintenance or the host fails (unplanned), so that the user's access is not interfered by scheduled or unplanned downtime.

The file system is used as a software mechanism for managing and storing file information in an operating system and is mainly responsible for establishing files, storing, reading and dumping the files, controlling the access of the files and the like for a user.

For example, Network Attached Storage (NAS) may be a computer data storage device connected to a computer network, and is also referred to as "network storage". NAS is an dedicated data storage server, and a storage device is completely separated from a server by centering data, and centrally manages data.

In a service for providing a file system, it is necessary to guarantee that groups of write requests all succeed or all fail under many scenarios (e.g., involving data storage systems of 2 or more nodes), which may be referred to as atomic writes.

It should be understood that atomic writes may also be referred to as group write requests are atomic or the group write requests are atomic operations (i.e., the group write requests are an uninterruptible series of or operations).

In the prior art, the atomicity of group write requests is guaranteed by the semantic lock of the file system, however, the added semantic lock may result in a complex system flow and increased system overhead.

In the data backup process, in order to ensure atomicity of write requests among more than 2 nodes and reduce complexity of a system process and system overhead, the embodiment of the application provides data writing methods, which can avoid complex processes and large system overhead caused by realization of atomicity through a file system semantic lock in the prior art.

The data writing method of the present embodiment is described in detail below with reference to fig. 2 based on the architecture shown in fig. 1, fig. 2 is a schematic flow chart of data writing methods provided by the present embodiment, the method shown in fig. 2 may include

steps

210 and 230, and the

steps

210 and 230 are described in detail below.

In step 210, the master node stores the N sub-logs of the log locally.

A concatenated sub-writes of the atomic writes may generate logs (logs), and N sub-writes of the atomic writes may generate N sub-logs (sublogs), where the sub-logs may record information of the sub-writes (that is, sub-writes may correspond to sub-logs), and the recorded information may include, but is not limited to, a deposited address (start address), written data, a length of the written data, and the like.

The master node may store the generated N sub-logs locally, and as examples, the master node may store the N sub-logs in corresponding log storage containers according to the recorded start addresses, for example, the master node may store the N sub-logs in corresponding chunk according to the recorded start addresses.

In step 220, the primary node backs up the N sub-logs into the standby node.

In this embodiment of the present application, the master node may backup N sub-logs generated in the standby node, the standby node may store the N sub-logs written by the master node in the standby node, as examples, the standby node may store the N sub-logs in corresponding log storage containers according to recorded starting addresses, for example, the standby node may store the N sub-logs in corresponding chunks according to recorded starting addresses, and a writing process of atomic write requests will be described in detail below with reference to fig. 3, and details are not described here.

It should be noted that the chunk address in the standby node is the same as the address in the main node, and the standby node may store the N sub-logs into the corresponding chunk according to the recorded start addresses, as in operation flow in the main node.

In step 230, the primary node marks the journal as valid if it determines that all of the N sub-journals have been stored to the backup node.

If the master node can determine that the backup node stores all the N sub-logs in the corresponding log storage containers, the master node can mark the logs as valid after storing all the N sub-logs in the local log, as examples, the master node can add marking bits in the logs, modify the state of the marking bits, and set the marking bits as valid.

It should be understood that, in the embodiment of the present application, when the master node marks the log as valid, it may indicate that N sub-logs of the log have been stored locally, and the N sub-logs have also been backed up in the standby node, and atomic write requests may be considered to have been completely written into the master node and the standby node, so that atomicity of group write requests between nodes may be ensured.

In the embodiment of the application, in the data backup process, the atomicity of the write request between the main node and the standby node is realized through a log mechanism, so that the semantic lock of a file system in the prior art can be avoided, and the complexity of a system flow and the system overhead can be reduced.

A specific implementation of the write process of the atomic write request in the embodiment of the present application is described in more detail below with reference to a specific example. It should be noted that the example of fig. 3 is only for assisting the skilled person in understanding the embodiments of the present application, and is not intended to limit the embodiments of the application to the specific values or specific scenarios illustrated. It will be apparent to those skilled in the art that various equivalent modifications or variations are possible in light of the given example of fig. 3, and such modifications and variations are intended to be included within the scope of the embodiments of the present application.

Fig. 3 is a schematic block diagram of an data writing system provided in an embodiment of the present application, the system shown in fig. 3 may include a chunk (chunk)310, a chunk (chunk)320, and a chunk (chunk) 330.

The Sublog stored in the chunk 310 in the form of a linked list may have: a child log (Sublog)312, a child log (Sublog)314, and a child log (Sublog) 316.

The Sublog stored in the form of a linked list in chunk 320 may have: a sub-log (Sublog)322, a sub-log (Sublog)324, and a sub-log (Sublog) 326.

The superbog stored in linked list form in chunk 330 may have: sublog (Sublog)332, sublog (Sublog)334, and sublog (Sublog) 336.

It should be understood that in the embodiment of the present application, consecutive sub-writes of atomic write requests may generate 0 logs (logs), N sub-writes of atomic writes may generate N sub-logs (sublogs), sub-logs may record addresses of the sub-writes, data to be written, length of data to be written, and the like, that is, atomic write requests may correspond to logs, and each sub-writes of atomic writes corresponds to sublogs.

In the embodiment of the present application, the chunk may be logically-divided block memories, and is basic data indexing units, and the node may store the blob according to the start address of the sub-write recorded in the blob, and store the blob in the chunk at the corresponding address in a linked list form.

The method comprises the steps that a superbog generated by a write request can find a chunk corresponding to the superbog according to the addresses of the sub-writes recorded in the superbog, and can be added to a superbog chain table on the chunk, as examples, the read request can find the chunk corresponding to the read request through the addresses and the offsets recorded in the superbog, as examples, and the write request can find the chunk corresponding to the write request according to the addresses and the offsets recorded in the superbog.

It should be understood that the linked list may be a non-sequential, non-sequential storage structure on physical storage units, and the logical order of the data elements may be implemented sequentially through pointer connections in the linked list.

The following description will be made in detail by taking a write request as an example.

Referring to fig. 3, the log311 generated by group write requests may include a Sublog316, a Sublog 324, and a Sublog334, wherein the Sublog316, the Sublog 324, and the Sublog334 may be organized within the log311 in a linked list.

Sublog can find the chunk corresponding to the recorded address and offset. For example, Sublog316 may be added to the link of chunk 310, Sublog 324 may be added to the link of chunk 320, and Sublog334 may be added to the link of chunk 320.

It should be noted that the solid line boxes in the graph may be used to indicate that a plurality of Sublogs included in bars log have all been stored in the corresponding chunk, and the plurality of Sublogs have been marked as valid ( the group of write requests has all been written in the memory, and atomicity of the group of write requests may be guaranteed), and may be read by the read request.

It should be understood that the Sublog 322 on the chunk 320 linked list may be used to indicate that the log311 has been stored in the chunk before being written, and the Sublog 326 may be used to indicate that other write data may be stored on the chunk 320 linked list while the log311 is being written.

For example, a flag bit may be added to the log311, and after all the superbogs in the log311 are stored in the corresponding chunk linked list, the flag bit may be set to be valid, and all the superbogs in the log311 may be read by a read request.

After logs are marked as valid (that is, all the superbogs on the logs are stored in the corresponding chunk linked list), it can be considered that atomic write request operations have been completed, the memory space occupied by the logs can be released, and the released memory space can be recycled, specifically referring to the system structure diagram shown in fig. 5, compared with the system structure diagram shown in fig. 4, the superbogs 316, 324 and 334 on the logs 311 in fig. 5 are all stored in the corresponding chunks, and the memory space occupied by the logs 311 can be released.

It should be appreciated that if a log of atomic write requests is released, all of the Sublogs on the log may also be considered to be stored in their entirety on the corresponding chunk linked list, atomic write requests have completed.

In the embodiment of the application, atomic write in the process can be realized through a mechanism of the journal and the sub-journal, atomicity of group write requests can be ensured, atomicity of group write requests realized through a file system semantic lock in the prior art can be avoided, and complexity of the process and system overhead can be reduced.

Optionally, in embodiments, in a data backup process, a standby node is used as a data backup of a primary node, and an order of data written by the standby node needs to be written according to an order of the primary node, that is, an order of data written by the standby node and data written by the primary node needs to be kept , as examples, the present application embodiment may generate identification numbers (IDs) for every logs, multiple superbogs above a log may also have the same log ID., and the standby node may hang the superbogs in corresponding chunks according to log IDs, and the superbogs in a superbog linked list in the chunks may be ordered according to log IDs.

It should be noted that the example of fig. 6 is only for helping those skilled in the art understand the embodiment of the present application, and does not limit the embodiment of the present application to the specific values or the specific scenarios illustrated.

Referring to fig. 6, it should be understood that the home node in fig. 6 corresponds to the above home node, and the opposite node corresponds to the above standby node. The method of FIG. 6 may include steps 610-670, which are described in detail below.

Step 610, the local end writes input/output (I/O).

The local side (understanding that the controller) may receive a write I/O instruction of an operating system and data to be written, in this embodiment, consecutive sub-writes in groups of write requests may generate logs, and the sub-writes may generate a blob, where the blob may record the data to be written, the length of the data, and an address where the data is stored.

Step 620, the local end generates a log (log) ID.

The local end may, after receiving a write I/O instruction of the operating system, the generated logs, and multiple Sublogs, generate log IDs for the logs, and the multiple Sublogs may have the same log ID as the log, so that the opposite end may write data in the order of the local end when writing data, so that the data written by the opposite end may correspond to the data written by the local end.

Step 630, the local end writes locally.

The local end may write the received multiple Sublogs locally, for example, the local end may write the Sublogs into corresponding log storage containers (for example, may be chunk) according to logIDs of each Sublog, and a Sublog linked list in the chunk may be ordered according to log IDs. For the local write processing flow of the local terminal, please refer to the data write method described in fig. 3, which is not described herein again.

Step 640, the mirror image is written to the peer.

An opposite end (which can understand a controller) may receive a plurality of superbogs to be backed up sent by the local end, where the plurality of superbogs may carry log IDs, and the opposite end may write the superbogs into corresponding log storage containers (for example, may be chunk) according to the log IDs of each superbog, and a superbog linked list in the chunk may be ordered according to the log IDs.

The process of writing the Sublog into the opposite end according to the log ID can also be called as a write mirror image, that is, a plurality of Sublogs stored by the local end are backed up to the opposite end. For the processing flow of the peer-to-peer write mirror, please refer to the data writing method described in fig. 3, which is not described herein again.

It should be noted that the local end may send only the multiple superbogs to be backed up to the opposite end, and the local end may also send both the log and the multiple superbogs of the log to the opposite end, which is not specifically limited in this embodiment of the present application. If the opposite end only receives a plurality of superbogs sent by the local end, the opposite end can generate a corresponding log.

Step 650, validate the endpoint data.

After the opposite end stores all the received multiple Sublogs to the corresponding chunk according to the log IDs, the opposite end can mark the log marking positions of the multiple Sublogs as valid. For a specific processing flow, please refer to the data writing method described with reference to fig. 3, which is not described herein again.

As another examples, if the peer stores all the superbogs to the corresponding chunks and the log flag is valid, the notification message sent by the peer to the peer may be used to indicate that atomic write requests of the peer have all been completed.

In step 660, the local side data is valid.

After the local terminal determines that the opposite terminal stores all the Sublogs to the corresponding chunk, the local terminal can mark the log marking positions of the Sublogs as valid. For a specific processing flow, please refer to the data writing method described with reference to fig. 3, which is not described herein again.

And step 670, the local end completes.

The log generated by the local end in the atomic write request is set to be valid, which can be used to indicate that the atomic write request has been successfully written in 2 nodes and can be read by the read request.

Optionally, in embodiments, if the local end does not set the mark position of the log to valid after determining that the opposite end is completely executed (the process fails to execute), the local end may be considered to be abnormal.

The following describes a specific implementation manner of the rollback mechanism in the embodiment of the present application in more detail with reference to a specific example. It should be noted that the example of fig. 7 is only for assisting the skilled person in understanding the embodiments of the present application, and is not intended to limit the embodiments of the present application to the specific values or specific scenarios illustrated. It will be apparent to those skilled in the art that various equivalent modifications or variations are possible in light of the example given in the example of fig. 7, and such modifications and variations also fall within the scope of the embodiments of the present application.

If the storage system needs to rollback, the local end can determine the Sublog related to the log according to the log needing to be rolled back, and can delete the Sublog from the chunk linked list in sequence to complete the rollback operation through the Sublog linked list in the log.

As an example of , if the opposite end fails to store the Sublog in the log311, the home end may delete the Sublog316, the Sublog 324, and the Sublog334 from the chunk linked list according to the Sublog linked list in the log311 to complete the rollback operation, as shown in fig. 3.

The system after the local rollback can refer to fig. 7, and the superbog in the log311 written in fig. 7 has been deleted from the chunk linked list.

The data writing method provided by the embodiment of the present application is described in detail above with reference to fig. 1 to 7, and the embodiment of the data writing apparatus of the present application is described in detail below with reference to fig. 8 to 11. It is to be understood that the description of the method embodiments corresponds to the description of the apparatus embodiments, and therefore reference may be made to the preceding method embodiments for parts not described in detail.

Fig. 8 is a schematic block diagram of data writing devices 800 provided by an embodiment of the present application, where the data writing devices 800 may include:

a storage module 810, configured to store N sub-logs of the log locally;

a sending module 820, configured to backup the N sub-logs to a standby node;

, a processing module 830 for marking the journal as valid if it is determined that all of the N sub-journals are stored to the standby node.

It should be understood that the log is generated from atomic write requests, the atomic write requests including N sub-writes, each of the N sub-logs recording information for each sub-write, N being a positive integer greater than 0

Optionally, in embodiments, the data writing apparatus 800 may further include:

a second processing module 840, configured to delete the N stored sub-logs when the N sub-logs in the standby node fail to be stored.

Optionally, in embodiments, the N sub-logs carry the same log identification number ID, and the storage module 810 is specifically configured to store the N sub-logs in order in a log storage container of the master node according to the ID.

Optionally, in embodiments, the processing module 830 is specifically configured to receive a notification message indicating that all of the N sub-logs are stored in the standby node, and mark the log as valid.

It should be noted that, in the entity apparatus 800 shown in fig. 8, the processor may implement the steps executed by each module by calling the computer program in the memory, for example, the processor may call the computer instructions stored in the cache to execute the steps required to be executed by each module (the storage module 810, the sending module 820, the -th processing module 830, the second processing module 840).

Fig. 9 is a schematic block diagram of data writing devices 900 provided in this embodiment of the present application, where the data writing devices 900 may include:

a storage module 910, configured to store N sub-logs of a log backed up by a master node in the backup node;

a processing module 920, configured to mark the log as valid when all the N sub-logs are stored in the standby node;

a sending module 930, configured to notify the master node that all of the N sub-logs are stored in the slave node by the slave node.

Optionally, in embodiments, the N sub-logs carry the same log identification number ID, and the storage module 910 is specifically configured to store the N sub-logs in order in a log storage container of the standby node according to the ID.

Optionally, in embodiments, the sending module 930 is specifically configured to send a notification message to the master node, where the notification message is used to indicate that all of the N sub-logs are stored in the log storage container of the standby node.

It should be noted that, in the entity apparatus 900 shown in fig. 9, the processor may implement the steps executed by the respective modules by calling the computer program in the memory. For example, computer instructions stored in the cache may be called by the processor to perform the steps required by the respective modules (the storage module 910, the processing module 920, and the sending module 930).

Fig. 10 is a schematic block diagram of host nodes 1000 provided by the embodiments of the present application, where the host node 1000 may include a memory 1001, a process 1002, and an input/output interface 1003.

The memory 1001, the processor 1002 and the input/output interface 1003 are connected via an internal connection path, the memory 1001 is used for storing program instructions, and the processor 1002 is used for executing the program instructions stored in the memory 1001 to control the input/output interface 1003 to receive input data and information and output data such as operation results.

It should be understood that, in the embodiment of the present application, the processor 1002 may adopt a Central Processing Unit (CPU), and the processor may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete or transistor logic devices, discrete hardware components, etc. the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. or the processor 1002 adopts or more integrated circuits for executing the relevant programs to realize the technical solutions provided by the embodiments of the present application.

The memory 1001 may include both read-only memory and random access memory and provide instructions and data to the processor 1002. the portion of the processor 1002 may also include non-volatile random access memory.

In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 1002. The method disclosed in the embodiments of the present application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 1001, and the processor 1002 reads the information in the memory 1001 and completes the steps of the method in combination with the hardware thereof. To avoid repetition, it is not described in detail here.

It should be understood that the host node 1000 according to the embodiment of the present application may correspond to the apparatus 800 in the embodiment of the present application, and may be configured to perform corresponding processes of the methods in fig. 2 to fig. 7 in the embodiment of the present application, and the above-mentioned and other operations and/or functions of each module in the host node 1000 are respectively for implementing the corresponding processes of the methods in fig. 2 to fig. 7 in the embodiment of the present application, and are not described herein again for brevity.

Fig. 11 is a schematic block diagram of standby nodes 1100 provided in this embodiment, where the standby node 1100 may include a memory 1101, a processor 1102, and an input/output interface 1103.

The memory 1101, the processor 1102 and the input/output interface 1103 are connected through an internal connection path, the memory 1101 is used for storing program instructions, and the processor 1102 is used for executing the program instructions stored in the memory 1101 to control the input/output interface 1103 to receive input data and information and output data such as operation results.

It should be understood that, in the embodiment of the present application, the processor 1102 may adopt a Central Processing Unit (CPU), and the processor may also adopt other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete or transistor logic devices, discrete hardware components, etc. the general purpose processor may also be a microprocessor or the processor may also be any conventional processor, etc. or the processor 1102 adopts or more integrated circuits for executing the relevant programs to realize the technical solutions provided by the embodiments of the present application.

The memory 1101 may include both read-only memory and random-access memory and provides instructions and data to the processor 1102 portion of the processor 1102 may also include non-volatile random-access memory.

In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 1102. The method disclosed in the embodiments of the present application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 1101, and the processor 1102 reads the information in the memory 1101, and completes the steps of the above method in combination with the hardware thereof. To avoid repetition, it is not described in detail here.

It should be understood that the standby node 1100 according to the embodiment of the present application may correspond to the apparatus 900 in the embodiment of the present application, and may be configured to perform corresponding processes of the methods in fig. 2 to fig. 7 in the embodiment of the present application, and the above-mentioned and other operations and/or functions of each module in the standby node 1100 are respectively for implementing the corresponding processes of the methods in fig. 2 to fig. 7 in the embodiment of the present application, and are not repeated herein for brevity.

Through the above description, the primary node 1000 and the standby node 1100 provided in the embodiment of the present application may ensure atomicity of group write requests through a log mechanism in a data backup process, so as to avoid implementing atomicity through a file system semantic lock in the prior art, thereby reducing process complexity and system overhead.

It should be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units into logical functional divisions may be realized in other ways, for example, multiple units or components may be combined or integrated into another systems, or features may be omitted or not executed, in another point, the shown or discussed coupling or direct coupling or communication connection between each other may be through interfaces, indirect coupling or communication connection between units or devices may be electrical, mechanical or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, that is, may be located in places, or may also be distributed on multiple network units.

In addition, functional units in the embodiments of the present application may be integrated into processing units, or each unit may exist alone physically, or two or more units are integrated into units.

Based on the understanding that the technical solution of the present application, in essence or a part contributing to the prior art, or a part of the technical solution, can be embodied in the form of a software product, which is stored in storage media and includes several instructions for causing computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application, the foregoing storage media include various media capable of storing program codes, such as a usb disk, a mobile hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1, A method for data writing, the method comprising:

the method comprises the steps that a main node stores N sub-logs of a log to the local, the log is generated according to atomic write requests, the atomic write requests comprise N sub-writes, information of the sub-writes is recorded in the sub-logs, and N is a positive integer larger than 0;

the main node backs up the N sub-logs to a standby node;

and the master node marks the logs as valid under the condition that the master node determines that all the N sub-logs are stored in the standby node.

2. The method of claim 1, further comprising:

and under the condition that the N sub-logs in the standby node fail to be stored, the main node deletes the stored N sub-logs.

3. The method according to claim 1 or 2, wherein the N sub-logs carry the same log identification number ID,

the main node stores N sub-logs of the log to the local, including:

and the main node orderly stores the N sub-logs into a log storage container of the main node according to the ID.

4. The method of any of claims 1-3, wherein the master node marking the log as valid if it is determined that all of the N sub-logs are stored to the backup node comprises:

the main node receives a notification message, wherein the notification message is used for indicating that all the N sub-logs are stored in the standby node;

the master node marks the log as valid.

A method of data writing of , said method comprising:

the backup node stores N sub-logs of a log backed up by a main node into the backup node, wherein the log is generated according to atomic write requests, each atomic write request comprises N sub-writes, the sub-logs record sub-write information, and N is a positive integer greater than 0;

when the N sub-logs are all stored in the standby node, the standby node marks the logs as valid;

and the standby node informs the main node that all the N sub-logs are stored in the standby node.

6. The method of claim 5, wherein the N sub-logs carry the same log identification number ID,

the backup node stores N sub-logs of the log backed up by the master node into the backup node, and the method comprises the following steps:

and the standby node orderly stores the N sub-logs into a log storage container of the standby node according to the ID.

7. The method of claim 5 or 6, wherein the notifying, by the standby node, that the N sub-logs are all stored in the standby node to the master node comprises:

and the standby node sends a notification message to the main node, wherein the notification message is used for indicating that all the N sub-logs are stored in a log storage container of the standby node.

An apparatus for data writing of the type 8, , said apparatus comprising:

a storage module, configured to store N sub-logs of a log locally, where the log is generated according to atomic write requests, the atomic write requests include N sub-writes, each sub-log in the N sub-logs records information of each sub-write, and N is a positive integer greater than 0;

the sending module is used for backing up the N sub-logs to the standby node;

, a processing module for marking the log as valid if it is determined that all of the N sub-logs are stored to the standby node.

9. The apparatus of claim 8, further comprising:

and the second processing module is used for deleting the stored N sub-logs under the condition that the N sub-logs in the standby node fail to be stored.

10. The apparatus according to claim 8 or 9, wherein the N sub-logs carry the same log identification number ID,

the storage module is specifically configured to:

11. The apparatus according to of claims 8-10, wherein the processing module is specifically configured to:

receiving a notification message, wherein the notification message is used for indicating that all the N sub-logs are stored in the standby node;

marking the log as valid.

12, apparatus for data writing, the method comprising:

the storage module is configured to store N sub-logs of a log backed up by a master node into the backup node, where the log is generated according to atomic write requests, each atomic write request includes N sub-writes, each sub-log of the N sub-logs records information of each sub-write, and N is a positive integer greater than 0;

the processing module is used for marking the logs as valid when all the N sub-logs are stored in the standby node;

and the sending module is used for informing the master node that all the N sub-logs are stored in the standby node by the standby node.

13. The apparatus of claim 12, wherein the N sub-logs carry the same log identification number (ID),

the storage module is specifically configured to:

and orderly storing the N sub-logs into a log storage container of the standby node according to the ID.

14. The apparatus according to claim 12 or 13, wherein the sending module is specifically configured to:

and sending a notification message to the master node, wherein the notification message is used for indicating that all the N sub-logs are stored in a log storage container of the standby node.

15, master node, comprising an input/output interface, a processor and a memory, wherein the processor is configured to control the input/output interface to send and receive information, the memory is configured to store a computer program, and the processor is configured to retrieve and execute the computer program from the memory, so that the master node performs the operation steps of the method of any of claims 1 to 4, wherein is the master node.

16, backup node, comprising an input output interface, a processor and a memory, wherein the processor is configured to control the input output interface to send and receive information, the memory is configured to store a computer program, and the processor is configured to retrieve and execute the computer program from the memory, so that the backup node performs the operation steps of the method of any of claims 5 to 7.

Storage system of claim 17, , comprising a master node according to claim 15 and a backup node according to claim 16.

A computer-readable storage medium comprising a computer program which, when run on a computer, causes the computer to perform the method of any of claims 1-7 .