CN118093730B - Data double-writing method, device, computer equipment and program product - Google Patents

Data double-writing method, device, computer equipment and program product Download PDF

Info

Publication number
CN118093730B
CN118093730B CN202410525955.2A CN202410525955A CN118093730B CN 118093730 B CN118093730 B CN 118093730B CN 202410525955 A CN202410525955 A CN 202410525955A CN 118093730 B CN118093730 B CN 118093730B
Authority
CN
China
Prior art keywords
data
data volume
volume
lock
writing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410525955.2A
Other languages
Chinese (zh)
Other versions
CN118093730A (en
Inventor
张廷雷
彭泽
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Inspur Jinan data Technology Co ltd
Original Assignee
Inspur Jinan data Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Inspur Jinan data Technology Co ltd filed Critical Inspur Jinan data Technology Co ltd
Priority to CN202410525955.2A priority Critical patent/CN118093730B/en
Publication of CN118093730A publication Critical patent/CN118093730A/en
Application granted granted Critical
Publication of CN118093730B publication Critical patent/CN118093730B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2308—Concurrency control
    • G06F16/2336—Pessimistic concurrency control approaches, e.g. locking or multiple versions without time stamps
    • G06F16/2343—Locking methods, e.g. distributed locking or locking implementation details
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2365—Ensuring data consistency and integrity

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Computing Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to the technical field of storage, and discloses a data double-writing method, a device, computer equipment and a program product, wherein the method comprises the following steps: receiving a data writing instruction, wherein the data writing instruction carries a locking range of a data object; generating an object identifier and a client identifier for the data object in response to the data write instruction; transmitting a first data write operation comprising a data write request and an object range lock to a first data volume and a second data volume, wherein the object range lock comprises a locking range, an object identifier and a client identifier; receiving a first response result of the first data volume responding to the first data writing operation and a second response result of the second data volume responding to the first data writing operation; and determining a first execution result of the first data writing operation according to the first response result and the second response result. And performing effective data write operation mutual exclusion processing to ensure the data consistency of write operation of the data write instruction in different data volumes.

Description

Data double-writing method, device, computer equipment and program product
Technical Field
The present invention relates to the field of storage technologies, and in particular, to a data double writing method, apparatus, computer device, and program product.
Background
In the related art, the method of data double writing is only to perform simple data reading and writing on data volumes at two ends, and the requirement of complex characteristics such as online data migration, synchronous remote copying and double activity of distributed block storage cannot be met for mutual exclusion processing of reading and writing operations.
Disclosure of Invention
In view of the above, the present invention provides a data double-writing method, apparatus, computer device and program product, so as to solve the problem that in the related art, the operation for data double-writing is only performed for the data volumes at both ends, and the reasonable mutual exclusion processing cannot be performed for the double-writing operation.
In a first aspect, the present invention provides a data double writing method, including:
receiving a data writing instruction, wherein the data writing instruction carries a locking range of a data object;
Generating an object identifier and a client identifier for the data object in response to the data write instruction;
transmitting a first data write operation comprising a data write request and an object range lock to a first data volume and a second data volume, wherein the object range lock comprises a locking range, an object identifier and a client identifier;
receiving a first response result of performing the first data write operation on the first data volume and a second response result of performing the first data write operation on the second data volume;
And determining a first execution result of the first data writing operation according to the first response result and the second response result.
According to the data double-writing method provided by the embodiment of the invention, the object range lock is added in the data writing operation, when the data writing operation is executed by the first data volume and the second data volume, the mutual exclusion problem of the writing operation can be effectively processed through the object range lock, and the response results of the writing operation executed by different data volumes are timely obtained, so that the execution result of the writing operation is determined based on the first response result of the first data writing operation executed by the first data volume and the second response result of the first data writing operation executed by the second data volume, and therefore the effective mutual exclusion processing of the data writing operation is carried out on the data writing instruction, and the data consistency of the writing operation executed by the data writing instruction by different data volumes is ensured.
In an alternative embodiment, determining the first execution result of the first data write operation according to the first response result and the second response result includes:
And when the first response result shows that the lock conflict failure occurs when the first data volume executes the first data write operation and/or the second response result shows that the lock conflict failure occurs when the second data volume executes the first data write operation, judging that the first execution result of the first data write operation is the execution failure.
The data double-writing method provided by the embodiment of the invention judges that the first execution result of the first data writing operation is the execution failure as long as the first response result shows that the first data writing operation is executed on the first data volume and the lock conflict failure occurs, or the second response result shows that the first data writing operation is executed on the second data volume and the lock conflict failure occurs. Therefore, the problem of data difference among a plurality of data volumes caused by different execution results of data writing operation on different data volumes is effectively avoided.
In an alternative embodiment, when the first execution result of the first data write operation is an execution failure, the method further includes:
if the first response result shows that the first data write operation is successfully executed on the first data volume and the second response result shows that the lock conflict failure occurs when the first data write operation is executed on the second data volume, releasing the object range lock of the first data write operation, which is successfully applied on the first data volume;
And if the second response result shows that the first data write operation is successfully executed on the second data volume, but the first response result shows that the lock conflict failure occurs when the first data write operation is executed on the first data volume, releasing the object range lock of the first data write operation which is successfully applied on the second data volume.
When the first execution result of the first data writing operation is the execution failure, if the data writing operation fails in a lock conflict in one of the different data volumes, the object range lock used when the data writing operation is successfully executed on the other data volumes is released in time, so that the occupation of storage resources is avoided, and the data consistency of a plurality of data volumes is ensured.
In an alternative embodiment, when the first execution result of the first data write operation is an execution failure, the method further includes: and carrying out serial processing on the data writing instruction.
According to the data double-writing method provided by the embodiment of the invention, when the first execution result of the first data writing operation is the execution failure, the data writing instruction is processed in series in time. The problem of user experience degradation caused by direct feedback of double writing failure of data to the user is effectively avoided. Meanwhile, serial processing of data writing operation is adopted, so that the probability of double writing success of data is remarkably improved, and user experience is effectively improved.
In an alternative embodiment, the serial processing of the data write instruction includes:
Applying for an object range lock to the first data volume and the second data volume based on the data write instruction;
If the data writing instruction applies successfully to the object range locks of the first data volume and the second data volume, sending a second data writing operation comprising a data writing request and an object range lock confirmation request to the first data volume and the second data volume;
Receiving a third response result of the second data writing operation performed on the first data volume and a fourth response result of the second data writing operation performed on the second data volume;
And determining a second execution result of the second data writing operation according to the third response result and the fourth response result.
In the process of carrying out serial processing on a data writing instruction, the method firstly applies object range lock to a first data volume and a second data volume based on the data writing instruction, and then when the data writing instruction is applied successfully to the object range lock of the first data volume and the second data volume, a second data writing operation comprising a data writing request and an object range lock confirmation request is sent to the first data volume and the second data volume. Therefore, the data conflict of the parallel data writing operation is effectively processed, and mutual exclusivity of a plurality of data double writing operations on the same data volume and data consistency of the same data writing operation on different data volumes are ensured.
In an alternative embodiment, before applying for the object range lock based on the data write instruction, the data write instruction is serially processed before the first data volume and the second data volume, further comprising: an execution order of the plurality of data write instructions is determined.
In an alternative embodiment, determining an order of execution of a plurality of data write instructions includes: acquiring an object identifier of a data object of the data write instruction; determining an object index of the object identifier; and determining the sequence of the data write instruction applying for the object range lock in the first data volume and the second data volume according to the object index.
In an alternative embodiment, determining an order in which the data write instructions apply for object scope locks on the first data volume and the second data volume according to the object index includes: performing double-modulo on the object index; under the condition that the object index modulo two result is zero, applying for object range lock on the first data volume preferentially according to the corresponding data writing instruction; and under the condition that the modulo-two result of the object index is one, applying the object range lock on the second data volume preferentially aiming at the corresponding data writing instruction.
According to the data double-writing method provided by the embodiment of the invention, based on the processing of the data object of the data writing instruction, one of the first data volume or the second data volume is randomly selected to apply for the object range lock, so that the data processing pressure of the first data volume and the second data volume is effectively balanced, and the data processing efficiency of the data double-writing operation is obviously improved.
In an alternative embodiment, after the first data volume and the second data volume apply for the object range lock, the serial processing is performed on the data write instruction, and further including: if at least one of the first data volume and the second data volume fails to lock when the object range lock is applied to based on the data writing instruction, the object range lock is applied to the first data volume and the second data volume again based on the data writing instruction until the data writing instruction is successfully applied to the object range locks of the first data volume and the second data volume.
In an alternative embodiment, after the first data volume and the second data volume apply for the object range lock, the serial processing is performed on the data write instruction, and further including: if at least one of the first data volume and the second data volume has a lock conflict failure when the application object range is locked based on the data writing instruction, the application object range lock is reapplied to the first data volume and the second data volume based on the data writing instruction until the application frequency of the application object range lock to the first data volume and the second data volume reaches the set serial application frequency.
In an alternative embodiment, after the number of applications for applying the object range lock to the first data volume and the second data volume reaches the set serial number of applications, the serial processing is performed on the data write command, and further including:
if the number of times of applying the object range lock to the first data volume and the second data volume reaches the set serial application number, the object range lock is successfully applied to the first data volume, but the object range lock is successfully applied to the second data volume, and the lock conflict failure occurs, the object range lock which is successfully applied to the first data volume is released;
or if the application of the object range lock to the first data volume fails in the lock conflict but the application of the object range lock to the second data volume succeeds, releasing the object range lock which succeeds in the application to the second data volume.
In an alternative embodiment, the method further comprises: and releasing the object range lock included in the second data write operation when the second execution result shows that the second data write operation is successfully executed.
In an alternative embodiment, the first response result is obtained according to the following operations:
Sequentially judging whether the object identifier, the client identifier and the locking range of the first data writing operation conflict with the object identifiers, the client identifiers and the locking ranges of the object range locks of other first data writing operations completed before the first data writing operation is executed by the first data volume; when it is determined that one of the object identifier, the client identifier and the locking range conflicts, generating a first response result for showing that a lock conflict failure occurs when the first data write operation is performed on the first data volume;
the second response result is obtained according to the following operation:
Sequentially judging whether the object identifier, the client identifier and the locking range of the first data writing operation conflict with the object range locks of other first data writing operations completed before the second data volume executes the first data writing operation; when it is determined that one of the object identification, the client identification, and the locking range conflicts, a second response result showing that a lock conflict failure occurred in performing the first data write operation on the second data volume is generated.
According to the data double-writing method provided by the embodiment of the invention, in different data volumes, based on the object identification, the client identification and the locking range of the first data writing operation, the object range lock of other data writing operation completed before the first data writing operation is compared, whether the first data writing operation collides with the object range lock of other data writing operation completed before the first data writing operation is judged, and the mutual exclusivity of different data writing operations in the same data volume is effectively ensured.
In an alternative embodiment, determining the first execution result of the first data write operation according to the first response result and the second response result includes: and when the first response result shows that the first data write operation is successfully executed on the first data volume, and the second response result shows that the first data write operation is successfully executed on the second data volume, judging that the first execution result of the first data write operation is successful.
According to the data double-writing method provided by the embodiment of the invention, when the first response result shows that the first data writing operation is successfully executed on the first data volume, and the second response result shows that the first execution result of the first data writing operation is judged to be the execution success when the second data volume successfully executes the first data writing operation. The data consistency of the same data writing operation on different data volumes is effectively ensured.
In an alternative embodiment, the locking range includes an offset and a data length of the data object.
In an alternative embodiment, the data write command is a data remote copy command, the first data volume is a home data volume, and the second data volume is a remote data volume.
In an alternative embodiment, the data write command is a data online migration command, the first data volume is a source data volume, and the second data volume is a target data volume.
The data double-writing method provided by the embodiment of the invention is simultaneously suitable for different scenes such as remote copying of the data stored in the distributed block, online migration of the data and the like, can meet the functional requirements of synchronous remote copying characteristics, online migration characteristics and double-activity characteristics of the distributed block storage, and ensures the mutual exclusivity and consistency of data operation.
In a second aspect, the present invention provides a data dual writing apparatus, the apparatus comprising:
the instruction receiving module is used for receiving a data writing instruction, wherein the data writing instruction carries a locking range of a data object;
the identification generation module is used for responding to the data writing instruction and generating an object identification and a client identification aiming at the data object;
The system comprises a write operation sending module, a write operation receiving module and a write operation receiving module, wherein the write operation sending module is used for sending a first data write operation comprising a data write request and an object range lock to a first data volume and a second data volume, and the object range lock comprises a locking range, an object identifier and a client identifier;
The response receiving module is used for receiving a first response result of executing the first data writing operation on the first data volume and a second response result of executing the first data writing operation on the second data volume;
and the writing result determining module is used for determining a first execution result of the first data writing operation according to the first response result and the second response result.
In a third aspect, the present invention provides a computer device comprising: the memory and the processor are in communication connection, computer instructions are stored in the memory, and the processor executes the computer instructions, so that the data double writing method of the first aspect or any corresponding implementation mode of the first aspect is executed.
In a fourth aspect, the present invention provides a computer program product comprising computer instructions for causing a computer to perform the data double writing method of the first aspect or any of its corresponding embodiments.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the related art, the drawings that are required to be used in the description of the embodiments or the related art will be briefly described, and it is apparent that the drawings in the description below are some embodiments of the present invention, and other drawings may be obtained according to the drawings without inventive effort for those skilled in the art.
FIG. 1 is a flow chart of a data double writing method according to an embodiment of the invention;
FIG. 2 is a flow chart of another data double writing method according to an embodiment of the invention;
FIG. 3 is a schematic diagram of an implementation flow of data double-write lock conflict determination in a specific application example of a data double-write method according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a lock conflict processing flow for a specific application example of a data dual writing method according to an embodiment of the present invention;
FIG. 5 is a block diagram of a data dual writing apparatus according to an embodiment of the present invention;
Fig. 6 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The distributed block storage provides storage resources for upper-layer applications such as cloud computing systems by creating data volumes on which data is stored into a distributed block storage cluster through RBD clients. Therefore, when complex characteristics of online data migration, synchronous remote copy, dual activity and the like of the distributed block storage are realized, the online migration characteristics are required to carry out data dual writing on a source data volume and a target data volume before the data migration is completed, and the synchronous remote copy and dual activity characteristics are required to carry out data dual writing on a local data volume and a remote data volume.
The invention provides a data double-writing method for distributed block storage. And performing mutual exclusion write operation on migration data operation and data write operation according to online migration characteristics, performing mutual exclusion operation on different client operations according to remote copy characteristics and dual activity characteristics, and ensuring consistency of data volumes.
Specifically, in an optional implementation manner, the RBD client is configured on a fixed data server, and when a data writing instruction of a user is received, the data double writing method provided by the embodiment of the invention is executed. The URL link of the user input data writing instruction may also be referred to as a client, and may also be configured in a fixed data server, where the data server may be the same data server or a different data server than the data server configuring the RBD client.
In an alternative implementation manner, the data write command may be a data remote copy command, where the first data volume is a local data volume and the second data volume is a remote data volume in this embodiment of the present invention.
In another alternative implementation manner, the data write instruction may be a data online migration instruction, where the first data volume in the embodiment of the present invention is a source data volume and the second data volume is a target data volume.
The data double-writing method provided by the embodiment of the invention is simultaneously suitable for different scenes such as remote copying of the data stored in the distributed block, online migration of the data and the like, can meet the functional requirements of synchronous remote copying characteristics, online migration characteristics and double-activity characteristics of the distributed block storage, and ensures the mutual exclusivity and consistency of data operation.
It should be noted that, the present invention is directed to a data double writing process under the scenarios of remote data replication, online data migration, etc., where the first data volume and the second data volume have the same data storage space. In order to facilitate data and storage space identification, different storage space identifications and the like may be configured for the first data volume and the second data volume, but without affecting the correspondence of the storage spaces of the first data volume and the second data volume.
In accordance with an embodiment of the present invention, a data double writing method embodiment is provided, it being noted that the steps shown in the flowchart of the figures may be performed in a computer system, such as a set of computer executable instructions, and, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order other than that shown or described herein.
In this embodiment, a data double writing method is provided, which may be used for the data server and the like, and fig. 1 is a flowchart of a data double writing method according to an embodiment of the present invention, as shown in fig. 1, where the flowchart includes the following steps:
step S101, a data write command is received, where the data write command carries a locking range of a data object.
In an alternative embodiment, the locking range may include an offset and a data length of the data object.
Specifically, the data write instruction may be a data online migration instruction or a data remote copy instruction, or the like. As is common in the art, data write instructions may include a lock range for a data object that requires a write operation, such as: offset of data and data length. Both offset and length are terms of art that are common to the art and are not specifically limited herein.
Step S102, in response to the data writing instruction, generating an object identifier and a client identifier for the data object.
Specifically, when a data write instruction is received, a client_id (client identifier) for transmitting the data write instruction may be acquired, and an object identifier for identifying the uniqueness of the data object may be generated for the data object of the data write instruction.
Step S103, a first data write operation comprising a data write request and an object range lock is sent to a first data volume and a second data volume, wherein the object range lock comprises a locking range, an object identification and a client identification.
Specifically, because the same data write command needs to perform data processing on the same data object in different data volumes, different data write commands need to perform data processing on different data objects in the same data volume. Thus, the same data write instruction issued concurrently to different data volumes herein may include the same object range lock.
Further, each object range lock sent to the same data volume should include a locking range, an object identification, and a client identification that are unique.
Step S104, a first response result of performing the first data writing operation on the first data volume and a second response result of performing the first data writing operation on the second data volume are received.
When the first data volume receives the first data write operation, lock conflict judgment can be performed according to an object range lock included in the first data write operation, and if the object range lock is allocated to other clients (users), it is judged that lock conflict failure occurs when the first data volume executes the first data write operation.
Specifically, the data object id (identifier), the client identifier and the locking range of the first data writing operation can be sequentially compared with the above information of the object range lock of the completed data writing operation, when the object identifiers are consistent, the client identifiers are compared, and if the client identifiers are still consistent, the locking range is further compared. If the object identifier, the client identifier and the locking range of the first data write operation are consistent with the above information of the object range lock of the certain completed data write operation, determining that the data write operation fails in the lock conflict in the write operation of the first data volume, and feeding back a first execution result for showing that the first data write operation fails in the lock conflict in the execution of the first data volume to the RBD client.
The locking range may include, among other things, an offset and a data length of the data object.
The second data volume performs a similar data processing procedure upon receipt of the first data write operation as the first data volume received the first data write operation.
Step S105, determining a first execution result of the first data writing operation according to the first response result and the second response result.
In an alternative embodiment, when the first answer result shows that the first data write operation is successfully performed on the first data volume, and the second answer result shows that the first data write operation is successfully performed on the second data volume, it is determined that the first execution result of the first data write operation is successful.
And if the first response result shows that the first data write operation is successfully executed on the first data volume and the second response result shows that the lock conflict failure occurs when the first data write operation is executed on the second data volume, or the second response result shows that the first data write operation is successfully executed on the second data volume and the first response result shows that the lock conflict failure occurs when the first data write operation is executed on the first data volume, or the first response result shows that the lock conflict failure occurs when the first data write operation is executed on the first data volume and the second response result shows that the lock conflict failure occurs when the first data write operation is executed on the second data volume, judging that the first execution result of the first data write operation is the execution failure.
According to the data double-writing method provided by the embodiment of the invention, when the first response result shows that the first data writing operation is successfully executed on the first data volume, and the second response result shows that the first execution result of the first data writing operation is judged to be the execution success when the second data volume successfully executes the first data writing operation. The data consistency of the same data writing operation on different data volumes is effectively ensured.
According to the data double-writing method provided by the embodiment of the invention, the object range lock is added in the data writing operation, when the data writing operation is executed by the first data volume and the second data volume, the mutual exclusion problem of the writing operation can be effectively processed through the object range lock, and the response results of the writing operation executed by different data volumes are timely obtained, so that the execution result of the writing operation is determined based on the first response result of the first data writing operation executed by the first data volume and the second response result of the first data writing operation executed by the second data volume, and therefore the effective mutual exclusion processing of the data writing operation is carried out on the data writing instruction, and the data consistency of the writing operation executed by the data writing instruction by different data volumes is ensured.
In this embodiment, a data double writing method is provided, which may be used for the data server and the like, and fig. 2 is a flowchart of the data double writing method according to an embodiment of the present invention, as shown in fig. 2, where the flowchart includes the following steps:
In step S201, a data write command is received, where the data write command carries a locking range of the data object.
Please refer to step S101 in the embodiment shown in fig. 1 in detail, which is not described herein.
In step S202, in response to the data write instruction, an object identification and a client identification for the data object are generated.
Please refer to step S102 in the embodiment shown in fig. 1 in detail, which is not described herein.
In step S203, a first data write operation including a data write request and an object range lock is sent to a first data volume and a second data volume, wherein the object range lock includes a locking range, an object identification, and a client identification.
Please refer to step S103 in the embodiment shown in fig. 1 in detail, which is not described herein.
Step S204 receives a first response result of performing the first data write operation on the first data volume and a second response result of performing the first data write operation on the second data volume.
In an alternative embodiment, the first response result may be obtained according to the following operations: and sequentially judging whether the object identifier, the client identifier and the locking range of the first data writing operation conflict with the object identifiers, the client identifiers and the locking ranges of the object range locks of other first data writing operations completed before the first data writing operation is executed on the first data volume. And generating a first response result for showing that the lock conflict fails when the first data write operation is performed on the first data volume when it is determined that the conflict occurs in one of the object identifier, the client identifier and the locking range.
The second response result is obtained according to the following operation: and sequentially judging whether the object identifier, the client identifier and the locking range of the first data writing operation conflict with the object range locks of other first data writing operations completed before the second data volume executes the first data writing operation. And generating a second response result for indicating that the lock conflict fails when the first data write operation is performed on the second data volume when it is determined that the conflict occurs in one of the object identifier, the client identifier and the locking range.
According to the data double-writing method provided by the embodiment of the invention, in different data volumes, based on the object identification, the client identification and the locking range of the first data writing operation, the object range lock of other data writing operation completed before the first data writing operation is compared, whether the first data writing operation collides with the object range lock of other data writing operation completed before the first data writing operation is judged, and the mutual exclusivity of different data writing operations in the same data volume is effectively ensured.
Step S205, determining a first execution result of the first data writing operation according to the first response result and the second response result.
Please refer to step S105 in the embodiment shown in fig. 1 in detail, which is not described herein.
According to the data double-writing method provided by the embodiment of the invention, if the first response result shows that the first data writing operation is executed on the first data volume and the lock conflict fails, or the second response result shows that the first data writing operation is executed on the second data volume and the lock conflict fails, the first execution result of the first data writing operation is judged to be the execution failure. Therefore, the problem of data difference among a plurality of data volumes caused by different execution results of data writing operation on different data volumes is effectively avoided.
In step S206, when the first execution result of the first data write operation is the execution failure, if the first response result shows that the first data write operation is executed successfully on the first data volume and the second response result shows that the lock conflict failure occurs when the first data write operation is executed on the second data volume, then the object range lock of the first data write operation that is applied successfully on the first data volume is released, and if the second response result shows that the first data write operation is executed successfully on the second data volume and the first response result shows that the lock conflict failure occurs when the first data write operation is executed on the first data volume, then the object range lock of the first data write operation that is applied successfully on the second data volume is released.
When the first execution result of the first data writing operation is the execution failure, if the data writing operation fails in a lock conflict in one of the different data volumes, the object range lock used when the data writing operation is successfully executed on the other data volumes is released in time, so that the occupation of storage resources is avoided, and the data consistency of a plurality of data volumes is ensured.
In step S207, when the first execution result of the first data writing operation is that the execution fails, the data writing instruction is further processed in series.
According to the data double-writing method provided by the embodiment of the invention, when the first execution result of the first data writing operation is the execution failure, the data writing instruction is processed in series in time. The problem of user experience degradation caused by direct feedback of double writing failure of data to the user is effectively avoided. Meanwhile, serial processing of data writing operation is adopted, so that the probability of double writing success of data is remarkably improved, and user experience is effectively improved.
In an alternative embodiment, step S207 may include:
Step S2071, based on the data writing instruction, applying object range lock to the first data volume and the second data volume.
Specifically, when the execution of the first data write operation issued to the first data volume and the second data volume in parallel fails, in order to ensure the probability of successful execution of the data write operation, step S207 performs serial processing on the data write instruction when the first execution result of the first data write operation is that the execution fails. The biggest difference between the serial processing of the data writing instruction and the parallel processing of the data writing operation is that the serial processing of the data writing instruction is to apply for the object range lock first, and when the object range lock is determined that neither the first data volume nor the second data volume is allocated to other clients, the first data writing operation is sent to the first data volume and the second data volume respectively. The execution efficiency of the data writing operation is effectively improved.
In an alternative embodiment, the following step S2070 is also performed before step S2071.
Step S2070, performing serial processing on the data write command, further includes: an execution order of the plurality of data write instructions is determined.
In an alternative embodiment, step S2070 may include:
step a1, an object identifier of a data object of a data write instruction is acquired.
Step a2, determining the object index of the object identifier.
And a3, determining the sequence of the data writing instruction for applying object range lock in the first data volume and the second data volume according to the object index.
Specifically, in an alternative embodiment, the object index may be subjected to modulo-two, and if the modulo-two result of the object index is zero, the object range lock is preferentially applied to the first data volume for the corresponding data write instruction. And under the condition that the modulo-two result of the object index is one, applying the object range lock on the second data volume preferentially aiming at the corresponding data writing instruction.
For example, an object index (objectindex) in an OID (Object Identifier ) of a data object is taken as a key value, which can be modulo 2. If the result of modulo 2 on the key value is 0, the data range lock may be applied to the first data volume first to execute the write operation of the data object, and then the data range lock may be applied to the second data volume to execute the write operation of the data object.
According to the data double-writing method provided by the embodiment of the invention, based on the processing of the data object of the data writing instruction, one of the first data volume or the second data volume is randomly selected to apply for the object range lock, so that the data processing pressure of the first data volume and the second data volume is effectively balanced, and the data processing efficiency of the data double-writing operation is obviously improved.
In an alternative embodiment, after the object range lock is applied to the first data volume and the second data volume according to step S2071, if at least one of the first data volume and the second data volume has failed to apply for the lock conflict when the object range lock is applied to the first data volume and the second data volume based on the data writing instruction, the object range lock is reapplied to the first data volume and the second data volume based on the data writing instruction until the data writing instruction is successful in applying for the object range lock of the first data volume and the second data volume.
Here, it is generally considered that, when the application flow of the object range lock is correct, the object range lock is applied to the first data volume and the second data volume again for multiple times, and the corresponding object range lock may be applied to both the first data volume and the second data volume successfully. The first data volume and the second data volume may be reapplied for the object range lock until the data write command applies successfully for the object range lock of both the first data volume and the second data volume.
In another alternative embodiment, after the object range lock is applied to the first data volume and the second data volume according to step S2071, if at least one of the first data volume and the second data volume has failed in lock conflict when the object range lock is applied to the first data volume and the second data volume based on the data writing instruction, the object range lock is applied to the first data volume and the second data volume again based on the data writing instruction until the number of times of application for the object range lock to the first data volume and the second data volume reaches the set serial application number.
Here, in order to avoid that the operation of newly applying the object range lock to the first data volume and the second data volume enters a dead loop due to a flow error or the like, the set serial application number is configured in advance. And stopping reapplying the object range lock to the first data volume and the second data volume when reapplying the object range lock to the first data volume and the second data volume until the application times of the object range lock to the first data volume and the second data volume reach the set serial application times. Therefore, the information of the failure of re-applying the object range lock to the first data volume and the second data volume can be fed back to the RBD client in time.
In an alternative embodiment, the reapplication of the object range lock to the first data volume and the second data volume is stopped while the number of applications of the object range lock to the first data volume and the second data volume reaches the set serial number of applications. And if the application of the object range lock to the first data volume is successful but the application of the object range lock to the second data volume fails in a lock conflict, releasing the object range lock which is successfully applied to the first data volume. And if the application of the object range lock to the first data volume fails in the lock conflict but the application of the object range lock to the second data volume succeeds, releasing the object range lock which succeeds in the application to the second data volume.
In step S2072, if the data write command is applied successfully to the object range locks of the first data volume and the second data volume, a second data write operation including a data write request and an object range lock confirmation request is sent to the first data volume and the second data volume.
Specifically, the object range lock validation request may include an object identification, a client identification, and a locking range of the data object of the second data write operation. Where the locking range may also include the offset and data length of the data object. If the data write command applies for success in the object range locks of the first data volume and the second data volume, the object range locks with success are locked in the first data volume and the second data volume.
Step S2073, receiving a third response result of the second data write operation performed on the first data volume and a fourth response result of the second data write operation performed on the second data volume.
When the second data write operation is received, the first data volume determines whether the object range lock carried by the second data write operation is consistent with the object range lock locked in the first data volume in step S2072, if so, the second data write operation is executed in the first data volume, and a third response result for indicating that the second data write operation is executed successfully in the first data volume is generated. If the first data volume and the second data volume are inconsistent, the second data writing operation is not continuously executed, and a third response result for showing that the second data writing operation fails to be executed on the first data volume is generated.
When the second data write operation is received, it is also required to determine whether the object range lock carried by the second data write operation is consistent with the object range lock locked in the first data volume in step S2072 in the second data volume, and if so, execute the second data write operation in the second data volume, and generate a third response result for indicating that the second data write operation is executed successfully in the second data volume. If the first data write operation is not consistent, the second data write operation is not continued, and a fourth response result for showing that the second data write operation fails to be executed on the second data volume is generated.
Step S2074, determining a second execution result of the second data writing operation according to the third response result and the fourth response result.
Here, similarly to step S105 and step S205, the second execution result of the second data write operation is determined to be execution success only when the third answer result shows that the second data write operation was execution success at the first data volume, and the fourth answer result shows that the second data write operation was execution success at the second data volume. In other cases, the second execution result of the second data write operation is determined to be the execution failure.
In the process of carrying out serial processing on a data writing instruction, the method firstly applies object range lock to a first data volume and a second data volume based on the data writing instruction, and then when the data writing instruction is applied successfully to the object range lock of the first data volume and the second data volume, a second data writing operation comprising a data writing request and an object range lock confirmation request is sent to the first data volume and the second data volume. Therefore, the data conflict of the parallel data writing operation is effectively processed, and mutual exclusivity of a plurality of data double writing operations on the same data volume and data consistency of the same data writing operation on different data volumes are ensured.
In step S208, when the second execution result shows that the second data write operation is successfully executed, the object range lock included in the second data write operation is released.
Specifically, in the process of executing serial processing on the data write instruction in step S207, the object range lock is applied first, and when the first data volume and the second data volume both successfully apply for the object range lock, the second data write operation is executed. The object range lock is realized when the second execution result shows that the second data writing operation is successfully executed. The object range lock included in the second data write operation is released when the second execution result shows that the second data write operation is successfully executed.
In this embodiment, a specific application example of a data double-writing method is provided, which may be used in the above-mentioned data server or the like, and fig. 3 is a schematic implementation flow diagram of data double-writing lock conflict judgment in the specific application example of the data double-writing method according to an embodiment of the present invention, and as shown in fig. 3, the specific application example of the data double-writing method provided in the embodiment of the present invention uses remote copy of data as an example, and for online migration characteristics, migration operations and data writing operations may be equivalent to remote copy characteristics, and perform data double-writing operations to a local volume and a remote volume. The specific application example of the data double writing method provided in this embodiment may include the following steps:
the RBD client (shown as clients 1&2 in fig. 3 and 4) receives a data write command (shown as IO in fig. 3 and 4) sent by the user.
The RBD client sends data write operations WriteOp to the home volume and the remote volume.
Different RBD clients respond to the IO, and can issue write operations WriteOp to the local volume and the remote volume simultaneously when performing data double write operations. Data WRITE operations WriteOp can be used to apply for object RANGE LOCKs, including WRITE Requests (WRITE) and object RANGE LOCK requests (range_lock_get), such as: data write operation WriteOp (oid 1, range1, data1, client1, new_range_lock). The object range lock request contains a data object id, a client_id, a data object lock range, where the data object lock range may include an offset and a data length.
The underlying data processing module of the RBD client obtains the object range lock, if it finds that the object range lock has been allocated to another client, then the obtaining of the object range lock fails, generates information for characterizing the failure of obtaining the object range lock, such as get_rw_range_locks () failure (oid 1, range1, client 1), and returns a response result for showing that a "lock conflict" failure occurs, such as: response (lock conflict, oid1, range1, clinet 1).
The bottom data processing module of the RBD client acquires the object range lock, if the object range lock is not allocated to other clients, the object range lock is acquired successfully, and a response result for showing that the response is successful is returned and a data writing operation is executed, for example: response (success, oid1, range1, clinet 1).
If the RBD client receives a response result returned from one of the home volume and the remote volume and used for indicating that the "lock conflict" failure occurs, recording the execution result of the data writing operation WriteOp as the "lock conflict" failure, and continuing to wait for the response result of the data writing operation WriteOp executed by the other of the home volume and the remote volume.
After receiving the response results of the data write operations WriteOp returned by the local volume and the remote volume, the RBD client receives the response results. If the response results of the concurrent issuing data writing operation WriteOp of the local volume and the remote volume pair are both response success, information for successful writing of the data writing instruction IO is returned to the user. If one of the response results of the concurrent issue data write WriteOp for the local volume and the remote volume pair is used for the failed response result of the lock conflict. Then immediately releasing the object range lock corresponding to the successfully executed write operation WriteOp, and returning to the initial state of the current IO processing of the data write instruction. For example: the RBD client sends RangeUnlockOp (oid 1, range1, clinet 1) and upon receiving information characterizing the success of the reply to RangeUnlockOp (oid 1, range1, clinet 1), determines that one of the home volume and the remote volume was successfully released prior to applying for a successful object range lock, such as: response (success, oid1, range1, clinet 1).
Further, according to the lock conflict processing flow schematic diagram of the specific application example of the data double-writing method provided by the embodiment of the present invention shown in fig. 4, a writing flow of serial application lock may be performed to process a "lock conflict" failure. If one of the response results of the concurrent issue data write WriteOp for the local volume and the remote volume pair is used for the failed response result of the lock conflict. The initial state of the IO processing of the data writing instruction is directly returned. Meanwhile, according to the lock conflict processing flow schematic diagram of the specific application example of the data double-writing method provided by the embodiment of the invention shown in fig. 4, a writing flow of serial lock application is performed to process the failure of lock conflict.
Specifically, the lock conflict processing flow of the specific application example of the data double-writing method provided by the embodiment of the invention may include the following steps:
first, the execution order of the serial application object range lock of the data write instruction IO is determined. Specifically, an object index of an OID (Object Identifier ) of a data object of the data write command IO may be used as a key value, the key value is modulo-2, if the result of the modulo-2 is 0, the object range lock of the data object of the data write command IO may be applied to the local volume first, and then the object range lock of the data object of the data write command IO may be applied to the remote volume. For example: when the RBD client applies for the object range lock of the data object of the data write instruction IO to the local volume, rangeLockOp (oid, range1, client 1) may be sent to the local volume, and the local volume locks the data object locking range corresponding to the object range lock based on get_rw_range_locks () (oid 1, range1, client 1). The home volume may feed back a response message to the RBD client to characterize success of the serial application object range lock, for example: reply (success, oid1, range1, client 1).
It should be noted that the OID of the data object may be used to generate an object identifier of the data object, for example: the object index of the OID of the data object may be taken as the object identification of the data object, i.e. the above-mentioned key value is taken as the object identification.
And secondly, when the RBD client applies for the object range lock to the local volume or the remote volume, if a lock conflict condition occurs, the bottom data processing module of the RBD client re-enqueues the object range lock request of the object range lock until the object range lock is acquired.
Further, after the RBD client applies for the object RANGE LOCK to the home volume or the remote volume, the RBD client issues a concurrent second data WRITE operation WriteOp2 to the home volume and the remote volume, where the data WRITE operation WriteOp includes a data WRITE Request (WRITE) and a RANGE LOCK acknowledge request (range_lock_info), for example: writeOp2 (oid, range1, data1, client 1). The range lock acknowledge request may also include a data object id, a client_id, a locking range of the data object, which may include an offset and a data length of the data object.
Further, the underlying data processing module of the RBD client receives the data write WriteOp, and compares the data object locked information, for example: chk_rw_range_lock is executed (oid 1, range1, data1, client 1). If the object range lock validation request of the first data volume and the object range lock validation request of the second data volume are both the same as the corresponding locked information, for example: the result of the execution is that get_rw_locks () succeeded. Then a data write operation WriteOp2 is performed, for example: judging that the member write transaction is successful, and returning a response result for showing that the data write operation WrintOp2 is successful to the RBD client after the data write operation WriteOp is performed, for example: response (success, oid, range1, client 1). And the RBD client releases the two-end data object range lock.
Note that, in parentheses oid of the above instructions such as the data writing operation in fig. 3 and fig. 4 may represent an object identifier of a data object of the RBD client transmitting the corresponding instruction, range1 may represent a locking range of the data object transmitted by the RBD client, client1 may represent a client identifier of the RBD client, and data1 may represent the data object transmitted by the RBD client. The "rough" on the remote volume indicates that the processing flow of the data writing operation and the like on the local volume is consistent with that of the remote volume, and the detailed description of the process flow on the local volume is given as an example, and the processing flow of the data writing operation and the like on the remote volume is omitted and will not be repeated.
The embodiment also provides a data dual writing device, which is used for implementing the above embodiment and the preferred implementation, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
The present embodiment provides a data dual writing apparatus, as shown in fig. 5, including:
The instruction receiving module 501 is configured to receive a data write instruction, where the data write instruction carries a locking range of a data object;
an identifier generation module 502, configured to generate an object identifier and a client identifier for a data object in response to a data write instruction;
A write operation sending module 503, configured to send a first data write operation including a data write request and an object range lock to a first data volume and a second data volume, where the object range lock includes a locking range, an object identifier, and a client identifier;
a reply receiving module 504, configured to receive a first reply result of performing the first data write operation on the first data volume and a second reply result of performing the first data write operation on the second data volume;
The write result determining module 505 is configured to determine a first execution result of the first data write operation according to the first response result and the second response result.
In some alternative embodiments, the write result determination module 505 includes:
The first judging unit is used for judging that the first execution result of the first data write operation is the execution failure when the first response result shows that the first data write operation is executed on the first data volume and/or the second response result shows that the first data write operation is executed on the second data volume and the lock conflict failure occurs.
In some alternative embodiments, the apparatus further comprises: the first releasing module is used for releasing the object range lock of the first data writing operation, which is applied to be successful in the first data volume, if the first response result shows that the first data writing operation is successfully executed in the first data volume and the second response result shows that the lock conflict failure occurs in the first data writing operation executed in the second data volume when the first execution result of the first data writing operation is the execution failure;
And the second releasing module is used for releasing the object range lock for which the first data writing operation is applied to be successful in the second data volume if the second response result shows that the first data writing operation is successfully executed in the second data volume but the first response result shows that the lock conflict failure occurs in the first data volume when the first data writing operation is executed in the first data volume.
In some alternative embodiments, the apparatus further comprises: and the conflict processing module is used for carrying out serial processing on the data writing instruction when the first execution result of the first data writing operation is the execution failure.
In some alternative embodiments, the conflict handling module includes:
the lock application unit is used for applying object range locks to the first data volume and the second data volume based on the data writing instruction;
the write request unit is used for sending a second data write operation comprising a data write request and an object range lock confirmation request to the first data volume and the second data volume if the data write instruction is applied successfully in the object range locks of the first data volume and the second data volume;
A result receiving unit for receiving a third response result of performing the second data write operation on the first data volume and a fourth response result of performing the second data write operation on the second data volume;
And the result determining unit is used for determining a second execution result of the second data writing operation according to the third response result and the fourth response result.
In some alternative embodiments, the conflict handling module further comprises: and the order determining unit is used for determining the execution order of the plurality of data writing instructions before the first data volume and the second data volume apply for the object range lock based on the data writing instructions.
In some alternative embodiments, the order determining unit comprises:
an identification acquisition subunit for acquiring an object identifier of a data object of the data write instruction;
An index determination subunit for determining an object index of the object identifier;
and the sequence determining subunit is used for determining the sequence of the data writing instruction applying for the object range lock in the first data volume and the second data volume according to the object index.
In some alternative embodiments, the order determination subunit employs the following operations to determine, from the object index, an order in which the data write instructions apply for the object range locks on the first data volume and the second data volume:
Performing double-modulo on the object index;
under the condition that the object index modulo two result is zero, applying for object range lock on the first data volume preferentially according to the corresponding data writing instruction;
and under the condition that the modulo-two result of the object index is one, applying the object range lock on the second data volume preferentially aiming at the corresponding data writing instruction.
In some alternative embodiments, the conflict handling module further comprises:
And the first repeating unit is used for re-applying the object range lock to the first data volume and the second data volume based on the data writing instruction if at least one of the first data volume and the second data volume fails to lock when the object range lock is applied to the first data volume and the second data volume based on the data writing instruction, and until the data writing instruction is successfully applied to the object range locks of the first data volume and the second data volume.
In some alternative embodiments, the conflict handling module further comprises:
And the second repeating unit is used for re-applying the object range lock to the first data volume and the second data volume based on the data writing instruction until the application times of applying the object range lock to the first data volume and the second data volume reach the set serial application times if at least one of the first data volume and the second data volume has lock conflict failure when applying the object range lock to the first data volume and the second data volume based on the data writing instruction.
In some alternative embodiments, the apparatus further comprises:
and the third releasing unit is used for releasing the object range lock included in the second data writing operation when the second execution result shows that the second data writing operation is successfully executed.
In some alternative embodiments, the first response result is obtained according to the following operations:
Sequentially judging whether the object identifier, the client identifier and the locking range of the first data writing operation conflict with the object identifiers, the client identifiers and the locking ranges of the object range locks of other first data writing operations completed before the first data writing operation is executed by the first data volume; when it is determined that one of the object identifier, the client identifier and the locking range conflicts, generating a first response result for showing that a lock conflict failure occurs when the first data write operation is performed on the first data volume;
the second response result is obtained according to the following operation:
Sequentially judging whether the object identifier, the client identifier and the locking range of the first data writing operation conflict with the object range locks of other first data writing operations completed before the second data volume executes the first data writing operation; when it is determined that one of the object identification, the client identification, and the locking range conflicts, a second response result showing that a lock conflict failure occurred in performing the first data write operation on the second data volume is generated.
In some alternative embodiments, the result determination module 505 includes:
and the success determining unit is used for judging that the first execution result of the first data writing operation is successful when the first response result shows that the first data writing operation is successfully executed on the first data volume and the second response result shows that the first execution result of the first data writing operation is successfully executed when the first data writing operation is successfully executed on the second data volume.
In some alternative embodiments, the locking range includes an offset and a data length of the data object.
In some alternative embodiments, the data write instruction is a data remote copy instruction, the first data volume being a home data volume and the second data volume being a remote data volume.
In some alternative embodiments, the data write instruction is a data online migration instruction, the first data volume is a source data volume, and the second data volume is a target data volume.
Further functional descriptions of the above respective modules and units are the same as those of the above corresponding embodiments, and are not repeated here.
The data double writing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application SPECIFIC INTEGRATED Circuit) Circuit, a processor and a memory executing one or more software or fixed programs, and/or other devices that can provide the above functions.
The embodiment of the invention also provides computer equipment, which is provided with the data double-writing device shown in the figure 5.
Referring to fig. 6, fig. 6 is a schematic structural diagram of a computer device according to an alternative embodiment of the present invention, as shown in fig. 6, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are communicatively coupled to each other using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the computer device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In some alternative embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple computer devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 10 is illustrated in fig. 6.
The processor 10 may be a central processor, a network processor, or a combination thereof. The processor 10 may further include a hardware chip, among others. The hardware chip may be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general-purpose array logic, or any combination thereof.
Wherein the memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform a method for implementing the embodiments described above.
The memory 20 may include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, memory 20 may optionally include memory located remotely from processor 10, which may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk, or solid state disk; the memory 20 may also comprise a combination of the above types of memories.
The computer device also includes a communication interface 30 for the computer device to communicate with other devices or communication networks.
The embodiments of the present invention also provide a computer readable storage medium, and the method according to the embodiments of the present invention described above may be implemented in hardware, firmware, or as a computer code which may be recorded on a storage medium, or as original stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and to be stored in a local storage medium, so that the method described herein may be stored on such software process on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, a solid state disk or the like; further, the storage medium may also comprise a combination of memories of the kind described above. It will be appreciated that a computer, processor, microprocessor controller or programmable hardware includes a storage element that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the methods illustrated by the above embodiments.
Portions of the present invention may be implemented as a computer program product, such as computer program instructions, which when executed by a computer, may invoke or provide methods and/or aspects in accordance with the present invention by way of operation of the computer. Those skilled in the art will appreciate that the form of computer program instructions present in a computer readable medium includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the manner in which the computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Herein, a computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by a computer.
Although embodiments of the present invention have been described in connection with the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope of the invention as defined by the appended claims.

Claims (18)

1. A method of double writing of data, the method comprising:
Receiving a data writing instruction, wherein the data writing instruction carries a locking range of a data object;
Generating an object identifier and a client identifier for the data object in response to the data write instruction;
Concurrently sending a first data write operation comprising a data write request and an object range lock to a first data volume and a second data volume, wherein the object range lock comprises the locking range, the object identification and the client identification;
Receiving a first response result of executing the first data writing operation on a first data volume and a second response result of executing the first data writing operation on a second data volume;
Determining a first execution result of the first data writing operation according to the first response result and the second response result;
when the first execution result of the first data writing operation is that the execution fails, carrying out serial processing on the data writing instruction;
Wherein the serial processing of the data write command includes:
applying an object range lock to the first data volume and the second data volume based on the data write instruction;
If the data writing instruction applies successfully to the object range locks of the first data volume and the second data volume, sending a second data writing operation comprising a data writing request and an object range lock confirmation request to the first data volume and the second data volume;
Receiving a third response result of the second data writing operation performed on the first data volume and a fourth response result of the second data writing operation performed on the second data volume;
and determining a second execution result of the second data writing operation according to the third response result and the fourth response result.
2. The method of claim 1, wherein determining the first execution result of the first data write operation based on the first answer result and the second answer result comprises:
and when the first response result shows that the first data write operation of the first data volume fails to have lock conflict and/or the second response result shows that the second data volume fails to have lock conflict, judging that the first execution result of the first data write operation is the execution failure.
3. The method of claim 1, wherein when the first execution result of the first data write operation is an execution failure, the method further comprises:
if the first response result shows that the first data write operation is successfully executed on the first data volume and the second response result shows that the lock conflict failure occurs when the first data write operation is executed on the second data volume, releasing the object range lock of the first data write operation, which is successfully applied for by the first data volume;
And if the second response result shows that the first data write operation is successfully executed on the second data volume but the first response result shows that the lock conflict failure occurs when the first data write operation is executed on the first data volume, releasing the object range lock of which the first data write operation is successfully applied on the second data volume.
4. The method of claim 1, wherein prior to applying for an object range lock based on the data write instruction, the serially processing the data write instruction further comprises:
Determining the execution sequence of a plurality of the data write instructions.
5. The method of claim 4, wherein determining the order of execution of the plurality of data write instructions comprises:
Acquiring an object identifier of a data object of the data writing instruction;
Determining an object index of the object identifier;
And determining the sequence of the data write instruction applying for object range lock in the first data volume and the second data volume according to the object index.
6. The method of claim 5, wherein determining the order in which the data write instructions apply for the object range locks on the first data volume and the second data volume based on the object index comprises:
performing double-modulo on the object index;
when the object index pair-wise modulo result is zero, preferentially applying for an object range lock on the first data volume according to a corresponding data writing instruction;
and if the result of the modulo-two of the object index is one, preferentially applying object range lock on the second data volume according to the corresponding data writing instruction.
7. The method of claim 1, wherein said serially processing said data write instructions after said first data volume and said second data volume apply for an object range lock further comprises:
and if at least one of the first data volume and the second data volume has a lock conflict failure when the object range lock is applied to the first data volume and the second data volume based on the data writing instruction, the object range lock is applied to the first data volume and the second data volume again based on the data writing instruction until the data writing instruction is applied successfully to the object range locks of the first data volume and the second data volume.
8. The method of claim 1, wherein said serially processing said data write instructions after said first data volume and said second data volume apply for an object range lock further comprises:
And if at least one of the first data volume and the second data volume is failed in lock conflict when the application object range is locked based on the data writing instruction, the application object range is newly applied to the first data volume and the second data volume based on the data writing instruction until the number of application times of the application object range lock to the first data volume and the second data volume reaches the set serial application times.
9. The method of claim 8, wherein the serially processing the data write instruction after the number of applications for the object range lock to the first data volume and the second data volume reaches a set serial number of applications, further comprising:
if the number of times of applying for the object range lock to the first data volume and the second data volume reaches the set serial number of times of applying for the object range lock to the first data volume is successful, but the lock conflict failure occurs when the object range lock is applied for the second data volume, releasing the object range lock which is successfully applied for the first data volume;
or if the application of the object range lock to the first data volume fails in lock conflict but the application of the object range lock to the second data volume succeeds, releasing the object range lock which succeeds in the application to the second data volume.
10. The method according to claim 1, wherein the method further comprises:
And releasing an object range lock included in the second data write operation when the second execution result shows that the second data write operation is successfully executed.
11. The method of claim 1, wherein the step of determining the position of the substrate comprises,
The first response result is obtained according to the following operation:
sequentially judging whether the object identifier, the client identifier and the locking range of the first data writing operation conflict with the object identifiers, the client identifiers and the locking ranges of object range locks of other first data writing operations completed before the first data volume executes the first data writing operation;
Generating a first response result for showing that a lock conflict failure occurs when the first data volume performs the first data write operation when it is determined that one of the object identifier, the client identifier and the locking range conflicts;
the second response result is obtained according to the following operation:
Sequentially judging whether the object identifier, the client identifier and the locking range of the first data writing operation conflict with the object identifiers, the client identifiers and the locking ranges of object range locks of other first data writing operations completed before the second data volume executes the first data writing operation;
and generating a second response result for showing that the lock conflict failure occurs when the second data volume executes the first data write operation when the object identifier, the client identifier and the locking range are judged to conflict.
12. The method of claim 1, wherein determining the first execution result of the first data write operation based on the first answer result and the second answer result comprises:
And when the first response result shows that the first data write operation is successfully executed on the first data volume, and the second response result shows that the first data write operation is successfully executed on the second data volume, judging that the first execution result of the first data write operation is successful execution.
13. The method of any of claims 1-12, wherein the locking range includes an offset and a data length of the data object.
14. The method of any of claims 1-12, wherein the data write instruction is a data remote copy instruction, the first data volume is a home data volume, and the second data volume is a remote data volume.
15. The method of any of claims 1-12, wherein the data write instruction is a data online migration instruction, the first data volume is a source data volume, and the second data volume is a target data volume.
16. A data dual writing apparatus, the apparatus comprising:
The instruction receiving module is used for receiving a data writing instruction, wherein the data writing instruction carries a locking range of a data object;
The identification generation module is used for responding to the data writing instruction and generating an object identification and a client identification aiming at the data object;
the write operation sending module is used for concurrently sending a first data write operation comprising a data write request and an object range lock to a first data volume and a second data volume, wherein the object range lock comprises the locking range, the object identifier and the client identifier;
The response receiving module is used for receiving a first response result of executing the first data writing operation on the first data volume and a second response result of executing the first data writing operation on the second data volume;
The writing result determining module is used for determining a first execution result of the first data writing operation according to the first response result and the second response result;
the conflict processing module is used for carrying out serial processing on the data writing instruction when the first execution result of the first data writing operation is that the execution fails;
Wherein the serial processing of the data write command includes:
a lock application unit, configured to apply, based on the data write instruction, an object range lock to the first data volume and the second data volume;
A write request unit, configured to send a second data write operation including a data write request and an object range lock acknowledgement request to the first data volume and the second data volume if the data write instruction applies successfully in the object range locks of the first data volume and the second data volume;
a result receiving unit configured to receive a third response result of the second data write operation performed on the first data volume and a fourth response result of the second data write operation performed on the second data volume;
and the result determining unit is used for determining a second execution result of the second data writing operation according to the third response result and the fourth response result.
17. A computer device, comprising:
A memory and a processor, the memory and the processor being communicatively coupled to each other, the memory having stored therein computer instructions, the processor executing the computer instructions to perform the data double writing method of any of claims 1 to 15.
18. A computer program product comprising computer instructions for causing a computer to perform the data double writing method of any of claims 1 to 15.
CN202410525955.2A 2024-04-29 2024-04-29 Data double-writing method, device, computer equipment and program product Active CN118093730B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410525955.2A CN118093730B (en) 2024-04-29 2024-04-29 Data double-writing method, device, computer equipment and program product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410525955.2A CN118093730B (en) 2024-04-29 2024-04-29 Data double-writing method, device, computer equipment and program product

Publications (2)

Publication Number Publication Date
CN118093730A CN118093730A (en) 2024-05-28
CN118093730B true CN118093730B (en) 2024-07-23

Family

ID=91142681

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410525955.2A Active CN118093730B (en) 2024-04-29 2024-04-29 Data double-writing method, device, computer equipment and program product

Country Status (1)

Country Link
CN (1) CN118093730B (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017113351A1 (en) * 2015-12-31 2017-07-06 华为技术有限公司 Method and device for writing data, and system

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111857545B (en) * 2019-04-28 2023-12-29 深信服科技股份有限公司 Read-write lock conflict judging method, system, equipment and computer medium
CN110489388A (en) * 2019-08-16 2019-11-22 上海霄云信息科技有限公司 A kind of implementation method and equipment locked for SCSI in distributed network storage system
US11494405B2 (en) * 2020-12-21 2022-11-08 EMC IP Holding Company LLC Lock contention resolution for active-active replication performed in conjunction with journal recovery
CN112925796A (en) * 2021-03-30 2021-06-08 中国建设银行股份有限公司 Write consistency control method, device, equipment and storage medium
CN113419870A (en) * 2021-06-20 2021-09-21 济南浪潮数据技术有限公司 Volume reading and writing method and device
CN116126234A (en) * 2022-12-26 2023-05-16 天津中科曙光存储科技有限公司 Data synchronization method, apparatus, device, storage medium, and program product
CN116204124B (en) * 2023-02-23 2024-03-22 安超云软件有限公司 Data processing method and system based on conflict lock and electronic equipment

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017113351A1 (en) * 2015-12-31 2017-07-06 华为技术有限公司 Method and device for writing data, and system

Also Published As

Publication number Publication date
CN118093730A (en) 2024-05-28

Similar Documents

Publication Publication Date Title
CN110647580A (en) Distributed container cluster mirror image management main node, slave node, system and method
CN112130748B (en) Data access method, network card and server
WO2012139064A2 (en) Cluster unique identifier
CN111400777B (en) Network storage system, user authentication method, device and equipment
CN113067866B (en) Batch file transmission method and device between heterogeneous systems
CN109040263B (en) Service processing method and device based on distributed system
CN109522043B (en) Method and device for managing configuration data and storage medium
CN116166198A (en) Data transmission method, device, system and storage medium
CN116701518A (en) Database cluster deployment method and device, computer equipment and storage medium
US20080172494A1 (en) Method and apparatus for determining communication protocol
CN115335803A (en) Equipment upgrading method, intelligent equipment and computer readable storage medium
CN115017169A (en) Management method and system of multi-cloud management platform
CN118093730B (en) Data double-writing method, device, computer equipment and program product
CN110798358A (en) Distributed service identification method and device, computer readable medium and electronic equipment
CN107172082B (en) File sharing method and system
CN115022307A (en) File downloading method and device, storage medium and electronic equipment
CN112306962B (en) File copying method, device and storage medium in computer cluster system
CN114553521A (en) Remote memory access method, device, equipment and medium
CN114116655A (en) Parallel file storage method, device, equipment and storage medium based on virtual machine
CN111935059B (en) Processing method, device and system for equipment identification
CN113434477B (en) Method, system, medium and server for storing log file
CN109376135B (en) Cluster file system management method and system
CN110502460B (en) Data processing method and node
CN113779052A (en) Data updating method, device, equipment and storage medium
CN118413573B (en) Resource management method, device, computer equipment, storage medium and product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant