CN109445687B - Data storage method and protocol server - Google Patents

Data storage method and protocol server Download PDF

Info

Publication number
CN109445687B
CN109445687B CN201811143358.4A CN201811143358A CN109445687B CN 109445687 B CN109445687 B CN 109445687B CN 201811143358 A CN201811143358 A CN 201811143358A CN 109445687 B CN109445687 B CN 109445687B
Authority
CN
China
Prior art keywords
data
stripe
stripes
mirror image
mirror
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811143358.4A
Other languages
Chinese (zh)
Other versions
CN109445687A (en
Inventor
任永强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN201811143358.4A priority Critical patent/CN109445687B/en
Publication of CN109445687A publication Critical patent/CN109445687A/en
Application granted granted Critical
Publication of CN109445687B publication Critical patent/CN109445687B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

A data storage method and a protocol server are used for solving the technical problem that data write-in rate is reduced for data protection of a distributed storage system in the prior art. The method comprises the following steps: the protocol server receives first data; the first data is used for updating data stored in a first data stripe of a target stripe, the target stripe comprises N data stripes and M check stripes, and the first data stripe is any one of the K data stripes; respectively sending a write instruction to storage nodes where the L mirror image stripes are located, wherein the write instruction comprises the first data and the addresses of the mirror image stripes; establishing a corresponding relation between the L mirror image stripes and the first data stripe; and based on the corresponding relation, the protocol server can convert the reading instruction of the first data stripe into the reading instruction of any mirror image stripe. The embodiment of the invention is used for data storage.

Description

Data storage method and protocol server
Technical Field
The present invention relates to the field of computers, and in particular, to a data storage method and a protocol server.
Background
In the existing distributed storage system, a plurality of storage nodes are connected to form a cluster, and each storage node has a data storage function. A distributed storage system generally provides one or more Protocol servers (hereinafter referred to as "Protocol servers") to provide a standard Server Message Block (SMB) Protocol service function for a user.
As shown in fig. 1, the distributed storage system includes a Protocol server and a plurality of storage nodes, fig. 1 shows storage nodes 1 to 5, and a user program may access the storage system through a Protocol client (full name: Protocol client) as shown in fig. 1, where the Protocol client is connected with the Protocol server through a front-end network, the storage nodes in the distributed storage system are connected through a back-end network, the front-end network is used for performing request and data interaction between a user service and the distributed storage system, and the back-end network is used for performing request and data interaction between the storage nodes in the distributed storage system.
In the prior art, a Forward Error Correction (FEC) protection mode is usually adopted, so that a distributed storage system has an ability to continue providing data services even when a storage node fails. The forward error correction data protection mode is also called an N + M protection mode, where N represents the number of storage nodes, and M represents the number of storage nodes that are allowed by the cluster to fail at the same time on the premise that data loss does not occur. Under the N + M protection mode, the protocol server reads and writes by taking the stripe as a unit, one stripe comprises N data stripes and M check stripes, specifically, when writing in large block data, the protocol server divides the large block data into N small block data with fixed length, and according to the N small block data, M erasure codes are obtained by calculating through an algorithm, then the N data stripes in one stripe are used for storing the N small block data, and the M check stripes are used for storing the M erasure codes, wherein the erasure codes have redundant information, and when a storage node fails, the lost data can be calculated by using the erasure codes.
However, since the erasure code is calculated from the entire stripe, when data is newly written in any data stripe of the stripe, the protocol server needs to read the data and the erasure code in the entire stripe first, combine the data and the erasure code with the newly written data, recalculate the erasure code to obtain a new stripe, and write the new stripe into the storage node. As illustrated in fig. 1, fig. 1 shows a stripe, which includes data stripes 1 through 4 and a parity stripe, wherein, the data stripes 1 to 4 store data 1 to 4, the check stripes store the erasure codes of the stripes, the protocol server, according to any three data in the data stripes 1 to 4 and the erasure codes, the fourth data can be calculated, at this time, if the protocol server newly writes the data 5 into the data stripe 3, it is necessary to read all the data 1 to 4 stored in the data stripes 1 to 4 and the erasure codes in the parity stripes, replace the data 3 with the data 5, and recalculating according to the data 1, the data 2, the data 5 and the data 4 to obtain a new erasure code, and finally rewriting the data 1, the data 2, the data 5 and the data 4 and the new erasure code into the stripe.
As can be seen from the above, in the existing N + M protection mode, the process of writing new data into one striped data stripe by the protocol server is very complicated, and the data writing rate is slow.
Disclosure of Invention
The invention provides a data storage method and a protocol server, which are used for solving the technical problem that the data writing speed is reduced for data protection of a distributed storage system in the prior art.
In order to achieve the purpose, the invention adopts the following technical scheme:
in a first aspect, a data storage method is provided, including:
the protocol server receives first data; the first data is used for updating data stored in a first data stripe of a target stripe, the target stripe comprises N data stripes and M check stripes, the N data stripes are respectively specific storage spaces on N storage nodes, the M check stripes are respectively specific storage spaces on M storage nodes, N is a positive integer greater than 2, and M is a positive integer less than N; the first data stripe is any one of the K data stripes;
respectively sending a write instruction to storage nodes where L mirror strips are located, wherein the write instruction comprises the first data and addresses of the mirror strips; the L mirror image strips are specific storage spaces on at least two storage nodes, and L is a positive integer greater than 1;
establishing a corresponding relation between the L mirror image stripes and the first data stripe; and based on the corresponding relation, the protocol server can convert the reading instruction of the first data stripe into the reading instruction of any mirror image stripe.
In a first possible implementation manner with reference to the first aspect, after the establishing correspondence between the L mirror stripes and the first data stripe, the method further includes:
receiving second data; the second data is used for updating data stored in all data stripes in the target sub-stripe;
dividing the second data into N data, and calculating to obtain M erasure codes according to the N data; after the N data are written into the N data stripes and the M erasure codes are written into the M check stripes, the first data stored in the L mirror image stripes and the corresponding relation are deleted.
With reference to the first aspect or the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, after the deleting the first data and the corresponding relationship stored in the L mirror image stripe, the method further includes:
receiving a reading instruction sent by a protocol client; the reading instruction is used for instructing the protocol server to read the data stored in the first data stripe;
if the corresponding relation between the first data strip and the L mirror image strips exists, reading the first data stored in any mirror image strip corresponding to the first data strip according to the corresponding relation;
and returning the first data to the protocol client.
With reference to any one possible implementation manner of the first aspect to the second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, after the establishing a correspondence between the L mirror stripes and the first data stripe, the method further includes:
receiving a reading instruction sent by a protocol client; the reading instruction is used for instructing the protocol server to read the data stored in the first data stripe;
reading a first version number of data stored within the first data stripe;
reading the version number of the first data stored in any mirror image strip corresponding to the first data strip according to the corresponding relation;
according to the first version number and the second version number, determining that the writing time of the first data is later than the writing time of the data stored in the first data stripe;
reading the first data stored in any mirror image strip corresponding to the first data strip; and are
And returning the first data to the protocol client.
With reference to any one possible implementation manner of the first aspect to the third possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, the method further includes:
according to the first version number and the second version number, determining that the writing time of the first data is earlier than the writing time of the data stored in the first data stripe;
reading data stored in the first data stripe; and are
And returning the data stored in the first data stripe to the protocol client.
With reference to any one possible implementation manner of the first aspect to the fourth possible implementation manner of the first aspect, in a fifth possible implementation manner of the first aspect, the reading the first data stored in any mirror image stripe corresponding to the first data stripe includes:
if the storage node where the first mirror image strip in the L mirror image strips corresponding to the first data strip is located fails, reading the first data stored in the second mirror image strip in the L mirror image strips corresponding to the first data strip according to the corresponding relation.
In a second aspect, a protocol server is provided, comprising:
a receiving unit configured to receive first data; the first data is used for updating data stored in a first data stripe of a target stripe, the target stripe comprises N data stripes and M check stripes, the N data stripes are respectively specific storage spaces on N storage nodes, the M check stripes are respectively specific storage spaces on M storage nodes, N is a positive integer greater than 2, and M is a positive integer less than N; the first data stripe is any one of the K data stripes;
a write unit, configured to send a write instruction to storage nodes where L mirror stripes are located, where the write instruction includes the first data and an address of the mirror stripe; the L mirror image strips are specific storage spaces on at least two storage nodes, and L is a positive integer greater than 1;
the establishing unit is used for establishing the corresponding relation between the L mirror image stripes and the first data stripe; and based on the corresponding relation, the protocol server can convert the reading instruction of the first data stripe into the reading instruction of any mirror image stripe.
In a first possible implementation manner combined with the second aspect, the system further includes a calculating unit and a deleting unit;
the receiving unit is further configured to: receiving second data; the second data is used for updating data stored in all data stripes in the target sub-stripe;
the computing unit is used for dividing the second data into N data and computing M erasure codes according to the N data;
the deleting unit is configured to delete the first data stored in the L mirror image stripe and the corresponding relationship after the data writing unit writes the N data into the N data stripes and writes the M erasure codes into the M parity stripes.
With reference to the second aspect or the first possible implementation manner of the second aspect, in a second possible implementation manner of the second aspect, the method further includes: a reading unit and a transmitting unit;
the receiving unit is also used for receiving a reading instruction sent by the protocol client; the reading instruction is used for instructing the protocol server to read the data stored in the first data stripe;
the reading unit is configured to, when there is a correspondence between the first data stripe and the L mirror image stripes, read the first data stored in any mirror image stripe corresponding to the first data stripe according to the correspondence;
the sending unit is configured to return the first data to the protocol client.
With reference to any one possible implementation manner of the second aspect to the second possible implementation manner of the second aspect, in a third possible implementation manner of the second aspect, the apparatus further includes a reading unit and a sending unit;
the receiving unit is also used for receiving a reading instruction sent by the protocol client; the reading instruction is used for instructing the protocol server to read the data stored in the first data stripe;
the reading unit is used for reading a first version number of the data stored in the first data strip;
reading the version number of the first data stored in any mirror image strip corresponding to the first data strip according to the corresponding relation;
when the writing time of the first data is determined to be later than the writing time of the data stored in the first data stripe according to the first version number and the second version number, reading the first data stored in any mirror image stripe corresponding to the first data stripe;
the sending unit is configured to return the first data to the protocol client.
With reference to any one of the second to the third possible implementation manners of the second aspect, in a fourth possible implementation manner of the second aspect, the reading unit is further configured to read the data stored in the first data stripe when it is determined that the writing time of the first data is earlier than the writing time of the data stored in the first data stripe according to the first version number and the second version number;
the sending unit is further configured to return the data stored in the first data stripe to the protocol client.
With reference to any one possible implementation manner of the second aspect to the fourth possible implementation manner of the second aspect, in a fifth possible implementation manner of the second aspect, the protocol server includes a reading unit, configured to, if a storage node where a first mirror stripe in the L mirror stripes corresponding to the first data stripe is located fails, read the first data stored in a second mirror stripe in the L mirror stripes corresponding to the first data stripe according to the correspondence relationship.
In a third aspect, a protocol server is provided, which includes:
a memory for storing program code;
a processor for implementing the following by executing the program code in the memory:
receiving first data; the first data is used for updating data stored in a first data stripe of a target stripe, the target stripe comprises N data stripes and M check stripes, the N data stripes are respectively specific storage spaces on N storage nodes, the M check stripes are respectively specific storage spaces on M storage nodes, and N is a positive integer greater than 2; (ii) a
Respectively sending a write-in instruction to storage nodes where L mirror image strips are located, wherein the write-in instruction comprises the first data and addresses of the mirror image strips in the corresponding storage nodes; the L mirror strips are positioned on at least two storage nodes, the L mirror strips are L specific storage spaces, and L is a positive integer greater than 1;
establishing a corresponding relation between the L mirror image stripes and the first data stripe; and converting the reading instruction of the first data stripe into the reading instruction of any mirror image stripe based on the corresponding relation.
By adopting the scheme, the protocol server receives first data, wherein the first data is used for updating data stored in a first data stripe of a target stripe; sending a data writing instruction to a storage node where L mirror stripes are located, wherein the data writing instruction comprises the first data and addresses of the mirror stripes; establishing a corresponding relation between the L mirror image stripes and the first data stripe; the corresponding relation is used for reading the first data stored in any mirror image strip corresponding to the first data strip according to the corresponding relation when the protocol server receives a data reading instruction for reading the first data strip. That is to say, the protocol server writes the first data for updating the first data stripe into the L mirror image stripes, and does not directly replace the data stored in the first data stripe, so that the data in all the data stripes in the target stripe is not changed, and the erasure code does not need to be recalculated, and further the data stored in all the data stripes in the target stripe does not need to be read out first, so as to recalculate the erasure code, thereby simplifying the data writing process and improving the data writing rate. And when receiving a data reading instruction for reading the first data stripe, the protocol server can read the first data stored in any mirror image stripe corresponding to the first data stripe according to the corresponding relationship, thereby ensuring the correctness of data reading. And because the L mirror image strips are positioned on at least two storage nodes, the first data stored in the L mirror image strips are mutually backup, and after a certain storage node fails, the protocol server can still read the first data from the mirror image strips on other storage nodes, thereby realizing mirror image protection of the first data.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic structural diagram of a distributed storage system according to the present invention;
fig. 2 is a schematic flowchart of a data storage method according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a distributed storage system according to an embodiment of the present invention;
FIG. 4 is a flow chart illustrating another data storage method according to an embodiment of the present invention;
fig. 5 is a schematic flowchart of a data reading method according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a protocol server according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of another protocol server according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of another protocol server according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example one
An embodiment of the present invention provides a data storage method, as shown in fig. 2, the method includes:
s201, the protocol server receives first data, and the first data is used for updating data stored in a first data stripe of the target stripe.
The target stripe comprises N data stripes and M check stripes, the N data stripes are respectively specific storage spaces on N storage nodes, the M check stripes are respectively specific storage spaces on M storage nodes, K is a positive integer larger than 2, M is a positive integer smaller than N, and the first data stripe is any one of the K data stripes.
S202, the protocol server respectively sends a write instruction to the storage nodes where the L mirror strips are located, wherein the write instruction comprises the first data and the addresses of the mirror strips.
Wherein the L mirror stripes are specific storage spaces on at least two storage nodes, and L is a positive integer greater than 1.
S203, the protocol server establishes the corresponding relation between the L mirror image strips and the first data strip.
And based on the corresponding relation, the protocol server can convert the reading instruction of the first data stripe into the reading instruction of any mirror image stripe.
Through the steps S201 to S203, the protocol server does not directly replace the data stored in the first data stripe during the process of writing the first data, so that the data in all the data stripes in the target stripe is not changed, and the erasure code does not need to be recalculated.
The following describes steps S201 to S203 in detail.
Specifically, in step S201, the protocol server may be configured to receive the first data sent by the protocol client, for example, the protocol client sends a write instruction to the protocol server, where the write instruction includes the first data and a write address, and after receiving the data write instruction, the protocol server may determine, according to the write address, that the first data is data stored in a first data stripe of a stripe for an update purpose.
For example, in step S203, the corresponding relationship established by the protocol server may be a corresponding relationship between the identifier of the storage node where the first data stripe is located, the address of the first data stripe, the identifier of the storage node where each mirror image stripe is located, and the address of each mirror image stripe, so that the protocol server can more accurately read the data stored in any mirror image stripe according to the corresponding relationship.
For example, if the first data stripe is a first storage space on the storage node 1, the mirror stripe 1 is a second storage space on the storage node 2, and the mirror stripe 2 is a third storage space on the storage node 3, the protocol server may establish a corresponding relationship as shown in the following table 1:
TABLE 1
Storage node 1 Storage node 2 Storage node 3
A first storage space Second storage space A third storage space
Further, the protocol server can convert, based on the corresponding relationship, the read instruction into a read instruction for any mirror image stripe when receiving the read instruction for reading the first data stripe, that is, after step S203, if the protocol server receives the read instruction sent by the protocol client, the data read instruction is used to instruct the protocol server to read data stored in the first data stripe, and at this time, the protocol server may read the first data stored in any mirror image stripe corresponding to the first data stripe according to the corresponding relationship, and return the first data to the protocol client.
That is, referring to table 1 above, the read instruction sent by the protocol client includes the identifier of the storage node 1 and the identifier of the first storage space, so that the protocol server can convert the data read instruction into the data read instruction for reading the mirror stripe 1 according to table 1 above, and the data read instruction for reading the mirror stripe 1 includes the identifier of the storage node 2 and the address of the second storage space.
Further, after step S203 and before the protocol server receives the data reading instruction for reading the first data stripe, the protocol server may receive second data for updating data of all stripes of the destination stripe, and at this time, one possible implementation manner for the protocol server to write the second data into the destination stripe is as follows: the protocol server receives the second data, divides the second data into N data, calculates M erasure codes according to the N data, writes the N data into the N data stripes, writes the M erasure codes into the M check stripes, and deletes the first data and the corresponding relation stored in the L mirror image stripe.
Therefore, in a preferred implementation manner of the embodiment of the present invention, when receiving a data reading instruction sent by a protocol client and used for reading the first data stripe, the protocol server may first determine whether a corresponding relationship between the first data stripe and the L mirror stripes exists, if the corresponding relationship between the first data stripe and the L mirror stripes exists, read the first data stored in any mirror stripe corresponding to the first data stripe according to the corresponding relationship, and return the first data to the protocol client, and if the corresponding relationship between the first data stripe and the L mirror stripes does not exist, read the data stored in the first data stripe, and return the data stored in the first data stripe to the protocol client, thereby ensuring the correctness of data reading.
In addition, after step S203 and before the protocol server receives the second data, if the protocol server receives third data for updating the first stripe data, at this time, the protocol server may write the third data into the L mirror stripes according to the corresponding relationship, and directly overwrite the first data, thereby ensuring that when the corresponding relationship between the first data stripe and the mirror stripe exists, the data stored in the mirror stripe is always the latest data written by the protocol server.
It should be noted that, after the protocol client sends the first data for updating the data stored in the first data stripe to the protocol server, it may not be able to sense that the protocol server writes the first data into the L mirror stripes, and therefore, when the protocol client needs to read the first data, the read address in the data read instruction sent to the protocol server is still the address of the first data stripe.
The following describes the distribution relationship of N data stripes and L mirror stripes included in the first stripe on each storage node in the distributed storage system.
In a possible implementation manner of the embodiment of the present invention, N data stripes and M check stripes of the target stripe are respectively located in K different storage nodes, where N + M is equal to K, and at this time, M storage nodes may be allowed to fail in the K storage nodes, and the L mirror stripes may be located on at least two storage nodes in the K storage nodes.
It should be noted that, in a possible implementation manner of the embodiment of the present invention, the number L of mirror stripes included in a destination stripe may be greater than or equal to the number of parity stripes in the destination stripe, that is, if the destination stripe includes M parity stripes, at this time, the number L of mirror stripes included in the destination stripe may be greater than or equal to M, for example, L is M +1, so that, for the N + M protection manner, the number of storage nodes that are allowed to fail is M, and for the M +1 mirror stripes, the number of storage nodes that are allowed to fail is also M. As shown in fig. 3, the destination stripe includes four data stripes, namely, a data stripe 1 to a data stripe 4, and a check stripe, wherein the data stripe 1 to the data stripe 4 are respectively located on a storage node 1 to a storage node 4, and the check stripe is located on a storage node 5, at this time, on the premise that it is ensured that data stored in the entire destination stripe can be correctly read, it is allowed that one storage node among the storage nodes 1 to the storage node 5 fails, fig. 3 also shows two mirror stripes, namely, a mirror stripe 1 and a mirror stripe 2, because the mirror stripe 1 and the mirror stripe 2 are respectively located on the storage node 1 and the storage node 5, on the premise that data stored in the entire destination stripe can be correctly read, it is also allowed that one storage node among the storage nodes 1 to the storage node 5 fails.
The above is only an example, and in a specific implementation process, the mirror stripes may be distributed in other manners, for example, the L mirror stripes may be located on other storage nodes besides the K storage nodes. The invention is not limited in this regard.
The following describes protection of data when a storage node fails according to an embodiment of the present invention.
Specifically, the reading, by the protocol server, the first data stored in any mirror image stripe corresponding to the first data stripe according to the corresponding relationship specifically includes: if the storage node where the first mirror image strip in the L mirror image strips corresponding to the first data strip is located fails, the first data stored in the second mirror image strip in the L mirror image strips corresponding to the first data strip is read according to the corresponding relation.
Therefore, the L mirror image strips are positioned on the at least two storage nodes, the first data stored in the L mirror image strips are mutually backup, and after a certain storage node fails, the protocol server can still read the first data from the mirror image strips on other storage nodes, so that the mirror image protection of the first data is realized.
In addition, because the erasure code of the target stripe is not updated when the protocol server writes in the first data, when the protocol server reads data in the second data stripe of the target stripe, if the protocol server determines that the storage node where the second data stripe is located fails, the protocol server may query whether the second data stripe has a corresponding mirror stripe, if so, read data in the mirror stripe corresponding to the second data stripe, if not, read data stored in the N data stripes of the target stripe except the second data stripe and the erasure code stored in the parity stripe, and calculate data stored in the second data stripe according to the erasure code and the data stored in the other data stripes, thereby ensuring that the erasure code is not updated, erasure codes still have a data protection effect.
In the above, the protocol server deletes the data in the mirror image stripe and the corresponding relationship between the data stripe and the mirror image stripe when updating the data in all the data stripes, thereby ensuring the latest data written by the protocol server when updating the data stored in the mirror image stripe, however, in another possible implementation manner of the embodiment of the present invention, the protocol server may also determine the precedence relationship of the data writing time by corresponding to the version number of each data, at this time, after the protocol server receives the reading instruction sent by the protocol client, the reading instruction is used to instruct the protocol server to read the data stored in the first data stripe, the first version number of the data stored in the first data stripe may be read, and the version number of the first data stored in any mirror image stripe corresponding to the first data stripe may be read according to the corresponding relationship, and when the writing time of the first data is determined to be later than the writing time of the data stored in the first data stripe according to the first version number and the second version number, reading the first data stored in any mirror image stripe corresponding to the first data stripe, and returning the first data to the protocol client.
Optionally, when determining that the write time of the first data is earlier than the write time of the data stored in the first data stripe according to the first version number and the second version number, the protocol server reads the data stored in the first data stripe, and returns the data stored in the first data stripe to the protocol client.
Specifically, the protocol server establishes and records a corresponding version number for data in each stripe of the written destination stripe, and the version number may be used to indicate a precedence relationship of writing time of each data.
For example, after step S202, the protocol server may record the version number of the first data in the L mirror stripes, where the version number of the first data is equal to max (ec-stripe-ver, mirror-stripe-ver) + n.
Wherein, ec-stripe-ver is the maximum value of the version numbers of the data stored in the N data stripes of the destination stripe, mirror-stripe-ver is the maximum value of the version numbers of the data currently stored in the L mirror stripes, and N is a positive number greater than 0.
When the protocol server writes data into the data stripe of the destination stripe, the protocol server records the version number of each data as described above, and the larger the version number is, the later the data writing time is. Therefore, the implementation manner of determining the precedence relationship between the write time of the first data and the first version number by the protocol server according to the first version number and the second version number specifically includes: and determining the numerical value relationship between the first version number and the second version number, if the first version number is smaller than the second version number, determining that the writing time of the first data is later than the writing time of the data stored in the first data strip, and if the first version number is larger than the second version number, determining that the writing time of the first data is earlier than the writing time of the data stored in the first data strip.
By adopting the data storage method provided by the embodiment of the invention, the protocol server writes the first data for updating the first data stripe into the L mirror image stripes, and does not directly replace the data stored in the first data stripe, so that the data in all the data stripes in the target stripe are not changed, the erasure code does not need to be recalculated, the data stored in all the data stripes in the target stripe does not need to be read out first, the erasure code is recalculated, the data writing process is simplified, and the data writing rate is improved. And when receiving a data reading instruction for reading the first data stripe, the protocol server can read the first data stored in any mirror image stripe corresponding to the first data stripe according to the corresponding relationship, thereby ensuring the correctness of data reading. And because the L mirror image strips are positioned on at least two storage nodes, the first data stored in the L mirror image strips are mutually backup, and after a certain storage node fails, the protocol server can still read the first data from the mirror image strips on other storage nodes, thereby realizing mirror image protection of the first data.
Example two
In order to make those skilled in the art understand the technical solution of the data storage method provided by the present invention more, the following takes the distributed storage system shown in fig. 3 as an example to describe the data storage method provided by the embodiment of the present invention in detail, as shown in fig. 3, the data stripe 1 to the data stripe 4 of the target stripe are respectively located on the storage node 1 to the storage node 4, the check stripe is located on the storage node 5, and the mirror stripe 1 and the mirror stripe 2 are respectively located on the storage node 1 and the storage node 5. Based on the distributed storage system shown in fig. 3, a data storage method provided by an embodiment of the present invention is shown in fig. 4, and includes:
s401, the protocol server receives data A, and the data A is used for updating data in all data stripes in the target stripe.
Wherein, the protocol server may receive the data a sent by the protocol client.
S402, the protocol server divides the data A into 4 parts of data, and calculates and obtains an erasure code e according to the divided data a, data b, data c and data d.
For example, if the data size of the data a is 512KB, the protocol server may divide the data a into 4 data pieces with the same data size, that is, the data sizes of the data a, the data b, the data c and the data d are all 128KB, and the data size of the erasure code e calculated by the protocol server according to the data a, the data b, the data c and the data d is also 128 KB.
S403, the protocol server sequentially writes the data a to the data d into the data stripes 1 to 4 on the storage nodes 1 to 4, and writes the erasure code e into the parity stripe on the storage node 5.
S404, the protocol server writes the version number v of the data a into the data stripes 1 to 4 respectively.
Specifically, if the data a is data initially written in the destination stripe, the version number v of the data a may be a preset initial value of the version number, for example, the initial value is 1, and if the data a is not data initially written in the destination stripe, the version number v may be equal to max (ec-stripe-ver, mirror-stripe-ver) + n.
Where, ec-stripe-ver is the maximum value of the version numbers of the data currently stored in the data stripes 1 to 4, mirror-stripe-ver is the maximum value of the version numbers of the data currently stored in the mirror stripes 1 and 2, and n is a positive number greater than 0.
In addition, in step S404, the data a corresponds to a version number v, which is only an optional implementation manner of the embodiment of the present invention, that is, the version numbers of the data a to the data d are the same and are all v, however, in another possible implementation manner of the embodiment of the present invention, the data a to the data d and the erasure code e may also correspond to different version numbers, which is not limited by the present invention.
S405, the protocol server receives the data f written into the data stripe 1 of the destination stripe.
S406, the protocol server sends a first write command to the storage node 1, where the first write command includes the data f and the address of the mirror stripe 1, and sends a second write command to the storage node 5, where the second write command includes the data f and the address of the mirror stripe 2.
That is, the protocol server writes the data f into the mirror stripe 1 and the mirror stripe 2, respectively.
S407, the protocol server writes the version number u of the data f in the mirror stripe 1 and the mirror stripe 2, respectively.
Specifically, the version number u of the data f is greater than the version number v of the data a.
S408, the protocol server establishes a corresponding relationship between the data strip 1 and the mirror image strip 2.
The corresponding relationship may be, for example, a corresponding relationship between the identity of the storage node 1, and the version number v, and the identity of the storage node where each mirror stripe is located, and the version number u of the data f.
In this way, in the process of writing the data f, the protocol server does not directly replace the data stored in the data stripe 1, so that the data in all the data stripes in the target stripe is not changed, and therefore, the erasure code does not need to be recalculated.
For the above data storage method, an embodiment of the present invention further provides a data reading method based on the distributed storage system shown in fig. 3, as shown in fig. 5, including:
s501, the protocol server receives a data reading instruction, and the data reading instruction is used for reading data stored in the data strip 1.
S502, if the data stripe 1 does not have a corresponding mirror stripe, the protocol server reads the data in the data stripe 1.
S503, if the data stripe 1 has a corresponding mirror stripe, the protocol server obtains a version number v stored in the data stripe.
S504, the protocol server obtains the version number u stored in any mirror image strip corresponding to the data strip.
S505, if the version number v is greater than the version number u, the protocol server reads data from the data stripe 1.
And S506, if the version number v is smaller than the version number u, the protocol server reads data in a mirror image strip corresponding to the data strip.
Through the steps S501 to S506, when the protocol server receives a data reading instruction for reading the data stripe 1, if the data f written by the protocol server is stored in the mirror image stripe corresponding to the data stripe 1, the protocol server can read the data f stored in any mirror image stripe corresponding to the data stripe 1 according to the corresponding relationship, thereby ensuring the correctness of data reading. Moreover, the data f is stored in the mirror image stripe 1 on the storage node 1 and the mirror image stripe 2 on the storage node 5 respectively, that is, the data f stored in the mirror image stripe 1 and the data f stored in the mirror image stripe 2 are backup to each other, so that after a failure occurs in a certain storage node, the protocol server can still read the data f from the mirror image stripes on other storage nodes, thereby implementing mirror image protection on the data f.
It should be noted that, for simplicity of description, the above method embodiments are described as a series of action combinations, but those skilled in the art should understand that the present invention is not limited by the described action sequence, and those skilled in the art should understand that the embodiments described in the specification belong to the preferred embodiments, and the actions involved are not necessarily required by the present invention.
EXAMPLE III
An embodiment of the present invention provides a protocol server 60, configured to implement the data storage method provided in the first embodiment, as shown in fig. 6, where the protocol server 60 includes:
a receiving unit 61, configured to receive the first data.
The first data is used for updating data stored in a first data stripe of a target stripe, the target stripe comprises N data stripes and M check stripes, the N data stripes are respectively specific storage spaces on N storage nodes, the M check stripes are respectively specific storage spaces on M storage nodes, N is a positive integer greater than 2, and M is a positive integer less than N; the first data stripe is any one of the K data stripes;
and a write unit 62, configured to send a write instruction to the storage nodes where the L mirror stripes are located, where the write instruction includes the first data and an address of the mirror stripe.
Wherein the L mirror stripes are specific storage spaces on at least two storage nodes, and L is a positive integer greater than 1.
The establishing unit 63 is configured to establish a corresponding relationship between the L mirror stripes and the first data stripe.
And the protocol server can convert the reading instruction of the first data stripe into the reading instruction of any mirror image stripe based on the corresponding relation.
The first data received by the receiving unit 61 may be data sent by a protocol client to the protocol server, and the protocol client sends a write instruction to the protocol server, where the write instruction includes the first data and a write address, and the protocol server may determine, after receiving the write instruction, that the first data is data stored in a first data stripe of a stripe for update purposes according to the write address.
The corresponding relationship established by the protocol server may be a corresponding relationship between the identifier of the storage node where the first data stripe is located, the address of the first data stripe, the identifier of the storage node where each mirror image stripe is located, and the address of each mirror image stripe, so that the protocol server can more accurately read the data stored in any mirror image stripe according to the corresponding relationship.
Optionally, as shown in fig. 7, the protocol server 60 further includes a calculating unit 64 and a deleting unit 65, and the receiving unit 61 is further configured to: receiving second data; the second data is used for updating the data stored in all the data stripes in the target stripe; the calculating unit 64 is configured to divide the second data into N data, and calculate M erasure codes according to the N data; the deleting unit 65 is configured to delete the first data and the corresponding relationship stored in the L mirror image stripe after the data writing unit writes the N data into the N data stripes and writes the M erasure codes into the M parity stripes.
In an optional implementation manner of the embodiment of the present invention, as shown in fig. 7, the protocol server 60 further includes: a reading unit 66 and a sending unit 67, where the receiving unit 61 is further configured to receive a reading instruction sent by a protocol client, where the reading instruction is used to instruct the protocol server to read data stored in the first data stripe; the reading unit 66 is configured to, when there is a corresponding relationship between the first data stripe and the L mirror image stripes, read the first data stored in any mirror image stripe corresponding to the first data stripe according to the corresponding relationship; the sending unit 67 is configured to return the first data to the protocol client.
In addition, before the protocol server receives the second data, if third data for updating the first stripe data is received, at this time, the protocol server may write the third data into the L mirror stripes according to the corresponding relationship, and directly overwrite the first data, thereby ensuring that the data stored in the mirror stripe is always the latest data written by the protocol server when the corresponding relationship between the first data stripe and the mirror stripe exists.
Therefore, when the first data stripe has the corresponding mirror image stripe, the data stored in the mirror image stripe is the latest data written by the protocol server all the time, so that when the protocol server receives a data reading instruction for reading the data stored in the first data stripe, whether the mirror image stripe corresponding to the first data stripe exists can be firstly inquired, if so, the protocol server reads the data from the mirror image stripe, and if not, the protocol server reads the data from the first data stripe, thereby ensuring the correctness of data reading.
The value of the first data stripe is that, after the protocol client sends the first data for updating the data stored in the first data stripe to the protocol server, it may not be able to sense that the protocol server writes the first data into the L mirror stripes, and therefore, when the protocol client needs to read the first data, the read address in the data read instruction sent to the protocol server is still the address of the first data stripe.
Optionally, the reading unit 66 is further configured to, if a storage node where a first mirror stripe in the L mirror stripes corresponding to the first data stripe is located fails, read the first data stored in a second mirror stripe in the L mirror stripes corresponding to the first data stripe according to the corresponding relationship.
Therefore, the L mirror image strips are positioned on the at least two storage nodes, the first data stored in the L mirror image strips are mutually backup, and after a certain storage node fails, the protocol server can still read the first data from the mirror image strips on other storage nodes, so that the mirror image protection of the first data is realized.
In another optional implementation manner of the embodiment of the present invention, the receiving unit 61 is further configured to receive a reading instruction sent by a protocol client; the reading instruction is used for instructing the protocol server to read the data stored in the first data stripe; the reading unit 66 is configured to read a first version number of data stored in the first data stripe, read a version number of the first data stored in any mirror image stripe corresponding to the first data stripe according to the corresponding relationship, and read the first data stored in any mirror image stripe corresponding to the first data stripe when it is determined that the writing time of the first data is later than the writing time of the data stored in the first data stripe according to the first version number and the second version number; the sending unit 67 is configured to return the first data to the protocol client.
Optionally, the reading unit 66 is further configured to, when it is determined that the writing time of the first data is earlier than the writing time of the data stored in the first data stripe according to the first version number and the second version number, read the data stored in the first data stripe; the sending unit 67 is further configured to return the data stored in the first data stripe to the protocol client.
Specifically, the protocol server establishes and records a corresponding version number for data in each stripe of the written destination stripe, and the version number may be used to indicate a precedence relationship of writing time of each data.
For example, the protocol server may record, in the L mirror stripes, a version number of the first data, where the version number of the first data is equal to max (ec-stripe-ver, mirror-stripe-ver) + n.
Wherein, ec-stripe-ver is the maximum value of the version numbers of the data stored in the N data stripes of the destination stripe, mirror-stripe-ver is the maximum value of the version numbers of the data currently stored in the L mirror stripes, and N is a positive number greater than 0.
When the protocol server writes data into the data stripe of the destination stripe, the protocol server records the version number of each data as described above, and the larger the version number is, the later the data writing time is. Therefore, the implementation manner of determining the precedence relationship between the write time of the first data and the first version number by the protocol server according to the first version number and the second version number specifically includes: and determining the numerical value relationship between the first version number and the second version number, if the first version number is smaller than the second version number, determining that the writing time of the first data is later than the writing time of the data stored in the first data strip, and if the first version number is larger than the second version number, determining that the writing time of the first data is earlier than the writing time of the data stored in the first data strip.
It should be noted that, as is clear to those skilled in the art, for convenience and brevity of description, the specific working process of the storage controller described above may refer to the corresponding process of the foregoing method embodiment, and is not described herein again.
In addition, the above unit division of the protocol server is only a logic function division, there may be another division manner in actual implementation, and the specific physical implementation manner of each unit is not limited in the present invention, for example, in the specific implementation process, the receiving unit 61 and the sending unit 67 shown in fig. 7 may be a transceiver, the writing unit 62 and the reading unit 66 may be processors executing data read-write threads, and other implementation manners that can be considered by reasonable analysis and reasoning by those skilled in the art also fall within the protection scope of the present invention.
By adopting the protocol server provided by the embodiment of the invention, the protocol server does not directly replace the data stored in the first data stripe in the process of writing the first data, so that the data in all the data stripes in the target stripe is not changed, and the erasure code does not need to be recalculated. And when the protocol server reads the first data, the protocol server can read the first data stored in any mirror image strip corresponding to the first data strip according to the corresponding relation, so that the correctness of data reading is ensured. Moreover, because the L mirror image stripes are located on the at least two storage nodes, the first data stored in the L mirror image stripes are backups for each other, and after a failure occurs in a certain storage node, the protocol server can still read the first data from the mirror image stripes on other storage nodes, thereby implementing mirror image protection on the first data.
Example four
Another protocol server 80 is provided in the embodiment of the present invention, as shown in fig. 8, where the protocol server 80 includes:
a processor (processor)81, a communication Interface (Communications Interface)82, a memory (memory)83, and a communication bus 84; wherein, the processor 81, the communication interface 82 and the memory 83 complete mutual communication through the communication bus 84.
The processor 81 may be a multi-core central processing unit CPU or an application Specific Integrated circuit asic or one or more Integrated circuits configured to implement embodiments of the present invention.
The memory 83 is used for storing program code, which includes computer operating instructions and network flow diagrams. The memory 83 may comprise a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 83 may also be a memory array. The storage 83 may also be partitioned and the blocks may be combined into virtual volumes according to certain rules.
The communication interface 82 is used for realizing connection communication between the devices.
The processor 81 is configured to execute the program code in the memory 83 to implement the following operations:
receiving first data; the first data is used for updating data stored in a first data stripe of a target stripe, the target stripe comprises N data stripes and M check stripes, the N data stripes are respectively specific storage spaces on N storage nodes, the M check stripes are respectively specific storage spaces on M storage nodes, N is a positive integer greater than 2, and M is a positive integer less than N; the first data stripe is any one of the K data stripes;
respectively sending a write instruction to storage nodes where L mirror strips are located, wherein the write instruction comprises the first data and addresses of the mirror strips; the L mirror image strips are specific storage spaces on at least two storage nodes, and L is a positive integer greater than 1;
establishing a corresponding relation between the L mirror image stripes and the first data stripe; and based on the corresponding relation, the protocol server can convert the reading instruction of the first data stripe into the reading instruction of any mirror image stripe.
Optionally, after the establishing the correspondence between the L mirror stripes and the first data stripe, the operations further include:
receiving second data; the second data is used for updating data stored in all data stripes in the target sub-stripe;
dividing the second data into N data, and calculating to obtain M erasure codes according to the N data; after the N data are written into the N data stripes and the M erasure codes are written into the M check stripes, the first data stored in the L mirror image stripes and the corresponding relation are deleted.
Optionally, after the deleting the first data stored in the L mirror image stripe and the corresponding relationship, the operation further includes:
receiving a reading instruction sent by a protocol client; the reading instruction is used for instructing the protocol server to read the data stored in the first data stripe;
if the corresponding relation between the first data strip and the L mirror image strips exists, reading the first data stored in any mirror image strip corresponding to the first data strip according to the corresponding relation;
and returning the first data to the protocol client.
Optionally, after the establishing of the correspondence between the L mirror stripes and the first data stripe, the operations further include:
receiving a reading instruction sent by a protocol client; the reading instruction is used for instructing the protocol server to read the data stored in the first data stripe;
reading a first version number of data stored within the first data stripe;
reading the version number of the first data stored in any mirror image strip corresponding to the first data strip according to the corresponding relation;
according to the first version number and the second version number, determining that the writing time of the first data is later than the writing time of the data stored in the first data stripe;
reading the first data stored in any mirror image strip corresponding to the first data strip; and are
And returning the first data to the protocol client.
Optionally, the operations further comprise:
according to the first version number and the second version number, determining that the writing time of the first data is earlier than the writing time of the data stored in the first data stripe;
reading data stored in the first data stripe; and are
And returning the data stored in the first data stripe to the protocol client.
Optionally, the reading the first data stored in any mirror image stripe corresponding to the first data stripe specifically includes:
if the storage node where the first mirror image strip in the L mirror image strips corresponding to the first data strip is located fails, reading the first data stored in the second mirror image strip in the L mirror image strips corresponding to the first data strip according to the corresponding relation.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute some steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other media capable of storing program codes.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims (18)

1. A method of storing data, comprising:
the protocol server receives first data; the first data is used for updating data stored in a first data stripe of a target stripe, the target stripe comprises N data stripes and M check stripes, the N data stripes are respectively specific storage spaces on N storage nodes, the M check stripes are respectively specific storage spaces on M storage nodes, N is a positive integer greater than 2, and M is a positive integer less than N; the first data stripe is any one of the N data stripes;
respectively sending a write-in instruction to storage nodes where L mirror image strips are located, wherein the write-in instruction comprises the first data and addresses of the mirror image strips in the corresponding storage nodes; the L mirror strips are positioned on at least two storage nodes, the L mirror strips are L specific storage spaces, and L is a positive integer greater than 1;
establishing a corresponding relation between the L mirror image stripes and the first data stripe; the correspondence relationship comprises an identifier of a storage node where the first data stripe is located, an address of the first data stripe, identifiers of storage nodes where the L mirror image stripes are located, and correspondence relationship between the addresses of the L mirror image stripes, and based on the correspondence relationship, the protocol server can convert a read instruction for the first data stripe into a read instruction for any mirror image stripe.
2. The method of claim 1, wherein after the establishing the correspondence between the L mirror stripes and the first data stripe, the method further comprises:
receiving second data; the second data is used for updating data stored in all data stripes in the target sub-stripe;
dividing the second data into N data, and calculating to obtain M erasure codes according to the N data; after the N data are written into the N data stripes and the M erasure codes are written into the M check stripes, the first data stored in the L mirror image stripes and the corresponding relation are deleted.
3. The method according to claim 2, wherein after deleting the first data stored in the L mirror stripes and the corresponding relationship, further comprising:
receiving a reading instruction sent by a protocol client; the reading instruction is used for indicating to read the data stored in the first data stripe;
when the corresponding relation between the first data stripe and the L mirror image stripes exists, reading the first data stored in any mirror image stripe corresponding to the first data stripe according to the corresponding relation;
and returning the first data to the protocol client.
4. The method according to claim 1, wherein after establishing the correspondence between the L mirror stripes and the first data stripe, further comprising:
receiving a reading instruction sent by a protocol client; the reading instruction is used for instructing the protocol server to read the data stored in the first data stripe;
reading a first version number of data stored within the first data stripe;
reading a second version number of the first data stored in any mirror image strip corresponding to the first data strip according to the corresponding relation;
comparing the first version number with the second version number, wherein the second version number is the version number of the data stored in the first data strip;
when the writing time of the first data is determined to be later than the writing time of the data stored in the first data stripe, reading the first data stored in any mirror stripe corresponding to the first data stripe; and are
And returning the first data to the protocol client.
5. The method of claim 4, further comprising:
when the writing time of the first data is determined to be earlier than the writing time of the data stored in the first data stripe according to the first version number and the second version number;
reading data stored in the first data stripe; and are
And returning the data stored in the first data stripe to the protocol client.
6. The method according to any one of claims 3 to 5, wherein the reading the first data stored in any mirror stripe corresponding to the first data stripe comprises:
if the storage node where the first mirror image strip in the L mirror image strips corresponding to the first data strip is located fails, reading the first data stored in the second mirror image strip in the L mirror image strips corresponding to the first data strip according to the corresponding relation.
7. A protocol server, comprising:
a receiving unit configured to receive first data; the first data is used for updating data stored in a first data stripe of a target stripe, the target stripe comprises N data stripes and M check stripes, the N data stripes are respectively specific storage spaces on N storage nodes, the M check stripes are respectively specific storage spaces on M storage nodes, N is a positive integer greater than 2, and M is a positive integer less than N; the first data stripe is any one of the N data stripes;
a write unit, configured to send a write instruction to storage nodes where L mirror strips are located, where the write instruction includes the first data and an address of the mirror strip in a corresponding storage node; the L mirror strips are positioned on at least two storage nodes, the L mirror strips are L specific storage spaces, and L is a positive integer greater than 1;
the establishing unit is used for establishing the corresponding relation between the L mirror image stripes and the first data stripe; the correspondence relationship comprises an identifier of a storage node where the first data stripe is located, an address of the first data stripe, identifiers of storage nodes where the L mirror image stripes are located, and correspondence relationship between the addresses of the L mirror image stripes, and based on the correspondence relationship, the protocol server can convert a read instruction for the first data stripe into a read instruction for any mirror image stripe.
8. The protocol server according to claim 7, wherein the protocol server further comprises a calculation unit and a deletion unit;
the receiving unit is further configured to: receiving second data; the second data is used for updating data stored in all data stripes in the target sub-stripe;
the computing unit is used for dividing the second data into N data and computing M erasure codes according to the N data;
the deleting unit is configured to delete the first data stored in the L mirror image stripes and the corresponding relationship after the writing unit writes the N data into the N data stripes and writes the M erasure codes into the M parity stripes.
9. The protocol server of claim 8, wherein the protocol server further comprises: a reading unit and a transmitting unit;
the receiving unit is also used for receiving a reading instruction sent by the protocol client; the reading instruction is used for instructing the protocol server to read the data stored in the first data stripe;
the reading unit is configured to, when there is a correspondence between the first data stripe and the L mirror image stripes, read the first data stored in any mirror image stripe corresponding to the first data stripe according to the correspondence;
the sending unit is configured to return the first data to the protocol client.
10. The protocol server according to claim 7, wherein the protocol server further comprises a reading unit and a sending unit;
the receiving unit is also used for receiving a reading instruction sent by the protocol client; the reading instruction is used for instructing the protocol server to read the data stored in the first data stripe;
the reading unit is used for reading a first version number of the data stored in the first data strip;
reading a second version number of the first data stored in any mirror image strip corresponding to the first data strip according to the corresponding relation;
when the writing time of the first data is determined to be later than the writing time of the data stored in the first data stripe according to the first version number and the second version number, reading the first data stored in any mirror image stripe corresponding to the first data stripe;
the sending unit is configured to return the first data to the protocol client.
11. The protocol server according to claim 10, wherein the reading unit is further configured to read the data stored in the first data stripe when it is determined that the write time of the first data is earlier than the write time of the data stored in the first data stripe according to the first version number and the second version number;
the sending unit is further configured to return the data stored in the first data stripe to the protocol client.
12. The protocol server according to any one of claims 7 to 11, wherein the protocol server includes a reading unit, configured to, if a storage node where a first mirror stripe in the L mirror stripes corresponding to the first data stripe is located fails, read the first data stored in a second mirror stripe in the L mirror stripes corresponding to the first data stripe according to the correspondence relationship.
13. A protocol server, comprising:
a memory for storing program code;
a processor for implementing the following by executing the program code in the memory:
receiving first data; the first data is used for updating data stored in a first data stripe of a target stripe, the target stripe comprises N data stripes and M check stripes, the N data stripes are respectively specific storage spaces on N storage nodes, the M check stripes are respectively specific storage spaces on M storage nodes, N is a positive integer greater than 2, and M is a positive integer less than N; the first data stripe is any one of the N data stripes;
respectively sending a write-in instruction to storage nodes where L mirror image strips are located, wherein the write-in instruction comprises the first data and addresses of the mirror image strips in the corresponding storage nodes; the L mirror strips are positioned on at least two storage nodes, the L mirror strips are L specific storage spaces, and L is a positive integer greater than 1;
establishing a corresponding relation between the L mirror image stripes and the first data stripe; the correspondence relationship comprises an identifier of a storage node where the first data stripe is located, an address of the first data stripe, identifiers of storage nodes where the L mirror image stripes are located, and correspondence relationship between the addresses of the L mirror image stripes, and based on the correspondence relationship, the protocol server can convert a read instruction for the first data stripe into a read instruction for any mirror image stripe.
14. The protocol server according to claim 13, wherein the processor is further configured to, after the establishing the correspondence between the L mirror stripes and the first data stripe, further perform:
receiving second data; the second data is used for updating data stored in all data stripes in the target sub-stripe;
dividing the second data into N data, and calculating to obtain M erasure codes according to the N data; after the N data are written into the N data stripes and the M erasure codes are written into the M check stripes, the first data stored in the L mirror image stripes and the corresponding relation are deleted.
15. The protocol server according to claim 14, wherein the processor is further configured to, after deleting the first data stored in the L mirror stripes and the corresponding relationship, further perform:
receiving a reading instruction sent by a protocol client; the reading instruction is used for indicating to read the data stored in the first data stripe;
when the corresponding relation between the first data stripe and the L mirror image stripes exists, reading the first data stored in any mirror image stripe corresponding to the first data stripe according to the corresponding relation;
and returning the first data to the protocol client.
16. The protocol server according to claim 13, wherein the processor is further configured to, after establishing the correspondence between the L mirror stripes and the first data stripe, further perform:
receiving a reading instruction sent by a protocol client; the reading instruction is used for instructing the protocol server to read the data stored in the first data stripe;
reading a first version number of data stored within the first data stripe;
reading a second version number of the first data stored in any mirror image strip corresponding to the first data strip according to the corresponding relation;
comparing the first version number with the second version number, wherein the second version number is the version number of the data stored in the first data strip;
when the writing time of the first data is determined to be later than the writing time of the data stored in the first data stripe, reading the first data stored in any mirror stripe corresponding to the first data stripe; and are
And returning the first data to the protocol client.
17. The protocol server of claim 16, wherein the processor is further configured to:
when the writing time of the first data is determined to be earlier than the writing time of the data stored in the first data stripe according to the first version number and the second version number;
reading data stored in the first data stripe; and are
And returning the data stored in the first data stripe to the protocol client.
18. The protocol server of claim 13, wherein the protocol server further comprises a communication interface.
CN201811143358.4A 2015-09-30 2015-09-30 Data storage method and protocol server Active CN109445687B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811143358.4A CN109445687B (en) 2015-09-30 2015-09-30 Data storage method and protocol server

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811143358.4A CN109445687B (en) 2015-09-30 2015-09-30 Data storage method and protocol server
CN201510646279.5A CN105242879B (en) 2015-09-30 2015-09-30 A kind of date storage method and protocol server

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN201510646279.5A Division CN105242879B (en) 2015-09-30 2015-09-30 A kind of date storage method and protocol server

Publications (2)

Publication Number Publication Date
CN109445687A CN109445687A (en) 2019-03-08
CN109445687B true CN109445687B (en) 2022-04-26

Family

ID=55040542

Family Applications (2)

Application Number Title Priority Date Filing Date
CN201510646279.5A Active CN105242879B (en) 2015-09-30 2015-09-30 A kind of date storage method and protocol server
CN201811143358.4A Active CN109445687B (en) 2015-09-30 2015-09-30 Data storage method and protocol server

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN201510646279.5A Active CN105242879B (en) 2015-09-30 2015-09-30 A kind of date storage method and protocol server

Country Status (1)

Country Link
CN (2) CN105242879B (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106708651B (en) 2016-11-16 2020-09-11 北京三快在线科技有限公司 Partial writing method and device based on erasure codes, storage medium and equipment
WO2019071595A1 (en) * 2017-10-13 2019-04-18 华为技术有限公司 Method and device for storing data in distributed block storage system, and computer readable storage medium
CN109726036B (en) * 2018-11-21 2021-08-20 华为技术有限公司 Data reconstruction method and device in storage system
WO2020124608A1 (en) * 2018-12-22 2020-06-25 华为技术有限公司 Distributed storage system and computer program product
CN111367863B (en) * 2018-12-25 2024-07-23 深圳市茁壮网络股份有限公司 Method and system for modifying part of file
EP3889752B1 (en) 2018-12-25 2024-09-25 Huawei Cloud Computing Technologies Co., Ltd. Data storage method and apparatus in distributed storage system, and computer program product
CN111367864B (en) * 2018-12-25 2024-07-23 深圳市茁壮网络股份有限公司 Seamless replacement method, reading method and related system for file
CN111399780B (en) * 2020-03-19 2021-08-24 蚂蚁金服(杭州)网络技术有限公司 Data writing method, device and equipment

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101154174A (en) * 2002-06-24 2008-04-02 网络装置公司 Using file system information in raid data reconstruction and migration
CN102662607A (en) * 2012-03-29 2012-09-12 华中科技大学 RAID6 level mixed disk array, and method for accelerating performance and improving reliability

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8402213B2 (en) * 2008-12-30 2013-03-19 Lsi Corporation Data redundancy using two distributed mirror sets
CN101556802B (en) * 2009-05-25 2011-04-13 杭州华三通信技术有限公司 RAID array transposing method and device thereof

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101154174A (en) * 2002-06-24 2008-04-02 网络装置公司 Using file system information in raid data reconstruction and migration
CN102662607A (en) * 2012-03-29 2012-09-12 华中科技大学 RAID6 level mixed disk array, and method for accelerating performance and improving reliability

Also Published As

Publication number Publication date
CN105242879A (en) 2016-01-13
CN109445687A (en) 2019-03-08
CN105242879B (en) 2018-11-06

Similar Documents

Publication Publication Date Title
CN109445687B (en) Data storage method and protocol server
US11579777B2 (en) Data writing method, client server, and system
CN113485636B (en) Data access method, device and system
CN108628541B (en) File storage method, device and storage system
CN104636286B (en) For the method and apparatus of data access
US10303374B2 (en) Data check method and storage system
CN113656501B (en) Data reading method, device, equipment and storage medium
US8903804B2 (en) Data existence judging device and data existence judging method
US11899533B2 (en) Stripe reassembling method in storage system and stripe server
US20190114076A1 (en) Method and Apparatus for Storing Data in Distributed Block Storage System, and Computer Readable Storage Medium
US20170262220A1 (en) Storage control device, method of controlling data migration and non-transitory computer-readable storage medium
JP2019212175A (en) Storage control system, and storage control method
US11775194B2 (en) Data storage method and apparatus in distributed storage system, and computer program product
CN110955545B (en) Data integrity verification method, system and related equipment
CN109753225B (en) Data storage method and equipment
CN117591009A (en) Data management method, storage device and server
US12032849B2 (en) Distributed storage system and computer program product
CN109542674A (en) Snapshot creation method, device, equipment and the medium of distributed system
CN114442962B (en) Data reading method, device, equipment and storage medium
CN112988034B (en) Distributed system data writing method and device
CN112445413A (en) Data storage method and device and related equipment
CN114327293B (en) Data reading method, device, equipment and storage medium
US20230236932A1 (en) Storage system
CN114697349A (en) Method for processing data by using intermediate device, computer system and intermediate device
JP6605762B2 (en) Device for restoring data lost due to storage drive failure

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant