WO2020037625A1

WO2020037625A1 - Distributed storage system and data read-write method therefor, and storage terminal and storage medium

Info

Publication number: WO2020037625A1
Application number: PCT/CN2018/102027
Authority: WO
Inventors: 袁振南; 简怀兵
Original assignee: 袁振南; 区链通网络有限公司
Priority date: 2018-08-23
Filing date: 2018-08-23
Publication date: 2020-02-27
Also published as: CN109691065B; CN109691065A

Abstract

Disclosed are a distributed storage system and a data read-write method therefor, and a storage terminal and a storage medium. The data read-write method for the distributed storage system comprises: a client generating multiple data read requests, wherein the multiple data read requests comprise a first data read request and a second data read request (S301); the client sending the first data read request to a first storage node and sending the second data read request to a read request coordination module (S302); the first storage node acquiring the first data read request and acquiring first data from stored data according to the first data read request; and the read request coordination module acquiring the second data read request and acquiring second data from data stored in a second storage node according to the second data read request (S303); the first storage node sending the first data to the client; and the read request coordination module sending the second data to the client (S304). In this way, a read expansion capability of the distributed storage system can be improved.

Description

Distributed storage system and data reading and writing method, storage terminal and storage medium

[Technical Field]

The present application relates to the field of information technology, and in particular, to a distributed storage system and a data read / write method, a storage terminal, and a storage medium.

【Background technique】

With the rapid increase in the amount of information, large-scale distributed storage systems have become increasingly important. To this end, Microsoft developed a general replication framework PacificA for large-scale distributed storage systems. PacificA can also be called a prototype system. The framework is simple, practical, provides strong consistency, and can be adapted to different replication strategies.

The inventor of this application has found in long-term research and development that Blockchain Communications Software (Blockchain Messenger, BCM) relies on a back-end distributed storage system to store massive data such as accounts and contacts. Currently, the back-end distributed storage system uses PacificA to replicate The protocol manages multiple copies of the same data. In the PacificA replication protocol, all read and write operations are initiated from the Leader node (master replica node). The read capacity of the data will not be improved as more data replicas are added, so its read scalability is poor.

[Summary of the Invention]

The technical problem mainly solved by this application is to provide a distributed storage system and a method for reading and writing data, a storage terminal, and a storage medium to improve the read expansion capability of the distributed storage system.

In order to solve the above technical problems, a technical solution adopted in the present application is to provide a data reading and writing method for a distributed storage system. The distributed storage system includes a client and a storage terminal, where the storage terminal includes a first storage node, at least two second storage nodes, and at least two read request coordination modules, and the read request coordination module is set corresponding to the second storage node, The first storage node is used to store data, and at least two second storage nodes are used to store data copies. The data read and write method includes: the client generates multiple data read requests, and the multiple data read requests include the first data read Request and the second data read request; the client sends the first data read request to the first storage node and the second data request to the read request coordination module; the first storage node obtains the first data request, and according to the first data request, The first data is obtained from the stored data; the read request coordination module obtains the second data read request, and obtains the second data from the data stored by the second storage node according to the second data request; the first storage node sends the first data to Client; the read request coordination module sends the second data to the client.

In order to solve the above technical problem, another technical solution adopted in the present application is to provide a distributed storage system including a client and a storage terminal, where the storage terminal includes a first storage node, at least two first storage nodes, Two storage nodes and at least two read request coordination modules. The read request coordination module is set corresponding to the second storage node. The first storage node is used to store data, and the at least two second storage nodes are used to store a copy of the data. For generating multiple data read requests, the multiple data read requests include a first data read request and a second data read request; the client is configured to send the first data read request to the first storage node and the second data read request to the read request coordination module. Data request; the first storage node is used to obtain the first data request, and obtains the first data from the stored data according to the first data request, and sends the first data to the client; the read request coordination module is used to obtain the second A data read request, and obtain the second data from the data stored by the second storage node according to the second data request, and copy the second data To the client.

In order to solve the above technical problem, another technical solution adopted in the present application is to provide a data reading and writing method for a distributed storage system. The distributed storage system includes a client and a storage terminal. The storage terminal includes a first storage node. At least two second storage nodes and at least two read request coordination modules, the read request coordination modules are set corresponding to the second storage node, the first storage node is used to store data, and the at least two second storage nodes are used to store data The copy and data read and write method includes: the first storage node obtains the first data read request from the client, and obtains the first data from the stored data according to the first data request, and sends the first data to the client; the read request The coordination module obtains the second data read request from the client, obtains the second data from the data stored by the second storage node according to the second data request, and sends the second data to the client.

In order to solve the above technical problem, another technical solution adopted in the present application is to provide a storage terminal. The storage terminal includes a first storage node for storing data, and the first storage node obtains a first data read request from a client. And obtaining the first data from the stored data according to the first data read request and sending the first data to the client; at least two second storage nodes for storing a copy of the data; a read request coordination module, Configured to obtain a second data read request from a client, obtain second data from data stored by a second storage node according to the second data request, and send the second data to the client.

In order to solve the above technical problem, another technical solution adopted in the present application is to provide a storage terminal, which includes: a storage module for storing data; a transceiver module for receiving data read requests from a client; a processing module , Used to obtain the requested data from the data stored in the storage module according to the data read request; the transceiver module further sends the requested data to the client.

In order to solve the above technical problem, another technical solution adopted in the present application is to provide a storage medium on which program data is stored, and when the program data is executed, the steps in the foregoing method are implemented.

The beneficial effect of the present application is that, unlike the prior art, the distributed storage system in the embodiment of the present application includes a client and a storage terminal. The storage terminal includes a first storage node, at least two second storage nodes, and at least two read nodes. The request coordination module, the read request coordination module is set corresponding to the second storage node, wherein the first storage node is used to store data, at least two second storage nodes are used to store a copy of the data, and the distributed storage system data read and write method The method includes: the client generates multiple data read requests, and the multiple data read requests include a first data read request and a second data read request; the client sends a first data read request to the first storage node and sends a first data read request to the read request coordination module. Two data requests; the first storage node obtains the first data request and obtains the first data from the stored data according to the first data request; the read request coordination module obtains the second data read request and retrieves the second data request from the second data request The second data is obtained from the data stored by the storage node; the first storage node sends the first data to the client; the read request A second transfer module transmits data to the client. In this way, the present application divides multiple data read requests generated by the client into a first data read request and a second data read request, and sends the first data read request to the first storage node to retrieve the data from the first storage. The node obtains the first data and sends a second data read request to the read request coordination module to obtain the second data from the second storage node through the read request coordination module. Therefore, the second storage node of the present application can communicate with the first storage The nodes jointly process multiple data read requests from the client, so that multiple data read requests generated by the client can be evenly distributed across the entire storage node cluster of the distributed storage system, thereby improving the read expansion capability of the distributed storage system.

[Brief Description of the Drawings]

In order to more clearly explain the embodiments of the present application or the technical solutions in the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some of the present application. For those of ordinary skill in the art, other embodiments may be obtained based on these drawings without paying creative labor.

FIG. 1 is a schematic structural diagram of an embodiment of a distributed storage system of the present application;

FIG. 2 is a structural block diagram of the distributed storage system of the embodiment of FIG. 1; FIG.

3 is a schematic structural diagram of an embodiment of a data reading and writing method of a distributed storage system according to the present application;

FIG. 4 is a schematic flowchart of step S203 in the data reading and writing method of the distributed storage system in the embodiment of FIG. 3; FIG.

5 is another schematic flowchart of step S203 in the data reading and writing method of the distributed storage system in the embodiment of FIG. 3;

FIG. 6 is a detailed flowchart of step S402 in the data reading and writing method of the distributed storage system in the embodiment of FIG. 5; FIG.

7 is a schematic structural diagram of another embodiment of a data reading and writing method for a distributed storage system of the present application;

8 is a schematic structural diagram of another embodiment of a distributed storage system of the present application;

9 is a schematic structural diagram of an embodiment of a storage terminal according to the present application;

10 is a schematic structural diagram of another embodiment of a storage terminal according to the present application;

FIG. 11 is a schematic structural diagram of an embodiment of a storage medium of the present application.

【detailed description】

The following describes the present application in detail with reference to the accompanying drawings and embodiments. It is particularly pointed out that the following examples are only used to illustrate the present application, but not to limit the scope of the present application. Similarly, the following embodiments are only some of the embodiments of the present application, but not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

In a distributed storage system, the mainstream data replication protocol is the PacificA replication protocol, which can reduce the cost of coordination and thereby improve the efficiency of reaching consensus. PacificA's replica data uses a master-slave framework to ensure data consistency. Multiple copies of the data are stored in a master replica node and multiple slave replica nodes. Data is read and written by the leader node. This data processing method is simple and can always be read from the leader node. Consistent data. Obviously, in this kind of system, as long as the Leader node is alive, the Follower node is in a warm-up state. Therefore, data read scalability is poor, and resources cannot be used well.

In order to solve the above problems, the present application first proposes a distributed storage system. As shown in FIG. 1, FIG. 1 is a schematic structural diagram of an embodiment of the distributed storage system of the present application; FIG. 2 is a schematic diagram of the distributed storage system of the embodiment of FIG. Structure diagram. The distributed storage system 101 of the present application includes a client 102 and a storage terminal 103, where the storage terminal 103 includes a first storage node 104, at least two second storage nodes 105, and at least two read request coordination modules 106, and read request coordination The module 106 is provided corresponding to the second storage node 105, wherein the first storage node 104 is used to store data, and at least two second storage nodes 105 are used to store a copy of the data.

The data stored by the first storage node 104 in this embodiment may be original data written by the client 102 or a copy of the original data, which is not specifically limited.

In an application scenario, the distributed storage system 101 in this embodiment may use the PacificA storage framework. The storage terminal 103 in this embodiment is used as a storage cluster, and is mainly responsible for reading and updating system data, and ensuring data reliability by using multiple copies. And availability; the distributed storage system 101 in this embodiment may further include a configuration management cluster, which is mainly responsible for maintaining replica information, such as the participating nodes of the replica, the master replica node, and the version number of the current replica.

The differences between the distributed storage system 101 of the present application and the traditional distributed storage system based on the PacificA storage framework include at least: the data storage point 103 of this embodiment is provided with a read request coordination module 106 and the read request coordination module 106 may be connected to the first storage node. 104 (Leader node) jointly processes multiple data read requests generated by the client 102. Therefore, in this way, multiple data read requests can be processed in a balanced manner through the Leader node and the Follower node, which can not only ensure the consistency of the read data It can improve the read scalability of data and make reasonable use of resources.

This application further proposes a data reading and writing method for a distributed storage system. As shown in FIG. 3, the method of this implementation is used to implement data reading of the distributed storage system 101 of the foregoing embodiment. The data reading and writing method of this embodiment It includes the following steps:

Step S301: The client 102 generates multiple data read requests, and the multiple data read requests include a first data read request and a second data read request.

Optionally, the client 102 in this embodiment may divide a plurality of data read requests into a first data read request and a second data read request according to a preset percentage x. Specifically, the preset percentage x in this embodiment is a ratio of the number of first data read requests to the total number of multiple data read requests.

Optionally, the preset percentage x in this embodiment satisfies the following formula:

Among them, p is the probability that the second storage node 105 is set in the second storage node set, n is the sum of the number of the first storage nodes 104 and the number of the second storage nodes 105, and n is greater than or equal to three.

The second storage node set is a majority formed by the read request coordination module 106 selecting a plurality of second storage nodes 105 according to P after receiving the second data read request. Majority is used for data copy management. It can adjust the number of write copies and read copies according to the actual situation, so that the data can achieve a balance between reliability and performance.

Optionally, p in this embodiment satisfies the following formula:

As shown in FIG. 2, n in this embodiment is equal to 3, that is, the storage terminal 103 includes one first storage node 104 and two second storage nodes 105. It can be known from the above formula that when n is equal to 3, p is equal to 1, that is, the probability that each second storage node 105 is set in the second storage node set is 1, so both the second storage nodes 105 are reading requests. The second storage node set formed by the coordination module 106.

Step S302: The client 102 sends a first data read request to the first storage node 104 and sends a second data request to the read request coordination module 106.

In this embodiment, the second data read request sent by the client 102 is managed by the read request coordination module 106, so that multiple second storage nodes 105 can process multiple second data read requests in a balanced manner.

Step S303: the first storage node 104 obtains the first data request, and obtains the first data from the stored data according to the first data request; the read request coordination module 106 obtains the second data read request, and retrieves the second data request from the first Acquire the second data from the data stored by the two storage nodes 105.

Optionally, the method shown in FIG. 4 is used in this application to implement the method for the first storage node 104 to obtain the first data request, and to obtain the first data from the stored data according to the first data request. Specifically, the method in this embodiment includes steps S401-S404:

Step S401: The first storage node 104 obtains a first data read request, and obtains first identification information of the first data from the first data read request.

Step S402: The first storage node 104 determines whether the first identification information matches the second identification information in the cache list of the first storage node, and if yes, proceeds to step S403, and if not, proceeds to step S404.

In the above application scenario, each storage node of the storage terminal 103 in this embodiment is provided with a prepare list, and the prepare list is used to cache at least the identification information of the data stored in the storage node.

Step S403: The first storage node 104 obtains the first data corresponding to the first identification information, and sends the first data corresponding to the first identification information to the client 102.

Of course, in this embodiment, after the first identification information and the second identification information are successfully matched, it can further determine whether the first data corresponding to the first identification information stored by the first storage node 104 is missing. If the first data is not missing, , The first data is sent to the client 102; if the first data is missing, the first identification information can be further matched with the third identification information stored by the first storage node 104, and operations such as data loss are not described herein. .

Step S404: The first storage node 104 sets the version number corresponding to the first data to zero, and sends the zeroed version number to the client 102.

When there is no first data in the cache list of the first storage node 104 or the first data is missing, the version number of the first data may be set to zero.

Optionally, in the present application, the method shown in FIG. 5 is used to implement the method for the read request coordination module 103 to obtain the second data read request, and to obtain the second data from the data stored by the second storage node 105 according to the second data request. . Specifically, the method in this embodiment includes steps S501-S505:

Step S501: The read request coordination module 106 obtains a second data read request, obtains a second storage node set, and sends the second data read request to the second storage node 105 in the second storage node set.

The second storage node set is not repeated here.

Step S502: The second storage node 105 receives the second data read request, obtains a version number corresponding to the second data according to the second data request, and sends the version number to the read request coordination module 106.

Step S503: The read request coordination module 106 receives the version numbers sent by all the second storage nodes 105, and obtains the maximum value of the multiple version numbers.

When the client 102 updates the data, the data enters the first storage node 104 for processing. The first storage node 104 first assigns a version number to the data, and then inserts the data into the prepare list. The data on the prepare list is arranged in order according to the version number, and then The data carrying the version number is sent to the second storage node 105, and the second storage node 105 also inserts the data into the prepare list. Each time the data is updated, the version number is incremented. Therefore, the second data corresponding to the largest version number is the latest second data.

Step S504: The read request coordination module 106 determines whether the second data corresponding to the maximum value is missing. If so, the process proceeds to step S505; if not, the process returns to step S501.

It should be noted that when returning to step S501, it is not necessary to receive the second data read request again.

During the data update process, the data update will be incomplete due to system power failure or data transmission problems, resulting in data loss. Therefore, you should determine whether the data is missing before obtaining the data.

Step S505: The read request coordination module 106 sends the second data corresponding to the maximum value to the client 102.

Step S304: The first storage node 104 sends the first data to the client 102, and the read request coordination module 106 sends the second data to the client 102.

In other embodiments, the read request coordination module may also be provided on each storage node of the storage terminal. When processing the read data request, only the read request coordination module set corresponding to the second storage node is selected to take effect.

Different from the prior art, the client 102 of the distributed storage system 101 in this embodiment divides the generated multiple data read requests into a first data read request and a second data read request, and sends the first data read request to the first A storage node 104 to obtain the first data from the first storage node 104 and send a second data read request to the read request coordination module 106 to obtain the second data from the second storage node 105 through the read request coordination module 106, Therefore, the second storage node 105 in this embodiment can co-process multiple data read requests from the client 102 with the first storage node 104, so that multiple data read requests generated by the client 102 can be evenly distributed in the distributed storage system. In the entire storage node cluster of 101, the read expansion capability of the distributed storage system 101 can be improved.

Optionally, the method shown in FIG. 6 may be used in this application to implement the method for the second storage node 105 to obtain the version number corresponding to the second data according to the second data request. Specifically, the method in this embodiment includes the following steps:

Step S601: The second storage node 105 obtains the first identification information of the second data from the second data read request.

Step S602: The second storage node 105 determines whether the first identification information matches the second identification information in the cache list of the second storage node, and if yes, proceeds to step S603.

Step S603: The second storage node 105 obtains a version number corresponding to the first identification information from the cache list, and sends the version number corresponding to the first identification information to the read request coordination module 106.

Further, if the first identification information does not match the second identification information in the cache list of the second storage node, the process proceeds to steps S604-S606.

Step S604: The second storage node 105 determines whether the first identification information matches the third identification information stored by the second storage node 105, and if yes, proceeds to step S605, and if not, proceeds to step S606.

Each storage node in this embodiment is equivalent to a server, which includes a cache list and disk storage space. When the required data is not in the cache list of the storage node, the data can be further queried from the disk storage space.

Step S605: The second storage node 105 obtains a version number corresponding to the first identification information, and sends the version number corresponding to the first identification information to the read request coordination module 106.

Step S606: The second storage node 105 sets the version number of the second data to zero, and sends the zeroed version number to the read request coordination module 106.

This application further proposes a data reading and writing method for a distributed storage system in another embodiment. As shown in FIG. 7, the method in this embodiment is used to implement data writing in the distributed storage system in the foregoing embodiment. Specifically, The method in this embodiment includes the following specific steps:

Step S701: The client 102 generates a data write request.

Step S702: The client 102 sends a data write request to the first storage node 104.

The data write requests of the distributed storage system 101 in this embodiment are all processed at the first storage node 104.

Step S703: The first storage node 104 receives the data write request, obtains the third data according to the data read request, and sends the third data to the second storage node 105.

Step S704: The second storage node 105 receives and saves the third data.

From the above analysis, it can be known that the second storage node 105 and the first storage node 104 in this embodiment can jointly process multiple data read requests from the client 102, so that the multiple data read requests generated by the client 102 can be evenly distributed in the distribution. In the entire storage node cluster of the distributed storage system 101, the read expansion capability of the distributed storage system 101 can be improved. At the same time, because the second storage node 105 shares the data read request, so that the first storage node 104 has more resources to process the write request, the throughput of the entire cluster data writing of the distributed storage system 101 can be improved.

This application further proposes a data reading and writing method of a distributed storage system according to another embodiment. The data reading and writing method of this embodiment includes: a first storage node 104 obtaining a first data read request from a client 102, and according to the first data Request to obtain the first data from the stored data, and send the first data to the client 102; the read request coordination module 106 obtains the second data read request from the client 102, and according to the second data request from the second storage The second data is acquired from the data stored by the node 105, and the second data is sent to the client 102.

Different from the prior art, this embodiment uses the second storage node 105 and the first storage node 104 to jointly process multiple data read requests from the client 102, so that multiple data read requests generated by the client 102 can be evenly distributed in the distribution. In the entire storage node cluster of the embedded storage system 101, the read expansion capability of the distributed storage system 101 can be improved

This application further proposes a distributed storage system according to another embodiment. As shown in FIG. 8, the distributed storage system 801 in this embodiment includes a client 802 and a storage terminal 803. The storage terminal 803 includes a first storage node 804, at least Two second storage nodes 805 and at least two read request coordination modules 806. The read request coordination module 806 is set corresponding to the second storage node 805. The first storage node 804 is used for storing data, and the at least two second storage nodes 805 are used for For storing a copy of the data; the client 802 is configured to generate multiple data read requests, and the multiple data read requests include a first data read request and a second data read request; the client 802 is configured to send a first to the first storage node 804 A data read request and a second data request to the read request coordination module 806; the first storage node 804 is configured to obtain the first data request, and obtain the first data from the stored data according to the first data request, and send the first data Sent to the client 802; the read request coordination module 806 is configured to obtain a second data read request, and from the data stored in the second storage node 805 according to the second data request A second data fetch, and the second data transmission to the client 802.

Different from the prior art, the client 802 of the distributed storage system 801 in this embodiment divides the generated multiple data read requests into a first data read request and a second data read request, and sends the first data read request to the first A storage node 804 to obtain the first data from the first storage node 804 and send a second data read request to the read request coordination module 806 to obtain the second data from the second storage node 805 through the read request coordination module 806, Therefore, the second storage node 805 in this embodiment can co-process multiple data read requests from the client 802 with the first storage node 804, so that multiple data read requests generated by the client 802 can be evenly distributed in the distributed storage system. In the entire storage node cluster of 801, the read expansion capability of the distributed storage system 801 can be improved.

Of course, the distributed storage system 801 in this embodiment may use the data reading and writing method in the foregoing method embodiment to read and write data, and details are not described herein.

This application further proposes a storage terminal. As shown in FIG. 9, FIG. 9 is a schematic structural diagram of an embodiment of a storage terminal according to the present application. The storage terminal 901 in this embodiment includes a first storage node 902, at least two second storage nodes 903, and a read request coordination module 904. The first storage node 902 is configured to store data, and the first storage node 902 obtains the first storage node from the client. A data read request, and obtains the first data from the stored data according to the first data read request, and sends the first data to the client; the second storage node 903 is used to store a copy of the data; the read request coordination module 904 is used A second data read request is obtained from the client, and the second data is obtained from the data stored by the second storage node 903 according to the second data request, and the second data is sent to the client.

Different from the prior art, the second storage node 903 of the storage terminal 901 in this embodiment can co-process the data read request of the client with the first storage node 902, so that multiple data read requests generated by the client can be evenly distributed on the storage terminal. In the entire storage node cluster of 901, the read expansion capability of the terminal 901 can be stored.

This application further proposes a storage terminal. As shown in FIG. 10, FIG. 10 is a schematic structural diagram of another embodiment of a storage terminal of the present application. The storage terminal 1001 in this embodiment includes a storage module 1002, a transceiver module 1003, and a processing module 1004. The storage module 1002 is configured to store data; the transceiver module 1003 is configured to receive a data read request from a client; the processing module 1004 is configured to read data Request to obtain the requested data from the data stored in the storage module 1002; the transceiver module 1003 further sends the requested data to the client.

Different from the prior art, this embodiment enables multiple data read requests generated by a client to be evenly distributed throughout the entire storage node cluster of the storage terminal 1001, thereby improving the read expansion capability of the storage terminal 1001.

This application further proposes a storage medium. As shown in FIG. 11, FIG. 11 is a schematic structural diagram of an embodiment of the storage medium of the present application. The storage medium 1101 of this embodiment is used to store the related data 1102 and program data 1103 of the foregoing embodiment, where the related data 1102 includes at least the above-mentioned read request and write request, and the program data 1103 can be executed by the method in the above method embodiment. The related data 1102 and the program data 1103 have been described in detail in the foregoing method embodiments, and are not described herein again.

The device 1101 with a storage function in this embodiment may be, but is not limited to, a U disk, an SD card, a PD optical drive, a mobile hard disk, a large-capacity floppy drive, a flash memory, a multimedia memory card, a server, and the like.

Different from the prior art, the distributed storage system in the embodiment of the present application includes a client and a storage terminal, where the storage terminal includes a first storage node, at least two second storage nodes, and at least two read request coordination modules, and read request coordination The module is set corresponding to a second storage node, where the first storage node is used to store data, and at least two second storage nodes are used to store a copy of the data. The distributed storage system data read and write method includes: the client generates multiple Data read request, the multiple data read requests include a first data read request and a second data read request; the client sends a first data read request to a first storage node and a second data request to a read request coordination module; the first storage The node obtains the first data request, and obtains the first data from the stored data according to the first data request; the read request coordination module obtains the second data read request, and obtains from the data stored by the second storage node according to the second data request. The second data; the first storage node sends the first data to the client; the read request coordination module sends the second data To the client. In this way, the present application divides multiple data read requests generated by the client into a first data read request and a second data read request, and sends the first data read request to the first storage node to retrieve the data from the first storage. The node obtains the first data and sends a second data read request to the read request coordination module to obtain the second data from the second storage node through the read request coordination module. Therefore, the second storage node of the present application can communicate with the first storage The nodes jointly process multiple data read requests from the client, so that multiple data read requests generated by the client can be evenly distributed across the entire storage node cluster of the distributed storage system, thereby improving the read expansion capability of the distributed storage system.

In addition, if the above functions are implemented in the form of software functions and sold or used as independent products, they can be stored in a mobile terminal readable storage medium, that is, this application also provides a storage device that stores program data. The program data can be executed to implement the method of the above embodiment, and the storage device may be, for example, a U disk, an optical disk, a server, or the like. That is, the present application may be embodied in the form of a software product, which includes several instructions for causing a smart terminal to perform all or part of the steps of the methods described in the embodiments.

In the description of the present application, the description with reference to the terms “one embodiment”, “some embodiments”, “examples”, “specific examples”, or “some examples” and the like means specific features described in conjunction with the embodiments or examples , Structure, material, or characteristic is included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms are not necessarily directed to the same embodiment or example. Moreover, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. In addition, without any contradiction, those skilled in the art may combine and combine different embodiments or examples and features of the different embodiments or examples described in this specification.

In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality" is at least two, for example, two, three, etc., unless it is specifically and specifically defined otherwise.

Any process or method description in a flowchart or otherwise described herein can be understood as a module, fragment, or portion of code that includes one or more executable instructions for implementing a particular logical function or step of a process And, the scope of the preferred embodiments of this application includes additional implementations in which the functions may be performed out of the order shown or discussed, including performing the functions in a substantially simultaneous manner or in the reverse order according to the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application pertain.

Logic and / or steps represented in a flowchart or otherwise described herein, for example, a sequenced list of executable instructions that may be considered to implement a logical function, may be embodied in any computer-readable medium, For use by, or in combination with, an instruction execution system, device, or device (which may be a personal computer, server, network device, or other system that can take instructions from the instruction execution system, device, or device and execute instructions) While using. For the purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connections (electronic devices) with one or more wirings, portable computer disk cartridges (magnetic devices), random access memory (RAM), Read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program can be printed, because, for example, by optically scanning the paper or other medium, followed by editing, interpretation, or other suitable Processing to obtain the program electronically and then store it in computer memory.

The above is only an implementation of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the description and drawings of the present application, or directly or indirectly applied to other related technologies The fields are equally included in the patent protection scope of this application.

Claims

A data reading and writing method for a distributed storage system, characterized in that the distributed storage system includes a client and a storage terminal, wherein the storage terminal includes a first storage node, at least two second storage nodes, and at least Two read request coordination modules, which are set corresponding to the second storage node, wherein the first storage node is used to store data and at least two of the second storage nodes are used to store the A copy of the data, the data reading and writing method comprising:

The client generates multiple data read requests, and the multiple data read requests include a first data read request and a second data read request;

Sending, by the client, the first data read request to the first storage node and sending the second data request to the read request coordination module;

The first storage node obtains the first data request, and obtains the first data from the stored data according to the first data request; the read request coordination module obtains the second data read request, and The second data request acquires second data from data stored by the second storage node;

The first storage node sends the first data to the client; the read request coordination module sends the second data to the client.
The data reading and writing method according to claim 1, wherein the read request coordination module obtains the second data read request, and from the data stored in the second storage node according to the second data request Obtaining the second data includes:

The read request coordination module obtains the second data read request, obtains a second storage node set, and sends the second data read request to all second storage nodes in the second storage node set;

Receiving, by the second storage node, the second data read request, obtaining a version number corresponding to the second data according to the second data request, and sending the version number to the read request coordination module;

The read request coordination module receives the version numbers sent by all the second storage nodes, and obtains a maximum value of the plurality of version numbers;

The read request coordination module determines whether the second data corresponding to the maximum value is missing;

If not, the read request coordination module sends second data corresponding to the maximum value to the client.
The method for reading and writing data according to claim 2, wherein if the second data corresponding to the maximum value is missing, the read request coordination module returns all the second data in the second storage node set to the second storage node set. The step of the storage node sending the second data read request.
The method for reading and writing data according to claim 2, wherein the obtaining, by the second storage node, a version number corresponding to the second data according to the second data request comprises:

Acquiring, by the second storage node, first identification information of the second data from the second data read request;

Determining whether the first identification information matches the second identification information in the cache list of the second storage node;

If yes, the second storage node obtains a version number corresponding to the first identification information from the cache list, and sends a version number corresponding to the first identification information to the read request coordination module.
The data reading and writing method according to claim 4, wherein if the first identification information does not match the second identification information in the cache list of the second storage node, determining the first identification information Whether it matches the third identification information stored by the second storage node;

If yes, the second storage node obtains a version number corresponding to the first identification information, and sends a version number corresponding to the first identification information to the read request coordination module.
The data reading and writing method according to claim 5, wherein if the first identification information does not match the third identification information stored by the second storage node, the second storage node will The version number corresponding to the second data is set to zero, and the version number after the zeroing is sent to the read request coordination module.
The data read / write method according to claim 2, wherein a ratio of the number of the first data read requests to a total number of the plurality of data read requests is a preset percentage.
The data reading and writing method according to claim 7, wherein the preset percentage satisfies a formula:

Where x is the preset percentage, p is the probability that the second storage node is set in the second storage node set, and n is the number of the first storage node and the second The sum of the number of storage nodes, and n is greater than or equal to three.
The data read / write method according to claim 8, wherein the p satisfies a formula:
The method for reading and writing data according to claim 1, wherein the acquiring the first data read request by the first storage node, and acquiring the first data from the stored data according to the first data request comprises:

Acquiring, by the first storage node, the first data read request, and acquiring first identification information of the first data from the first data read request;

Determining, by the first storage node, whether the first identification information matches second identification information in a cache list of the first storage node;

If yes, the first storage node acquires first data corresponding to the first identification information, and sends the first data corresponding to the first identification information to the client.
The method for reading and writing data according to claim 1, wherein the method for reading and writing data further comprises:

The client generates a data write request;

Sending, by the client, the data write request to the first storage node;

Receiving, by the first storage node, the data write request, obtaining third data according to the data write request, and sending the third data to the second storage node;

The second storage node receives and saves the third data.
A distributed storage system, characterized in that the distributed storage system includes a client and a storage terminal, wherein the storage terminal includes a first storage node, at least two second storage nodes, and at least two read request coordinations. A module, the read request coordination module is set corresponding to the second storage node, the first storage node is used to store data, and at least two of the second storage nodes are used to store a copy of the data;

The client is configured to generate multiple data read requests, and the multiple data read requests include a first data read request and a second data read request; the client is configured to send the first data read request to the first storage node. A data read request and sending the second data request to the read request coordination module;

The first storage node is configured to obtain the first data request, obtain the first data from the stored data according to the first data request, and send the first data to the client;

The read request coordination module is configured to obtain the second data read request, and obtain second data from data stored in the second storage node according to the second data request, and send the second data to The client.
A data reading and writing method for a distributed storage system, characterized in that the distributed storage system includes a client and a storage terminal, wherein the storage terminal includes a first storage node, at least two second storage nodes, and at least Two read request coordination modules, which are set corresponding to the second storage node, the first storage node is used to store data, and at least two of the second storage nodes are used to store the data A copy, the data reading and writing method includes:

The first storage node obtains the first data read request from the client, obtains the first data from the stored data according to the first data request, and sends the first data to the client The read request coordination module obtains the second data read request from the client, and obtains the second data from the data stored by the second storage node according to the second data request, and The second data is sent to the client.
A storage terminal, characterized in that the storage terminal includes:

A first storage node for storing data, the first storage node obtaining a first data read request from a client, and obtaining the first data from the stored data according to the first data read request, and Sending data to the client;

At least two second storage nodes for storing a copy of the data;

The read request coordination module is configured to obtain a second data read request from the client, obtain second data from data stored in the second storage node according to the second data request, and convert the second data To the client.
A storage terminal, characterized in that the storage terminal includes:

Storage module for storing data;

A transceiver module for receiving a data read request from a client;

A processing module, configured to obtain requested data from data stored in the storage module according to the data read request;

The transceiver module further sends the requested data to the client.
A storage medium having program data stored thereon, characterized in that when the program data is executed, the steps in the method according to any one of claims 1-11, 13 are implemented.