CN109691065A - Distributed memory system and its data read-write method, storage terminal and storage medium - Google Patents
Distributed memory system and its data read-write method, storage terminal and storage medium Download PDFInfo
- Publication number
- CN109691065A CN109691065A CN201880002511.3A CN201880002511A CN109691065A CN 109691065 A CN109691065 A CN 109691065A CN 201880002511 A CN201880002511 A CN 201880002511A CN 109691065 A CN109691065 A CN 109691065A
- Authority
- CN
- China
- Prior art keywords
- data
- memory node
- read request
- request
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application discloses a kind of distributed memory system and its data read-write methods, storage terminal and storage medium.The distributed memory system data read-write method includes: that client generates multiple data read requests, and multiple data read requests include the first data read request and the second data read request；Client sends the first data read request to the first memory node and sends the second request of data to read request Coordination module；First memory node obtains the first request of data, and the first data are obtained from the data of storage according to the first request of data；Read request Coordination module obtains the second data read request, and the second data are obtained from the data that the second memory node stores according to the second request of data；First data are sent to client by the first memory node；Second data are sent to client by read request Coordination module.In this way, it is able to ascend the reading extended capability of distributed memory system.
Description
Technical field
This application involves information technology field, more particularly to a kind of distributed memory system and its data read-write method,
Store terminal and storage medium.
Background technique
With the sharp increase of information content, large-scale distributed storage system is become more and more important.For this purpose, Microsoft is
Large-scale distributed storage system develops a general duplication frame PacificA, and PacificA is also referred to as a prototype
System.The frame is simple, practical, provides strong consistency, and can be adapted to different replication strategies.
Present inventor has found in long-term R&D work, block chain communication software
The distributed memory system that (BlockchainMessenger, BCM) relies on rear end stores the mass datas such as account, address list,
Rear end distributed memory system manages multiple data copies of the same data using PacificA replication protocol at present.?
In PacificA replication protocol, all read-write operations are all initiated from Leader node (primary copy node), the reading ability of data
It will not get a promotion with the addition of more data copies, therefore its reading scalability is poor.
Summary of the invention
The application is mainly solving the technical problems that provide a kind of distributed memory system and its data read-write method, storage
Terminal and storage medium, to promote the reading extended capability of distributed memory system.
In order to solve the above technical problems, the technical solution that the application uses is: providing a kind of distributed memory system
Data read-write method.The distributed memory system includes client and storage terminal, wherein storage terminal includes the first storage
Node, at least two second memory nodes and at least two read request Coordination modules, read request Coordination module are saved with the second storage
Point be correspondingly arranged, wherein the first memory node for storing data, the pair of at least two second memory nodes for storing data
This, data read-write method includes: that client generates multiple data read requests, and multiple data read requests include the first data read request
With the second data read request；Client sends the first data read request to the first memory node and sends to read request Coordination module
Second request of data；First memory node obtains the first request of data, and is obtained from the data of storage according to the first request of data
Take the first data；Read request Coordination module obtains the second data read request, and according to the second request of data from the second memory node
The second data are obtained in the data of storage；First data are sent to client by the first memory node；Read request Coordination module will
Second data are sent to client.
In order to solve the above technical problems, another technical solution that the application uses is: providing a kind of distributed storage system
System, the distributed memory system include client and storage terminal, wherein storage terminal includes the first memory node, at least two
A second memory node and at least two read request Coordination modules, read request Coordination module are correspondingly arranged with the second memory node,
First memory node for storing data, the copy of at least two second memory nodes for storing data；Client is for giving birth to
At multiple data read requests, multiple data read requests include the first data read request and the second data read request；Client is used for
The first data read request is sent to the first memory node and sends the second request of data to read request Coordination module；First storage section
Point and obtains the first data according to the first request of data for obtaining the first request of data from the data of storage, and by first
Data are sent to client；Read request Coordination module is for obtaining the second data read request, and according to the second request of data from the
The second data are obtained in the data of two memory nodes storage, and the second data are sent to client.
In order to solve the above technical problems, another technical solution that the application uses is: providing a kind of distributed storage system
The data read-write method of system, the distributed memory system include client and storage terminal, wherein storage terminal is deposited including first
Store up node, at least two second memory nodes and at least two read request Coordination modules, read request Coordination module and the second storage
Node is correspondingly arranged, the first memory node for storing data, the copy of at least two second memory nodes for storing data,
Data read-write method, which includes: the first memory node, obtains the first data read request from client, and according to the first request of data from
The first data are obtained in the data of storage, and the first data are sent to client；Read request Coordination module is obtained from client
Second data read request, and the second data are obtained from the data that the second memory node stores according to the second request of data, and will
Second data are sent to client.
In order to solve the above technical problems, another technical solution that the application uses is: providing a kind of storage terminal, this is deposited
Storing up terminal includes: the first memory node, and for storing data, the first memory node obtains the first data read request from client,
And the first data are obtained from the data of storage according to the first data read request, and the first data are sent to the client；
At least two second memory nodes, copy for storing data；Read request Coordination module, for obtaining the second number from client
The second data are obtained from the data that the second memory node stores according to read request, and according to the second request of data, and second is counted
According to being sent to client.
In order to solve the above technical problems, another technical solution that the application uses is: providing a kind of storage terminal, this is deposited
Storing up terminal includes: memory module, for storing data；Transceiver module, for receiving data read request from client；Handle mould
Block, for obtaining requested data from the data that memory module stores according to data read request；Transceiver module further will
Requested data are sent to client.
In order to solve the above technical problems, another technical solution that the application uses is: providing a kind of storage medium, thereon
It is stored with program data, which is performed the step realized in the above method.
The beneficial effect of the application is: being different from the prior art, the embodiment of the present application distributed memory system includes client
End and storage terminal, wherein storage terminal includes that the first memory node, at least two second memory nodes and at least two readings are asked
Coordination module is sought, read request Coordination module is correspondingly arranged with the second memory node, wherein the first memory node is for storing number
According to the copy of at least two second memory nodes for storing data, which includes: visitor
Family end generates multiple data read requests, and multiple data read requests include the first data read request and the second data read request；Client
It holds and sends the first data read request to the first memory node and send the second request of data to read request Coordination module；First storage
Node obtains the first request of data, and the first data are obtained from the data of storage according to the first request of data；Read request is coordinated
Module obtains the second data read request, and obtains the second number from the data that the second memory node stores according to the second request of data
According to；First data are sent to client by the first memory node；Second data are sent to client by read request Coordination module.It is logical
This mode is crossed, multiple data read requests that the application generates client are divided into the first data read request and the reading of the second data is asked
It asks, and the first data read request is sent to the first memory node, to obtain the first data from the first memory node, while by the
Two data read requests are sent to read request Coordination module, to obtain the second number from the second memory node by read request Coordination module
According to, therefore, the second memory node of the application can be jointly processed by multiple data read requests of client with the first memory node,
The multiple data read requests for generating client are balancedly distributed in the entire memory node cluster of distributed memory system
In, so as to promote the reading extended capability of distributed memory system.
Detailed description of the invention
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, below will be to institute in embodiment
Attached drawing to be used is needed to be briefly described, it should be apparent that, the accompanying drawings in the following description is only some implementations of the application
Example, for those of ordinary skill in the art, without creative efforts, can also obtain according to these attached drawings
Obtain other attached drawings.
Fig. 1 is the structural schematic diagram of one embodiment of the application distributed memory system；
Fig. 2 is the structural block diagram of Fig. 1 embodiment distributed memory system；
Fig. 3 is the structural schematic diagram of one embodiment of data read-write method of the application distributed memory system；
Fig. 4 is the detailed process signal of step S203 in the data read-write method of Fig. 3 embodiment distributed memory system
Figure；
Fig. 5 is that another detailed process of step S203 in the data read-write method of Fig. 3 embodiment distributed memory system is shown
It is intended to；
Fig. 6 is the idiographic flow schematic diagram of step S402 in the data read-write method of Fig. 5 embodiment distributed memory system；
Fig. 7 is the structural schematic diagram of another embodiment of data read-write method of the application distributed memory system；
Fig. 8 is the structural schematic diagram of another embodiment of the application distributed memory system；
Fig. 9 is the structural schematic diagram that the application stores one embodiment of terminal；
Figure 10 is the structural schematic diagram that the application stores another embodiment of terminal；
Figure 11 is the structural schematic diagram of one embodiment of the application storage medium.
Specific embodiment
With reference to the accompanying drawings and examples, the application is described in further detail.It is emphasized that following implement
Example is merely to illustrate the application, but is not defined to scope of the present application.Likewise, following embodiment is only the portion of the application
Point embodiment and not all embodiments, institute obtained by those of ordinary skill in the art without making creative efforts
There are other embodiments, shall fall in the protection scope of this application.
In distributed memory system, the data replication protocol of mainstream is PacificA replication protocol, it can reduce association
The expense of tune, to promote the efficiency reached common understanding.The copy data of PacificA uses master-slave mode frame, to guarantee data one
Cause property.Multiple copies of data are stored in primary copy (Leader) node and multiple from copy (Follower) node, number
According to read-write all by Leader node processing, this data processing method is simple, and can read always from Leader node
The preferable data of consistency.Obviously, in this kind of system, as long as Leader node is survived, Follower node is at one kind
Warm standby state, therefore, it is poor that data read scalability, and resource cannot be well used.
To solve the above problems, the application proposes a kind of distributed memory system first, as shown in Figure 1, Fig. 1 is the application
The structural schematic diagram of one embodiment of distributed memory system；Fig. 2 is the structural block diagram of Fig. 1 embodiment distributed memory system.This
Application distributed memory system 101 includes: client 102 and storage terminal 103, wherein storage terminal 103 includes the first storage
Node 104, at least two second memory nodes 105 and at least two read request Coordination modules 106, read request Coordination module 106
It is correspondingly arranged with the second memory node 105, wherein for storing data, at least two second storages save the first memory node 104
The copy of point 105 for storing data.
Wherein, the present embodiment the first memory node 104 storage data can be client 102 write-in former data or
The copy of person's original data, specifically without limitation.
In an application scenarios, PacificA storing framework, this reality is can be used in the present embodiment distributed memory system 101
The storage terminal 103 of example is applied as storage cluster, the reading and update of system data are mainly responsible for, by using more copy modes
Guarantee data reliability and availability；The present embodiment distributed memory system 101 can further comprise configuration management cluster, mainly
It is responsible for maintenance copy information, such as the participation node of copy, primary copy node, the version number etc. of current copy.
The application distributed memory system 101 and traditional distributed memory system based on PacificA storing framework
Difference includes at least: the dot data memory 103 of the present embodiment is provided with 106 read request Coordination module 106 of read request Coordination module
Multiple data read requests of the generation of client 102 can be jointly processed by with the first memory node 104 (Leader node), therefore,
In this way, multiple data read requests balanced can be handled by Leader node and Follower node, no
It can only guarantee the consistency of reading data, and be able to ascend data and read scalability, make resource rational utilization.
The application is it is further proposed that a kind of data read-write method of distributed memory system, as shown in figure 3, the side of this implementation
For realizing the reading data of above-described embodiment distributed memory system 101, the data read-write method of the present embodiment specifically wraps method
Include following steps:
Step S301: client 102 generates multiple data read requests, and multiple data read requests include the first data read request
With the second data read request.
Optionally, multiple data read requests can be divided into according to preset percentage x by the client 102 of the present embodiment
One data read request and the second data read request.Specifically, the preset percentage x of the present embodiment is the number of the first data read request
The ratio between the total quantity of amount and multiple data read requests.
Optionally, the preset percentage x of the present embodiment meets following formula:
Wherein, p is that the probability in the second memory node set is arranged in the second memory node 105, and n is the first memory node
The sum of the quantity of 104 quantity and the second memory node 105, and n is greater than or equal to 3.
Wherein, the second memory node set is to be selected after read request Coordination module 106 receives the second data read request according to P
Select the majority that multiple second memory nodes 105 form.Majority is used for data copy management, can adjust according to the actual situation
It is whole to write number of copies and read number of copies, so that data is obtained balance between reliability and performance.
Optionally, the p of the present embodiment meets following formula:
As shown in Fig. 2, the n of the present embodiment is equal to 3, i.e., storage terminal 103 includes first memory node 104 and two
Second memory node 105.As shown from the above formula, when n is equal to 3, p is equal to 1, that is to say, that each second memory node 105 is set
Setting the probability in the second memory node set is 1, and therefore, two the second memory nodes 105 are in read request Coordination module 106
In the second memory node set formed.
Step S302: client 102 sends the first data read request to the first memory node 104 and coordinates mould to read request
Block 106 sends the second request of data.
The present embodiment is managed by the second data read request that read request Coordination module 106 sends client 102,
So that multiple second memory nodes 105 are capable of multiple second data read requests of processing of equilibrium.
The S303: the first memory node of step 104 obtains the first request of data, and according to the first request of data from the number of storage
According to the first data of middle acquisition；Read request Coordination module 106 obtains the second data read request, and according to the second request of data from second
The second data are obtained in the data that memory node 105 stores.
Optionally, the application realizes that above-mentioned first memory node 104 obtains the first data and asks using method as shown in Figure 4
It asks, and obtains the method for the first data from the data of storage according to the first request of data.Specifically, the method packet of the present embodiment
Include step S401-S404:
The S401: the first memory node of step 104 obtains the first data read request, and obtains from the first data read request the
First identification information of one data.
The S402: the first memory node of step 104 judge the first identification information whether the cache list with the first memory node
In the second identification information matching, if so, S403 is entered step, if it is not, then entering step S404.
In above-mentioned application scenarios, each memory node of the storage terminal 103 of the present embodiment is provided with prepare
List, prepare list are at least buffer memory in the identification information of the data of memory node.
The S403: the first memory node of step 104 obtains the first data corresponding with the first identification information, and will know with first
Corresponding first data of other information are sent to client 102.
Certainly, the present embodiment can also further judge after the first identification information and the second identification information successful match
Whether the first data corresponding with the first identification information of the first memory node 104 storage lack, if first data do not lack
It loses, then first data is sent to client 102；It, can be further by the first identification information if first shortage of data
Match with the third identification information of the first memory node 104 storage and shortage of data etc. operates, not repeated here.
The S404: the first memory node of step 104 sets zero for version number corresponding with the first data, and will be after zero setting
Version number is sent to client 102.
It, can should when there is no the first data or the first shortage of data in the cache list of the first memory node 104
Version number's zero setting of first data.
Optionally, the application realizes that above-mentioned read request Coordination module 103 obtains the second data using method as shown in Figure 5
Read request, and the method that the second data are obtained according to the second request of data from the data that the second memory node 105 stores.Specifically
Ground, the method for the present embodiment include step S501-S505:
Step S501: read request Coordination module 106 obtains the second data read request, and obtains the second memory node set,
And the second memory node 105 into the second memory node set sends the second data read request.
It is not repeated here about the second memory node set.
The S502: the second memory node of step 105 receives the second data read request, and is obtained and the according to the second request of data
The corresponding version number of two data, and version number is sent to read request Coordination module 106.
Step S503: read request Coordination module 106 receives the version number that all second memory nodes 105 are sent, and obtains
The maximum value of multiple version numbers.
When the more new data of client 102, for data into the processing of the first memory node 104, the first memory node 104 is first should
Data distribute version number, and the data are then inserted into prepare list, and the data on prepare list have according to version number
Then the data for carrying version number are sent to the second memory node 105, the second memory node 105 is equally by the number by sequence arrangement
According to being inserted into prepare list.Data are every to be updated once, and version number increases primary.Therefore, corresponding with maximum version number
Two data are newest second data.
Step S504: read request Coordination module 106 judges whether the second data corresponding with maximum value lack, if so,
S505 is entered step, if it is not, then return step S501.
It should be noted that not needing to receive the second data read request again when return step S501.
It can be caused because system cut-off or data cause data update imperfect the problems such as transmission in data updating process
Shortage of data, therefore should first judge whether data lack before obtaining data.
Step S505: the second data corresponding with maximum value are sent to client 102 by read request Coordination module 106.
First data are sent to client 102 by the S304: the first memory node of step 104, and read request Coordination module 106 will
Second data are sent to client 102.
In other embodiments, read request Coordination module can also be arranged on each memory node of storage terminal,
The read request Coordination module that only selection is correspondingly arranged with the second memory node when handling read data request comes into force.
It is different from the prior art, the client 102 of the present embodiment distributed memory system 101 reads multiple data of generation
Request is divided into the first data read request and the second data read request, and the first data read request is sent to the first memory node
104, to obtain the first data from the first memory node 104, while the second data read request is sent to read request Coordination module
106, to obtain the second data, therefore, the second of the present embodiment from the second memory node 105 by read request Coordination module 106
Memory node 105 can be jointly processed by multiple data read requests of client 102 with the first memory node 104, make client 102
The multiple data read requests generated can be balancedly distributed in the entire memory node cluster of distributed memory system 101, from
And it is able to ascend the reading extended capability of distributed memory system 101.
Optionally, above-mentioned second memory node 105 of method realization as shown in FIG. 6 can be used according to the second data in the application
The method of request version number corresponding with the second data.Specifically, the present embodiment method the following steps are included:
The S601: the second memory node of step 105 obtains the first identification information of the second data from the second data read request.
The S602: the second memory node of step 105 judge the first identification information whether the cache list with the second memory node
In the second identification information whether match, if so, entering step S603.
The S603: the second memory node of step 105 obtains version number corresponding with the first identification information from cache list, and will
Version number corresponding with the first identification information is sent to read request Coordination module 106.
Further, the second identification information in the first identification information and the cache list of the second memory node mismatches,
Then enter step S604-S606.
The S604: the second memory node of step 105 judge the first identification information whether the with the storage of the second memory node 105
Three identification informations are matched, if so, S605 is entered step, if it is not, then entering step S606.
Wherein, each memory node of the present embodiment is equivalent to a server comprising cache list and disk storage
Space, when, without required data, can further inquire from disk storage space should in the cache list of memory node
Data.
The S605: the second memory node of step 105 obtains version number corresponding with the first identification information, and will be with the first identification
The corresponding version number of information is sent to read request Coordination module 106.
The S606: the second memory node of step 105 sets zero for the version number of the second data, and by the version number after zero setting
It is sent to read request Coordination module 106.
The application it is further proposed that a kind of distributed memory system of another embodiment data read-write method, such as Fig. 7 institute
To show, the method for the present embodiment is written for realizing the data of above-described embodiment distributed memory system, specifically, the present embodiment
Method comprising the following specific steps
Step S701: client 102 generates data write request.
Step S702: client 102 sends data write request to the first memory node 104.
The data write request of the present embodiment distributed memory system 101 is handled in the first memory node 104.
The S703: the first memory node of step 104 receives data write request, and obtains third data according to data read request,
And third data are sent to the second memory node 105.
The S704: the second memory node of step 105 receives and saves third data.
As the above analysis, second memory node 105 of the present embodiment can be jointly processed by with the first memory node 104
Multiple data read requests of client 102, the multiple data read requests for generating client 102 are balancedly distributed in distribution
In the entire memory node cluster of formula storage system 101, so as to promote the reading extended capability of distributed memory system 101.
Meanwhile because of 105 sharing data read request of the second memory node, so that there are more resources to handle for the first memory node 104
Write request has been able to ascend the handling capacity of the entire company-data write-in of distributed memory system 101.
The application it is further proposed that the distributed memory system of another embodiment data read-write method, the number of the present embodiment
Include: the first memory node 104 from the first data read request of acquisition of client 102 according to reading/writing method, and is asked according to the first data
It asks and obtains the first data from the data of storage, and the first data are sent to the client 102；Read request Coordination module
106 obtain the second data read request, and the number stored according to the second request of data from the second memory node 105 from client 102
According to the second data of middle acquisition, and the second data are sent to client 102.
It is different from the prior art, the present embodiment is jointly processed by visitor by the second memory node 105 and the first memory node 104
Multiple data read requests at family end 102, the multiple data read requests for generating client 102 are balancedly distributed in distribution
In the entire memory node cluster of storage system 101, it is able to ascend the reading extended capability of distributed memory system 101
The application it is further proposed that another embodiment distributed memory system, as shown in figure 8, the present embodiment distribution is deposited
Storage system 801 includes client 802 and storage terminal 803, wherein storage terminal 803 includes the first memory node 804, at least
Two the second memory nodes 805 and at least two read request Coordination modules 806, read request Coordination module 806 and the second storage save
Point 805 is correspondingly arranged, and for storing data, at least two second memory nodes 805 are for storing data for the first memory node 804
Copy；Client 802 includes the first data read request and second for generating multiple data read requests, multiple data read requests
Data read request；Client 802 is used to send the first data read request to the first memory node 804 and to read request Coordination module
806 send the second request of data；First memory node 804 for obtain the first request of data, and according to the first request of data from
The first data are obtained in the data of storage, and the first data are sent to client 802；Read request Coordination module 806 is for obtaining
The second data read request is taken, and the second data are obtained from the data that the second memory node 805 stores according to the second request of data,
And the second data are sent to client 802.
It is different from the prior art, the client 802 of the present embodiment distributed memory system 801 reads multiple data of generation
Request is divided into the first data read request and the second data read request, and the first data read request is sent to the first memory node
804, to obtain the first data from the first memory node 804, while the second data read request is sent to read request Coordination module
806, to obtain the second data, therefore, the second of the present embodiment from the second memory node 805 by read request Coordination module 806
Memory node 805 can be jointly processed by multiple data read requests of client 802 with the first memory node 804, make client 802
The multiple data read requests generated can be balancedly distributed in the entire memory node cluster of distributed memory system 801, from
And it is able to ascend the reading extended capability of distributed memory system 801.
Certainly, the present embodiment distributed memory system 801 can be using the data read-write method in above method embodiment
The read-write for carrying out data, does not repeat here.
The application is it is further proposed that a kind of storage terminal, as shown in figure 9, Fig. 9 is that the application stores one embodiment of terminal
Structural schematic diagram.The present embodiment stores terminal 901 and includes the first memory node 902, at least two second memory nodes 903 and read
Request Coordination module 904, wherein for storing data, the first memory node 902 obtains the first memory node 902 from client
First data read request, and the first data are obtained from the data of storage according to the first data read request, and the first data are sent out
It send to client；The copy of second memory node 903 for storing data；Read request Coordination module 904 from client for obtaining
The second data read request is taken, and the second data are obtained from the data that the second memory node 903 stores according to the second request of data,
And the second data are sent to client.
It is different from the prior art, the second memory node 903 that the present embodiment stores terminal 901 can be with the first memory node
902 are jointly processed by the data read request of client, and the multiple data read requests for generating client, which are balancedly distributed in, deposits
In the entire memory node cluster for storing up terminal 901, so as to store the reading extended capability of terminal 901.
The application is it is further proposed that a kind of storage terminal, as shown in Figure 10, Figure 10 are that the application stores another implementation of terminal
The structural schematic diagram of example.It includes memory module 1002, transceiver module 1003 and processing module that the present embodiment, which stores terminal 1001,
1004, wherein memory module 1002 is for storing data；Transceiver module 1003 is used to receive data read request from client；Place
Reason module 1004 is used to obtain requested data from the data that memory module 1002 stores according to data read request；Receive and dispatch mould
Requested data are further sent to client by block 1003.
It is different from the prior art, multiple data read requests that the present embodiment generates client, which are balancedly distributed in, deposits
In the entire memory node cluster for storing up terminal 1001, so as to promote the reading extended capability of storage terminal 1001.
The application is it is further proposed that a kind of storage medium, as shown in figure 11, Figure 11 are one embodiments of the application storage medium
Structural schematic diagram.The present embodiment storage medium 1101 is used to store the related data 1102 and program data of above-described embodiment
1103, wherein related data 1102 includes at least above-mentioned read request and write request etc., and program data 1103 can be performed above
The method for stating embodiment of the method.Related data 1102 and program data 1103 have carried out in above method embodiment detailed
Narration, does not repeat here.
There is the present embodiment the device 1101 of store function can be, but not limited to, USB flash disk, SD card, PD CD-ROM drive, movement firmly
Disk, large capacity floppy drive, flash memory, multimedia memory card, server etc..
It is different from the prior art, the embodiment of the present application distributed memory system includes client and storage terminal, wherein is deposited
Storing up terminal includes the first memory node, at least two second memory nodes and at least two read request Coordination modules, read request association
Mode transfer block is correspondingly arranged with the second memory node, wherein for storing data, at least two second storages save the first memory node
The copy of point for storing data, the distributed memory system data read-write method include: that the multiple data readings of client generation are asked
It asks, multiple data read requests include the first data read request and the second data read request；Client is sent to the first memory node
First data read request and to read request Coordination module send the second request of data；First memory node obtains the first data and asks
It asks, and the first data is obtained from the data of storage according to the first request of data；Read request Coordination module obtains the second data and reads
Request, and the second data are obtained from the data that the second memory node stores according to the second request of data；First memory node will
First data are sent to client；Second data are sent to client by read request Coordination module.In this way, the application
Multiple data read requests that client generates are divided into the first data read request and the second data read request, and the first data are read
Request is sent to the first memory node, to obtain the first data from the first memory node, while the second data read request being sent
To read request Coordination module, to obtain the second data, therefore, the application's from the second memory node by read request Coordination module
Second memory node can be jointly processed by multiple data read requests of client with the first memory node, generate client more
A data read request can be balancedly distributed in the entire memory node cluster of distributed memory system, so as to be promoted point
The reading extended capability of cloth storage system.
If in addition, above-mentioned function realized in the form of software function and as stand-alone product when selling or using, can deposit
Storage is in a mobile terminal read/write memory medium, that is, and the application also provides a kind of storage device for being stored with program data,
The method that described program data can be performed to realize above-described embodiment, the storage device can be for such as USB flash disk, CD, services
Device etc..That is, the application can be embodied in the form of software products comprising some instructions are used so that an intelligence
Energy terminal executes all or part of the steps of each embodiment the method.
In the description of the present application, reference term " one embodiment ", " some embodiments ", " example ", " specific example ",
Or the description of " some examples " etc. means particular features, structures, materials, or characteristics packet described in conjunction with this embodiment or example
In at least one embodiment or example contained in the application.In the present specification, schematic expression of the above terms are necessarily
It is directed to identical embodiment or example.Moreover, particular features, structures, materials, or characteristics described can be at any one
Or it can be combined in any suitable manner in multiple embodiment or examples.In addition, without conflicting with each other, those skilled in the art
Member the feature of different embodiments or examples described in this specification and different embodiments or examples can be combined and
Combination.
In addition, term " first ", " second " are used for descriptive purposes only and cannot be understood as indicating or suggesting relative importance
Or implicitly indicate the quantity of indicated technical characteristic.Define " first " as a result, the feature of " second " can be expressed or
Implicitly include at least one this feature.In the description of the present application, the meaning of " plurality " is at least two, such as two, three
It is a etc., unless otherwise specifically defined.
Any process described otherwise above or method description are construed as in flow chart or herein, and expression includes
It is one or more for realizing specific logical function or process the step of executable instruction code module, segment or portion
Point, and the range of the preferred embodiment of the application includes other realization, wherein can not press shown or discussed suitable
Sequence, including according to related function by it is basic simultaneously in the way of or in the opposite order, Lai Zhihang function, this should be by the application
Embodiment person of ordinary skill in the field understood.
Expression or logic and/or step described otherwise above herein in flow charts, for example, being considered use
In the order list for the executable instruction for realizing logic function, may be embodied in any computer-readable medium, for
Instruction execution system, device or equipment (can be personal computer, server, the network equipment or other can be from instruction execution
System, device or equipment instruction fetch and the system executed instruction) it uses, or combine these instruction execution systems, device or equipment
And it uses.For the purpose of this specification, " computer-readable medium " can be it is any may include, store, communicate, propagate, or transport
Program is for instruction execution system, device or equipment or the device used in conjunction with these instruction execution systems, device or equipment.
The more specific example (non-exhaustive list) of computer-readable medium include the following: there is the electrical connection of one or more wirings
Portion's (electronic device), portable computer diskette box (magnetic device), random access memory (RAM), read-only memory (ROM) can
It wipes editable read-only memory (EPROM or flash memory), fiber device and portable optic disk read-only storage
(CDROM).In addition, computer-readable medium can even is that the paper that can print described program on it or other suitable Jie
Matter, because can then be edited, be interpreted or when necessary with other for example by carrying out optical scanner to paper or other media
Suitable method is handled electronically to obtain described program, is then stored in computer storage.
The foregoing is merely presently filed embodiments, are not intended to limit the scope of the patents of the application, all to utilize this
Equivalent structure or equivalent flow shift made by application specification and accompanying drawing content, it is relevant to be applied directly or indirectly in other
Technical field similarly includes in the scope of patent protection of the application.
Claims (16)
1. a kind of data read-write method of distributed memory system, which is characterized in that the distributed memory system includes client
End and storage terminal, wherein the storage terminal includes the first memory node, at least two second memory nodes and at least two
Read request Coordination module, the read request Coordination module are correspondingly arranged with second memory node, wherein first storage
For storing data, at least two second memory nodes are used to store the copy of the data, the reading and writing data to node
Method includes:
The client generates multiple data read requests, and multiple data read requests include the first data read request and the second number
According to read request；
The client to first memory node sends first data read request and to the read request Coordination module
Send second request of data；
First memory node obtains first request of data, and according to first request of data from the data of storage
Obtain the first data；The read request Coordination module obtains second data read request, and according to second request of data
The second data are obtained from the data that second memory node stores；
First data are sent to the client by first memory node；The read request Coordination module is by described
Two data are sent to the client.
2. data read-write method according to claim 1, which is characterized in that the read request Coordination module obtains described the
Two data read requests, and the second data are obtained from the data that second memory node stores according to second request of data
Include:
The read request Coordination module obtains second data read request, and obtains the second memory node set, and to described
All second memory nodes in second memory node set send second data read request；
Second memory node receives second data read request, and is obtained and described the according to second request of data
The corresponding version number of two data, and the version number is sent to the read request Coordination module；
The read request Coordination module receives the version number that all second memory nodes are sent, and obtains multiple versions
Number maximum value；
The read request Coordination module judges whether the second data corresponding with the maximum value lack；
If it is not, then the second data corresponding with the maximum value are sent to the client by the read request Coordination module.
3. data read-write method according to claim 2, which is characterized in that if the second data corresponding with the maximum value
Missing then returns to all second memory nodes of the read request Coordination module into the second memory node set and sends institute
The step of stating the second data read request.
4. data read-write method according to claim 2, which is characterized in that second memory node is according to described second
Request of data obtains version number corresponding with second data
Second memory node obtains the first identification information of second data from second data read request；
Judge whether first identification information match with the second identification information in the cache list of the second memory node；
If so, second memory node obtains version number corresponding with first identification information from the cache list,
And version number corresponding with first identification information is sent to the read request Coordination module.
5. data read-write method according to claim 4, which is characterized in that if first identification information and described second
Whether the second identification information in the cache list of memory node mismatches, then judge first identification information with described second
The third identification information matching that memory node is stored；
If so, second memory node obtains version number corresponding with first identification information, and will be with described first
The corresponding version number of identification information is sent to the read request Coordination module.
6. data read-write method according to claim 5, which is characterized in that if first identification information and described second
Third identification information that memory node is stored mismatches, then second memory node is by the corresponding version of second data
Number it is set as zero, and the version number after zero setting is sent to the read request Coordination module.
7. data read-write method according to claim 2, which is characterized in that the quantity of first data read request and more
The ratio between total quantity of a data read request is preset percentage.
8. data read-write method according to claim 7, which is characterized in that the preset percentage meets formula:
Wherein, x is the preset percentage, and the p is arranged for second memory node in institute
State the probability in the second memory node set, quantity and second memory node of the n for first memory node
The sum of quantity, and the n is more than or equal to 3.
9. data read-write method according to claim 8, which is characterized in that the p meets formula:
10. data read-write method according to claim 1, which is characterized in that first memory node obtains described the
One data read request, and the first data are obtained from the data of storage according to first request of data and include:
First memory node obtains first data read request, and obtains from first data read request described the
First identification information of one data；
First memory node judge first identification information whether in the cache list of first memory node
The matching of second identification information；
If so, first memory node obtains the first data corresponding with first identification information, and will be with described the
Corresponding first data of one identification information are sent to the client.
11. data read-write method according to claim 1, which is characterized in that the data read-write method further comprises:
The client generates data write request；
The client sends the data write request to first memory node；
First memory node receives the data write request, and obtains third data according to the data write request, and will
The third data are sent to second memory node；
Second memory node receives and saves the third data.
12. a kind of distributed memory system, which is characterized in that the distributed memory system include client and storage terminal,
Wherein, the storage terminal includes that the first memory node, at least two second memory nodes and at least two read requests coordinate mould
Block, the read request Coordination module are correspondingly arranged with second memory node, first memory node for storing data,
At least two second memory nodes are used to store the copy of the data；
For the client for generating multiple data read requests, multiple data read requests include the first data read request and the
Two data read requests；The client is used to send first data read request to first memory node and to the reading
Coordination module is requested to send second request of data；
First memory node is for obtaining first request of data, and according to first request of data from the number of storage
According to the first data of middle acquisition, and first data are sent to the client；
The read request Coordination module is for obtaining second data read request, and according to second request of data from described
The second data are obtained in the data of second memory node storage, and second data are sent to the client.
13. a kind of data read-write method of distributed memory system, which is characterized in that the distributed memory system includes client
End and storage terminal, wherein the storage terminal includes the first memory node, at least two second memory nodes and at least two
Read request Coordination module, the read request Coordination module are correspondingly arranged with second memory node, first memory node
For storing data, at least two second memory nodes are used to store the copy of the data, the data read-write method
Include:
First memory node obtains first data read request from the client, and according to first request of data
The first data are obtained from the data of storage, and first data are sent to the client；The read request coordinates mould
Block obtains second data read request from the client, and according to second request of data from second memory node
The second data are obtained in the data of storage, and second data are sent to the client.
14. a kind of storage terminal, which is characterized in that the storage terminal includes:
First memory node, for storing data, first memory node obtain the first data read request, and root from client
The first data are obtained from the data of storage according to first data read request, and first data are sent to the client
End；
At least two second memory nodes, for storing the copy of the data；
Read request Coordination module, for obtaining the second data read request from the client, and according to second request of data
The second data are obtained from the data that second memory node stores, and second data are sent to the client.
15. a kind of storage terminal, which is characterized in that the storage terminal includes:
Memory module, for storing data；
Transceiver module, for receiving data read request from client；
Processing module, for obtaining requested number from the data that the memory module stores according to the data read request
According to；
The requested data are further sent to the client by the transceiver module.
16. a kind of storage medium, is stored thereon with program data, which is characterized in that described program data are performed realization power
Benefit requires the step in the method for any one of 1-11,13.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2018/102027 WO2020037625A1 (en) | 2018-08-23 | 2018-08-23 | Distributed storage system and data read-write method therefor, and storage terminal and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109691065A true CN109691065A (en) | 2019-04-26 |
| CN109691065B CN109691065B (en) | 2021-11-09 |
Family
ID=66190472
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201880002511.3A Active CN109691065B (en) | 2018-08-23 | 2018-08-23 | Distributed storage system and data read-write method thereof, storage terminal and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109691065B (en) |
| WO (1) | WO2020037625A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112347335A (en) * | 2020-10-30 | 2021-02-09 | 银盛支付服务股份有限公司 | Method, system, computer equipment and storage medium for efficient retrieval of Elasticissearch |
| CN113254415A (en) * | 2021-05-19 | 2021-08-13 | 浪潮商用机器有限公司 | Method and device for processing read request of distributed file system |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113204536A (en) * | 2021-05-27 | 2021-08-03 | 济南浪潮数据技术有限公司 | Read-write optimization method and device for distributed storage system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1840767A2 (en) * | 2006-03-28 | 2007-10-03 | Sun Microsystems, Inc. | Systems and methods for a distributed cache |
| CN103248656A (en) * | 2012-02-10 | 2013-08-14 | 联想(北京)有限公司 | Method for achieving data reading and writing as well as DSF (distributed file system) and client side |
| CN105468302A (en) * | 2015-11-23 | 2016-04-06 | 华为技术有限公司 | Data processing method, apparatus and system |
| US20170024411A1 (en) * | 2005-01-12 | 2017-01-26 | Wandisco, Inc. | Distributed file system using consensus nodes |
| CN106372136A (en) * | 2010-12-30 | 2017-02-01 | 脸谱公司 | Distributed cache system and method and storage medium |
| CN106850856A (en) * | 2017-03-28 | 2017-06-13 | 南京卓盛云信息科技有限公司 | A kind of distributed memory system and its synchronization caching method |
| CN108228750A (en) * | 2017-12-21 | 2018-06-29 | 浪潮软件股份有限公司 | A kind of distributed data base and its method that data are managed |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103635887B (en) * | 2013-09-23 | 2015-07-08 | 华为技术有限公司 | Data caching method and storage system |
| CN104580428B (en) * | 2014-12-27 | 2018-09-04 | 北京奇虎科技有限公司 | A kind of data routing method, data administrator and distributed memory system |
-
2018
- 2018-08-23 CN CN201880002511.3A patent/CN109691065B/en active Active
- 2018-08-23 WO PCT/CN2018/102027 patent/WO2020037625A1/en active Application Filing
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170024411A1 (en) * | 2005-01-12 | 2017-01-26 | Wandisco, Inc. | Distributed file system using consensus nodes |
| EP1840767A2 (en) * | 2006-03-28 | 2007-10-03 | Sun Microsystems, Inc. | Systems and methods for a distributed cache |
| CN106372136A (en) * | 2010-12-30 | 2017-02-01 | 脸谱公司 | Distributed cache system and method and storage medium |
| CN103248656A (en) * | 2012-02-10 | 2013-08-14 | 联想(北京)有限公司 | Method for achieving data reading and writing as well as DSF (distributed file system) and client side |
| CN105468302A (en) * | 2015-11-23 | 2016-04-06 | 华为技术有限公司 | Data processing method, apparatus and system |
| CN106850856A (en) * | 2017-03-28 | 2017-06-13 | 南京卓盛云信息科技有限公司 | A kind of distributed memory system and its synchronization caching method |
| CN108228750A (en) * | 2017-12-21 | 2018-06-29 | 浪潮软件股份有限公司 | A kind of distributed data base and its method that data are managed |
Non-Patent Citations (1)
| Title |
|---|
| 王千阁等: "区块链系统的数据存储与查询技术综述", 《计算机科学》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112347335A (en) * | 2020-10-30 | 2021-02-09 | 银盛支付服务股份有限公司 | Method, system, computer equipment and storage medium for efficient retrieval of Elasticissearch |
| CN113254415A (en) * | 2021-05-19 | 2021-08-13 | 浪潮商用机器有限公司 | Method and device for processing read request of distributed file system |
| CN113254415B (en) * | 2021-05-19 | 2022-11-04 | 浪潮商用机器有限公司 | Method and device for processing read request of distributed file system |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020037625A1 (en) | 2020-02-27 |
| CN109691065B (en) | 2021-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10108632B2 (en) | Splitting and moving ranges in a distributed system | |
| US20110196828A1 (en) | Method and System for Dynamically Replicating Data Within A Distributed Storage System | |
| CN109691065A (en) | Distributed memory system and its data read-write method, storage terminal and storage medium | |
| CN102387220A (en) | Offline downloading method and system based on cloud storage | |
| CN106375416B (en) | Consistency dynamic adjusting method and device in distributed data-storage system | |
| JP7378403B2 (en) | Data processing methods, equipment, and computing devices | |
| CN102413156B (en) | Distributed data storage system and method | |
| CN103399894A (en) | Distributed transaction processing method on basis of shared storage pool | |
| CN109933312A (en) | A method of containerization relevant database I/O consumption is effectively reduced | |
| CN110222076A (en) | Distributed caching method, server and storage medium | |
| CN107277128A (en) | The method and device of request processing order-preserving in a kind of distributed storage agreement | |
| JP2023541298A (en) | Transaction processing methods, systems, devices, equipment, and programs | |
| CN113051102B (en) | File backup method, device, system, storage medium and computer equipment | |
| Lv et al. | An optimized RGA supporting selective undo for collaborative text editing systems | |
| CN110402435A (en) | Dull transaction in more primary databases with loose couplings node | |
| CN105339906B (en) | The method for controlling the data write-in to permanent storage appliance | |
| CN109271105A (en) | Date storage method and distributed storage cluster applied to distributed storage cluster | |
| CN104363269B (en) | It is a kind of to pass through FC link transmissions, the method and device of reception NAS data | |
| CN114661818B (en) | Method, system, and medium for real-time synchronization of data between clusters in a graph database | |
| CN103095765B (en) | Peer-to-peer communication network and data content distributing method thereof and communication node thereof | |
| CN109992447A (en) | Data copy method, device and storage medium | |
| CN114490540A (en) | Data storage method, medium, device and computing equipment | |
| CN110288309B (en) | Data interaction method, device, system, computer equipment and storage medium | |
| CN114443677A (en) | Object storage service system | |
| Cheriere et al. | How fast can one resize a distributed file system? |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |