CN105915595A - Cluster storage system data accessing method and cluster storage system - Google Patents
Cluster storage system data accessing method and cluster storage system Download PDFInfo
- Publication number
- CN105915595A CN105915595A CN201610223289.2A CN201610223289A CN105915595A CN 105915595 A CN105915595 A CN 105915595A CN 201610223289 A CN201610223289 A CN 201610223289A CN 105915595 A CN105915595 A CN 105915595A
- Authority
- CN
- China
- Prior art keywords
- program
- data block
- caching
- storage
- copy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Abstract
The invention discloses a cluster storage system data accessing method and a cluster storage system. The cluster storage system comprises memory spaces and buffer memory spaces. The cluster storage system runs memory programs, buffer memory programs and client end programs that are all interconnected, the memory programs correspond to the memory spaces in a one-to-one manner, and the buffer memory programs correspond to the buffer memory spaces in a one-to-one manner; when data to be stored is received, the data to be stored is divided into a plurality of data blocks, N+1 copies of the data blocks are generated, each copy of the data blocks is stored in one buffer memory space in a buffered manner, each of the data blocks is also stored in one buffer memory space, the data blocks are input into the memory spaces when a data block utilization rate is low, some buffer memory spaces are spared, and occupied buffer memory spaces can be reduced. Based on the above mode, via use of the cluster storage system data accessing method and the cluster storage system, read-write performance of the cluster storage system can be optimized, cost of the cluster storage system can be lowered, and reliability of the cluster storage system can be improved.
Description
Technical field
The present invention relates to cloud storage technical field, particularly relate to a kind of cluster storage system access number
According to method and cluster storage system.
Background technology
Cluster storage system be the memory space in multiple stage storage device is aggregated into one can be to
Application server provides unified access interface and the storage pool of administration interface, and application can be by accessing
Interface accesses pellucidly and utilizes the system of disk in all storage devices.Cluster storage system can
To give full play to performance and the disk utilization of storage device, and easily extend depositing of its storage pool
Storage capacity.
The memory space of prior art cluster storage system is used mostly mechanical hard disk.Mechanical hard disk has
There are advantages such as low, the life-span length of price, but mechanical hard disk equally exists the shortcomings such as read or write speed is slow.
The memory space of cluster storage system all uses mechanical hard disk, and its read or write speed is very slow, especially same
When one time existed the access request of a large amount of concurrency, the readwrite performance of mechanical hard disk self can be directly
Have influence on the overall readwrite performance of cluster storage system.
Summary of the invention
The technical problem that present invention mainly solves is to provide a kind of cluster storage system storage data
Method and cluster storage system, both optimized the readwrite performance of cluster storage system, reduced collection
The cost of group's storage system.
For solving above-mentioned technical problem, the technical scheme that the present invention uses is: provide one
Cluster storage system, including memory space and spatial cache, described cluster storage system has also run
Storage program, caching program and client-side program, described storage program, caching program and client
Interconnect between program, wherein, described memory space, spatial cache, storage program and caching program
Quantity be multiple, and described in one, store the corresponding memory space of program, described in one, cache journey
Ordered pair answers a spatial cache, and the readwrite performance of described spatial cache is better than the read-write of memory space
Energy；Described client-side program, for when receiving data to be stored, is calculated according to preset data segmentation
Data to be stored are divided into several data blocks by method, and set up the mark of described data block and treat
Corresponding relation between the mark of storage data, and according to presetting the first storage algorithm, select respectively
Select the storage program that each described data block is corresponding, and send correspondence to selected storage program
Data block.Described selected storage program is for caching journey according to preset buffer memory algorithms selection original
Sequence, and selected caching program is cached program as original, send out to described original caching program
Send the data block received, and self mark described is write the metadata of described data block
In；Described original caching program is for arriving the metadata cache of described data block and described data block
The spatial cache of self correspondence, associates the mark of the mark of described data block with original caching program
Relation write cache set, and store algorithm according to default copy, select N-1 described caching journey
Sequence, and selected described caching program is cached program as copy, and generate described data
N-1 part copy of block, sends the pair of a described data block to copy caching program described in respectively
This, wherein, N is the natural number more than 1；
The copy of described data block is delayed by the copy caching program of the copy receiving described data block
Deposit the spatial cache to self correspondence, and associating the mark of the mark of self and described data block
Relation write cache set, and send copy caching successful respond to described original caching program；
Described original caching program is additionally operable to described after receiving copy caching successful respond being selected
Storage program send data buffer storage successful respond；Described selected storage server is additionally operable to
When receiving data buffer storage successful respond, send data storage successful respond to client-side program.
Wherein, described caching program is additionally operable to according to presetting filtering algorithm, from the spatial cache of self
Middle screening data block to be stored, and process described according to the metadata acquisition of described data block to be stored
The storage program of data block to be stored, and wait to deposit to described in the transmission of the described storage program got
Storage data block；Described data block to be stored is stored to self corresponding by the described storage program got
Memory space, and return data brush and enter successful respond.
Wherein, described caching program for receiving after described data brush enters successful respond, according to
Described cache set acquisition process described data block to be stored other caching program, and from self
Spatial cache deletes described data block to be stored；Sending deletion copy to other caching program described please
Asking, wherein, the mark of described data block to be stored is carried in the request of described deletion copy；Described other
Caching program, according to deleting copy request, deletes described data block to be stored from own cache space,
And to returning copy deletion successful respond after deletion completes；Described caching program receive described
Copy is deleted after successful respond, from described cache set delete the mark of described data block to be stored with
Incidence relation between the mark of caching program.
Wherein, described default filtering algorithm includes recent minimum use algorithm；Described caching program root
According to default filtering algorithm, from the spatial cache of self correspondence, screen data block to be stored include: institute
State caching program detect the idle remaining space of spatial cache of self correspondence less than threshold value time,
According to recent minimum use algorithm, from the spatial cache of self, screen data block to be stored.
Wherein, described memory space is mechanical hard disk, and described spatial cache is solid state hard disc.
Wherein, described client-side program is additionally operable to when receiving data read request, obtains described
The data block to be read corresponding to data to be read that data read request is asked, and process institute
State the storage program of data block to be read, send described data block to the described storage program got
Read requests；The described storage program got is after receiving described data block read requests, logical
Cross and search whether described data block to be read exists caching from described cache set, if there is caching,
Then send data block read requests to the caching program of described correspondence；Receive described data block to read
The caching program of request reads described data block to be read from the spatial cache of self, and returns institute
State data block to be read；The described storage program got returns described to be read to client-side program
Data block.
Wherein, the storage program got described in can not find from described cache set described to be read
During the caching of data block, from the storage space of self, extract described data block to be read, and to described
Client-side program returns described data block to be read.
For solving above-mentioned technical problem, another technical solution used in the present invention is: provide one
The method of cluster storage system access data, described cluster storage system includes memory space and caching
Space, described cluster storage system has also run storage program, caching program and client-side program,
Described storage program, interconnection between caching program and client-side program, wherein, described memory space,
The quantity of spatial cache, storage program and caching program is multiple, and stores program described in one
A corresponding memory space, caches the corresponding spatial cache of program, the reading of described spatial cache described in one
Write performance is better than the readwrite performance of memory space, and method includes: receives and sends from client-side program
Data block, wherein, described data block is to be received, by client-side program, the number to be stored uploaded
According to time, according to preset data partitioning algorithm, the data block that described data to be stored are divided into, institute
State to set up between mark and the mark of data to be stored of data block and have corresponding relation；Slow according to presetting
Deposit algorithms selection caching program, and selected caching program is cached program as original, by institute
The mark stating self writes the metadata that described data block is corresponding, and caches program to described original
Send described data block, so that described original caching program is used for described data-block cache to self
Corresponding spatial cache, and by slow for incidence relation write between self mark and the mark of data block
Deposit set；
Store algorithm according to default copy, select N-1 described caching program, and will be selected
Caching program cache program as copy, and generate N-1 part copy of described data block, to
Described in one, copy caching program sends the copy of a described data block, so that receiving described data
The copy of described data block is cached the caching to self correspondence by the copy caching program of the copy of block
Space, and by the incidence relation write cache set of the mark of self with the mark of described data block,
Wherein, N is the natural number more than 1.
Wherein, described memory space is mechanical hard disk, and described spatial cache is solid state hard disc.
The invention has the beneficial effects as follows: be different from the situation of prior art, the present invention treats receiving
During storage data, data to be stored are divided into multiple data block, first data block is generated N-1 part
Copy, and respectively each copy of data block is buffered in a spatial cache, data block itself is also
It is buffered in a spatial cache, owing to the readwrite performance of spatial cache is better than memory space, can be significantly
Improving the writing speed of data block, data block is cached with altogether N part together with its copy, effectively carries
The reliability of high cluster storage system；It addition, at storage processing technology field, the number being most recently stored
It is accessed for probability recently according to block the highest, by spatial cache read block, is also greatly improved
The reading speed of data block, optimizes the reading performance of cluster storage system；Again, counting
According to the utilization rate of block relatively low time, data block brush is entered memory space, discharges spatial cache, reduce slow
Deposit the usage amount in space, reduce the cost of cluster storage system.
Accompanying drawing explanation
Fig. 1 is the schematic diagram of cluster storage system embodiment of the present invention；
Fig. 2 is the flow chart of the method embodiment of a kind of cluster storage system of present invention storage data.
Detailed description of the invention
Referring to Fig. 1, cluster storage system 20 includes memory space 21 and spatial cache 22, collection
Group's storage system has also run storage program 211, caching program 221 and client-side program 231,
Storage program 211, interconnection between caching program 221 and client-side program 231, wherein, storage
The quantity of space 21, spatial cache 22, storage program 211 and caching program 221 is multiple,
And a storage corresponding memory space 21 of program 211, the corresponding caching sky of a caching program 221
Between 22, the readwrite performance of spatial cache 22 is better than memory space 21.What deserves to be explained is: cluster
Storage system 20 is combined by multiple memory nodes 2, and each memory node 2 can be hung
Carry one or more than one memory space 21 and/or memory space 21, and cache program 221
Operating on memory node 2 with storage program 211, client-side program 231 may be mounted at client
End 3, or be arranged on some memory node 2.
Data to be stored, for according to preset data partitioning algorithm, are divided into by client-side program 231
Several data blocks, and the mark pass corresponding with between the mark of data to be stored setting up data block
System, and according to presetting the first storage algorithm, be chosen as each data block and select a storage program
211, send data block to selected storage program 211.In the mark of data block with to be stored
After setting up corresponding relation between the mark of data, according to this corresponding relation, data block combinations can be gone out
Complete data to be stored.Preset data partitioning algorithm and the default first storage algorithm present invention do not make
Concrete restriction, can be set according to practical situation, such as: preset data partitioning algorithm is
Divided in equal amounts algorithm, presetting the first storage algorithm is load-balancing algorithm etc..
Receive the storage program 211 of data block for caching journey according to preset buffer memory algorithms selection
Sequence 221, and selected caching program 221 is cached program (sign), Xiang Zheng as original
This caching program sends data block, and the metadata by the mark writing data blocks of self.Depositing
After in the metadata of the mark writing data blocks of storage program 211, can be directly according to the unit of data block
Data find the storage program 211 storing this data block.The mark of storage program 211 can be pre-
Coding, it is also possible to for storing the address of program 211.Original caching program for by data block and
The mark of data block, to the spatial cache 22 of self correspondence, is write caching by data block metadata cache
Set, and store algorithm according to default copy, select N-1 caching program, and by selected
Caching program caches program, and N-1 part copy of generation data block as copy, to a copy
Caching program sends the copy of a data block.
The copy of data block is cached to self by the copy caching program of the copy receiving data block
Corresponding spatial cache, and by the incidence relation write caching of the mark of self with the mark of data block
Set, wherein, N is the natural number more than 1.Mark and data block in copy caching program
After the incidence relation write cache set of mark, caching data block can be found by cache set
Copy caching program and original caching program
Copy caching program, after caching completes, caches to original caching program transmission copy and successfully should
Answer, after original caching program receives the copy caching successful respond of copy caching program, to being selected
Storage program 211 send data buffer storage successful respond.The storage server selected is additionally operable to connect
Receive data buffer storage successful respond, and send data storage successful respond to client-side program 231.Visitor
Family end program 231, after receiving data storage successful respond, has known current data to be stored
Completed by storage.
The process performance of spatial cache 22 is better than the process performance of memory space 21, in storage data
Time, first store data in spatial cache 22 and be greatly improved writing of data write cluster storage system
Enter speed, improve the write performance of cluster storage system.It addition, in storage processing technology field,
The data being most recently stored, being accessed for recently may be the highest, and the data of the newest storage are divided into data
Block is stored in spatial cache 22, is also beneficial to optimize the process performance of cluster storage system.Furthermore,
The present invention when caching data block, direct caching data block itself and N-1 copy of data block,
It is greatly improved the reliability of cluster storage system.In the present embodiment, it is preferred that memory space
21 is mechanical hard disk, and spatial cache 22 is solid state hard disc, and the read or write speed of solid state hard disc is better than machine
Tool hard disk, but its price is the most expensive in mechanical hard disk.
Owing to the size of each spatial cache 22 is limited, in order to avoid spatial cache 22 not
Foot, the data block brush in spatial cache 22 can also be entered memory space 21 by caching program 221,
Concrete, caching program 221 is additionally operable to according to presetting filtering algorithm, from the spatial cache 22 of self
Middle screening data block to be stored, and obtain according to the metadata of data block to be stored and process to be stored
The storage program 211 of data block, and send data to be stored to the storage program 211 got
Block.Owing in the metadata of data block to be stored, record has the mark with data to be stored to be associated
The mark of storage program 211, therefore, it can the metadata acquisition according to data block to be stored and processes
The storage program 211 of data block to be stored.The storage program 211 got is by data block to be stored
Store the memory space 21 to self correspondence, and return data brush and enter successful respond.
In data block to be stored after spatial cache 22 brushes memory space 21, by this number to be stored
Delete from all of spatial cache 22 according to block, then caching program 221 is for receiving data brush
After entering successful respond, process other caching program of data block to be stored according to cache set acquisition
221, and from the spatial cache 22 of self, delete data block to be stored.Program 221 is cached to other
Send copy request of deleting, wherein, delete copy request and carry the mark of data block to be stored.Institute
State other caching program 221 and ask according to deleting copy, delete and delete from himself spatial cache 22
Except data block to be stored, and to returning copy deletion successful respond after deletion completes.Caching program
221, after receiving copy deletion successful respond, delete the mark of data block to be stored from cache set
Know the incidence relation between the mark of caching program 221.
Further, default filtering algorithm includes recent minimum use algorithm.By recent minimum use
Data block brush enter memory space 21, be conducive to optimize cluster storage system spatial cache 22 He
Memory space 21 storage configuration, optimizes the process performance of cluster storage system.Caching program 221
According to default filtering algorithm, from the spatial cache 22 of self correspondence, screen data block bag to be stored
Include: caching program 221 is little at the idle remaining space of the spatial cache 22 self correspondence being detected
When threshold value, according to recent minimum use algorithm, screen from the spatial cache 22 of self and wait to deposit
Storage data block.
It addition, can also keep being connected, when a certain between original caching program and copy caching program
During individual copy caching program malfunctions, immediately data block brush is entered memory space 21, concrete,
Original caching program has heartbeat detection each other with copy caching program, when finding that certain copy delays
When depositing Programmable detection less than heart beating, original caching program is according to depositing described in the metadata of data block
The mark of storage program 211 sends data brush to storage program 211 and enters request and data block be sent to
Corresponding storage program 211.Storage program 211 receives will after data brush enters to ask summed data block
The storage sky shen of data block write self, and just entering request to transmission data brush after writing successfully
This caching program sends data brush and enters successful respond, and original caching program enters into receiving data brush
Sending copy removal request to storage other copy of this data block caching program after merit response, other is secondary
This caching program deletes this data block from the spatial cache 22 of self after receiving copy removal request,
Returning copy after deleting successfully and delete successful respond, original caching program receives copy and deletes successfully
In the metadata of data block, mark this data block after response brushed into memory space 21 and deleted
Except the copy in spatial cache 22.If can't detect heart beating is that original caches program, then will
The copy caching program depositing the first authentic copy rises to original caching program and performs original caching journey
The action that sequence should be taked in this state.Meanwhile, the topological structure of notice storage program 211 is
Altered, with ensure storage program 211 can find when read block deposit original caching journey
Sequence.
At client-side program 231 when reading data, preferentially read to improve from spatial cache 22 and read
Taking speed, concrete, client-side program 231 is additionally operable to, when receiving data read request, obtain
The data block to be read corresponding to data to be read that the read requests that fetches data is asked, and process
The storage program 211 of data block to be read, sends to the storage program 211 got and reads according to block
Request.
The storage program 211 got is after receiving data block read requests, by from cache set
Close and search whether data block to be read exists caching, if there is caching, then to corresponding caching program
221 send data block read requests.Receive the caching program 221 of data block read requests from certainly
The spatial cache 22 of body reads data block to be read, and returns data block to be read；Get
Storage program 211 return data block to be read to client-side program 231.
The storage program 211 got is finding data block to be read to there is not caching from cache set
Time, from the storage space of self, extract data block to be read, and return to client-side program 231231
Return data block to be read.The data chunk to be read synthesis that client-side program 231 can will read
The most whole data.
The invention has the beneficial effects as follows: be different from the situation of prior art, the present invention treats receiving
During storage data, data to be stored are divided into multiple data block, first data block is generated N-1 part
Copy, and respectively each copy of data block is buffered in a spatial cache, data block itself is also
It is buffered in a spatial cache, owing to the readwrite performance of spatial cache is better than memory space, can be significantly
Improving the writing speed of data block, data block is cached with altogether N part together with its copy, effectively carries
The reliability of high cluster storage system；It addition, at storage processing technology field, the number being most recently stored
It is accessed for probability recently according to block the highest, by spatial cache read block, is also greatly improved
The reading speed of data block, optimizes the reading performance of cluster storage system；Again, counting
According to the utilization rate of block relatively low time, data block brush is entered memory space, discharges spatial cache, reduce slow
Depositing the usage amount in space, the cost present invention reducing cluster storage system provides again cluster storage system
The method embodiment of access data.Cluster storage system includes memory space and spatial cache, collection
Group's storage system has also run storage program, caching program and client-side program, described storage program,
Interconnect between caching program and client-side program, wherein, memory space, spatial cache, storage journey
The quantity of sequence and caching program is multiple, and a corresponding memory space of storage program, and one delays
The corresponding spatial cache of program of depositing, the readwrite performance of spatial cache is better than memory space.It is worth explanation
: cluster storage system is combined by multiple memory nodes, and each memory node can
Carry one or more than one memory space and/or memory space, and cache program and storage
Program operates on memory node, but when a certain memory node only has memory space or caching sky
Between, then this memory node has only run caching program or storage program, refers to Fig. 2, method bag
Include:
Step S301: receive from client-side program send data block, wherein, data block be by
Client-side program is when receiving the data to be stored uploaded, according to preset data partitioning algorithm, and will
The data block that data to be stored are split to form, between mark and the mark of data to be stored of data block
Foundation has corresponding relation；
Client-side program, after data to be stored are divided into data block, goes back root according to presetting the first storage
Algorithm, selects the storage program of storage data block, sends data block to selected storage program.
It addition, the data block split can be combined into data to be stored.
Step S302: cache program according to preset buffer memory algorithms selection, and by selected caching journey
Sequence caches program as original, by metadata corresponding for the mark writing data blocks of self and to just
This caching program sends data block, so that original caching program is for right to self by data-block cache
The spatial cache answered, and by incidence relation write caching between self mark and the mark of data block
Set.
The readwrite performance of spatial cache is better than the readwrite performance of memory space, is cached by spatial cache
Data block, is greatly improved the readwrite performance of cluster storage system.When the utilization rate of data block is relatively low,
Again data block brush being entered memory space, used by memory space and caching interference fit, ratio is merely
Only there is the cluster storage system of memory space storage data block, the read-write of the storage system of the present invention
Performance is more preferable, the simple cluster storage system only with spatial cache of ratio, the storage system of the present invention
Cheaper.In the present embodiment, it is preferred that memory space is mechanical hard disk, caching sky
Between be solid state hard disc.
Step S303: store algorithm according to default copy, selects N-1 caching program, and will
The caching program selected caches program, and N-1 part copy of generation data block as copy,
The copy of a described data block is sent, so that receiving the pair of data block to a copy caching program
The copy of data block is cached the spatial cache to self correspondence by this copy caching program, and will be from
The mark of body writes cache set with the incidence relation of the mark of described data block, and wherein, N is big
In the natural number of 1；
In addition to caching data block itself, also cache the copy of N-1 data block, and each
Data block copy and data block are originally penetrated and are positioned at different spatial cache, i.e. N-1 part copy damages,
There is also at least a data block, substantially increase the reliability of data block.
For other operating process of cluster storage system, such as: many copies store, by data block
Enter memory space, digital independent etc. from spatial cache brush, see cluster storage system embodiment party
In formula, repeat the most one by one.
The invention has the beneficial effects as follows: be different from the situation of prior art, the present invention treats receiving
During storage data, data to be stored are divided into multiple data block, first data block is generated N-1 part
Copy, and respectively each copy of data block is buffered in a spatial cache, data block itself is also
It is buffered in a spatial cache, owing to the readwrite performance of spatial cache is better than memory space, can be significantly
Improving the writing speed of data block, data block is cached with altogether N part together with its copy, effectively carries
The reliability of high cluster storage system；It addition, at storage processing technology field, the number being most recently stored
It is accessed for probability recently according to block the highest, by spatial cache read block, is also greatly improved
The reading speed of data block, optimizes the reading performance of cluster storage system；Again, counting
According to the utilization rate of block relatively low time, data block brush is entered memory space, discharges spatial cache, reduce slow
Deposit the usage amount in space, reduce the cost of cluster storage system.
The foregoing is only embodiments of the present invention, not thereby limit the patent model of the present invention
Enclosing, every equivalent structure utilizing description of the invention and accompanying drawing content to be made or equivalence flow process become
Change, or be directly or indirectly used in other relevant technical fields, be the most in like manner included in the present invention's
In scope of patent protection.
Claims (9)
1. a cluster storage system, it is characterised in that include memory space and spatial cache, institute
State cluster storage system and also run storage program, caching program and client-side program, described storage
Program, interconnection between caching program and client-side program, wherein, described memory space, caching sky
Between, storage program and caching program quantity be multiple, and described in one store program correspondence one
Memory space, caches the corresponding spatial cache of program, the readwrite performance of described spatial cache described in one
It is better than the readwrite performance of memory space；
Described client-side program, for when receiving data to be stored, is calculated according to preset data segmentation
Data to be stored are divided into several data blocks by method, and set up the mark of described data block and treat
Corresponding relation between the mark of storage data, and according to presetting the first storage algorithm, be respectively
Each described data block selection one stores program, and sends correspondence to selected described storage program
Data block；
Described selected storage program is used for caching program according to preset buffer memory algorithms selection, and will
The caching program selected caches program as original, sends to described original caching program and receives
Data block, and by described self mark write described data block metadata in；
Described original caching program is for by described data block and the metadata cache of described data block
To the spatial cache of self correspondence, the mark of described data block and the mark of original caching program are closed
Connection relation write cache set, and store algorithm according to default copy, select N-1 described caching
Program, and selected described caching program is cached program as copy, and generate described number
According to N-1 part copy of block, send a described data block to copy caching program described in respectively
Copy, wherein, N is the natural number more than 1；
The copy of described data block is delayed by the copy caching program of the copy receiving described data block
Deposit the spatial cache to self correspondence, and associating the mark of the mark of self and described data block
Relation write cache set, and send copy caching successful respond to described original caching program；
Described original caching program is additionally operable to after receiving copy caching successful respond to described selected
Storage program sends data buffer storage successful respond；
Described selected storage program is additionally operable to when receiving data buffer storage successful respond, to institute
State client-side program and send data storage successful respond.
Cluster storage system the most according to claim 1, it is characterised in that
Described caching program is additionally operable to, according to presetting filtering algorithm, screen from the spatial cache of self
Data block to be stored, and process described to be stored according to the metadata acquisition of described data block to be stored
The storage program of data block, and send described data to be stored to the described storage program got
Block；
Described data block to be stored is stored depositing to self correspondence by the described storage program got
Storage space, and return data brush and enter successful respond.
Cluster storage system the most according to claim 2, it is characterised in that
Described caching program is for receiving after described data brush enters successful respond, according to described slow
Deposit set acquisition and process other caching program of described data block to be stored, and empty from the caching of self
Between delete described data block to be stored；
Sending, to other caching program described, copy request of deleting, wherein, described deletion copy is asked
Carry the mark of described data block to be stored；
Other caching program described, according to deleting copy request, is treated from described in the deletion of own cache space
Storage data block, and to returning copy deletion successful respond after deletion completes；
Described caching program is after receiving described copy deletion successful respond, from described cache set
Delete the incidence relation between the mark of described data block to be stored and the mark of caching program.
Cluster storage system the most according to claim 2, it is characterised in that
Described default filtering algorithm includes recent minimum use algorithm；
Described caching program, according to default filtering algorithm, is screened from the spatial cache of self correspondence and is treated
Storage data block includes:
Described caching program is less than at the idle remaining space of the spatial cache self correspondence being detected
During threshold value, according to recent minimum use algorithm, from the spatial cache of self, screen data to be stored
Block.
Cluster storage system the most according to claim 2, it is characterised in that
Described memory space is mechanical hard disk, and described spatial cache is solid state hard disc.
Cluster storage system the most according to claim 1, it is characterised in that
Described client-side program is additionally operable to when receiving data read request, obtains described data and reads
The data block to be read corresponding to data to be read that the request of taking is asked, and continue described in process
Fetching data the storage program of block, sending that described data block reads to the described storage program got please
Ask；
The described storage program got is after receiving described data block read requests, by from institute
State cache set and search whether described data block to be read exists caching, if there is caching, then to institute
The caching program stating correspondence sends data block read requests；
The caching program receiving described data block read requests reads from the spatial cache of self
Described data block to be read, and return described data block to be read；
The described storage program got returns described data block to be read to client-side program.
Cluster storage system the most according to claim 6, it is characterised in that
The described storage program got can not find described data to be read from described cache set
During the caching of block, from the storage space of self, extract described data block to be read, and to described client
End program returns described data block to be read.
8. the method for cluster storage system access data, it is characterised in that described cluster-based storage
System include memory space and spatial cache, described cluster storage system also run storage program,
Caching program and client-side program, described storage program, between caching program and client-side program mutually
Even, wherein, the quantity of described memory space, spatial cache, storage program and caching program is
Multiple, and described in one, store the corresponding memory space of program, cache program corresponding described in one and delay
Depositing space, the readwrite performance of described spatial cache is better than the readwrite performance of memory space, and method includes:
Receiving the data block sent from client-side program, wherein, described data block is by client
Program, when receiving the data to be stored uploaded, according to preset data partitioning algorithm, is treated described
The data block that storage data are divided into, between mark and the mark of data to be stored of described data block
Foundation has corresponding relation；
Program is cached according to preset buffer memory algorithms selection, and using selected caching program as just
This caching program, writes the metadata that described data block is corresponding, Yi Jixiang by described identifying of self
Described original caching program sends described data block, so that described original caching program is for by described
Data-block cache to the spatial cache of self correspondence, and by the mark of the mark of self and data block it
Between incidence relation write cache set；
Store algorithm according to default copy, select N-1 described caching program, and will be selected
Caching program cache program as copy, and generate N-1 part copy of described data block, to
Described in one, copy caching program sends the copy of a described data block, so that receiving described data
The copy of described data block is cached the caching to self correspondence by the copy caching program of the copy of block
Space, and by the incidence relation write cache set of the mark of self with the mark of described data block,
Wherein, N is the natural number more than 1.
Method the most according to claim 8, it is characterised in that
Described memory space is mechanical hard disk, and described spatial cache is solid state hard disc.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610223289.2A CN105915595B (en) | 2016-04-11 | 2016-04-11 | Method for cluster storage system to access data and cluster storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610223289.2A CN105915595B (en) | 2016-04-11 | 2016-04-11 | Method for cluster storage system to access data and cluster storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105915595A true CN105915595A (en) | 2016-08-31 |
| CN105915595B CN105915595B (en) | 2020-05-26 |
Family
ID=56745682
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610223289.2A Active CN105915595B (en) | 2016-04-11 | 2016-04-11 | Method for cluster storage system to access data and cluster storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105915595B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106599292A (en) * | 2016-12-26 | 2017-04-26 | 东方网力科技股份有限公司 | Method and system for storing real-time video data and image data |
| CN108932203A (en) * | 2017-05-29 | 2018-12-04 | 爱思开海力士有限公司 | Data processing system and data processing method |
| CN109002260A (en) * | 2018-07-02 | 2018-12-14 | 深圳市茁壮网络股份有限公司 | A kind of data cached processing method and processing system |
| CN110119248A (en) * | 2016-12-05 | 2019-08-13 | 华为技术有限公司 | Control method, storage equipment and the system of reading and writing data order |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8700841B2 (en) * | 2010-04-19 | 2014-04-15 | International Business Machines Corporation | Sub-LUN input/output profiling for SSD devices |
| CN103885728A (en) * | 2014-04-04 | 2014-06-25 | 华中科技大学 | Magnetic disk cache system based on solid-state disk |
| US20140207995A1 (en) * | 2013-01-22 | 2014-07-24 | International Business Machines Corporation | Use of differing granularity heat maps for caching and migration |
| CN104484130A (en) * | 2014-12-04 | 2015-04-01 | 北京同有飞骥科技股份有限公司 | Construction method of horizontal expansion storage system |
-
2016
- 2016-04-11 CN CN201610223289.2A patent/CN105915595B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8700841B2 (en) * | 2010-04-19 | 2014-04-15 | International Business Machines Corporation | Sub-LUN input/output profiling for SSD devices |
| US20140207995A1 (en) * | 2013-01-22 | 2014-07-24 | International Business Machines Corporation | Use of differing granularity heat maps for caching and migration |
| CN103885728A (en) * | 2014-04-04 | 2014-06-25 | 华中科技大学 | Magnetic disk cache system based on solid-state disk |
| CN104484130A (en) * | 2014-12-04 | 2015-04-01 | 北京同有飞骥科技股份有限公司 | Construction method of horizontal expansion storage system |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110119248A (en) * | 2016-12-05 | 2019-08-13 | 华为技术有限公司 | Control method, storage equipment and the system of reading and writing data order |
| CN110119248B (en) * | 2016-12-05 | 2021-10-15 | 华为技术有限公司 | Control method of data read-write command, storage device and system |
| CN106599292A (en) * | 2016-12-26 | 2017-04-26 | 东方网力科技股份有限公司 | Method and system for storing real-time video data and image data |
| CN106599292B (en) * | 2016-12-26 | 2020-05-15 | 东方网力科技股份有限公司 | Method and system for storing real-time video data and image data |
| CN108932203A (en) * | 2017-05-29 | 2018-12-04 | 爱思开海力士有限公司 | Data processing system and data processing method |
| CN108932203B (en) * | 2017-05-29 | 2022-03-11 | 爱思开海力士有限公司 | Data processing system and data processing method |
| CN109002260A (en) * | 2018-07-02 | 2018-12-14 | 深圳市茁壮网络股份有限公司 | A kind of data cached processing method and processing system |
| CN109002260B (en) * | 2018-07-02 | 2021-08-13 | 深圳市茁壮网络股份有限公司 | Processing method and processing system for cache data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105915595B (en) | 2020-05-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9836514B2 (en) | Cache based key-value store mapping and replication | |
| CN107169083B (en) | Mass vehicle data storage and retrieval method and device for public security card port and electronic equipment | |
| CN108647151A (en) | It is a kind of to dodge system metadata rule method, apparatus, equipment and storage medium entirely | |
| CN105872040B (en) | A method of write performance is stored using gateway node cache optimization distributed block | |
| US6785771B2 (en) | Method, system, and program for destaging data in cache | |
| CN105915595A (en) | Cluster storage system data accessing method and cluster storage system | |
| CN101673192B (en) | Method for time-sequence data processing, device and system therefor | |
| US10656839B2 (en) | Apparatus and method for cache provisioning, configuration for optimal application performance | |
| WO2020098654A1 (en) | Data storage method and device based on cloud storage, and storage medium | |
| CN108089825B (en) | Storage system based on distributed cluster | |
| CN103198027A (en) | Method and device for storing and providing files | |
| CN104054071A (en) | Method for accessing storage device and storage device | |
| CN107422989B (en) | Server SAN system multi-copy reading method and storage system | |
| CN104572505A (en) | System and method for ensuring eventual consistency of mass data caches | |
| CN107888687B (en) | Proxy client storage acceleration method and system based on distributed storage system | |
| CN109766318A (en) | File reading and device | |
| CN106991118A (en) | Entangled based on CEPH and delete fault-tolerant reading document method and device | |
| CN105243030A (en) | Data caching method | |
| CN104158863A (en) | Cloud storage mechanism based on transaction-level whole-course high-speed buffer | |
| CN110008197A (en) | A kind of data processing method, system and electronic equipment and storage medium | |
| CN104899161A (en) | Cache method based on continuous data protection of cloud storage environment | |
| CN111399760B (en) | NAS cluster metadata processing method and device, NAS gateway and medium | |
| CN109254958A (en) | Distributed data reading/writing method, equipment and system | |
| JP2009245004A (en) | Bidirectional data arrangement system, access analysis server, data movement server, bidirectional data arrangement method and program | |
| CN111427920B (en) | Data acquisition method, device, system, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |