CN106648464A - Multi-node mixed block cache data read-writing method and system based on cloud storage - Google Patents
Multi-node mixed block cache data read-writing method and system based on cloud storage Download PDFInfo
- Publication number
- CN106648464A CN106648464A CN201611199823.7A CN201611199823A CN106648464A CN 106648464 A CN106648464 A CN 106648464A CN 201611199823 A CN201611199823 A CN 201611199823A CN 106648464 A CN106648464 A CN 106648464A
- Authority
- CN
- China
- Prior art keywords
- cache
- block
- client
- disk
- read
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Abstract
The embodiment of the invention discloses a multi-node mixed block cache data read-writing method and system based on cloud storage. The method comprises the steps that any one cache client-side receives a general block layer read-writing request sent by a virtual disk device, wherein the virtual disk device is obtained through mapping of a local cache disk of the cache client-side, other cache client-sides, located in the same local area network, of the cache client-side and a source storage disk on a metadata server-side, and the general block layer read-writing request contains a read-writing disk start sector position; the cache client-side adopts a pre-defined cache strategy to transfer the general block layer read-writing request to a corresponding physical disk device for processing. The embodiment of the invention provides a multi-node network mixed cache system suitable for multiple virtual machine examples to share and use. Improvement of the block-level cloud storage read-writing performance of the virtual machine examples is achieved, total improvement of cloud storage I/O performance is finally achieved, and stable and safe operation of a whole cloud storage system is ensured.
Description
Technical field
The present embodiments relate to cloud storage technical field, more particularly to the multinode mixed block caching number based on cloud storage
According to reading/writing method and system.
Background technology
Cloud computing is that one kind utilizes Intel Virtualization Technology, and by the Internet the shared software and hardware money of dynamic extending is provided a user with
The service in source.The characteristic such as dynamic, elastic of cloud computing allow users on demand using and by being paid using resource quantity, user
Can be according to the loading condition of current service, dynamic adjustresources usage amount, to reduce the wasting of resources.Cloud storage system is cloud meter
One important component part of calculation system.From the perspective of from the angle of technology, cloud storage system is actually one large-scale distributed
Network store system, it is a kind of special shared storage.Actually used cloud storage system can be largely classified into object and deposit
Storage and block store two classes.Wherein block storage (Cinder) is mainly used in providing the disk storage space of extension to virtual machine instance,
And the data storage characteristic with persistence, its rear end can support various storages by drivers, such as Ceph, Net App,
Ocean Stor, Nexenta, Solid Fire and Zadara etc..
Cloud computing system now often possesses multiple large-scale data centers, there is respective piece of rank cloud storage subsystem
System, and it is distributed in many ground.Because the network bandwidth between Liang Ge data centers is limited, when virtual machine instance and cloud storage subsystem
When being in different pieces of information center, the I/O performances of storage system will be had a strong impact on, affect the stable operation of virtual machine instance, finally
Affect the stable and high effective operation of virtual machine instance.
The content of the invention
The embodiment of the present invention provides the data cached reading/writing method of multinode mixed block and system based on cloud storage, to improve
The efficiency of data cached read-write.
On the one hand, a kind of data cached read-write side of multinode mixed block based on cloud storage is embodiments provided
Method, including：
Arbitrary cache client receives the generic block layer read-write requests that virtual disk equipment sends, wherein virtual disk equipment
By the cache client local cache disk and the cache client be located at same LAN in other cache clients and
The source storage disk mapping at meta data server end is obtained, comprising read-write disk initial sector position in the general layer read-write requests
Put；
The generic block layer read-write requests are forwarded to respective physical by the cache client using predefined cache policy
Disk unit is processed.
On the other hand, the embodiment of the present invention additionally provides the data cached read-write system of multinode mixed block based on cloud storage
System, including meta data server end and multiple cache clients in same LAN；
Cache client is used to receive the generic block layer read-write requests that virtual disk equipment sends, and using predefined slow
Deposit strategy the generic block layer read-write requests are forwarded to into respective physical disk unit to be processed, the general layer read-write requests
In comprising read-write disk initial sector position；
Wherein virtual disk equipment is located at same office by the local cache disk and the cache client of the cache client
The source storage disk mapping of other cache clients and meta data server end in the net of domain is obtained, the general layer read-write requests
In comprising read-write disk initial sector position.
Technical scheme provided in an embodiment of the present invention, the reading and writing data distributed cloud storage produced for the network bandwidth this
One bottleneck problem, proposes in LAN pattern, the method read and write using multinode mixed block buffered data is realized to block rank
The improvement of the readwrite performance of cloud storage, finally realizes the general increase of cloud storage I/O performance, it is ensured that entirely cloud storage system is steady
Fixed, safe operation.
Description of the drawings
Fig. 1 is a kind of data cached read-write side of multinode mixed block based on cloud storage provided in the embodiment of the present invention one
The flow chart of method；
Fig. 2 is the frame of the data cached read-write system of multinode mixed block based on cloud storage provided in the embodiment of the present invention
Frame figure；
Fig. 3 is the structure chart at the meta data server end provided in the embodiment of the present invention；
Fig. 4 is the structure chart of the cache client provided in the embodiment of the present invention；
Fig. 5 is the caching write-in policy schematic diagram provided in the embodiment of the present invention；
Fig. 6 is the test environment topological diagram provided in the embodiment of the present invention.
Specific embodiment
With reference to the accompanying drawings and examples the present invention is described in further detail.It is understood that this place is retouched
The specific embodiment stated is used only for explaining the present invention, rather than limitation of the invention.It also should be noted that, in order to just
Part related to the present invention rather than entire infrastructure are illustrate only in description, accompanying drawing.
Fig. 1 is a kind of data cached reading/writing method of multinode mixed block based on cloud storage provided in the embodiment of the present invention
Flow chart.The method of the present embodiment can be performed by the data cached read-write system of multinode mixed block based on cloud storage.Ginseng
Fig. 1 is examined, specifically should can be included based on the data cached reading/writing method of multinode mixed block of cloud storage as follows：
Step 11, arbitrary cache client receive the generic block layer read-write requests that virtual disk equipment sends.
Wherein, virtual disk equipment is located at same by the local cache disk and the cache client of the cache client
The source storage disk mapping of other cache clients and meta data server end in LAN is obtained, and the general layer read-write please
Comprising read-write disk initial sector position in asking.
With reference to Fig. 2, the caching system can include a meta data server end and multiple in same LAN
Cache client.Source storage disk in meta data server end passes through iSCSI (Internet Small by external network
Computer System Interface, small computer system interface) agreement is connected to all caching clients in LAN
End, uses for all cache clients are shared.The local cache disk of each cache client is connected to again by iSCSI protocol
Other cache clients, are written and read for other cache clients are data cached to distal end.Also, each cache client uses slow
Deposit client-side program by local cache disk, multiple distal ends caching disk (other the cache client nodes i.e. in LAN, can
Being a server or multiple disk arrays) and source storage disk become a virtual disk with caching function
Equipment.This virtual disk equipment possesses identical capacity with source storage disk, and user stores magnetic using virtual disk equipment replacement source
Disk.Cache client program forwards the block I/O of the virtual disk equipment, is carried out by respective cache disk or source storage disk
Reason, realizes caching function.
With reference to Fig. 3, meta data server end is mainly the caching that each cache client provides all cache client nodes
Hit situation query function, by synchronization request the metadata synchronization with each cache client is realized.And cache client is then born
Duty realizes that actual caching function passes through predefined cache policy and communicate with meta data server end to realize caching function.
It is responsible for focusing on the metadata of each cache client and cache hit inquiry being provided servicing in meta data server end.
The realization at meta data server end mainly includes three steps：
Firstth, initial configuration.Meta data server end initialization when record each cache client caching block size,
Caching number of blocks, the caching cached parameters such as write-in policy, and the configuration information such as each cache client node serial number, IP address.
Secondth, metadata synchronization.Store part metadata related to cache hit inquiry in each cache client node
(corresponding source storage disk initial sector information of status information, cache blocks including each cache blocks etc.), meta data server end
By the metadata synchronization request for receiving each cache client, the metadata updates operation of server end is carried out.
3rd, cache hit inquiry.Meta data server to each cache client provides cache hit inquiry service, receives
The inquiry request (initial sector information including cache blocks etc.) of cache client simultaneously returns the Query Result (visitor that cache blocks are located
Family end numbering, status information etc.).The meta data server end of caching is inquired about with the metadata synchronization of client and cache hit
Realized by UDP (User Datagram Protocol, UDP) communication protocol.
With reference to Fig. 4, cache client is responsible for real cache function, and according to default cache policy caching is carried out.It is slow
Depositing the realization of client mainly includes three parts：
First, initial configuration.It is identical with meta data server, cache client record buffer memory block first in initialization
The cached parameters such as size, caching number of blocks, caching write-in policy, and each cache client node serial number and correspondence distal end caching
Disk device information.
Second, metadata synchronization.Each cache client is put into respective local cache metadata information in kernel internal memory,
And cache hit is inquired about into related part metadata synchronization to meta data server end.Cache client is in local cache unit number
After changing, metadata synchronization information is sent to meta data server end immediately.Each cache client is received and located simultaneously
Modification of the reason meta data server end to local cache metadata is asked.
3rd, caching is realized.To realize caching function, each cache client utilizes Linux Device Mapper mechanism,
Local cache disk, multiple distal ends caching disk and source storage disk are mapped as a virtual disk with caching function and are set
It is standby.For the block I/O of this virtual disk equipment, cache client inquires about local and distal end cache metadata, and according to predefined
Cache policy, the generic block layer bio that virtual disk equipment the is sent io of block device (request) be forwarded to respective physical disk
Equipment is processed.
Realization for caching function also includes following five main points：
(1) real cache data are deposited in the caching disk of each cache client node, data cached according to initialization
The cache blocks size of configuration is organized into block, and each caching data block only preserves a in all cache client nodes.
(2) for the mapping between cache blocks and data block, it is connected mode using group, realizes that source stores magnetic by hash methods
The mapping of the data block of disk and the cache blocks of caching disk.
(3) in terms of the replacement policy of cache blocks, using LRU policy.
(4) due to caching multiple cache client nodes that disk is distributed in LAN, therefore the local of each node is delayed
Deposit disk and there is performance difference with distal end caching disk.According to caching mapping ruler, cache client can preferential performance compared with
Good local cache disk.
(5) cache client is realized by kernel module form, therefore user can according to demand dynamically by cache module
Insertion linux kernel is removed from kernel.
Client communicate with service end realization it is as follows：
Meta data server end includes two parts with the data communication of each cache client：Metadata synchronization and cache hit
Inquiry.Metadata synchronization part includes the metadata updates at meta data server end and the metadata updates of cache client.
Wherein, the metadata updates request at meta data server end is initiated by cache client.When the cache metadata of cache client
Trigger during change, by the corresponding source storage disk initial sector of metadata evolution cache blocks, caching block number, caching bulk state letter
Breath and the cache client node serial number information of transmission renewal request send to meta data server end and carry out metadata updates.
And the metadata updates request of cache client is initiated by meta data server end, and the only generation when write operation is carried out.When
When cache hit inquiry causes cache metadata to change, trigger this and update request, metadata evolution is cached into block number and caching
Block state information sends to correspondence cache client and carries out metadata updates.It is true that cache client equally responds renewal request numbering
Recognize.Cache hit query portion then includes cache blocks inquiry request and its response.Cache blocks inquiry request is sent out by cache client
Rise.This request is triggered when caching is local miss, the corresponding source storage disk initial sector of cache blocks and I/O read-write sides is sent
To meta data server end, and inquiry request is waited to respond.
Specifically, arbitrary cache client receives the generic block layer read-write requests (bio requests) that virtual disk equipment sends,
Comprising read-write disk initial sector position in the bio structures.
The generic block layer read-write requests are forwarded to phase by step 12, the cache client using predefined cache policy
Physical disk equipment is answered to be processed.
Wherein, predefined cache policy can mainly include cache blocks mapping policy, caching write-in policy and caching
Block replacement policy.Wherein, cache write-in policy and the degree of association (assoc) of cache blocks mapping policy can be at the beginning of caching system
By User Defined during beginningization, in addition user can also be to the caching block size of caching system, caching number of blocks and caching visitor
The parameters such as family end IP are set.
Specifically, the cache client adopts cache blocks mapping policy, determines according to read-write disk initial sector position logical
Cached write-in policy and be written and read place with the respective physical disk unit of block layer read-write requests, and the employing of respective physical disk unit
Reason.
For write operation, main to provide write through and write back two kinds of write-in policies, write through strategies favor is in data
Safety, and write back strategy and be then more biased towards in write performance, user can according to demand arrange the write-in policy of caching.
With reference to Fig. 5, when using write through strategy, all block I/O data are write direct the source storage disk of distal end, and
It is not cached to caching disk.If the data of write are locally or remotely caching disk buffering, need correspondence caching bulk state
It is invalid to be set to.This write-in policy ensure that the data consistency between caching disk and source storage disk, in caching client
End node goes wrong and in the case of causing data cached loss, still is able to ensure the safety of data.
With reference to Fig. 5, during using writing back strategy, all block I/O data write direct local cache disk or distal end caching
Disk, all data cached being delayed by write back source storage disk.Those postpone written-back operation will be when cache blocks be replaced
Occur, or cache disk using finish unloaded when, operation is rinsed by cache client program, unifying will be all dirty
Data in cache blocks write back the source storage disk of distal end.Certainly, caching flushing operation can be to respective cache client node
Performance produces certain impact.Writing speed can be lifted in theory using strategy is write back, and reduces far-end network storage
The load pressure of system.But using strategy is write back while there is also risk, when cache client one malfunctions, will appear from
The situation of data cached loss.
For the mapping between cache blocks and data block, system is by the way of group is connected.It is connected according to predefined caching
Degree, will often cache degree of association cache blocks and be combined into a caching group, and by hash (Hash) method source storage disk is realized
Data block with caching disk cache blocks mapping.
Exemplary, step 12 can include：
A, the cache client are connected according to read-write disk initial sector position, default cache blocks mask and caching
Degree, determines target cache block.
Further, step A specifically can include：
, according to read-write disk initial sector position and default cache blocks mask, obtaining read-write please for A1, the cache client
Ask initial sector position of the data corresponding data block in the storage disk of source.
Specifically, the cache client be directed to from topmost paper system or VFS (virtual File System, virtually
File system) layer generic block layer bio requests, the read-write disk initial sector position bi_sector in bio structures is first
It is first step-by-step is carried out with default cache blocks mask block_mask to draw read-write disk initial sector position bi_sector with operation
Side-play amount offset in corresponding data block, then deducts bi_sector side-play amount offset and show that data block is deposited in source
Initial sector request_block in storage disk unit.
The initial sector position of A2, the cache client according to the data block in the storage disk of source, calculates the number
According to block correspondence caching group number.
Specifically, the cache client is first using hash_block functions according to initial sector position request_block
Database accession number is first calculated, being then divided by with caching degree of association assoc draws caching group number set_number.
A3, the cache client determine the caching according to data block correspondence caching group number and caching degree of association
The starting caching block number of group.
Specifically, the cache client is by data block correspondence caching group number set_number and caches degree of association assoc phases
It is multiplied go out the caching group starting caching block number index.
A4, the cache client cache block number according to the starting and the caching degree of association determines target cache block.
Specifically, the cache blocks of starting caching block number index to index+assoc are defined as mesh by the cache client
Mark cache blocks.
B, the cache client are entered by the metadata of local cache disk or meta data server end to target cache block
Row cache hit is inquired about, and determines the respective physical disk of the generic block layer read-write requests.
Specifically, cache client carries out hit inquiry by the metadata of local cache disk to target cache block, with
Whether the respective physical disk for determining generic block layer read-write requests is local cache disk；Cache client passes through Metadata Service
Whether device end carries out cache hit inquiry to target cache block, to determine the respective physical disk of generic block layer read-write requests for it
His cache client or source storage disk.
Made a look up using cache blocks metadata struct cacheblock, search caching block number index to index+
The metadata cacheblock item of assoc.In searching all cacheblock, caching bulk state be VALID, DIRTY,
(state illustrates that this cache blocks carries out caching write or caching writes back to retain or writing back for RESERVED or WRITEBACK
Operation, current bio requests will be placed into delay disposal in the bios lists of cacheblock) and cache blocks corresponding data block
Sector original position block is equal with request_block.If finding cacheblock items, cache hit；If not finding,
Then cache miss.
C, the generic block layer read-write requests are forwarded to into respective physical disk unit are processed.
Exemplary, the cache client carries out cache hit inquiry by meta data server end to target cache block,
Determine the respective physical disk of the generic block layer read-write requests, including：The cache client to meta data server end sends
Cache hit inquiry request comprising target cache block；Meta data server end determines and returns target to the cache client delays
The status information of counterfoil place cache client numbering and target cache block；The cache blocks client is according to meta data server end
Cache client numbering and the status information of target cache block that the target cache block of return is located, determine that the generic block layer is read
The respective physical disk of write request.
Exemplary, meta data server end determines and returns the caching visitor that target cache block is located to the cache client
The status information with target cache block is numbered at family end, including：Meta data server end determines mesh according to metadata synchronous in advance
The status information of mark cache blocks；Meta data server end determines the caching visitor that target cache block is located according to initial configuration information
Number at family end；The cache client numbering that meta data server end is located the status information of target cache block and target cache block
Return to the cache client.
Exemplary, said method can also include：In the local cache metadata updates of arbitrary cache client, to
Metadata Service end sends metadata synchronization request, and update content is synchronized in Metadata Service end.
Specifically, each cache client is put into respective local cache metadata information in kernel internal memory, and will caching
The related part metadata synchronization of hit inquiry is to meta data server end.Cache client becomes in local cache metadata
After more, metadata synchronization information is sent to meta data server end immediately.
Exemplary, said method can also include：Cache blocks replacement instruction is detected in arbitrary cache client, and is not had
When having the cache blocks in disarmed state, then using LRU (Least Recently Used, at least use in the recent period) algorithm policy
Select cache blocks to be replaced.
Specifically, in terms of the replacement policy of cache blocks, using LRU policy.The access time stamp of each cache blocks is preserved,
And the newest timestamp of whole local cache.The access time stamp of wherein each cache blocks deposits in correspondence metadata struct
Counter items in cacheblock structures, and the whole newest timestamp of local cache deposits in cache information struct
Counter items in cache_c structures.When cache blocks are accessed, the newest timestamp value of whole local cache is added 1 and assignment
Stab in the access time of cache blocks, will be all when the newest timestamp of whole local cache is maximum more than counter variables to be limited
Counter sets to 0.When cache blocks are replaced, prioritizing selection state is invalid cache blocks, by relatively more each if caching is full
The access time stamp of cache blocks selects untapped cache blocks to be at most replaced.
The embodiment of the present invention is produced for the network bandwidth to distributed cloud storage reading and writing data this bottleneck problem, proposition
In LAN pattern, the method read and write using multinode mixed block buffered data realizes the read-write to block rank cloud storage
The improvement of energy, finally realizes the general increase of cloud storage I/O performance, it is ensured that whole the stable of cloud storage system, safe operation.Tool
Body, same level network storage volume is used for multi-dummy machine example in the cloud computing system in enterprises and institutions garden is shared,
And virtual machine instance causes network storage volume I/O poor-performings with block rank cloud storage subsystem in different pieces of information center
Situation, designs a kind of block rank net of the multinode that physical disk device map is carried out based on Linux Device Mapper machines
Network caching system, using local area network relative to external network superiority bandwidth, by data cached shared in LAN
Realize the performance boost that external network is stored.
The embodiment of the present invention additionally provides a kind of data cached read-write system of multinode mixed block based on cloud storage.This is
System includes meta data server end and multiple cache clients in same LAN；
Cache client is used to receive the generic block layer read-write requests that virtual disk equipment sends, and using predefined slow
Deposit strategy the generic block layer read-write requests are forwarded to into respective physical disk unit to be processed, the general layer read-write requests
In comprising read-write disk initial sector position；
Wherein virtual disk equipment is located at same office by the local cache disk and the cache client of the cache client
The source storage disk mapping of other cache clients and meta data server end in the net of domain is obtained, the general layer read-write requests
In comprising read-write disk initial sector position.
It is exemplary, cache client specifically for：
According to read-write disk initial sector position bi_sector, default cache blocks mask block_mask and caching
Degree of association accoc, determines target cache block；
Cache hit inquiry is carried out to target cache block by the metadata or meta data server end of local cache disk,
Determine the respective physical disk of the generic block layer read-write requests；
The generic block layer read-write requests are forwarded to into respective physical disk unit to be processed.
It is exemplary, cache client specifically for：
According to read-write disk initial sector position bi_sector and default cache blocks mask block_mask, read
Initial sector position request_block of the write request data corresponding data block in the storage disk of source；
Initial sector position request_block according to the data block in the storage disk of source, calculates the data
Block correspondence caching group number set_number；
According to data block correspondence caching group number set_number and caching degree of association accoc, the caching is determined
The starting caching block number index of group；
The block number index and caching degree of association accoc is cached according to the starting determine target cache block.
Exemplary, cache client is used to be looked into meta data server end cache hit of the transmission comprising target cache block
Ask request；
Meta data server end is used to determine and returns target cache block place cache client volume to the cache client
Number and target cache block status information；
The cache blocks client is used for the cache client that target cache block is located returned according to meta data server end
The status information of numbering and target cache block, determines the respective physical disk of the generic block layer read-write requests.
It is exemplary, meta data server end specifically for：
Determine the status information of target cache block according to metadata synchronous in advance；
Determine the cache client numbering that target cache block is located according to initial configuration information；
The cache client numbering that the status information of target cache block and target cache block are located returns to caching visitor
Family end.
Exemplary, the cache client is additionally operable to：
In local cache metadata updates, metadata synchronization request is sent to Metadata Service end, by update content
In being synchronized to Metadata Service end.
Exemplary, the cache client is additionally operable to：
Detecting cache blocks replacement instruction, and when being not on the cache blocks of disarmed state, then selected using LRU policy
Cache blocks to be replaced.
In order to verify the characteristic model that this patent is proposed, we have done substantial amounts of experiment, and these experiments are mainly in office
The network data operation and transmission of long period have been carried out under the net pattern of domain on cloud storage service device.By to experimental result point
Analysis, it was demonstrated that the effectiveness and stability of the model.
Experiment relevant information is as follows：
1) hardware environment
Cloud storage test platform of the present invention is as follows：
Generic storage server test platform is as follows：
2) software environment
| Title | Operating system | Software | Function |
| Storage server | CentOS6.2X86_64 | Storage server modules | Data storage |
| Data server | CentOS6.2X86_64 | Meta data block | Data management |
| Application server | Windows 7 | Application server module |
Method of testing：
Fig. 6 is the test environment topological diagram provided in the embodiment of the present invention.With reference to Fig. 6, transport on all application servers
The preset iozone scripts of row, (script is the citing of iozone test commands line parameter：iozone1.bat)：
iozone.exe-s 30g-t 12-F f:\f1f:\f2f:\f3f:\f4f:\f5f:\f6f:\f7f:\f8f:\
f9f:\f10-c-C-w-e-r 1m-+n-+k-i 0-i 1>>iozone_1_64k.log
Test is divided into 2 processes：
1) reading and writing data is carried out using generic storage server, duration data write is carried out by IOZONE, write node
For 10 clients, each client writes 10 files simultaneously, each file 10G (each client write 100G file),
Check the time of write data costs.
2) reading and writing data is carried out using data cloud storage architecture of the present invention, duration data write is carried out by IOZONE, write
Ingress is 10 clients, and each client writes 10 files, each file 10G (each client write 100G simultaneously
File), check the time of write data costs.
Jing tests find, when carrying out reading and writing data using generic storage server, the network bandwidth difference of 10 clients
Be 17379KB/s, 19190KB/s, 17270KB/s, 21826KB/s, 17028KB/s, 25977KB/s, 22274KB/s,
16661KB/s, 16818KB/s and 16818KB/s.The averaging network of each client data storage is passed in normal cache system
Defeated speed is 1974KB/S.When carrying out reading and writing data using caching system of the present invention, the network bandwidth of 10 clients is respectively
74729KB/s、75146KB/s、74663KB/s、74286KB/s、74364KB/s、74637KB/s、74947KB/s、
74558KB/s, 75325KB/s and 75325KB/s.Therefore, each client putting down to cloud storage in caching system of the present invention
Network transfer speeds are 7671KB/S, and 4 node cloud storage maximums can provide the network polymerization bandwidth of 767MB/s.Therefore, should
Method has larger improvement really in LAN to the reading and writing data of cloud storage, has reached expected purpose.
Note, above are only presently preferred embodiments of the present invention and institute's application technology principle.It will be appreciated by those skilled in the art that
The invention is not restricted to specific embodiment described here, can carry out for a person skilled in the art various obvious changes,
Readjust and substitute without departing from protection scope of the present invention.Therefore, although the present invention is carried out by above example
It is described in further detail, but the present invention is not limited only to above example, without departing from the inventive concept, also
More other Equivalent embodiments can be included, and the scope of the present invention is determined by scope of the appended claims.
Claims (8)
1. the data cached reading/writing method of multinode mixed block of cloud storage is based on, it is characterised in that included：
Arbitrary cache client receives the generic block layer read-write requests that virtual disk equipment sends, and wherein virtual disk equipment is by this
The local cache disk and the cache client of cache client is located at other cache clients in same LAN and first number
Obtain according to the source storage disk mapping of server end, comprising read-write disk initial sector position in the general layer read-write requests；
The generic block layer read-write requests are forwarded to respective physical disk by the cache client using predefined cache policy
Equipment is processed.
2. method according to claim 1, it is characterised in that the cache client is using predefined cache policy by institute
State generic block layer read-write requests and be forwarded to respective physical disk unit and processed, including：
The cache client is according to read-write disk initial sector position, default cache blocks mask and caches degree of association, it is determined that
Target cache block；
The cache client is cached by the metadata of local cache disk or meta data server end to target cache block
Hit inquiry, determines the respective physical disk of the generic block layer read-write requests；
The generic block layer read-write requests are forwarded to respective physical disk unit and are processed by the cache client.
3. method according to claim 2, it is characterised in that the cache client is according to read-write disk initial sector position
Put, default cache blocks mask and caching degree of association, determine target cache block, including：
The cache client obtains read-write requests data pair according to read-write disk initial sector position and default cache blocks mask
Answer initial sector position of the data block in the storage disk of source；
Initial sector position of the cache client according to the data block in the storage disk of source, calculates the data block correspondence
Caching group number；
The cache client determines the starting of the caching group according to data block correspondence caching group number and caching degree of association
Caching block number；
The cache client caches block number according to the starting and the caching degree of association determines target cache block.
4. method according to claim 2, it is characterised in that the cache client is by meta data server end to target
Cache blocks carry out cache hit inquiry, determine the respective physical disk of the generic block layer read-write requests, including：
The cache client to meta data server end sends the cache hit inquiry request comprising target cache block；
Meta data server end determines and returns target cache block place cache client numbering and target to the cache client
The status information of cache blocks；
Cache client numbering and mesh that the cache blocks client is located according to the target cache block that meta data server end returns
The status information of mark cache blocks, determines the respective physical disk of the generic block layer read-write requests.
5. method according to claim 4, it is characterised in that meta data server end determines simultaneously to be returned to the cache client
The cache client numbering at target cache block place and the status information of target cache block are returned, including：
Meta data server end determines the status information of target cache block according to metadata synchronous in advance；
Meta data server end determines the cache client numbering that target cache block is located according to initial configuration information；
The cache client numbering that meta data server end is located the status information of target cache block and target cache block is returned
Give the cache client.
6. method according to claim 1, it is characterised in that also include：
In the local cache metadata updates of arbitrary cache client, metadata synchronization request is sent to Metadata Service end,
So that update content is synchronized in Metadata Service end.
7. method according to claim 1, it is characterised in that also include：
Cache blocks replacement instruction is detected in arbitrary cache client, and when being not on the cache blocks of disarmed state, is then adopted
LRU policy selects cache blocks to be replaced.
8. the data cached read-write system of multinode mixed block of cloud storage is based on, it is characterised in that including meta data server end
With multiple cache clients in same LAN；
Cache client is used to receive the generic block layer read-write requests that virtual disk equipment sends, and using predefined caching plan
Slightly the generic block layer read-write requests are forwarded to into respective physical disk unit to be processed, are wrapped in the general layer read-write requests
The disk initial sector position containing read-write；
Wherein virtual disk equipment is located at same LAN by the local cache disk and the cache client of the cache client
The source storage disk mapping of interior other cache clients and meta data server end is obtained, and is wrapped in the general layer read-write requests
The disk initial sector position containing read-write.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611199823.7A CN106648464B (en) | 2016-12-22 | 2016-12-22 | Multi-node mixed block cache data reading and writing method and system based on cloud storage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611199823.7A CN106648464B (en) | 2016-12-22 | 2016-12-22 | Multi-node mixed block cache data reading and writing method and system based on cloud storage |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106648464A true CN106648464A (en) | 2017-05-10 |
| CN106648464B CN106648464B (en) | 2020-01-21 |
Family
ID=58833633
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611199823.7A Active CN106648464B (en) | 2016-12-22 | 2016-12-22 | Multi-node mixed block cache data reading and writing method and system based on cloud storage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106648464B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107357633A (en) * | 2017-07-17 | 2017-11-17 | 郑州云海信息技术有限公司 | The creation method and creating device of a kind of virtual machine |
| CN107395732A (en) * | 2017-07-29 | 2017-11-24 | 安徽云帮邦网络技术有限公司 | A kind of business data storage inquiry cloud platform |
| CN107797769A (en) * | 2017-11-06 | 2018-03-13 | 长沙曙通信息科技有限公司 | A kind of memory virtualization system cache management strategy implementation method |
| CN109634874A (en) * | 2018-12-07 | 2019-04-16 | 联想(北京)有限公司 | A kind of data processing method, device, electronic equipment and storage system |
| CN109992373A (en) * | 2017-12-29 | 2019-07-09 | 华为技术有限公司 | Resource regulating method, approaches to IM and device and task deployment system |
| CN110471613A (en) * | 2018-05-09 | 2019-11-19 | 杭州海康威视系统技术有限公司 | The method of storing data, the method, apparatus and system for reading data |
| CN111984191A (en) * | 2020-08-05 | 2020-11-24 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Multi-client caching method and system supporting distributed storage |
| CN112650694A (en) * | 2019-10-12 | 2021-04-13 | 北京达佳互联信息技术有限公司 | Data reading method and device, cache proxy server and storage medium |
| CN116578741A (en) * | 2023-07-12 | 2023-08-11 | 南京奥看信息科技有限公司 | View hybrid storage method, device and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1564517A (en) * | 2004-03-26 | 2005-01-12 | 清华大学 | Memory-network memory-magnetic disc high speed reliable storage system and its reading/writing method |
| CN1852318A (en) * | 2006-04-19 | 2006-10-25 | 华中科技大学 | Distributed multi-stage buffer storage system suitable for object network storage |
| CN103268252A (en) * | 2013-05-12 | 2013-08-28 | 南京载玄信息科技有限公司 | Virtualization platform system based on distributed storage and achieving method thereof |
| CN103617003A (en) * | 2013-11-20 | 2014-03-05 | 浪潮电子信息产业股份有限公司 | Organization method and architecture for storage resources in cloud storage environment |
| CN105335441A (en) * | 2014-08-12 | 2016-02-17 | 阳平 | Local area network based distributed file system |
| CN105808449A (en) * | 2014-12-31 | 2016-07-27 | 中国电信股份有限公司 | Virtual storage image version management method and system for virtual machine |
-
2016
- 2016-12-22 CN CN201611199823.7A patent/CN106648464B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1564517A (en) * | 2004-03-26 | 2005-01-12 | 清华大学 | Memory-network memory-magnetic disc high speed reliable storage system and its reading/writing method |
| CN1852318A (en) * | 2006-04-19 | 2006-10-25 | 华中科技大学 | Distributed multi-stage buffer storage system suitable for object network storage |
| CN103268252A (en) * | 2013-05-12 | 2013-08-28 | 南京载玄信息科技有限公司 | Virtualization platform system based on distributed storage and achieving method thereof |
| CN103617003A (en) * | 2013-11-20 | 2014-03-05 | 浪潮电子信息产业股份有限公司 | Organization method and architecture for storage resources in cloud storage environment |
| CN105335441A (en) * | 2014-08-12 | 2016-02-17 | 阳平 | Local area network based distributed file system |
| CN105808449A (en) * | 2014-12-31 | 2016-07-27 | 中国电信股份有限公司 | Virtual storage image version management method and system for virtual machine |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107357633A (en) * | 2017-07-17 | 2017-11-17 | 郑州云海信息技术有限公司 | The creation method and creating device of a kind of virtual machine |
| CN107395732A (en) * | 2017-07-29 | 2017-11-24 | 安徽云帮邦网络技术有限公司 | A kind of business data storage inquiry cloud platform |
| CN107797769A (en) * | 2017-11-06 | 2018-03-13 | 长沙曙通信息科技有限公司 | A kind of memory virtualization system cache management strategy implementation method |
| CN109992373B (en) * | 2017-12-29 | 2021-04-09 | 华为技术有限公司 | Resource scheduling method, information management method and device and task deployment system |
| CN109992373A (en) * | 2017-12-29 | 2019-07-09 | 华为技术有限公司 | Resource regulating method, approaches to IM and device and task deployment system |
| CN110471613A (en) * | 2018-05-09 | 2019-11-19 | 杭州海康威视系统技术有限公司 | The method of storing data, the method, apparatus and system for reading data |
| CN110471613B (en) * | 2018-05-09 | 2021-01-12 | 杭州海康威视系统技术有限公司 | Data storage method, data reading method, device and system |
| CN109634874A (en) * | 2018-12-07 | 2019-04-16 | 联想(北京)有限公司 | A kind of data processing method, device, electronic equipment and storage system |
| CN112650694A (en) * | 2019-10-12 | 2021-04-13 | 北京达佳互联信息技术有限公司 | Data reading method and device, cache proxy server and storage medium |
| CN112650694B (en) * | 2019-10-12 | 2024-01-12 | 北京达佳互联信息技术有限公司 | Data reading method and device, cache proxy server and storage medium |
| CN111984191A (en) * | 2020-08-05 | 2020-11-24 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Multi-client caching method and system supporting distributed storage |
| CN116578741A (en) * | 2023-07-12 | 2023-08-11 | 南京奥看信息科技有限公司 | View hybrid storage method, device and system |
| CN116578741B (en) * | 2023-07-12 | 2023-10-20 | 南京奥看信息科技有限公司 | View hybrid storage method, device and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106648464B (en) | 2020-01-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106648464A (en) | Multi-node mixed block cache data read-writing method and system based on cloud storage | |
| US10291739B2 (en) | Systems and methods for tracking of cache sector status | |
| Saemundsson et al. | Dynamic performance profiling of cloud caches | |
| JP5006348B2 (en) | Multi-cache coordination for response output cache | |
| US11561930B2 (en) | Independent evictions from datastore accelerator fleet nodes | |
| CN109327539A (en) | A kind of distributed block storage system and its data routing method | |
| US20160212203A1 (en) | Multi-site heat map management | |
| CN103338242B (en) | A kind of mixed cloud storage system based on multi-level buffer and method | |
| CN107025243A (en) | A kind of querying method of resource data, inquiring client terminal and inquiry system | |
| CN105518631B (en) | EMS memory management process, device and system and network-on-chip | |
| CN111782612A (en) | File data edge caching method in cross-domain virtual data space | |
| CN110147345A (en) | A kind of key assignments storage system and its working method based on RDMA | |
| CN104899161A (en) | Cache method based on continuous data protection of cloud storage environment | |
| CN114844846A (en) | Multi-level cache distributed key value storage system based on programmable switch | |
| US10802748B2 (en) | Cost-effective deployments of a PMEM-based DMO system | |
| CN108319634A (en) | The directory access method and apparatus of distributed file system | |
| JP5661355B2 (en) | Distributed cache system | |
| US7529891B2 (en) | Balanced prefetching exploiting structured data | |
| Raindel et al. | Replicate and bundle (rnb)--a mechanism for relieving bottlenecks in data centers | |
| Otoo et al. | Accurate modeling of cache replacement policies in a data grid | |
| CN114390069B (en) | Data access method, system, equipment and storage medium based on distributed cache | |
| Hou et al. | Pacaca: mining object correlations and parallelism for enhancing user experience with cloud storage | |
| Xu et al. | Practical optimizations for lightweight distributed file system on consumer devices | |
| CN105095105B (en) | A kind of method and device of Cache subregions | |
| Wang et al. | A new hierarchical data cache architecture for iSCSI storage server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |