CN106648464B - Multi-node mixed block cache data reading and writing method and system based on cloud storage - Google Patents
Multi-node mixed block cache data reading and writing method and system based on cloud storage Download PDFInfo
- Publication number
- CN106648464B CN106648464B CN201611199823.7A CN201611199823A CN106648464B CN 106648464 B CN106648464 B CN 106648464B CN 201611199823 A CN201611199823 A CN 201611199823A CN 106648464 B CN106648464 B CN 106648464B
- Authority
- CN
- China
- Prior art keywords
- cache
- block
- client
- disk
- read
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Abstract
The embodiment of the invention discloses a multi-node mixed block cache data reading and writing method and system based on cloud storage. The method comprises the following steps: any cache client receives a universal block layer read-write request sent by virtual disk equipment, wherein the virtual disk equipment is obtained by mapping a local cache disk of the cache client, other cache clients and source storage disks of a metadata server end, which are positioned in the same local area network with the cache client, and the universal layer read-write request comprises the initial sector position of a read-write disk; and the cache client forwards the universal block layer read-write request to corresponding physical disk equipment for processing by adopting a predefined cache strategy. The embodiment of the invention provides a multi-node network hybrid cache system suitable for sharing and using multiple virtual machine instances, the read-write performance of the virtual machine instances to block-level cloud storage is improved, the overall improvement of the I/O performance of the cloud storage is finally realized, and the stable and safe operation of the whole cloud storage system is ensured.
Description
Technical Field
The embodiment of the invention relates to the technical field of cloud storage, in particular to a multi-node mixed block cache data reading and writing method and system based on cloud storage.
Background
Cloud computing is a service that can dynamically expand shared software and hardware resources through the internet by using a virtualization technology. The dynamic and elastic characteristics of the cloud computing enable users to use the resources as required and pay according to the used resource amount, and the users can dynamically adjust the resource usage amount according to the load condition of the current service so as to reduce resource waste. The cloud storage system is an important component of the cloud computing system. From the technical point of view, the cloud storage system is actually a large distributed network storage system, which is a special shared storage. The cloud storage system in practical use can be mainly divided into two types of object storage and block storage. The block storage (shader) is mainly used for providing an extended disk storage space for the virtual machine instance, has a persistent data storage characteristic, and can support various storages such as Ceph, Net App, Ocean Stor, Nexenta, Solid Fire, Zadara and the like through drivers.
Today's cloud computing systems often have a plurality of large data centers, each having a respective block-level cloud storage subsystem, and distributed in multiple locations. Due to the limitation of network bandwidth between the two data centers, when the virtual machine instance and the cloud storage subsystem are in different data centers, the I/O performance of the storage system is seriously influenced, the stable operation of the virtual machine instance is influenced, and the stable and efficient operation of the virtual machine instance is finally influenced.
Disclosure of Invention
The embodiment of the invention provides a multi-node mixed block cache data reading and writing method and system based on cloud storage, and aims to improve the efficiency of reading and writing cache data.
In one aspect, an embodiment of the present invention provides a multi-node mixed block cache data read-write method based on cloud storage, including:
any cache client receives a universal block layer read-write request sent by virtual disk equipment, wherein the virtual disk equipment is obtained by mapping a local cache disk of the cache client, other cache clients and source storage disks of a metadata server end, which are positioned in the same local area network with the cache client, and the universal layer read-write request comprises the initial sector position of a read-write disk;
and the cache client forwards the universal block layer read-write request to corresponding physical disk equipment for processing by adopting a predefined cache strategy.
On the other hand, the embodiment of the invention also provides a multi-node mixed block cache data read-write system based on cloud storage, which comprises a metadata server end and a plurality of cache client ends positioned in the same local area network;
the cache client is used for receiving a universal block layer read-write request sent by the virtual disk device and forwarding the universal block layer read-write request to the corresponding physical disk device for processing by adopting a predefined cache strategy, wherein the universal block layer read-write request comprises the initial sector position of a read-write disk;
the virtual disk device is obtained by mapping a local cache disk of the cache client, other cache clients and source storage disks of the metadata server end, wherein the cache clients and the source storage disks of the metadata server end are positioned in the same local area network, and the read-write request of the universal layer comprises the initial sector position of the read-write disk.
According to the technical scheme provided by the embodiment of the invention, aiming at the bottleneck problem of data reading and writing generated by the distributed cloud storage by network bandwidth, a method for buffering data reading and writing by adopting a multi-node mixed block is provided in a local area network mode, the improvement of the reading and writing performance of the block-level cloud storage is realized, the overall improvement of the I/O performance of the cloud storage is finally realized, and the stable and safe operation of the whole cloud storage system is ensured.
Drawings
Fig. 1 is a flowchart of a multi-node hybrid block cache data read-write method based on cloud storage according to a first embodiment of the present invention;
fig. 2 is a framework diagram of a multi-node mixed block cache data read-write system based on cloud storage according to an embodiment of the present invention;
fig. 3 is a structural diagram of a metadata server provided in an embodiment of the present invention;
fig. 4 is a structural diagram of a cache client provided in the embodiment of the present invention;
FIG. 5 is a diagram illustrating a cache write strategy according to an embodiment of the present invention;
fig. 6 is a topology diagram of a test environment provided in an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Fig. 1 is a flowchart of a multi-node mixed block cache data read-write method based on cloud storage according to an embodiment of the present invention. The method of the embodiment can be executed by a multi-node mixed block cache data read-write system based on cloud storage. Referring to fig. 1, the method for reading and writing data of a multi-node mixed block cache based on cloud storage may specifically include the following steps:
and step 11, any cache client receives a universal block layer read-write request sent by the virtual disk device.
The virtual disk device is obtained by mapping a local cache disk of the cache client, other cache clients located in the same local area network with the cache client and source storage disks of the metadata server, and the read-write request of the universal layer comprises the initial sector position of the read-write disk.
Referring to fig. 2, the caching system may include a metadata server and a plurality of caching clients located in the same lan. The source storage disk in the metadata server is connected to all cache clients in the local area network through an iSCSI (Internet Small computer System Interface) protocol by an external network, so that all cache clients can share and use the source storage disk. The local cache disk of each cache client is also connected to other cache clients through an iSCSI protocol, so that the other cache clients can read and write remote cache data. And each cache client maps a local cache disk, a plurality of remote cache disks (namely, other cache client nodes in the local area network, which may be a server or a plurality of disk arrays) and a source storage disk into a virtual disk device with a cache function by using a cache client program. The virtual disk device and the source storage disk have the same capacity, and a user uses the virtual disk device to replace the source storage disk. The cache client program forwards the block I/O of the virtual disk device, and the block I/O is processed by the corresponding cache disk or the source storage disk, so that the cache function is realized.
Referring to fig. 3, the metadata server mainly provides a cache hit query function of all cache client nodes for each cache client, and implements metadata synchronization with each cache client through a synchronization request. And the cache client is responsible for realizing the actual cache function and realizing the cache function through a predefined cache strategy and the communication with the metadata server.
The metadata server side is responsible for intensively processing the metadata of each cache client side and providing cache hit query service. The implementation of the metadata server mainly comprises three steps:
first, initialization configuration. And the metadata server side records cache parameters such as cache block size, cache block quantity, cache write-in strategy and the like of each cache client side, and configuration information such as node number, IP address and the like of each cache client side during initialization.
Second, metadata synchronization. And storing partial metadata (including state information of each cache block, initial sector information of a source storage disk corresponding to the cache block and the like) related to cache hit query in each cache client node, and performing metadata updating operation of a server side by a metadata server side by receiving a metadata synchronization request of each cache client.
Third, cache hit queries. The metadata server provides cache hit query service for each cache client, receives a query request (including initial sector information of a cache block and the like) of the cache client and returns a query result (a client number, state information and the like where the cache block is located). The metadata synchronization between the cached metadata server and the client and the cache hit query are both realized by a User Datagram Protocol (UDP) communication Protocol.
Referring to fig. 4, the cache client is responsible for an actual cache function, and performs a cache operation according to a preset cache policy. The implementation of the cache client mainly comprises three parts:
first, the configuration is initialized. As with the metadata server, the cache client first records cache parameters such as cache block size, cache block number, cache write-in policy, and the like, as well as node numbers of each cache client and information of corresponding remote cache disk devices during initialization.
Second, the metadata is synchronized. And each cache client puts the local cache metadata information into the kernel memory and synchronizes part of metadata related to cache hit query to the metadata server. And after the local cache metadata is changed, the cache client side immediately sends metadata synchronization information to the metadata server side. And each cache client simultaneously receives and processes the modification request of the metadata server end to the local cache metadata.
Third, cache implementation. In order to realize the cache function, each cache client maps the local cache disk, the remote cache disks and the source storage disk into a virtual disk Device with the cache function by using a Linux Device Mapper mechanism. For the block I/O of the virtual disk device, the cache client inquires local and remote cache metadata, and forwards a general block layer bio (io request of the block device) sent by the virtual disk device to the corresponding physical disk device for processing according to a predefined cache policy.
The implementation for the cache function also includes the following five main points:
(1) the actual cache data is stored in the cache disk of each cache client node, the cache data is organized into blocks according to the size of the cache blocks in the initialization configuration, and each cache data block only stores one copy in all the cache client nodes.
(2) And for the mapping between the cache block and the data block, a group association mode is adopted, and the mapping between the data block of the source storage disk and the cache block of the cache disk is realized by a hash method.
(3) In terms of the replacement policy of the cache block, the LRU policy is used.
(4) Since the cache disks are distributed in a plurality of cache client nodes in the local area network, the local cache disk and the remote cache disk of each node have performance difference. According to the cache mapping rule, the cache client side can preferentially use the local cache disk with better performance.
(5) The cache client is realized in a kernel module mode, so that a user can dynamically insert or remove the cache module into or from the Linux kernel according to requirements.
The client and server communication is realized as follows:
the data communication between the metadata server and each cache client comprises two parts: metadata synchronization and cache hit lookup. The metadata synchronization part comprises metadata updating of a metadata server side and metadata updating of a cache client side. The metadata updating request of the metadata server side is initiated by the cache client side. And triggering when cache metadata of the cache client side is changed, and sending the initial sector of the source storage disk, the cache block number, the cache block state information and the cache client side node number information which sends an updating request of the cache block which is corresponding to the metadata change cache block to the metadata server side for updating the metadata. And the metadata updating request of the cache client is initiated by the metadata server and is generated only when the write operation is carried out. When the cache metadata are changed due to cache hit query, the updating request is triggered, and the metadata change cache block number and the cache block state information are sent to the corresponding cache client side for updating the metadata. The caching client also responds to the update request number confirmation. The cache hit query portion includes a cache block query request and a response thereto. The cache block query request is initiated by the cache client. And triggering the request when the cache is not locally hit, sending the initial sector of the source storage disk and the I/O read-write direction corresponding to the cache block to the metadata server, and waiting for the response of the query request.
Specifically, any cache client receives a general block layer read-write request (bio request) sent by the virtual disk device, and the bio structure includes a read-write disk start sector position.
And step 12, the cache client forwards the general block layer read-write request to corresponding physical disk equipment for processing by adopting a predefined cache strategy.
The predefined caching policy mainly includes a caching block mapping policy, a caching write policy, and a caching block replacement policy. The association degree (assoc) of the cache write-in strategy and the cache block mapping strategy can be customized by a user when the cache system is initialized, and the user can set parameters such as the size of cache blocks, the number of cache blocks, the IP (Internet protocol) of the cache client and the like of the cache system.
Specifically, the cache client determines the physical disk device corresponding to the read-write request of the general block layer according to the initial sector position of the read-write disk by using a cache block mapping strategy, and the corresponding physical disk device performs read-write processing by using a cache write strategy.
Aiming at write-in operation, two write-through strategies and write-back strategies are mainly provided, the write-through strategy is biased to data security, the write-back strategy is biased to write-in performance, and a user can set a cache write-in strategy according to requirements.
Referring to FIG. 5, when using the write-through strategy, all block I/O data is written directly to the remote source storage disk and not cached to the cache disk. If the written data is cached in the local or remote cache disk, the state of the corresponding cache block needs to be set as invalid. The write strategy can ensure the data consistency between the cache disk and the source storage disk, and can still ensure the data security under the condition that cache data is lost due to the problem of the cache client node.
Referring to fig. 5, when using the write-back policy, all block I/O data is written directly to the local cache disk or the remote cache disk, and all cache data will be deferred to be written back to the source storage disk. The delayed write-back operation is carried out when the cache block is replaced or when the cache disk is unloaded after being used, the cache client program carries out flushing operation, and data in all dirty cache blocks are uniformly written back to a remote source storage disk. Of course, cache flush operations may have some impact on the performance of the corresponding cache client node. The write-back strategy is theoretically used for improving the data writing speed and reducing the load pressure of the remote network storage system. However, there is also a risk of using the write-back policy, and when a cache client node fails, cache data will be lost.
For the mapping between the cache block and the data block, the system adopts a group association mode. Combining the cache blocks of each cache association degree into a cache group according to the predefined cache association degree, and mapping the data block of the source storage disk and the cache block of the cache disk by a hash (hash) method.
Illustratively, step 12 may include:
A. the cache client determines a target cache block according to the initial sector position of the read-write disk, a preset cache block mask and cache association degree.
Further, step a may specifically include:
a1, the cache client obtains the initial sector position of the data block corresponding to the read-write request data in the source storage disk according to the initial sector position of the read-write disk and the preset cache block mask.
Specifically, for a general block layer bio request from an upper File System or a VFS (virtual File System) layer, according to a read-write disk start sector position bi _ sector in a bio structure, the cache client performs bitwise and operation with a preset cache block mask _ mask to obtain an offset of the read-write disk start sector position bi _ sector in a corresponding data block, and then subtracts the offset from the bi _ sector to obtain a start sector request _ block of the data block in the source storage disk device.
A2, the cache client calculates the cache group number corresponding to the data block according to the initial sector position of the data block in the source storage disk.
Specifically, the cache client calculates the database number by using a hash _ block function according to the initial sector position request _ block, and then divides the database number by the cache association degree assoc to obtain the cache group number set _ number.
A3, the cache client determines the initial cache block number of the cache group according to the cache group number corresponding to the data block and the cache association degree.
Specifically, the cache client multiplies the cache group number set _ number corresponding to the data block by the cache association degree assoc to obtain the starting cache block number index of the cache group.
A4, the cache client determines the target cache block according to the number of the initial cache block and the cache associativity.
Specifically, the cache client determines the cache blocks with the starting cache block numbers index to index + assoc as target cache blocks.
B. And the cache client performs cache hit query on a target cache block through metadata of a local cache disk or a metadata server to determine a corresponding physical disk of the read-write request of the universal block layer.
Specifically, the cache client performs hit query on a target cache block through metadata of a local cache disk to determine whether a corresponding physical disk of the read-write request of the universal block layer is the local cache disk; and the cache client performs cache hit query on the target cache block through the metadata server to determine whether the corresponding physical disk of the read-write request of the universal block layer is other cache clients or source storage disks.
And searching for a metadata cacheblock item from the cache block number index to the index + assoc by using the cache block metadata struct cacheblock. Looking up all cacheblocks, wherein the state of the cache block is VALID, DIRTY, RESERVED or write back (if the state is RESERVED or write back, it indicates that the cache block is performing cache write or cache write back operation, the current bio request is put into the bios list of cacheblock for delay processing), and the sector start position block of the data block corresponding to the cache block is equal to request _ block. If the cacheblock item is found, the cache is hit; if not, then the cache misses.
C. And forwarding the read-write request of the general block layer to corresponding physical disk equipment for processing.
Illustratively, the cache client performs cache hit query on a target cache block through a metadata server, and determines a corresponding physical disk of the universal block layer read-write request, including: the cache client sends a cache hit query request containing a target cache block to a metadata server; the metadata server side determines and returns the number of the cache client side where the target cache block is located and the state information of the target cache block to the cache client side; and the cache block client determines a corresponding physical disk of the read-write request of the universal block layer according to the cache client number of the target cache block returned by the metadata server and the state information of the target cache block.
Exemplarily, the determining and returning the cache client number where the target cache block is located and the state information of the target cache block to the cache client by the metadata server includes: the metadata server side determines the state information of a target cache block according to the pre-synchronized metadata; the metadata server side determines the number of the cache client side where the target cache block is located according to the initialization configuration information; and the metadata server side returns the state information of the target cache block and the number of the cache client side where the target cache block is located to the cache client side.
Illustratively, the method may further include: and when the local cache metadata of any cache client side is updated, sending a metadata synchronization request to the metadata server side so as to synchronize the updated content to the metadata server side.
Specifically, each cache client puts the local cache metadata information into the kernel memory, and synchronizes part of metadata related to cache hit query to the metadata server. And after the local cache metadata is changed, the cache client side immediately sends metadata synchronization information to the metadata server side.
Illustratively, the method may further include: when any cache client detects a cache block replacement instruction and has no cache block in an invalid state, an LRU (Least Recently Used) algorithm strategy is adopted to select the cache block to be replaced.
Specifically, in terms of the replacement policy of the cache block, the LRU policy is used. The access timestamp of each cache block is saved, as well as the latest timestamp of the entire local cache. The access timestamp of each cache block is stored in the counter entry in the corresponding metadata structcacheblock structure, and the latest timestamp of the whole local cache is stored in the counter entry in the cache information structcachec structure. When the cache block is accessed, the entire local cache latest timestamp value is incremented by 1 and assigned to the cache block's access timestamp, and all counters are set to 0 when the entire local cache latest timestamp exceeds the counter variable maximum limit. When the cache block is replaced, the cache block with invalid state is preferentially selected, and if the cache is full, the cache block which is not used for the longest time is selected for replacement by comparing the access time stamps of all the cache blocks.
Aiming at the bottleneck problem of data reading and writing generated by the distributed cloud storage through network bandwidth, the embodiment of the invention provides a method for buffering data reading and writing by adopting a multi-node mixed block in a local area network mode, so that the improvement of the reading and writing performance of the block-level cloud storage is realized, the overall improvement of the I/O performance of the cloud storage is finally realized, and the stable and safe operation of the whole cloud storage system is ensured. Specifically, aiming at the condition that multiple virtual machine instances share and use the same block-level network storage volume in a cloud computing system in an enterprise park, and the I/O performance of the network storage volume is poor due to the fact that the virtual machine instances and a block-level cloud storage subsystem are located in different data centers, a multi-node block-level network cache system for mapping physical disk equipment based on a Linux Device Mapper is designed, and the performance improvement of external network storage is realized through cache data sharing in a local area network by utilizing the bandwidth advantage of the local area network relative to an external network.
The embodiment of the invention also provides a multi-node mixed block cache data reading and writing system based on cloud storage. The system comprises a metadata server and a plurality of cache clients located in the same local area network;
the cache client is used for receiving a universal block layer read-write request sent by the virtual disk device and forwarding the universal block layer read-write request to the corresponding physical disk device for processing by adopting a predefined cache strategy, wherein the universal block layer read-write request comprises the initial sector position of a read-write disk;
the virtual disk device is obtained by mapping a local cache disk of the cache client, other cache clients and source storage disks of the metadata server end, wherein the cache clients and the source storage disks of the metadata server end are positioned in the same local area network, and the read-write request of the universal layer comprises the initial sector position of the read-write disk.
Illustratively, the cache client is specifically configured to:
determining a target cache block according to the initial sector position bi _ sector of the read-write disk, a preset cache block _ mask and a cache associativity accoc;
performing cache hit query on a target cache block through metadata of a local cache disk or a metadata server end, and determining a corresponding physical disk of the read-write request of the universal block layer;
and forwarding the read-write request of the general block layer to corresponding physical disk equipment for processing.
Illustratively, the cache client is specifically configured to:
obtaining an initial sector position request _ block of a data block corresponding to read-write request data in a source storage disk according to an initial sector position bi _ sector of the read-write disk and a preset cache block mask _ mask;
calculating a cache group number set _ number corresponding to the data block according to an initial sector position request _ block of the data block in a source storage disk;
determining the initial cache block number index of the cache group according to the cache group number set _ number corresponding to the data block and the cache association degree accoc;
and determining a target cache block according to the initial cache block number index and the cache association degree accoc.
Illustratively, the cache client is configured to send a cache hit query request including a target cache block to the metadata server;
the metadata server is used for determining and returning the number of the cache client where the target cache block is located and the state information of the target cache block to the cache client;
the cache block client is used for determining a corresponding physical disk of the read-write request of the universal block layer according to the cache client number of the target cache block returned by the metadata server and the state information of the target cache block.
Illustratively, the metadata server is specifically configured to:
determining state information of a target cache block according to pre-synchronized metadata;
determining the number of a cache client where the target cache block is located according to the initialization configuration information;
and returning the state information of the target cache block and the number of the cache client where the target cache block is positioned to the cache client.
Illustratively, the cache client is further configured to:
and when the local cache metadata is updated, sending a metadata synchronization request to the metadata server so as to synchronize the updated content into the metadata server.
Illustratively, the cache client is further configured to:
when a cache block replacing instruction is detected and no cache block in an invalid state exists, the cache block to be replaced is selected by adopting an LRU strategy.
In order to verify the characteristic model provided by the patent, a large number of experiments are performed, and the experiments mainly include that long-time network data operation and transmission are performed on a cloud storage server in a local area network mode. The effectiveness and stability of the model are proved by analyzing the experimental results.
The relevant information of the experiment is as follows:
1) hardware environment
The cloud storage test platform comprises the following components:
the test platform of the common storage server comprises the following steps:
2) software environment
| Name (R) | Operating system | Software | Function(s) |
| Storage server | CentOS6.2X86_64 | Storage server module | Data storage |
| Data server | CentOS6.2X86_64 | Metadata module | Data management |
| Application server | Windows 7 | Application server module |
The test method comprises the following steps:
fig. 6 is a topology diagram of a test environment provided in an embodiment of the present invention. Referring to fig. 6, a preset iozone script is run on all application servers, and the iozone test command line parameters are exemplified (the script is iozone1. bat):
iozone.exe-s 30g-t 12-F f:\f1f:\f2f:\f3f:\f4f:\f5f:\f6f:\f7f:\f8f:\f9f:\f10-c-C-w-e-r 1m-+n-+k-i 0-i 1>>iozone_1_64k.log
the test is divided into 2 processes:
1) the data are read and written by adopting a common storage server, continuous data writing is carried out through the IOZONE, the writing node is 10 clients, each client writes 10 files simultaneously, each file is 10G (each client writes 100G files), and the time spent on writing data is checked.
2) The data cloud storage structure is adopted for data reading and writing, continuous data writing is carried out through the IOZONE, the writing nodes are 10 clients, each client writes 10 files simultaneously, each file is 10G (each client writes 100G files), and the time spent on writing data is checked.
Tests show that when a common storage server is used for data reading and writing, the network bandwidths of 10 clients are 17379KB/s, 19190KB/s, 17270KB/s, 21826KB/s, 17028KB/s, 25977KB/s, 22274KB/s, 16661KB/s, 16818KB/s and 16818KB/s respectively. The average network transmission speed of each client in the common cache system for storing data is 1974 KB/S. When the cache system is adopted for data reading and writing, the network bandwidths of 10 clients are 74729KB/s, 75146KB/s, 74663KB/s, 74286KB/s, 74364KB/s, 74637KB/s, 74947KB/s, 74558KB/s, 75325KB/s and 75325KB/s respectively. Therefore, the average network transmission speed from each client to the cloud storage in the cache system is 7671KB/S, and the maximum network aggregation bandwidth of 767MB/S can be provided by 4 node cloud storage. Therefore, the method has great improvement on the data reading and writing of the cloud storage in the local area network, and achieves the expected purpose.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (8)
1. A multi-node mixed block cache data read-write method based on cloud storage is characterized by comprising the following steps:
any cache client receives a universal block layer read-write request sent by virtual disk equipment, wherein the virtual disk equipment is obtained by mapping a local cache disk of the cache client, other cache clients and source storage disks of a metadata server end, which are positioned in the same local area network with the cache client, and the universal layer read-write request comprises the initial sector position of a read-write disk;
the cache client forwards the universal block layer read-write request to corresponding physical disk equipment for processing by adopting a predefined cache strategy;
the virtual disk equipment has a cache function and has the same capacity as a source storage disk, and a user uses the virtual disk equipment to replace the source storage disk;
the physical disk device comprises the local cache disk, other cache clients or a source storage disk.
2. The method of claim 1, wherein the cache client forwards the generic block-level read-write request to a corresponding physical disk device for processing using a predefined cache policy, comprising:
the cache client determines a target cache block according to the initial sector position of the read-write disk, a preset cache block mask and cache association degree;
the cache client performs cache hit query on a target cache block through metadata of a local cache disk or a metadata server end, and determines a corresponding physical disk of the read-write request of the universal block layer;
and the cache client forwards the universal block layer read-write request to corresponding physical disk equipment for processing.
3. The method of claim 2, wherein the determining, by the cache client, the target cache block according to the starting sector location of the read/write disk, a preset cache block mask, and the cache association degree comprises:
the cache client obtains the initial sector position of the data block corresponding to the read-write request data in the source storage disk according to the initial sector position of the read-write disk and a preset cache block mask;
the cache client calculates a cache group number corresponding to the data block according to the initial sector position of the data block in a source storage disk;
the cache client determines the initial cache block number of the cache group according to the cache group number corresponding to the data block and the cache association degree;
and the cache client determines a target cache block according to the number of the initial cache block and the cache associativity.
4. The method of claim 2, wherein the cache client performs cache hit query on a target cache block through the metadata server to determine a corresponding physical disk of the generic block layer read-write request, comprising:
the cache client sends a cache hit query request containing a target cache block to a metadata server;
the metadata server side determines and returns the number of the cache client side where the target cache block is located and the state information of the target cache block to the cache client side;
and the cache block client determines a corresponding physical disk of the read-write request of the universal block layer according to the cache client number of the target cache block returned by the metadata server and the state information of the target cache block.
5. The method of claim 4, wherein the determining and returning, by the metadata server, the number of the cache client where the target cache block is located and the state information of the target cache block to the cache client comprises:
the metadata server side determines the state information of a target cache block according to the pre-synchronized metadata;
the metadata server side determines the number of the cache client side where the target cache block is located according to the initialization configuration information;
and the metadata server side returns the state information of the target cache block and the number of the cache client side where the target cache block is located to the cache client side.
6. The method of claim 1, further comprising:
and when the local cache metadata of any cache client side is updated, sending a metadata synchronization request to the metadata server side so as to synchronize the updated content to the metadata server side.
7. The method of claim 1, further comprising:
and when any cache client detects a cache block replacing instruction and has no cache block in an invalid state, selecting the cache block to be replaced by adopting an LRU (least recently used) strategy.
8. The multi-node mixed block cache data read-write system based on cloud storage is characterized by comprising a metadata server and a plurality of cache clients located in the same local area network;
the cache client is used for receiving a universal block layer read-write request sent by the virtual disk device and forwarding the universal block layer read-write request to the corresponding physical disk device for processing by adopting a predefined cache strategy, wherein the universal block layer read-write request comprises the initial sector position of a read-write disk;
the virtual disk device is obtained by mapping a local cache disk of the cache client, other cache clients which are positioned in the same local area network with the cache client and a source storage disk of the metadata server, and the read-write request of the universal layer comprises the initial sector position of the read-write disk;
the virtual disk equipment has a cache function and has the same capacity as a source storage disk, and a user uses the virtual disk equipment to replace the source storage disk;
the physical disk device comprises the local cache disk, other cache clients or a source storage disk.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611199823.7A CN106648464B (en) | 2016-12-22 | 2016-12-22 | Multi-node mixed block cache data reading and writing method and system based on cloud storage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611199823.7A CN106648464B (en) | 2016-12-22 | 2016-12-22 | Multi-node mixed block cache data reading and writing method and system based on cloud storage |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106648464A CN106648464A (en) | 2017-05-10 |
| CN106648464B true CN106648464B (en) | 2020-01-21 |
Family
ID=58833633
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611199823.7A Active CN106648464B (en) | 2016-12-22 | 2016-12-22 | Multi-node mixed block cache data reading and writing method and system based on cloud storage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106648464B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107357633A (en) * | 2017-07-17 | 2017-11-17 | 郑州云海信息技术有限公司 | The creation method and creating device of a kind of virtual machine |
| CN107395732A (en) * | 2017-07-29 | 2017-11-24 | 安徽云帮邦网络技术有限公司 | A kind of business data storage inquiry cloud platform |
| CN107797769A (en) * | 2017-11-06 | 2018-03-13 | 长沙曙通信息科技有限公司 | A kind of memory virtualization system cache management strategy implementation method |
| CN109992373B (en) * | 2017-12-29 | 2021-04-09 | 华为技术有限公司 | Resource scheduling method, information management method and device and task deployment system |
| CN110471613B (en) * | 2018-05-09 | 2021-01-12 | 杭州海康威视系统技术有限公司 | Data storage method, data reading method, device and system |
| CN109634874A (en) * | 2018-12-07 | 2019-04-16 | 联想(北京)有限公司 | A kind of data processing method, device, electronic equipment and storage system |
| CN112650694B (en) * | 2019-10-12 | 2024-01-12 | 北京达佳互联信息技术有限公司 | Data reading method and device, cache proxy server and storage medium |
| CN111984191A (en) * | 2020-08-05 | 2020-11-24 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Multi-client caching method and system supporting distributed storage |
| CN116578741B (en) * | 2023-07-12 | 2023-10-20 | 南京奥看信息科技有限公司 | View hybrid storage method, device and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1564517A (en) * | 2004-03-26 | 2005-01-12 | 清华大学 | Memory-network memory-magnetic disc high speed reliable storage system and its reading/writing method |
| CN1852318A (en) * | 2006-04-19 | 2006-10-25 | 华中科技大学 | Distributed multi-stage buffer storage system suitable for object network storage |
| CN103268252A (en) * | 2013-05-12 | 2013-08-28 | 南京载玄信息科技有限公司 | Virtualization platform system based on distributed storage and achieving method thereof |
| CN103617003A (en) * | 2013-11-20 | 2014-03-05 | 浪潮电子信息产业股份有限公司 | Organization method and architecture for storage resources in cloud storage environment |
| CN105335441A (en) * | 2014-08-12 | 2016-02-17 | 阳平 | Local area network based distributed file system |
| CN105808449A (en) * | 2014-12-31 | 2016-07-27 | 中国电信股份有限公司 | Virtual storage image version management method and system for virtual machine |
-
2016
- 2016-12-22 CN CN201611199823.7A patent/CN106648464B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1564517A (en) * | 2004-03-26 | 2005-01-12 | 清华大学 | Memory-network memory-magnetic disc high speed reliable storage system and its reading/writing method |
| CN1852318A (en) * | 2006-04-19 | 2006-10-25 | 华中科技大学 | Distributed multi-stage buffer storage system suitable for object network storage |
| CN103268252A (en) * | 2013-05-12 | 2013-08-28 | 南京载玄信息科技有限公司 | Virtualization platform system based on distributed storage and achieving method thereof |
| CN103617003A (en) * | 2013-11-20 | 2014-03-05 | 浪潮电子信息产业股份有限公司 | Organization method and architecture for storage resources in cloud storage environment |
| CN105335441A (en) * | 2014-08-12 | 2016-02-17 | 阳平 | Local area network based distributed file system |
| CN105808449A (en) * | 2014-12-31 | 2016-07-27 | 中国电信股份有限公司 | Virtual storage image version management method and system for virtual machine |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106648464A (en) | 2017-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106648464B (en) | Multi-node mixed block cache data reading and writing method and system based on cloud storage | |
| US10523786B2 (en) | I/O bandwidth reduction using storage-level common page information | |
| EP1569085B1 (en) | Method and apparatus for increasing data storage capacity | |
| US9330108B2 (en) | Multi-site heat map management | |
| US9798655B2 (en) | Managing a cache on storage devices supporting compression | |
| US8046422B2 (en) | Automatic load spreading in a clustered network storage system | |
| EP2502148B1 (en) | Selective file system caching based upon a configurable cache map | |
| US7058763B2 (en) | File system for caching web proxies | |
| US9251003B1 (en) | Database cache survivability across database failures | |
| US11656803B2 (en) | Tiering data strategy for a distributed storage system | |
| US20130232215A1 (en) | Virtualized data storage system architecture using prefetching agent | |
| US7529972B2 (en) | Methods and apparatus for reconfiguring a storage system | |
| US10102147B1 (en) | Phased based distributed LRU for shared cache systems | |
| US11809707B2 (en) | File operations in a distributed storage system | |
| US10042763B2 (en) | Write back coordination node for cache latency correction | |
| US10942867B2 (en) | Client-side caching for deduplication data protection and storage systems | |
| US10452619B1 (en) | Decreasing a site cache capacity in a distributed file system | |
| Lensing et al. | hashfs: Applying hashing to optimize file systems for small file reads | |
| US20200379686A1 (en) | Flash registry with write leveling | |
| US7539838B1 (en) | Methods and apparatus for increasing the storage capacity of a storage system | |
| US20210286734A1 (en) | Persistent read cache in a scale out storage system | |
| CN115129618A (en) | Method and apparatus for optimizing data caching | |
| US9430387B2 (en) | Decoupling data and metadata in hierarchical cache system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |