WO2017113329A1 - 一种主机集群中缓存管理方法及主机 - Google Patents

一种主机集群中缓存管理方法及主机 Download PDF

Info

Publication number
WO2017113329A1
WO2017113329A1 PCT/CN2015/100190 CN2015100190W WO2017113329A1 WO 2017113329 A1 WO2017113329 A1 WO 2017113329A1 CN 2015100190 W CN2015100190 W CN 2015100190W WO 2017113329 A1 WO2017113329 A1 WO 2017113329A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
cache device
host
application
target
Prior art date
Application number
PCT/CN2015/100190
Other languages
English (en)
French (fr)
Inventor
刘叙友
明亮
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN201580002565.6A priority Critical patent/CN107851062A/zh
Priority to PCT/CN2015/100190 priority patent/WO2017113329A1/zh
Publication of WO2017113329A1 publication Critical patent/WO2017113329A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems

Definitions

  • the present invention relates to the field of information technology, and in particular, to a cache management method and a host in a host cluster.
  • VMs Virtual Machines
  • VMs 2 VMs 3, and VMs 4 run on the host (or server) shown in FIG.
  • Each VM on the host has a separate operating system.
  • Application 1 runs on VM 1
  • VM 2 runs Application 2
  • VM 3 runs Application 3
  • VM 4 runs Application 4.
  • the host divides the cache device into a fixed partition as a virtual cache device and allocates it to the VM. .
  • partition 1 is assigned as a virtual cache device to VM 1
  • partition 2 is assigned to VM 2 as a virtual cache device
  • partition 3 is assigned to VM 3 as a virtual cache device
  • partition 4 is assigned to VM 4 as a virtual cache device.
  • the virtual cache device can be allocated to the VM, thereby improving the IO performance of the VM, once the virtual cache device is allocated to the VM, the capacity of the virtual cache device is actually allocated, and the VM is virtualized during operation.
  • the capacity of the cache device is not adjustable, which causes a waste of cache space of the virtual cache device.
  • the present invention provides a cache management method in a host cluster, where the host cluster includes a cache resource pool composed of a local cache device of one or more hosts, such as a target host.
  • the cache resource pool includes a plurality of data blocks, each of which is mapped to a physical block of the local cache device, where the target host runs an application; the method includes:
  • the central processing unit CPU of the target host receives a data request sent by the application
  • the CPU of the target host allocates a target data block from the plurality of data blocks of the cache resource pool according to an attribute of the virtual cache device to buffer data operated by the data request;
  • the CPU of the target host establishes a correspondence between the target data block and the virtual cache device.
  • the virtual cache device management method provided by the implementation of the present invention can dynamically allocate data blocks according to data requests of the application, improve the utilization of the virtual cache device, and reduce the waste of the cache space of the virtual cache device.
  • the CPU of the target host establishes a mapping relationship between an address of the volume carried by the data request and a logical address of the target data block, to indicate that the target data block in the virtual cache device caches the The data that the data request operates on.
  • the target host stores a correspondence between the identifier of the application and the identifier of the virtual cache device, where the CPU of the target host determines, according to the data request, that the virtual cache device of the application specifically includes: The CPU of the target host determines the virtual cache device according to the correspondence between the identifier of the application and the identifier of the virtual cache device.
  • the target host stores a correspondence between an identifier of the volume that is mounted by the application and an identifier of the virtual cache device, where the CPU of the target host determines the virtual cache device of the application according to the data request. Specifically, the CPU of the target host determines the virtual cache device according to the identifier of the volume of the application mounted by the CPU and the identifier of the virtual cache device.
  • the CPU of the target host allocates a target data block from the plurality of data blocks of the cache resource pool according to an attribute of the virtual cache device to cache the
  • the data operated by the data request is specifically: the CPU of the target host allocates the target data block from the plurality of data blocks of the cache resource pool according to an attribute of the virtual cache device to cache the write data.
  • the CPU of the target host allocates a target data block from the plurality of data blocks of the cache resource pool according to an attribute of the virtual cache device to cache
  • the data operated by the data request is specifically: the CPU of the target host reads data to be read from the volume mounted by the application; the CPU of the target host is based on the attribute of the virtual cache device. Allocating the target data block in the plurality of data blocks of the cache resource pool to cache the data to be read.
  • the size of the data blocks in the cache resource pool are different to provide different granularity of cache.
  • data blocks and physical blocks of the same size are used to establish a mapping.
  • the application is a virtual machine, an application running in a virtual machine, or an application running on a host.
  • the present invention also provides a host for performing the above solution. Accordingly, the present invention also provides a non-transitory computer readable storage medium and computer program product to implement the above aspects.
  • FIG. 1 is a schematic diagram of an application scenario in the prior art
  • FIG. 3 is a schematic diagram of a local cache resource pool of the host 1;
  • FIG. 4 is a schematic diagram of a relationship between a virtual cache device and a cache resource pool
  • FIG. 5 is a schematic diagram of a relationship between a virtual cache device and a cache resource pool
  • FIG. 6 is a schematic diagram of a relationship between a virtual cache device and a cache resource pool
  • FIG. 7 is a schematic diagram of a relationship between a virtual cache device and a cache resource pool
  • FIG. 9 is a schematic diagram of an application scenario according to an embodiment of the present invention.
  • FIG. 10 is a schematic diagram of a cache management method in a host cluster according to an embodiment of the present invention.
  • FIG. 11 is a schematic structural diagram of a target host according to an embodiment of the present invention.
  • the host cluster includes host 1, host 2 to host N, and the hosts communicate with each other, where N is an integer.
  • a plurality of VMs are run on each host, and in the embodiment of the present invention, VM 1 , VM 2 to VM N are respectively indicated.
  • the number of hosts and the number of VMs in the present invention are merely exemplary, and the number of hosts and the number of VMs can be determined according to actual needs.
  • a host also known as a server or node, runs a VM by using virtualization software. Virtualization software can be (Huawei Technologies Co., Ltd. calculates virtualization software).
  • the hardware of each host contains a local cache device.
  • the hardware of the host includes a local cache device, where the local cache device is built in the host or the local cache device is connected to the host, and is hereinafter referred to as a local cache device of the host.
  • the permanent storage device required by the host may also be located outside the host or built in the host, which is not limited in this embodiment of the present invention.
  • the local cache device may be a solid state storage device (SSD), a Peripheral Component Interconnect Express (PCIe) flash card, or other device as a cache device, which is not limited in this embodiment of the present invention.
  • the hardware of the host further includes a CPU, and those skilled in the art may know that the host includes other hardware, such as main memory, etc., in addition to the hardware device shown in FIG. No longer listed.
  • each host pools the local cache device.
  • the so-called pooling is to logically divide the local cache device into fixed-size data blocks.
  • the blocks constitute a cache resource pool, and the data block corresponds to a fixed-size physical block on the local cache device, and each data block is mapped to a physical block on the local cache device, that is, the logical block of the data block is mapped to the physical block on the local cache device.
  • the physical address, or data block corresponds to a physical block on the local cache device. Therefore, the data block in the embodiment of the present invention is a logical concept.
  • the local cache device may have physical blocks of different sizes.
  • each host local cache device can form a cache resource pool. Specifically, each host local cache device may form a cache resource pool, or multiple host local cache devices form a cache resource pool, or all host local cache devices form a cache resource pool.
  • the local cache device 1 of the host 1 forms a cache resource pool 301a in which the local cache device 1 is logically divided into fixed-size data blocks, such as 4 KB.
  • a 4KB-sized data block is mapped to a 4KB-sized physical block in the local cache device 1, that is, a logical address of a 4KB-sized data block is mapped to a 4KB-sized physical block in the local cache device 1.
  • the local cache devices from Host 2 to Host N also form a similar cache resource pool.
  • the size of the data block in the cache resource pool formed by the local cache device of each host may be the same or different.
  • the cache resource pool 301a is also referred to as a block device.
  • the host 1 creates different virtual cache devices Vcache 11 to Vcache 1M in the cache resource pool, where M is an integer.
  • Host 1 creates virtual cache devices Vcache 11 through Vcache 1M and records the metadata of Vcache 11 to Vcache 1M, respectively.
  • Vcache 11 provides a cache for VM 11 on host 1.
  • the metadata of Vcache 11 includes the correspondence between Vcache 11 and volume V1 mounted by VM 11 (for example, by recording the identifier of Vcache 11 and the identifier of V1).
  • the corresponding relationship is implemented), the identifier of Vcache 11, and the size of Vcache 11, etc., and may also include the correspondence between Vcache 11 and VM 11. (It can be realized by recording the correspondence between the identifier of Vcache 11 and the identifier of VM 11).
  • the correspondence between the Vcache 11 and the volume V1 mounted by the VM 11 for example, by recording the correspondence between the identifier of the Vcache 11 and the identifier of the V1 and the correspondence between the Vcache 11 and the VM 11 (specifically It can be realized by recording the correspondence between the identifier of the Vcache 11 and the identifier of the VM 11) or not in the metadata of the Vcache 11, but stored in a specific area of the host 1 for the VM 11 to access the Vcache 11. Therefore, the Vcache 11, the cache resource pool 301a and the local cache device 1 of the host 1 form a relationship as shown in FIG.
  • creating a Vcache does not require allocating data blocks for the Vcache.
  • the correspondence between the Vcache and the data block is established, that is, the data block is allocated for the data cached by the Vcache, thereby improving the utilization of the local cache device and reducing the waste of the cache space of the local cache device.
  • Vcache 11 is used to cache data written by VM 11 to V1, or to cache data read by VM 11 from V1.
  • the CPU of the host 1 receives the write data request sent by the VM 11, the write data request carries the identifier of the VM 11, and the CPU queries the correspondence between the identifier of the VM 11 and the identifier of the Vcache 11.
  • the write data request carries the identifier of the volume V1 mounted by the VM 11
  • the CPU queries the correspondence between the identifier of V1 and the identifier of the Vcache 11, and determines the virtuality of the VM 11
  • the cache device Vcache 11 allocates a data block from the cache resource pool 301a for the write request according to the attribute of the Vcache 11 to buffer the data carried by the write data request.
  • the allocation of the data block for the write request from the cache resource pool 301a according to the attribute of the Vcache 11 includes: from the cache resource pool according to the size of the middle data block of the Vcache 11.
  • the data block of the same size is selected in the 301a, and the data carried by the write request is stored in the physical block of the local cache device corresponding to the data block (the other expression of the same meaning is to store the data carried by the write request to the data.
  • the CPU establishes a correspondence between the address to be written (the address in V1) carried by the write request and the logical address of the data block, to indicate that the logical address of the data block is the cache address of the data carried by the write request.
  • the CPU of the host 1 receives the write data request sent by the VM 11, and allocates a data block for the write request from the cache resource pool 301a to buffer the data carried by the write request.
  • the data carried by the write request is stored to the mounted volume V1 of the VM 11, and a write success response is sent to the VM 11.
  • the attribute of the Vcache 11 further includes a write back mode
  • the CPU of the host 1 receives the write data request sent by the VM 11, and allocates a data block for the write request from the cache resource pool 301a to buffer the data carried by the write request.
  • VM 11 returns a write success response.
  • the data block is allocated from the cache resource pool 301a for the write request to buffer the data carried by the write request, specifically, the data block is allocated from the cache resource pool 301a for the write request, and the write request is carried.
  • the data is stored in the physical block of the local cache device corresponding to the allocated data block.
  • the read data request When the VM 11 sends a read data request, the read data request carries the identifier of the VM 11, and the CPU queries the correspondence between the identifier of the VM 11 and the identifier of the Vcache 11, and determines the virtual cache device Vcache 11 of the VM 11 (another implementation, The read data request carries the identifier of the volume V1 mounted by the VM 11. The CPU queries the correspondence between the identifier of the V1 and the identifier of the Vcache 11, determines the virtual cache device Vcache 11 of the VM 11, and queries whether the Vcache 11 caches the data to be read. When the data block allocated by the Vcache 11 buffers the data to be read, the Vcache 11 returns the data to be read to the VM 11.
  • the CPU queries whether the logical address of the data block and the address of the data to be read are in the Vcache11.
  • the mapping relationship of the address in V1 indicates that the data to be read is buffered in the data block in Vcache11.
  • the data to be read is read from the volume V1 mounted by the VM 11, and the data block is allocated from the cache resource pool 301a to the Vcache 11 according to the attribute of the Vcache 11, and the data block is to be read.
  • the data to be read Cached in the physical block of the local cache device corresponding to the data block, the mapping relationship between the logical address of the data block and the address (the address in V1) of the data to be read is established.
  • the CPU reads the data to be read from the logical address of the data block in the Vcache 11 of the VM 11.
  • the Vcache 11 and the allocation are also included.
  • the correspondence between the data blocks indicates that the allocated data block belongs to Vcache 11, or indicates that the allocated data block is used to provide buffer space for Vcache 11.
  • the Vcache 11 attribute includes a deduplication function, and the local cache device where the data block allocated by the VM 11 or the data to be written is cached is required to support the deduplication function.
  • the Vcache 11 attribute includes a cache elimination algorithm, and the data block allocated when the VM 11 reads data or the written data is cached to support a specific cache elimination algorithm, and details are not described herein.
  • Vcache 11 can provide cache for multiple VMs of host 1, and the Vcaches required by multiple VMs have the same attributes, and the cache capacity of Vcache 11 satisfies the cache requirements of multiple VMs.
  • the metadata of Vcache 11 records the correspondence between multiple VMs and Vcache 11, or records the correspondence between Vcache 11 and multiple VM mounted volumes, and multiple VMs access Vcache 11 Refer to the implementation of the previous VM 11, and I will not go into details here.
  • Vcache 11 can provide caching for specific applications running in VM 11, instead of providing caching for VM 11.
  • Vcache 11 metadata records the specific applications in VM 11.
  • the CPU of the host 1 determines the Vcache through the identifier of the specific application in the VM 11, and the implementation of the latter can refer to the implementation of the previous VM 11, and the details are not described herein. .
  • the embodiment of the invention improves the utilization of the local cache device of the host, reduces the waste of the cache space of the local cache device, and improves the flexibility of the configuration of the virtual cache device.
  • Vcache 11 as a cache for VM 11 is taken as an example.
  • VM 11 migrates to host 2, which becomes the target virtual machine VM 22, an implementation,
  • Vcache 11 still provides caching for VM 22, and the metadata of Vcache11 includes the correspondence between Vcache 11 and the volume mounted by VM 22 (by recording the identifier of Vcache 11 and the identity of the volume mounted by VM 22), or VM. Correspondence between 22 and Vcache 11 (by recording the correspondence between the identifier of VM 22 and the identifier of Vcache 11).
  • the VM 22 is still V1, and the VM 22 on the host 2 can access the Vcache 11 on the host 1 through the network.
  • the scenario in which Vcache 11 provides caching for a specific application in VM 11 running on host 1 is similar to that, and will not be described again.
  • Vcache 21 has the same properties as Vcache 11.
  • Vcache 11 For the properties of Vcache 11, please refer to the previous description.
  • VM 11 migrates to the host 2
  • another implementation is to allocate a virtual cache device having the same attributes as the Vcache 11 that has been created in the host 2 to the VM 22, which is also capable of providing the Vcache 11 for the VM 22. Cache capacity.
  • VM 22 shares the virtual cache device with other virtual machines.
  • the local cache device of multiple hosts forms a cache resource pool.
  • the local cache device 1 of the host 1 and the local cache device 2 of the host 2 are logically configured.
  • a data block divided into fixed sizes, such as 4 KB, constitutes a cache resource pool 601.
  • the size of the data blocks in the same cache resource pool 601 may be the same or different.
  • the data block is mapped to the physical block of the local cache device, that is, the logical address of the data block is mapped to the physical address of the physical block of the local cache device, and the data stored in the data block is actually stored in the local cache corresponding to the data block. In the physical block of the device. As shown in FIG.
  • the host 1 can create a virtual cache device Vcache 611 in the cache resource pool 601 according to the requirements of the VM 11 running on the host 1.
  • Host 1 creates a virtual cache device Vcache 611 and records the metadata of Vcache 611.
  • Vcache 611 metadata The correspondence between the Vcache 611 and the V11 may be included in the Vcache 611, the size of the Vcache 611, and the VVM 611 and the VM 11 mounted volume V1. Therefore, the Vcache 611, the cache resource pool 601 and the local cache device of the host 1 and the host 2 form a mapping relationship as shown in FIG. 8, and the data cached in the data block of the Vcache 611 is finally stored in the local cache device 1 and the local cache device.
  • the data buffered in the data block of Vcache 611 may also be stored only in the physical block of the local cache device 1 or the local cache device 2, or uniformly stored in the local cache device. 1 and in the physical block of the local cache device 2.
  • the Vcache 611 when the Vcache 611 is created, it is not necessary to allocate the data blocks in the cache resource pool 601 when the Vcache 611 is created. Instead, the Vcache 611 is allocated a data block according to the Vcache 611 usage state, thereby improving the utilization of the local cache device 1 and the local cache device 2, and reducing the waste of the cache space of the local cache device.
  • the process for the host to create a virtual cache device is as shown in FIG. 8, and includes:
  • the host receives a virtual cache device Vcache creation request.
  • the Vcache creation request carries the attributes of the Vcache.
  • the Vcache attributes include the size of the Vcache, the size of the block in the Vcache, etc., and may also include the performance requirements of the Vcache, such as input and output per second (IOPS), and Vcache write. Mode (such as Write back or Write Through).
  • the attributes of Vcache can also include deduplication, specific IO elimination algorithm requirements, and so on. For the size, write mode, and deduplication function of the data block block included in the Vcache attribute, refer to the description of the previous embodiment. At the same time, other information included in the attributes of the Vcache described herein is also applicable to the previous embodiment.
  • the host creates a Vcache based on the Vcache attribute carried by the Vcache creation request, and does not allocate a data block from the local cache device for the Vcache.
  • the host creates a Vcache based on the Vcache attribute and records the Vcache metadata.
  • the metadata includes the correspondence between the Vcache and the volume mounted by the virtual machine running by the host (by recording the Vcache identifier and the volume identifier), the Vcache identifier, and the size of the Vcache.
  • the mapping between the Vcache and the virtual machine is recorded.
  • the specific implementation can be implemented by recording the Vcache identifier and the VM identifier.
  • the Vcache is not allocated data blocks from the local cache device until the Vcache has no cached data.
  • the virtual cache creation method provided by the embodiment of the invention improves the flexibility of virtual cache creation, and can set the attributes of the virtual cache according to requirements, and does not need to allocate data blocks before creating the virtual cache without actually buffering the data, thereby improving the The utilization of the virtual cache reduces the waste of the cache space of the local cache device.
  • a virtualization management platform is generally required to implement management of a virtual machine. Specifically, the command host creates a VM, the VM migrates between different hosts, and VM resource allocation.
  • the virtualization management platform is usually implemented by a server running virtualization management software.
  • VMs running on different hosts are used to jointly run distributed applications, and therefore, Vcaches having the same attributes are required.
  • hosts running these VMs respectively create Vcaches of the same attribute to provide caching for VMs running distributed applications.
  • a Vcache can be created to provide a cache for the VMs.
  • the host cluster includes host 1, host 2 to host N, and the hosts communicate with each other, where N is an integer.
  • a host also known as a server or node.
  • the local cache device is included in the hardware of each host.
  • the local cache device includes a local cache device built in the host or a local cache device connected to the host, and is hereinafter referred to as a local cache device of the host.
  • the permanent storage device required by the host may also be located outside the host or built in the host, and the present invention is implemented The example is not limited.
  • the local cache device may be an SSD, a PCIe flash card, or other device as a cache device, which is not limited in this embodiment of the present invention.
  • the hardware of the host further includes a CPU.
  • the host includes other hardware, such as a main memory, in addition to the hardware devices listed in FIG.
  • a Vcache is created for the application running on the host, and the mapping between the application and the Vcache (the correspondence between the identifier of the application and the identifier of the virtual cache device) or the corresponding volume of the application and the Vcache is established.
  • the relationship between the identifier of the mounted volume and the identifier of the Vcache is as follows: For the method of accessing the Vcache, refer to the method of accessing the Vcache by the VM shown in Figure 2-8, and no further details are provided here.
  • step 1001 the CPU of the target host (host 1 in FIG. 2-9) receives and runs on the target host.
  • the data request sent by the application may specifically be a write data request or a read data request.
  • step 1002 The CPU of the target host determines the virtual cache of the application according to the data request. Specifically, the target host saves the identifier of the application and the corresponding relationship between the identifier of the virtual cache device and the identifier of the virtual cache device.
  • the CPU of the target host determines the virtual cache device according to the correspondence between the identifier of the application and the identifier of the virtual cache device, or the target host stores Corresponding relationship between the identifier of the volume mounted by the application and the identifier of the virtual cache device (such as the metadata of the virtual cache or the identifier of the volume mounted by the application in the specific area of the target host and the identifier of the virtual cache device
  • the CPU of the target host determines the virtual cache device according to the identifier of the volume of the application mounted by the application and the identifier of the virtual cache device.
  • Step 1003 The CPU of the target host allocates the target data block from the plurality of data blocks of the cache resource pool (such as 301a) according to the attributes of the virtual cache device to cache the data of the data request operation.
  • the data request is a write data request
  • the CPU of the target host allocates the target data block from the plurality of data blocks of the cache resource pool according to the attribute of the virtual cache device to buffer the write data request.
  • data is a read data request
  • the CPU of the target host reads the data to be read from the volume mounted by the application.
  • the CPU of the target host allocates a target data block from a plurality of data blocks of the cache resource pool according to the attributes of the virtual cache device to cache the data to be read.
  • Step 1004 The CPU of the target host establishes a correspondence between the target data block and the virtual cache device.
  • the virtual cache device management method provided by the embodiment of the present invention can dynamically allocate data blocks according to data requests of the application, improve the utilization of the virtual cache device, and reduce the waste of the cache space of the virtual cache device.
  • the Vcache is allocated to the application on the host (including the virtual machine, the application in the virtual machine, or the directly running application on the host), and the specific implementation of the access cache when the application accesses the data may refer to the cache access principle. The present invention will not be described again herein.
  • the attribute information of the Vcache can be recorded in the metadata of the Vcache after the Vcache is created.
  • the attribute of the Vcache has the same meaning as the representation of the metadata according to the Vcache, for example, the CPU. Allocating a target data block from a plurality of data blocks of the cache resource pool according to the attribute of the Vcache to buffer the data operated by the data request and the CPU allocates the target data from the plurality of data blocks of the cache resource pool according to the metadata of the Vcache.
  • the data that the block operates with the cached data request has the same meaning, but the representation is different.
  • the write data request and the read data request are collectively referred to as a data request, and the data carried by the write data request and the data read by the read data request are collectively referred to as data operated by the data operation request.
  • the data carried by the write data request is also referred to as the data to be written or the data in the write data request.
  • an embodiment of the present invention provides a target host 110 for running an application, which is applied to a host cluster, where the target host 110 includes a local cache device, and the host cluster includes the target host 110.
  • a cache resource pool composed of a local cache device of one or more hosts, the cache resource pool includes a plurality of data blocks, each data block is mapped to one physical block of the local cache device; the target host 110 further includes receiving A unit 1101, a determining unit 1102, an allocating unit 1103, and an establishing unit 1104.
  • receiving The unit 1101 is configured to receive a data request sent by the application, and the determining unit 1102 is configured to determine, according to the data request, a virtual cache device of the application, and an allocating unit 1103, configured to use the cache according to the attribute of the virtual cache device.
  • the target data block is allocated in the plurality of data blocks of the resource pool to buffer the data operated by the data request; the establishing unit 1104 is configured to establish a correspondence between the target data block and the virtual cache device.
  • the virtual cache device management method provided by the embodiment of the present invention can dynamically allocate data blocks according to data requests of the application, improve the utilization of the virtual cache device, and reduce the waste of the cache space of the virtual cache device.
  • the target host 110 shown in FIG. 11 further includes a storage unit, configured to store a correspondence between the identifier of the application and the identifier of the virtual cache device, where the determining unit 1102 is specifically configured to: The corresponding relationship between the identifier of the application and the identifier of the virtual cache device determines the virtual cache device.
  • the target host 110 shown in FIG. 11 further includes a storage unit, configured to store a correspondence between an identifier of the volume mounted by the application and an identifier of the virtual cache device, where the determining unit 1102 is specific. And determining, by using the identifier of the volume that is mounted by the application and the identifier of the virtual cache device, the virtual cache device.
  • the allocating unit 1103 is specifically configured to: when the data request is a write data request, according to an attribute of the virtual cache device, from the cache resource pool.
  • the target data block is allocated in the plurality of data blocks to buffer data carried by the write data request.
  • the allocating unit 1103 is configured to: read, when the data request is a read data request, read from the volume mounted by the application. Data; allocating the target data block from the plurality of data blocks of the cache resource pool according to an attribute of the virtual cache device to cache the data to be read.
  • the establishing unit 1104 is further configured to establish a physical location of the volume of the volume carried by the data request and the target data block.
  • a mapping relationship is used to represent data in the virtual cache device in which the data request is cached.
  • the target host 110 is implemented by the target host 110 having the above-mentioned unit, and the unit can be loaded into the main memory of the target host 110, and the CPU of the target host 110 executes the instructions in the main memory.
  • the functions in the corresponding embodiments of the present invention are implemented; in another implementation, the units included in the target host 110 may be implemented by hardware or by a combination of software and hardware.
  • the above units are also referred to as structural units.
  • the virtualization management platform instructs the host to create a Vcache for a VM or a particular application.
  • the first software component runs on each host to form a local cache device into a resource pool, or a local cache device on several hosts to form a resource pool.
  • the second software component is run on the host to provide a Vcache for the application on the host.
  • the specific process is as described in the previous embodiment. That is, the CPU of the host executes computer instructions in the main memory to implement the technical solutions described in the previous embodiments. Accordingly, embodiments of the present invention also provide non-transitory computer readable storage media and computer program products to implement embodiments of the present invention.
  • the disclosed apparatus and method may be implemented in other manners.
  • the division of the units described in the device embodiments described above is only one logical function division, and may be further divided in actual implementation, for example, multiple units or components may be combined or may be integrated into another system, or Some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the unit described as a separate component may or may not be physically Separately, the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种主机集群中缓存管理方法，所述主机集群包括目标主机在内的一台或多台主机的本地缓存设备组成的缓存资源池(301a)，所述缓存资源池(301a)包含多个数据块，每一个数据块映射到所述本地缓存设备的一个物理块，所述目标主机运行有应用；目标主机CPU根据应用的数据请求从缓存资源池(301a)的多个数据块中分配目标数据块用于缓存数据操作请求所操作的数据。

Description

一种主机集群中缓存管理方法及主机 技术领域
本发明涉及信息技术领域，尤其涉及一种主机集群中缓存管理方法及主机。
背景技术
在过去的几十年中，信息技术迅猛发展，计算机的中央处理单元(Centric Processing Unit,CPU)的计算能力依然在按照摩尔定律在发展，但是存储的性能并没有显著的提高，造成存储的性能和计算能力发展的不匹配。为改善这一问题，引入缓存技术。
如图1所示，提供了一种在虚拟化环境下基于缓存技术改善存储性能的方案。虚拟机(Virtual Machine,VM)1、VM 2、VM 3和VM 4运行在图1所示的主机(或称为服务器)上。主机上的每台VM有独立的操作系统。VM 1上运行应用1，VM 2运行应用2，VM 3运行应用3，VM 4运行应用4。由于缓存设备的输入输出(Input/Output，IO)性能要优于外部存储设备的IO性能，因此，为提高VM的IO性能，主机将缓存设备划分为固定的分区作为虚拟缓存设备分配给VM使用。例如，将分区1作为虚拟缓存设备分配给VM 1，分区2作为虚拟缓存设备分配给VM 2，分区3作为虚拟缓存设备分配给VM 3，分区4作为虚拟缓存设备分配给VM 4。
图1所示的方案中，虽然可以通过为VM分配虚拟缓存设备，从而提升VM的IO性能，但是一旦为VM分配虚拟缓存设备，即实际分配了虚拟缓存设备的容量，在VM运行过程中虚拟缓存设备容量即不可调整，造成虚拟缓存设备的缓存空间的浪费。
发明内容
本发明提供了一种主机集群中缓存管理方法，所述主机集群包括目标主机在内的一台或多台主机的本地缓存设备组成的缓存资源池， 所述缓存资源池包含多个数据块，每一个数据块映射到所述本地缓存设备的一个物理块，所述目标主机运行有应用；所述方法包括：
所述目标主机的中央处理单元CPU接收所述应用发送的数据请求；
所述目标主机的CPU根据所述数据请求确定所述应用的虚拟缓存设备；
所述目标主机的CPU根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配目标数据块以缓存所述数据请求所操作的数据；
所述目标主机的CPU建立所述目标数据块与所述虚拟缓存设备的对应关系。本发明实施提供的虚拟缓存设备管理方法，可根据应用的数据请求动态分配数据块，提高了虚拟缓存设备的利用率，减少了虚拟缓存设备的缓存空间的浪费。可选的，所述目标主机的CPU建立所述数据请求携带的卷的地址与所述目标数据块的逻辑地址的映射关系，以用于表示虚拟缓存设备中所述目标数据块缓存了所述数据请求所操作的数据。
可选的，所述目标主机存储有所述应用的标识与所述虚拟缓存设备的标识的对应关系，所述目标主机的CPU根据所述数据请求确定所述应用的虚拟缓存设备具体包括：所述目标主机的CPU根据所述应用的标识与所述虚拟缓存设备的标识的对应关系确定所述虚拟缓存设备。
可选的，所述目标主机存储有所述应用挂载的卷的标识与所述虚拟缓存设备的标识的对应关系，所述目标主机的CPU根据所述数据请求确定所述应用的虚拟缓存设备具体包括：所述目标主机的CPU根据所述应用挂载的卷的标识与所述虚拟缓存设备的标识对应关系确定所述虚拟缓存设备。
可选的，当所述数据请求为写数据请求时，所述目标主机的CPU根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配目标数据块以缓存所述数据请求所操作的数据具体为：所述目标主机的CPU根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配所述目标数据块以缓存所述写数据请求携带的数据。
可选的，当所述数据请求为读取数据请求时，所述目标主机的CPU根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配目标数据块以缓存所述数据请求所操作的数据具体为：所述目标主机的CPU从所述应用挂载的卷中读取待读取的数据；所述目标主机的CPU根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配所述目标数据块以缓存所述取待读取的数据。
可选的，所述缓存资源池中数据块的大小不同，以提供不同粒度的缓存。
可选的，具有相同大小的数据块和物理块建立映射。
可选的，应用是虚拟机、虚拟机中运行的应用或主机上运行的应用。
相应的，本发明还提供了主机用于执行上述方案。相应的，本发明还提供了非易失性计算机可读存储介质和计算机程序产品以实现上述方案。
附图说明
图1为现有技术中应用场景图；
图2为本发明实施例应用场景图；
图3为主机1本地缓存资源池示意图；
图4为虚拟缓存设备与缓存资源池关系示意图；
图5为虚拟缓存设备与缓存资源池关系示意图；
图6为虚拟缓存设备与缓存资源池关系示意图；
图7为虚拟缓存设备与缓存资源池关系示意图；
图8为创建虚拟缓存设备的流程图；
图9为本发明实施例应用场景图；
图10为本发明实施例为主机集群中缓存管理方法示意图；
图11为本发明实施例目标主机的结构示意图。
具体实施例
如图2所示，为本发明实施例的一个应用场景。图2中，主机集群中包括主机1、主机2至主机N，主机之间相互通信，其中N为整数。每个主机上运行多台VM，在本发明实施例中，分别用VM 1、VM 2至VM N表示。本发明中主机的数量以及VM的数量只是示例性的，可根据实际需要确定主机的数量和VM的数量。主机，也称为服务器或者节点，通过使用虚拟化软件运行VM。虚拟化软件可以为
Figure PCTCN2015100190-appb-000001
(华为技术有限公司计算虚拟化软件)。每台主机的硬件包含本地缓存设备。本发明实施例中，主机的硬件包含本地缓存设备是指本地缓存设备内置于主机或者本地缓存设备与主机连接，以下称为主机的本地缓存设备。而主机需要的永久存储设备也可位于主机外部或内置于主机，对此本发明实施例不作限定。本地缓存设备可以是固态存储硬件(Solid State Disk,SSD)、快速外围组件互连(Peripheral Component Interconnect Express,PCIe)闪存卡或其他设备作为缓存设备，本发明实施例对此不作限定。图2所示的实施例中，主机的硬件还包含CPU，本领域技术人员可知，主机除图2中所示的硬件设备外，还包括其他硬件，如主存储器(Main Memory)等，在此不再列举。
图2所示的实施例中，每台主机将本地缓存设备进行池化，所谓进行池化是将本地缓存设备逻辑上划分成固定大小的数据块，由数据 块组成缓存资源池，数据块对应本地缓存设备上固定大小的物理块，每一个数据块会映射到本地缓存设备上一个物理块，即数据块的逻辑地块映射到本地缓存设备上物理块的物理地址，或者数据块与本地缓存设备上的物理块对应。因此，本发明实施例中数据块是逻辑上的概念。可选的，本地缓存设备可以有不同大小的物理块，相应的，数据块也可以有不同的大小，具有相同大小的数据块和物理块建立映射。在图2中，各主机本地缓存设备可以组成缓存资源池。具体地，每个主机本地缓存设备可以组成一个缓存资源池，或者多个主机的本地缓存设备组成一个缓存资源池，或者所有主机的本地缓存设备组成一个缓存资源池。
如图3所示，主机1的本地缓存设备1形成一个缓存资源池301a，在该缓存资源池中本地缓存设备1被逻辑上划分成固定大小的数据块，如4KB。如前所述，一个4KB大小的数据块映射到本地缓存设备1中一个4KB大小的物理块，即一个4KB大小的数据块的逻辑地址映射到本地缓存设备1中一个4KB大小的物理块。主机2至主机N的本地缓存设备也分别组成类似的缓存资源池。在另一种实施场景下，每台主机的本地缓存设备形成的缓存资源池中的数据块大小可根据实际需求，大小均相同，也可不相同。因为缓存资源池301a的存储资源是以数据块的形式呈现的，因此，缓存资源池301a也称为块设备。主机1根据VM的需求，如图4所示，在缓存资源池中创建不同的虚拟缓存设备Vcache 11至Vcache 1M，其中，M为整数。主机1创建虚拟缓存设备Vcache 11至Vcache 1M，并且分别记录Vcache 11至Vcache 1M的元数据。以Vcache 11为例，Vcache 11为主机1上的VM 11提供缓存，Vcache 11的元数据包括Vcache 11与VM 11挂载的卷V1的对应关系(例如，通过记录Vcache 11的标识与V1的标识的对应关系实现)、Vcache 11的标识以及Vcache 11的大小等，还可以包括Vcache 11与VM 11的对应关系 (具体可通过记录Vcache 11的标识与VM 11的标识的对应关系实现)。在另一种实现方式中，Vcache 11与VM 11挂载的卷V1的对应关系(例如，通过记录Vcache 11的标识与V1的标识的对应关系实现)和Vcache 11与VM 11的对应关系(具体可通过记录Vcache 11的标识与VM  11的标识的对应关系实现)也可不记录在Vcache 11的元数据中，而存储在主机1的特定区域以供VM 11访问Vcache11时查询。因此，Vcache 11、缓存资源池301a与主机1的本地缓存设备1形成如图5所示的关系，Vcache 11中缓存的数据存储在Vcache 11的数据块对应的本地缓存设备1的物理块中。本发明实施例中，创建Vcache，并不需要为Vcache分配数据块。当需要缓存数据时，才建立Vcache和数据块的对应关系，也即为Vcache缓存的数据分配数据块，从而提高了本地缓存设备的利用率，减少本地缓存设备的缓存空间的浪费。
例如，为Vcache 11创建的缓存空间大小为4GB，则Vcache 11用于缓存VM 11写入V1的数据，或者缓存VM 11从V1读取的数据。具体地，以缓存VM 11写入V1的数据为例，主机1的CPU接收VM 11发送的写数据请求，写数据请求携带VM 11的标识，CPU查询VM 11的标识与Vcache 11的标识的对应关系，确定VM 11的虚拟缓存设备Vcache 11(另一种实现，写数据请求携带VM 11挂载的卷V1的标识，CPU查询V1的标识与Vcache 11的标识的对应关系，确定VM 11的虚拟缓存设备Vcache 11)，根据Vcache 11的属性从缓存资源池301a中为该写请求分配数据块以缓存写数据请求携带的数据。例如，当Vcache 11的属性包括Vcache 11中数据块的大小，则根据Vcache 11的属性从缓存资源池301a中为该写请求分配数据块包括：根据Vcache 11的中数据块的大小从缓存资源池301a中选择相同大小的数据块，将该写请求携带的数据存储到该数据块对应的本地缓存设备的物理块中(另一种相同含义的表述为将该写请求携带的数据存储到该数据块对应的本地缓 存设备的物理块中)。CPU建立该写请求携带的待写入地址(V1中的地址)与该数据块的逻辑地址的对应关系，以表示该数据块的逻辑地址为该写请求携带的数据的缓存地址。
当Vcache 11的属性还包括透写(write through)模式时，主机1的CPU接收VM 11发送的写数据请求，从缓存资源池301a中为该写请求分配数据块以缓存写请求携带的数据，将写请求携带的数据存储到VM 11的挂载的卷V1，向VM 11发送写成功响应。当Vcache 11的属性还包括回写(write back)模式，主机1的CPU接收VM 11发送的写数据请求，从缓存资源池301a中为该写请求分配数据块以缓存写请求携带的数据，向VM 11返回写成功响应。
无论是透写还是回写模式，从缓存资源池301a中为该写请求分配数据块以缓存写请求携带的数据具体包括，从缓存资源池301a中为写请求分配数据块，将写请求携带的数据存储到分配的数据块对应的本地缓存设备的物理块中。
当VM 11发送读取数据请求时，读取数据请求携带VM 11的标识，CPU查询VM 11的标识与Vcache 11的标识的对应关系，确定VM 11的虚拟缓存设备Vcache 11(另一种实现，读取数据请求携带VM 11挂载的卷V1的标识，CPU查询V1的标识与Vcache 11的标识的对应关系，确定VM 11的虚拟缓存设备Vcache 11)，查询Vcache 11是否缓存待读取的数据，当Vcache 11分配的数据块缓存了待读取数据时，则Vcache 11向VM 11返回待读取的数据，具体的，CPU查询Vcache11中是否有数据块的逻辑地址与待读取数据的地址(V1中的地址)的映射关系，当存在该映射关系，则表明Vcache11中该数据块中缓存了待读取数据。当Vcache 11没有缓存待读取数据时，从VM 11挂载的卷V1读取该待读取数据，并且根据Vcache 11的属性从缓存资源池301a中为Vcache 11分配数据块，并将该待读取数据至Vcache 11，即将该待读取的数据 缓存在该数据块对应的本地缓存设备的物理块中，建立该数据块的逻辑地址与与待读取数据的地址(V1中的地址)的映射关系。CPU从VM 11的Vcache 11中该数据块的逻辑地址中读取该待读取的数据，上述几种情况，为Vcache 11从缓存资源池301a中分配数据块后，还包括建立Vcache 11与分配的数据块之间的对应关系，表明该分配的数据块属于Vcache 11，或者表明该分配的数据块用于为Vcache 11提供缓存空间。可选的，Vcache 11的属性包括重复数据删除功能，则为缓存VM 11读取的数据或写入的数据时分配的数据块所在的本地缓存设备要支持重复数据删除功能。可选的，Vcache 11的属性包括缓存淘汰算法，则为缓存VM 11读取的数据或写入的数据时分配的数据块要支持特定的缓存淘汰算法，在此不再赘述。
另一种实现方式，Vcache 11可以为主机1的多个VM提供缓存，则多个VM所需要的Vcache具有相同的属性，Vcache 11的缓存容量满足多个VM的缓存需求。在创建Vcache 11时，Vcache 11的元数据中记录的是多个VM与Vcache 11的对应关系，或者，记录Vcache 11与多个VM挂载的卷的对应关系，而多个VM访问Vcache 11可参考前面VM 11的实现方式，在此不赘述。
另一种实现方式，Vcache 11可以为VM 11中运行的特定应用提供缓存，而不是为VM 11提供缓存，则在创建Vcache 11时，Vcache 11的元数据中记录的是VM 11中特定应用与Vcache 11的对应关系，VM 11中特定的应用访问Vcache 11时，主机1的CPU通过VM 11中特定应用的标识确定Vcache 11，后面的实现方式可参考前面VM 11的实现方式，在此不赘述。
本发明实施例提高了主机本地缓存设备的利用率，减少了本地缓存设备的缓存空间的浪费，同时提高了虚拟缓存设备配置的灵活性。
图5所示的实施方式中，以Vcache 11为VM 11提供缓存为例，当 VM 11迁移至主机2，即成为目标虚拟机VM 22，一种实现方式，
Vcache 11仍然为VM 22提供缓存，则Vcache11的元数据包括Vcache 11与VM 22挂载的卷的对应关系(通过记录Vcache 11的标识与VM 22挂载的卷的标识对应关系实现)，或者VM 22与Vcache 11的对应关系(通过记录VM 22的标识与Vcache 11的标识的对应关系实现)。其中，VM 22挂载的卷仍然为V1，则主机2上的VM 22可通过网络访问主机1上的Vcache 11。同理，Vcache 11为主机1上运行的VM 11中的特定应用提供缓存的场景与此类似，不再赘述。另一种实现方式，当VM 11迁移至主机2，即目标虚拟机VM 22，主机2为VM 22创建Vcache 21，为VM 22分配Vcache 21，具体方式可参考前面创建Vcache 11及为VM 11分配Vcache 11的方式，在此不再赘述。其中，Vcache 21与Vcache 11具有相同的属性，关于Vcache 11的属性，请参见前面的描述。当VM 11迁移至主机2，另一种实现方式为将主机2中已经创建的与Vcache 11具有相同属性的虚拟缓存设备分配给VM 22，该虚拟缓存设备还要能够为VM 22提供Vcache 11的缓存容量，该种实现方式下，VM 22与其他虚拟机共享该虚拟缓存设备。
在本发明实施例图2所示的实现场景中，多个主机的本地缓存设备形成一个缓存资源池，如图6所示，主机1的本地缓存设备1和主机2的本地缓存设备2被逻辑上划分成固定大小的数据块，如4KB，组成一个缓存资源池601。同一缓存资源池601中的数据块的大小可相同，也可不相同。数据块与本地缓存设备的物理块建立映射关系，即数据块的逻辑地址与本地缓存设备的物理块的物理地址建立映射关系，数据块中存储的数据实际要存储到该数据块对应的本地缓存设备的物理块中。如图7所示，主机1可以根据主机1上运行的VM 11的需求在缓存资源池601中创建虚拟缓存设备Vcache 611。主机1创建虚拟缓存设备Vcache 611，并且记录Vcache 611的元数据。Vcache 611的元数据 包括Vcache 611的标识、Vcache 611的大小、Vcache 611与VM 11挂载的卷V1的对应关系，还可以包括Vcache 611与VM 11的对应关系等。因此，Vcache 611、缓存资源池601与主机1和主机2的本地缓存设备形成如图8所示的映射关系，Vcache 611的数据块中缓存的数据最终将存储在本地缓存设备1和本地缓存设备2的物理块中，当然本领域技术人员可知，Vcache 611的数据块中缓存的数据最终也可只在存储在本地缓存设备1或本地缓存设备2的物理块中，或者均匀存储在本地缓存设备1和本地缓存设备2的物理块中。本发明实施例中，创建Vcache 611时，并不需要在创建Vcache 611时就分配缓存资源池601中的数据块。而是根据Vcache 611使用状态，为Vcache 611分配数据块，从而提高了本地缓存设备1和本地缓存设备2的利用率，减少了本地缓存设备的缓存空间的浪费。
根据图2至图7所示的实施例，主机创建虚拟缓存设备的流程如图8所示，包括：
801：主机接收虚拟缓存设备Vcache创建请求；
其中，Vcache创建请求携带Vcache的属性，Vcache的属性包括Vcache的大小、Vcache中块的大小等，还可以包括Vcache的性能需求，如每秒钟输入输出(InputOutput per Second,IOPS)，Vcache的写模式(如Write back或者Write Through)等。Vcache的属性，还可以包括重复数据删除功能、特定的IO淘汰算法需求等。关于Vcache属性包含的数据块块的大小、写模式以及重复数据删除功能，可参见前面实施例描述。同时，此处描述的Vcache的属性包括的其他信息也可应用于前面实施例。
802：主机根据Vcache创建请求携带的Vcache属性创建Vcache，而不为Vcache从本地缓存设备分配数据块。
主机根据Vcache属性创建Vcache，并记录Vcache的元数据。其中， 元数据包括Vcache与主机运行的虚拟机挂载的卷的对应关系(通过记录Vcache的标识与卷的标识实现)、Vcache的标识、Vcache的大小等。创建Vcache后，还包括记录Vcache与虚拟机的对应关系，具体实现可通过记录Vcache标识与虚拟机标识实现。在Vcache没有缓存数据之前，不为Vcache从本地缓存设备中分配数据块。本发明实施例提供的虚拟缓存创建方法，提高了虚拟缓存创建的灵活性，可根据需要设定虚拟缓存的属性，同时在创建虚拟缓存而没有实际缓存数据之前，不需要分配数据块，提高了虚拟缓存的利用率，减少本地缓存设备的缓存空间的浪费。
在图2至图7所示的实施例场景下，通常需要虚拟化管理平台来实现虚拟机的管理，具体地，包括指令主机创建VM，VM在不同主机之间迁移、VM资源分配等。虚拟化管理平台通常由运行虚拟化管理软件的服务器来实现。
在图2至图5所示的场景中包括不同主机上运行的VM共同运行分布式应用，因此，需要具有相同属性的Vcache。一种实施方式，运行这些VM的主机分别创建相同属性的Vcache为运行分布式应用的VM提供缓存。具体实现方式可参考前面实施例描述，在此不再赘述。另一种实现方式，可创建一个Vcache为这些VM提供缓存，具体实现方式可参考前面实施例关于一个为多个VM提供缓存的描述，在此不再赘述。
如图9所示，为本发明另一实施场景。主机集群中包括主机1、主机2至主机N，主机之间相互通信，其中N为整数。主机，也称为服务器或者节点。每台主机的硬件中包含本地缓存设备。本发明实施例中，主机的硬件中包含本地缓存设备是指本地缓存设备内置于主机或者本地缓存设备与主机连接，以下称为主机的本地缓存设备。而主机需要的永久存储设备也可位于主机外部或内置于主机，对此本发明实施 例不作限定。本地缓存设备可以是SSD、PCIe闪存卡或其他设备作为缓存设备，本发明实施例对此不作限定。图9所示的实施例中，主机的硬件还包含CPU，本领域技术人员可知，主机除图9中所列的硬件设备外，还包括其他硬件，如主存储器，在此不再列举。在图9所示的实施场景中，为主机运行的应用创建Vcache，建立应用与Vcache的对应关系(应用的标识与该虚拟缓存设备的标识的对应关系)或应用挂载的卷与Vcache的对应关系(应用挂载的卷的标识与该Vcache的标识的对应关系)，应用访问Vcache的方式可参考图2-8所示的VM访问Vcache的方式，在此不再赘述。
根据图2-9所示的实施方式，如图10所示，在主机集群中缓存管理方法中，步骤1001：目标主机(如图2-9中的主机1)的CPU接收运行在目标主机上的应用发送的数据请求，具体地可以为写数据请求或读取数据请求。步骤1002：目标主机的CPU根据数据请求确定该应用的虚拟缓存，具体地，目标主机保存该应用的标识与该虚拟缓存设备的标该应用的标识与该虚拟缓存设备的标识的对应关系识的对应关系(如在虚拟缓存的元数据或在目标主机特定区域中记录)，目标主机的CPU根据该应用的标识与该虚拟缓存设备的标识的对应关系确定该虚拟缓存设备，或者目标主机存储有该应用挂载的卷的标识与该虚拟缓存设备的标识的对应关系(如在虚拟缓存的元数据或在目标主机特定区域中记录该应用挂载的卷的标识与该虚拟缓存设备的标识的对应关系)，则目标主机的CPU根据该应用挂载的卷的标识与该虚拟缓存设备的标识对应关系确定该虚拟缓存设备。步骤1003：目标主机的CPU根据该虚拟缓存设备的属性从缓存资源池(如301a)的多个数据块中分配目标数据块以缓存该数据请求操作的数据。当数据请求为写数据请求时，目标主机的CPU根据该虚拟缓存设备的属性从该缓存资源池的多个数据块中分配目标数据块以缓存写数据请求携带的 数据。当数据请求为读取数据请求时，目标主机的CPU从该应用挂载的卷中读取待读取的数据。目标主机的CPU根据该虚拟缓存设备的属性从该缓存资源池的多个数据块中分配目标数据块以缓存该取待读取的数据。步骤1004：目标主机的CPU建立目标数据块与虚拟缓存设备的对应关系。本发明实施例提供的虚拟缓存设备管理方法，可根据应用的数据请求动态分配数据块，提高了虚拟缓存设备的利用率，减少了虚拟缓存设备的缓存空间的浪费。
本发明实施例中，为主机上的应用(包括虚拟机、虚拟机中的应用或主机上的直接运行的应用)分配Vcache，应用在进行数据访问时访问缓存的具体实现，可以参考缓存访问原理，本发明在此不再赘述。
本发明实施例中，Vcache的属性信息在Vcache创建后，可以记录在Vcache的元数据中，则本发明实施例中，根据Vcache的属性与根据Vcache的元数据的表述具有相同的含义，例如CPU根据所述Vcache的属性从缓存资源池的多个数据块中分配目标数据块以缓存数据请求所操作的数据与CPU根据所述Vcache的元数据从缓存资源池的多个数据块中分配目标数据块以缓存数据请求所操作的数据具有相同的含义，只是表述的不同。本发明实施例中，写数据请求和读取数据请求统称为数据请求，写数据请求携带的数据和读取数据请求待读取的数据统称为数据操作请求所操作的数据。写数据请求携带的数据也称为待写入数据或写数据请求中的数据。
如图11所示，本发明实施例提供了一种运行应用的目标主机110，应用于主机集群，所述目标主机110包括本地缓存设备，所述主机集群包括所述目标主机110在内的一台或多台主机的本地缓存设备组成的缓存资源池，所述缓存资源池包含多个数据块，每一个数据块映射到所述本地缓存设备的一个物理块；所述目标主机110还包括接收单元1101、确定单元1102、分配单元1103和建立单元1104。其中，接收 单元1101，用于接收应用发送的数据请求；确定单元1102，用于根据所述数据请求确定所述应用的虚拟缓存设备；分配单元1103，用于根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配目标数据块以缓存所述数据请求所操作的数据；建立单元1104，用于建立所述目标数据块与所述虚拟缓存设备的对应关系。本发明实施例提供的虚拟缓存设备管理方法，可根据应用的数据请求动态分配数据块，提高了虚拟缓存设备的利用率，减少了虚拟缓存设备的缓存空间的浪费。
可选的，在图11所示的目标主机110中，还包括存储单元，用于存储所述应用的标识与所述虚拟缓存设备的标识的对应关系，所述确定单元1102具体用于：根据所述应用的标识与所述虚拟缓存设备的标识的对应关系确定所述虚拟缓存设备。
可选的，在图11所示的目标主机110中，还包括存储单元，用于存储所述应用挂载的卷的标识与所述虚拟缓存设备的标识的对应关系，所述确定单元1102具体用于：根据所述应用挂载的卷的标识与所述虚拟缓存设备的标识对应关系确定所述虚拟缓存设备。
可选的，在图11所示的目标主机110中，所述分配单元1103，具体用于当所述数据请求为写数据请求时，根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配所述目标数据块以缓存所述写数据请求携带的数据。
可选的，在图11所示的目标主机110中，所述分配单元1103，具体用于当所述数据请求为读取数据请求时，从所述应用挂载的卷中读取待读取的数据；根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配所述目标数据块以缓存所述待读取的数据。
可选的，在图11所示的目标主机110中，建立单元1104，还用于建立所述数据请求携带的卷的地块与所述目标数据块的逻辑地址的 映射关系，以用于表示虚拟缓存设备中缓存了所述数据请求所操作的数据。
如图11所示的目标主机110，一种实现方式为目标主机110安装有上述单元，上述单元可被加载到目标主机110的主存储器中，由目标主机110的CPU执行主存储器中的指令，实现本发明对应的实施例中的功能；另一种实现，目标主机110中包含的单元可以由硬件来实现，或者由软件与硬件组合实现。上述单元也称为结构单元。
本领域技术人员可知，上述各本发明实施例中，无论是VM，还是VM中运行特定应用，或者是图9所示的主机中的应用，实际上均是运行在主机上的一种应用。本发明实施例在图2-图7的所示的实施例中，虚拟化管理平台指令主机为VM或者特定应用创建Vcache。各主机上运行第一软件组件将本地缓存设备组成资源池，或将几个台主机上的本地缓存设备组成资源池。主机上运行第二软件组件，为主机上的应用提供Vcache，具体过程如前面实施例所述。也就是主机的CPU执行主存储器中的计算机指令以实现前面实施例中描述的技术方案。因此，本发明实施例，还提供了非易失性计算机可读存储介质和计算机程序产品以实现本发明实施例。
在本发明所提供的几个实施例中，应该理解到，所公开的装置、方法，可以通过其它的方式实现。例如，以上所描述的装置实施例所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上 分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。

Claims (15)

  1. 一种主机集群中缓存管理方法，其特征在于，所述主机集群包括目标主机在内的一台或多台主机的本地缓存设备组成的缓存资源池，所述缓存资源池包含多个数据块，每一个数据块映射到所述本地缓存设备的一个物理块，所述目标主机运行有应用；所述方法包括：
    所述目标主机的中央处理单元CPU接收所述应用发送的数据请求；
    所述目标主机的CPU根据所述数据请求确定所述应用的虚拟缓存设备；
    所述目标主机的CPU根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配目标数据块以缓存所述数据请求所操作的数据；
    所述目标主机的CPU建立所述目标数据块与所述虚拟缓存设备的对应关系。
  2. 根据权利要求1所述的方法，其特征在于，所述目标主机存储有所述应用的标识与所述虚拟缓存设备的标识的对应关系，所述目标主机的CPU根据所述数据请求确定所述应用的虚拟缓存设备具体包括：所述目标主机的CPU根据所述应用的标识与所述虚拟缓存设备的标识的对应关系确定所述虚拟缓存设备。
  3. 根据权利要求1所述的方法，其特征在于，所述目标主机存储有所述应用挂载的卷的标识与所述虚拟缓存设备的标识的对应关系，所述目标主机的CPU根据所述数据请求确定所述应用的虚拟缓存设备具体包括：所述目标主机的CPU根据所述应用挂载的卷的标识与所述虚拟缓存设备的标识对应关系确定所述虚拟缓存设备。
  4. 根据权利要求1所述的方法，其特征在于，当所述数据请求为 写数据请求时，所述目标主机的CPU根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配目标数据块以缓存所述数据请求所操作的数据具体为：所述目标主机的CPU根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配所述目标数据块以缓存所述写数据请求携带的数据。
  5. 根据权利要求1所述的方法，其特征在于，当所述数据请求为读取数据请求时，所述目标主机的CPU根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配目标数据块以缓存所述数据请求所操作的数据具体为：所述目标主机的CPU从所述应用挂载的卷中读取待读取的数据；所述目标主机的CPU根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配所述目标数据块以缓存所述取待读取的数据。
  6. 一种运行应用的目标主机，应用于主机集群，其特征在于，所述目标主机包括中央处理单元CPU和本地缓存设备，所述CPU和所述本地缓存设备通信，所述主机集群包括所述目标主机在内的一台或多台主机的本地缓存设备组成的缓存资源池，所述缓存资源池包含多个数据块，每一个数据块映射到所述本地缓存设备的一个物理块；所述目标主机的CPU用于执行如下步骤：
    接收应用发送的数据请求；
    根据所述数据请求确定所述应用的虚拟缓存设备；
    根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配目标数据块以缓存所述数据请求所操作的数据；
    建立所述目标数据块与所述虚拟缓存设备的对应关系。
  7. 根据权利要求6所述的目标主机，其特征在于，所述目标主机存储有所述应用的标识与所述虚拟缓存设备的标识的对应关系，根据所述数据请求确定所述应用的虚拟缓存设备具体包括：根据所述应用 的标识与所述虚拟缓存设备的标识的对应关系确定所述虚拟缓存设备。
  8. 根据权利要求6所述的目标主机，其特征在于，所述目标主机存储有所述应用挂载的卷的标识与所述虚拟缓存设备的标识的对应关系，根据所述数据请求确定所述应用的虚拟缓存设备具体包括：根据所述应用挂载的卷的标识与所述虚拟缓存设备的标识对应关系确定所述虚拟缓存设备。
  9. 根据权利要求6所述的目标主机，其特征在于，当所述数据请求为写数据请求时，根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配所述目标数据块以缓存所述数据请求所操作的数据具体为，根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配所述目标数据块以缓存所述写数据请求携带的数据。
  10. 根据权利要求6所述的目标主机，其特征在于，当所述数据请求为读取数据请求时，根据所述虚拟缓存设备的属性从所述缓存资源池中分配数据块以缓存所述数据请求操作的数据具体为，从所述应用挂载的卷中读取待读取的数据；根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配所述目标数据块以缓存所述待读取的数据。
  11. 一种运行应用的目标主机，应用于主机集群，其特征在于，所述目标主机包括本地缓存设备，所述主机集群包括所述目标主机在内的一台或多台主机的本地缓存设备组成的缓存资源池，所述缓存资源池包含多个数据块，每一个数据块映射到所述本地缓存设备的一个物理块；所述目标主机还包括：
    接收单元，用于接收应用发送的数据请求；
    确定单元，用于根据所述数据请求确定所述应用的虚拟缓存设备；
    分配单元，用于根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配目标数据块以缓存所述数据请求所操作的数据；
    建立单元，用于建立所述目标数据块与所述虚拟缓存设备的对应关系。
  12. 根据权利要求11所述的目标主机，其特征在于，还包括存储单元，用于存储所述应用的标识与所述虚拟缓存设备的标识的对应关系，所述确定单元具体用于：根据所述应用的标识与所述虚拟缓存设备的标识的对应关系确定所述虚拟缓存设备。
  13. 根据权利要求11所述的目标主机，其特征在于，还包括存储单元，用于存储所述应用挂载的卷的标识与所述虚拟缓存设备的标识的对应关系，所述确定单元具体用于：根据所述应用挂载的卷的标识与所述虚拟缓存设备的标识对应关系确定所述虚拟缓存设备。
  14. 根据权利要求11所述的目标主机，其特征在于，所述分配单元，具体用于当所述数据请求为写数据请求时，根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配所述目标数据块以缓存所述写数据请求携带的数据。
  15. 根据权利要求11所述的目标主机，其特征在于，所述分配单元，具体用于当所述数据请求为读取数据请求时，从所述应用挂载的卷中读取待读取的数据；根据所述虚拟缓存设备的属性从所述缓存资源池的所述多个数据块中分配所述目标数据块以缓存所述待读取的数据。
PCT/CN2015/100190 2015-12-31 2015-12-31 一种主机集群中缓存管理方法及主机 WO2017113329A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201580002565.6A CN107851062A (zh) 2015-12-31 2015-12-31 一种主机集群中缓存管理方法及主机
PCT/CN2015/100190 WO2017113329A1 (zh) 2015-12-31 2015-12-31 一种主机集群中缓存管理方法及主机

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2015/100190 WO2017113329A1 (zh) 2015-12-31 2015-12-31 一种主机集群中缓存管理方法及主机

Publications (1)

Publication Number Publication Date
WO2017113329A1 true WO2017113329A1 (zh) 2017-07-06

Family

ID=59224324

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/100190 WO2017113329A1 (zh) 2015-12-31 2015-12-31 一种主机集群中缓存管理方法及主机

Country Status (2)

Country Link
CN (1) CN107851062A (zh)
WO (1) WO2017113329A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117827699B (zh) * 2023-12-27 2024-09-24 中电云计算技术有限公司 并行读cache持久化方法、装置、设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101604226A (zh) * 2009-07-14 2009-12-16 浪潮电子信息产业股份有限公司 一种基于虚拟raid构建动态缓存池提高存储系统性能的方法
CN102467408A (zh) * 2010-11-12 2012-05-23 阿里巴巴集团控股有限公司 一种虚拟机数据的访问方法和设备
US8478931B1 (en) * 2008-07-17 2013-07-02 Virident Systems Inc. Using non-volatile memory resources to enable a virtual buffer pool for a database application
CN103635969A (zh) * 2011-05-31 2014-03-12 美光科技公司 包含存储器系统控制器的设备和相关方法

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10346095B2 (en) * 2012-08-31 2019-07-09 Sandisk Technologies, Llc Systems, methods, and interfaces for adaptive cache persistence
CN102880557B (zh) * 2012-09-13 2015-06-10 浙江金大科技有限公司 一种异构数据源的多级分布式高速缓存的查找方法
WO2014101218A1 (zh) * 2012-12-31 2014-07-03 华为技术有限公司 一种计算存储融合的集群系统
CN103823709B (zh) * 2014-02-28 2017-06-20 华为技术有限公司 虚拟化集群系统、资源分配方法和管理节点
CN105007328A (zh) * 2015-07-30 2015-10-28 山东超越数控电子有限公司 一种基于一致性hash的网络cache设计方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8478931B1 (en) * 2008-07-17 2013-07-02 Virident Systems Inc. Using non-volatile memory resources to enable a virtual buffer pool for a database application
CN101604226A (zh) * 2009-07-14 2009-12-16 浪潮电子信息产业股份有限公司 一种基于虚拟raid构建动态缓存池提高存储系统性能的方法
CN102467408A (zh) * 2010-11-12 2012-05-23 阿里巴巴集团控股有限公司 一种虚拟机数据的访问方法和设备
CN103635969A (zh) * 2011-05-31 2014-03-12 美光科技公司 包含存储器系统控制器的设备和相关方法

Also Published As

Publication number Publication date
CN107851062A (zh) 2018-03-27

Similar Documents

Publication Publication Date Title
US11093177B2 (en) Virtualized OCSSDs spanning physical OCSSD channels
EP3206124B1 (en) Method, apparatus and system for accessing storage device
CN110597451B (zh) 一种虚拟化缓存的实现方法及物理机
US8782335B2 (en) Latency reduction associated with a response to a request in a storage system
US9648081B2 (en) Network-attached memory
KR20200017363A (ko) 호스트 스토리지 서비스들을 제공하기 위한 NVMe 프로토콜에 근거하는 하나 이상의 호스트들과 솔리드 스테이트 드라이브(SSD)들 간의 관리되는 스위칭
CN107203411B (zh) 一种基于远程ssd的虚拟机内存扩展方法及系统
KR102321913B1 (ko) 불휘발성 메모리 장치, 및 그것을 포함하는 메모리 시스템
WO2016119468A1 (zh) 虚拟机内存管理方法、物理主机、pcie设备及其配置方法以及迁移管理设备
US20220066928A1 (en) Pooled memory controller for thin-provisioning disaggregated memory
CN111124951A (zh) 管理数据访问的方法、设备和计算机程序产品
WO2015180598A1 (zh) 对存储设备的访问信息处理方法和装置、系统
JP2020502694A (ja) バイトアドレス可能メモリとして不揮発性メモリにアクセスする方法及び装置
WO2018103022A1 (zh) 帧缓存实现方法、装置、电子设备和计算机程序产品
CN111367472A (zh) 虚拟化方法和装置
US11513849B2 (en) Weighted resource cost matrix scheduler
CN107577733B (zh) 一种数据复制的加速方法及系统
WO2024060710A1 (zh) 一种页面换入方法以及装置
CN104426965B (zh) 自管理存储方法和系统
CN112703481B (zh) 混合式存储器系统接口
WO2017113329A1 (zh) 一种主机集群中缓存管理方法及主机
US10168911B1 (en) Defragmentation of persistent main memory
US11163475B2 (en) Block input/output (I/O) accesses in the presence of a storage class memory
WO2024060711A1 (zh) 一种页面换出方法、装置、设备及数据处理系统
CN115756742A (zh) 直通i/o虚拟化的性能优化设计方法、系统、介质及设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15911955

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15911955

Country of ref document: EP

Kind code of ref document: A1