CN111522621A

CN111522621A - Snapshot cache creating method and device, electronic equipment and computer storage medium

Info

Publication number: CN111522621A
Application number: CN201910105731.5A
Authority: CN
Inventors: 佘海斌
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2019-02-01
Filing date: 2019-02-01
Publication date: 2020-08-11

Abstract

The embodiment of the invention discloses a snapshot cache creating method, a snapshot cache creating device, electronic equipment and a computer storage medium, wherein the method comprises the following steps: determining a snapshot cache creating position; acquiring data of a source block device which can be taken at the creation position, and creating a snapshot based on the data of the source block device, wherein the data of the source block device comprises a source index file, the snapshot comprises a snapshot index file, and the snapshot shares a data file with the source block device; and creating a snapshot cache based on the snapshot, wherein the snapshot cache comprises a snapshot cache index file and a snapshot cache data file. The technical scheme can accelerate the creation of a system disk and a virtual machine, effectively relieve the pressure of a business peak on a block device storage cluster and a mirror image snapshot center, avoid the interference on the operation of the virtual machine, and finally meet the elastic computing requirement of a cloud computing system.

Description

Snapshot cache creating method and device, electronic equipment and computer storage medium

Technical Field

The embodiment of the invention relates to the technical field of data processing, in particular to a snapshot cache creating method and device, electronic equipment and a computer storage medium.

Background

With the development of data technology, cloud computing is more and more widely applied. An important part in cloud computing is elastic computing, and the resource usage in the elastic computing can be elastically expanded, so that the method can adaptively cope with the fluctuating business flow. However, in actual internet applications, the fluctuation of traffic flow is often very severe, and when related applications run on a cloud computing platform, the flexibility of the quantity of the required Virtual Machines (VM) is also very large, that is, when a traffic peak comes, a large number of virtual machines need to be started in a short time, since the disk snapshot includes an operating system and data required by the virtual machine to run, this means that when the traffic peak comes, a large number (for example, tens of thousands) of virtual machine disks need to be concurrently loaded from the disk snapshot/mirror, which will bring a great pressure to a block equipment storage cluster (EBS) and a mirror snapshot center (image center), and at the same time, the running of the virtual machines is also greatly interfered.

Disclosure of Invention

The embodiment of the invention provides a snapshot cache creating method and device, electronic equipment and a computer storage medium.

In a first aspect, an embodiment of the present invention provides a snapshot cache creating method.

Specifically, the snapshot cache creation method includes:

determining a snapshot cache creating position;

acquiring data of a source block device which can be taken at the creation position, and creating a snapshot based on the data of the source block device, wherein the data of the source block device comprises a source index file, the snapshot comprises a snapshot index file, and the snapshot shares a data file with the source block device;

and creating a snapshot cache based on the snapshot, wherein the snapshot cache comprises a snapshot cache index file and a snapshot cache data file.

With reference to the first aspect, in a first implementation manner of the first aspect, the creating a snapshot cache based on the snapshot includes:

determining the positions of the snapshot index file and the data file;

and creating a snapshot cache based on the snapshot index file and the data file according to the positions of the snapshot index file and the data file.

With reference to the first aspect and the first implementation manner of the first aspect, in a second implementation manner of the first aspect, the creating a snapshot cache based on the snapshot index file and the data file according to the locations of the snapshot index file and the data file includes:

determining whether the snapshot index file and the data file are in a local cluster;

and when the snapshot index file and the data file are in the local cluster, creating a snapshot cache in a first creation mode, otherwise, creating the snapshot cache in a second creation mode.

With reference to the first aspect, the first implementation manner of the first aspect, and the second implementation manner of the first aspect, in a third implementation manner of the first aspect, the first creation manner is a hard link creation manner, and the second creation manner is a delayed load creation manner.

In a second aspect, an embodiment of the present invention provides a method for creating a clone block device using a snapshot cache created by any one of the above methods.

Specifically, the method for creating a clone block device includes:

determining a clone device creation location;

acquiring a source snapshot cache of which the distance from the creating position of the clone block device meets a preset condition;

creating a clone block device based on the source snapshot cache.

With reference to the second aspect, in a first implementation manner of the second aspect, the obtaining a source snapshot cache whose distance from the creation location of the clone block device satisfies a preset condition includes:

searching a source snapshot cache with a distance meeting a preset condition in a local cluster range where the creation position of the clone block device is located;

and when the local range of the cluster where the clone block device is located does not have a source snapshot cache with a distance meeting a preset condition, searching upwards step by step.

With reference to the second aspect and the first implementation manner of the second aspect, in a second implementation manner of the second aspect, an embodiment of the present invention provides that the apparatus for creating a clone block based on the source snapshot cache includes:

determining the position of the source snapshot cache;

and creating a clone block device according to the position of the source snapshot cache.

With reference to the second aspect, the first implementation manner of the second aspect, and the second implementation manner of the second aspect, in a third implementation manner of the second aspect, the creating a clone block device according to the location of the source snapshot cache includes:

determining a relationship between a location of the source snapshot cache and the clone device creation location;

when the source snapshot cache and the creation position of the clone block device are in the same cluster, creating the clone block device through a first creation mode based on the source snapshot cache;

when the creation positions of the source snapshot cache and the clone block device are in different clusters, a local cluster snapshot cache is created through a second creation mode based on the source snapshot cache, a local device is created through the second creation mode based on the local cluster snapshot cache, and after downloading of the local cluster snapshot cache is completed, a clone device is created through the second creation mode.

With reference to the second aspect, the first implementation manner of the second aspect, the second implementation manner of the second aspect, and the third implementation manner of the second aspect, in a fourth implementation manner of the second aspect, the first creation manner is a hard link creation manner, and the second creation manner is a delayed load creation manner.

In a third aspect, an embodiment of the present invention provides a snapshot cache creating apparatus.

Specifically, the snapshot cache creating apparatus includes:

a first determination module configured to determine a snapshot cache creation location;

a first obtaining module configured to obtain source block device data retrievable at the creation location, and create a snapshot based on the source block device data, wherein the source block device data includes a source index file, the snapshot includes a snapshot index file, and the snapshot shares a data file with the source block device;

a first creation module configured to create a snapshot cache based on the snapshot, the snapshot cache including a snapshot cache index file and a snapshot cache data file.

With reference to the third aspect, in a first implementation manner of the third aspect, the embodiment of the present invention includes:

a first determining submodule configured to determine locations of the snapshot index file and the data file;

and the first creating sub-module is configured to create a snapshot cache based on the snapshot index file and the data file according to the positions of the snapshot index file and the data file.

With reference to the third aspect and the first implementation manner of the third aspect, in a second implementation manner of the third aspect, an embodiment of the present invention includes that the first creating sub-module includes:

a second determining submodule configured to determine whether the snapshot index file and the data file are within a local cluster;

and the second creating sub-module is configured to create the snapshot cache in the first creating mode when the snapshot index file and the data file are in the local cluster, and otherwise, create the snapshot cache in the second creating mode.

With reference to the third aspect, the first implementation manner of the third aspect, and the second implementation manner of the third aspect, in a third implementation manner of the third aspect, the first creation manner is a hard link creation manner, and the second creation manner is a delayed load creation manner.

In a fourth aspect, an embodiment of the present invention provides an apparatus for creating a clone block device using a snapshot cache created by any one of the above apparatuses.

Specifically, the apparatus for creating a clone block includes:

a second determination module configured to determine a clone device creation location;

the second acquisition module is configured to acquire a source snapshot cache of which the distance from the creation position of the clone block device meets a preset condition;

a second creation module configured to create a clone block device based on the source snapshot cache.

With reference to the fourth aspect, in a first implementation manner of the fourth aspect, the second obtaining module includes:

the first searching submodule is configured to search a source snapshot cache with a distance meeting a preset condition in a local cluster range where the creation position of the clone block device is located;

and the second searching submodule is configured to search upwards step by step when no source snapshot cache with a distance meeting a preset condition exists in the local range of the cluster where the clone block device creation position is located.

With reference to the fourth aspect and the first implementation manner of the fourth aspect, in a second implementation manner of the fourth aspect, the second creating module includes:

a third determining submodule configured to determine a location of the source snapshot cache;

a third creating submodule configured to create a clone block device from the location of the source snapshot cache.

With reference to the fourth aspect, the first implementation manner of the fourth aspect, and the second implementation manner of the fourth aspect, in a third implementation manner of the fourth aspect, the third creating sub-module includes:

a fourth determination submodule configured to determine a relationship between a location of the source snapshot cache and the clone block device creation location;

a fourth creating submodule configured to create a clone block device in a first creating manner based on the source snapshot cache when the source snapshot cache and the clone block device creation position are in the same cluster;

and the fifth creating submodule is configured to create a local cluster snapshot cache in a second creating manner based on the source snapshot cache when the creating positions of the source snapshot cache and the clone block device are in different clusters, create a local device in the second creating manner based on the local cluster snapshot cache, and create a clone device in the second creating manner after downloading of the local cluster snapshot cache is completed.

With reference to the fourth aspect, the first implementation manner of the fourth aspect, the second implementation manner of the fourth aspect, and the third implementation manner of the fourth aspect, in a fourth implementation manner of the fourth aspect, the first creation manner is a hard link creation manner, and the second creation manner is a delayed load creation manner.

In a fifth aspect, an embodiment of the present invention provides an electronic device, which includes a memory and a processor, where the memory is configured to store one or more computer instructions that support a snapshot cache creation apparatus to execute the snapshot cache creation method in the first and second aspects, and the processor is configured to execute the computer instructions stored in the memory. The snapshot cache creating device may further include a communication interface for the snapshot cache creating device to communicate with other devices or a communication network.

In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, configured to store computer instructions for a snapshot cache creation apparatus, where the computer instructions include computer instructions for executing the snapshot cache creation method in the first and second aspects to the snapshot cache creation apparatus.

The technical scheme provided by the embodiment of the invention has the following beneficial effects:

according to the technical scheme, the snapshot is created based on the source block device data, in order to save storage space, the source block device data and the snapshot share a data file, and an available snapshot cache in an effective range is created based on the snapshot, so that when a system disk of a virtual machine is created in a certain cluster, the snapshot cache of the cluster closest to the cluster can be selected as a cloned data source, further, the creation of the system disk and the virtual machine is accelerated, the pressure of a service peak on the block device storage cluster and a mirror image snapshot center is effectively relieved, the running of the virtual machine is prevented from being interfered, and the elastic computing requirement of a cloud computing system is finally met.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of embodiments of the invention.

Drawings

Other features, objects and advantages of embodiments of the invention will become more apparent from the following detailed description of non-limiting embodiments thereof, when taken in conjunction with the accompanying drawings. In the drawings:

FIG. 1 illustrates a flow diagram of a snapshot cache creation method according to an embodiment of the present invention;

FIG. 2 illustrates a cloud computing resource architecture diagram in accordance with an embodiment of the present invention;

FIG. 3 illustrates a cloud computing system architecture diagram according to an embodiment of the invention;

FIG. 4 illustrates a LSBD device architecture diagram based on a distributed file system in accordance with an embodiment of the present invention;

FIG. 5 illustrates a disk block device latency loading implementation architecture diagram according to an embodiment of the present invention;

FIG. 6 shows a flowchart of step S103 of the snapshot cache creation method according to the embodiment shown in FIG. 1;

FIG. 7 shows a flowchart of step S602 of the snapshot cache creation method according to the embodiment shown in FIG. 6;

FIG. 8 shows a flow diagram of a method of creating a clone block device in accordance with an embodiment of the present invention;

fig. 9 shows a flowchart of step S802 of the method of creating a clone block device according to the embodiment shown in fig. 8;

fig. 10 shows a flowchart of step S803 of the method of creating a clone block device according to the embodiment shown in fig. 8;

fig. 11 shows a flowchart of step S1002 of the method of creating a clone block device according to the embodiment shown in fig. 10;

fig. 12 is a block diagram showing the configuration of a snapshot cache creation apparatus according to an embodiment of the present invention;

fig. 13 is a block diagram illustrating a first creating module 1203 of the snapshot cache creating apparatus according to the embodiment illustrated in fig. 12;

fig. 14 is a block diagram illustrating a first creating submodule 1302 of the snapshot cache creating apparatus according to the embodiment illustrated in fig. 13;

fig. 15 is a block diagram showing the construction of an apparatus for creating a clone block device according to an embodiment of the present invention;

fig. 16 is a block diagram showing a second acquisition module 1502 of the apparatus for creating a clone device according to the embodiment shown in fig. 15;

fig. 17 is a block diagram showing a second creating module 1503 of the apparatus for creating a clone device according to the embodiment shown in fig. 15;

fig. 18 is a block diagram showing the structure of a third creating submodule 1702 of the apparatus for creating a clone device according to the embodiment shown in fig. 17;

FIG. 19 shows a block diagram of an electronic device according to an embodiment of the invention;

FIG. 20 is a block diagram of a computer system suitable for use in implementing a snapshot cache creation or clone creation facility method in accordance with an embodiment of the present invention.

Detailed Description

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. Also, for the sake of clarity, parts not relevant to the description of the exemplary embodiments are omitted in the drawings.

In the embodiments of the present invention, it is to be understood that terms such as "including" or "having", etc., are intended to indicate the presence of the features, numbers, steps, actions, components, parts, or combinations thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, actions, components, parts, or combinations thereof may be present or added.

It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict. Embodiments of the present invention will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

The technical scheme provided by the embodiment of the invention is used for creating the snapshot based on the data of the source block device, wherein in order to save the storage space, the data of the source block device and the snapshot share the data file, and the usable snapshot cache in the effective range is created based on the snapshot, so that when a system disk of a virtual machine is created by a certain cluster, the snapshot cache of the cluster closest to the cluster can be selected as a cloned data source, further, the creation of the system disk and the virtual machine is accelerated, the pressure of a service peak on the storage cluster of the block device and a mirror image snapshot center is effectively relieved, the running of the virtual machine is prevented from being interfered, and the elastic computing requirement of a cloud computing system is finally met.

Fig. 1 shows a flowchart of a snapshot cache creation method according to an embodiment of the present invention, as shown in fig. 1, the snapshot cache creation method includes the following steps S101 to S103:

in step S101, a snapshot cache creation position is determined;

in step S102, obtaining source block device data retrievable at the creation position, and creating a snapshot based on the source block device data, where the source block device data includes a source index file, the snapshot includes a snapshot index file, and the snapshot shares a data file with the source block device;

in step S103, a snapshot cache is created based on the snapshot, where the snapshot cache includes a snapshot cache index file and a snapshot cache data file.

As mentioned above, with the development of data technology, cloud computing is increasingly widely used. An important part in cloud computing is elastic computing, and the resource usage in the elastic computing can be elastically expanded, so that the method can adaptively cope with the fluctuating business flow. However, in actual internet applications, the fluctuation of traffic flow is often very severe, and when related applications run on a cloud computing platform, the flexibility of the quantity of the required Virtual Machines (VM) is also very large, that is, when a traffic peak comes, a large number of virtual machines need to be started in a short time, since the disk snapshot includes an operating system and data required by the virtual machine to run, this means that when the traffic peak comes, a large number (for example, tens of thousands) of virtual machine disks need to be concurrently loaded from the disk snapshot/mirror, which will bring a great pressure to a block equipment storage cluster (EBS) and a mirror snapshot center (image center), and at the same time, the running of the virtual machines is also greatly interfered.

In view of the above problems, in this embodiment, a snapshot cache creating method is provided, where a snapshot is created based on source block device data, and in order to save a storage space, the source block device data and the snapshot share a data file, and an available snapshot cache in an effective range is created based on the snapshot, so that when a system disk of a virtual machine is created in a certain cluster, a snapshot cache of a cluster closest to the cluster can be selected as a cloned data source, thereby accelerating the creation of the system disk and the virtual machine, effectively relieving pressure on a block device storage cluster and a mirror snapshot center caused by a business peak, avoiding interference on the operation of the virtual machine, and finally meeting an elastic computing requirement of a cloud computing system.

In an embodiment of the present invention, the snapshot cache creating method is applicable to a distributed system. The snapshot cache is a cache file which is created to quickly propagate snapshot data and quickly clone needed block device data for a short time virtual machine and comprises a snapshot cache index file and a snapshot cache data file. When creating a snapshot cache, first, a creation location of the snapshot cache needs to be determined, where the creation location of the snapshot cache may be set as a certain Cluster (Cluster) in a certain available area (azo, available Zone) of a certain cloud computing area (Region), and specifically may be set according to a need of an actual application.

First, a basic architecture of cloud computing resources is introduced, and as shown in fig. 2, in the cloud computing resources, an allocation unit of the cloud computing resources includes: a cloud computing area (Region), an available Zone (available Zone), and a Cluster (Cluster), wherein a Region refers to a cloud computing area, which may be a city, for example; the zones refer to a computing unit with independent power and network resources, the quality of network access is consistent in the same zone, generally, one Region may contain 3 zones, each Region has a mirror center for storing snapshot/mirror images of a virtual machine system disk, and all zones within the range of the Region are served; the Cluster refers to a management unit of a computing storage host, one azo can contain a plurality of clusters, and the host size of the clusters is generally in the order of hundreds.

Fig. 3 is a basic architecture of a cloud computing system, and as shown in fig. 3, in the cloud computing system, storage and computation are separated from each other, so the cloud computing cluster can be divided into a storage cluster/snapshot center storage cluster including a plurality of storage nodes and a computing cluster including a plurality of computing nodes, where the storage cluster is responsible for providing storage resources of disk block devices (i.e., disks (disks) of VMs), and the computing cluster is responsible for providing computing resources (i.e., CPUs/memories of VMs), and the storage cluster, the snapshot center storage cluster and the computing cluster exchange and transmit data to be processed through a switch.

The storage cluster is realized based on a Log Structured Block Device (LSBD) block storage technology, the LSBD adopts a log file (log file) to construct a disk block device, the log file is a distributed file which can only be additionally written and cannot be overwritten, and the device is constructed based on the log file. Fig. 4 illustrates an LSBD device architecture diagram based on a distributed file system according to an embodiment of the present invention, as shown in fig. 4, the LSBD device is implemented based on a log file of the distributed file system, and multiple devices and snapshots may share the same log file through a hard link (Hardlink), wherein the distributed file system guarantees data security and availability through a multi-copy (Replica) manner.

Further, the LSBD Device may be divided into a plurality of Device Segment groups (Device Segment groups) consisting of a plurality of Device segments (Device Segment), wherein each Device Segment is handled by a block server (BlockServer), so that the LSBD block Device system can be easily load balanced. Further, the device segment is composed of an Index file (Index file), a data file (data file) and a Txn file (Txn file), which are all log files in a distributed file, and can only be additionally written and cannot be overwritten. The index file is responsible for recording the corresponding relation between the logical address interval (LBARange) of the device and the physical address interval of the data file, the data file is responsible for storing the data of the device segment, and the Txn file is responsible for recording and modifying the Transaction Log (Transaction Log) of the device segment. Specifically, for a device segment, the process of writing data can be described as: firstly writing data in a data file, then writing a transaction log in a Txn file, and finally updating an index file.

Based on the above, the implementation principle of the delayed loading of the disk Block device is introduced again, and fig. 5 shows an implementation architecture diagram of the delayed loading of the disk Block device according to an embodiment of the present invention, as shown in fig. 5, a snapshot/image is cut into a plurality of data blocks (blocks) according to a length of 2M, and during the delayed loading (Lazyload), a loading operation is performed in units of 2M data blocks, a system background process gradually downloads snapshot/image data, and the downloaded snapshot/image data can be used at the same time, that is, even if the data is not completely downloaded, the downloaded data can also be used. In addition, when the data content of a certain data block needs to be read during the running of the virtual machine, the system can preferentially download the content in the corresponding data block in the snapshot/mirror image.

In the above scheme, the source block device data available at the creation location refers to source block device data available at the creation location. In an embodiment of the present invention, the source block device data that can be fetched may be, for example, source block device data whose distance from the snapshot cache creation position satisfies a preset condition, and therefore, the source block device data may be source block device data located in the same Cluster (Cluster) as the snapshot cache, may be source block device data located in the same available area (azo) but different Cluster as the snapshot cache, or may be source block device data located in the same area (Region) but different available area as the snapshot cache. Therefore, the required source block device data can be quickly obtained, and the accelerated creation of the system disk and the virtual machine can be realized.

In an embodiment of the present invention, after the source block device Data is obtained, a snapshot may be created based on the source block device Data, and in order to effectively save a storage space, both the snapshot and the source block device only have their index files (index files), and the Data files (Data files) are shared.

In an embodiment of the present invention, as shown in fig. 6, the step S103 of creating a snapshot cache based on the snapshot includes the following steps S601-S602:

in step S601, the positions of the snapshot index file and the data file are determined;

in step S602, a snapshot cache is created based on the snapshot index file and the data file according to the positions of the snapshot index file and the data file.

In the above, the creating position of the snapshot cache may be set as a Cluster (Cluster) in an available Zone (azo, available Zone) of a certain cloud computing area (Region), so that, as a data source of the snapshot cache, the snapshot may be in the same Cluster as the snapshot cache, may also be in a different Cluster of the same available Zone, or may also be in a different available Zone of the same cloud computing area.

In an optional implementation manner of this embodiment, as shown in fig. 7, the step S602, that is, the step of creating a snapshot cache based on the snapshot index file and the data file according to the positions of the snapshot index file and the data file, includes the following steps S701 to S702:

in step S701, it is determined whether the snapshot index file and the data file are in a local cluster;

in step S702, when the snapshot index file and the data file are in the local cluster, a snapshot cache is created in a first creation manner, otherwise, a snapshot cache is created in a second creation manner.

In an optional implementation manner of this embodiment, the first creation manner is a hard link (hardlink) creation manner, and the second creation manner is a lazy load (Lazyload) creation manner.

In this embodiment, when the snapshot index file and the data file are in a local cluster, that is, the snapshot index file and the data file are in the same cluster as the creation location of the snapshot cache, in order to not increase the storage space, a snapshot cache is created in a hard link creation manner, that is, the index file and the data file of the snapshot cache are created based on the snapshot index file and the data file through hard links; otherwise, when the snapshot index file and the data file are not in the local cluster, that is, the snapshot index file and the data file are in different clusters in the same available area as the snapshot cache creation location, or in different available areas in the same cloud computing area, the snapshot cache needs to be created in a deferred loading creation manner, and at this time, the deferred loading object is not the snapshot index file and the data file, but an available snapshot cache in a non-local cluster, such as a snapshot cache in a mirror center or other clusters.

Fig. 8 shows a flowchart of a method for creating a clone block device using a snapshot cache created by any one of the above methods according to an embodiment of the present invention, and as shown in fig. 8, the method for creating a clone block device includes the following steps S801 to S803:

in step S801, a clone device creation position is determined;

in step S802, a source snapshot cache whose distance from the creation location of the clone block device satisfies a preset condition is obtained;

in step S803, a clone block device is created based on the source snapshot cache.

As mentioned above, when the elastic computing service peak comes, a large number of virtual machines need to be started in a short time, and the disk snapshot includes an operating system and data required for running the virtual machines, which means that when the service peak comes, a large number (e.g., tens of thousands) of virtual machine disks need to be loaded from the disk snapshot/mirror concurrently, which will bring great pressure to the block equipment storage cluster (EBS) and the mirror snapshot center (image center), and also cause great interference to the running of the virtual machines.

In view of the above problems, in this embodiment, a method for creating a clone block device using a snapshot cache is provided, where when a system disk of a virtual machine is created in a cluster, a snapshot cache closest to a creation location of the clone block device or meeting a preset distance condition is selected as a cloned data source, so as to accelerate creation of the system disk and the virtual machine, effectively relieve pressure on a block device storage cluster and a mirror snapshot center from a service peak, avoid interference on operation of the virtual machine, and finally meet an elastic computing requirement of a cloud computing system.

In an embodiment of the present invention, when creating a clone device, first, a creation location of the clone device needs to be determined, where the creation location of the clone device may be set as a certain Cluster (Cluster) in a certain available area (azo, available Zone) of a certain cloud computing area (Region), and may be specifically set according to requirements of actual applications. In this way, the clone device and clone-source data, that is, source snapshot cache data may be in the same cluster, different clusters of the same available area, or different available areas of the same cloud computing area, and for a plurality of available clone-source data, a snapshot cache whose distance from the creation location of the clone device satisfies a preset condition may be selected as the clone-source data of the clone device, where the preset condition may be selected according to the needs of actual applications and the characteristics of the clone device, for example, a snapshot cache closest to the creation location of the clone device may be selected as the clone-source data of the clone device.

In an optional implementation manner of this embodiment, as shown in fig. 9, the step S802, that is, the step of obtaining the source snapshot cache whose distance from the creation location of the clone block device satisfies the preset condition, includes the following steps S901 to S902:

in step S901, a source snapshot cache whose distance satisfies a preset condition is searched in a local cluster range where the creation location of the clone device is located;

in step S902, when there is no source snapshot cache whose distance satisfies a preset condition in the local range of the cluster where the creation location of the clone device is located, the source snapshot cache is searched upward step by step.

In order to save the cloning time of the block device and reduce the impact on the IO performance of the user, in this embodiment, when obtaining a source snapshot cache whose distance from the creation location of the clone block device satisfies a preset condition, first, a source snapshot cache whose distance satisfies the preset condition is searched in a local range of a cluster where the creation location of the clone block device is located, if the source snapshot cache exists, the clone block device may be created directly by a hard link creation manner, otherwise, the search is performed step by step, for example, a snapshot cache whose distance satisfies the preset condition is searched in an available area range of the cluster where the creation location of the clone block device is located is first searched, if the snapshot cache exists, the local snapshot cache may be created in the local cluster by a delayed loading creation manner, then, based on the local snapshot cache, the local block device may be created by a delayed loading creation manner, and after the local snapshot cache data is completely downloaded, the data is then imported into the local block device, in addition, after the local snapshot cache data is completely downloaded, a clone block device can be directly created in a hard link mode, otherwise, a source snapshot cache with a distance meeting a preset condition is searched in a cloud computing area range where an available area where the clone block device is located, similarly, if the source snapshot cache exists, a local snapshot cache can be created in the cluster in a delayed loading creation mode, based on the local snapshot cache, the local block device is created in a delayed loading creation mode, after the local snapshot cache data is completely downloaded, the data is imported into the local block device, and based on the local snapshot cache with the data downloaded, the clone block device can be created.

In an optional implementation manner of this embodiment, as shown in fig. 10, the step S803, that is, the step of creating a clone block device based on the source snapshot cache, includes the following steps S1001 to S1002:

in step S1001, determining a location of the source snapshot cache;

in step S1002, a clone block device is created according to the location of the source snapshot cache.

Since the location of the source snapshot cache is related to the manner in which the clone device is created, the location of the source snapshot cache needs to be determined first when creating the clone device based on the source snapshot cache.

In an optional implementation manner of this embodiment, as shown in fig. 11, the step S1002, that is, the step of creating a clone block device according to the location of the source snapshot cache, includes the following steps S1101-S1103:

in step S1101, a relationship between the location of the source snapshot cache and the clone device creation location is determined;

in step S1102, when the source snapshot cache and the clone device creation location are in the same cluster, creating a clone device in a first creation manner based on the source snapshot cache;

in step S1103, when the source snapshot cache and the creation location of the clone block device are in different clusters, a local cluster snapshot cache is created in a second creation manner based on the source snapshot cache, a local device is created in the second creation manner based on the local cluster snapshot cache, and after downloading of the local cluster snapshot cache is completed, a clone device is created in the second creation manner.

In an optional implementation manner of this embodiment, the first creation manner is a hard link creation manner, and the second creation manner is a delayed load creation manner.

As mentioned above, if there exists a source snapshot cache locally in the cluster where the clone block device creation location is located, and the distance between the source snapshot cache and the cluster meets the preset condition, in order not to increase the storage space, the clone block device may be created directly by a hard link creation manner; if a source snapshot cache with a distance meeting a preset condition from the creating position of the clone block device exists outside the cluster, the local snapshot cache can be created in the cluster in a delayed loading creating mode, then the local block device is created in the delayed loading creating mode based on the local snapshot cache, the data is imported into the local block device after the data of the local snapshot cache is completely downloaded, and in addition, the clone block device can be directly created in a hard link mode after the data of the local snapshot cache is completely downloaded.

In specific implementation, because Ocean is a metadata management center at a Region level, river master is a metadata management center at an Azone level, and blockamaster is a metadata management center at a Cluster level, Ocean is responsible for collecting location information of all snapshots/snapshot caches/images, Ocean collects location information to river master subordinate thereto, river master collects location information to blockack master subordinate thereto, and Ocean can also collect image location information to image centers. When data is read or downloaded, each Snapshot Segment (Snapshot Segment) in the deferred loading Snapshot cache created based on deferred loading is downloaded by a Snapshot server (Snapshot server); when a lazy load block Device (Lazyload Device) needs to read snapshot data content, a corresponding BlockServer requests a corresponding snapshot server to read; when the mirror image delay loading is carried out from the mirror image center, a local snapshot server requests to download an object storage (Objectstorage) of the mirror image center; and when the loading is delayed from the snapshot cache of the source cluster, the local snapshot server requests the snapshot server corresponding to the source cluster for downloading.

In addition, in order to further save the storage space, in an optional implementation manner of this embodiment, when all the devices that are created on a cluster and share the same snapshot cache are deleted, that is, after the snapshot cache is not referenced, the snapshot cache will be automatically deleted.

The following are embodiments of the apparatus of the present invention that may be used to perform embodiments of the method of the present invention.

Fig. 12 is a block diagram illustrating a snapshot cache creation apparatus according to an embodiment of the present invention, which may be implemented as part or all of an electronic device by software, hardware, or a combination of the two. As shown in fig. 12, the snapshot cache creation apparatus includes:

a first determining module 1201 configured to determine a snapshot cache creation location;

a first obtaining module 1202 configured to obtain source block device data retrievable at the creation location, and create a snapshot based on the source block device data, wherein the source block device data includes a source index file, the snapshot includes a snapshot index file, and the snapshot shares a data file with the source block device;

a first creating module 1203 configured to create a snapshot cache based on the snapshot, the snapshot cache including a snapshot cache index file and a snapshot cache data file.

In view of the above problems, in this embodiment, a snapshot cache creating apparatus is provided, where the apparatus creates a snapshot based on source block device data, and in order to save a storage space, the source block device data and the snapshot share a data file, and an available snapshot cache in an effective range is created based on the snapshot, so that when a system disk of a virtual machine is created in a certain cluster, a snapshot cache of a cluster closest to the cluster can be selected as a cloned data source, thereby accelerating the creation of the system disk and the virtual machine, effectively relieving pressure on a block device storage cluster and a mirror snapshot center from a service peak, avoiding interference on the operation of the virtual machine, and finally meeting an elastic computing requirement of a cloud computing system.

In an embodiment of the present invention, the snapshot cache creating apparatus may be applied to a distributed system. The snapshot cache is a cache file which is created to quickly propagate snapshot data and quickly clone needed block device data for a short time virtual machine and comprises a snapshot cache index file and a snapshot cache data file. When creating a snapshot cache, first, a creation location of the snapshot cache needs to be determined, where the creation location of the snapshot cache may be set as a certain Cluster (Cluster) in a certain available area (azo, available Zone) of a certain cloud computing area (Region), and specifically may be set according to a need of an actual application.

In an embodiment of the present invention, as shown in fig. 13, the first creating module 1203 includes:

a first determining sub-module 1301 configured to determine the locations of the snapshot index file and the data file;

a first creating sub-module 1302, configured to create a snapshot cache based on the snapshot index file and the data file according to the positions of the snapshot index file and the data file.

In an optional implementation manner of this embodiment, as shown in fig. 14, the first creating sub-module 1302 includes:

a second determining sub-module 1401 configured to determine whether the snapshot index file and the data file are within a local cluster;

and the second creating sub-module 1402 is configured to create the snapshot cache in the first creating manner when the snapshot index file and the data file are in the local cluster, and otherwise, create the snapshot cache in the second creating manner.

In this embodiment, when the second determining sub-module 1401 determines that the snapshot index file and the data file are in a local cluster, that is, the snapshot index file and the data file are in the same cluster as the creation location of the snapshot cache, in order not to increase the storage space, the second creating sub-module 1402 creates the snapshot cache by a hard link creation manner, that is, based on the snapshot index file and the data file, the index file and the data file of the snapshot cache are created by a hard link; otherwise, when the second determining sub-module 1401 determines that the snapshot index file and the data file are not in the local cluster, that is, the snapshot index file and the data file are in different clusters in the same available area as the snapshot cache creation location, or in different available areas in the same cloud computing area, the second creating sub-module 1402 creates the snapshot cache by a deferred loading creation manner, and at this time, the deferred loading object is not the snapshot index file and the data file but an available snapshot cache in a non-local cluster, such as a snapshot cache in a mirror center or other cluster.

Fig. 15 is a block diagram illustrating an apparatus for creating a clone block using a snapshot cache created by any one of the above apparatuses, which may be implemented as part or all of an electronic device by software, hardware, or a combination of the two, according to an embodiment of the present invention. As shown in fig. 15, the snapshot cache creation apparatus includes:

a second determination module 1501 configured to determine a clone device creation location;

a second obtaining module 1502 configured to obtain a source snapshot cache whose distance from the creation location of the clone block device satisfies a preset condition;

a second creation module 1503 configured to create a clone device based on the source snapshot cache.

In view of the above problems, in this embodiment, an apparatus for creating a clone block device using a snapshot cache is provided, where when a system disk of a virtual machine is created in a cluster, the apparatus selects a snapshot cache that is closest to a creation location of the clone block device or meets a preset distance condition as a cloned data source, so as to accelerate creation of the system disk and the virtual machine, effectively relieve pressure on a block device storage cluster and a mirror snapshot center from a service peak, avoid interference on operation of the virtual machine, and finally meet an elastic computing requirement of a cloud computing system.

In an optional implementation manner of this embodiment, as shown in fig. 16, the second obtaining module 1502 includes:

a first search submodule 1601 configured to search a source snapshot cache whose distance meets a preset condition in a local range of a cluster where the creation location of the clone block device is located;

and a second searching sub-module 1602, configured to perform searching upwards step by step when there is no source snapshot cache whose distance satisfies a preset condition in the local range of the cluster where the creating location of the cloned block device is located.

In order to save the cloning time of the block device and reduce the impact on the IO performance of the user, in this embodiment, when the second obtaining module 1502 obtains the source snapshot cache whose distance from the creation location of the clone block device meets the preset condition, the first searching sub-module 1601 first searches the source snapshot cache whose distance meets the preset condition in the local range of the cluster where the creation location of the clone block device is located, if so, the clone block device can be created directly through the hard link creation mode, otherwise, the second searching sub-module 1602 searches upwards step by step, for example, first searches the snapshot cache whose distance meets the preset condition in the available area range of the cluster where the creation location of the clone block device is located, if so, the local snapshot cache can be created in the present cluster through the delayed load creation mode, and then creates the local block device through the delayed load creation mode based on the local snapshot cache, and importing the data into the local block equipment after the local snapshot cache data are completely downloaded, otherwise, directly establishing the clone block equipment in a hard link mode after the local snapshot cache data are completely downloaded, otherwise, searching a source snapshot cache with a distance meeting a preset condition in a cloud computing area range of an available area where the clone block equipment is established, similarly, if the source snapshot cache exists, establishing the local snapshot cache in the cluster in a delayed loading establishing mode, establishing the local block equipment in a delayed loading establishing mode based on the local snapshot cache, importing the data into the local block equipment after the local snapshot cache data are completely downloaded, and establishing the clone block equipment based on the snapshot cache after the data are completely downloaded.

In an optional implementation manner of this embodiment, as shown in fig. 17, the second creating module 1503 includes:

a third determining submodule 1701 configured to determine a location of the source snapshot cache;

a third create sub-module 1702 configured to create a clone device based on the location of the source snapshot cache.

Since the location of the source snapshot cache is related to the manner in which the clone device is created, the third determination submodule 1701 needs to first determine the location of the source snapshot cache when creating the clone device based on the source snapshot cache.

In an optional implementation manner of this embodiment, as shown in fig. 18, the third creating sub-module 1702 includes:

a fourth determining submodule 1801 configured to determine a relationship between the location of the source snapshot cache and the clone block device creation location;

a fourth creating submodule 1802 configured to create a clone block device by a first creating means based on the source snapshot cache when the source snapshot cache and the clone block device creation location are in the same cluster;

a fifth creating sub-module 1803, configured to, when the source snapshot cache and the creation location of the clone block device are in different clusters, create a local cluster snapshot cache in a second creating manner based on the source snapshot cache, create a local device in the second creating manner based on the local cluster snapshot cache, and create a clone device in the second creating manner after downloading of the local cluster snapshot cache is completed.

As mentioned above, if there is a source snapshot cache locally in the cluster where the creating location of the clone block device is located, and the distance between the source snapshot cache and the cluster meets the preset condition, in order not to increase the storage space, the fourth creating sub-module 1802 may create the clone block device directly by a hard link creating manner; if a source snapshot cache whose distance from the creating position of the clone block device is beyond the cluster exists and meets the preset condition, the fifth creating sub-module 1803 may create a local snapshot cache in the cluster in a delayed loading creating manner, create the local block device in a delayed loading creating manner based on the local snapshot cache, and import the data into the local block device after the local snapshot cache data is completely downloaded, and in addition, create the clone block device directly in a hard link manner after the local snapshot cache data is completely downloaded.

Fig. 19 is a block diagram illustrating a structure of an electronic device according to an embodiment of the invention, and as shown in fig. 19, the electronic device 1900 includes a memory 1901 and a processor 1902; wherein the content of the first and second substances,

the memory 1901 is used to store one or more computer instructions, which are executed by the processor 1902 to perform any of the method steps described above.

Fig. 20 is a schematic structural diagram of a computer system suitable for implementing the snapshot cache creation method according to the embodiment of the present invention.

As shown in fig. 20, the computer system 2000 includes a Central Processing Unit (CPU)2001, which can execute various processes in the above-described embodiments according to a program stored in a Read Only Memory (ROM)2002 or a program loaded from a storage section 2008 into a Random Access Memory (RAM) 2003. In the RAM2003, various programs and data necessary for the operation of the system 2000 are also stored. The CPU2001, ROM2002, and RAM2003 are connected to each other via a bus 2004. An input/output (I/O) interface 2005 is also connected to bus 2004.

The following components are connected to the I/O interface 2005: an input portion 2006 including a keyboard, a mouse, and the like; an output portion 2007 including a display device such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 2008 including a hard disk and the like; and a communication section 2009 including a network interface card such as a LAN card, a modem, or the like. The communication section 2009 performs communication processing via a network such as the internet. Drive 2010 is also connected to I/O interface 2005 as needed. A removable medium 2011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 2010 as necessary, so that a computer program read out therefrom is mounted in the storage section 2008 as necessary.

In particular, the above described method may be implemented as a computer software program according to an embodiment of the present invention. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a medium readable thereby, the computer program comprising program code for performing the snapshot cache creation method. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 2009, and/or installed from the removable medium 2011.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, a program segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units or modules described in the embodiments of the present invention may be implemented by software, or may be implemented by hardware. The units or modules described may also be provided in a processor, and the names of the units or modules do not in some cases constitute a limitation of the units or modules themselves.

As another aspect, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium may be a computer-readable storage medium included in the apparatus in the foregoing embodiment; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the methods described in the embodiments of the present invention.

The foregoing description is only exemplary of the preferred embodiments of the invention and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention according to the embodiments of the present invention is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept. For example, the above features and (but not limited to) the features with similar functions disclosed in the embodiments of the present invention are mutually replaced to form the technical solution.

Claims

1. A snapshot cache creation method, comprising:

determining a snapshot cache creating position;

2. The method of claim 1, wherein creating a snapshot cache based on the snapshot comprises:

determining the positions of the snapshot index file and the data file;

3. The method of claim 2, wherein creating a snapshot cache based on the snapshot index file and the data file according to the locations of the snapshot index file and the data file comprises:

4. The method according to any one of claims 1 to 3, wherein the first creation means is a hard link creation means and the second creation means is a delayed load creation means.

5. A method for creating a clone block device using a snapshot cache created by the method of any one of claims 1-4, comprising:

determining a clone device creation location;

creating a clone block device based on the source snapshot cache.

6. The method according to claim 5, wherein obtaining a source snapshot cache whose distance from the cloning device creation location satisfies a preset condition comprises:

7. The method according to claim 5 or 6, wherein said creating a clone block device based on said source snapshot cache comprises:

determining the position of the source snapshot cache;

8. The method of claim 7, wherein creating a clone block device from the location of the source snapshot cache comprises:

9. The method according to any one of claims 5 to 8, wherein the first creation means is a hard link creation means and the second creation means is a lazy load creation means.

10. A snapshot cache creation apparatus, comprising:

11. The apparatus of claim 10, wherein the first creating module comprises:

12. The apparatus of claim 11, wherein the first create submodule comprises:

13. The apparatus according to any of claims 10-12, wherein the first creation means is a hard link creation means and the second creation means is a lazy load creation means.

14. An apparatus for creating a clone block device using a snapshot cache created by the apparatus of any of claims 10-13, comprising:

15. The apparatus of claim 14, wherein the second obtaining module comprises:

16. The apparatus of claim 14 or 15, wherein the second creating module comprises:

17. The apparatus of claim 16, wherein the third creation submodule comprises:

18. The apparatus according to any of claims 14-17, wherein the first creation means is a hard link creation means and the second creation means is a lazy load creation means.

19. An electronic device comprising a memory and a processor; wherein the content of the first and second substances,

the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps of any of claims 1-9.

20. A computer-readable storage medium having stored thereon computer instructions, characterized in that the computer instructions, when executed by a processor, carry out the method steps of any of claims 1-9.