CN117714476A - Cloud coil control method, cloud coil control system, electronic equipment and storage medium - Google Patents

Abstract

The application provides a cloud coil control method, a cloud coil control system, electronic equipment and a storage medium, and relates to the technical field of distributed storage. The method comprises the following steps: acquiring cloud disk identification information of a target cloud disk; the cloud disk identification information is used for identifying a target cloud disk, a target storage management and control node corresponding to the target cloud disk and a target cluster; the target cluster comprises a plurality of storage control nodes, and the storage control nodes comprise target storage control nodes; the target storage management and control node is used for creating a target cloud disk on at least one storage service node of the target cluster and generating cloud disk identification information; and determining a target cluster, a target storage management node and a target cloud disk according to the cloud disk identification information. According to the technical scheme, the resource utilization rate can be improved, the large-scale cluster is supported, and the explosion radius is reduced.

Description

Cloud coil control method, cloud coil control system, electronic equipment and storage medium

Technical Field

The application relates to the technical field of distributed storage, in particular to a cloud management and control method, a cloud management and control system, electronic equipment and a storage medium.

Background

Distributed storage is a data storage technology that disperses data stored on devices in a network of multiple physical locations. The block storage system is a low latency, persistent and highly reliable block level random distributed storage system. In a block storage system, data is partitioned into blocks (blocks), each of which operates as a separate hard disk. This approach provides high performance and flexibility, and is particularly suited for processing large amounts of data, such as databases and large-scale virtual machine environments. In the application scene of distributed block storage, the cloud disk is provided for block storage service of a user data block level, the storage space of a physical hard disk is virtualized, the virtual storage space is provided for a user through a network, and the user can use the cloud disk like a local hard disk to perform data read-write operation. Therefore, in the block storage system, how to realize efficient management and control of the cloud disk, and reduce the fault domain of the block storage system are very important.

Disclosure of Invention

The embodiment of the application provides a cloud coil control method, a cloud coil control system, electronic equipment and a storage medium, so as to alleviate or solve one or more technical problems in the prior art.

In a first aspect, an embodiment of the present application provides a cloud management and control method, including: acquiring cloud disk identification information of a target cloud disk; the cloud disk identification information is used for identifying the target cloud disk and a target storage management node and a target cluster corresponding to the target cloud disk; the target cluster comprises a plurality of storage control nodes, and the storage control nodes comprise the target storage control nodes; the target storage management node is used for creating the target cloud disk on at least one storage service node of the target cluster and generating the cloud disk identification information; and determining the target cluster, the target storage management and control node and the target cloud disk according to the cloud disk identification information.

In a second aspect, an embodiment of the present application provides a cloud management and control method, including: creating a target cloud disk on at least one storage service node of the target cluster in response to the Yun Pan create request; the target cluster comprises a plurality of storage control nodes, and the storage control nodes comprise the target storage control nodes; generating cloud disk identification information of the target cloud disk; the cloud disk identification information is used for identifying the target cloud disk, the target storage management and control node and the target cluster.

In a third aspect, embodiments of the present application provide a cloud management and control system, including: a cluster management node, configured to perform a method provided by an embodiment of the first aspect of the present application; at least one cluster, the cluster including a plurality of storage management nodes and a plurality of storage service nodes, the storage management nodes being configured to perform a method provided by an embodiment of the second aspect of the present application.

In a fourth aspect, embodiments of the present application provide an electronic device including a memory, a processor, and a computer program stored on the memory, the processor implementing the method provided by any of the embodiments of the present application when the computer program is executed.

In a fifth aspect, embodiments of the present application provide a computer readable storage medium having a computer program stored therein, the computer program, when executed by a processor, implementing a method provided by any embodiment of the present application.

According to the cloud management and control method, storage management and control nodes in one storage cluster are horizontally expanded, namely one storage cluster comprises a plurality of storage management and control nodes, and each storage management and control node is responsible for management and control of a plurality of groups of cloud disks (for example, a plurality of cloud disk sets). On one hand, each storage control node works simultaneously, so that resource waste caused by a main multi-standby architecture is avoided, the resource utilization rate is improved, and a large-scale cluster can be supported; on the other hand, at a certain storage management node failure, the affected cloud disk in the storage cluster is only a part, so the explosion radius is far smaller than that of the centralized BM architecture. Furthermore, an effective cluster resource scheduling scheme can be provided for a cloud disk control architecture of the distributed storage control node, so that the cloud disk scheduling is not affected by architecture update.

The foregoing summary is for the purpose of the specification only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the present application will become apparent by reference to the drawings and the following detailed description.

Drawings

In the drawings, the same reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily drawn to scale. It is appreciated that these drawings depict only some embodiments according to the disclosure and are not therefore to be considered limiting of its scope.

FIG. 1 illustrates an architecture diagram of a cloud-based management and control system provided in the related art;

fig. 2 shows an architecture diagram of a cloud management and control system provided in an embodiment of the present application;

fig. 3 illustrates an application example diagram of cloud disk scheduling provided in an embodiment of the present application;

fig. 4 illustrates another application example diagram of cloud disk scheduling provided in an embodiment of the present application;

fig. 5 shows a flowchart of a cloud management control method provided in an embodiment of the present application;

fig. 6 shows a flowchart of a cloud management control method provided in an embodiment of the present application;

Fig. 7 shows a block diagram of an electronic device provided in an embodiment of the present application.

Detailed Description

Hereinafter, only certain exemplary embodiments are briefly described. As will be recognized by those of skill in the pertinent art, the described embodiments may be modified in various different ways without departing from the spirit or scope of the present application. Accordingly, the drawings and description are to be regarded as illustrative in nature and not as restrictive.

In order to facilitate understanding of the technical solutions of the embodiments of the present application, the following description is given of related technologies of the embodiments of the present application, and the following related technologies may be optionally combined with the technical solutions of the embodiments of the present application as an alternative, which all belong to the protection scope of the embodiments of the present application.

The following terms will be used hereinafter for explanation:

block Storage (Block Storage) system: a low latency, persistent and highly reliable block-level random access memory system divides data into blocks, each of which operates as a separate hard disk drive. This means that users are free to create their own file system on these blocks, thereby better controlling the data environment, and being particularly suited for handling large amounts of data, such as databases and large-scale virtual machine environments.

Cloud Disk (Cloud Disk): based on the distributed storage architecture, block storage services at the data block level are provided. The data of the cloud disk is stored in the distributed storage system of the back end in a lasting mode, and downtime of any node of the computing end cannot affect the cloud disk.

Radius of explosion: the size of the fault domain is represented, namely the influence surface when a certain node fails. The smaller the explosion radius, the smaller the fault domain; the larger the explosion radius, the larger the fault domain.

Internet data center (Internet Data Center, IDC): is a facility that stores, manages, and distributes large amounts of data. It provides a range of services including server hosting, network connectivity, data storage and backup, security management, etc.

Availability zone (Availability Zone): representing a physical location having its own power supply, network connections, security facilities, etc. Within the same geographic area (Region), there may be one or more available regions. Each of the available regions is independent of the other available regions to prevent affecting the other available regions when a problem occurs in one of the available regions. For example, if one available area is problematic due to a power failure, network disruption, or natural disaster, the other available area may continue to operate normally. In a cloud computing environment, one availability zone contains one or more internet data centers (Internet Data Center, IDC).

Cluster (Cluster): are a group of mutually independent computers interconnected via a high-speed computer network, which form a group and are managed in a single system mode. Computers or servers in a cluster are typically connected together through a high-speed network, sharing storage, software, or other resources, such as a computing cluster, storage cluster, and the like.

Cluster management and control nodes: service nodes, such as Resource management nodes (RMs), for managing the individual storage clusters.

Storage management node: in this embodiment, each storage cluster has a plurality of storage management nodes, such as a Block Master (BM), and the plurality of storage management nodes commonly manage one storage cluster.

Storage service node: each storage cluster has a plurality of storage service nodes, and each storage service node is used for writing or reading data according to the scheduling of the storage management node, such as a Block storage back end service node (BS).

Virtual block Device (Device): virtualized storage devices that emulate the functionality of a physical hard drive such that virtual machines can access storage space like physical hard drives, thereby reading or writing data in cloud disks.

Creation of cloud disk (creation disk): the system performs resource scheduling according to the size and type of the cloud disk designated by the user and the resource use condition of the current cluster, and comprises the following steps: creating a new cloud disk on the selected storage service node, and setting related metadata such as the size, type, state and the like of the storage volume; the newly created cloud disk is registered into the metadata service of the cluster so that the user can find and use it.

Mounting cloud disk (open disk): the cloud disk is mounted to a server instance, such as an elastic computing service (Elastic Compute Service, ECS) instance, so that the cloud disk serves as a data disk for the server instance.

Garbage collection (Garbage Collection, GC): an automatic memory management technique. In a computer system, when objects are no longer used by a program, garbage collector automatically recovers the memory space occupied by the objects for reuse. In a clustered environment, GC tasks are designed to purge useless blocks or objects of data to free up storage space and increase resource utilization.

In the related art, a storage cluster adopts a centralized BM architecture. Fig. 1 shows an architecture diagram of a cloud-based management and control system in the related art. As shown in fig. 1, the cloud management and control system generally includes a Client (Client), a cluster management node RM, and at least one storage cluster (fig. 1 is exemplified by storage cluster 1 and storage cluster 2), and the storage cluster includes a storage management node BM and a plurality of storage service nodes BS (fig. 1 is exemplified by BS1, BS2, BS3, … …).

On the one hand, in the centralized BM architecture, one storage cluster deploys a group of BMs no matter the size of the storage cluster, and a single group of BMs manages metadata of clients, BSs and cloud disks related to the whole storage cluster and is responsible for scheduling cloud disk fragments (segments) and distributing background tasks such as garbage collection and the like to complete all management and control work at the cluster level. The single-group BM adopts a main and two-standby service mode, namely, in a normal working state, one main BM is used for completing all control work of the storage cluster, and when the main BM is down, the standby BM is used for taking over the control work of the storage cluster. Therefore, for a single cluster, only one BM works, and other BMs are in a dormant state, so that resource waste is caused. Moreover, when a group of BMs of one storage cluster is abnormal, there is no way for another storage cluster to support the abnormal storage cluster even if there are sufficient machines.

On the other hand, the centralized management mode has a relatively simple architecture, and the client can acquire the position of Segment through the BM and then read and write data on the corresponding BS. The centralized BM is responsible for the management, scheduling and operation and maintenance related work of the whole storage cluster, and can work more efficiently under the conditions of less number of single-cluster cloud disks, limited number of segments and smaller cluster scale. However, with the increase of the scale, the single cluster is expected to support more segments and cloud disks, so that 1000 tens of thousands of segments are required to be supported for calculation in the future, that a single BM is required to provide 360G memory, and it is obvious that the existing server cannot support the requirement, and once the centralized BM is down, the management and control operation of the whole cluster granularity fails until one of three BMs can be started to provide services normally. Thus, the explosion radius of a storage cluster of a centralized BM architecture is all cloud disks of the entire storage cluster.

The embodiment of the application aims at providing a cloud management and control scheme based on distributed BM. Fig. 2 shows an architecture diagram of a cloud-based management and control system 200 according to an embodiment of the present application. As shown in fig. 2, cloud management and control system 200 includes cluster management and control node 201 and one or more clusters 202.

Wherein the cluster management and control node 201 is a service node for managing each cluster 202. Illustratively, cluster management node 201 may be a resource management node RM. Illustratively, when a computing node creates a corresponding storage resource, one of the clusters 202 is determined as the target cluster by the cluster management node 201 managing the respective storage cluster.

Cluster 202 is a storage cluster. For one cluster 202, its deployment includes multiple storage management nodes and multiple storage service nodes. Illustratively, the storage management node may store a back-end control node (BM) for the block, as shown in fig. 2, with a plurality of storage management nodes, such as BM1, BM2, BM3, and BM4. The storage service node may store a back-end service node (BS) for the block, as shown in fig. 2, a plurality of storage service nodes such as BS1, BS2, BS3, BS4, BS5, BS6, BS7, and BS8. It should be noted that, in the embodiment of the present application, the number of clusters 202, and the number of storage management nodes and storage service nodes deployed by each cluster are not specifically limited.

Each storage management node is configured to manage a plurality of cloud disk sets (portions). For example, BM1 may manage cloud disk set 1 and cloud disk set 8 (for cloud disk set list 1), BM2 may manage cloud disk set 2 and cloud disk set 3 (for cloud disk set list 2), BM3 may manage cloud disk set 4 and cloud disk set 6 (for cloud disk set list 3), and BM4 may manage cloud disk set 5 and cloud disk set 7 (for cloud disk set list 4). For one cluster 202, a plurality of storage management nodes and a plurality of cloud disk collection lists (partition lists) have a one-to-one mapping relationship, that is, a cloud disk collection and a storage management node have a mapping relationship.

The cloud disk set is a logic concept, is a set of a lot of cloud disks, and a certain cloud disk can only belong to one cloud disk set. After the cloud disk set is created, the corresponding storage management node is required to load (load) to provide corresponding services. Illustratively, for logical simplicity, cloud disk collection (part) supports dynamic creation, but cannot be modified and deleted.

Storage management nodes manage their respective cloud disk collections, including, but not limited to: creating a cloud disk, and generating corresponding cloud disk identification information (Volume ID) for the created cloud disk; mounting a cloud disk (opening a disk), deleting Yun Pan, scheduling the cloud disk and the like based on the cloud disk identification information.

The cloud disk identification information comprises a Cluster identification (Cluster ID), a cloud disk set identification (Partition ID) and a cloud disk identification (Yun Pan) of the cloud disk set. Yun Pan the position on the set of cloud disks may be represented by a cloud disk number or a cloud disk random number, for one set of cloud disks, each cloud disk in the set of cloud disks having a unique cloud disk number or a unique cloud disk random number. Illustratively, the cloud disk identification information may further include a reserved field (or called a preset field), which may identify a reserved function or preset function of the cloud disk, or other information, etc.

For one cluster 202, its deployment may also include a Central Master (CM) for managing work responsible for bypassing from the global perspective of the cluster 202, including: managing the up-down lines of BMs in the cluster 202; managing the up-down lines of BSs in the cluster 202; and distributing and creating a cloud disk set, creating and configuring corresponding resources for the cloud disk set, and the like.

Illustratively, the cloud disk management and control system 200 may further include a client 203, which may be deployed in a computing cluster, and interact with the cluster management and control node 201 to obtain cloud disk identification information, so as to access a corresponding cloud disk, such as a mounted cloud disk, based on the cloud disk identification information. The client 203 may interact with the storage management node through an application program interface (Application Programming Interface, API) to obtain information of the storage service node where the Segment is located, and further read and write data on the storage service node.

An example of an application of the cloud management and control system 200 in performing a dynamic authoring process according to an embodiment of the present application is described below with reference to fig. 2. The wound flow mainly comprises the following steps:

(1) The cluster management and control node 201 selects one from a plurality of storage management and control nodes of the cluster 201 as a target storage management and control node (for example, BM 1), and the selection policy may be a random selection or a hash-based (hash) rule;

(2) The cluster management and control node 201 sends a cloud disk creation request to the BM 1;

(3) BM1 may be configured with a proxy or a management component (proxy), the proxy of BM1 selecting one from a plurality of cloud disk sets as a target cloud disk set (e.g., cloud disk set 8);

(4) The BM1 completes creation of the target cloud disk, generates cloud disk identification information (Volume ID) of the target cloud disk, and returns information of successful creation of the target cloud disk and the cloud disk identification information of the target cloud disk to the cluster management and control node 201 through the original path.

An example of an application of the cloud-based management and control system 200 in performing a dynamic open flow according to an embodiment of the present application is described below with reference to fig. 2. The opening flow mainly comprises:

(1) The client 203 queries the cluster management and control node 201 for the information of the target cluster, the cloud disk identification information of the target cloud disk and the information of the target storage management and control node through the device identification information (DEVID) of the virtual block device;

(2) The Cluster management and control node 201 analyzes corresponding cloud disk identification information (Volume ID) based on the device identification information given by the client 203, obtains a Cluster identification (Cluster ID), a cloud disk set identification (Partition ID) and a cloud disk identification (Yun Pan) by analyzing the Volume ID, and then determines a storage management and control node (for example, BM 1) corresponding to the cloud disk set identification (Partition ID) as a target storage management and control node based on a mapping relationship between the cloud disk set and the storage management and control node;

(3) The cluster management node 201 returns to the client 203 a cluster identification (e.g. cluster name) of the target cluster, an internet protocol (Internet Protocol, IP) address of BM1, a Volume ID;

(4) The client 203 sends a cloud disk mounting request for the target cloud disk to the BM1 based on the IP address of the BM 1;

(5) And the BM1 responds to the cloud disk mounting request to mount the target cloud disk.

An application example of cloud disk scheduling by the cloud disk control system according to the embodiment of the present application is described below with reference to fig. 3. In this application example, the cloud disk scheduling process mainly includes:

(1) Each BM in a cluster obtains traffic information for each BS in the cluster. The acquisition mode may be that each BS in the cluster sends heartbeat to a central control node (CM) and reports the flow information of each BS; the acquisition mode may also be that each BM in the cluster sends a slow heartbeat to each BS in the cluster, so that each BM can obtain flow information of each BS, and the slow heartbeat is sent once every 10 seconds, for example;

(2) Each BM performs cloud disk scheduling in turn according to a preset sequence, for example, BM1, BM2, BM3, and BM4 perform cloud disk scheduling in turn, including traffic balancing scheduling (dynamic or static), so as to schedule cloud disk fragments (segments) of a cloud disk to multiple storage service nodes. The CM is used for controlling each BM to execute cloud disk scheduling in turn according to a preset sequence from a global angle of the cluster according to a global coordination mechanism, and performs certain interference, and if the CM is down, each BM executes cloud disk scheduling in turn all the time.

Another application example of cloud disk scheduling by the cloud disk control system according to the embodiment of the present application is described below with reference to fig. 4. In this application example, the cloud disk scheduling process mainly includes:

(1) Each BS in the cluster is divided into a plurality of groups to form a storage service node group, each BM corresponds to each storage service node group one by one, and the BM has the jurisdiction of the storage service node group corresponding to the BM and initiates downtime operation to the BS of the storage service node group. For example: BS1 and BS2 form group 1, bm1 corresponds to group 1, having jurisdictions for BS1 and BS 2; BS3 and BS4 form group 2, with BS2 corresponding to group 2, having jurisdictions over BS3 and BS 4; BS5 and BS6 form group 3, with BS3 corresponding to group 3, having jurisdictions over BS5 and BS 6; BS7 and BS8 form group 4, with BS4 corresponding to group 4, having jurisdictions over BS7 and BS 8;

(2) The BM sends a quick heartbeat to the BS corresponding to the BM, for example, once every 200 milliseconds, so as to obtain the running state information of the corresponding BS, and the running state information is used for performing fault judgment (downtime judgment) on the BS;

(3) If the BM judges that the BS in the group has a fault BS, that is, a BS with a downtime, the BS is subjected to downtime, for example, the BM1 judges that the BS2 in the group 1 is downtime, and the BM1 can perform downtime on the BS to realize downtime scheduling (also called fault scheduling or abnormal scheduling).

An application example of heartbeat deployment of the cloud-based management system of the embodiments of the present application is described below in conjunction with fig. 2. As shown in fig. 2, for a cluster 202, each BM reports heartbeat to an RM, and reports respective running state information, that is, determines that the BM has not failed; the information setting of the cluster granularity is reported by one BM in the cluster (e.g., defaulting to BM 1). The central control node reports heartbeat to each BM, acquires running state information of each BM, such as whether fault exists or not, and acquires load conditions of each cloud disk set on the BM.

The cloud management and control system of the embodiment of the application provides a cloud management and control scheme based on distributed storage management and control nodes, which horizontally expands the storage management and control nodes in one storage cluster, namely the storage cluster comprises a plurality of storage management and control nodes, and each storage management and control node is responsible for management and control of a plurality of cloud disk sets. On one hand, each storage control node works simultaneously, so that resource waste caused by a main multi-standby architecture is avoided, and the resource utilization rate is improved; on the other hand, when one storage control node fails, other storage control nodes take over the cloud disk set which is in charge of the storage control node, and in the process, the affected cloud disks in the storage cluster are only part of the cloud disks, so that the explosion radius is far smaller than that of the centralized BM architecture; on the other hand, the method can support large-scale clusters, and the scheme of the embodiment of the application can support 200 or even 1000 clusters with larger scale through simulation; in still another aspect, the embodiment of the application also provides an effective cluster resource scheduling scheme for a cloud disk control architecture of a distributed storage control node, so that the cloud disk scheduling is not affected by architecture update.

It should be noted that, the application scenario provided in the embodiment of the present application is for convenience of understanding, and the application scenario of the embodiment of the present application is not specifically limited. In addition, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the relevant data is required to comply with relevant laws and regulations and standards of relevant countries and regions, and is provided with corresponding operation entries for the user to select or edit authorization or rejection.

The following describes the technical solution of the present application and how the technical solution of the present application solves the foregoing technical problems in detail with specific embodiments. The specific embodiments illustrated may be combined with one another and the same or similar concepts or processes may not be described in detail in some embodiments.

Fig. 5 shows a flowchart of a cloud-management-control method according to an embodiment of the present application. The cloud management and control method can be applied to cluster management and control nodes. As shown in fig. 5, the cloud-point management method includes:

Step S501: acquiring cloud disk identification information of a target cloud disk; the cloud disk identification information is used for identifying a target cloud disk, a target storage management and control node corresponding to the target cloud disk and a target cluster; the target cluster comprises a plurality of storage control nodes, and the storage control nodes comprise target storage control nodes; the target storage management node is used for creating a target cloud disk on at least one storage service node of the target cluster and generating cloud disk identification information.

In the embodiment of the present application, the target cluster is a storage cluster, for example, the cluster 202 shown in fig. 2. The target cluster includes a plurality of storage management nodes and a plurality of storage service nodes. For example, as shown in FIG. 2, the storage management node may store backend control nodes for blocks, as shown in FIG. 2, a plurality of storage management nodes, such as BM1, BM2, BM3, and BM4. The storage service node may store back-end service nodes for blocks, as shown in fig. 2, a plurality of storage service nodes are, for example, BS1, BS2, BS3, BS4, BS5, BS6, BS7, and BS8. The plurality of storage management nodes comprise target storage management nodes, wherein the target storage management nodes are storage management nodes for creating target cloud disks, and cloud disk identification information (Volume ID) of the target cloud disks is generated.

Step S502: and determining a target cluster, a target storage management node and a target cloud disk according to the cloud disk identification information.

The cloud disk identification information comprises: information for determining the target Cluster, such as Cluster identification (Cluster ID); information for determining a target storage management node; and the information is used for determining the target cloud disk. And determining a target cluster, a target storage management and control node and a target cloud disk corresponding to the target cloud disk based on the cloud disk identification information of the target cloud disk, so that the target storage management and control node can perform corresponding management and control on the target cloud disk, including but not limited to mounting, deleting, scheduling and the like.

In one embodiment, each storage management node is configured to manage a plurality of cloud disk sets (parts), where cloud disk identification information includes a cloud disk identifier, a cloud disk set identifier, and a cluster identifier, and in step S501, determining, according to the cloud disk identifier information, a target cluster, a target storage management node, and a target cloud disk may include: respectively determining a target cluster, a target cloud disk set to which the target cloud disk belongs and the position of the target cloud disk on the target cloud disk set by utilizing the cluster identifier, the cloud disk set identifier and the cloud disk identifier of the target cloud disk; and determining the storage control node corresponding to the target cloud disk set as a target storage control node based on the mapping relation between the cloud disk set and the storage control node.

The partition is a logic concept, and is a collection of a batch of cloud disks, and a certain cloud disk can only belong to one partition. After the partition is created, the corresponding storage management node needs to load (load) to provide the corresponding service. Illustratively, for logic simplicity, part supports dynamic creation, but cannot be modified and deleted.

Each storage management node is used for managing a plurality of parts, and for one cluster, the parts and the storage management nodes have a mapping relation. Therefore, the part identification of the target cloud disk can be utilized to determine the part to which the target cloud disk belongs, and further, the information of the corresponding target storage management node is obtained based on the mapping relation between the part and the storage management node. Illustratively, a part has a mapping relationship with the IP address of the BM, and based on the part identifier of the target cloud disk, the IP address of the target BM may be determined.

In one implementation, the method of the embodiment of the present application may further include: selecting a target storage control node from a plurality of storage control nodes; sending a cloud disk creation request to a target storage management and control node so that the target storage management and control node creates a target cloud disk on at least one storage service node and generates cloud disk identification information; and receiving cloud disk identification information returned by the target storage management and control node.

Illustratively, when a computing node creates a corresponding storage resource, one of the clusters is decided by the cluster management and control node as a target cluster; the cluster management and control node can select one from a plurality of storage management and control nodes of the target cluster as the target storage management and control node, and the selection strategy can be selected randomly or based on a hash rule, which is not particularly limited in the embodiment of the present application; further, the cluster management and control node sends a cloud disk creation request to the target storage management and control node, the target storage management and control node completes creation of the target cloud disk, cloud disk identification information (Volume ID) of the target cloud disk is generated, and the Volume ID of the target cloud disk is returned to the cluster management and control node.

In one embodiment, the client may query the cluster management node for information of the target cluster, cloud disk identification information of the target cloud disk, and information of the target storage management node through device identification information (DEVID) of the virtual block device.

Illustratively, the Cluster management and control node may parse the corresponding Volume ID based on the device identification information, obtain a Cluster identifier (Cluster ID), a cloud disk set identifier (Partition ID), and a cloud disk identifier (Yun Pan) by parsing the Volume ID, then determine an IP address of the target storage management and control node corresponding to the cloud disk set identifier (Partition ID) based on a mapping relationship between the cloud disk set and the storage management and control node, and return the Cluster identifier (for example, a Cluster name) of the target Cluster, the IP address of the target storage management and control node, and the Volume ID to the client; based on the information, the client may send a management and control request for the target cloud disk, such as a cloud disk mounting request, a Yun Pan deletion request, and the like, to the target storage management and control node.

Fig. 6 shows a flowchart of a cloud-management-control method according to an embodiment of the present application. The cloud disk management and control method can be applied to storage management and control nodes, and the embodiment of the application takes the target storage management and control node as an example, namely the storage management and control node of the target cloud disk is created. As shown in fig. 6, the cloud-point management method includes:

step S601: creating a target cloud disk on at least one storage service node of the target cluster in response to the Yun Pan create request; the target cluster comprises a plurality of storage control nodes, and the storage control nodes comprise target storage control nodes;

step S602: generating cloud disk identification information of a target cloud disk; the cloud disk identification information is used for identifying a target cloud disk, a target storage management and control node and a target cluster.

Each storage management node is used for managing and controlling a plurality of cloud disk sets (parts), and the cloud disk identification information comprises a cluster identification, a part identification and a cloud disk identification; the cluster identifier of the target cloud disk is used for determining a target cluster; the part identifier of the target cloud disk is used for identifying the target part to which the target cloud disk belongs, and the storage control node corresponding to the target part is determined to be used as the target storage control node based on the mapping relation between the part and the storage control node; the cloud disk identification of the target cloud disk is used for determining the position of the target cloud disk on the target part.

In step S601, creating a target cloud disk on at least one storage service node of a target cluster, including: determining a target part from a plurality of parts corresponding to the target storage management and control node based on a preset load strategy; and selecting a target cloud disk from the target part, and creating the target cloud disk on at least one storage service node.

Illustratively, the storage management node selects one of the plurality of parts as the target part in response to the Yun Pan create request. The selection policy may be a load balancing policy, that is, according to the number of cloud disks that have been created by each part, a part with a small load is selected as a target part. The selection policy may also be a random selection policy, or the selection policy may also be a policy based on distance preference, that is, selecting a part that is closer to the network topology of the computing node as the target part, which is not limited in the embodiment of the present application.

The storage management node may be configured with an agent or management component, configured to select one of the plurality of parts as a target part, and send a corresponding operation request to the corresponding storage management node. The application examples of the above disc creation process can be specifically referred to, and will not be described herein.

In one implementation, the method of the embodiment of the present application may further include: and responding to a cloud disk mounting request aiming at the target cloud disk, and mounting the target cloud disk. The application examples of the dynamic open flow can be specifically referred to, and will not be described herein.

According to the cloud management and control method, storage management and control nodes in one storage cluster are horizontally expanded, namely one storage cluster comprises a plurality of storage management and control nodes, and each storage management and control node is responsible for management and control of a plurality of groups of cloud disks (for example, a plurality of cloud disk sets). On one hand, each storage control node works simultaneously, so that resource waste caused by a main multi-standby architecture is avoided, the resource utilization rate is improved, and a large-scale cluster can be supported; on the other hand, in the case of a failure of a certain storage management node, the affected cloud disk in the storage cluster is only a part, so the explosion radius is far smaller than that of the centralized BM (one master multiple slave) architecture.

In an embodiment, the method performed by the target storage management node in the embodiment of the present application may further include: acquiring flow information of each storage service node in a target cluster; and responding to the round of cloud disk scheduling of the target storage management and control node, and scheduling at least one cloud disk corresponding to the target storage management and control node based on the flow information of each storage service node.

Illustratively, each storage management node in the target cluster obtains traffic information for each storage service node of the target cluster. The acquisition mode can be that each storage service node in the target cluster sends heartbeat to a central control node (CM) and reports the flow information of each storage service node; the obtaining mode may be that each storage management node in the target cluster sends a slow heartbeat to each storage service node in the target cluster, so that the target storage management node can obtain flow information of each storage service node in the target cluster. Further, each storage control node executes cloud disk scheduling in turn according to a preset sequence, and when the target storage control node is in turn to execute the cloud disk scheduling, the target storage control node can execute the cloud disk scheduling on each part managed by the target storage control node, including dynamic flow balance scheduling or static flow balance scheduling.

In one implementation manner, each storage service node of the target cluster is divided into a plurality of storage service node groups, and the plurality of storage control nodes are in one-to-one correspondence with the plurality of storage service node groups, and the method executed by the target storage control node in the embodiment of the present application may further include: judging whether a fault storage service node exists in the storage service node group corresponding to the target storage management node based on the running state information of each storage service node in the storage service node group corresponding to the target storage management node; and responding to the existence of the fault storage service node, and executing downtime operation on the fault storage service node. See specifically the cloud disk scheduling application example in connection with fig. 4, and will not be described in detail herein.

In one implementation, the method performed by the target storage management node in the embodiment of the present application may further include: and responding to the fault storage management and control node in the target cluster, and managing and controlling each cloud disk corresponding to the fault storage management and control node. That is, at a failure of one storage management node, the other storage management node takes over its responsible cloud disk.

According to the cloud disk control method, an effective cluster resource scheduling scheme can be provided for the cloud disk control framework of the distributed storage control node, so that cloud disk scheduling is not affected by framework updating.

Corresponding to the method provided by the embodiment of the present application, the embodiment of the present application provides a cloud management and control device, where the cloud management and control device is applied to a cluster management and control node, and may include: the cloud disk identification information acquisition module is used for acquiring cloud disk identification information of a target cloud disk; the cloud disk identification information is used for identifying the target cloud disk and a target storage management node and a target cluster corresponding to the target cloud disk; the target cluster comprises a plurality of storage control nodes, and the storage control nodes comprise the target storage control nodes; the target storage management node is used for creating the target cloud disk on at least one storage service node of the target cluster and generating the cloud disk identification information; and the target storage management and control node determining module is used for determining the target cluster, the target storage management and control node and the target cloud disk according to the cloud disk identification information.

In one embodiment, each storage management node is configured to manage a plurality of cloud disk sets, where the cloud disk identification information includes a cloud disk identifier, a cloud disk set identifier, and a cluster identifier, and the target storage management node determining module is specifically configured to: respectively determining the target cluster, a target cloud disk set to which the target cloud disk belongs and the position of the target cloud disk on the target cloud disk set by utilizing the cluster identifier, the cloud disk set identifier and the cloud disk identifier of the target cloud disk; and determining a storage control node corresponding to the target cloud disk set as the target storage control node based on the mapping relation between the cloud disk set and the storage control node.

In one embodiment, the apparatus may further include: the target storage control node selection module is used for selecting the target storage control node from the storage control nodes; the cloud disk creation request sending module is used for sending a cloud disk creation request to the target storage management and control node so that the target storage management and control node creates the target cloud disk on the at least one storage service node and generates the cloud disk identification information; and the cloud disk identification information receiving module is used for receiving the cloud disk identification information returned by the target storage management and control node.

Corresponding to the method provided by the embodiment of the present application, the embodiment of the present application provides a cloud management and control device, where the cloud management and control device is applied to a target storage management and control node, may include: a target cloud disk creation module for creating a target cloud disk on at least one storage service node of the target cluster in response to the Yun Pan creation request; the target cluster comprises a plurality of storage control nodes, and the storage control nodes comprise the target storage control nodes; the cloud disk identification information generation module is used for generating cloud disk identification information of the target cloud disk; the cloud disk identification information is used for identifying the target cloud disk, the target storage management and control node and the target cluster.

In one embodiment, each storage management node is configured to manage a plurality of cloud disk sets, where the cloud disk identification information includes a cluster identifier, a cloud disk set identifier, and a cloud disk identifier; the cluster identifier of the target cloud disk is used for determining the target cluster; the cloud disk set identifier of the target cloud disk is used for identifying the target cloud disk set to which the target cloud disk belongs, so that the storage management node corresponding to the target cloud disk set is determined to serve as the target storage management node based on the mapping relation between the cloud disk set and the storage management node; the cloud disk identification of the target cloud disk is used for determining the position of the target cloud disk on the target cloud disk set.

In one embodiment, the target cloud disk creation module is specifically configured to: determining a target cloud disk set from a plurality of cloud disk sets corresponding to the target storage management and control node based on a preset load strategy; selecting the target cloud disk from the target cloud disk set, and creating the target cloud disk on the at least one storage service node.

In one embodiment, the apparatus may further include a mounting module configured to mount the target cloud disk in response to a cloud disk mounting request for the target cloud disk.

In an embodiment, the plurality of storage management and control nodes alternately execute cloud disk scheduling according to a preset sequence, and the apparatus may further include: the flow information acquisition module is used for acquiring flow information of each storage service node in the target cluster; and the scheduling module is used for scheduling at least one cloud disk corresponding to the target storage management node based on the flow information of each storage service node in response to the round of cloud disk scheduling of the target storage management node.

In one embodiment, each storage service node of the target cluster is divided into a plurality of storage service node groups, and the plurality of storage management nodes are in one-to-one correspondence with the plurality of storage service node groups, and the apparatus may further include: the judging module is used for judging whether a fault storage service node exists in the storage service node group corresponding to the target storage control node based on the running state information of each storage service node in the storage service node group corresponding to the target storage control node; and the downtime operation module is used for responding to the existence of the fault storage service node and executing downtime operation on the fault storage service node.

In one embodiment, the device further includes a failure management and control module, configured to respond to the existence of a failure storage management and control node in the target cluster, and manage each cloud disk corresponding to the failure storage management and control node.

The functions of each module in each device of the embodiments of the present application may be referred to the corresponding descriptions in the above methods, and have corresponding beneficial effects, which are not described herein.

Fig. 7 is a block diagram of an electronic device used to implement an embodiment of the present application. As shown in fig. 7, the electronic device includes: a memory 701 and a processor 702, the memory 701 storing a computer program executable on the processor 702. The processor 702, when executing the computer program, implements the methods of the embodiments described above. The number of memories 701 and processors 702 may be one or more.

The electronic device further includes: and the communication interface 703 is used for communicating with external equipment and performing data interaction transmission.

If the memory 701, the processor 702, and the communication interface 703 are implemented independently, the memory 701, the processor 702, and the communication interface 703 may be connected to each other and perform communication with each other through buses. The bus may be an industry standard architecture (Industry Standard Architecture, ISA) bus, an external device interconnect (Peripheral Component Interconnect, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in fig. 7, but not only one bus or one type of bus.

Alternatively, in a specific implementation, if the memory 701, the processor 702, and the communication interface 703 are integrated on a chip, the memory 701, the processor 702, and the communication interface 703 may communicate with each other through internal interfaces.

Embodiments of the present application provide a computer readable storage medium storing a computer program which, when executed by a processor, implements a method provided in any of the embodiments of the present application.

The embodiment of the application also provides a chip, which comprises a processor, and is used for calling the instructions stored in the memory from the memory and running the instructions stored in the memory, so that the communication device provided with the chip executes the method provided by any embodiment of the application.

The embodiment of the application also provides a chip, which comprises: the input interface, the output interface, the processor and the memory are connected through an internal connection path, the processor is used for executing codes in the memory, and when the codes are executed, the processor is used for executing the method provided by any embodiment of the application.

It should be appreciated that the processor may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processing, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field Programmable gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or any conventional processor or the like. It is noted that the processor may be a processor supporting an advanced reduced instruction set machine (Advanced RISC Machines, ARM) architecture.

Alternatively, the memory may include a read-only memory and a random access memory, and may also include a nonvolatile random access memory. The memory may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), programmable ROM (PROM), erasable Programmable ROM (EPROM), electrically Erasable EPROM (EEPROM), or flash Memory, among others. Volatile memory can include random access memory (Random Access Memory, RAM), which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available. For example: static RAM (SRAM), dynamic RAM (Dynamic Random Access Memory, DRAM), synchronous DRAM (SDRAM), double Data Rate Synchronous DRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), and Direct RAM (DRRAM).

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions in accordance with the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. Computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another.

It should be noted that, user information (including, but not limited to, user equipment information, user personal information, user operation information, etc.) and data (including, but not limited to, data for processing, analyzed data, stored data, presented data, etc.) and the like, which are information and data authorized by a user or sufficiently authorized by each party, and the collection, use and processing of related information and data need to comply with related laws and regulations and standards of related countries and regions, and are provided with corresponding operation entries for the user to select authorization or rejection.

In the description of the present specification, a description referring to the terms "one embodiment," "some embodiments," "examples," "particular examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.

Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.

Any process or method description in a flowchart or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process. And the scope of the preferred embodiments of the present application includes additional implementations in which functions may be performed in a substantially simultaneous manner or in an opposite order from that shown or discussed, including in accordance with the functions that are involved.

Logic and/or steps represented in the flow diagrams or otherwise described herein, e.g.: may be considered a ordered listing of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.

It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the methods of the embodiments described above may be performed by a program that, when executed, comprises one or a combination of the steps of the method embodiments, instructs the associated hardware to perform the method.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules described above, if implemented in the form of software functional modules and sold or used as a stand-alone product, may also be stored in a computer-readable storage medium. The storage medium may be a read-only memory, a magnetic or optical disk, or the like.

The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think of various changes or substitutions within the technical scope of the present application, and these should be covered in the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (15)

1. A cloud-based management and control method applied to cluster management and control nodes, the method comprising:

acquiring cloud disk identification information of a target cloud disk; the cloud disk identification information is used for identifying the target cloud disk and a target storage management node and a target cluster corresponding to the target cloud disk; the target cluster comprises a plurality of storage control nodes, and the storage control nodes comprise the target storage control nodes; the target storage management node is used for creating the target cloud disk on at least one storage service node of the target cluster and generating the cloud disk identification information;

and determining the target cluster, the target storage management and control node and the target cloud disk according to the cloud disk identification information.

2. The method of claim 1, wherein each storage management node is configured to manage a plurality of cloud disk sets, the cloud disk identification information includes a cloud disk identification, a cloud disk set identification, and a cluster identification, and determining the target cluster, the target storage management node, and the target cloud disk according to the cloud disk identification information includes:

Respectively determining the target cluster, a target cloud disk set to which the target cloud disk belongs and the position of the target cloud disk on the target cloud disk set by utilizing the cluster identifier, the cloud disk set identifier and the cloud disk identifier of the target cloud disk;

and determining a storage control node corresponding to the target cloud disk set as the target storage control node based on the mapping relation between the cloud disk set and the storage control node.

3. The method of claim 1, further comprising:

selecting the target storage management node from the plurality of storage management nodes;

sending a cloud disk creation request to the target storage management and control node so that the target storage management and control node creates the target cloud disk on the at least one storage service node and generates the cloud disk identification information;

and receiving the cloud disk identification information returned by the target storage management and control node.

4. A cloud-based management and control method applied to a target storage management and control node, the method comprising:

creating a target cloud disk on at least one storage service node of the target cluster in response to the Yun Pan create request; the target cluster comprises a plurality of storage control nodes, and the storage control nodes comprise the target storage control nodes;

Generating cloud disk identification information of the target cloud disk; the cloud disk identification information is used for identifying the target cloud disk, the target storage management and control node and the target cluster.

5. The method of claim 4, wherein each storage management node is configured to manage a plurality of cloud disk sets, the cloud disk identification information including a cluster identification, a cloud disk set identification, and a cloud disk identification; the cluster identifier of the target cloud disk is used for determining the target cluster; the cloud disk set identifier of the target cloud disk is used for identifying the target cloud disk set to which the target cloud disk belongs, so that the storage management node corresponding to the target cloud disk set is determined to serve as the target storage management node based on the mapping relation between the cloud disk set and the storage management node; the cloud disk identification of the target cloud disk is used for determining the position of the target cloud disk on the target cloud disk set.

6. The method of claim 5, wherein creating a target cloud disk on at least one storage service node of a target cluster comprises:

determining a target cloud disk set from a plurality of cloud disk sets corresponding to the target storage management and control node based on a preset load strategy;

Selecting the target cloud disk from the target cloud disk set, and creating the target cloud disk on the at least one storage service node.

7. The method of claim 4, further comprising:

and responding to a cloud disk mounting request aiming at the target cloud disk, and mounting the target cloud disk.

8. The method of any of claims 4 to 7, wherein the plurality of storage management nodes alternately perform cloud disk scheduling in a preset order, the method further comprising:

acquiring flow information of each storage service node in the target cluster;

and responding to the round of cloud disk scheduling of the target storage management and control node, and scheduling at least one cloud disk corresponding to the target storage management and control node based on the flow information of each storage service node.

9. The method of any of claims 4 to 7, wherein each storage service node of the target cluster is divided into a plurality of storage service node groups, the plurality of storage management nodes being in one-to-one correspondence with the plurality of storage service node groups, the method further comprising:

and judging whether a fault storage service node exists in the storage service node group corresponding to the target storage management node based on the running state information of each storage service node in the storage service node group corresponding to the target storage management node.

10. The method of claim 9, further comprising:

and responding to the existence of the fault storage service node, and executing downtime operation on the fault storage service node.

11. The method of any of claims 4 to 7, further comprising:

and responding to the fault storage control node in the target cluster, and controlling each cloud disk corresponding to the fault storage control node.

12. A cloud-based management system, comprising:

a cluster management node for performing the method of any of claims 1 to 3;

at least one cluster comprising a plurality of storage management nodes and a plurality of storage service nodes, the storage management nodes being adapted to perform the method of any of claims 4 to 11.

13. The cloud disk management and control system of claim 12, wherein each cluster further comprises a central management and control node, wherein the central management and control node is used for sending flow information of each storage service node in the cluster to each storage management and control node, and controlling each storage management and control node in the cluster to execute cloud disk scheduling in turn according to a preset sequence; and/or the central management and control node is used for creating a plurality of cloud disk sets for each storage management and control node distribution.

14. An electronic device comprising a memory, a processor and a computer program stored on the memory, the processor implementing the method of any one of claims 1 to 11 when the computer program is executed.

15. A computer readable storage medium having a computer program stored therein, which when executed by a processor, implements the method of any of claims 1 to 11.