WO2024087717A1 - 云上资源管理系统及其部署方法 - Google Patents

云上资源管理系统及其部署方法 Download PDF

Info

Publication number
WO2024087717A1
WO2024087717A1 PCT/CN2023/104680 CN2023104680W WO2024087717A1 WO 2024087717 A1 WO2024087717 A1 WO 2024087717A1 CN 2023104680 W CN2023104680 W CN 2023104680W WO 2024087717 A1 WO2024087717 A1 WO 2024087717A1
Authority
WO
WIPO (PCT)
Prior art keywords
management component
resource
basic
cluster
virtual instance
Prior art date
Application number
PCT/CN2023/104680
Other languages
English (en)
French (fr)
Inventor
江涛
温嘉佳
Original Assignee
华为云计算技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202310402195.1A external-priority patent/CN117978802A/zh
Application filed by 华为云计算技术有限公司 filed Critical 华为云计算技术有限公司
Publication of WO2024087717A1 publication Critical patent/WO2024087717A1/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network

Definitions

  • the present application relates to the field of resource management technology, and in particular to a cloud resource management system and a deployment method thereof.
  • resource pools are usually divided into two parts: the management area and the tenant area.
  • the management area resource pool and the tenant area resource pool are completely independent of each other.
  • Cloud services deployed in any type of resource pool are fixedly deployed in that type of resource pool throughout the life cycle, and there is no scenario of expanding to the other area for co-deployment or migration deployment.
  • the virtual instances created in the management area resource pool are used to deploy the resource management system, basic cloud services, operation and maintenance services, security services, etc. of the tenant area resource pool.
  • management area resource pool In order to cope with the performance impact of peak business traffic, the management area resource pool generally selects large-scale virtual machines and deploys large-scale business clusters.
  • This application provides a cloud resource management system and its deployment method.
  • the basic management component of this application only manages the basic resource group in the external resource cluster, effectively controls the resources occupied by the basic management component, and helps to reduce the construction cost of the cloud resource management system.
  • the technical solution provided by this application is as follows:
  • the present application provides a cloud resource management system, including: a cloud resource management node, an underlying management component and an external resource management component, wherein: the cloud resource management node is used to deploy the underlying management component, the underlying management component is used to manage the internal resource cluster, and the external resource management component is deployed based on the internal resource cluster; the external resource management component is used to manage the external resource cluster, and the external resource management component includes a basic management component, the basic management component is used to manage the basic resource group in the external resource cluster, create a virtual instance based on the basic resource group, and deploy the tenant's business in the virtual instance.
  • the basic management component only manages the basic resource groups in the external resource cluster, rather than all the resources in the external resource cluster. This controls the amount of resources in the external resource cluster managed by the basic management component, effectively controls the resources occupied by the basic management component, and reduces the probability that the scale of the basic management component will increase in tandem with the increase in the scale of the external resource cluster, which helps to reduce the construction cost of the cloud resource management system.
  • the basic management component by managing some resources in the external resource cluster through the basic management component, it provides conditions for the basic management component to expand the entire control plane of the cloud resource management system based on the external resource cluster, so that the cloud resource management system has the ability to control ultra-large-scale infrastructure resource pools.
  • the external resource management component also includes an extended management component.
  • the basic management component is specifically used to deploy the extended management component based on the basic resource group.
  • the basic management component specifically provides cloud services to tenants through the extended management component.
  • the extended management component is used to manage the extended resource group in the external resource cluster, create a virtual instance based on the extended resource group, and deploy the tenant's business in the virtual instance.
  • the basic management component is also used to deploy the extended management component based on the basic resource group, and the extended management component is used to manage the extended resource group in the external resource cluster.
  • the basic management component is specifically used to create a virtual instance based on the remaining resource groups in the basic resource group that are not occupied by the extended management component, and deploy the tenant's business in the virtual instance.
  • the above process of deploying extended management components can be regarded as the process of further deploying external resource management components when the business volume increases.
  • the resources occupied by basic management components can be effectively controlled, and the scale of the minimum startup unit can be limited.
  • the scale of the minimum startup unit does not need to increase in conjunction with the increase in the scale of the external resource cluster. It has stable and reliable characteristics and reduces the construction cost of the cloud resource management system.
  • the basic management component is specifically used to create a first virtual instance cluster based on the basic resource group, and deploy the extension management component in the first virtual instance cluster.
  • the basic management component is specifically used to create a first virtual instance cluster based on the basic resource group; the underlying management component is also used to deploy an extended management component in the first virtual instance cluster.
  • the resources used by the deployed underlying management components are isolated from the external resource cluster and/or from the internal resource cluster.
  • the underlying management components do not depend on the internal resource cluster and the external resource cluster.
  • the underlying management components can be regarded as a completely self-contained service of the cloud resource management node, which realizes the decoupling of the underlying management components from the internal resource cluster and the external resource cluster, and realizes the physical isolation of the underlying management components from the internal resource cluster and the external resource cluster, which can achieve the effect of fault isolation and avoid the spread of faults.
  • the complete cloud resource management system can still be rebuilt based on the underlying management component.
  • the internal resource cluster and the external resource cluster can also be isolated from each other.
  • the underlying management components may be deployed on multiple cloud resource management nodes so that the underlying management components have disaster recovery capabilities.
  • the underlying management component is specifically used to create a second virtual instance cluster based on the internal resource cluster, and deploy the basic management component in the external resource management component in the second virtual instance cluster.
  • the basic management component may include at least one of basic public services and essential middleware.
  • public services are services that can be shared by cloud services that can be provided by the cloud resource management system through an external resource cluster.
  • public services can be database services.
  • Essential middleware is middleware that is required to be used by cloud services that can be provided by the cloud resource management system through an external resource cluster.
  • essential middleware can be cache and message queue.
  • the basic management component may not include at least one of the basic public services and the required middleware.
  • the underlying management component is also used to deploy at least one of the basic public services and the required middleware based on the internal resource cluster.
  • the implementation process includes: the underlying management component creates a third virtual instance cluster based on the internal resource cluster, and deploys at least one of the basic public services and the required middleware in the third virtual instance cluster.
  • the second virtual instance cluster and the third virtual instance cluster can be an instance cluster created by the underlying management component at one time based on all internal resource clusters, or can be an instance cluster created by the underlying management component multiple times and each time based on a part of the resources in the internal resource cluster. This application does not make any specific limitations on this.
  • the subsystem including the underlying management components and the basic management components can be regarded as the most basic management subsystem in the cloud resource management system.
  • the most basic management subsystem is the smallest scale form that the cloud resource management system has to maintain normal operation when the tenant business volume is the least. Therefore, the most basic management subsystem is also called the minimum startup unit of the cloud resource management system.
  • the minimum startup unit includes the most basic and completely self-contained computing and storage root services, the internal resource cluster managed by the root service, the basic public services and necessary middleware issued on the basis of the internal resource cluster, and the basic management components capable of managing external resource clusters of a specific scale.
  • the minimum startup unit exists independently of the external resource cluster, realizes complete decoupling from the external resource cluster, and plays the role of fault isolation from the external resource cluster.
  • the scale of the minimum startup unit is controlled and will not grow synchronously with the growth of the capacity of the external resource cluster, which effectively controls the cost of the cloud resource management system.
  • This minimum startup unit is equivalent to the basic input and output system of a computer. It is the most solid and reliable fortress of the entire cloud resource management system. It can complete the boot and initialization of the entire cloud resource management system at the lowest cost, and can expand a complete cloud resource management system based on this minimum startup unit, ensuring that after a serious disaster occurs, it can still be used as a basis to gradually rebuild the complete cloud resource management system.
  • the external resource management component may include multiple extended management components, any two extended management components may be isolated from each other, and/or the basic management component and any extended management component may be isolated from each other.
  • the external resource management component may adopt a grid architecture, that is, the entire external management component adopts a multi-service grid form, and the basic management component and each extended management component are an independent service grid.
  • the propagation of faults between the extended management components can be avoided, thereby ensuring the reliability of the extended management components.
  • the basic management component and the extended management component are isolated from each other, the propagation of faults between the extended management component and the basic management component can be avoided, thereby ensuring the reliability of the extended management component and the basic management component.
  • the virtual instance is a container or a virtual machine.
  • the present application provides a deployment method for a cloud resource management system, which is applied to a cloud resource management system.
  • the cloud resource management system includes a cloud resource management node, an underlying management component and an external resource management component.
  • the method includes: deploying the underlying management component on the cloud resource management node; the underlying management component manages the internal resource cluster, and deploying the external resource management component based on the internal resource cluster; the external resource management component manages the external resource cluster, wherein the external resource management component includes a basic management component, and the basic management component manages the external resource cluster
  • the basic resource group in the group creates a virtual instance based on the basic resource group and deploys the tenant's business in the virtual instance.
  • the external resource management component also includes an extended management component
  • the method also includes: the basic management component deploys the extended management component based on the basic resource group; the extended management component manages the extended resource group in the external resource cluster, creates a virtual instance based on the extended resource group, and deploys the tenant's business in the virtual instance.
  • the external resource management component also includes an extended management component
  • the method also includes: the basic management component deploys the extended management component based on the basic resource group, and the extended management component is used to manage the extended resource group in the external resource cluster; the basic management component creates a virtual instance based on the remaining resource groups in the basic resource group that are not occupied by the extended management component, and deploys the tenant's business in the virtual instance.
  • the basic management component deploys the extended management component based on the basic resource group, including: the basic management component creates a first virtual instance cluster based on the basic resource group, and deploys the extended management component in the first virtual instance cluster.
  • the basic management component deploys the extended management component based on the basic resource group, including: the basic management component creates a first virtual instance cluster based on the basic resource group; and the underlying management component deploys the extended management component in the first virtual instance cluster.
  • resources used by the deployment underlying management components are isolated from external resource clusters and/or isolated from internal resource clusters.
  • the underlying management component has disaster recovery capabilities.
  • the underlying management component deploys a basic management component of the external resource management component based on the internal resource cluster, including: the underlying management component creates a second virtual instance cluster based on the internal resource cluster, and deploys the external resource management component in the second virtual instance cluster.
  • the basic management component includes at least one of basic public services and essential middleware.
  • the method further includes: the underlying management component deploys at least one of basic public services and necessary middleware based on the internal resource cluster.
  • the underlying management component deploys at least one type of basic public services and middleware based on the internal resource cluster, including: the underlying management component creates a third virtual instance cluster based on the internal resource cluster, and deploys at least one type of basic public services and required middleware in the third virtual instance cluster.
  • the external resource management component includes a plurality of extended management components, any two extended management components are isolated from each other, and/or the basic management component and any extended management component are isolated from each other.
  • the virtual instance is a container or a virtual machine.
  • the present application provides a computing device including a memory and a processor, wherein the memory stores program instructions, and the processor runs the program instructions to execute the method provided in the second aspect of the present application and any possible implementation thereof.
  • the present application provides a computing device cluster, including multiple computing devices, the multiple computing devices including multiple processors and multiple memories, the multiple memories storing program instructions, and the multiple processors running the program instructions, so that the computing device cluster executes the method provided in the second aspect of the present application and any possible implementation thereof.
  • the present application provides a computer-readable storage medium, which is a non-volatile computer-readable storage medium, and the computer-readable storage medium includes program instructions.
  • the program instructions When the program instructions are executed on a computing device, the computing device executes the method provided in the second aspect of the present application and any possible implementation thereof.
  • the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method provided in the second aspect of the present application and any possible implementation thereof.
  • FIG1 is a schematic diagram of a structure of an implementation environment involved in a method for deploying a cloud resource management system provided in an embodiment of the present application;
  • FIG2 is a schematic diagram of a deployment process of a cloud resource management system provided in an embodiment of the present application.
  • FIG3 is a flow chart of a method for deploying a cloud resource management system provided in an embodiment of the present application
  • FIG4 is a schematic diagram of a deployment bottom management component provided in an embodiment of the present application.
  • FIG5 is a schematic diagram of an underlying management component managing an internal resource cluster provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a deployment basic management component, basic public services and necessary middleware provided in an embodiment of the present application;
  • FIG. 7 is a schematic diagram of a basic management component managing a basic resource group provided in an embodiment of the present application.
  • FIG8 is a flow chart of another method for deploying a cloud resource management system provided in an embodiment of the present application.
  • FIG9 is a schematic diagram of an extended K8s cluster provided in an embodiment of the present application.
  • FIG. 10 is a schematic diagram of an extended management component managing an extended resource group provided in an embodiment of the present application.
  • FIG11 is a flowchart of another method for deploying a cloud resource management system provided in an embodiment of the present application.
  • FIG. 12 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application.
  • Virtualization is a resource management technology that abstracts and transforms the various physical resources of the host, such as computing resources, network resources, and storage resources, to break the inseparable barriers between the physical structures of the host, so that users can use these resources in a better way than the original configuration.
  • the resources used through virtualization are called virtualized resources, which are not restricted by the construction method, setting region or physical configuration of existing physical resources.
  • Host A physical resource used to carry virtualization technology.
  • a host is also called a physical machine.
  • a host used to deploy a virtual instance is a physical server.
  • a physical machine has multiple physical devices.
  • a physical server has physical devices such as a processor and a memory.
  • Multiple virtual instances can be deployed in a host, and multiple virtual instances deployed on the same host share the physical resources of the host. According to different usage scenarios, a host can deploy virtual instances belonging to only one user, or it can deploy virtual instances belonging to multiple users.
  • a resource pool is a collection of various hardware resources and software resources involved in a cloud computing data center.
  • resources in a resource pool can be divided into computing resources, storage resources, and network resources.
  • the virtual instance runs on the host's operating system, and the virtual instance itself is equipped with an operating system.
  • the operating system of the virtual instance runs an application program, which is used to implement the user's business.
  • the virtual instance can use the host's hardware resources, and different virtual instances are isolated from each other.
  • the virtual instance can be a virtual machine or a container.
  • Virtual machine refers to a complete computer system that is simulated by virtualization technology and has complete hardware system functions and runs in a completely isolated environment. Some instruction subsets of the virtual machine can be processed in the host machine, and other instructions can be executed in a simulated manner. A virtual machine is also called a virtual server.
  • a virtual machine can be considered as a collection of several virtual devices, which is a complete computer system with complete hardware system functions and running in a completely isolated environment.
  • Virtual devices are virtualized based on physical devices that can share resources based on virtualization technology.
  • a virtual processor virtualized based on a processor based on virtualization technology is a virtual device.
  • a training card virtualized based on a field-programmable gate array (FPGA) based on virtualization technology is also a virtual device.
  • FPGA field-programmable gate array
  • the container provides a lightweight virtual operating environment.
  • the container can be obtained by packaging all the code, libraries, and dependencies of the user's application into an image.
  • the image runs in the virtual operating environment.
  • the container is a runtime instance of the image, similar to a lightweight sandbox, which can be started, started, stopped, and deleted.
  • the image will not share the host's memory, processor (such as the central processing unit (CPU)) and disk resources with other images, realizing container isolation between the image and the host, and between the image and other images, ensuring that the process in the container cannot monitor any process or resources outside the container.
  • processor such as the central processing unit (CPU)
  • Container technologies include Docker, Kubernetes, CoreOS, and other container technologies. The following uses Kubernetes container technology as an example.
  • Kubernetes is an open source container orchestration engine from Google that supports automated deployment, large-scale scalability, and containerized application management. In Kubernetes, you can create multiple containers and run an application instance in each container. When deploying an application in a production environment, you can usually deploy multiple application instances of the application, and then manage, discover, and access multiple application instances through the built-in load balancing strategy, and the implementation of these details does not require complex manual configuration and processing by operation and maintenance personnel.
  • the component that manages the container using Kubernetes container technology can be called the Kubernetes management component (K8S management component for short).
  • the container implemented by Kubernetes container technology can be called the Kubernetes container (K8S container for short).
  • the embodiment of the present application provides a cloud resource management system and a deployment method thereof.
  • the cloud resource management node of the cloud resource management system deploys an underlying management component, and then the underlying management component manages the internal resource cluster, and deploys an external resource management component based on the internal resource cluster, the external resource management component manages the external resource cluster, and the external resource management component includes a basic management component, the basic management component manages the basic resource group in the external resource cluster, and deploys the tenant's business based on the basic resource group.
  • the basic management component only manages the basic resource groups in the external resource cluster, not Manage all resources in external resource clusters. This controls the amount of resources in the external resource clusters managed by the basic management component, effectively controls the resources occupied by the basic management component, and reduces the probability that the scale of the basic management component will increase in tandem with the scale of the external resource cluster, which helps to reduce the construction cost of the cloud resource management system.
  • the basic management component by managing some resources in the external resource cluster through the basic management component, it provides conditions for the basic management component to expand the entire control plane of the cloud resource management system based on the external resource cluster, so that the cloud resource management system has the ability to control ultra-large-scale infrastructure resource pools.
  • Figure 1 is a structural diagram of an implementation environment involved in a deployment method of a cloud resource management system provided in an embodiment of the present application.
  • the implementation environment includes: a client 01, a cloud resource management node 02, an internal resource cluster 03, and an external resource cluster 04.
  • a communication connection can be established between the client 01 and the cloud resource management node 02, between the cloud resource management node 02 and the internal resource cluster 03, between the cloud resource management node 02 and the external resource cluster 04, and between the internal resource cluster 03 and the external resource cluster 04.
  • a communication connection can be established between the client 01 and the cloud resource management node 02 through a network.
  • the network can be a local area network, the Internet, or other networks, which is not limited by the embodiments of the present application.
  • the operation and maintenance personnel of the cloud resource management system can interact with the cloud resource management node 02 through the client 01.
  • the operation and maintenance personnel can send an instruction to deploy the cloud resource management system to the cloud resource management node 02 through the client 01 to instruct the cloud resource management node 02 to deploy the cloud resource management system based on the resources of the cloud resource management node 02, the internal resource cluster 03 and the external resource cluster 04.
  • the cloud resource management system is used to manage the resources of the external resource cluster 04 and provide cloud services to tenants based on the resources of the external resource cluster 04.
  • the cloud resource management node 02 can execute the deployment method of the cloud resource management system provided in the embodiment of the present application based on the resources of the cloud resource management node 02, the internal resource cluster 03 and the external resource cluster 04.
  • the cloud resource management node 02 can deploy underlying management components based on the resources of the cloud resource management node 02, and control the underlying management components to manage the internal resource cluster 03, create a container cluster management system (Kubernetes, K8s for short) cluster based on the internal resource cluster 03, and deploy at least one type of basic public services and required middleware in the K8s cluster, as well as deploy external resource management components, and then manage the external resource cluster 04 through the external management components, and provide cloud services to tenants based on the external resource cluster 04.
  • Kubernetes, K8s for short container cluster management system
  • the client 01 can be a computer, a personal computer, a laptop computer, a mobile phone, a smart phone, a tablet computer, a cloud host, a portable mobile terminal, a multimedia player, an e-book reader, a wearable device, a smart home appliance, an artificial intelligence device, a smart wearable device, a smart vehicle-mounted device or an Internet of Things device, etc.
  • the cloud resource management node 02 can be a cloud server or a cloud physical machine, or a cloud server cluster or a physical machine cluster composed of several cloud servers, or a cloud computing service center.
  • the internal resource cluster 03 and the external resource cluster 04 can be a server cluster composed of several servers, or a cloud computing service center.
  • a large number of basic resources owned by a cloud service provider are deployed in the cloud computing service center.
  • computing resources, storage resources, and network resources are deployed in the cloud computing service center.
  • the cloud computing service center can implement the deployment method of the cloud resource management system provided in the embodiment of the present application based on the large number of basic resources.
  • executing the deployment method of the cloud resource management system provided in the embodiment of the present application is actually to build a resource management and control architecture of the cloud computing service center, so as to provide cloud services to tenants based on the resource management and control architecture.
  • cloud resource management node 02, internal resource cluster 03 and external resource cluster 04 in the implementation environment can also be implemented through other resource platforms besides the cloud computing service center, and the embodiments of the present application do not specifically limit them.
  • the deployment method of the cloud resource management system includes the following steps:
  • Step 301 The cloud resource management node deploys the underlying management components.
  • the cloud resource management node can first deploy the underlying management component in the cloud resource management node.
  • the underlying management component can be regarded as the basis of the entire cloud resource management system. Only after the underlying management component is deployed can other components in the cloud resource management system be deployed based on the underlying management component.
  • the implementation method of deploying the underlying management component in the cloud resource management node includes: the cloud resource management node deploys the underlying management component based on the resources of the cloud resource management node.
  • the cloud resource management node can be a physical machine, etc. At this time, deploying the underlying management component is actually deploying the underlying management component based on the storage resources and computing resources of the physical machine.
  • the resources used to deploy the underlying management components can be isolated from the external resource cluster and/or from the internal resource cluster.
  • the source cluster is used by the management components of the cloud resource management system
  • the external resource cluster is used to provide cloud services to tenants.
  • Both the internal resource cluster and the external resource cluster can be server clusters composed of several servers, or a cloud computing service center.
  • the underlying management components can be regarded as a completely self-contained service of the cloud resource management node, which realizes the decoupling of the underlying management components from the internal resource cluster and the external resource cluster, and realizes the physical isolation of the underlying management components from the internal resource cluster and the external resource cluster, which can achieve the effect of fault isolation and avoid the spread of faults.
  • the complete cloud resource management system can still be rebuilt based on the underlying management components.
  • the internal resource cluster and the external resource cluster can also be isolated from each other.
  • the underlying management components can be deployed on multiple cloud resource management nodes so that the underlying management components have disaster recovery capabilities.
  • the black squares in Figure 4 represent physical machines
  • the unfilled squares in Figure 4 represent internal resource clusters.
  • Figure 4 shows three physical machines, which can deploy underlying management components based on their own resources, so that the underlying management components have node-level local disaster recovery capabilities.
  • the deployment of the underlying management component depends on the resources of the cloud resource management node, but has no dependence on resources other than the cloud resource management node.
  • the underlying management component can be regarded as a service undertaken by the cloud resource management node, and the external resource management component can be regarded as a higher-level service deployed based on the underlying management component. Therefore, the underlying management component is also called a root service. Since the deployment of the underlying management component has no dependence on resources other than the cloud resource management node, and the cloud resource management node does not belong to the external resource cluster, the root service can be regarded as the bottom-level, completely self-contained computing and storage root service in the cloud resource management system.
  • the underlying management component does not need to grow in conjunction with the increase in the scale of the external resource cluster, has stable and reliable characteristics, and can reduce the construction cost of the cloud resource management system.
  • Step 302 The bottom-level management component manages the internal resource cluster.
  • the underlying management component can manage the internal resource cluster by installing an operating system, allocating an Internet Protocol (IP) address, and installing a client for the servers in the internal resource cluster. After installing the operating system, allocating an IP address, and installing a client on the servers in the internal resource cluster, the underlying management component can obtain the resource usage of the server through the client and allocate the resources of the server, thereby managing the internal resource cluster. As shown in Figure 5, the unfilled blocks in Figure 5 represent the servers in the internal resource cluster. The underlying management component can install an operating system, allocate an IP address, and install a client on these servers to manage these servers, that is, to manage the internal resource cluster.
  • IP Internet Protocol
  • Step 303 The underlying management component deploys the basic management component of the external resource management component based on the internal resource cluster, or deploys at least one of the basic public services and necessary middleware based on the internal resource cluster, and the basic management component of the external resource management component.
  • the external resource management component is used to manage the external resource cluster.
  • the external resource management component includes a basic management component and an extended management component.
  • the extended management component is deployed based on the resources managed by the basic management component.
  • the basic management component can be regarded as the base for the external resource management component to expand outward. Therefore, the control scope of the basic management component must at least cover the resources required to deploy the extended management component.
  • the basic management component is used to manage the resources used to deploy the extended management component.
  • the basic management component can be deployed across regions, so that the basic management component has remote disaster recovery capabilities and ensures the reliability of the basic management component.
  • the basic management component may include at least one of basic public services and essential middleware.
  • basic public services are services that must be used for the operation of the basic management component.
  • the basic public services may be database services.
  • Essential middleware is middleware that must be used for the operation of the basic management component.
  • the essential middleware may be a cache and a message queue.
  • step 303 is actually the underlying management component deploying the entire basic management component based on the internal resource cluster.
  • the implementation process includes: the underlying management component creates a second virtual instance cluster based on the internal resource cluster, and deploys the entire basic management component in the second virtual instance cluster.
  • the virtual instance cluster includes multiple virtual instances.
  • the virtual instance may be a container or a virtual machine.
  • the second virtual instance cluster may be a K8s cluster
  • the basic management component may be software with management functions implemented in a container form.
  • the basic management component may not include at least one of the basic public services and required middleware.
  • step 303 is actually the underlying management component deploying at least one of the basic public services and required middleware based on the internal resource cluster, and then deploying the basic management component based on the at least one of the basic public services and required middleware.
  • the underlying management component deploys at least one of the basic public services and required middleware based on the internal resource cluster, and its implementation process includes: the underlying management component creates a third virtual instance cluster based on the internal resource cluster, and in the third virtual instance cluster, deploys at least one of the basic public services and required middleware.
  • the third virtual instance cluster can be a K8s cluster, and the basic public services and required middleware It can be software with relevant functions implemented in the form of containers.
  • the underlying management component can create a K8s cluster based on the internal resource cluster, and deploy basic public services, necessary middleware and basic management components in the K8s cluster respectively.
  • the second virtual instance cluster and the third virtual instance cluster can be an instance cluster created by the underlying management component based on all internal resource clusters at one time, or it can be an instance cluster created by the underlying management component multiple times and each time based on a part of the resources in the internal resource cluster.
  • the embodiments of this application do not specifically limit it.
  • the basic management component does not include basic public services and required middleware
  • the basic public services and required middleware are first deployed based on the internal resource cluster, and then the basic management component is deployed based on the basic public services and required middleware.
  • This maximizes the sharing of cross-cloud service public capabilities, reduces the implementation cost of basic management components, and simplifies the deployment complexity of basic management components.
  • the implementation method of this deployment embodies the principle of service-on-service development. Among them, the principle of service-on-service development means that the upper-level services should be deployed based on the underlying cloud services with layered, stable, and sustainable supply.
  • the most basic management subsystem is the smallest scale form that the cloud resource management system has to maintain normal operation when the tenant business volume is the least. Therefore, the most basic management subsystem is also called the minimum starting unit (MSU) of the cloud resource management system.
  • MSU minimum starting unit
  • the scope of the minimum starting unit is shown in Figure 6.
  • the minimum starting unit includes the bottom-level completely self-contained computing and storage root services, the internal resource cluster managed by the root service, the basic public services and necessary middleware issued on the basis of the internal resource cluster, and the basic management components capable of managing external resource clusters of a specific scale.
  • the minimum starting unit exists independently of the external resource cluster, realizes complete decoupling from the external resource cluster, and plays the effect of fault isolation from the external resource cluster.
  • the scale of the minimum starting unit is controlled and will not grow synchronously with the growth of the capacity of the external resource cluster, effectively controlling the cost of the cloud resource management system.
  • This minimum startup unit is equivalent to the computer's basic input output system (BIOS), and is the most solid and reliable fortress of the entire cloud resource management system. It can complete the boot and initialization of the entire cloud resource management system at the lowest cost, and can expand a complete cloud resource management system based on this minimum startup unit, ensuring that after a serious disaster occurs, it can still be used as a basis to gradually rebuild the complete cloud resource management system.
  • BIOS basic input output system
  • Step 304 The basic management component manages the basic resource group in the external resource cluster.
  • the basic management component manages the basic resource group in the external resource cluster, which can be achieved by installing the operating system, allocating IP addresses, installing clients, etc. for the server that provides the basic resource group. After installing the operating system, allocating IP addresses, and installing clients on the server that provides the basic resource group, the basic management component can obtain the resource usage of the server through the client and allocate the resources of the server, so that the basic resource group can be managed.
  • host group 1 in Figure 7 provides the basic resource group in the external resource cluster. Host group 1 includes multiple servers.
  • the basic management component can install the operating system, allocate IP addresses, and install clients on each server in host group 1 to achieve the management of the basic resource group.
  • Step 305 The basic management component provides cloud services to tenants based on the basic resource group.
  • step 305 is described below by taking the implementation of the basic management component providing cloud services to tenants directly and indirectly based on the basic resource group as an example.
  • the basic management component indirectly provides cloud services to tenants based on the basic resource group.
  • the implementation process includes:
  • Step 305a1 The basic management component deploys the extended management component of the external resource management component based on the basic resource group.
  • the basic management component of the external resource management component belongs to the minimum startup unit of the cloud resource management system.
  • the minimum startup unit is the minimum size of the cloud resource management system to maintain normal operation.
  • the scale of the resources managed by the cloud resource management system needs to increase. For example, if the scale of the external resource cluster is increased, the scale of the cloud resource management system needs to increase accordingly.
  • the basic management component can deploy the extended management component based on the basic resource group to manage the increased resources based on the extended management component.
  • the process of deploying the extended management component can be regarded as the process of expanding the scale of the external resource management component when the business volume increases.
  • the scale of the minimum startup unit can be limited.
  • the scale of the external resource cluster needs to change according to the business volume, the scale of the minimum startup unit does not need to increase in conjunction with the increase in the scale of the external resource cluster, which has stable and reliable characteristics and reduces the construction cost of the cloud resource management system.
  • step 305a1 there may be multiple implementations of step 305a1, and the following two implementations are used as examples to illustrate the implementations.
  • the basic management component deploys the extended management component based on the basic resource group, including: the basic management component creates a first virtual instance cluster based on the basic resource group, and deploys the extended management component in the first virtual instance cluster.
  • the first virtual instance cluster may be a container cluster management system K8s cluster
  • the extended management component may be software with management functions implemented in a container form.
  • the basic management component deploys the extended management component based on the basic resource group, including: the basic management component creates a first virtual instance cluster based on the basic resource group, and the underlying management component deploys the extended management component in the first virtual instance cluster.
  • the first virtual instance cluster can be a container cluster management system K8s cluster
  • the extended management component can be software with management functions implemented in a container form.
  • the basic management component provides the ability to virtualize a virtual instance cluster based on resources.
  • the underlying management component schedules the first virtual instance cluster and deploys the extended management component based on the first virtual instance cluster.
  • the basic management component can provide the underlying management component with relevant information of the first virtual instance cluster, so that the underlying management component can schedule the first virtual instance cluster based on the relevant information.
  • the underlying management component can merge the first virtual instance cluster with the second virtual instance cluster into one virtual instance cluster, and then schedule the expanded virtual instance cluster.
  • the process of merging the first virtual instance cluster with the second virtual instance cluster can be regarded as adding the first virtual instance cluster on the basis of the second virtual instance cluster, and expanding the second virtual instance cluster into a logical large cluster spanning internal resource clusters and external resource clusters.
  • the Virtual Kublet technology can be used to merge the K8s cluster created based on the basic resource group with the K8s cluster created based on the internal resource cluster, so as to expand the K8s cluster created based on the internal resource cluster into a large logical cluster spanning the internal resource cluster and the external resource cluster.
  • the underlying management component merges the first virtual instance cluster, the second virtual instance cluster, and the third virtual instance cluster.
  • the implementation process of the merger please refer to the implementation process of merging the first virtual instance cluster and the second virtual instance cluster.
  • the internal resource cluster and the external resource cluster can be isolated from each other, and relatively independent resource cluster management technology can be used.
  • the external resource management component may include multiple extended management components, any two extended management components may be isolated from each other, and/or the basic management component and any extended management component may be isolated from each other.
  • the external resource management component may adopt a grid architecture, that is, the entire external management component adopts a multi-service grid form, and the basic management component and each extended management component are an independent service grid.
  • the propagation of faults between the extended management components can be avoided, thereby ensuring the reliability of the extended management components.
  • the basic management component and the extended management component are isolated from each other, the propagation of faults between the extended management component and the basic management component can be avoided, thereby ensuring the reliability of the extended management component and the basic management component.
  • Step 305a2 The extension management component manages the extension resource group in the external resource cluster, creates a virtual instance based on the extension resource group, and deploys the tenant's services in the virtual instance.
  • the extended management component can manage the extended resource group by installing an operating system, assigning IP addresses, installing clients, etc. for the servers in the extended resource group.
  • host groups 2, 3, 4, etc. in Figure 10 provide extended resource groups in an external resource cluster. These host groups all include multiple servers.
  • the extended management component can install an operating system, assign IP addresses, and install clients on each server in these host groups to achieve management of the extended resource group.
  • a virtual instance can be created based on the extended resource group, and the tenant's business can be deployed in the virtual instance to provide cloud services to the tenant.
  • the basic management component directly provides cloud services to tenants based on the basic resource group.
  • the implementation process includes: the basic management component creates a virtual instance based on the basic resource group, and deploys the tenant's business in the virtual instance. For example, after the basic management component manages the basic resource group, when it is necessary to provide cloud services to tenants, the basic management component can create a virtual instance based on the basic resource group, and deploy the tenant's business in the virtual instance to provide cloud services to the tenant.
  • the basic management component can also deploy an extended management component based on the basic resource group.
  • the extended management component is used to manage the extended resource group in the external resource cluster, create a virtual instance based on the extended resource group, and deploy the tenant's services in the virtual instance.
  • the implementation method of deploying the extended management component and the implementation method of deploying the tenant's services by the extended management component can refer to the previous related content accordingly, and will not be repeated here.
  • the extended management component since the extended management component is used to manage the extended resource group in the external resource cluster and provide cloud services to tenants based on it, in order to ensure that the extended resource group can be effectively utilized, the extended management component can have priority in using the basic resource group over the virtual instance used to deploy the tenant. That is, the basic management component can use the basic resource group to deploy the extended management component first, and when it is necessary to deploy the tenant's business, create a virtual instance based on the remaining resource groups in the basic resource group that are not occupied by the extended management component, and deploy the tenant's business in the virtual instance. For example, as shown in FIG. 11, the basic management component creates a virtual instance based on the basic resource group, and in the virtual instance Before deploying the tenant's service in the example, the method further includes:
  • Step 305b1 The basic management component deploys the extended management component based on the basic resource group.
  • step 305b1 For the implementation process of step 305b1, please refer to the implementation process of step 305a1, which will not be repeated here.
  • the basic management component creates a virtual instance based on the basic resource group, and deploys the tenant's business in the virtual instance, including: step 305b2, the basic management component creates a virtual instance based on the remaining resource groups in the basic resource group that are not occupied by the extended management component, and deploys the tenant's business in the virtual instance.
  • the minimum startup unit is used to provide the starting specifications of various services deployed therein (including basic management components, basic public services, and necessary middleware) to meet the resource needs of normal business loads.
  • extended management components need to be deployed in external resource clusters to increase the business processing capabilities of resource clusters. Therefore, after the business peak has passed, the resources occupied by them in the external resource pool can be released in a timely manner, including the release of resources used by the corresponding business and the resources occupied by extended management components, to achieve elastic scaling based on resources, so as to optimize the cost occupation of the cloud resource management system.
  • a bottom-level management component that is completely independent of the outside world is constructed in the minimum startup unit, and the internal resource cluster is managed by the bottom-level management component.
  • the basic management component, basic public service and necessary middleware of the external resource management component are deployed based on the internal resource cluster, and then the extended management component of the external resource management component is constructed, and then the control surface of the cloud resource management system is expanded and deployed from the minimum startup unit to the external resource cluster.
  • This method of building a cloud resource management system based on bootstrapping and partial order dependency can not only ensure that the scale of the internal resource pool of the minimum startup unit is limited, but also help to improve the overall resource utilization, effectively reducing the construction cost of the cloud resource management system.
  • this method of realizing the deployment of the entire cloud resource management system on the basis of the minimum startup unit not only enables the cloud resource management system to have the ability to control the ultra-large-scale infrastructure resource pool, but also realizes the isolation of the minimum startup unit and the external resource cluster on the basis of controlling the cost of the cloud resource management system, avoiding the propagation of faults between each other.
  • the cloud resource management system issues virtual instances of the same resource form during the resource management process, it can ensure that the minimum startup unit is simple and stable enough, and it is easier to achieve self-containment of the minimum startup unit, which can ensure the reliability of the cloud resource management system.
  • most of the deployment resources required for the extended management component are extended in the external resource pool, and the resource pool is shared with tenants, which largely avoids the fragmentation of the hardware resource pool and helps to optimize costs.
  • the embodiment of the present application also provides an on-cloud resource management system.
  • Figures 7 and 10 are schematic diagrams of the structure of the on-cloud resource management system provided by the embodiment of the present application. It should be understood that the on-cloud resource management system may include more additional components than the components shown or omit some of the components shown therein, and the embodiment of the present application does not limit this.
  • the on-cloud resource management system can be configured on a cloud platform. As shown in Figure 7, the on-cloud resource management system may include: an on-cloud resource management node, an underlying management component, and an external resource management component.
  • the underlying management component is used to manage the internal resource cluster and is the basic management component for deploying external resource management components based on the internal resource cluster.
  • the external resource management component is used to manage the external resource cluster, wherein the external resource management component includes a basic management component, and the basic management component is used to manage the basic resource group in the external resource cluster, create a virtual instance based on the basic resource group, and deploy the tenant's business in the virtual instance.
  • the basic management component only manages the basic resource groups in the external resource cluster, rather than all the resources in the external resource cluster. This controls the amount of resources in the external resource cluster managed by the basic management component, effectively controls the resources occupied by the basic management component, and reduces the probability that the scale of the basic management component will increase in tandem with the scale of the external resource cluster, which helps to reduce the construction cost of the cloud resource management system.
  • the basic management component by managing some resources in the external resource cluster through the basic management component, it provides conditions for the basic management component to expand the entire control plane of the cloud resource management system based on the external resource cluster, so that the cloud resource management system has the ability to control ultra-large-scale infrastructure resource pools.
  • the system further includes: an extension management component of the external resource management component.
  • the basic management component is specifically used to deploy the extension management component based on the basic resource group.
  • the basic management component specifically provides cloud services to tenants through the extension management component.
  • the extension management component is used to manage the extension resource group in the external resource cluster, create a virtual instance based on the extension resource group, and deploy the tenant's business in the virtual instance.
  • the basic management component is specifically used to create a virtual instance based on the basic resource group, and deploy the tenant's services in the virtual instance.
  • the basic management component is also used to deploy the extended management component based on the basic resource group.
  • the component is specifically used to create a virtual instance based on the remaining resource groups in the basic resource group that are not occupied by the extended management component, and deploy the tenant's business in the virtual instance.
  • the above process of deploying the extended management component can be regarded as the process of further deploying the external resource management component when the business volume increases.
  • the remaining resource group occupied by the basic management component can be effectively controlled, and the size of the minimum startup unit can be limited.
  • the size of the minimum startup unit does not need to increase in conjunction with the increase in the size of the external resource cluster. It has stable and reliable characteristics and reduces the construction cost of the cloud resource management system.
  • the basic management component is specifically used to create a first virtual instance cluster based on the basic resource group, and deploy the extension management component in the first virtual instance cluster.
  • the basic management component is specifically used to create a first virtual instance cluster based on the basic resource group; the underlying management component is also used to deploy an extended management component in the first virtual instance cluster.
  • resources used by the deployment bottom management component are isolated from the external resource cluster and/or isolated from the internal resource cluster.
  • the internal resource cluster and the external resource cluster may also be isolated from each other.
  • the underlying management components may be deployed on multiple cloud resource management nodes so that the underlying management components have disaster recovery capabilities.
  • the underlying management component is specifically used to create a second virtual instance cluster based on the internal resource cluster, and deploy a basic management component of the external resource management component in the second virtual instance cluster.
  • the basic management component may include at least one of basic public services and essential middleware.
  • public services are services that can be shared by cloud services that can be provided by the cloud resource management system through an external resource cluster.
  • public services can be database services.
  • Essential middleware is middleware that is required to be used by cloud services that can be provided by the cloud resource management system through an external resource cluster.
  • essential middleware can be cache and message queue.
  • the basic management component may not include at least one of the basic public services and the required middleware.
  • the underlying management component deploys at least one of the basic public services and the required middleware based on the internal resource cluster, and the implementation process includes: the underlying management component creates a third virtual instance cluster based on the internal resource cluster, and deploys at least one of the basic public services and the required middleware in the third virtual instance cluster.
  • the subsystem including the underlying management components and the basic management components can be regarded as the most basic management subsystem in the cloud resource management system.
  • the most basic management subsystem is the smallest scale form that the cloud resource management system has to maintain normal operation when the tenant business volume is the least. Therefore, the most basic management subsystem is also called the minimum startup unit of the cloud resource management system.
  • the minimum startup unit includes the most basic and completely self-contained computing and storage root services, the internal resource cluster managed by the root service, the basic public services and necessary middleware issued on the basis of the internal resource cluster, and the basic management components capable of managing external resource clusters of a specific scale.
  • the minimum startup unit exists independently of the external resource cluster, realizes complete decoupling from the external resource cluster, and plays the role of fault isolation from the external resource cluster.
  • the scale of the minimum startup unit is controlled and will not grow synchronously with the growth of the capacity of the external resource cluster, which effectively controls the cost of the cloud resource management system.
  • This minimum startup unit is equivalent to the basic input and output system of a computer. It is the most solid and reliable fortress of the entire cloud resource management system. It can complete the boot and initialization of the entire cloud resource management system at the lowest cost, and can expand a complete cloud resource management system based on this minimum startup unit, ensuring that after a serious disaster occurs, it can still be used as a basis to gradually rebuild the complete cloud resource management system.
  • the external resource management component may include multiple extended management components, any two extended management components may be isolated from each other, and/or the basic management component and any extended management component may be isolated from each other.
  • the virtual instance is a container or a virtual machine.
  • the cloud resource management system provided by the embodiment of the present application can not only ensure that the internal resource pool size of the minimum startup unit is limited, but also help to improve the overall resource utilization, effectively reducing the construction cost of the cloud resource management system.
  • the cloud resource management system is deployed on the basis of the minimum startup unit, which not only enables the cloud resource management system to have the ability to control the ultra-large-scale infrastructure resource pool, but also realizes the isolation of the minimum startup unit from the external resource cluster on the basis of controlling the cost of the cloud resource management system, avoiding the propagation of faults between each other.
  • the cloud resource management system issues virtual instances of the same resource form during the resource control process, it can ensure that the minimum startup unit is simple and stable enough, and it is easier to achieve the self-containment of the minimum startup unit, which can ensure the reliability of the cloud resource management system.
  • most of the deployment resources required for the extended management component are extended in the external resource pool, and the resource pool is shared with tenants, which largely avoids the fragmentation of the hardware resource pool and helps cost optimization.
  • FIG. 12 is a structural diagram of a computing device provided in an embodiment of the present application.
  • the computing device 1200 includes a processor 1201, a memory 1202, a communication interface 1203 and a bus 1204. Among them, the processor 1201, the memory 1202, and the communication interface 1203 realize communication connection with each other through the bus 1204.
  • Processor 1201 may include a general-purpose processor and/or a dedicated hardware chip.
  • a general-purpose processor may include: a central processing unit (CPU), a microprocessor or a graphics processing unit (GPU).
  • the CPU is, for example, a single-core processor (single-CPU) or a multi-core processor (multi-CPU).
  • a dedicated hardware chip is a hardware module for high-performance processing.
  • the dedicated hardware chip includes at least one of a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or a network processor (NP).
  • Processor 1201 may also be an integrated circuit chip with signal processing capabilities. In the implementation process, some or all of the functions of the deployment method of the cloud resource management system of the present application can be completed by the hardware integrated logic circuit in the processor 1201 or instructions in software form.
  • the memory 1202 is used to store computer programs, and the computer programs include an operating system 1202a and executable codes (i.e., program instructions) 1202b.
  • the memory 1202 is, for example, a read-only memory or other types of static storage devices that can store static information and instructions, or a random access memory or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory, a read-only optical disc or other optical disc storage, an optical disc storage (including a compressed optical disc, a laser disc, an optical disc, a digital versatile disc, a blue-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired executable code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto.
  • the memory 1202 is used to store an outbound port queue, etc.
  • the memory 1202 is, for example, independent and connected to the processor 1201 via a bus 1204. Or the memory 1202 and the processor 1201 are integrated together.
  • the memory 1202 can store executable code.
  • the processor 1201 is used to execute part or all of the functions of the deployment method of the cloud resource management system provided in the embodiment of the present application.
  • the implementation method of the processor 1201 executing the process please refer to the relevant description in the aforementioned embodiment.
  • the memory 1202 may also include software modules and data required for other running processes such as the operating system.
  • the communication interface 1203 uses a transceiver module such as, but not limited to, a transceiver to achieve communication with other devices or communication networks.
  • a transceiver module such as, but not limited to, a transceiver to achieve communication with other devices or communication networks.
  • the communication interface 1203 can be any one or any combination of the following devices: a network interface (such as an Ethernet interface), a wireless network card, and other devices with network access functions.
  • the bus 1204 is any type of communication bus for interconnecting the internal devices of the computing device (e.g., the memory 1202, the processor 1201, and the communication interface 1203).
  • a system bus for interconnecting the internal devices of the computing device.
  • the embodiment of the present application takes the interconnection of the above-mentioned devices inside the computing device through the bus 1204 as an example.
  • the above-mentioned devices inside the computing device 1200 can also be connected to each other in communication with each other using other connection methods other than the bus 1204.
  • the above-mentioned devices inside the computing device 1200 are interconnected through an internal logical interface.
  • the above-mentioned multiple devices can be respectively arranged on independent chips, or at least partially or completely arranged on the same chip. Whether to independently arrange each device on different chips or to integrate and arrange it on one or more chips often depends on the needs of product design.
  • the embodiments of the present application do not limit the specific implementation form of the above-mentioned devices.
  • the descriptions of the processes corresponding to the above-mentioned figures have different focuses. For the parts not described in detail in a certain process, please refer to the relevant descriptions of other processes.
  • all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof.
  • all or part of the embodiments may be implemented in the form of a computer program product.
  • the computer program product providing the program development platform includes one or more computer instructions, and when these computer program instructions are loaded and executed on a computing device, all or part of the functions of the deployment method of the cloud resource management system provided in the embodiment of the present application are implemented.
  • computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means.
  • the computer-readable storage medium stores computer program instructions that provide a program development platform.
  • the present application also provides a computing device cluster.
  • the computing device cluster includes at least one computing device.
  • the computing device may be a server, such as a central server, an edge server, or a local server in a local data center.
  • the computing device cluster includes at least one computing device.
  • the computing device can also be a terminal device such as a desktop computer, a laptop computer or a smart phone.
  • the structure of at least one computing device included in the computing device cluster can refer to the computing device 1200 shown in Figure 12.
  • the memory 1202 in one or more computing devices 1200 in the computing device cluster may store the same instructions for executing the deployment method of the cloud resource management system.
  • the memory 1202 of one or more computing devices 1200 in the computing device cluster may also store partial instructions for executing the deployment method of the cloud resource management system.
  • the combination of one or more computing devices 1200 can jointly execute the instructions for executing the deployment method of the cloud resource management system.
  • the memory 1202 in different computing devices 1200 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the cloud resource management system. That is, the instructions stored in the memory 1202 in different computing devices 1200 can implement the functions of one or more modules of the first determination module, the second determination module and the third determination module.
  • multiple computing devices in a computing device cluster may be connected via a network.
  • the network may be a wide area network or a local area network, etc. Any two computing devices are connected via the network.
  • the network is connected via a communication interface in each computing device.
  • each computing device includes a bus, a processor, a memory, and a communication interface.
  • An embodiment of the present application also provides a computer-readable storage medium, which is a non-volatile computer-readable storage medium.
  • the computer-readable storage medium includes program instructions. When the program instructions are executed on a computing device, the computing device implements a deployment method for a cloud resource management system as provided in an embodiment of the present application.
  • the embodiment of the present application also provides a computer program product comprising instructions.
  • the computer program product When the computer program product is executed on a computer, the computer implements the deployment method of the cloud resource management system provided by the embodiment of the present application.
  • the information including but not limited to user device information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.
  • the original data and executable code involved in this application are all obtained with full authorization.
  • the terms “first”, “second” and “third” are used for descriptive purposes only and are not to be understood as indicating or implying relative importance.
  • the term “at least one” means one or more, and the term “plurality” means two or more, unless otherwise expressly defined.
  • a and/or B can represent: A exists alone, A and B exist at the same time, and B exists alone.
  • the character "/" in this article generally indicates that the associated objects before and after are in an "or" relationship.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Stored Programmes (AREA)

Abstract

本申请公开了一种云上资源管理系统及其部署方法。该云上资源管理系统包括：云上资源管理节点，底层管理组件和外部资源管理组件，其中：云上资源管理节点用于部署底层管理组件，该底层管理组件用于纳管内部资源集群，基于内部资源集群部署外部资源管理组件；该外部资源管理组件用于纳管外部资源集群，外部资源管理组件包括基础管理组件，基础管理组件用于纳管外部资源集群中的基础资源组，基于基础资源组创建虚拟实例，在虚拟实例中部署租户的业务。本申请的基础管理组件仅纳管外部资源集群中的基础资源组，有效控制基础管理组件占用的剩余资源组，有助于降低云上资源管理系统的建设成本。

Description

云上资源管理系统及其部署方法
本申请要求于2022年10月24日提交的申请号为202211300013.1、发明名称为“资源集群管理系统部署方法”的中国专利申请的优先权，以及于2023年04月14日提交的申请号为202310402195.1、发明名称为“云上资源管理系统及其部署方法”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及资源管理技术领域，特别涉及一种云上资源管理系统及其部署方法。
背景技术
在公有云或者企业私有云中，资源池通常分为管理区和租户区两部分。管理区资源池和租户区资源池彼此完全独立，部署在任意一类资源池中的云服务在整个生命周期中都固定地部署在该类资源池，不存在扩展到对方区域实现共部署或者迁移部署的情景。管理区资源池中创建的虚拟实例用于部署纳管租户区资源池的资源管理系统、基础云服务、运维服务、安全服务等。
目前，管理区资源池中的云服务都是虚拟实例形态，不具有弹性伸缩的能力。为了应对业务峰值流量的性能冲击，管理区资源池普遍选择大规格虚拟机，部署大规模业务集群。
但是，业务峰值流量存续的时间通常非常短，造成了管理区资源池大量的资源浪费，导致资源管理系统的建设成本较高。
发明内容
本申请提供了一种云上资源管理系统及其部署方法。本申请的基础管理组件仅纳管外部资源集群中的基础资源组，有效控制基础管理组件占用的资源，有助于降低云上资源管理系统的建设成本。本申请提供的技术方案如下：
第一方面，本申请提供了一种云上资源管理系统，包括：云上资源管理节点，底层管理组件和外部资源管理组件，其中：云上资源管理节点用于部署底层管理组件，底层管理组件用于纳管内部资源集群，基于内部资源集群部署外部资源管理组件；外部资源管理组件用于纳管外部资源集群，该外部资源管理组件包括基础管理组件，基础管理组件用于纳管外部资源集群中的基础资源组，基于基础资源组创建虚拟实例，在虚拟实例中部署租户的业务。
在该云上资源管理系统中，基础管理组件仅纳管外部资源集群中的基础资源组，而不是纳管外部资源集群的所有资源。这样一方面控制了基础管理组件纳管的外部资源集群的资源量，有效控制基础管理组件占用的资源，减小了基础管理组件的规模随着外部资源集群的规模的增加而联动增长的概率，有助于降低云上资源管理系统的建设成本。另一方面，通过基础管理组件对外部资源集群中部分资源进行纳管，为基础管理组件基于外部资源集群扩展出云上资源管理系统的整个管控面提供了条件，使得云上资源管理系统具备管控超大规模基础设施资源池的能力。
在一种实现方式中，外部资源管理组件还包括扩展管理组件。此时，基础管理组件具体用于基于基础资源组，部署扩展管理组件。基础管理组件具体通过扩展管理组件向租户提供云服务。例如，扩展管理组件用于纳管外部资源集群中的扩展资源组，基于扩展资源组创建虚拟实例，在虚拟实例中部署租户的业务。
可选地，基础管理组件还用于基于基础资源组，部署扩展管理组件，扩展管理组件用于纳管外部资源集群中的扩展资源组。则基础管理组件具体用于基于基础资源组中未被扩展管理组件占用的剩余资源组创建虚拟实例，在虚拟实例中部署租户的业务。
以上部署扩展管理组件的过程，可视为在有业务量增加时，进一步部署外部资源管理组件的过程。这样一来，通过基于内部资源集群部署基础管理组件，基于外部资源集群部署扩展管理组件，能够有效控制基础管理组件占用的资源，限制最小启动单元的规模，在外部资源集群的规模需要根据业务量变化时，使得最小启动单元的规模无需随着外部资源集群的规模的增加而联动增长，具有稳定可靠的特性，且降低了云上资源管理系统的建设成本。
在一种实现方式中，基础管理组件具体用于基于基础资源组创建第一虚拟实例集群，在第一虚拟实例集群中部署扩展管理组件。
在另一种实现方式中，基础管理组件具体用于基于基础资源组创建第一虚拟实例集群；底层管理组件还用于在第一虚拟实例集群中部署扩展管理组件。
可选地，部署底层管理组件使用的资源与外部资源集群隔离，和/或，与内部资源集群隔离。当部署底层管理组件使用的资源与外部资源集群、内部资源集群隔离时，底层管理组件不依赖于内部资源集群和外部资源集群，底层管理组件可视为云上资源管理节点完全自包含的服务，其实现了底层管理组件与内部资源集群、外部资源集群的解耦，实现了底层管理组件与内部资源集群、外部资源集群的物理隔离，能够起到故障隔离的效果，避免了故障的扩散，在故障发生后仍能以该底层管理组件为基础重建完整的云上资源管理系统。另外，内部资源集群与外部资源集群也可以互相隔离。
可选地，为了进一步保证云上资源管理系统的安全性，可以在多个云上资源管理节点上部署底层管理组件，使得底层管理组件具有容灾能力。
在一种实现方式中，底层管理组件具体用于基于内部资源集群创建第二虚拟实例集群，在第二虚拟实例集群中部署外部资源管理组件中的基础管理组件。
可选地，基础管理组件可以包括基础公共服务和必需中间件中的至少一类。其中，公共服务是云上资源管理系统通过外部资源集群能够提供的云服务能够公用的服务。例如，公共服务可以为数据库服务。必需中间件是云上资源管理系统通过外部资源集群能够提供的云服务都需要使用到的中间件。例如，必需中间件可以为缓存和消息队列。
或者，基础管理组件也可以不包括基础公共服务和必需中间件中的至少一类。在一种实现方式中，底层管理组件还用于基于内部资源集群，部署基础公共服务和必需中间件中的至少一类。其实现过程包括：底层管理组件基于内部资源集群创建第三虚拟实例集群，在第三虚拟实例集群中，部署基础公共服务和必需中间件中的至少一类。
其中，第二虚拟实例集群和第三虚拟实例集群可以是底层管理组件一次性地基于全部内部资源集群创建的一个实例集群，也可以是底层管理组件分多次，并在每次基于内部资源集群中的一部分资源创建的实例集群，本申请对其不做具体限定。
包含底层管理组件和基础管理组件的子系统可以视为云上资源管理系统中最基础的管理子系统。该最基础的管理子系统是在租户业务量最少的情况下，云上资源管理系统维持正常运行状态具有的最小规模的形态。因此，该最基础的管理子系统也称为云上资源管理系统的最小启动单元。该最小启动单元包括最底层完全自包含的计算和存储根服务，由根服务纳管的内部资源集群，在内部资源集群的基础上发放的基础公共服务和必需中间件，以及具备管理特定规模外部资源集群的基础管理组件。可以看出该最小启动单元相对于外部资源集群独立存在，实现了与外部资源集群的完全解耦，起到了与外部资源集群进行故障隔离的效果。并且，该最小启动单元的规模受控，不会随着外部资源集群容量的增长而同步增长，有效的控制了云上资源管理系统的成本。该最小启动单元相当于计算机的基本输入输出系统，是整个云上资源管理系统最坚实可靠的堡垒，能够以最小代价完成整个云上资源管理系统的引导和初始化，并能够以该最小启动单元为基础扩展出完整的云上资源管理系统，确保严重灾难发生后仍能够以此为基础，逐步重建完整的云上资源管理系统。
可选地，外部资源管理组件可以包括多个扩展管理组件，任两个扩展管理组件之间可以相互隔离，和/或，基础管理组件与任一扩展管理组件之间可以相互隔离。例如，外部资源管理组件可以采用网格化架构，即整个外部管理组件采用多服务格形态，基础管理组件和每个扩展管理组件均为一个独立的服务格。当任两个扩展管理组件之间相互隔离时，能够避免扩展管理组件之间的故障传播，保证了扩展管理组件的可靠性。类似的，当基础管理组件与扩展管理组件之间相互隔离时，能够避免扩展管理组件与基础管理组件之间的故障传播，保证了扩展管理组件和基础管理组件的可靠性。
在一种实现方式中，虚拟实例为容器或虚拟机。
第二方面，本申请提供了一种云上资源管理系统的部署方法，方法应用于云上资源管理系统，云上资源管理系统包括云上资源管理节点，底层管理组件和外部资源管理组件，方法包括：云上资源管理节点部署底层管理组件；底层管理组件纳管内部资源集群，基于内部资源集群部署外部资源管理组件；外部资源管理组件纳管外部资源集群，其中，外部资源管理组件包括基础管理组件，基础管理组件纳管外部资源集 群中的基础资源组，基于基础资源组创建虚拟实例，在虚拟实例中部署租户的业务。
可选地，外部资源管理组件还包括扩展管理组件，方法还包括：基础管理组件基于基础资源组，部署扩展管理组件；扩展管理组件纳管外部资源集群中的扩展资源组，基于扩展资源组创建虚拟实例，在虚拟实例中部署租户的业务。
可选地，外部资源管理组件还包括扩展管理组件，方法还包括：基础管理组件基于基础资源组，部署扩展管理组件，扩展管理组件用于纳管外部资源集群中的扩展资源组；基础管理组件基于基础资源组中未被扩展管理组件占用的剩余资源组创建虚拟实例，在虚拟实例中部署租户的业务。
可选地，基础管理组件基于基础资源组，部署扩展管理组件，包括：基础管理组件基于基础资源组创建第一虚拟实例集群，在第一虚拟实例集群中部署扩展管理组件。
可选地，基础管理组件基于基础资源组，部署扩展管理组件，包括：基础管理组件基于基础资源组创建第一虚拟实例集群；底层管理组件在第一虚拟实例集群中部署扩展管理组件。
可选地，部署底层管理组件使用的资源与外部资源集群隔离，和/或，与内部资源集群隔离。
可选地，底层管理组件具有容灾能力。
可选地，底层管理组件基于内部资源集群部署外部资源管理组件的基础管理组件，包括：底层管理组件基于内部资源集群创建第二虚拟实例集群，在第二虚拟实例集群中部署外部资源管理组件。
可选地，基础管理组件包括基础公共服务和必需中间件中的至少一类。
可选地，该方法还包括：底层管理组件基于内部资源集群，部署基础公共服务和必需中间件中的至少一类。
可选地，底层管理组件基于内部资源集群部署基础公共服务和中间件中的至少一类，包括：底层管理组件基于内部资源集群创建第三虚拟实例集群，在第三虚拟实例集群中，部署基础公共服务和必需中间件中的至少一类。
可选地，外部资源管理组件包括多个扩展管理组件，任两个扩展管理组件之间相互隔离，和/或，基础管理组件与任一扩展管理组件之间相互隔离。
可选地，虚拟实例为容器或虚拟机。
第三方面，本申请提供了一种计算设备，包括存储器和处理器，存储器存储有程序指令，处理器运行程序指令以执行本申请第二方面以及其任一种可能的实现方式中提供的方法。
第四方面，本申请提供了一种计算设备集群，包括多个计算设备，多个计算设备包括多个处理器和多个存储器，多个存储器中存储有程序指令，多个处理器运行程序指令，使得计算设备集群执行本申请第二方面以及其任一种可能的实现方式中提供的方法。
第五方面，本申请提供了一种计算机可读存储介质，该计算机可读存储介质为非易失性计算机可读存储介质，该计算机可读存储介质包括程序指令，当程序指令在计算设备上运行时，使得计算设备执行本申请第二方面以及其任一种可能的实现方式中提供的方法。
第六方面，本申请提供了一种包含指令的计算机程序产品，当计算机程序产品在计算机上运行时，使得计算机执行本申请第二方面以及其任一种可能的实现方式中提供的方法。
附图说明
图1是本申请实施例提供的一种云上资源管理系统的部署方法涉及的实施环境的结构示意图；
图2是本申请实施例提供的一种云上资源管理系统的部署过程示意图；
图3是本申请实施例提供的一种云上资源管理系统的部署方法的流程图；
图4是本申请实施例提供的一种部署底层管理组件的示意图；
图5是本申请实施例提供的一种底层管理组件纳管内部资源集群的示意图；
图6是本申请实施例提供的一种部署基础管理组件、基础公共服务和必需中间件的示意图；
图7是本申请实施例提供的一种基础管理组件纳管基础资源组的示意图；
图8是本申请实施例提供的另一种云上资源管理系统的部署方法的流程图；
图9是本申请实施例提供的一种扩展K8s集群的示意图；
图10是本申请实施例提供的一种扩展管理组件纳管扩展资源组的示意图；
图11是本申请实施例提供的又一种云上资源管理系统的部署方法的流程图；
图12是本申请实施例提供的一种计算设备的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
为便于理解，下面先对本申请实施例中涉及的技术和背景进行解释。
虚拟化(virtualization)是一种资源管理技术，是将主机的各种实体资源，如计算资源、网络资源及存储资源等，予以抽象、转换后呈现出来，打破主机的实体结构间的不可切割的障碍，使用户可以采用比原本的组态更好的方式来应用这些资源。通过虚拟化使用的资源称为虚拟化资源，虚拟化资源不受现有实体资源的架设方式、设置地域或物理组态所限制。
主机(physical machine，PM)：用于承载虚拟化技术的物理资源。主机也称物理机。通常的，用于部署虚拟实例的主机为物理服务器。物理机具有多个物理设备。例如，物理服务器具有处理器和存储器等物理设备。一台主机中可以部署多个虚拟实例，部署在同一台主机上的多个虚拟实例共享主机的物理资源。按照不同的使用情况，一台主机可以仅部署属于一个用户的虚拟实例，也可以部署属于多个用户的虚拟实例。
资源池是云计算数据中心中所涉及到的各种硬件资源和软件资源的集合。通常的，按资源的类型，资源池中的资源可分为计算资源、存储资源和网络资源。
虚拟实例运行在主机的操作系统上，且虚拟实例自身设置有操作系统，虚拟实例的操作系统运行有应用程序，该应用程序用于实现用户的业务。虚拟实例可使用主机的硬件资源，不同虚拟实例之间相互隔离。通常的，虚拟实例可以为虚拟机或容器。
虚拟机(virtual machine，VM)：指通过虚拟化技术模拟得到的具有完整硬件系统功能的、运行在一个完全隔离环境中的完整的计算机系统。虚拟机的部分指令子集可以在宿主(host)机中处理，其它部分指令可以以仿真的方式执行。虚拟机也称虚拟服务器。
虚拟机可视为若干个虚拟设备的集合，该若干个设备的集合为具有完整硬件系统功能的、且运行在一个完全隔离环境中的完整的计算机系统。虚拟设备通过基于虚拟化技术，在能够被共享资源的物理设备的基础上虚拟得到。例如，基于虚拟化技术，在处理器的基础上虚拟得到的虚拟处理器是一种虚拟设备。又例如，基于虚拟化技术，在现场可编程逻辑门阵列(field-programmable gate array，FPGA)的基础上虚拟得到的训练卡也是一种虚拟设备。
容器提供了一个轻量级的虚拟运行环境。容器可以通过将用户的应用程序的所有代码、库和依赖关系等打包成镜像得到。执行镜像时，镜像运行在虚拟运行环境中。此时容器是镜像的运行时实例，类似于一个轻量级的沙箱子，它可以被启动、开始、停止、删除。该镜像不会与其他镜像共享宿主机的内存、处理器(如中央处理器(central processing unit，CPU))以及磁盘等资源，实现了该镜像与主机之间、该镜像与其他镜像之间的容器隔离，保证了容器内的进程不能监控容器外的任何进程或资源。
容器技术包括docker、kubernetes、coreos以及其他容器技术。下面以kubernetes容器技术为例进行说明。
kubernetes是谷歌(Google)开源的一个容器编排引擎，它支持自动化部署、大规模可伸缩、应用容器化管理。在kubernetes中，可以创建多个容器，在每个容器里面运行一个应用实例。当在生产环境中部署一个应用程序时，通常可以部署该应用程序的多个应用实例，然后通过内置的负载均衡策略，对多个应用实例进行管理、发现、访问，而这些细节的实现不需要运维人员进行复杂的手工配置和处理。使用kubernetes容器技术对容器进行管理的组件可称为kubernetes管理组件(简称K8S管理组件)。通过kubernetes容器技术实现的容器可称为kubernetes容器(简称K8S容器)。
本申请实施例提供了一种云上资源管理系统及其部署方法。在该云上资源管理系统的部署方法中，云上资源管理系统的云上资源管理节点部署底层管理组件，然后，底层管理组件纳管内部资源集群，并基于内部资源集群部署外部资源管理组件，外部资源管理组件纳管外部资源集群，且外部资源管理组件包括基础管理组件，基础管理组件纳管外部资源集群中的基础资源组，基于基础资源组部署租户的业务。
在该云上资源管理系统及其部署方法中，基础管理组件仅纳管外部资源集群中的基础资源组，而不是 纳管外部资源集群的所有资源。这样一方面控制了基础管理组件纳管的外部资源集群的资源量，有效控制基础管理组件占用的资源，减小了基础管理组件的规模随着外部资源集群的规模的增加而联动增长的概率，有助于降低云上资源管理系统的建设成本。另一方面，通过基础管理组件对外部资源集群中部分资源进行纳管，为基础管理组件基于外部资源集群扩展出云上资源管理系统的整个管控面提供了条件，使得云上资源管理系统具备管控超大规模基础设施资源池的能力。
图1是本申请实施例提供的一种云上资源管理系统的部署方法涉及的实施环境的结构示意图。如图1所示，该实施环境包括：客户端01、云上资源管理节点02、内部资源集群03和外部资源集群04。客户端01和云上资源管理节点02之间，云上资源管理节点02和内部资源集群03之间，云上资源管理节点02和外部资源集群04之间，内部资源集群03和外部资源集群04之间均能够建立通信连接。例如，客户端01和云上资源管理节点02之间，可以通过网络建立通信连接。可选的，该网络可以为局域网，也可以为互联网，还可以为其它网络，本申请实施例不作限定。
在该实施环境中，云上资源管理系统的运维人员可以通过客户端01与云上资源管理节点02交互。例如，运维人员可以通过客户端01向云上资源管理节点02发送部署云上资源管理系统的指示，以指示云上资源管理节点02基于云上资源管理节点02的资源、内部资源集群03和外部资源集群04，部署云上资源管理系统。该云上资源管理系统用于对该外部资源集群04的资源进行管理，并基于该外部资源集群04的资源为租户提供云服务。如图2所示，云上资源管理节点02接收到该指示后，能够基于云上资源管理节点02的资源、内部资源集群03和外部资源集群04，执行本申请实施例提供的云上资源管理系统的部署方法。例如，云上资源管理节点02可以基于云上资源管理节点02的资源部署底层管理组件，并控制底层管理组件纳管内部资源集群03，基于内部资源集群03创建容器集群管理系统(Kubernetes，简称K8s)集群，并在该K8s集群中部署基础公共服务和必需中间件中的至少一类，以及部署外部资源管理组件，然后通过外部管理组件纳管外部资源集群04，并基于外部资源集群04为租户提供云服务。
可选地，客户端01可以为计算机、个人电脑、膝上型计算机、移动电话、智能手机、平板电脑、云主机、便携式移动终端、多媒体播放器、电子书阅读器、可穿戴式设备、智能家电、人工智能设备、智能可穿戴设备、智能车载设备或物联网设备等。云上资源管理节点02可以是一台云服务器或云物理机，或者由若干台云服务器组成的云服务器集群或物理机集群，或者是一个云计算服务中心。内部资源集群03和外部资源集群04可以是由若干台服务器组成的服务器集群，或者是一个云计算服务中心。其中，云计算服务中心中部署有云服务提供商拥有的大量基础资源。例如云计算服务中心中部署有计算资源、存储资源和网络资源等。云计算服务中心可以基于该大量基础资源，实现本申请实施例提供的云上资源管理系统的部署方法。当云上资源管理节点02、内部资源集群03和外部资源集群04通过云计算服务中心实现时，执行本申请实施例提供的云上资源管理系统的部署方法实际是构建云计算服务中心的资源管控架构，以便于基于该资源管控架构为租户提供云服务。
需要说明的是，该实施环境中的云上资源管理节点02、内部资源集群03和外部资源集群04，也可以通过除云计算服务中心外的其他资源平台实现，本申请实施例对其不做具体限定。
应当理解的是，以上内容是对本申请实施例提供的云上资源管理系统的部署方法的应用场景的示例性说明，并不构成对于该云上资源管理系统的部署方法的应用场景的限定，本领域普通技术人员可知，随着业务需求的改变，其应用场景可以根据应用需求进行调整，本申请实施例对其不做一一列举。
下面先对本申请实施例提供的云上资源管理系统的部署方法进行说明。如图3所示，该云上资源管理系统的部署方法包括以下步骤：
步骤301、云上资源管理节点部署底层管理组件。
在部署云上资源管理系统的过程中，云上资源管理节点可以先在云上资源管理节点中部署底层管理组件。底层管理组件可视为整个云上资源管理系统的基础，在部署底层管理组件后，才能基于底层管理组件继续部署云上资源管理系统中的其他组件。在一种实现方式中，云上资源管理节点部署底层管理组件的实现方式包括：云上资源管理节点基于云上资源管理节点的资源部署底层管理组件。云上资源管理节点可以是物理机等。此时，部署底层管理组件实际是基于物理机的存储资源和计算资源等部署底层管理组件。
其中，部署底层管理组件使用的资源可以与外部资源集群隔离，和/或，与内部资源集群隔离。内部资 源集群用于供云上资源管理系统的管理组件使用，外部资源集群用于向租户提供云服务。内部资源集群和外部资源集群均可以是由若干台服务器组成的服务器集群，或者是一个云计算服务中心。当部署底层管理组件使用的资源与外部资源集群、内部资源集群隔离时，底层管理组件不依赖于内部资源集群和外部资源集群，底层管理组件可视为云上资源管理节点完全自包含的服务，其实现了底层管理组件与内部资源集群、外部资源集群的解耦，实现了底层管理组件与内部资源集群、外部资源集群的物理隔离，能够起到故障隔离的效果，避免了故障的扩散，在故障发生后仍能以该底层管理组件为基础重建完整的云上资源管理系统。另外，内部资源集群与外部资源集群也可以互相隔离。
可选地，为了进一步保证云上资源管理系统的安全性，可以在多个云上资源管理节点上部署底层管理组件，使得底层管理组件具有容灾能力。例如，图4中黑色的方块表示物理机，图4中未填充的方块标识内部资源集群。如图4所示，图4示出了3台物理机，这3台物理机可以基于自身的资源部署底层管理组件，使得底层管理组件具备节点级本地容灾能力。
根据该底层管理组件的部署方式可以看出：底层管理组件的部署除了依赖于云上资源管理节点的资源，对除云上资源管理节点外的资源没有依赖，该底层管理组件可视为云上资源管理节点承担的服务，外部资源管理组件等可视为基于该底层管理组件部署的更上层服务。因此，该底层管理组件也称为根服务。由于底层管理组件的部署对除云上资源管理节点外的资源没有依赖，且云上资源管理节点不属于外部资源集群，可视为该根服务为云上资源管理系统中最底层完全自包含的计算和存储根服务。这样一来，在外部资源集群的规模需要根据业务量变化时，底层管理组件不需要随着外部资源集群的规模的增加而联动增长，具有稳定可靠的特性，且能够降低云上资源管理系统的建设成本。
步骤302、底层管理组件纳管内部资源集群。
底层管理组件纳管内部资源集群，可以通过为内部资源集群中的服务器安装操作系统、分配互联网协议(internet protocol，IP)地址、安装客户端等操作实现。在内部资源集群中的服务器上安装操作系统、分配IP地址和安装客户端等后，底层管理组件就能够通过客户端获取服务器的资源使用情况，并对服务器的资源进行分配，因此能够实现对内部资源集群的纳管。如图5所示，图5中未填充的方块表示内部资源集群中的服务器，底层管理组件可以在这些服务器上安装操作系统、分配IP地址和安装客户端等，以实现对这些服务器的纳管，即实现对内部资源集群的纳管。
步骤303、底层管理组件基于内部资源集群部署外部资源管理组件的基础管理组件，或者，基于内部资源集群部署基础公共服务和必需中间件中的至少一类，以及部署外部资源管理组件的基础管理组件。
外部资源管理组件用于对外部资源集群进行管理。外部资源管理组件包括基础管理组件和扩展管理组件。扩展管理组件基于基础管理组件纳管的资源部署得到。该基础管理组件可视为外部资源管理组件向外扩张的根据地，因此，基础管理组件的管控范围至少需要覆盖部署扩展管理组件所需的资源。例如，当外部资源管理组件由基础管理组件和一级扩展管理组件组成，且该扩展管理组件用于对外部资源集群中除部署该扩展管理组件使用的资源外的所有资源进行管理时，基础管理组件用于对部署该扩展管理组件使用的资源进行管理。其中，基础管理组件可以跨地部署，使得基础管理组件具有异地容灾能力，保证基础管理组件的可靠性。
可选地，基础管理组件可以包括基础公共服务和必需中间件中的至少一类。其中，基础公共服务是基础管理组件运行必须要使用的服务。例如，基础公共服务可以为数据库服务。必需中间件是基础管理组件运行必须要使用的中间件。例如，必需中间件可以为缓存和消息队列。此时，该步骤303实际是底层管理组件基于内部资源集群部署整个基础管理组件。在一种实现方式中，其实现过程包括：底层管理组件基于内部资源集群创建第二虚拟实例集群，在第二虚拟实例集群中部署整个基础管理组件。虚拟实例集群包括多个虚拟实例。在本申请实施例中，虚拟实例可以为容器或虚拟机。示例的，第二虚拟实例集群可以为K8s集群，基础管理组件可以为通过容器形态实现的具有管理功能的软件。
或者，基础管理组件也可以不包括基础公共服务和必需中间件中的至少一类。此时，该步骤303实际是底层管理组件基于内部资源集群部署基础公共服务和必需中间件中的至少一类，然后在该基础公共服务和必需中间件中的至少一类的基础上，部署基础管理组件。部署基础管理组件的实现过程请参考上一段中的相关描述。底层管理组件基于内部资源集群部署基础公共服务和必需中间件中的至少一类，其实现过程包括：底层管理组件基于内部资源集群创建第三虚拟实例集群，在第三虚拟实例集群中，部署基础公共服务和必需中间件中的至少一类。示例的，第三虚拟实例集群可以为K8s集群，基础公共服务和必需中间件 可以为通过容器形态实现的具有相关功能的软件。如图6所示，底层管理组件可以基于内部资源集群创建K8s集群，并在K8s集群中分别部署基础公共服务、必需中间件和基础管理组件。其中，第二虚拟实例集群和第三虚拟实例集群可以是底层管理组件一次性地基于全部内部资源集群创建的一个实例集群，也可以是底层管理组件分多次，并在每次基于内部资源集群中的一部分资源创建的实例集群，本申请实施例对其不做具体限定。
当基础管理组件不包括基础公共服务和必需中间件时，通过基于内部资源集群先部署基础公共服务和必需中间件，然后在该基础公共服务和必需中间件的基础上，部署基础管理组件。这样最大化地实现了跨云服务公共能力的共享，降低了基础管理组件的实现代价，简化了基础管理组件的部署复杂度。该部署的实现方式体现了基于服务开发服务(service on service)的原则。其中，基于服务开发服务的原则是指在具有分层、稳定、可持续供应的底层云服务前提下，上层服务应基于该底层云服务进行部署。
执行完步骤303后，就完成了云上资源管理系统中最基础的管理子系统的部署。该最基础的管理子系统是在租户业务量最少的情况下，云上资源管理系统维持正常运行状态具有的最小规模的形态。因此，该最基础的管理子系统也称为云上资源管理系统的最小启动单元(minimum starting unit，MSU)。示例地，该最小启动单元的范围如图6所示。该最小启动单元包括最底层完全自包含的计算和存储根服务，由根服务纳管的内部资源集群，在内部资源集群的基础上发放的基础公共服务和必需中间件，以及具备管理特定规模外部资源集群的基础管理组件。可以看出该最小启动单元相对于外部资源集群独立存在，实现了与外部资源集群的完全解耦，起到了与外部资源集群进行故障隔离的效果。并且，该最小启动单元的规模受控，不会随着外部资源集群容量的增长而同步增长，有效的控制了云上资源管理系统的成本。该最小启动单元相当于计算机的基本输入输出系统(basic input output system，BIOS)，是整个云上资源管理系统最坚实可靠的堡垒，能够以最小代价完成整个云上资源管理系统的引导和初始化，并能够以该最小启动单元为基础扩展出完整的云上资源管理系统，确保严重灾难发生后仍能够以此为基础，逐步重建完整的云上资源管理系统。
步骤304、基础管理组件纳管外部资源集群中的基础资源组。
基础管理组件纳管外部资源集群中的基础资源组，可以通过为提供基础资源组的服务器安装操作系统、分配IP地址、安装客户端等操作实现。在提供基础资源组的服务器上安装操作系统、分配IP地址和安装客户端等后，基础管理组件就能够通过客户端获取服务器的资源使用情况，并对服务器的资源进行分配，因此能够实现对基础资源组的纳管。如图7所示，图7中主机组1提供外部资源集群中的基础资源组，主机组1包括多个服务器，基础管理组件可以在主机组1中每个服务器上安装操作系统、分配IP地址和安装客户端等，以实现对基础资源组的纳管。
步骤305、基础管理组件基于基础资源组向租户提供云服务。
基础管理组件基于基础资源组向租户提供云服务的实现方式有多种。下面以基础管理组件基于基础资源组，直接地和间接地向租户提供云服务的实现方式为例，对该步骤305的实现方式进行说明。
在步骤305的第一种实现方式中，基础管理组件基于基础资源组，间接地向租户提供云服务。如图8所示，其实现过程包括：
步骤305a1、基础管理组件基于基础资源组，部署外部资源管理组件的扩展管理组件。
根据前面描述可知，外部资源管理组件的基础管理组件属于云上资源管理系统的最小启动单元，最小启动单元是云上资源管理系统维持正常运行状态具有的最小规模的形态。当租户业务量增加时，云上资源管理系统管理的资源的规模需要增加，如增加外部资源集群的规模，云上资源管理系统的规模需要相应增加。此时，基础管理组件可以基于基础资源组，部署扩展管理组件，以便于基于该扩展管理组件对增加的资源进行管理。部署扩展管理组件的过程可视为在有业务量增加时，扩展外部资源管理组件规模的过程。这样一来，通过基于内部资源集群部署基础管理组件，基于外部资源集群部署扩展管理组件，能够有效控制基础管理组件占用的资源，限制最小启动单元的规模，在外部资源集群的规模需要根据业务量变化时，使得最小启动单元的规模无需随着外部资源集群的规模的增加而联动增长，具有稳定可靠的特性，且降低了云上资源管理系统的建设成本。
可选的，步骤305a1的实现方式也可以有多种，下面以以下两种实现方式为例对其进行说明。
在部署扩展管理组件的第一种实现方式中，基础管理组件基于基础资源组，部署扩展管理组件，包括：基础管理组件基于基础资源组创建第一虚拟实例集群，在第一虚拟实例集群中部署扩展管理组件。示例的， 第一虚拟实例集群可以为容器集群管理系统K8s集群，扩展管理组件可以为通过容器形态实现的具有管理功能的软件。
在部署扩展管理组件的第二种实现方式中，基础管理组件基于基础资源组，部署扩展管理组件，包括：基础管理组件基于基础资源组创建第一虚拟实例集群，底层管理组件在第一虚拟实例集群中部署扩展管理组件。在该实现方式中，第一虚拟实例集群可以为容器集群管理系统K8s集群，扩展管理组件可以为通过容器形态实现的具有管理功能的软件。
在该第二种实现方式中，可视为基础管理组件提供基于资源虚拟出虚拟实例集群的能力，在基础管理组件创建第一虚拟实例集群后，由底层管理组件对该第一虚拟实例集群进行调度，基于该第一虚拟实例集群部署扩展管理组件。其中，基础管理组件创建第一虚拟实例集群后，可以向底层管理组件提供该第一虚拟实例集群的相关信息，以便于底层管理组件基于该相关信息对第一虚拟实例集群进行调度。在底层管理组件对第一虚拟实例集群进行调度之前，底层管理组件可以将该第一虚拟实例集群与第二虚拟实例集群合并成一个虚拟实例集群，然后对扩张后的虚拟实例集群进行调度。将第一虚拟实例集群与第二虚拟实例集群合并的过程，可视为在第二虚拟实例集群的基础上，加入第一虚拟实例集群，将第二虚拟实例集群扩展为跨越内部资源集群和外部资源集群的逻辑大集群。例如，如图9所示，当第一虚拟实例集群和第二虚拟实例集群都是K8s集群时，可以使用Virtual Kublet技术将基于基础资源组创建的K8s集群与基于内部资源集群创建的K8s集群合并，以将基于内部资源集群创建的K8s集群扩展为跨越内部资源集群和外部资源集群的逻辑大集群。当第二虚拟实例集群与第三虚拟实例集群是不同的集群时，底层管理组件是将第一虚拟实例集群、第二虚拟实例集群和第三虚拟实例集群合并，其合并的实现过程请相应参考将第一虚拟实例集群和第二虚拟实例集群合并的实现过程。进一步的，为保证第一虚拟实例集群和第二虚拟实例集群的隔离性，内部资源集群与外部资源集群可以相互隔离，并使用相对独立的资源集群管理技术。
可选地，外部资源管理组件可以包括多个扩展管理组件，任两个扩展管理组件之间可以相互隔离，和/或，基础管理组件与任一扩展管理组件之间可以相互隔离。例如，外部资源管理组件可以采用网格(grid)化架构，即整个外部管理组件采用多服务格形态，基础管理组件和每个扩展管理组件均为一个独立的服务格。当任两个扩展管理组件之间相互隔离时，能够避免扩展管理组件之间的故障传播，保证了扩展管理组件的可靠性。类似的，当基础管理组件与扩展管理组件之间相互隔离时，能够避免扩展管理组件与基础管理组件之间的故障传播，保证了扩展管理组件和基础管理组件的可靠性。
步骤305a2、扩展管理组件纳管外部资源集群中的扩展资源组，基于扩展资源组创建虚拟实例，在虚拟实例中部署租户的业务。
完成扩展管理组件的部署后，扩展管理组件可以通过为扩展资源组中的服务器安装操作系统、分配IP地址、安装客户端等，以对扩展资源组进行纳管。例如，如图10所示，图10中主机组2、3、4等提供外部资源集群中的扩展资源组，这些主机组均包括多个服务器，扩展管理组件可以在这些主机组中每个服务器上安装操作系统、分配IP地址和安装客户端等，以实现对扩展资源组的纳管。扩展管理组件纳管扩展资源组后，在需要为租户提供云服务时，就可以基于扩展资源组创建虚拟实例，并在虚拟实例中部署租户的业务，以向租户提供云服务。
在步骤305的第二种实现方式中，基础管理组件基于基础资源组，直接地向租户提供云服务。其实现过程包括：基础管理组件基于基础资源组创建虚拟实例，在虚拟实例中部署租户的业务。例如，基础管理组件纳管基础资源组后，在需要为租户提供云服务时，基础管理组件可以基于基础资源组创建虚拟实例，并在该虚拟实例中部署租户的业务，以向租户提供云服务。
可选地，除了基于基础资源组部署租户的业务，基础管理组件还可以基于基础资源组，部署扩展管理组件。该扩展管理组件用于对外部资源集群中的扩展资源组进行纳管，并基于扩展资源组创建虚拟实例，在虚拟实例中部署租户的业务。其中，部署扩展管理组件的实现方式，扩展管理组件部署租户业务的实现方式，可以相应参考前面的相关内容，此处不再赘述。
在一种实现方式中，由于该扩展管理组件用于对外部资源集群中的扩展资源组进行纳管，并基于其为租户提供云服务，为保证该扩展资源组能够被有效利用，扩展管理组件相比于用于部署租户的虚拟实例，可以具有使用基础资源组的优先权。也即是，在基础管理组件可以优先使用基础资源组部署扩展管理组件，并在需要部署租户的业务时，基于基础资源组中未被扩展管理组件占用的剩余资源组创建虚拟实例，在虚拟实例中部署租户的业务。例如，如图11所示，在基础管理组件基于基础资源组创建虚拟实例，在虚拟实 例中部署租户的业务之前，该方法还包括：
步骤305b1、基础管理组件基于基础资源组，部署扩展管理组件。
该步骤305b1的实现过程请相应参考步骤305a1的实现过程，此处不再赘述。
此时，如图11所示，基础管理组件基于基础资源组创建虚拟实例，在虚拟实例中部署租户的业务，包括：步骤305b2、基础管理组件基于基础资源组中未被扩展管理组件占用的剩余资源组创建虚拟实例，在虚拟实例中部署租户的业务。
根据上面内容可知，最小启动单元用于提供部署在其中的各类服务(包括基础管理组件、基础公共服务和必需中间件)的起步规格，满足平常业务负载的资源需要，在业务高峰时段，需部署在外部资源集群中部署扩展管理组件，以增加资源集群的业务处理能力。因此，在业务高峰过去后，可以将其在外部资源池中占用的资源及时释放掉，包括释放部署对应业务的使用的资源和扩展管理组件占用的资源，实现对资源基于的弹性伸缩，以优化云上资源管理系统的成本占用。
综上所述，在本申请实施例提供的云上资源管理系统的部署方法中，首先构建最小启动单元中对外完全无依赖的底层管理组件，利用底层管理组件纳管内部资源集群，然后基于内部资源集群部署外部资源管理组件的基础管理组件、基础公共服务和必需中间件，然后构建外部资源管理组件的扩展管理组件，进而将云上资源管理系统的管控面从最小启动单元扩展部署到外部资源集群中。这种依靠自举和偏序依赖构建云上资源管理系统的方式，既可以保证最小启动单元的内部资源池规模受限，又有助于提升整体资源利用率，有效地降低了云上资源管理系统的建设成本。并且，这种在最小启动单元的基础上实现整个云上资源管理系统的部署的方法，不仅使得云上资源管理系统具备管控超大规模基础设施资源池的能力，还在控制云上资源管理系统的成本的基础上，实现了最小启动单元与外部资源集群的隔离，避免了互相之间的故障扩散。另外，当该云上资源管理系统在资源管控过程中发放同一种资源形态的虚拟实例时，能够确保最小启动单元足够简单和稳定，较易实现最小启动单元的自包含，能够保证云上资源管理系统的可靠性。同时，扩展管理组件所需的部署资源，大部分都扩展在外部资源池中，与租户共享资源池，在较大程度上避免了硬件资源池的碎片化，有助于成本优化。
需要说明的是，本申请实施例提供的云上资源管理系统的部署方法的步骤先后顺序可以进行适当调整，步骤也可以根据情况进行相应增减。任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化的方法，都应涵盖在本申请的保护范围之内，因此不再赘述。
以上介绍了本申请实施例的云上资源管理系统的部署方法，与上述方法对应，本申请实施例还提供了云上资源管理系统。图7和图10是本申请实施例提供的云上资源管理系统的结构示意图。应理解到，该云上资源管理系统可以包括比所示组件更多的附加组件或者省略其中所示的一部分组件，本申请实施例对此并不进行限制。可选的，该云上资源管理系统可配置于云平台。如图7所示，该云上资源管理系统可以包括：云上资源管理节点，底层管理组件和外部资源管理组件。
底层管理组件用于纳管内部资源集群，基于内部资源集群部署外部资源管理组件的基础管理组件。
外部资源管理组件用于纳管外部资源集群，其中，外部资源管理组件包括基础管理组件，基础管理组件用于纳管外部资源集群中的基础资源组，基于基础资源组创建虚拟实例，在虚拟实例中部署租户的业务。
在该云上资源管理系统中，基础管理组件仅纳管外部资源集群中的基础资源组，而不是纳管外部资源集群的所有资源。这样一方面控制了基础管理组件纳管的外部资源集群的资源量，有效控制基础管理组件占用的资源，减小了基础管理组件的规模随着外部资源集群的规模的增加而联动增长的概率，有助于降低云上资源管理系统的建设成本。另一方面，通过基础管理组件对外部资源集群中部分资源进行纳管，为基础管理组件基于外部资源集群扩展出云上资源管理系统的整个管控面提供了条件，使得云上资源管理系统具备管控超大规模基础设施资源池的能力。
在一种实现方式中，如图10所示，该系统还包括：外部资源管理组件的扩展管理组件。此时，基础管理组件具体用于基于基础资源组，部署扩展管理组件。基础管理组件具体通过扩展管理组件向租户提供云服务。例如，扩展管理组件用于纳管外部资源集群中的扩展资源组，基于扩展资源组创建虚拟实例，在虚拟实例中部署租户的业务。
或者，基础管理组件具体用于基于基础资源组创建虚拟实例，在虚拟实例中部署租户的业务。
可选地，在这种实现方式中，基础管理组件还用于基于基础资源组，部署扩展管理组件。则基础管理 组件具体用于基于基础资源组中未被扩展管理组件占用的剩余资源组创建虚拟实例，在虚拟实例中部署租户的业务。
以上部署扩展管理组件的过程，可视为在有业务量增加时，进一步部署外部资源管理组件的过程。这样一来，通过基于内部资源集群部署基础管理组件，基于外部资源集群部署扩展管理组件，能够有效控制基础管理组件占用的剩余资源组，限制最小启动单元的规模，在外部资源集群的规模需要根据业务量变化时，使得最小启动单元的规模无需随着外部资源集群的规模的增加而联动增长，具有稳定可靠的特性，且降低了云上资源管理系统的建设成本。
在一种实现方式中，基础管理组件具体用于基于基础资源组创建第一虚拟实例集群，在第一虚拟实例集群中部署扩展管理组件。
在另一种实现方式中，基础管理组件具体用于基于基础资源组创建第一虚拟实例集群；底层管理组件还用于在第一虚拟实例集群中部署扩展管理组件。
可选地，部署底层管理组件使用的资源与外部资源集群隔离，和/或，与内部资源集群隔离。另外，内部资源集群与外部资源集群也可以互相隔离。
可选地，为了进一步保证云上资源管理系统的安全性，可以在多个云上资源管理节点上部署底层管理组件，使得底层管理组件具有容灾能力。
在一种实现方式中，底层管理组件具体用于基于内部资源集群创建第二虚拟实例集群，在第二虚拟实例集群中部署外部资源管理组件的基础管理组件。
可选地，基础管理组件可以包括基础公共服务和必需中间件中的至少一类。其中，公共服务是云上资源管理系统通过外部资源集群能够提供的云服务能够公用的服务。例如，公共服务可以为数据库服务。必需中间件是云上资源管理系统通过外部资源集群能够提供的云服务都需要使用到的中间件。例如，必需中间件可以为缓存和消息队列。
或者，基础管理组件也可以不包括基础公共服务和必需中间件中的至少一类。在一种实现方式中，底层管理组件基于内部资源集群部署基础公共服务和必需中间件中的至少一类，其实现过程包括：底层管理组件基于内部资源集群创建第三虚拟实例集群，在第三虚拟实例集群中，部署基础公共服务和必需中间件中的至少一类。
包含底层管理组件和基础管理组件的子系统可以视为云上资源管理系统中最基础的管理子系统。该最基础的管理子系统是在租户业务量最少的情况下，云上资源管理系统维持正常运行状态具有的最小规模的形态。因此，该最基础的管理子系统也称为云上资源管理系统的最小启动单元。该最小启动单元包括最底层完全自包含的计算和存储根服务，由根服务纳管的内部资源集群，在内部资源集群的基础上发放的基础公共服务和必需中间件，以及具备管理特定规模外部资源集群的基础管理组件。可以看出该最小启动单元相对于外部资源集群独立存在，实现了与外部资源集群的完全解耦，起到了与外部资源集群进行故障隔离的效果。并且，该最小启动单元的规模受控，不会随着外部资源集群容量的增长而同步增长，有效的控制了云上资源管理系统的成本。该最小启动单元相当于计算机的基本输入输出系统，是整个云上资源管理系统最坚实可靠的堡垒，能够以最小代价完成整个云上资源管理系统的引导和初始化，并能够以该最小启动单元为基础扩展出完整的云上资源管理系统，确保严重灾难发生后仍能够以此为基础，逐步重建完整的云上资源管理系统。
可选地，外部资源管理组件可以包括多个扩展管理组件，任两个扩展管理组件之间可以相互隔离，和/或，基础管理组件与任一扩展管理组件之间可以相互隔离。在一种实现方式中，虚拟实例为容器或虚拟机。
综上所述，本申请实施例提供的云上资源管理系统，既可以保证最小启动单元的内部资源池规模受限，又有助于提升整体资源利用率，有效地降低了云上资源管理系统的建设成本。并且，该云上资源管理系统通过在最小启动单元的基础上部署得到，不仅使得云上资源管理系统具备管控超大规模基础设施资源池的能力，还在控制云上资源管理系统的成本的基础上，实现了最小启动单元与外部资源集群的隔离，避免了互相之间的故障扩散。另外，当该云上资源管理系统在资源管控过程中发放同一种资源形态的虚拟实例时，能够确保最小启动单元足够简单和稳定，较易实现最小启动单元的自包含，能够保证云上资源管理系统的可靠性。同时，扩展管理组件所需的部署资源，大部分都扩展在外部资源池中，与租户共享资源池，在较大程度上避免了硬件资源池的碎片化，有助于成本优化。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的云上资源管理系统和各组 件的具体工作过程，可以参考前述方法实施例中的对应内容，在此不再赘述。
本申请实施例提供了一种计算设备。该计算设备用于实现本申请实施例提供的云上资源管理系统的部署方法中的部分或全部功能。图12是本申请实施例提供的一种计算设备的结构示意图。如图12所示，该计算设备1200包括处理器1201、存储器1202、通信接口1203和总线1204。其中，处理器1201、存储器1202、通信接口1203通过总线1204实现彼此之间的通信连接。
处理器1201可以包括通用处理器和/或专用硬件芯片。通用处理器可以包括：中央处理器(central processing unit，CPU)、微处理器或图形处理器(graphics processing unit，GPU)。CPU例如是一个单核处理器(single-CPU)，又如是一个多核处理器(multi-CPU)。专用硬件芯片是一个高性能处理的硬件模块。专用硬件芯片包括数字信号处理器、专用集成电路(application-specific integrated circuit，ASIC)、现场可编程逻辑门阵列(field-programmable gate array，FPGA)或者网络处理器(network processer，NP)中的至少一项。处理器1201还可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，本申请的云上资源管理系统的部署方法的部分或全部功能，可以通过处理器1201中的硬件的集成逻辑电路或者软件形式的指令完成。
存储器1202用于存储计算机程序，计算机程序包括操作系统1202a和可执行代码(即程序指令)1202b。存储器1202例如是只读存储器或可存储静态信息和指令的其它类型的静态存储设备，又如是随机存取存储器或者可存储信息和指令的其它类型的动态存储设备，又如是电可擦可编程只读存储器、只读光盘或其它光盘存储、光碟存储(包括压缩光碟、激光碟、光碟、数字通用光碟、蓝光光碟等)、磁盘存储介质或者其它磁存储设备，或者是能够用于携带或存储具有指令或数据结构形式的期望的可执行代码并能够由计算机存取的任何其它介质，但不限于此。例如存储器1202用于存放出端口队列等。存储器1202例如是独立存在，并通过总线1204与处理器1201相连接。或者存储器1202和处理器1201集成在一起。存储器1202可以存储可执行代码，当存储器1202中存储的可执行代码被处理器1201执行时，处理器1201用于执行本申请实施例提供的云上资源管理系统的部署方法的部分或全部功能。处理器1201执行该过程的实现方式请相应参考前述实施例中的相关描述。存储器1202中还可以包括操作系统等其他运行进程所需的软件模块和数据等。
通信接口1203使用例如但不限于收发器一类的收发模块，来实现与其他设备或通信网络之间的通信。例如，通信接口1203可以是以下器件的任一种或任一种组合：网络接口(如以太网接口)、无线网卡等具有网络接入功能的器件。
总线1204是任何类型的，用于实现计算设备的内部器件(例如，存储器1202、处理器1201、通信接口1203)互连的通信总线。例如系统总线。本申请实施例以计算设备内部的上述器件通过总线1204互连为例说明，可选地，计算设备1200内部的上述器件还可以采用除了总线1204之外的其他连接方式彼此通信连接。例如，计算设备1200内部的上述器件通过内部的逻辑接口互连。
需要说明的是，上述多个器件可以分别设置在彼此独立的芯片上，也可以至少部分的或者全部的设置在同一块芯片上。将各个器件独立设置在不同的芯片上，还是整合设置在一个或者多个芯片上，往往取决于产品设计的需要。本申请实施例对上述器件的具体实现形式不做限定。且上述各个附图对应的流程的描述各有侧重，某个流程中没有详述的部分，可以参见其他流程的相关描述。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。提供程序开发平台的计算机程序产品包括一个或多个计算机指令，在计算设备上加载和执行这些计算机程序指令时，全部或部分地实现本申请实施例提供的云上资源管理系统的部署方法的功能。
并且，计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线)或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质存储有提供程序开发平台的计算机程序指令。
本申请实施例还提供了一种计算设备集群。该计算设备集群包括至少一台计算设备。该计算设备可以是服务器，例如是中心服务器、边缘服务器，或者是本地数据中心中的本地服务器。在一些实施例中，计 算设备也可以是台式机、笔记本电脑或者智能手机等终端设备。
可选地，计算设备集群包括的至少一个计算设备的结构可参见图12示出的计算设备1200。计算设备集群中的一个或多个计算设备1200中的存储器1202中可以存有相同的用于执行云上资源管理系统的部署方法的指令。
在一些可能的实现方式中，该计算设备集群中的一个或多个计算设备1200的存储器1202中也可以分别存有用于执行云上资源管理系统的部署方法的部分指令。换言之，一个或多个计算设备1200的组合可以共同执行用于执行云上资源管理系统的部署方法的指令。
需要说明的是，计算设备集群中的不同的计算设备1200中的存储器1202可以存储不同的指令，分别用于执行云上资源管理系统的部分功能。也即，不同的计算设备1200中的存储器1202存储的指令可以实现第一确定模块、第二确定模块和第三确定模块中的一个或多个模块的功能。
在一些可能的实现方式中，计算设备集群中的多个计算设备可以通过网络连接。其中，所述网络可以是广域网或局域网等等。任意两个计算设备之间通过网络进行连接。具体地，通过各个计算设备中的通信接口与所述网络进行连接。在这一类可能的实现方式中，每个计算设备包括总线、处理器、存储器和通信接口。
本申请实施例还提供了一种计算机可读存储介质，该计算机可读存储介质为非易失性计算机可读存储介质，该计算机可读存储介质包括程序指令，当程序指令在计算设备上运行时，使得计算设备实现如本申请实施例提供的云上资源管理系统的部署方法。
本申请实施例还提供了一种包含指令的计算机程序产品，当计算机程序产品在计算机上运行时，使得计算机实现本申请实施例提供的云上资源管理系统的部署方法。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成，也可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，上述提到的存储介质可以是只读存储器，磁盘或光盘等。
需要说明的是，本申请所涉及的信息(包括但不限于用户设备信息、用户个人信息等)、数据(包括但不限于用于分析的数据、存储的数据、展示的数据等)以及信号，均为经用户授权或者经过各方充分授权的，且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准。例如，本申请中涉及到的原始数据和可执行代码等都是在充分授权的情况下获取的。
在本申请实施例中，术语“第一”、“第二”和“第三”仅用于描述目的，而不能理解为指示或暗示相对重要性。术语“至少一个”是指一个或多个，术语“多个”指两个或两个以上，除非另有明确的限定。
本申请中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。
以上所述仅为本申请的可选实施例，并不用以限制本申请，凡在本申请的构思和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (21)

  1. 一种云上资源管理系统，其特征在于，包括：云上资源管理节点，底层管理组件和外部资源管理组件，其中：
    所述云上资源管理节点用于部署所述底层管理组件；
    所述底层管理组件用于纳管内部资源集群，基于所述内部资源集群部署所述外部资源管理组件；
    所述外部资源管理组件用于纳管外部资源集群，其中，所述外部资源管理组件包括基础管理组件，所述基础管理组件用于纳管所述外部资源集群中的基础资源组，基于所述基础资源组创建虚拟实例，在所述虚拟实例中部署租户的业务。
  2. 根据权利要求1所述的系统，其特征在于，所述外部资源管理组件还包括扩展管理组件；
    所述基础管理组件还用于基于所述基础资源组，部署所述扩展管理组件；
    所述扩展管理组件用于纳管所述外部资源集群中的扩展资源组，基于所述扩展资源组创建虚拟实例，在所述虚拟实例中部署所述租户的业务。
  3. 根据权利要求1所述的系统，其特征在于，所述外部资源管理组件还包括扩展管理组件；
    所述基础管理组件还用于基于所述基础资源组，部署所述扩展管理组件，所述扩展管理组件用于纳管所述外部资源集群中的扩展资源组；
    所述基础管理组件还用于基于所述基础资源组中未被所述扩展管理组件占用的剩余资源组创建虚拟实例，在所述虚拟实例中部署所述租户的业务。
  4. 根据权利要求2或3所述的系统，其特征在于，
    所述基础管理组件还用于基于所述基础资源组创建第一虚拟实例集群，在所述第一虚拟实例集群中部署所述扩展管理组件。
  5. 根据权利要求2或3所述的系统，其特征在于，
    所述基础管理组件还用于基于所述基础资源组创建第一虚拟实例集群；
    所述底层管理组件还用于在所述第一虚拟实例集群中部署所述扩展管理组件。
  6. 根据权利要求1至5任一所述的系统，其特征在于，
    所述底层管理组件还用于基于所述内部资源集群创建第二虚拟实例集群，在所述第二虚拟实例集群中部署所述外部资源管理组件。
  7. 根据权利要求1至6任一所述的系统，其特征在于，所述基础管理组件包括所述基础公共服务和必需中间件中的至少一类。
  8. 根据权利要求1至6任一所述的系统，其特征在于，所述系统还包括：所述基础公共服务和必需中间件中的至少一类；
    所述底层管理组件还用于基于所述内部资源集群，部署所述基础公共服务和必需中间件中的至少一类。
  9. 根据权利要求8所述的系统，其特征在于，
    所述底层管理组件还用于基于所述内部资源集群创建第三虚拟实例集群，在所述第三虚拟实例集群中部署所述基础公共服务和必需中间件中的至少一类。
  10. 一种云上资源管理系统的部署方法，其特征在于，所述方法应用于云上资源管理系统，所述云上资源管理系统包括云上资源管理节点，底层管理组件和外部资源管理组件，所述方法包括：
    所述云上资源管理节点部署所述底层管理组件；
    所述底层管理组件纳管内部资源集群，基于所述内部资源集群部署所述外部资源管理组件；
    所述外部资源管理组件纳管外部资源集群，其中，所述外部资源管理组件包括基础管理组件，所述基础管理组件纳管所述外部资源集群中的基础资源组，基于所述基础资源组创建虚拟实例，在所述虚拟实例中部署租户的业务。
  11. 根据权利要求10所述的方法，其特征在于，所述外部资源管理组件还包括扩展管理组件，所述方法还包括：
    所述基础管理组件基于所述基础资源组，部署所述扩展管理组件；
    所述扩展管理组件纳管所述外部资源集群中的扩展资源组，基于所述扩展资源组创建虚拟实例，在所述虚拟实例中部署所述租户的业务。
  12. 根据权利要求10所述的方法，其特征在于，所述外部资源管理组件还包括扩展管理组件，所述方法还包括：
    所述基础管理组件基于所述基础资源组，部署所述扩展管理组件，所述扩展管理组件用于纳管所述外部资源集群中的扩展资源组；
    所述基础管理组件基于所述基础资源组中未被所述扩展管理组件占用的剩余资源组创建虚拟实例，在所述虚拟实例中部署所述租户的业务。
  13. 根据权利要求11或12所述的方法，其特征在于，所述基础管理组件基于所述基础资源组，部署所述扩展管理组件，包括：
    所述基础管理组件基于所述基础资源组创建第一虚拟实例集群，在所述第一虚拟实例集群中部署所述扩展管理组件。
  14. 根据权利要求11或12所述的方法，其特征在于，所述基础管理组件基于所述基础资源组，部署所述扩展管理组件，包括：
    所述基础管理组件基于所述基础资源组创建第一虚拟实例集群；
    所述底层管理组件在所述第一虚拟实例集群中部署所述扩展管理组件。
  15. 根据权利要求10至14任一所述的方法，其特征在于，所述底层管理组件基于所述内部资源集群部署所述外部资源管理组件，包括：
    所述底层管理组件基于所述内部资源集群创建第二虚拟实例集群，在所述第二虚拟实例集群中部署所述外部资源管理组件。
  16. 根据权利要求10至15任一所述的方法，其特征在于，所述基础管理组件包括所述基础公共服务和必需中间件中的至少一类。
  17. 根据权利要求10至15任一所述的方法，其特征在于，所述方法还包括：
    所述底层管理组件基于所述内部资源集群，部署所述基础公共服务和必需中间件中的至少一类。
  18. 根据权利要求17所述的方法，其特征在于，所述底层管理组件基于所述内部资源集群，部署所述基础公共服务和中间件中的至少一类，包括：
    所述底层管理组件基于所述内部资源集群创建第三虚拟实例集群，在所述第三虚拟实例集群中部署所述基础公共服务和必需中间件中的至少一类。
  19. 一种计算设备集群，其特征在于，包括多个计算设备，所述多个计算设备包括多个处理器和多个存储器，所述多个存储器中存储有程序指令，所述多个处理器运行程序指令，使得所述计算设备集群执行权 利要求10至18任一所述的方法。
  20. 一种计算机可读存储介质，其特征在于，包括程序指令，当所述程序指令在计算设备上运行时，使得所述计算设备执行如权利要求10至18任一所述的方法。
  21. 一种计算机程序产品，其特征在于，当所述计算机程序产品在计算机上运行时，使得所述计算机执行如权利要求10至18任一所述的方法。
PCT/CN2023/104680 2022-10-24 2023-06-30 云上资源管理系统及其部署方法 WO2024087717A1 (zh)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202211300013 2022-10-24
CN202211300013.1 2022-10-24
CN202310402195.1 2023-04-14
CN202310402195.1A CN117978802A (zh) 2022-10-24 2023-04-14 云上资源管理系统及其部署方法

Publications (1)

Publication Number Publication Date
WO2024087717A1 true WO2024087717A1 (zh) 2024-05-02

Family

ID=90829909

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/104680 WO2024087717A1 (zh) 2022-10-24 2023-06-30 云上资源管理系统及其部署方法

Country Status (1)

Country Link
WO (1) WO2024087717A1 (zh)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102314391A (zh) * 2011-05-27 2012-01-11 运软网络科技(上海)有限公司 一种基于硬件基础工具监视虚拟化系统性能的架构及方法
US20140075499A1 (en) * 2012-09-07 2014-03-13 Oracle International Corporation Security infrastructure for cloud services
CN106033648A (zh) * 2015-03-13 2016-10-19 浙江成瑞软件科技有限公司 基于云平台的交互式系统、交互式联合教育系统及其方法
CN107959588A (zh) * 2017-12-07 2018-04-24 郑州云海信息技术有限公司 数据中心的云资源管理方法、云资源管理平台和管理系统
CN109032785A (zh) * 2018-08-14 2018-12-18 北京交通大学 一种基于虚拟桌面的工作流程管控方法及系统
CN110324424A (zh) * 2019-05-13 2019-10-11 上海分布信息科技有限公司 一种基于区块链的可信云服务架构
CN114090329A (zh) * 2020-07-30 2022-02-25 华为云计算技术有限公司 一种全卸载架构下的服务器重启方法及相关设备

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102314391A (zh) * 2011-05-27 2012-01-11 运软网络科技(上海)有限公司 一种基于硬件基础工具监视虚拟化系统性能的架构及方法
US20140075499A1 (en) * 2012-09-07 2014-03-13 Oracle International Corporation Security infrastructure for cloud services
CN106033648A (zh) * 2015-03-13 2016-10-19 浙江成瑞软件科技有限公司 基于云平台的交互式系统、交互式联合教育系统及其方法
CN107959588A (zh) * 2017-12-07 2018-04-24 郑州云海信息技术有限公司 数据中心的云资源管理方法、云资源管理平台和管理系统
CN109032785A (zh) * 2018-08-14 2018-12-18 北京交通大学 一种基于虚拟桌面的工作流程管控方法及系统
CN110324424A (zh) * 2019-05-13 2019-10-11 上海分布信息科技有限公司 一种基于区块链的可信云服务架构
CN114090329A (zh) * 2020-07-30 2022-02-25 华为云计算技术有限公司 一种全卸载架构下的服务器重启方法及相关设备

Similar Documents

Publication Publication Date Title
US11263719B2 (en) Graphical rendering using multiple graphics processors
CN107220039B (zh) 一种基于云环境的异构资源标准化封装系统
US8762999B2 (en) Guest-initiated resource allocation request based on comparison of host hardware information and projected workload requirement
US9021046B2 (en) Provisioning server resources in a cloud resource
EP2354935A1 (en) Extending functionality of legacy services in computing system environment
CN113900773A (zh) 一种基于国产平台虚拟机和容器的超融合调度方法
CN102567116A (zh) 数字家庭的云计算和虚拟化系统
CN116800616B (zh) 虚拟化网络设备的管理方法及相关装置
US11184244B2 (en) Method and system that determines application topology using network metrics
Yang et al. High-performance docker integration scheme based on OpenStack
WO2024087717A1 (zh) 云上资源管理系统及其部署方法
US20230138867A1 (en) Methods for application deployment across multiple computing domains and devices thereof
Rac et al. At the edge of a seamless cloud experience
CN114615268B (zh) 基于Kubernetes集群的服务网络、监控节点、容器节点及设备
CN114816665B (zh) 混合编排系统及超融合架构下虚拟机容器资源混合编排方法
CN116954810A (zh) 容器应用实例的创建方法、系统、存储介质及程序产品
CN111562968B (zh) 一种实现ICS对Kata container管理的方法、装置、设备和介质
CN117978802A (zh) 云上资源管理系统及其部署方法
CN112181401A (zh) 应用构建方法及应用构建平台
WO2023274014A1 (zh) 容器集群的存储资源管理方法、装置及系统
US11704159B2 (en) System and method for unified infrastructure architecture
Zhao et al. The deployment of FPGA Based on Network in Ultra-large-scale Data Center
Pires Caravela: Cloud@ Edge
CN115426251A (zh) 一种云主机的容灾方法、装置及介质
Fonseca FairCloud-Auction-driven Cloud Scheduling

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23881306

Country of ref document: EP

Kind code of ref document: A1