CN112965817B - Resource management method and device and electronic equipment - Google Patents

Resource management method and device and electronic equipment Download PDF

Info

Publication number
CN112965817B
CN112965817B CN202110200163.4A CN202110200163A CN112965817B CN 112965817 B CN112965817 B CN 112965817B CN 202110200163 A CN202110200163 A CN 202110200163A CN 112965817 B CN112965817 B CN 112965817B
Authority
CN
China
Prior art keywords
cluster
virtual machine
slave
node
offline task
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110200163.4A
Other languages
Chinese (zh)
Other versions
CN112965817A (en
Inventor
王端
李想成
吴爽
史征宇
赵增
刘柏
李仁杰
范长杰
胡志鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Netease Hangzhou Network Co Ltd
Original Assignee
Netease Hangzhou Network Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Netease Hangzhou Network Co Ltd filed Critical Netease Hangzhou Network Co Ltd
Priority to CN202110200163.4A priority Critical patent/CN112965817B/en
Publication of CN112965817A publication Critical patent/CN112965817A/en
Application granted granted Critical
Publication of CN112965817B publication Critical patent/CN112965817B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)
  • Stored Programmes (AREA)

Abstract

The invention provides a resource management method, a resource management device and electronic equipment. Wherein, this method is applied to virtual machine cluster, and this method includes: determining idle resources of a virtual machine cluster; determining tide nodes of the virtual machine cluster according to the idle resources; constructing a slave cluster of the container technology cluster based on the tidal node; transmitting configuration information of the slave clusters to a master cluster of the container technology cluster; the configuration information is used to characterize a first execution environment from tidal nodes in the cluster; responding to a target offline task sent by a main cluster of the container technology cluster, and calling a slave cluster to execute the target offline task; the target offline task is an offline task in the main cluster, wherein the execution environment of the offline task is the same as the first execution environment. The method can realize the target offline task of the virtual machine cluster execution container technology, which is sent from the cluster, can improve the resource utilization rate of the virtual machine, does not need to expand K8s Node and adapting work, has low cost and is beneficial to rapid expansion and contraction of resources.

Description

Resource management method and device and electronic equipment
Technical Field
The present invention relates to the field of cloud services, and in particular, to a method and an apparatus for resource management, and an electronic device.
Background
The container technology is a technology which is popular at present and is the forefront, and after the container technology is pushed out from a docker (application container engine) and a k8s (container orchestration engine), the infrastructure based on k8s cluster design is more popular, the deployment of software is easier, and the 'one-time deployment and operation everywhere' is truly realized. However, a large number of application scenarios exist on the basis of the virtual machine technology because of factors such as similar usage mode and physical machine, convenience in old service migration and the like.
In different business scenarios, container technology or virtual machine technology needs to be used. The scenario using the virtual machine technology has a problem of low resource utilization, because the user usually uses the virtual machine as a remote host, and is generally active during daytime working hours, and a large amount of non-working hours are in an idle state.
In the scene of using the container technology, kubuead of K8s and some third-party open source schemes (for example Cluster Autoscaler) can automatically expand K8s Node (K8 s working Node) when K8s cluster resources are insufficient, so as to adapt to the calculation power requirement expansion of offline tasks and achieve the purpose of elastic calculation power.
However, the above-described manner shares API (Application Programming Interface, application program interface) support of clouds, such as AWS (Amazon Web Services, amazon cloud service), aliCloud (AliCloud), and the like; or K8s Node is built on a bare computer manually, cluster is added to complete the process, high cost is required for new machines or new public cloud resources, and flexibility is not high for machines or public cloud resources with different configurations. For example: the characteristics of the new machine lead to the fact that K8s with different versions are required to be used, or the resources provided by the public cloud of the third party are inconsistent with the existing Cluster version, so that a large amount of adaptation work is required, and the quick expansion and contraction of the resources are not facilitated.
In summary, the scenario using the virtual machine technology has the problem of low resource utilization rate, and the scenario using the container technology can expand K8s Node, but requires higher cost and a large amount of adaptation work, which is not beneficial to rapid expansion and contraction of resources.
Disclosure of Invention
Therefore, the present invention aims to provide a resource management method, a device and an electronic device, so as to improve the resource utilization rate of a virtual machine, without expanding the K8s Node and adapting work, with low cost and being beneficial to rapid expansion and contraction of resources.
In a first aspect, an embodiment of the present invention provides a resource management method, applied to a virtual machine cluster, where the method includes: determining idle resources of a virtual machine cluster; determining tide nodes of the virtual machine cluster according to the idle resources; constructing a slave cluster of the container technology cluster based on the tidal node; transmitting configuration information of the slave clusters to a master cluster of the container technology cluster; the configuration information is used to characterize a first execution environment from tidal nodes in the cluster; responding to a target offline task sent by a main cluster of the container technology cluster, and calling a slave cluster to execute the target offline task; the target offline task is an offline task in the main cluster, wherein the execution environment of the offline task is the same as the first execution environment.
In a preferred embodiment of the present invention, the step of constructing the slave cluster of the container technology cluster based on tidal nodes comprises: determining at least one mirror node from the tidal node; establishing a slave cluster of the container technology cluster in the virtual machine cluster according to the mirror node; selecting at least one idle resource from the virtual machine cluster, and adding the idle resource into the slave cluster as a tide node; the step of calling the target offline task to be executed from the cluster comprises the following steps: the tidal node is caused to perform a target offline task by starting the tidal node from the mirror node of the cluster.
In a preferred embodiment of the present invention, the step of selecting at least one free resource from the virtual machine cluster and adding the free resource as a tide node to the slave cluster includes: determining idle resources in the virtual machine cluster, and determining tide nodes based on the idle resources; establishing a working node and a master node of the slave cluster based on the tidal node; the step of sending configuration information of the slave cluster to the master cluster of the container technology cluster includes: transmitting configuration information of the slave cluster to a master cluster of the container technology cluster by a master node of the slave cluster; the step of calling the target offline task to be executed from the cluster comprises the following steps: receiving a target offline task from a main node of the cluster, and distributing the target offline task to a corresponding working node; and executing the target offline task through the corresponding working node.
In a preferred embodiment of the present invention, the determining the idle resources of the virtual machine cluster determines the idle resources according to the monitoring information time sequence of the virtual machine cluster; a step of determining tidal nodes of a virtual machine cluster from the free resources, comprising: the tidal nodes are determined according to the order of the size of the free resources.
In a preferred embodiment of the present invention, the step of determining the idle resources according to the monitoring information time sequence of the virtual machine cluster includes: determining the variance of the time sequence, the idle resource size of the virtual machine cluster and the resource proportion of the virtual machine cluster according to the monitoring information time sequence of the virtual machine cluster; the resource ratio represents the ratio of processor resources to memory resources of the virtual machine cluster; determining a resource reduction ratio of the virtual machine cluster based on the idle resource size of the virtual machine cluster; and selecting resources with the time sequence variance smaller than a preset variance threshold from the virtual machine cluster based on the resource reduction ratio, wherein the resource proportion of the virtual machine cluster meets the preset proportion condition as idle resources.
In a preferred embodiment of the present invention, after the step of constructing the slave clusters of the container technology cluster based on the tidal nodes, the method further comprises: and sending the cluster identification information and the key information of the slave cluster to the master cluster of the container technology cluster, so that the master cluster sends the target offline task to the slave cluster based on the cluster identification information and the key information of the slave cluster.
In a preferred embodiment of the present invention, the method further includes: monitoring the resource utilization rate of the virtual machine cluster; and if the resource utilization exceeds a preset first threshold, performing target offline task deletion and/or tide node deletion from the cluster.
In a preferred embodiment of the present invention, the step of deleting the target offline task from the cluster includes: interrupting the task that the tidal node is in an executing state; the task of tidal nodes in wait state is deleted.
In a second aspect, an embodiment of the present invention further provides a master cluster applied to a container technology cluster, where the method includes: receiving configuration information of a slave cluster sent by a virtual machine cluster; wherein the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster, the tidal node being a free resource in the virtual machine cluster; acquiring a target offline task and a second execution environment of the target offline task; judging whether the first execution environment is the same as the second execution environment; if so, the target offline task is sent to the virtual machine cluster, so that the virtual machine cluster executes the target offline task through the slave cluster.
In a preferred embodiment of the present invention, the step of receiving configuration information of a slave cluster sent by a virtual machine cluster includes: receiving cluster identification and cluster authentication information of a virtual machine cluster; searching configuration information of a virtual machine cluster from cluster authentication information; the step of sending the target offline task to the virtual machine cluster includes: and sending the target offline task to the virtual machine cluster based on the key information in the cluster authentication information and the cluster identification of the virtual machine cluster.
In a preferred embodiment of the present invention, the method further includes: monitoring the resource utilization rate of a slave cluster of the container technology cluster; if the resource utilization rate of the slave cluster is higher than a preset second threshold value, scheduling the task in a waiting state in the slave cluster to other slave clusters or the master cluster; the other slave clusters are slave clusters whose resource utilization is below a second threshold.
In a third aspect, an embodiment of the present invention further provides a resource management device, applied to a virtual machine cluster, where the device includes: the idle resource determining module is used for determining idle resources of the virtual machine cluster; the tide node determining module is used for determining tide nodes of the virtual machine cluster according to the idle resources; a slave cluster construction module for constructing a slave cluster of the container technology cluster based on the tidal node; the configuration information sending module is used for sending the configuration information of the slave cluster to the master cluster of the container technology cluster; the configuration information is used to characterize a first execution environment from tidal nodes in the cluster; the offline task execution module is used for responding to a target offline task sent by a main cluster of the container technology cluster and calling a slave cluster to execute the target offline task; the target offline task is an offline task in the main cluster, wherein the execution environment of the offline task is the same as the first execution environment.
In a fourth aspect, an embodiment of the present invention further provides a resource management device, applied to a master cluster of a container technology cluster, where the device includes: the configuration information receiving module is used for receiving configuration information of the slave cluster sent by the virtual machine cluster; wherein the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster, the tidal node being a free resource in the virtual machine cluster; the offline task acquisition module is used for acquiring a target offline task and a second execution environment of the target offline task; the execution environment judging module is used for judging whether the first execution environment is the same as the second execution environment; and the offline task sending module is used for sending the target offline task to the virtual machine cluster if the target offline task is sent to the virtual machine cluster, so that the virtual machine cluster executes the target offline task through the slave cluster.
In a fifth aspect, an embodiment of the present invention further provides an electronic device, including a processor and a memory, where the memory stores computer executable instructions executable by the processor, and the processor executes the computer executable instructions to implement the steps of the resource management method described above.
In a sixth aspect, embodiments of the present invention further provide a computer-readable storage medium storing computer-executable instructions that, when invoked and executed by a processor, cause the processor to implement the steps of the resource management method described above.
The embodiment of the invention has the following beneficial effects:
the embodiment of the invention provides a resource management method, a resource management device and electronic equipment, wherein a slave cluster of a container technology is established in a virtual machine cluster, and configuration information of the slave cluster is sent to a master cluster of the container technology cluster; and responding to the target offline task sent by the slave cluster of the container technology and calling the slave cluster to execute the target offline task. The method can realize the target offline task of the virtual machine cluster execution container technology, which is sent from the cluster, can improve the resource utilization rate of the virtual machine, does not need to expand K8s Node and adapting work, has low cost and is beneficial to rapid expansion and contraction of resources.
Additional features and advantages of the disclosure will be set forth in the description which follows, or in part will be obvious from the description, or may be learned by practice of the techniques of the disclosure.
The foregoing objects, features and advantages of the disclosure will be more readily apparent from the following detailed description of the preferred embodiments taken in conjunction with the accompanying drawings.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a resource management method according to an embodiment of the present invention;
FIG. 2 is a flowchart of another resource management method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a process of reorganizing idle resources according to an embodiment of the present invention;
FIG. 4 is a flowchart of another resource management method according to an embodiment of the present invention;
FIG. 5 is a flowchart of another resource management method according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating multi-cluster management of a container K8s cluster according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a K8s scheduling process and a dan-to-stove cloud platform overall scheduling process according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a resource management device according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of another resource management device according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of another resource management device according to an embodiment of the present invention;
FIG. 11 is a schematic diagram of another resource management device according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
At present, the problem of low resource utilization rate exists in the scene using the virtual machine technology, and although K8s Node can be expanded in the scene using the container technology, higher cost and a large amount of adaptation work are required, which is not beneficial to the rapid expansion and contraction of resources. Based on the above, the resource management method, the device and the electronic equipment provided by the embodiment of the invention can be applied to equipment such as computers, mobile phones, tablet computers and servers, and particularly can be applied to equipment with virtual machine functions.
For the sake of understanding the present embodiment, a detailed description will be given of a resource management method disclosed in the embodiment of the present invention.
The present embodiment provides a resource management method applied to a virtual machine cluster, referring to a flowchart of a resource management method shown in fig. 1, where the resource management method includes the following steps:
step S102, determining idle resources of the virtual machine cluster.
The virtual machine in this embodiment may be a virtual machine managed and maintained in Openstack (a cloud computing management platform), which is also referred to as a virtual machine instance. The virtual machine may provide services out through Openstack-bound floating IP (Internet Protocol, internetworking protocol). The Openstack is used as a cloud computing management platform and is an open-source application for managing the virtual machine.
The resource virtualization technology provided by Openstack enables the resource virtualization technology to allocate resources exceeding the actual resource amount for users. When the user quantity is large, the technology can ensure the utilization rate of cluster resources and ensure the statistical balance between available resources of users. Overstock of CPU (Central Processing Unit ) resources is common, and overstock of Memory resources also exists.
The idle resources may be resources in an idle state in the virtual machine cluster, i.e. resources that are not working.
Step S104, determining tide nodes of the virtual machine cluster according to the idle resources.
The tidal node may be understood as a free resource in the virtual machine cluster, and in this embodiment, the free resource in the virtual machine cluster may be used as a tidal node in a slave cluster of the container technology cluster, and the task of the container technology cluster may be performed through the tidal node.
Step S106, constructing a slave cluster of the container technology cluster based on the tidal node.
The container technique may partition the resources of a single operating system into isolated groups to better balance conflicting resource usage requirements among the isolated groups. In this embodiment, the container technology cluster may be managed through K8S, where K8S (Kubernetes) is an open source application for managing containerization on multiple hosts in the cloud platform.
Where Pod is the smallest deployable unit that can create and manage kubernetes computation, consists of one or more containers, and is also the basic granularity of kubernetes scheduling. Deployment is a resource controller of K8s that can ensure that the policed Pod is in a desired state (running) and number (copy number). Job is a resource controller of K8s, and can guarantee the state of the Pod under jurisdiction before the successful end of operation according to the setting. Node (working Node) is a server Node in the K8s cluster. Cluster is a K8s Cluster and consists of Master Node (Master Node) and a plurality of Node nodes (working nodes). Pod Life cycle is the life cycle of Pod and comprises Running, pending, terminating and other foundations.
And if the remaining resources of all nodes of the current cluster resource cannot meet the resource requirement of the Pod to be scheduled, the Pod can enter a Pending state. And the K8s scheduling can detect the Pod of the Pending and all the current clusters satisfying the Pod scheduling in the subsequent flow at regular time, if the Pod scheduling is satisfied, the scheduling is performed, and if the Pod scheduling is not satisfied, the Pending is continued.
Step S108, sending configuration information of the slave clusters to the master cluster of the container technology cluster; the configuration information is used to characterize a first execution environment from tidal nodes in the cluster.
After the step of building the slave cluster in the virtual machine cluster is completed, the virtual machine cluster may send configuration information of the slave cluster to the master cluster of the container technology cluster. Wherein, the main cluster of the container technology cluster can perform task scheduling. In performing the task scheduling, the master cluster of the container technology cluster may determine, through the first execution environment of the tidal nodes in the slave cluster as characterized by the configuration information, whether the tidal nodes of the slave cluster may perform the task.
The first execution environment may include an operating system and hardware configuration of the tidal node, and may also include Cluster version information for the tidal node. Here, it should be noted that, in this embodiment, the master cluster or the slave cluster refers to the master cluster or the slave cluster of the container technology cluster, and will not be described herein.
Step S110, a target offline task sent by a main cluster of the container technology cluster is responded, and a slave cluster is called to execute the target offline task; the target offline task is an offline task in the main cluster, wherein the execution environment of the offline task is the same as the first execution environment.
Offline tasks may refer to tasks that a user deploys without having to be aware of the task execution state, tasks that end after a period of time have been run, such as AI (Artificial Intelligence ) training tasks, timed tasks, big data computing tasks, and so forth. Offline tasks typically have failure tolerance in cloud scenarios, allowing tasks to be interrupted and continued.
Each offline task has an execution environment suitable for its execution, which may be referred to as a second execution environment. After receiving the target offline task, the target offline task can be executed by selecting the slave cluster with the same first execution environment and the second execution environment.
The embodiment of the invention provides a resource management method, which is characterized in that a slave cluster of a container technology is established in a virtual machine cluster, and configuration information of the slave cluster is sent to a master cluster of the container technology cluster; and responding to the target offline task sent by the slave cluster of the container technology and calling the slave cluster to execute the target offline task. The method can realize the target offline task of the virtual machine cluster execution container technology, which is sent from the cluster, can improve the resource utilization rate of the virtual machine, does not need to expand K8s Node and adapting work, has low cost and is beneficial to rapid expansion and contraction of resources.
The present embodiment provides another resource management method, which is implemented on the basis of the above embodiment; this example focuses on a specific implementation of a slave cluster to build a container technology cluster. As shown in fig. 2, the resource management method in this embodiment includes the following steps:
Step S202, determining idle resources of the virtual machine cluster.
Step S204, determining tide nodes of the virtual machine cluster according to the idle resources.
Step S206, determining at least one mirror node from the tidal node.
In this embodiment, a new K8s cluster (i.e., a slave cluster) may be established in a target Openstack cluster (i.e., a virtual machine cluster), and first, at least one mirror Node may be established in the virtual machine cluster, so that the mirror Node mirrors the K8s Node (i.e., a working Node in the slave cluster).
The mirror node of this embodiment is used to launch a tidal node, e.g., to invoke steps to perform a target offline task from the cluster, including: the tidal node is caused to perform a target offline task by starting the tidal node from the mirror node of the cluster. If the tidal node A is required to perform a target offline task, the tidal node may be enabled by mirroring the node A from the cluster to cause the tidal node to perform the target offline task.
Step S208, a slave cluster of the container technology cluster is established in the virtual machine cluster according to the mirror node.
The necessary components (e.g., docker, kubectl, etc.) may be pre-installed as a base image of a public image (Yun Jingxiang) of a debrian (a general operating system) authority. The kubead tool is used to initialize the master node of the slave cluster and to implant the authentication file into the mirror node. The mirror image node uses Cloud-init (an initialization tool of a virtual machine in a Cloud environment) to initialize the mirror image environment, accesses the slave cluster master node Apiserver after the base component is started successfully by using an authentication file, registers node information synchronization state with etcd (a distributed system with distributed and reliable key-value storage), and completes integration of the cluster component and related resources.
Step S210, at least one free resource is selected from the virtual machine cluster, and the free resource is added into the slave cluster as a tide node.
After the secondary cluster is established, free resources in the virtual machine may be added to the secondary cluster as tidal nodes, and the master nodes in the secondary cluster are established. For example: determining idle resources in the virtual machine cluster, and determining tide nodes based on the idle resources; the working node and the master node of the slave cluster are established based on the tidal node.
The master node is used for communicating with the master cluster, and can distribute target offline tasks to corresponding working nodes, and the corresponding working nodes execute the target offline tasks. For example: transmitting configuration information of the slave cluster to a master cluster of the container technology cluster by a master node of the slave cluster; receiving a target offline task from a main node of the cluster, and distributing the target offline task to a corresponding working node; and executing the target offline task correspondingly.
The method comprises the steps that a master node sends configuration information of a slave cluster to the master cluster, receives a target offline task sent by the master cluster, and distributes the target offline task to a corresponding working node; the worker node performs the target offline task distributed by the master node.
Wherein tidal nodes may be determined according to the order of the size of the free resources, for example: determining idle resources according to the monitoring information time sequence of the virtual machine cluster; the tidal nodes are determined according to the order of the size of the free resources.
And determining the remaining available resource size of the Openstack in real time according to the Openstack cluster monitoring information time sequence. A K8s node virtual machine (namely a tide node) is created according to the size of the resource in a super-selling ratio, and a master node is added to expand the resource.
There are 1 core CPUs in the super-market actual bare metal, but 16 core CPUs in the machine can be assumed when creating resources for the user. Super selling is applicable to CPU and memory resources.
The machine where Openstack is located basically allocates CPUs for users in a dedicated manner, that is, the users create 1-core 1G virtual machines, and then 1-core CPU resources and 1G memory are actually available. Openstack also supports shared mode allocation CPUs implemented in qemu process preemption. Different overstock ratios can be set for different Openstack nodes: there are 1: 16. 1:2, etc.
Prometaus is an open source service monitoring system and time series database, and can be used for capturing and storing real-time data of resource usage of physical machines. When the triggering resource pre-selection action is to call data of the last week in Prometheus, analyzing and screening. The free resources may be screened by: determining the variance of the time sequence, the idle resource size of the virtual machine cluster and the resource proportion of the virtual machine cluster according to the monitoring information time sequence of the virtual machine cluster; the resource ratio represents the ratio of processor resources to memory resources of the virtual machine cluster; determining a resource reduction ratio of the virtual machine cluster based on the idle resource size of the virtual machine cluster; and selecting resources with the time sequence variance smaller than a preset variance threshold from the virtual machine cluster based on the resource reduction ratio, wherein the resource proportion of the virtual machine cluster meets the preset proportion condition as idle resources.
1. The variance of the time series data is analyzed to determine that there is no large fluctuation in the physical machine load.
2. The average value of time sequence data on each resource dimension is analyzed to determine the available idle resource size of the physical machine, and according to the service attribute associated with the virtual machine running on the node, a service importance ratio is selected to reduce the available idle resource size (for example, for the machine where important service is located, a resource reduction ratio of 0.5 is selected to ensure smaller risk of influencing the service of the virtual machine).
The average value of the resource dimension may include an average value of cpu usage within 7 days, an average value of memory usage within 7 days, and the like.
3. Checking that there is no resource short board in each resource dimension (typically, the allocation ratio of the CPU resource and the Memory resource needs to be kept 1:2 or 1:1), and the resource needs to meet the required basic resource for maintaining one K8s node.
Referring to a schematic diagram of a process of reorganizing free resources shown in fig. 3, after determining the sizes of the target machine and the available resources according to the above screening method, the scheduler starts the tidal node by using node mirroring, and combines the available free resources into a K8s slave cluster.
Step S212, sending configuration information of the slave clusters to the master cluster of the container technology cluster; the configuration information is used to characterize a first execution environment from tidal nodes in the cluster.
After the cluster is completed, a K8s cluster management service can be requested, and the cluster identification and cluster authentication information are registered. The cluster authentication information may include configuration information of the slave cluster, and may also include key information. Thus, the above method further comprises: the cluster identification information and key information of the slave cluster are transmitted to the master cluster of the container technology cluster, so that the master cluster transmits the target-off task to the slave cluster based on the cluster identification information and key information of the slave cluster.
The authentication information includes key information included in an admin.conf file generated by creating the K8s cluster, and the identification information refers to names and uis (User Identification ) used by the cluster in the platform, such as reinforcement learning clusters, online publishing clusters, and the like.
Step S214, in response to the target offline task sent by the main cluster of the container technology cluster, calling the slave cluster to execute the target offline task; the target offline task is an offline task in the main cluster, wherein the execution environment of the offline task is the same as the first execution environment.
When the virtual machine cluster executes the target offline task, the resource utilization rate of the virtual machine cluster can be monitored in real time, and if the resource utilization rate of the virtual machine cluster is too high, the execution of the target offline task can be stopped, for example: monitoring the resource utilization rate of the virtual machine cluster; and if the resource utilization exceeds a preset first threshold, performing target offline task deletion and/or tide node deletion from the cluster.
The present embodiment aims to improve the resource utilization rate without affecting the virtual machine service, and thus delete the tide node in the Openstack node if the monitoring data is fully analyzed and the Openstack node resource is determined to be tense (when the resource utilization rate exceeds a preset first threshold). And replace the Openstack node with the preselected resource phase candidate. In this scenario, the corresponding K8s slave cluster node resources will be released, the target offline task on the node will be evicted (task interrupt), and an attempt will be made to run on the other slave cluster nodes (task continue). If the target offline task has insufficient resources for running after the resources are released, the task will enter a Pending state (task waiting) to wait for the new tidal node resources to join the slave cluster.
Further, the task being performed from the cluster may be performed by: interrupting the task that the tidal node is in an executing state; the task of tidal nodes in wait state is deleted.
And according to the resource index grasped by the monitoring system, when the virtual machine service load is increased, the resource release of the tide node is triggered after the resource utilization rate is increased. The resource release will force the deletion of tidal node virtual machines and send tidal node drain operations and tidal node delete operations to the slave cluster Apiserver. Because of the use of Job resources as an encapsulation resource for the target offline task, the target offline task may be interrupted and the task container may attempt to be recreated on the other tidal nodes of the slave cluster after being drained. If the resources of the other tidal nodes meet the resource requirements of the mission configuration, the mission container may be re-created and the target offline mission may continue automatically. If all tidal clusters on the slave cluster do not meet the resource requirements of the task configuration, the task container enters a Pending state, or waits for new tidal node resources to join the slave cluster, or triggers a target offline task rescheduling behavior.
The present embodiment provides another resource management method, which is applied to a master cluster of a container technology cluster; the embodiment focuses on describing a specific implementation manner of offline task scheduling by the main cluster. As shown in fig. 4, the resource management method in this embodiment includes the following steps:
step S402, receiving configuration information of a slave cluster sent by a virtual machine cluster; wherein the configuration information is used to characterize a first execution environment from a tidal node in the cluster, the tidal node being a free resource in the virtual machine cluster.
The master cluster of the container technology cluster may create a cluster management service that performs task scheduling via cluster management tasks. The primary cluster may obtain a first execution environment for the tidal node in the secondary cluster from configuration information of the secondary cluster sent from the virtual machine cluster. The first execution environment may be understood as the environment in which the tidal node performs tasks.
Step S404, obtaining the target offline task and the second execution environment of the target offline task.
The main cluster can acquire the target offline task and a second execution environment of the target offline task, and the second execution environment can be the environment for executing the target offline task.
In step S406, it is determined whether the first execution environment is the same as the second execution environment. If yes, go to step S408; if not, step S402 is performed.
Determining whether the first execution environment is the same as the second execution environment may determine whether the tidal node from the cluster is suitable for performing offline tasks. If the same, it means that the tidal node from the cluster is adapted to perform the offline task, step S408 may be performed; if not, it means that the tidal node of the slave cluster is not suitable to perform the offline task, step S402 may be performed to retrieve the configuration information of the new slave cluster.
In step S408, the offline task is sent to the virtual machine cluster, so that the virtual machine cluster performs the offline task through the slave cluster.
If the first execution environment is the same as the second execution environment, the tidal node of the slave cluster is adapted to perform the offline task, and the master cluster may send the offline task into the virtual machine cluster to cause the virtual machine cluster to perform the offline task through the slave cluster.
According to the resource management method provided by the embodiment of the invention, the master cluster can acquire the configuration information of the slave cluster transmitted by the virtual machine cluster, and the off-line task is transmitted to the slave cluster with the first execution environment being the same as the second execution environment of the off-line task, so that the virtual machine cluster executes the off-line task through the slave cluster. The method can carry out the scheduling of the offline tasks by the main cluster in a straight line and execute the offline tasks in the virtual machine cluster, thereby improving the resource utilization rate of the virtual machine, avoiding the expansion of K8s Node and the adaptation work, having lower cost and being beneficial to the rapid expansion and contraction of resources.
The present embodiment provides another resource management method, and the present embodiment focuses on a specific implementation manner in which a master cluster receives configuration information of a slave cluster sent by a virtual machine cluster and sends a target offline task to the virtual machine cluster. As shown in fig. 5, the resource management method in this embodiment includes the following steps:
step S502, receiving cluster identification and cluster authentication information of a virtual machine cluster.
The cluster authentication information may contain configuration information of the slave cluster and may also contain key information. The cluster identity is the name and Uid that can be used by the cluster to manage within the platform.
Step S504, searching configuration information of the virtual machine cluster from cluster authentication information.
The K8s main cluster can perform resource sensing and scheduling, and re-use the recombined idle resources: and synchronizing the available state and idle resources of the slave cluster according to the authentication information registered by the K8s slave node, and screening the K8s slave cluster identification according to the property of the task to be scheduled (whether the task is a target offline task). And the task scheduler acquires the slave cluster authentication information from the cluster management service according to the cluster identification and requests the slave cluster ApiServer to complete task scheduling.
Referring to a schematic diagram of multi-cluster management of a container K8s cluster shown in fig. 6, the K8s multi-cluster management service in the present embodiment acts to discover and evaluate K8s slave clusters; the authentication information of the slave cluster and the scheduling part participating in cluster screening are stored. The adoption of multiple clusters is that an administrator can not consider the model and configuration differences of newly added computing resources, and does not need to pay attention to the media attribute (physical machine or third party public cloud resource or virtual machine) of the bottom layer of the container clusters. The newly added resources only need to select proper own K8s configuration and ensure network connectivity with the main cluster. The control service of the cloud platform is deployed in the master cluster, and the management of the multi-container cluster is achieved by requesting, recovering and integrating the information of each slave cluster by using the authentication information of different clusters.
Step S506, obtaining the target offline task and the second execution environment of the target offline task.
Referring to a schematic diagram of a K8s scheduling procedure and a dane cloud platform overall scheduling procedure shown in fig. 7, the K8s scheduler encapsulates the target offline task service through Job resources (sometimes using a depoyment resource in a scenario where the target offline task needs to reside). Job resources manage containers (embodied in Pod) of the target offline task, monitor the state of the managed containers during operation, and ensure that the operational state of the containers is guaranteed as much as possible before the target offline task containers return a signal that execution is successful (the deviyment resources always maintain the operational state of the containers and attempt to restart the task even if the task execution is completed). The scheduler packages the target offline task into Job resources, selects a target slave cluster through the multi-cluster management service, and acquires an authentication file to access the Apiserver to complete task scheduling.
Step S508, determining whether the first execution environment is the same as the second execution environment. If yes, go to step S510; if not, step S502 is performed.
And step S510, based on the key information in the cluster authentication information and the cluster identification of the virtual machine cluster, sending the target offline task to the virtual machine cluster.
The master cluster can also reschedule the target offline task, when the resources of the slave cluster are in a depleted state, the K8s centralized controller can find and reschedule the target offline task in a Pending state (task waiting) on the slave cluster, reschedule the target offline task on other slave clusters with idle resources or reschedule the target offline task on the K8s master cluster under the condition that the resources of the master cluster are sufficient. The rescheduling can be performed by: monitoring the resource utilization rate of a slave cluster of the container technology cluster; if the resource utilization rate of the slave cluster is higher than a preset second threshold value, scheduling the task in a waiting state in the slave cluster to other slave clusters or the master cluster; the other slave clusters are slave clusters whose resource utilization is below a second threshold.
When the resources of the slave clusters are in a depleted state, the K8s collection controller can find and reschedule the target offline tasks in the Pending state (task waiting) on the slave clusters, reschedule the target offline tasks on other slave clusters with idle resources or reschedule the target offline tasks on the K8s master clusters under the condition that the master cluster resources are sufficient.
The method provided by the embodiment of the invention is mainly used for extracting the idle resources based on the virtual machine technical business for the target offline task based on the container technology by using the Openstack super-selling technology, so that the utilization rate is improved. In the mode, the computing resource waste caused by low load in the recombined virtual machine service can be reused by the off-line service, so that the aim of improving the utilization rate of the whole resource is fulfilled, the cost is saved, and meanwhile, the computing capacity support of the service provider can be provided more flexibly.
Corresponding to the method embodiment, the embodiment of the invention provides a resource management device which is applied to a virtual machine cluster. A schematic structure of a resource management device as shown in fig. 8, the resource management device comprising:
a free resource determining module 81, configured to determine a free resource of the virtual machine cluster;
a tide node determining module 82 for determining a tide node of the virtual machine cluster from the free resources;
a slave cluster construction module 83 for constructing a slave cluster of the container technology clusters based on the tidal nodes;
a configuration information sending module 84, configured to send configuration information of the slave cluster to the master cluster of the container technology cluster; the configuration information is used to characterize a first execution environment from tidal nodes in the cluster;
An offline task execution module 85, configured to invoke a slave cluster to execute a target offline task in response to the target offline task sent by the master cluster of the container technology cluster; the target offline task is an offline task in the main cluster, wherein the execution environment of the offline task is the same as the first execution environment.
The embodiment of the invention provides a resource management device, which establishes a slave cluster of a container technology in a virtual machine cluster and sends configuration information of the slave cluster to a master cluster of the container technology cluster; and responding to the target offline task sent by the slave cluster of the container technology and calling the slave cluster to execute the target offline task. The method can realize the target offline task of the virtual machine cluster execution container technology, which is sent from the cluster, can improve the resource utilization rate of the virtual machine, does not need to expand K8s Node and adapting work, has low cost and is beneficial to rapid expansion and contraction of resources.
The slave cluster construction module is used for determining at least one mirror image node according to the tide node; establishing a slave cluster of the container technology cluster in the virtual machine cluster according to the mirror node; selecting at least one idle resource from the virtual machine cluster, and adding the idle resource into the slave cluster as a tide node; the offline task execution module is used for enabling the tide node to execute the target offline task by starting the tide node from the mirror image node of the cluster.
The slave cluster construction module is used for determining idle resources in the virtual machine cluster and determining tide nodes based on the idle resources; establishing a working node and a master node of the slave cluster based on the tidal node; the configuration information sending module is used for sending the configuration information of the slave cluster to the master cluster of the container technology cluster through the master node of the slave cluster; the offline task execution module is used for receiving the target offline task from the main node of the cluster and distributing the target offline task to the corresponding working node; and executing the target offline task through the corresponding working node. The idle resource determining module is used for determining idle resources according to the monitoring information time sequence of the virtual machine cluster; the tide node determining module is used for determining tide nodes according to the order of the sizes of the free resources
The idle resource determining module is used for determining idle resources according to the monitoring information time sequence of the virtual machine cluster; the tidal nodes are determined according to the order of the size of the free resources.
The slave cluster construction module is used for determining the variance of the time sequence, the idle resource size of the virtual machine cluster and the resource proportion of the virtual machine cluster according to the monitoring information time sequence of the virtual machine cluster; the resource ratio represents the ratio of processor resources to memory resources of the virtual machine cluster; determining a resource reduction ratio of the virtual machine cluster based on the idle resource size of the virtual machine cluster; and selecting resources with the time sequence variance smaller than a preset variance threshold from the virtual machine cluster based on the resource reduction ratio, wherein the resource proportion of the virtual machine cluster meets the preset proportion condition as idle resources.
Referring to a schematic structure of another resource management device shown in fig. 9, the resource management device further includes: the cluster identification information and key information sending module 86 is connected to the slave cluster constructing module 83, and the cluster identification information and key information sending module 86 is configured to send the cluster identification information and key information of the slave cluster to the master cluster of the container technology cluster, so that the master cluster sends the target offline task to the slave cluster based on the cluster identification information and key information of the slave cluster.
Referring to a schematic structure of another resource management device shown in fig. 9, the resource management device further includes: the offline task and tide node deleting module 87 is connected with the offline task executing module 85, and the offline task and tide node deleting module 87 is used for monitoring the resource utilization rate of the virtual machine cluster; and if the resource utilization exceeds a preset first threshold, performing target offline task deletion and/or tide node deletion from the cluster.
The offline task and tide node deleting module is used for interrupting the task of the tide node in an executing state; the task of tidal nodes in wait state is deleted.
Corresponding to the above method embodiment, another resource management device is provided in the embodiment of the present invention, which is applied to the master cluster of the container technology cluster. Another resource management device, as shown in fig. 10, is a schematic structural diagram, comprising:
A configuration information receiving module 1001, configured to receive configuration information of a slave cluster sent by a virtual machine cluster; wherein the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster, the tidal node being a free resource in the virtual machine cluster;
an offline task acquiring module 1002, configured to acquire a target offline task and a second execution environment of the target offline task;
an execution environment determining module 1003, configured to determine whether the first execution environment is the same as the second execution environment;
the offline task sending module 1004 is configured to send the offline task to the virtual machine cluster, so that the virtual machine cluster performs the offline task through the slave cluster.
According to the resource management device provided by the embodiment of the invention, the master cluster can acquire the configuration information of the slave cluster transmitted by the virtual machine cluster, and the target offline task is transmitted to the slave cluster with the first execution environment being the same as the second execution environment of the target offline task, so that the virtual machine cluster can execute the target offline task through the slave cluster. The method can carry out scheduling of the target offline tasks by the main cluster in a straight line and execute the target offline tasks in the virtual machine cluster, can improve the resource utilization rate of the virtual machine, does not need to expand K8s Node and adaptation work, has low cost and is beneficial to rapid expansion and contraction of resources.
The configuration information receiving module is used for receiving cluster identification and cluster authentication information of the virtual machine clusters; searching configuration information of a virtual machine cluster from cluster authentication information; the offline task sending module is configured to send the target offline task to the virtual machine cluster based on the key information in the cluster authentication information and the cluster identifier of the virtual machine cluster.
Referring to a schematic structure of another resource management device shown in fig. 11, the resource management device further includes: the offline task scheduling rescheduling module 1005 is connected with the offline task sending module 1004, and the offline task scheduling rescheduling module 1005 is used for monitoring the resource utilization rate of the slave cluster of the container technology cluster; if the resource utilization rate of the slave cluster is higher than a preset second threshold value, scheduling the task in a waiting state in the slave cluster to other slave clusters or the master cluster; the other slave clusters are slave clusters whose resource utilization is below a second threshold.
The resource management device provided by the embodiment of the invention has the same technical characteristics as the resource management method provided by the embodiment, so that the same technical problems can be solved, and the same technical effects can be achieved.
The embodiment of the invention also provides electronic equipment for running the resource management method; referring to a schematic structural diagram of an electronic device shown in fig. 12, the electronic device includes a memory 100 and a processor 101, where the memory 100 is configured to store one or more computer instructions, and the one or more computer instructions are executed by the processor 101 to implement the above-mentioned resource management method.
Further, the electronic device shown in fig. 12 further includes a bus 102 and a communication interface 103, and the processor 101, the communication interface 103, and the memory 100 are connected through the bus 102.
The memory 100 may include a high-speed random access memory (RAM, random Access Memory), and may further include a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory. The communication connection between the system network element and at least one other network element is implemented via at least one communication interface 103 (which may be wired or wireless), and may use the internet, a wide area network, a local network, a metropolitan area network, etc. Bus 102 may be an ISA bus, a PCI bus, an EISA bus, or the like. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in FIG. 12, but not only one bus or type of bus.
The processor 101 may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in the processor 101 or instructions in the form of software. The processor 101 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processor, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in the memory 100 and the processor 101 reads information in the memory 100 and in combination with its hardware performs the steps of the method of the previous embodiments.
The embodiment of the invention also provides a computer readable storage medium, which stores computer executable instructions that, when being called and executed by a processor, cause the processor to implement the above resource management method, and the specific implementation can be referred to the method embodiment and will not be described herein.
The resource management method, the device and the computer program product of the electronic equipment provided by the embodiment of the invention comprise a computer readable storage medium storing program codes, and the instructions included in the program codes can be used for executing the method in the previous method embodiment, and specific implementation can be referred to the method embodiment and will not be repeated here.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described system and/or apparatus may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
In addition, in the description of embodiments of the present invention, unless explicitly stated and limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. are based on the directions or positional relationships shown in the drawings, are merely for convenience of describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above examples are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention, but it should be understood by those skilled in the art that the present invention is not limited thereto, and that the present invention is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (15)

1. A method of resource management, applied to a cluster of virtual machines, the method comprising:
determining idle resources of the virtual machine cluster;
taking the idle resources of the virtual machine cluster as tide nodes of the virtual machine cluster;
constructing a slave cluster of a container technology cluster based on the tidal node;
Transmitting configuration information of the slave cluster to a master cluster of the container technology cluster; the configuration information is used to characterize a first execution environment of the tidal nodes in the slave cluster;
responding to a target offline task sent by a master cluster of the container technology cluster, and calling the slave cluster to execute the target offline task; the target offline task is an offline task of which the execution environment in the main cluster is the same as the first execution environment.
2. The method of claim 1, wherein the step of constructing a slave cluster of a container technology cluster based on the tidal node comprises:
determining at least one mirror node from the tidal node;
establishing a slave cluster of a container technology cluster in the virtual machine cluster according to the mirror node;
selecting at least one free resource from the virtual machine cluster, and adding the free resource into the slave cluster as a tide node;
the step of calling the slave cluster to execute the target offline task comprises the following steps:
the tidal node is started by the mirror node of the slave cluster to cause the tidal node to perform the target offline task.
3. The method of claim 2, wherein selecting at least one free resource from the cluster of virtual machines, adding the free resource as a tidal node to the slave cluster, comprises:
Determining an idle resource in the virtual machine cluster, and determining a tide node based on the idle resource;
establishing a working node and a master node of the slave cluster based on the tidal node;
the step of sending configuration information of the slave cluster to a master cluster of the container technology cluster includes:
transmitting configuration information of the slave cluster to a master cluster of the container technology cluster through a master node of the slave cluster;
the step of calling the slave cluster to execute the target offline task comprises the following steps:
receiving the target offline task through the master node of the slave cluster, and distributing the target offline task to the corresponding working node;
and executing the target offline task through the corresponding working node.
4. The method of claim 1, wherein determining the free resources of the virtual machine cluster determines the free resources according to a monitored information time sequence of the virtual machine cluster;
the step of using the free resources of the virtual machine cluster as tidal nodes of the virtual machine cluster comprises:
and determining tide nodes according to the sequence of the sizes of the free resources.
5. The method of claim 4, wherein the step of determining the free resources from the monitored information time series of the virtual machine cluster comprises:
Determining the variance of the time sequence, the idle resource size of the virtual machine cluster and the resource proportion of the virtual machine cluster according to the monitoring information time sequence of the virtual machine cluster; the resource ratio represents the ratio of processor resources to memory resources of the virtual machine cluster;
determining a resource reduction ratio of the virtual machine cluster based on a free resource size of the virtual machine cluster;
and selecting resources, of which the variance of the time sequence is smaller than a preset variance threshold, from the virtual machine cluster based on the resource reduction ratio, wherein the resource proportion of the virtual machine cluster meets a preset proportion condition as the idle resources.
6. The method of claim 1, wherein after the step of constructing a slave cluster of a container technology cluster based on the tidal node, the method further comprises:
and sending the cluster identification information and the key information of the slave cluster to a master cluster of the container technology cluster, so that the master cluster sends the target offline task to the slave cluster based on the cluster identification information and the key information of the slave cluster.
7. The method according to claim 1, wherein the method further comprises:
Monitoring the resource utilization rate of the virtual machine cluster;
and if the resource utilization rate exceeds a preset first threshold value, performing target offline task deletion and/or tide node deletion on the slave cluster.
8. The method of claim 7, wherein the step of performing target offline task deletion from the cluster comprises: interrupting the task of the tidal node in an executing state; and deleting the task that the tide node is in a waiting state.
9. A method of resource management, applied to a master cluster of a container technology cluster, the method comprising:
receiving configuration information of a slave cluster sent by a virtual machine cluster; wherein the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster, the tidal node being a free resource in the virtual machine cluster;
acquiring a target offline task and a second execution environment of the target offline task;
judging whether the first execution environment is the same as the second execution environment;
and if so, sending the target offline task to the virtual machine cluster, so that the virtual machine cluster executes the target offline task through the slave cluster.
10. The method of claim 9, wherein the step of receiving configuration information of the slave cluster transmitted by the virtual machine cluster comprises:
receiving cluster identification and cluster authentication information of the virtual machine cluster;
searching configuration information of the virtual machine cluster from the cluster authentication information;
the step of sending the target offline task to the virtual machine cluster comprises the following steps:
and sending the target offline task to the virtual machine cluster based on the key information in the cluster authentication information and the cluster identification of the virtual machine cluster.
11. The method according to claim 9, wherein the method further comprises:
monitoring the resource utilization rate of the slave clusters of the container technology cluster;
if the resource utilization rate of the slave cluster is higher than a preset second threshold value, scheduling the task in the waiting state in the slave cluster to other slave clusters or the master cluster; the other slave clusters are slave clusters with resource utilization below the second threshold.
12. A resource management apparatus for use with a virtual machine cluster, the apparatus comprising:
the idle resource determining module is used for determining idle resources of the virtual machine cluster;
The tide node determining module is used for taking the idle resources of the virtual machine cluster as tide nodes of the virtual machine cluster;
a slave cluster construction module for constructing a slave cluster of a container technology cluster based on the tidal node;
the configuration information sending module is used for sending the configuration information of the slave cluster to the master cluster of the container technology cluster; the configuration information is used to characterize a first execution environment of the tidal nodes in the slave cluster;
the offline task execution module is used for responding to a target offline task sent by a master cluster of the container technology cluster and calling the slave cluster to execute the target offline task; the target offline task is an offline task of which the execution environment in the main cluster is the same as the first execution environment.
13. A resource management apparatus for use with a master cluster of container technology clusters, the apparatus comprising:
the configuration information receiving module is used for receiving configuration information of the slave cluster sent by the virtual machine cluster; wherein the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster, the tidal node being a free resource in the virtual machine cluster;
The offline task acquisition module is used for acquiring a target offline task and a second execution environment of the target offline task;
the execution environment judging module is used for judging whether the first execution environment is the same as the second execution environment;
and the offline task sending module is used for sending the target offline task to the virtual machine cluster if the target offline task is sent to the virtual machine cluster, so that the virtual machine cluster executes the target offline task through the slave cluster.
14. An electronic device comprising a processor and a memory, the memory storing computer executable instructions executable by the processor, the processor executing the computer executable instructions to implement the steps of the resource management method of any of claims 1-11.
15. A computer readable storage medium storing computer executable instructions which, when invoked and executed by a processor, cause the processor to implement the steps of the resource management method of any one of claims 1 to 11.
CN202110200163.4A 2021-02-22 2021-02-22 Resource management method and device and electronic equipment Active CN112965817B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110200163.4A CN112965817B (en) 2021-02-22 2021-02-22 Resource management method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110200163.4A CN112965817B (en) 2021-02-22 2021-02-22 Resource management method and device and electronic equipment

Publications (2)

Publication Number Publication Date
CN112965817A CN112965817A (en) 2021-06-15
CN112965817B true CN112965817B (en) 2023-08-11

Family

ID=76285621

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110200163.4A Active CN112965817B (en) 2021-02-22 2021-02-22 Resource management method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN112965817B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113791885B (en) * 2021-09-18 2024-09-17 上海中通吉网络技术有限公司 Method for automatically scheduling offline application according to application type
CN114640677A (en) * 2022-05-17 2022-06-17 云宏信息科技股份有限公司 Method for adding working nodes to cluster working nodes and APIServer by client configuration
CN117648710B (en) * 2024-01-30 2024-04-05 北京点聚信息技术有限公司 OFD format document authority control method

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103713935A (en) * 2013-12-04 2014-04-09 中国科学院深圳先进技术研究院 Method and device for managing Hadoop cluster resources in online manner
CN104461740A (en) * 2014-12-12 2015-03-25 国家电网公司 Cross-domain colony computing resource gathering and distributing method
CN105516325A (en) * 2015-12-18 2016-04-20 内蒙古农业大学 Cloud load balancing method for carrying out elastic expansion and traffic distribution expansion according to application load
CN110399307A (en) * 2019-07-31 2019-11-01 网宿科技股份有限公司 A kind of test method, test platform and destination server
CN110597664A (en) * 2019-09-17 2019-12-20 深信服科技股份有限公司 High-availability cluster resource deployment method, device and related components
CN111343288A (en) * 2020-05-21 2020-06-26 北京并行科技股份有限公司 Job scheduling method and system and computing device
US10896060B1 (en) * 2020-01-14 2021-01-19 Capital One Services, Llc Resource monitor for monitoring long-standing computing resources

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016074211A1 (en) * 2014-11-14 2016-05-19 华为技术有限公司 Data forwarding method and controller
US10474488B2 (en) * 2016-10-06 2019-11-12 Vmware, Inc. Configuration of a cluster of hosts in virtualized computing environments

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103713935A (en) * 2013-12-04 2014-04-09 中国科学院深圳先进技术研究院 Method and device for managing Hadoop cluster resources in online manner
CN104461740A (en) * 2014-12-12 2015-03-25 国家电网公司 Cross-domain colony computing resource gathering and distributing method
CN105516325A (en) * 2015-12-18 2016-04-20 内蒙古农业大学 Cloud load balancing method for carrying out elastic expansion and traffic distribution expansion according to application load
CN110399307A (en) * 2019-07-31 2019-11-01 网宿科技股份有限公司 A kind of test method, test platform and destination server
CN110597664A (en) * 2019-09-17 2019-12-20 深信服科技股份有限公司 High-availability cluster resource deployment method, device and related components
US10896060B1 (en) * 2020-01-14 2021-01-19 Capital One Services, Llc Resource monitor for monitoring long-standing computing resources
CN111343288A (en) * 2020-05-21 2020-06-26 北京并行科技股份有限公司 Job scheduling method and system and computing device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于虚拟机迁移的虚拟机集群资源调度;周文煜;陈华平;杨寿保;方君;;华中科技大学学报(自然科学版)(第S1期);130-133 *

Also Published As

Publication number Publication date
CN112965817A (en) 2021-06-15

Similar Documents

Publication Publication Date Title
CN112965817B (en) Resource management method and device and electronic equipment
US11704144B2 (en) Creating virtual machine groups based on request
CN106708622B (en) Cluster resource processing method and system and resource processing cluster
CN108737468B (en) Cloud platform service cluster, construction method and device
CN110612705B (en) Method for service deployment under server-free architecture and function management platform
CN109150987B (en) Two-layer container cluster elastic expansion method based on host layer and container layer
CN107251486B (en) Method, device and system for expanding linkage
CN109964507B (en) Network function management method, management unit and system
CN112860282B (en) Cluster plug-in upgrading method, device and server
CN113382077B (en) Micro-service scheduling method, micro-service scheduling device, computer equipment and storage medium
CN113297031B (en) Container group protection method and device in container cluster
US20240118935A1 (en) Pod deployment method and apparatus
CN113886069A (en) Resource allocation method and device, electronic equipment and storage medium
CN108900435A (en) A kind of method, apparatus and computer storage medium of service deployment
CN115617497A (en) Thread processing method, scheduling component, monitoring component, server and storage medium
CN113886058A (en) Cross-cluster resource scheduling method and device
EP3661140B1 (en) Method for applying for common service resource and related device
CN109905258B (en) PaaS management method, device and storage medium
CN113608838A (en) Deployment method and device of application image file, computer equipment and storage medium
CN111143033B (en) Operation execution method and device based on scalable operation system
CN112306640A (en) Container dispensing method, apparatus, device and medium therefor
JP2024501005A (en) Management method and device for container clusters
CN106550002B (en) paas cloud hosting system and method
CN112015515A (en) Virtual network function instantiation method and device
CN108696550B (en) System and method for quickly building and replicating clouds

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant