CN112965817A - Resource management method and device and electronic equipment - Google Patents

Resource management method and device and electronic equipment Download PDF

Info

Publication number
CN112965817A
CN112965817A CN202110200163.4A CN202110200163A CN112965817A CN 112965817 A CN112965817 A CN 112965817A CN 202110200163 A CN202110200163 A CN 202110200163A CN 112965817 A CN112965817 A CN 112965817A
Authority
CN
China
Prior art keywords
cluster
virtual machine
slave
node
offline task
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110200163.4A
Other languages
Chinese (zh)
Other versions
CN112965817B (en
Inventor
王端
李想成
吴爽
史征宇
赵增
刘柏
李仁杰
范长杰
胡志鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Netease Hangzhou Network Co Ltd
Original Assignee
Netease Hangzhou Network Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Netease Hangzhou Network Co Ltd filed Critical Netease Hangzhou Network Co Ltd
Priority to CN202110200163.4A priority Critical patent/CN112965817B/en
Publication of CN112965817A publication Critical patent/CN112965817A/en
Application granted granted Critical
Publication of CN112965817B publication Critical patent/CN112965817B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061Partitioning or combining of resources
    • G06F9/5077Logical partitioning of resources; Management or configuration of virtualized resources
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)
  • Stored Programmes (AREA)

Abstract

The invention provides a resource management method, a resource management device and electronic equipment. The method is applied to the virtual machine cluster and comprises the following steps: determining idle resources of a virtual machine cluster; determining tide nodes of the virtual machine cluster according to the idle resources; constructing a slave cluster of the container technology cluster based on the tidal nodes; sending configuration information of a slave cluster to a master cluster of a container technology cluster; the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster; responding to a target offline task sent by a main cluster of the container technology cluster, and calling a slave cluster to execute the target offline task; the target offline task is the offline task with the execution environment in the main cluster being the same as the first execution environment. The method can realize the target off-line task sent from the cluster by the virtual machine cluster execution container technology, can improve the resource utilization rate of the virtual machine, does not need to expand K8s Node and adapt work, has low cost and is beneficial to the rapid expansion and contraction of resources.

Description

Resource management method and device and electronic equipment
Technical Field
The invention relates to the technical field of cloud services, in particular to a resource management method and device and electronic equipment.
Background
The container technology is the most popular technology at present, and after the container technology is introduced from docker (application container engine) and k8s (Kubernetes, container arrangement engine), a bottom layer framework designed based on a k8s cluster is more popular, the deployment of software is easier, and one-time deployment and all-around operation are really realized. Meanwhile, a large number of application scenarios also exist due to the fact that the virtual machine technology is similar to a physical machine in use mode, old service migration is convenient, and the like.
Under different business scenarios, container technology or virtual machine technology needs to be used. The reason why the user usually uses the virtual machine as a remote host is that the virtual machine is generally active in working hours in the daytime and a large amount of non-working hours are in an idle state.
In a scene using a container technology, the kubemdm of K8s and some third-party open source schemes (such as Cluster Autoscaler) can automatically expand the K8s Node (K8s working Node) when the K8s Cluster resources are insufficient, so as to adapt to the expansion of the computing power requirement of an offline task and achieve the purpose of elastic computing power.
However, the above method is supported by API (Application Programming Interface) of public cloud, such as AWS (Amazon Web Services), allicloud (ali cloud), and the like; or a K8s Node needs to be manually built on a bare computer and a Cluster (Cluster) is added to complete the operation, and a new machine or a new public cloud resource needs higher cost, and the flexibility is not high when different machines or public cloud resources are configured. For example: the characteristics of the new machine result in that different versions of K8s have to be used, or the resources provided by the third-party public cloud are inconsistent with the existing Cluster version, which results in a large amount of adaptation work being required, and is not beneficial to rapid expansion and contraction of resources.
In summary, the scenario using the virtual machine technique has a problem of low resource utilization rate, and although the scenario using the container technique can expand K8s Node, it needs higher cost and a lot of adaptation work, which is not beneficial to fast expansion and contraction of resources.
Disclosure of Invention
In view of the above, an object of the present invention is to provide a method, an apparatus, and an electronic device for resource management, so as to improve the resource utilization rate of a virtual machine, without expanding a K8s Node and performing adaptation work, with low cost, and facilitating rapid expansion and contraction of resources.
In a first aspect, an embodiment of the present invention provides a resource management method, which is applied to a virtual machine cluster, and the method includes: determining idle resources of a virtual machine cluster; determining tide nodes of the virtual machine cluster according to the idle resources; constructing a slave cluster of the container technology cluster based on the tidal nodes; sending configuration information of a slave cluster to a master cluster of a container technology cluster; the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster; responding to a target offline task sent by a main cluster of the container technology cluster, and calling a slave cluster to execute the target offline task; the target offline task is the offline task with the execution environment in the main cluster being the same as the first execution environment.
In a preferred embodiment of the present invention, the step of constructing a slave cluster of a container technology cluster based on tidal nodes includes: determining at least one mirror node from the tidal nodes; establishing a slave cluster of the container technology cluster in the virtual machine cluster according to the mirror image node; selecting at least one idle resource from the virtual machine cluster, and adding the idle resource into the slave cluster as a tide node; the step of invoking the slave cluster to execute the target offline task includes: the tidal node performs the target offline task by launching the tidal node from the mirror node of the cluster.
In a preferred embodiment of the present invention, the step of selecting at least one idle resource from the virtual machine cluster and adding the idle resource as a tide node into the slave cluster includes: determining idle resources in the virtual machine cluster, and determining tide nodes based on the idle resources; establishing working nodes and main nodes of the slave clusters based on the tide nodes; the step of sending the configuration information of the slave cluster to the master cluster of the container technology cluster includes: sending configuration information of the slave cluster to a master cluster of the container technology cluster through a master node of the slave cluster; the step of invoking the slave cluster to execute the target offline task includes: receiving a target offline task by a master node of a slave cluster, and distributing the target offline task to a corresponding working node; and executing the target offline task through the corresponding working node.
In a preferred embodiment of the present invention, the determining of the idle resource of the virtual machine cluster determines the idle resource according to the monitoring information time sequence of the virtual machine cluster; the step of determining a tidal node of a cluster of virtual machines from free resources, comprising: tidal nodes are determined according to the order of the size of the free resources.
In a preferred embodiment of the present invention, the step of determining idle resources according to the monitoring information time sequence of the virtual machine cluster includes: determining the variance of the time sequence, the size of idle resources of the virtual machine cluster and the resource ratio of the virtual machine cluster according to the monitoring information time sequence of the virtual machine cluster; the resource ratio represents the ratio of processor resources to memory resources of the virtual machine cluster; determining a resource reduction ratio of the virtual machine cluster based on the size of the free resources of the virtual machine cluster; and selecting the resources, of which the variance of the time sequence is smaller than a preset variance threshold value and the resource ratio of the virtual machine cluster meets a preset ratio condition, from the virtual machine cluster as idle resources based on the resource reduction ratio.
In a preferred embodiment of the present invention, after the step of building a slave cluster of a container technology cluster based on tidal nodes, the method further comprises: and sending the cluster identification information and the key information of the slave cluster to a master cluster of the container technology cluster, so that the master cluster sends the target offline task to the slave cluster based on the cluster identification information and the key information of the slave cluster.
In a preferred embodiment of the present invention, the method further includes: monitoring the resource utilization rate of the virtual machine cluster; and if the resource utilization rate exceeds a preset first threshold value, performing target offline task deletion and/or tide node deletion on the slave cluster.
In a preferred embodiment of the present invention, the step of deleting the target offline task from the cluster includes: interrupting the task in which the tidal node is in an execution state; and deleting the task that the tide node is in a waiting state.
In a second aspect, an embodiment of the present invention further provides a master cluster applied to a container technology cluster, where the method includes: receiving configuration information of a slave cluster sent by a virtual machine cluster; wherein the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster, the tidal node being an idle resource in the virtual machine cluster; acquiring a target offline task and a second execution environment of the target offline task; judging whether the first execution environment is the same as the second execution environment; if so, the target offline task is sent to the virtual machine cluster, so that the virtual machine cluster executes the target offline task from the cluster.
In a preferred embodiment of the present invention, the step of receiving configuration information of a slave cluster sent by a virtual machine cluster includes: receiving cluster identification and cluster authentication information of a virtual machine cluster; searching configuration information of the virtual machine cluster from the cluster authentication information; the step of sending the target offline task to the virtual machine cluster includes: and sending the target offline task to the virtual machine cluster based on the key information in the cluster authentication information and the cluster identification of the virtual machine cluster.
In a preferred embodiment of the present invention, the method further includes: monitoring resource utilization of a slave cluster of a container technology cluster; if the resource utilization rate of the slave cluster is higher than a preset second threshold value, scheduling the tasks in the waiting state in the slave cluster to other slave clusters or the master cluster; the other slave clusters are slave clusters with resource utilization rate lower than a second threshold value.
In a third aspect, an embodiment of the present invention further provides a resource management apparatus, which is applied to a virtual machine cluster, and the apparatus includes: an idle resource determining module, configured to determine idle resources of a virtual machine cluster; the tide node determining module is used for determining tide nodes of the virtual machine cluster according to the idle resources; a slave cluster construction module for constructing a slave cluster of the container technology cluster based on the tidal nodes; the configuration information sending module is used for sending the configuration information of the slave cluster to the master cluster of the container technology cluster; the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster; the offline task execution module is used for responding to a target offline task sent by a main cluster of the container technology cluster and calling a slave cluster to execute the target offline task; the target offline task is the offline task with the execution environment in the main cluster being the same as the first execution environment.
In a fourth aspect, an embodiment of the present invention further provides a resource management apparatus, which is applied to a master cluster of a container technology cluster, and the apparatus includes: the configuration information receiving module is used for receiving the configuration information of the slave cluster, which is sent by the virtual machine cluster; wherein the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster, the tidal node being an idle resource in the virtual machine cluster; the offline task obtaining module is used for obtaining a target offline task and a second execution environment of the target offline task; the execution environment judgment module is used for judging whether the first execution environment is the same as the second execution environment; and the offline task sending module is used for sending the target offline task to the virtual machine cluster if the target offline task is the target offline task, so that the virtual machine cluster executes the target offline task from the cluster.
In a fifth aspect, an embodiment of the present invention further provides an electronic device, which includes a processor and a memory, where the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the steps of the resource management method described above.
In a sixth aspect, the embodiments of the present invention further provide a computer-readable storage medium, which stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions cause the processor to implement the steps of the resource management method described above.
The embodiment of the invention has the following beneficial effects:
according to the resource management method, the resource management device and the electronic equipment provided by the embodiment of the invention, the slave cluster of the container technology is established in the virtual machine cluster, and the configuration information of the slave cluster is sent to the master cluster of the container technology cluster; responding to the target offline task sent by the container technology from the cluster and calling the slave cluster to execute the target offline task. The method can realize the target off-line task sent from the cluster by the virtual machine cluster execution container technology, can improve the resource utilization rate of the virtual machine, does not need to expand K8s Node and adapt work, has low cost and is beneficial to the rapid expansion and contraction of resources.
Additional features and advantages of the disclosure will be set forth in the description which follows, or in part may be learned by the practice of the above-described techniques of the disclosure, or may be learned by practice of the disclosure.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of a resource management method according to an embodiment of the present invention;
FIG. 2 is a flowchart of another resource management method according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an idle resource reconfiguration process according to an embodiment of the present invention;
FIG. 4 is a flowchart of another resource management method according to an embodiment of the present invention;
FIG. 5 is a flowchart of another resource management method according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of multi-cluster management of a container K8s cluster according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a K8s scheduling process and an overall scheduling process of a dan furnace cloud platform according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a resource management apparatus according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of another resource management apparatus according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of another resource management apparatus according to an embodiment of the present invention;
fig. 11 is a schematic structural diagram of another resource management apparatus according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
At present, the problem of low resource utilization rate exists in a scene using a virtual machine technology, and although a K8s Node can be expanded in a scene using a container technology, higher cost and a large amount of adaptation work are required, which are not beneficial to rapid expansion and contraction of resources. Based on this, the resource management method, the resource management device and the electronic device provided by the embodiment of the invention can be applied to devices such as computers, mobile phones, tablet computers and servers, and can be particularly applied to devices with a virtual machine function.
To facilitate understanding of the embodiment, a detailed description is first given of a resource management method disclosed in the embodiment of the present invention.
The present embodiment provides a resource management method, which is applied to a virtual machine cluster, and referring to a flowchart of the resource management method shown in fig. 1, the resource management method includes the following steps:
step S102, determining idle resources of the virtual machine cluster.
The virtual machine in this embodiment may be a virtual machine managed and maintained in an Openstack (a cloud computing management platform), which is also referred to as a virtual machine instance. The virtual machine may provide services outside through a floating IP (Internet Protocol) bound Openstack. The Openstack is used as a cloud computing management platform and is an open-source application for managing the virtual machine.
The resource virtualization technology provided by Openstack enables a user to be allocated with resources exceeding the actual resource amount. When the number of users is large, the technology can achieve statistical balance between the cluster resource utilization rate and the available resources of the users. The over-selling of CPU (Central Processing Unit) resources is common, and the over-selling of Memory resources also exists.
The idle resource may be a resource in an idle state in the virtual machine cluster, that is, a resource that does not work.
And step S104, determining tide nodes of the virtual machine cluster according to the idle resources.
The tidal node can be understood as an idle resource in the virtual machine cluster, and in the embodiment, the idle resource in the virtual machine cluster can be used as a tidal node in a slave cluster of the container technology cluster, and the tasks of the container technology cluster are performed through the tidal node.
And step S106, constructing a slave cluster of the container technology cluster based on the tide node.
The container technique can partition resources of a single operating system into isolated groups to better balance conflicting resource usage requirements among the isolated groups. In this embodiment, the container technology cluster may be managed by K8S, and K8S (kubernets) is an open source application for managing containerization on multiple hosts in a cloud platform.
Wherein Pod is the smallest deployable unit that can create and manage kubernets computation, and is composed of one or more containers, and is also the basic granularity of kubernets scheduling. The Deployment is a resource controller for K8s that can guarantee that the policed Pod is in a desired state (running) and number (number of copies). Job is a resource controller for K8s that can be set to ensure that the managed Pod is in a state before the end of its successful run. Node (worker Node) is a server Node in the K8s cluster. Cluster is a K8s Cluster and consists of a Master Node (Master Node) and a plurality of Node nodes (working nodes). The Pod lifecycles are the life cycle of the Pod and comprise the foundations of Running, Pending and Terminating, Terminating and the like.
And if the residual resources of all the nodes of the current cluster resource cannot meet the resource requirement of the Pod to be scheduled, the Pod enters a Pending state. And the K8s scheduling regularly detects the Pod of the Pending and all current clusters meeting the Pod scheduling in the subsequent flow, if the Pod scheduling is met, scheduling is carried out, and if the Pod scheduling is not met, the Pending continues.
Step S108, sending the configuration information of the slave cluster to the master cluster of the container technology cluster; the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster.
After the step of constructing the slave cluster in the virtual machine cluster is completed, the virtual machine cluster may send configuration information of the slave cluster to a master cluster of the container technology cluster. Wherein, the main cluster of the container technology cluster can carry out the scheduling of tasks. In scheduling a task, a master cluster of a container technology cluster may determine whether a tidal node of a slave cluster can perform the task by a first execution environment of the tidal node in the slave cluster characterized by configuration information.
The first execution environment may include an operating system and hardware configuration of the tidal node and may also include Cluster version information of the tidal node. It should be noted that, in this embodiment, the master cluster or the slave cluster refers to a master cluster or a slave cluster of a container technology cluster, and details thereof are not described herein again.
Step S110, responding to a target offline task sent by a main cluster of the container technology cluster, and calling a slave cluster to execute the target offline task; the target offline task is the offline task with the execution environment in the main cluster being the same as the first execution environment.
The offline task may refer to a task that is deployed by a user and ends after running for a period of time without being aware of a task execution state, such as an AI (Artificial Intelligence) training task, a timing task, a big data computing task, and the like. Offline tasks are typically failure tolerant in cloud scenarios, allowing tasks to be interrupted and continued.
Each offline task has an execution environment suitable for its execution, which may be referred to as a second execution environment. After receiving the target offline task, the slave cluster with the same first execution environment and the same second execution environment may be selected to execute the target offline task.
In the resource management method provided by the embodiment of the invention, a slave cluster of a container technology is established in a virtual machine cluster, and configuration information of the slave cluster is sent to a master cluster of the container technology cluster; responding to the target offline task sent by the container technology from the cluster and calling the slave cluster to execute the target offline task. The method can realize the target off-line task sent from the cluster by the virtual machine cluster execution container technology, can improve the resource utilization rate of the virtual machine, does not need to expand K8s Node and adapt work, has low cost and is beneficial to the rapid expansion and contraction of resources.
The present embodiment provides another resource management method, which is implemented on the basis of the foregoing embodiments; this embodiment focuses on a specific implementation of a slave cluster building a container technology cluster. As shown in fig. 2, another flow chart of a resource management method, the resource management method in this embodiment includes the following steps:
step S202, determining the idle resources of the virtual machine cluster.
And step S204, determining tide nodes of the virtual machine cluster according to the idle resources.
In step S206, at least one mirror node is determined from the tidal nodes.
In this embodiment, a new K8s cluster (i.e., a slave cluster) may be established in a target Openstack cluster (i.e., a virtual machine cluster), and first, at least one mirror Node may be established in the virtual machine cluster, so that the mirror Node mirrors a K8s Node (i.e., a working Node in the slave cluster).
The mirror node of this embodiment is used to start a tidal node, for example, to invoke the step of executing a target offline task from a cluster, and includes: the tidal node performs the target offline task by launching the tidal node from the mirror node of the cluster. If the tidal node A is required to perform the target offline task, the tidal node can be caused to perform the target offline task by starting the tidal node from the mirror node A of the cluster.
And step S208, establishing a slave cluster of the container technology cluster in the virtual machine cluster according to the mirror image node.
The necessary components (e.g., Docker, Kubectl, etc.) may be pre-installed based on the cloud image of the Debian (a common operating system) authority. And initializing the main node of the slave cluster by using a Kubeads tool, and implanting an authentication file into the mirror image node. The mirror image node initializes the mirror image environment by using Cloud-init (an initialization tool of a virtual machine in a Cloud environment), accesses the Apiserver of the slave cluster main node after the basic component is successfully started by using the authentication file, and registers the node information synchronization state to the etcd (a distributed system with distributed and reliable key-value storage), thereby completing the integration of the cluster component and related resources.
And step S210, selecting at least one idle resource from the virtual machine cluster, and adding the idle resource into the slave cluster as a tide node.
After the slave cluster is established, free resources in the virtual machine may be added to the slave cluster as tidal nodes and the master node in the slave cluster is established. For example: determining idle resources in the virtual machine cluster, and determining tide nodes based on the idle resources; the working nodes and master nodes of the slave clusters are established based on the tidal nodes.
The master node is used for communicating with the master cluster, the master node can distribute the target offline tasks, the target offline tasks are distributed to the corresponding working nodes, and the corresponding working nodes execute the target offline tasks. For example: sending configuration information of the slave cluster to a master cluster of the container technology cluster through a master node of the slave cluster; receiving a target offline task by a master node of a slave cluster, and distributing the target offline task to a corresponding working node; and executing the target offline task correspondingly.
The master node sends configuration information of the slave cluster to the master cluster, receives a target offline task sent by the master cluster, and distributes the target offline task to a corresponding working node; the worker node performs the target offline task distributed by the master node.
Wherein tidal nodes can be determined in order of the size of the free resources, such as: determining idle resources according to the monitoring information time sequence of the virtual machine cluster; tidal nodes are determined according to the order of the size of the free resources.
And determining the size of the Openstack available resource in real time according to the Openstack cluster monitoring information time sequence. A K8s node virtual machine (namely a tide node) is created according to the resource size in a super-selling ratio, and a master node is added to expand the resource.
There are 1-core CPUs in the supermale real bare machine, but it can be assumed that there are 16-core CPUs in the machine when creating resources for the user. The reselling applies to CPU and memory resources.
The machine where Openstack is located basically allocates CPUs to users in a dedicated manner, that is, a virtual machine with 1 core and 1G is created by a user, and 1 core CPU resources and 1G memory are actually available at the bottom. However, Openstack also supports shared mode allocation of a CPU, and is realized in a qemu process preemption mode. Different over-sell ratios can be set for different Openstack nodes: the common is 1: 16. 1: 2, etc.
Prometheus is an open source service monitoring system and a time sequence database, and can be used for capturing and storing real-time data of resource usage of a physical machine. When the trigger resource preselection behavior is yes, the data of the Prometheus for the last week is called and analyzed for screening. Free resources may be screened by: determining the variance of the time sequence, the size of idle resources of the virtual machine cluster and the resource ratio of the virtual machine cluster according to the monitoring information time sequence of the virtual machine cluster; the resource ratio represents the ratio of processor resources to memory resources of the virtual machine cluster; determining a resource reduction ratio of the virtual machine cluster based on the size of the free resources of the virtual machine cluster; and selecting the resources, of which the variance of the time sequence is smaller than a preset variance threshold value and the resource ratio of the virtual machine cluster meets a preset ratio condition, from the virtual machine cluster as idle resources based on the resource reduction ratio.
1. The variance of the time series data is analyzed to determine that there is no major fluctuation in the physical machine load.
2. Analyzing the mean value of the time sequence data to each resource dimension to determine the size of the available free resources of the physical machine, and selecting a service importance ratio to reduce the size of the available free resources according to the service attribute associated with the virtual machine running on the node (for example, for the machine where the important service is located, a resource reduction ratio of 0.5 is selected to ensure less risk of affecting the service of the virtual machine).
The average value of the resource dimension may include a cpu usage average value within 7 days, a memory usage average value within 7 days, and the like.
3. Checking that there is no resource stub in each resource dimension (usually the CPU resource and Memory resource need to maintain a 1: 2 or 1: 1 allocation ratio), the resource needs to meet the basic resource needed to maintain one K8s node.
Referring to fig. 3, a schematic diagram of a free resource reorganization process, after determining the sizes of the target machines and the available resources according to the screening method, the scheduler starts the tidal nodes by using the node images, and the available free resources are combined into the K8s slave cluster.
Step S212, sending the configuration information of the slave cluster to the master cluster of the container technology cluster; the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster.
After the cluster is completed, the K8s cluster management service may be requested to register the cluster identification and the cluster authentication information. The cluster authentication information may include configuration information of the slave cluster, and may also include key information. Accordingly, the above method further comprises: and sending the cluster identification information and the key information of the slave cluster to a master cluster of the container technology cluster, so that the master cluster sends the off-target-line task to the slave cluster based on the cluster identification information and the key information of the slave cluster.
The authentication information includes key information included in admin. conf files generated by the K8s cluster creation, and the Identification information refers to names and Uid (User Identification) used by the cluster in management in the platform, such as a reinforcement learning cluster, an online publishing cluster, and the like.
Step S214, responding to a target offline task sent by a main cluster of the container technology cluster, and calling a slave cluster to execute the target offline task; the target offline task is the offline task with the execution environment in the main cluster being the same as the first execution environment.
When the virtual machine cluster executes the target offline task, the resource utilization rate of the virtual machine cluster can be monitored in real time, and if the resource utilization rate of the virtual machine cluster is too high, the execution of the target offline task can be stopped, for example: monitoring the resource utilization rate of the virtual machine cluster; and if the resource utilization rate exceeds a preset first threshold value, performing target offline task deletion and/or tide node deletion on the slave cluster.
The embodiment aims to improve the resource utilization rate without affecting the virtual machine service, so that the tide node in the node is deleted when the monitoring data is sufficiently analyzed and the Openstack node is judged to be in resource shortage (when the resource utilization rate exceeds a preset first threshold). And replace the Openstack node back to the pre-selected resource phase candidate. In this scenario, the corresponding K8s slave cluster node resource will be released, the target offline task on the node will be evicted (task interrupted), and an attempt will be made to run on the other slave cluster node (task continued). If the target offline task has insufficient resources available for it to run after the resources are released, the task will enter a Pending state (task waiting) to wait for new tidal node resources to join the slave cluster.
Further, a task being executed from a cluster may be performed by: interrupting the task in which the tidal node is in an execution state; and deleting the task that the tide node is in a waiting state.
According to the resource indexes captured by the monitoring system, when the virtual machine service load is increased, the resource utilization rate is increased, and then the resource release of the tide node is triggered. The resource release will force the removal of the tide node virtual machine and send a tide node drain operation and a tide node delete operation to the slave cluster Apiserver. Since the Job resource is used as an encapsulation resource for the target offline task, the target offline task may be interrupted and the task container may attempt to be recreated on the other tidal nodes of the slave cluster after being drained for eviction. If the resources of other tidal nodes meet the resource requirements of the task configuration, the task container will be recreated and the target offline task may continue automatically. If all tide clusters on the slave cluster do not meet the resource requirement of task configuration, the task container enters a Pending state, or waits for new tide node resources to join the slave cluster, or triggers a target offline task rescheduling behavior.
The embodiment provides another resource management method, which is applied to a main cluster of a container technology cluster; this embodiment focuses on a specific implementation of offline task scheduling performed by the master cluster. As shown in fig. 4, another flow chart of a resource management method, the resource management method in this embodiment includes the following steps:
step S402, receiving configuration information of a slave cluster sent by a virtual machine cluster; wherein the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster, the tidal node being a free resource in the virtual machine cluster.
The main cluster of the container technology cluster can create a cluster management service, and task scheduling is carried out through a cluster management task. In configuration information of the slave cluster sent from the virtual machine cluster, the master cluster may obtain a first execution environment of the tidal node in the slave cluster. The first execution environment may be understood as the environment in which the tidal node performs tasks.
Step S404, a target offline task and a second execution environment of the target offline task are obtained.
The main cluster can acquire the target offline task and a second execution environment of the target offline task, and the second execution environment can be an environment for executing the target offline task.
Step S406, determining whether the first execution environment is the same as the second execution environment. If yes, go to step S408; if not, step S402 is performed.
Determining whether the first execution environment is the same as the second execution environment may determine whether the slave cluster's tidal node is suitable for performing offline tasks. If the same, meaning that the tidal node from the cluster is eligible to perform the offline task, step S408 may be performed; if not, meaning that the tidal node of the slave cluster is not suitable for performing the offline task, step S402 may be performed to retrieve new slave cluster configuration information.
Step S408 is to send the offline task to the virtual machine cluster, so that the virtual machine cluster executes the offline task through the slave cluster.
If the first execution environment is the same as the second execution environment, the tidal node of the slave cluster is adapted to perform the offline task, and the master cluster may send the offline task into the virtual machine cluster to cause the virtual machine cluster to perform the offline task through the slave cluster.
In the resource management method provided by the embodiment of the present invention, the master cluster may obtain the configuration information of the slave cluster sent by the virtual machine cluster, and send the offline task to the slave cluster in which the first execution environment is the same as the second execution environment of the offline task, so that the virtual machine cluster executes the offline task through the slave cluster. By the method, the offline task can be linearly scheduled by the main cluster, and the offline task is executed in the virtual machine cluster, so that the resource utilization rate of the virtual machine can be improved, the expansion of the K8s Node and the adaptation work are not needed, the cost is low, and the rapid expansion and contraction of resources are facilitated.
This embodiment mainly describes a specific implementation manner in which a master cluster receives configuration information of a slave cluster sent by a virtual machine cluster and sends a target offline task to the virtual machine cluster. As shown in fig. 5, another flow chart of a resource management method, the resource management method in this embodiment includes the following steps:
step S502, receiving the cluster identification and the cluster authentication information of the virtual machine cluster.
The cluster authentication information may contain configuration information of the slave cluster and may also contain key information. The cluster identifier is a name and a Uid that can be used by the cluster for management within the platform.
Step S504, the configuration information of the virtual machine cluster is searched from the cluster authentication information.
The K8s master cluster can perform resource sensing and scheduling, and reorganize the reuse of idle resources: and synchronizing the available state and the idle resources of the cluster according to the authentication information registered by the K8s slave nodes, and screening the K8s slave cluster identifier according to the property of the task needing to be scheduled (whether the task is a target offline task). And the task scheduler acquires the slave cluster authentication information from the cluster management service according to the cluster identifier and requests the slave cluster ApiServer to complete task scheduling.
Referring to fig. 6, a schematic diagram of multi-cluster management of a container K8s cluster, the K8s multi-cluster management service in this embodiment is used for discovering and evaluating K8s slave clusters; the slave cluster authentication information and the scheduling portion participating in the cluster screening are stored. The multiple clusters are adopted in consideration that an administrator can not consider the model and configuration difference of the newly added computing resources, and does not need to be aware of the media attributes (physical machines or third-party public cloud resources or virtual machines) at the bottom layer of the container cluster. The newly added resource only needs to select the proper own K8s configuration and ensure the network connectivity with the master cluster. The control service of the cloud platform is deployed in the master cluster, and management of the multi-container cluster is achieved by requesting, recovering and integrating the authentication information of different clusters to each slave cluster information.
Step S506, a target offline task and a second execution environment of the target offline task are obtained.
Referring to a schematic diagram of a K8s scheduling process and a dan furnace cloud platform overall scheduling process shown in fig. 7, a K8s scheduler encapsulates a target offline task service through a Job resource (sometimes, a Deployment resource is used in a scenario where a target offline task needs to reside). The Job resource manages a container (embodied in Pod) of the target offline task, monitors the running state of the managed container, and ensures that the running state of the container is guaranteed as much as possible before the target offline task container returns a signal of successful execution (the deplayment resource always keeps the running state of the container, and even if the task is executed completely, the task is attempted to be restarted). The dispatcher encapsulates the target offline task into Job resources, selects a target slave cluster through the multi-cluster management service, and acquires an authentication file to access Apiserver to complete task dispatching.
In step S508, it is determined whether the first execution environment is the same as the second execution environment. If yes, go to step S510; if not, step S502 is performed.
Step S510, sending the target offline task to the virtual machine cluster based on the key information in the cluster authentication information and the cluster identifier of the virtual machine cluster.
The master cluster can also reschedule the target offline task, and when the resources of the slave cluster are in a dry state, the K8s cluster controller finds and reschedules the target offline task in a Pending state (task waiting) on the slave cluster, reschedules the target offline task on another slave cluster having idle resources, or reschedules the target offline task on the K8s master cluster in a scenario where the resources of the master cluster are sufficient. Rescheduling may be performed by: monitoring resource utilization of a slave cluster of a container technology cluster; if the resource utilization rate of the slave cluster is higher than a preset second threshold value, scheduling the tasks in the waiting state in the slave cluster to other slave clusters or the master cluster; the other slave clusters are slave clusters with resource utilization rate lower than a second threshold value.
When the resources of the slave cluster are in a depletion state, the K8s cluster controller finds and reschedules the target offline task in a Pending state (task waiting) on the slave cluster, reschedules it on other slave clusters with free resources, or reschedules it on the K8s master cluster in a scenario where the master cluster resources are sufficient.
The method provided by the embodiment of the invention mainly aims to extract the idle resources based on the virtual machine technology service for the target offline task based on the container technology by utilizing the Openstack over-selling technology, so that the utilization rate is improved. In the method, the computing resources are wasted due to low load in the recombined virtual machine service, so that the recombined virtual machine service can be reused by the offline service, the aim of improving the overall resource utilization rate is fulfilled, the cost is saved, and meanwhile, the computing power support of a service party can be provided more flexibly.
Corresponding to the above method embodiment, an embodiment of the present invention provides a resource management apparatus, which is applied to a virtual machine cluster. Fig. 8 is a schematic structural diagram of a resource management device, where the resource management device includes:
an idle resource determining module 81, configured to determine idle resources of the virtual machine cluster;
a tide node determining module 82, configured to determine a tide node of the virtual machine cluster according to the idle resource;
a slave cluster construction module 83 for constructing slave clusters of the container technology cluster based on the tidal nodes;
a configuration information sending module 84, configured to send configuration information of the slave cluster to a master cluster of the container technology cluster; the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster;
the offline task execution module 85 is configured to invoke the slave cluster to execute a target offline task in response to the target offline task sent by the master cluster of the container technology cluster; the target offline task is the offline task with the execution environment in the main cluster being the same as the first execution environment.
In the resource management device provided in the embodiment of the present invention, a slave cluster of a container technology is established in a virtual machine cluster, and configuration information of the slave cluster is sent to a master cluster of the container technology cluster; responding to the target offline task sent by the container technology from the cluster and calling the slave cluster to execute the target offline task. The method can realize the target off-line task sent from the cluster by the virtual machine cluster execution container technology, can improve the resource utilization rate of the virtual machine, does not need to expand K8s Node and adapt work, has low cost and is beneficial to the rapid expansion and contraction of resources.
The slave cluster building module is configured to determine at least one mirror node from the tidal nodes; establishing a slave cluster of the container technology cluster in the virtual machine cluster according to the mirror image node; selecting at least one idle resource from the virtual machine cluster, and adding the idle resource into the slave cluster as a tide node; the offline task execution module is used for starting the tide node from the mirror image node of the cluster so as to enable the tide node to execute the target offline task.
The slave cluster building module is used for determining idle resources in the virtual machine cluster and determining tide nodes based on the idle resources; establishing working nodes and main nodes of the slave clusters based on the tide nodes; the configuration information sending module is configured to send the configuration information of the slave cluster to the master cluster of the container technology cluster through the master node of the slave cluster; the offline task execution module is used for receiving the target offline task from the main node of the cluster and distributing the target offline task to the corresponding working node; and executing the target offline task through the corresponding working node. The idle resource determining module is configured to determine an idle resource according to the monitoring information time sequence of the virtual machine cluster; the tide node determining module is used for determining tide nodes according to the sequence of the size of the idle resources
The idle resource determining module is configured to determine an idle resource according to the monitoring information time sequence of the virtual machine cluster; tidal nodes are determined according to the order of the size of the free resources.
The slave cluster building module is used for determining the variance of the time sequence, the size of the idle resources of the virtual machine cluster and the resource ratio of the virtual machine cluster according to the monitoring information time sequence of the virtual machine cluster; the resource ratio represents the ratio of processor resources to memory resources of the virtual machine cluster; determining a resource reduction ratio of the virtual machine cluster based on the size of the free resources of the virtual machine cluster; and selecting the resources, of which the variance of the time sequence is smaller than a preset variance threshold value and the resource ratio of the virtual machine cluster meets a preset ratio condition, from the virtual machine cluster as idle resources based on the resource reduction ratio.
Referring to fig. 9, a schematic structural diagram of another resource management apparatus is shown, where the resource management apparatus further includes: the cluster identification information and key information sending module 86 is connected to the slave cluster constructing module 83, and the cluster identification information and key information sending module 86 is configured to send the cluster identification information and key information of the slave cluster to the master cluster of the container technology cluster, so that the master cluster sends the target offline task to the slave cluster based on the cluster identification information and key information of the slave cluster.
Referring to fig. 9, a schematic structural diagram of another resource management apparatus is shown, where the resource management apparatus further includes: the offline task and tide node deleting module 87 is connected with the offline task executing module 85, and the offline task and tide node deleting module 87 is used for monitoring the resource utilization rate of the virtual machine cluster; and if the resource utilization rate exceeds a preset first threshold value, performing target offline task deletion and/or tide node deletion on the slave cluster.
The off-line task and tide node deleting module is used for interrupting the task of the tide node in an execution state; and deleting the task that the tide node is in a waiting state.
Corresponding to the foregoing method embodiment, an embodiment of the present invention provides another resource management apparatus, which is applied to a master cluster of a container technology cluster. As shown in fig. 10, another schematic structural diagram of a resource management device includes:
a configuration information receiving module 1001, configured to receive configuration information of a slave cluster sent by a virtual machine cluster; wherein the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster, the tidal node being an idle resource in the virtual machine cluster;
an offline task obtaining module 1002, configured to obtain a target offline task and a second execution environment of the target offline task;
an execution environment determining module 1003, configured to determine whether the first execution environment is the same as the second execution environment;
an offline task sending module 1004, configured to send the offline task to the virtual machine cluster if yes, so that the virtual machine cluster executes the offline task through the slave cluster.
In the resource management device provided in the embodiment of the present invention, the master cluster may obtain the configuration information of the slave cluster sent by the virtual machine cluster, and send the target offline task to the slave cluster in which the first execution environment is the same as the second execution environment of the target offline task, so that the virtual machine cluster executes the target offline task through the slave cluster. By the method, the target offline task can be linearly scheduled by the main cluster, and the target offline task is executed in the virtual machine cluster, so that the resource utilization rate of the virtual machine can be improved, expansion of a K8s Node and adaptation work are not needed, the cost is low, and rapid expansion and contraction of resources are facilitated.
The configuration information receiving module is configured to receive a cluster identifier and cluster authentication information of a virtual machine cluster; searching configuration information of the virtual machine cluster from the cluster authentication information; the offline task sending module is configured to send the target offline task to the virtual machine cluster based on the key information in the cluster authentication information and the cluster identifier of the virtual machine cluster.
Referring to fig. 11, a schematic structural diagram of another resource management apparatus is shown, where the resource management apparatus further includes: an offline task scheduling rescheduling module 1005 connected to the offline task sending module 1004, where the offline task scheduling rescheduling module 1005 is configured to monitor resource utilization of a slave cluster of the container technology cluster; if the resource utilization rate of the slave cluster is higher than a preset second threshold value, scheduling the tasks in the waiting state in the slave cluster to other slave clusters or the master cluster; the other slave clusters are slave clusters with resource utilization rate lower than a second threshold value.
The resource management device provided by the embodiment of the invention has the same technical characteristics as the resource management method provided by the embodiment, so that the same technical problems can be solved, and the same technical effects can be achieved.
The embodiment of the invention also provides the electronic equipment, which is used for operating the resource management method; referring to fig. 12, a schematic structural diagram of an electronic device includes a memory 100 and a processor 101, where the memory 100 is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor 101 to implement the resource management method.
Further, the electronic device shown in fig. 12 further includes a bus 102 and a communication interface 103, and the processor 101, the communication interface 103, and the memory 100 are connected by the bus 102.
The Memory 100 may include a high-speed Random Access Memory (RAM) and may further include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 103 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, and the like can be used. The bus 102 may be an ISA bus, PCI bus, EISA bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 12, but that does not indicate only one bus or one type of bus.
The processor 101 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 101. The Processor 101 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 100, and the processor 101 reads the information in the memory 100, and completes the steps of the method of the foregoing embodiment in combination with the hardware thereof.
An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions cause the processor to implement the resource management method.
The resource management method, the resource management apparatus, and the computer program product of the electronic device provided in the embodiments of the present invention include a computer-readable storage medium storing a program code, where instructions included in the program code may be used to execute the method in the foregoing method embodiments, and specific implementations may refer to the method embodiments and are not described herein again.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and/or the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In addition, in the description of the embodiments of the present invention, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, an electronic device, or a network device) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (15)

1. A resource management method is applied to a virtual machine cluster, and comprises the following steps:
determining idle resources of the virtual machine cluster;
determining tide nodes of the virtual machine cluster according to the idle resources;
building a slave cluster of a container technology cluster based on the tidal nodes;
sending configuration information of the slave cluster to a master cluster of the container technology cluster; the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster;
in response to a target offline task sent by a master cluster of the container technology cluster, calling the slave cluster to execute the target offline task; the target offline task is the offline task with the same execution environment as the first execution environment in the main cluster.
2. The method of claim 1, wherein the step of building a slave cluster of a container technology cluster based on the tidal nodes comprises:
determining at least one mirror node from the tidal node;
establishing a slave cluster of a container technology cluster in the virtual machine cluster according to the mirror image node;
selecting at least one free resource from the virtual machine cluster, and adding the free resource into the slave cluster as a tide node;
the step of calling the slave cluster to execute the target offline task comprises the following steps:
launching the tidal node through a mirror node of the slave cluster to cause the tidal node to perform the target offline task.
3. The method of claim 2, wherein the step of selecting at least one free resource from the cluster of virtual machines, joining the free resource as a tidal node in the slave cluster comprises:
determining idle resources in the virtual machine cluster, determining tidal nodes based on the idle resources;
establishing working nodes and master nodes of the slave cluster based on the tidal nodes;
the step of sending configuration information of the slave cluster to a master cluster of the container technology cluster comprises:
sending, by a master node of the slave cluster, configuration information for the slave cluster to a master cluster of the container technology cluster;
the step of calling the slave cluster to execute the target offline task comprises the following steps:
receiving the target offline task through the master node of the slave cluster, and distributing the target offline task to the corresponding working node;
and executing the target offline task through the corresponding working node.
4. The method of claim 1, wherein determining the free resources of the virtual machine cluster determines the free resources according to a monitoring information time sequence of the virtual machine cluster;
the step of determining a tidal node of the virtual machine cluster from the free resources comprises:
and determining tide nodes according to the sequence of the sizes of the free resources.
5. The method of claim 4, wherein the step of determining the free resources based on the monitoring information time series of the virtual machine cluster comprises:
determining the variance of the time sequence, the size of idle resources of the virtual machine cluster and the resource ratio of the virtual machine cluster according to the monitoring information time sequence of the virtual machine cluster; the resource ratio represents the ratio of processor resources to memory resources of the virtual machine cluster;
determining a resource reduction ratio for the virtual machine cluster based on an idle resource size of the virtual machine cluster;
selecting, from the virtual machine cluster, a resource, of which the variance of the time series is smaller than a preset variance threshold and the resource ratio of the virtual machine cluster meets a preset ratio condition, as the idle resource, based on the resource reduction ratio.
6. The method of claim 1, wherein after the step of building a slave cluster of a container technology cluster based on the tidal nodes, the method further comprises:
sending the cluster identification information and the key information of the slave cluster to a master cluster of the container technology cluster, so that the master cluster sends the target offline task to the slave cluster based on the cluster identification information and the key information of the slave cluster.
7. The method of claim 1, further comprising:
monitoring the resource utilization rate of the virtual machine cluster;
and if the resource utilization rate exceeds a preset first threshold value, performing target offline task deletion and/or tide node deletion on the slave cluster.
8. The method of claim 7, wherein the step of performing target offline task deletion on the slave cluster comprises: interrupting a task in which the tidal node is in an executing state; and deleting the task of which the tide node is in a waiting state.
9. A method of resource management applied to a master cluster of a container technology cluster, the method comprising:
receiving configuration information of a slave cluster sent by a virtual machine cluster; wherein the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster, the tidal node being a free resource in the virtual machine cluster;
acquiring a target offline task and a second execution environment of the target offline task;
judging whether the first execution environment is the same as the second execution environment;
if so, sending the target offline task to the virtual machine cluster so that the virtual machine cluster executes the target offline task through the slave cluster.
10. The method of claim 9, wherein the step of receiving configuration information sent by the virtual machine cluster from the cluster comprises:
receiving cluster identification and cluster authentication information of the virtual machine cluster;
searching the configuration information of the virtual machine cluster from the cluster authentication information;
the step of sending the target offline task to the virtual machine cluster includes:
and sending the target offline task to the virtual machine cluster based on the key information in the cluster authentication information and the cluster identification of the virtual machine cluster.
11. The method of claim 9, further comprising:
monitoring resource utilization of a slave cluster of the container technology cluster;
if the resource utilization rate of the slave cluster is higher than a preset second threshold value, scheduling the task in the waiting state in other slave clusters or the master cluster; the other slave clusters are slave clusters with resource utilization rate lower than the second threshold.
12. A resource management apparatus, applied to a virtual machine cluster, the apparatus comprising:
an idle resource determining module, configured to determine idle resources of the virtual machine cluster;
a tide node determining module for determining tide nodes of the virtual machine cluster according to the idle resources;
a slave cluster construction module for constructing a slave cluster of a container technology cluster based on the tidal nodes;
a configuration information sending module, configured to send configuration information of the slave cluster to a master cluster of the container technology cluster; the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster;
the offline task execution module is used for responding to a target offline task sent by a main cluster of the container technology cluster, and calling the slave cluster to execute the target offline task; the target offline task is the offline task with the same execution environment as the first execution environment in the main cluster.
13. An apparatus for resource management, applied to a master cluster of a container technology cluster, the apparatus comprising:
the configuration information receiving module is used for receiving the configuration information of the slave cluster, which is sent by the virtual machine cluster; wherein the configuration information is used to characterize a first execution environment of a tidal node in the slave cluster, the tidal node being a free resource in the virtual machine cluster;
the offline task obtaining module is used for obtaining a target offline task and a second execution environment of the target offline task;
the execution environment judging module is used for judging whether the first execution environment is the same as the second execution environment;
and if so, sending the target offline task to the virtual machine cluster so that the virtual machine cluster executes the target offline task through the slave cluster.
14. An electronic device comprising a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to perform the steps of the resource management method of any of claims 1-11.
15. A computer-readable storage medium having stored thereon computer-executable instructions that, when invoked and executed by a processor, cause the processor to perform the steps of the resource management method of any of claims 1-11.
CN202110200163.4A 2021-02-22 2021-02-22 Resource management method and device and electronic equipment Active CN112965817B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110200163.4A CN112965817B (en) 2021-02-22 2021-02-22 Resource management method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110200163.4A CN112965817B (en) 2021-02-22 2021-02-22 Resource management method and device and electronic equipment

Publications (2)

Publication Number Publication Date
CN112965817A true CN112965817A (en) 2021-06-15
CN112965817B CN112965817B (en) 2023-08-11

Family

ID=76285621

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110200163.4A Active CN112965817B (en) 2021-02-22 2021-02-22 Resource management method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN112965817B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113791885A (en) * 2021-09-18 2021-12-14 上海中通吉网络技术有限公司 Method for automatically scheduling off-line application according to application type
CN114640677A (en) * 2022-05-17 2022-06-17 云宏信息科技股份有限公司 Method for adding working nodes to cluster working nodes and APIServer by client configuration
CN117648710A (en) * 2024-01-30 2024-03-05 北京点聚信息技术有限公司 OFD format document authority control method

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103713935A (en) * 2013-12-04 2014-04-09 中国科学院深圳先进技术研究院 Method and device for managing Hadoop cluster resources in online manner
CN104461740A (en) * 2014-12-12 2015-03-25 国家电网公司 Cross-domain colony computing resource gathering and distributing method
CN105516325A (en) * 2015-12-18 2016-04-20 内蒙古农业大学 Cloud load balancing method for carrying out elastic expansion and traffic distribution expansion according to application load
US20170251384A1 (en) * 2014-11-14 2017-08-31 Huawei Technologies Co., Ltd. Data forwarding method and controller
US20180101395A1 (en) * 2016-10-06 2018-04-12 Vmware, Inc. Configuration of a cluster of hosts in virtualized computing environments
CN110399307A (en) * 2019-07-31 2019-11-01 网宿科技股份有限公司 A kind of test method, test platform and destination server
CN110597664A (en) * 2019-09-17 2019-12-20 深信服科技股份有限公司 High-availability cluster resource deployment method, device and related components
CN111343288A (en) * 2020-05-21 2020-06-26 北京并行科技股份有限公司 Job scheduling method and system and computing device
US10896060B1 (en) * 2020-01-14 2021-01-19 Capital One Services, Llc Resource monitor for monitoring long-standing computing resources

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103713935A (en) * 2013-12-04 2014-04-09 中国科学院深圳先进技术研究院 Method and device for managing Hadoop cluster resources in online manner
US20170251384A1 (en) * 2014-11-14 2017-08-31 Huawei Technologies Co., Ltd. Data forwarding method and controller
CN104461740A (en) * 2014-12-12 2015-03-25 国家电网公司 Cross-domain colony computing resource gathering and distributing method
CN105516325A (en) * 2015-12-18 2016-04-20 内蒙古农业大学 Cloud load balancing method for carrying out elastic expansion and traffic distribution expansion according to application load
US20180101395A1 (en) * 2016-10-06 2018-04-12 Vmware, Inc. Configuration of a cluster of hosts in virtualized computing environments
CN110399307A (en) * 2019-07-31 2019-11-01 网宿科技股份有限公司 A kind of test method, test platform and destination server
CN110597664A (en) * 2019-09-17 2019-12-20 深信服科技股份有限公司 High-availability cluster resource deployment method, device and related components
US10896060B1 (en) * 2020-01-14 2021-01-19 Capital One Services, Llc Resource monitor for monitoring long-standing computing resources
CN111343288A (en) * 2020-05-21 2020-06-26 北京并行科技股份有限公司 Job scheduling method and system and computing device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
周文煜;陈华平;杨寿保;方君;: "基于虚拟机迁移的虚拟机集群资源调度", 华中科技大学学报(自然科学版), no. 1, pages 130 - 133 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113791885A (en) * 2021-09-18 2021-12-14 上海中通吉网络技术有限公司 Method for automatically scheduling off-line application according to application type
CN114640677A (en) * 2022-05-17 2022-06-17 云宏信息科技股份有限公司 Method for adding working nodes to cluster working nodes and APIServer by client configuration
CN117648710A (en) * 2024-01-30 2024-03-05 北京点聚信息技术有限公司 OFD format document authority control method
CN117648710B (en) * 2024-01-30 2024-04-05 北京点聚信息技术有限公司 OFD format document authority control method

Also Published As

Publication number Publication date
CN112965817B (en) 2023-08-11

Similar Documents

Publication Publication Date Title
CN106708622B (en) Cluster resource processing method and system and resource processing cluster
CN112965817B (en) Resource management method and device and electronic equipment
CN110612705B (en) Method for service deployment under server-free architecture and function management platform
CN108737468B (en) Cloud platform service cluster, construction method and device
JP7192103B2 (en) DATA PROCESSING METHOD AND APPARATUS, AND COMPUTING NODE
CN115328663B (en) Method, device, equipment and storage medium for scheduling resources based on PaaS platform
WO2017170470A1 (en) Network function virtualization management orchestration device, method and program
CN109150987B (en) Two-layer container cluster elastic expansion method based on host layer and container layer
CN111641515B (en) VNF life cycle management method and device
JP6293306B2 (en) Cloud application processing method, cloud application deployment method, and related apparatus and system
CN108121511B (en) Data processing method, device and equipment in distributed edge storage system
CN106790092B (en) Remote procedure call server control system and method
CN112860282B (en) Cluster plug-in upgrading method, device and server
CN113382077B (en) Micro-service scheduling method, micro-service scheduling device, computer equipment and storage medium
CN110245020B (en) Mobile phone content forensics method and system based on multiple forensics devices
CN113297031B (en) Container group protection method and device in container cluster
CN113835865A (en) Task deployment method and device, electronic equipment and storage medium
CN108900435B (en) Service deployment method, device and computer storage medium
WO2016115896A1 (en) Method and device for management of virtual network function (vnf)
CN113886069A (en) Resource allocation method and device, electronic equipment and storage medium
CN115617497A (en) Thread processing method, scheduling component, monitoring component, server and storage medium
CN113965576B (en) Container-based big data acquisition method, device, storage medium and equipment
CN114565502A (en) GPU resource management method, scheduling method, device, electronic equipment and storage medium
CN109905258B (en) PaaS management method, device and storage medium
CN109413011B (en) Public service resource application method, related equipment and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant