CN112199194A

CN112199194A - Container cluster-based resource scheduling method, device, equipment and storage medium

Info

Publication number: CN112199194A
Application number: CN202011097911.2A
Authority: CN
Inventors: 林嘉; 田帅; 王兴刚; 王华夏
Original assignee: Guangzhou Huya Technology Co Ltd
Current assignee: Guangzhou Huya Technology Co Ltd
Priority date: 2020-10-14
Filing date: 2020-10-14
Publication date: 2021-01-08
Anticipated expiration: 2040-10-14

Abstract

The embodiment of the invention provides a resource scheduling method, a device, equipment and a storage medium based on a container cluster, wherein the method comprises the following steps: reading a currently effective scheduling strategy, counting resources occupied by the offline service in the container cluster according to the scheduling strategy, obtaining a resource occupation value, loading the offline service in a node instance, determining adjustment operation on the offline service according to the scheduling strategy so as to enable the resource occupation value to converge to a specified resource occupation target, adjusting the instance occupied by the offline service in the container cluster according to the adjustment operation, setting a corresponding scheduling strategy according to the characteristics of the online service, dynamically adjusting the resources occupied by the offline service based on real-time monitoring, realizing 24-hour offline service and online service mixed part landing, stabilizing the CPU utilization rate of the container cluster at more than 40% all day, greatly improving the resource utilization rate and greatly reducing the cost of the container cluster.

Description

Container cluster-based resource scheduling method, device, equipment and storage medium

Technical Field

The embodiment of the invention relates to the technical field of container clusters, in particular to a method, a device, equipment and a storage medium for resource scheduling based on a container cluster.

Background

As container clusters kubernets become de facto standard for container orchestration, more and more enterprises build container management platforms based on container clusters kubenetes.

In the container management platform, although the amount of resources is large, the daily average utilization rate of a CPU (Central Processing Unit) of a container cluster is about 10%, and the resource utilization rate is insufficient, and for this situation, the industry generally adopts a mixed manner of offline service and online service to increase the resource utilization rate.

The off-line mixing part of the mainstream in the industry utilizes the tide characteristic of the on-line service, and mixes the off-line service in the early morning period through the off-peak scheduling of the on-line service in the peak period and the valley period so as to improve the resource utilization rate.

However, the load of the online service changes with time, and the container cluster kubenetes fixes the resource specification of the offline service, so that the container cluster kubenetes has a lot of resource waste and high cost.

Disclosure of Invention

The embodiment of the invention provides a resource scheduling method, a device, equipment and a storage medium based on a container cluster, and aims to solve the problem that a large amount of resources are wasted in the container cluster kubenes.

In a first aspect, an embodiment of the present invention provides a method for resource scheduling based on a container cluster, where a controller is loaded on a node in the container cluster, and the method is applied to the controller, and includes:

reading a currently effective scheduling strategy;

counting resources occupied by the offline service in the container cluster according to the scheduling strategy to obtain a resource occupation value, wherein the offline service is loaded in an instance in the node;

determining adjustment operation for the offline service according to the scheduling policy so as to make the resource occupation value converge towards a specified resource occupation target;

and adjusting the instances occupied by the offline service in the container cluster according to the adjustment operation.

In a second aspect, an embodiment of the present invention further provides an apparatus for scheduling resources based on a container cluster, where a controller is loaded at a node in the container cluster, and the apparatus is located at the controller, and includes:

the scheduling strategy reading module is used for reading the currently effective scheduling strategy;

a resource occupation value counting module, configured to count resources occupied by the offline service in the container cluster according to the scheduling policy, to obtain a resource occupation value, where the offline service is loaded in an instance in the node;

an adjustment operation determining module, configured to determine an adjustment operation for the offline service according to the scheduling policy, so that the resource occupancy value converges to an assigned resource occupancy target;

and the adjusting operation executing module is used for adjusting the instances occupied by the offline service in the container cluster according to the adjusting operation.

In a third aspect, an embodiment of the present invention further provides a computer device, where the computer device includes:

one or more processors;

a memory for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the method for container cluster based resource scheduling according to the first aspect.

In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method for scheduling resources based on a container cluster according to the first aspect.

In the embodiment, a currently effective scheduling policy is read, resources occupied by offline services in a container cluster are counted according to the scheduling policy, a resource occupation value is obtained, an instance of the offline services loaded in a node is determined, an adjustment operation is determined for the offline services according to the scheduling policy, so that the resource occupation value converges to a specified resource occupation target, the instance occupied by the offline services in the container cluster is adjusted according to the adjustment operation, a corresponding scheduling policy is set according to the characteristics of the online services, the resources occupied by the offline services are dynamically adjusted based on real-time monitoring, the 24-hour offline services and the online services are mixed, the CPU utilization rate of the container cluster is stabilized at more than 40% all day, the resource utilization rate is greatly improved, and the cost of the container cluster is greatly reduced.

Drawings

Fig. 1 is a flowchart of a method for scheduling resources based on a container cluster according to an embodiment of the present invention;

fig. 2 is a schematic diagram of an architecture of a container cluster and a resource manager according to an embodiment of the present invention;

fig. 3 is a flowchart of a resource scheduling method based on a container cluster according to a second embodiment of the present invention;

fig. 4 is a flowchart of a resource scheduling method based on a container cluster according to a third embodiment of the present invention;

fig. 5 is a flowchart of a resource scheduling method based on a container cluster according to a fourth embodiment of the present invention;

fig. 6 is a schematic structural diagram of a resource scheduling apparatus based on a container cluster according to a fifth embodiment of the present invention;

fig. 7 is a schematic structural diagram of a computer device according to a sixth embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.

Example one

Fig. 1 is a flowchart of a resource scheduling method based on a container cluster according to an embodiment of the present invention, which is applicable to a situation where resources occupied by offline services are dynamically scaled according to a load of the container cluster kubenes, so as to implement mixed distribution of online services and offline services.

In specific implementation, the container cluster kubenees is a platform for automated container operation, provides functions such as application deployment, maintenance, and extension mechanisms, and can be used for managing cross-cluster running containerized applications.

The cluster in the container cluster kubenees refers to a group of nodes which can be physical machines or virtual machines, and a kubenetes platform is installed on the cluster.

As shown in fig. 2, the container cluster kubenes may include the following structure:

master (master node): a container cluster kubenes has a master node master with a series of components, such as an API Server (Application Programming Interface Server) that provides REST (Representational State Transfer) endpoints that can be used to interact with the container cluster kubenes. The master can be used to create and copy the RC of the instance pod (a Replication Controller, RC can be used to guarantee the number of copies of the instance pod are run at any time, thus being able to guarantee that the instance pod is always available).

node (node): a node is a physical machine or a virtual machine, as a worker, also commonly referred to as Minion, and can run the following components:

kubelet: a master node proxy.

Kube-proxy: service uses it to route links to pod.

Docker or Rocker: container clustering kubernets uses container technology to create containers.

And the instance pod is a minimum or simplest basic unit for creating or deploying the container cluster kubernets, one instance pod represents a process running on the container cluster kubernets, and one instance pod encapsulates one or more containers, storage resources, an independent IP (Internet Protocol) address and a policy option for managing and controlling the operation mode of the containers.

The container is an abstraction of program codes and the dependency relationship thereof in an application layer, and a plurality of containers respectively serve as independent processes to run in a user space on the same machine and share an operating system kernel.

Typically, offline traffic is loaded in an instance pod in a node.

dynamic-hybrid-controller (controller): the controller dynamic-hybrid-controller is not a native component of the container cluster kubernets, but a component newly introduced in the container cluster kubernets in the embodiment is a monitoring-based dynamic scaling component implemented in a CRD (custom resource definition) mode of the container cluster kubernets, and controls the main logic of the whole dynamic scaling: the method comprises the steps of acquiring monitoring data, calculating available resources, dynamically performing telescopic operation, notifying resource change and displaying resource allocation.

Typically, a controller dynamic-hybrid-controller is typically provided in a container cluster kubernets, and may be loaded on any node in the container cluster controller kubernets.

The method may be applied to a controller dynamic-hybrid-controller, and is executed by a container cluster-based resource scheduling apparatus, which may be implemented by software and/or hardware, and may be configured in a computer device, such as a server, a workstation, a personal computer, and the like, and specifically includes the following steps:

step 101, reading the currently effective scheduling policy.

In a specific implementation, the controller may continuously monitor creation of a CRD in the container cluster kubenes, where the CRD is an API (Application Program Interface) extension of the container cluster kubenes, and may create a custom API object through the CRD.

In one example, the scheduling policy is expressed as follows:

when rs (reiciaset, used to ensure that the number of copies of a container application always remains at the number of user-defined copies, is created or updated by CRD, i.e. if there is a container exception exited, a new instance pod is automatically created to replace, and if there are more containers in the exception, a new instance pod is automatically recycled, and an explicit update or the like function is provided), the controller dynamic-hybrid-controller may trigger a timed loop process on rs representing a scheduling policy.

Generally, the online service and the offline service have different characteristics in different time periods, for example, in the early morning and early morning time periods, more resources can be scheduled to the offline service, in the daytime time periods, the online service is busy, more resources can be scheduled to the online service, and the like.

Considering different characteristics of online service and offline service, different scheduling strategies can be configured for different time periods, and the controller dynamic-hybrid-controller can read a preset scheduling strategy at regular time, wherein the scheduling strategy has an effective time period, such as a timeRange field in a hybrid strategys (attribute of scheduling strategy) field, and if the current time is within the effective time period, the scheduling strategy is determined to be effective.

It should be noted that, for some service scenarios, the scheduling policy may set the effective time period to be 0 to 24, that is, the scheduling policy is effective 24 hours all day, so as to support mixed floor of offline service and online service for 24 hours, and ensure isolation between offline service and offline service while providing elasticity, which is not limited in this embodiment.

And 102, counting resources occupied by the offline service in the container cluster according to a scheduling strategy to obtain a resource occupation value.

In this embodiment, the controller dynamic-hybrid-controller analyzes the scheduling policy in the CRD, and counts the resources occupied by the offline service in the container cluster kubenetes to obtain a resource occupation value.

In one embodiment of the invention, the granularity of monitoring may be different for different types of resources, for example, for Memory (Memory), IOWait (input/output latency), monitoring resource occupancy of nodes in Memory has a better meaning for resource scheduling, for CPU, monitoring resource occupancy of CPU at instance pod has a better meaning for resource scheduling, etc., so the level of the resource (e.g., metricType field in the resource scalestratecategory field), type (e.g., metric field in the resource scalestratecategory field) may be set in the scheduling policy.

The level metricType may include a first level host and a second level pod, where the first level host is used to indicate a resource occupied by the monitoring offline service at a node, and the second level pod is used to indicate a resource occupied by the monitoring offline service at an instance pod; the type metric may include CPU, memory, IOWait, etc.

If the level metrctype is the first level host, obtaining an instance pod where all offline services are located through match label in a scheduling policy, and pulling corresponding monitoring data from prometheus according to the name of the instance pod, so as to count resources under an offline service occupation type metric in a node where the offline services are located, and obtain a resource occupation value.

If the level metricType is the second level pod, obtaining an instance pod where all offline services are located through match label in a scheduling policy, querying a node where the instance pod is located according to the name of the instance pod, and pulling corresponding monitoring data according to prometheus from the name of the name node of the instance pod, so as to count resources under an offline service occupation type metric in the instance pod where the offline services are located, and obtain a resource occupation value.

It should be noted that, besides performing monitoring in a hierarchical manner, resources under the offline service occupation type metric may be collected and a resource occupation value may be obtained in a unified manner in a node where the offline service is located, or resources under the offline service occupation type metric may be collected and a resource occupation value may be obtained in a unified manner in an instance pod where the offline service is located, and the like, which is not limited in this embodiment.

And 103, determining adjustment operation for the offline service according to the scheduling strategy so as to make the resource occupation value converge towards the specified resource occupation target.

In this embodiment, the controller dynamic-hybrid-controller analyzes the scheduling policy in the CRD, determines an occupation target set for the resource, and calculates an adjustment operation to be performed on the offline service at this time, where the adjustment operation may adjust the resource occupation value so that the resource occupation value converges toward the resource occupation target, that is, the resource occupation value increases or decreases toward the resource occupation target.

In an embodiment of the present invention, the adjusting operation includes a capacity reduction operation and a capacity expansion operation, where the capacity reduction operation is used to reduce the instance pod occupied by the offline service in the container cluster kubenes, and the capacity expansion operation is used to expand the instance pod occupied by the offline service in the container cluster.

There is a threshold set for the resource in the scheduling policy, such as a threshold field.

In this embodiment, a threshold is used as a reference, a resource occupation target is set for the resource based on the threshold, and the resource occupation value is compared with the corresponding resource occupation target.

If the resource occupation value is larger than the resource occupation target, determining to execute capacity reduction operation on the offline service so as to reduce the resource occupation value to the resource occupation target.

And if the resource occupation value is less than or equal to the resource occupation target, determining to execute capacity expansion operation on the offline service so as to increase the resource occupation value to the resource occupation target.

In one example, if the resource is the processor resource CPU, then the threshold is set as the resource occupancy target.

In this example, the CPUUsed is analyzed, and if the CPUUsed is greater than threshold, it is determined that a capacity reduction operation is performed on the offline service, and if the CPUUsed is not greater than threshold, it is determined that a capacity expansion operation is performed on the offline service, where the CPUUsed is a resource occupancy value of the CPU resource.

In another example, if the resource is a Memory resource Memory, the allocable value and the floppy drive gradual value of the Memory resource in the node where the offline service is located are read.

And subtracting the soft eviction value and the difference value of the threshold value from the allocable value in sequence to be used as a resource occupation target.

When the available resources are low, the container cluster kubenees still need to ensure the stability of the node, which is particularly important when dealing with incompressible resources (such as memory or disk space), when these resources are exhausted, the node becomes unstable, the container cluster kubenees monitors and prevents the available resources from being exhausted, in this case, the container cluster kubenees terminates one or more instances pod to retrieve the exhausted resources, and at this time, the threshold value triggering the container cluster kubenees to recover the resources may be referred to as a floppy drive threshold value.

In this example, the MemUsed is analyzed, and if the MemUsed is greater than node.mem. allocatable-node.mem. allocatable 10% -threshold, it is determined to perform a capacity reduction operation on the offline service, and if the MemUsed is less than or equal to node.mem. allocatable-node.mem. allocatable 10% -threshold, it is determined to perform a capacity expansion operation on the offline service, where the MemUsed is a resource occupancy value of the memory resource, the node.mem. allocatable is an allocatable value of the memory resource, and the node.mem. allocatable 10% is a floppy drive gradual value.

Of course, the above manner of the calculation resource occupation target is only an example, and when the embodiment of the present invention is implemented, other manners of the calculation resource occupation target may be set according to actual situations, which is not limited in this embodiment of the present invention. In addition, besides the above manner of occupying the target by the computing resource, a person skilled in the art may also adopt other manners of occupying the target by the computing resource according to actual needs, and the embodiment of the present invention is not limited to this.

And 104, adjusting the occupied instance of the offline service in the container cluster according to the adjustment operation.

In this embodiment, the controller dynamic-hybrid-controller may perform the adjustment operation after determining the adjustment operation, thereby adjusting (decreasing or increasing) the instance pod occupied by the offline service in the container cluster kubenes.

The online service has the possibility of burst flow, when the burst online service arrives, the resources of the offline service are often moved to the online service, and the offline service is directly expelled under the original container cluster kubernets, so that the offline service has certain loss.

In an embodiment of the present invention, the scheduling policy has a first time window set for a capacity reduction operation, such as a scaledowncooldowwindows minutes field, and a second time window set for a capacity expansion operation, such as a scaleuccoolddownwindows minutes field.

If the adjustment operation is a capacity reduction operation, the last time of performing the capacity reduction operation is taken as a starting point, and a first time window scaledownwindowminutes is set, for example, the last time of performing the capacity reduction operation is 10:00, the first time window scaledowncondowwindowminutes is 30, and the first time window scaledowncooldowwindowminutes represents 10:00-10: 30.

The current time is read from the operating system or network and compared to the first time window scaledolcoldownwindows minutes.

And if the current time is within the first time window, pausing the capacity reduction operation.

And if the current time is out of the first time window, executing a capacity reduction operation to reduce the instance pod occupied by the offline service in the container cluster kubenes.

If the adjustment operation is a capacity expansion operation, the time of performing the capacity expansion operation last time is used as a starting point, and a second time window scaleupcooldownwindows minutes is set, for example, the time of performing the capacity expansion operation last time is 10:00, the second time window scaleupcooldownwindows minutes is 30, and the second time window scaleupcooldownwindows minutes represents 10:00-10: 30.

The current time is read from the operating system or network and compared to the second time window scaleuccoolddownwindows minutes.

And if the current time is within the second time window scaleuccoolddownwindows minutes, suspending the execution of the capacity expansion operation.

And if the current time is outside the second time window scaleuccollowlndowminites, executing capacity expansion operation to increase the instance pod occupied by the offline service in the container cluster kubenes.

In this embodiment, a first time window is set for the capacity reduction operation, a second time window is set for the capacity expansion operation, the capacity reduction operation is executed after the first time window is spaced, the capacity expansion operation is executed after the second time window is spaced, and the capacity reduction operation and the capacity expansion operation are gradually executed, so that frequent execution of the capacity reduction operation and the capacity expansion operation is avoided, influences on the offline service are reduced, and stability of the offline service is ensured.

In another embodiment of the present invention, the scheduling policy has a delay time set for the capacity reduction operation, such as a graceful scaleporiods field.

When the offline service occupies more resources, if the capacity reduction operation is executed in a short time, a part of the offline service may be forcibly driven away, which may cause fluctuation and even error of the offline service, and affect the stability of the offline service.

In another embodiment of the present invention, the scheduling policy has a first step size, such as scaledonstep field, set for the capacity reduction operation and a second step size, such as scaleupdep field, set for the capacity expansion operation.

If the adjusting operation is a capacity reduction operation, reducing the number of instances pod of the first step size scaleDownStep in the container cluster kubenes, thereby realizing the capacity reduction.

If the adjustment operation is a capacity expansion operation, adding the number of instances pod of the second step size scaleUpStep in the container cluster kubenes, thereby realizing the capacity expansion.

In this embodiment, the first step length set for the capacity reduction operation and the second step length set for the capacity expansion operation are reduced by the instances with the first step length each time, and the instances with the second step length each time, so that the amplitude of executing the capacity reduction operation and the capacity expansion operation is appropriate, the instances are prevented from being greatly reduced or increased, the influence on the offline service is reduced, and the stability of the offline service is ensured.

The example pod is used as a carrier of mixed online business and offline business, under the current container cluster kubernets system, the resource specification of the example pod needs to be stated in advance, the native container cluster kubernets does not support online adjustment of the specification of the example pod after the example pod is started, and the example pod needs to be created again for adjustment of the resource specification.

In still another embodiment of the present invention, as shown in fig. 2, a rescaled (scheduling component) may be set in the container cluster kubernets, where the rescaled is not a native component of the container cluster kubernets, but a component newly introduced in the container cluster kubernets in this embodiment is loaded on each node in a daemon manner, and may monitor state changes of all instances pod on the local computer, and dynamically adjust a CPU bound by the instances pod according to a stated binding configuration, so as to modify a configuration file cgroup without restarting the nodes pod, thereby achieving an effect of resource scaling.

Wherein, daemonSet ensures that a copy of an instance pod is run on all (or some) of the node nodes in the container cluster kubernets.

cgroup is a resource restriction mechanism provided by linux, and it can make configuration modification by editing the form of file, and the containers in the container cluster can make resource restriction based on cgroup.

The controller dynamic-hybrid-controller may reduce or increase the instance pod occupied by the offline service in the container cluster kubenetes according to the adjustment operation, determine resources (such as CPUs, memories, and the like) allocated to the instance pod, and obtain resource allocation information.

Calling an update interface patch of the container cluster kubernets to write resource allocation information into a configuration file pod.

And when the configuration file pod is monitored by the scheduling component vscaled, modifying the configuration file cgroup of the container according to the resource allocation information pod, and allocating resources for the instance pod.

In the specific implementation, a scheduling component vscaled deployed on each node monitors pod update on the node, analyzes vertical capacity expansion configuration in resource allocation information pod.

Wherein, the cgroup paths that different resources need to be modified are respectively:

the cpuiset (the cpuiset is a subsystem in a CPU group, and the cpuiset basically functions to limit a certain group of processes or containers to only run on some CPUs and memory nodes, and isolates the online service and the offline service by setting the cpuiset, which has stronger isolation compared with the traditional mixed part scheme):

/sys/fs/cgroup/cpuset/kubepods/$pod_id/$container_id/cpuset.cpus

Example two

Fig. 3 is a flowchart of a resource scheduling method based on a container cluster according to a second embodiment of the present invention, where the present embodiment further details the operation of allocating CPU resources by a scheduling component based on the foregoing embodiment, and the method specifically includes the following steps:

step 301, receiving an update event of an instance.

The container cluster kubernets provides a resource model of request/limit to describe resource declaration of the container, the resource declaration of the request is the basis of the container cluster kubernets in scheduling, and the resource declaration of the limit is the resource upper limit which can be used by the container in actual operation.

Based on different requests/limits ratios, three QoS levels of guarded, burst and BestEffort are defined by container cluster kubernets, and in an actual production environment, most service containers are overfall to a certain degree so as to provide resource utilization rate.

Wherein, when limits is greater than requests, instance pod belongs to the over-sell state.

On the node, the request/limit of the container is converted into a configuration file linux group, and the purpose of limiting the use of the container resource is achieved through the configuration file cggroup.

In terms of the use of the processor CPU, the configuration file cgroup provides two resource limitation modes, namely a preemption mode in which a CPU share is used as a weight through CPU.

In the resource model of the container cluster kubernets, the processor CPU is supported to be used in a preemption mode through CPU share, and the processor CPU is supported to be used in a binding mode through CPU set, but the CPU set is used on the premise that the request/limit ratio of the container is one to one, namely the QoS level of the container is the highest QoS level of Guaranteed, the priority is highest when the eviction and preemption occur, and the highest priority is not in line with the actual priority of the offline service.

In practical applications, a user may modify the indication field of metadata in the instance pod of the online service or the offline service according to the requirement of the online service or the offline service, and declare a processor that desires to bind, such as "cpu set.

The present embodiment can be applied to a scheduling component vscaled, which can monitor the state change of an instance pod on the same node.

When the association field of metadata in an instance pod is modified, the scheduling component vsccaled in the node where the instance pod is located may receive an update event for the instance pod, the update event including configuration information for the binding processor.

Further, the scheduling component vscaled may listen to the state change of the instance pod on the same node through the list/watch mechanism of the container cluster kubenees.

The container cluster kubenes is provided with an interface service API server, an Etcd stores data information of the cluster, the interface service API server serves as a uniform entry, data operation generally passes through the interface service API server, and a client (kubenet/scheduler (scheduler is a component of a kubenetes control plane of the container cluster and is responsible for scheduling and binding of container resources and hosts)/controller-manager/scheduling component vscaled) monitors the create, update and delete events of resources (pod/rs/rc and the like) in the interface service API server through list-watch, and calls corresponding event processing functions according to event types.

list is list API resources for calling resources, and is realized based on HTTP (Hypertext Transfer Protocol) short link; the watch is a watch API for calling resources to monitor resource change events and is realized based on an HTTP long link.

The Watch API is often taken as "true", which means that an HTTP long connection is used to continuously listen to pod-related events (e.g., update events), and a Watch event is returned whenever an event arrives.

Therefore, the scheduling component vcscaled can send a walk request to the interface service API server to subscribe the configuration information of the instance pod binding processor, after receiving the walk request in the interface service API server, call rw.walk () to create a router, use the serveWatch () to process the request, that is, record whether the configuration information of the instance pod in the same node changes, when the change occurs, read an event object from the result channel (channel) of the router, generate an update event, encode and write the update event into a response stream, and send the update event to the scheduling component vcscaled, so that the scheduling component vcscaled receives the update event sent by the interface service API server when the configuration information of the instance pod in the same node changes.

Step 302, in the processor in the node, a logic core which is in an idle state and satisfies the configuration information is searched as a target core.

In this embodiment, the scheduling component vscaled may read the topology of the processor in the node from matadata (metadata) of the node where the scheduling component vscaled is located, and maintain the state of the processor in the memory, where the state includes an idle state and a bound state, the idle state indicates that the processor is not bound, and the bound state indicates that the processor is bound.

Typically, the topology is a hierarchy of socket, core, hyperThread.

The socket is a card slot where the processor is located.

The core is a physical core/processor of the processor, has independent circuit elements and L1 and L2 caches, and can independently execute instructions.

hyperThread is a Hyper-threading (HT) on a logical core/processor (LCPU). A logical core refers to a core at a logical level within the same physical core. A hyper-thread may allocate a time slice to one logical core while the other logical core waits for the interval of instruction execution (waiting for the next instruction to be fetched from cache or memory). High speed switches between the two logic cores, making this interval imperceptible to the application.

In a specific implementation, when monitoring the configuration information of the node binding processor, the scheduling component vscaled may search, in the processor in the current node, for a logical core that is in an idle state (i.e., not bound) and satisfies the configuration information, as a target core.

Taking the number as an example of configuration information, the scheduling component vsccaled may read a target value from the configuration information, e.g. the target value is 4 in the configuration information "cpu set.

And searching the logic cores which are in an idle state and the number of the logic cores is the target value in the processor in the node as the target cores.

In addition, the logic core is configured with a serial number, and system components such as a network card and the like in the node generally preferentially use the logic core with the serial number of 0, so to avoid preempting the logic core with the serial number of 0, the scheduling component vscaled may perform descending sorting on the logic cores in the processor in the node according to the serial numbers of the logic cores, and search the logic core which is in an idle state and satisfies the configuration information according to the sequence after the descending sorting to serve as a target core.

For example, after the scheduling component vscaled is started, a list of logical cores is maintained, and the sequence numbers are [0, 31], if the two previous instance pods sequentially declare the binding processor, 4 logical cores and 2 logical cores are respectively needed, the sequence numbers of the sequentially allocated logical cores are [28, 31], [26, 27], and then the other instance pods are bound in the logical core with the sequence number [0, 25 ].

Under the condition that the number of bound logic cores is large, the logic core in an idle state (namely unbound) does not meet the requirement of the current example pod, if the scheduling component vscaled does not find the target core, a binding failure event can be generated, the binding failure event is bound to the current example pod, and the upper-layer application of the container cluster kubernets is notified.

And 303, if the target core is found, binding the instance and the target core.

If the scheduling component vscaled finds a logical core in an idle state (i.e., unbound) and that satisfies the configuration information as the target core, the instance pod may be bound with the target core.

In a specific implementation, the sequence number of the target core may be set to the binding mode cpuiset in the first configuration file cgroup of the instance pod.

And calling a docker update interface to update the sequence number of the target core to a binding mode cpuiset in a second configuration file cgroup of the container in the instance pod, wherein when the container in the instance pod calls the processor to execute the service, the container calls a logic core (namely the target core) corresponding to the sequence number according to the binding mode cpuiset in the second configuration file cgroup to execute the service.

And if the first configuration file cgroup of the instance pod is successfully set and the second configuration file cgroup of the container in the instance pod is successfully updated, determining that the instance pod is successfully bound with the target core.

And if the binding of the instance pod and the target core is successful, modifying the state of the target core from an idle state to a binding state so as to prevent the subsequent calling by other instance pods.

Further, the first configuration file cgroup and the second configuration file cgroup both belong to the configuration file cgroup, the configuration file cgroup is a resource restriction mechanism provided by the linux operating system, and is divided into a plurality of subsystems, which respectively restrict different types of resources, such as CPUs, memories (memories), cpsets, and the like, wherein the cpsets cgroup restricts the range of processor cores that can be used by the process, and processor cores that exceed the range cannot be used, and in a mixed part scene, the restriction of the cpsets can prevent iowait of offline services from spreading over all the processor cores to affect the online services.

For a native container cluster kubernets, if a container bound processor core is adjusted, a container needs to be created again, which has a great influence on a service, but in the embodiment, dynamic modification of the binding is realized by dynamically adjusting a first configuration file of a node and a second configuration file of the container, and the binding configuration can be modified and can take effect immediately without creating an instance pod and the container again, so that the influence on the service can be greatly reduced.

If updating the second configuration file cgroup of the container in the instance pod fails, it is determined that the binding between the instance pod and the target core fails, at this time, an update failure event may be generated, the update failure event is bound to the current instance pod, and the upper layer application of the container cluster kubernets is notified.

The scheduling component is in a component form, and the container cluster kubenetes originally supports the loading and running of the scheduling component, so that the container cluster kubenetes does not need to be invaded by a code of the container cluster kubenetes, the influence on the running of the container cluster kubenetes can be reduced, and the scheduling component is used as a bypass plug-in and is plug-and-play, so that the method has good universality.

The scheduling component receives an update event of the instance, the update event comprises configuration information used for binding the processor, the logic core which is in an idle state and meets the configuration information is searched in the processor in the node and is used as a target core, and if the target core is searched, the instance and the target core are bound.

The scheduling component realizes the automatic management of bound processor cores, automatically distributes the bound processor cores through the declarative configuration of the nodes, greatly reduces the operation and maintenance cost, and can enable I/O intensive offline services to be isolated on a specific processor core by means of binding the processor cores due to the fact that iowait promotion brought by over-selling is carried out in the operation process of the offline services, so that the online services cannot be influenced, and the stability of the online services under a mixed scene is guaranteed.

EXAMPLE III

Fig. 4 is a flowchart of a binding method for a processor according to a third embodiment of the present invention, where the present embodiment further adds an operation of deleting an instance pod and detecting a binding condition by a scheduling component vsccaled based on the foregoing embodiment, and the method specifically includes the following steps:

step 401, receiving an update event of an instance.

Wherein the update event includes configuration information for the binding processor.

Step 402, in the processor in the node, a logic core which is in an idle state and satisfies configuration information is searched as a target core.

And step 403, if the target core is found, binding the instance and the target core.

Step 404, receiving a deletion event of the instance.

In practical application, according to the requirement of the online service or the offline service, the instance pod where the online service or the offline service is located can be deleted through a command such as "kubecect delete pod name".

The scheduling component vscaled can monitor the state change of the instance pod on the same node through a list/watch mechanism and the like, and when a user deletes a certain instance pod, the scheduling component vscaled in the node where the instance pod is located can receive a deletion event of the instance pod.

In a specific implementation, the scheduling component vscaled may send a walk request to the interface service API server to subscribe to information of deleting an instance pod, in the interface service API server, after receiving the walk request, call rw.

Step 405, responding to the deletion event, inquiring the logic core currently bound by the instance as a candidate core.

And the scheduling component vscaled responds to the deletion event of the instance pod and queries the logic core currently bound by the instance pod to be deleted as a candidate core.

In general, a second configuration file cgroup may be read from a container of the to-be-deleted instance pod, a sequence number in the binding mode may be read from the second configuration file cgroup, and a logical core represented by the sequence number may be determined to be currently bound by the to-be-deleted instance pod, thereby obtaining a candidate core.

Of course, in addition to querying the currently bound logical core of the instance by using the second configuration file cgroup of the container, the currently bound logical core of the instance pod may also be queried by using the first configuration file cgroup of the instance pod, that is, reading the first configuration file cgroup from the to-be-deleted instance pod, reading the sequence number in the binding mode from the first configuration file cgroup, and determining the logical core represented by the currently bound sequence number of the to-be-deleted instance pod, so as to obtain the candidate core, which is not limited in this embodiment.

Step 406, modify the candidate core from the bound state to an idle state.

For the logic core (i.e. candidate core) bound by the instance pod to be deleted, the state of the logic core can be modified from the binding state to the idle state, so that the logic core can be preempted and bound by other instance pods, the normal allocation of the logic core is ensured, and the condition of idle occupation, namely bound and no service allocation execution, is avoided.

Step 407, inquiring the logic core currently bound by the instance as a reference.

In some cases, the logical core to which the instance pod actually binds may change such that it does not conform to the logical core to which it was previously expected to bind.

For example, since the scheduling component vscaled is not a control program of the container cluster kubernets, when the container cluster kubernets is restarted due to a failure, the instance pod and the logical core may be bound in a default manner, and the instance pod and the logical core are not bound according to the configuration information recorded by the scheduling component vscaled.

In this embodiment, the scheduling component vscaled may periodically query the logical core currently bound by each instance pod in the current node as a reference core.

In general, a second configuration file cgroup may be read from a container of each instance pod, a sequence number in the binding mode may be read from the second configuration file cgroup, and a logical core represented by the sequence number currently bound by each instance pod in the current node may be determined, so as to obtain a reference core.

Of course, in addition to querying the logical core to which the instance pod is currently bound by the second configuration file cgroup of the container, the logical core to which the instance pod is currently bound may also be queried by using the first configuration file cgroup of the instance pod, that is, reading the first configuration file cgroup from each instance pod, reading the sequence number in the binding mode from the first configuration file cgroup, and determining the logical core to which the to-be-deleted instance pod is currently bound and which is represented by the sequence number, so as to obtain the reference score, and the like, which is not limited in this embodiment.

And step 408, if the parameter check is different from the target core, switching the example from the bound parameter check to the bound target core.

For the same instance pod, the reference core is a logic core actually bound, the target core is a logic core expected to be bound, and the scheduling component vscaled can compare the reference core with the target core.

If the reference core is the same as the target core, the fact that the logic core actually bound is the same as the logic core expected to be bound is shown, and the scheduling component vscaled does not process the logic core and waits for the next detection.

If the parameter core is different from the target core, the logic core which indicates the actual binding is different from the logic core which expects the binding, and the scheduling component vscaled switches the instance pod from the binding parameter core to the binding target core, so that the logic core which actually binds is the same as the logic core which expects the binding.

For the unbound reference cores after switching the bound target core, the state of the reference core may be set to an idle state.

Example four

Fig. 5 is a flowchart of a resource scheduling method based on a container cluster according to a second embodiment of the present invention, where the present embodiment is based on the foregoing embodiment, and further adds an operation of notifying a resource manager to schedule offline services and resource display, where the method specifically includes the following steps:

and step 501, reading the currently effective scheduling policy.

Step 502, counting the resources occupied by the offline service in the container cluster according to the scheduling policy, and obtaining a resource occupation value.

Wherein the offline traffic is loaded in instances in the nodes.

Step 503, determining an adjustment operation for the offline service according to the scheduling policy, so that the resource occupation value converges to the specified resource occupation target.

And step 504, adjusting the occupied instance of the offline service in the container cluster according to the adjustment operation.

And 505, if the adjustment operation is completed, inquiring the resources allocated to the instance to obtain resource allocation information.

Step 506, notifying the resource manager of the resource allocation information.

Since the container cluster kubernets is more a container arrangement system oriented to online services, offline services are mostly scheduled through a resource manager framework, wherein the resource manager is used for scheduling offline services, such as yarn, in the container cluster.

As shown in fig. 2, yann is a system responsible for scheduling and running offline services, and is composed of resource manager, node manager, app master, and service processes.

The resource manager is responsible for the unified management and allocation of all resources in the yann cluster, receives resource report information from each node manager, and allocates the information to the app master of each application program according to a certain strategy.

The node manager is an agent on a single node in the yann cluster and is responsible for managing a single computing node, and functions include maintaining communication with the resource manager, managing the life cycle of a container, monitoring the resource usage (memory, CPU, etc.) of a container, tracking node health, managing logs, and attached services for different applications.

The app master is a container which is first started when a yarn task runs and is responsible for running the whole task, including application, starting, kill (closing), state checking and the like of the container.

Therefore, in the mixed scenario, the computing resources of the containers are delivered by the container cluster kubernets, the offline service is scheduled by the resource manager (e.g., yann), and both perform scheduling coordination to some extent, that is, the container resource specification of the offline service changes, so that the resource manager (e.g., yann) senses the number to perform reasonable task scheduling, which depends on a coordination notification mechanism between the container cluster kubernets and the resource manager (e.g., yann).

In this embodiment, after performing dynamic adjustment operation each time, the resource occupied by the new offline service is obtained, and the resource allocation information is obtained, the controller dynamic-hybrid-controller may notify the resource manager (e.g., yann) of the resource allocation information, and the resource manager (e.g., yann) schedules the offline service of the container cluster kubernets according to the resource allocation information, so as to ensure normal operation of the offline service.

In a specific implementation, an event notifier dynamic-resource-event may be preset, the event notifier dynamic-resource-event is a component of interaction between the controller dynamic-hybrid-controller and the resource manager (e.g., yann), and may be an HTTP service, and is independent from the container cluster kubernets and the resource manager (e.g., yann), the controller dynamic-hybrid-controller may send the resource allocation information to the event notifier dynamic-resource-event according to an HTTP protocol, and the event notifier dynamic-resource-event is used for sending the resource allocation information to the resource manager according to the HTTP service.

And 507, writing the resource allocation information into preset index configuration.

In a specific implementation, as shown in fig. 2, a monitor, such as prometheus (an open source container cluster monitoring system, which may provide real-time monitoring data of multidimensional resources such as memory and CPU of each host and container), may be preset, and the monitor (such as prometheus) is independent of the container cluster kubernets and the resource manager (such as yarn).

The controller dynamic-hybrid-controller can write resource allocation information into the index configuration metrics, the monitor (such as prometheus) regularly pulls the index configuration metrics exposed by the controller dynamic-hybrid-controller, and when the index configuration is pulled by the monitor, the index configuration metrics are displayed in a node unit, so that a user can browse resource conditions allocated to offline services in a web (webpage) mode and the like in the node unit, thereby quickly responding to possible abnormalities of the container cluster kubernets and ensuring normal operation of the container cluster kubernets.

It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.

EXAMPLE five

Fig. 6 is a block diagram of a resource scheduling apparatus based on a container cluster according to a fifth embodiment of the present invention, where a controller is loaded on a node in the container cluster, and the apparatus is located in the controller, and specifically includes the following modules:

a scheduling policy reading module 601, configured to read a currently effective scheduling policy;

a resource occupation value statistics module 602, configured to count resources occupied by the offline service in the container cluster according to the scheduling policy, to obtain a resource occupation value, where the offline service is loaded in an instance in the node;

an adjustment operation determining module 603, configured to determine an adjustment operation for the offline service according to the scheduling policy, so that the resource occupancy value converges to an assigned resource occupancy target;

and an adjusting operation executing module 604, configured to adjust an instance occupied by the offline service in the container cluster according to the adjusting operation.

In an embodiment of the present invention, the scheduling policy reading module 601 includes:

the system comprises a preset strategy reading submodule and a scheduling module, wherein the preset strategy reading submodule is used for reading a preset scheduling strategy, and the scheduling strategy has an effective time period;

and the effective determining submodule is used for determining that the scheduling strategy is effective if the current time is within the effective time period.

In one embodiment of the present invention, the scheduling policy has a level and a type of resource;

the resource occupancy value statistics module 602 includes:

a node resource counting submodule, configured to count, in a node where an offline service is located, resources occupied by the offline service in the type if the level is a first level, and obtain a resource occupation value;

and the example resource counting submodule is used for counting the resources occupied by the offline service in the type in the example of the offline service if the level is the second level, and acquiring a resource occupation value.

In an embodiment of the present invention, the adjusting operation includes a capacity reduction operation and a capacity expansion operation, and the scheduling policy has a threshold set for the resource;

the adjustment operation determination module 603 includes:

a resource occupation target setting submodule for setting a resource occupation target for the resource based on the threshold;

a capacity reduction operation determining submodule, configured to determine to perform a capacity reduction operation on the offline service if the resource occupancy value is greater than the resource occupancy target, so that the resource occupancy value is reduced toward the resource occupancy target;

and the capacity expansion operation determining submodule is used for determining to execute capacity expansion operation on the offline service if the resource occupation value is less than or equal to the resource occupation target so as to increase the resource occupation value to the resource occupation target.

In an embodiment of the present invention, the resource occupation target setting sub-module includes:

a threshold setting unit, configured to set the threshold as a resource occupation target if the resource is a processor resource;

a resource information reading unit, configured to read an allocable value and a floppy drive gradual value of a memory resource in a node where the offline service is located, if the resource is the memory resource;

and the difference value setting unit is used for sequentially subtracting the difference values of the soft eviction value and the threshold value from the allocable value to serve as a resource occupation target.

In an embodiment of the present invention, the scheduling policy has a first time window set for capacity reduction operation and a second time window set for capacity expansion operation;

the adjusting operation executing module 604 includes:

a first time window setting submodule, configured to set the first time window by using the time for executing the capacity reduction operation last time as a starting point if the adjustment operation is a capacity reduction operation;

a first time window setting submodule, configured to execute the capacity reduction operation if the current time is outside the first time window, so as to reduce an instance occupied by the offline service in the container cluster;

alternatively, the first and second electrodes may be,

a second time window setting submodule, configured to set the second time window by using the time for performing the expansion operation last time as a starting point if the adjustment operation is an expansion operation;

and the second time window setting submodule is used for executing the capacity expansion operation if the current time is outside the second time window so as to increase the instances occupied by the offline service in the container cluster.

In an embodiment of the present invention, the scheduling policy has a delay time set for the capacity reduction operation;

the adjusting operation executing module 604 includes:

and the delay and capacity reduction submodule is used for executing the capacity reduction operation after the delay time is exceeded if the adjustment operation is the capacity reduction operation so as to reduce the instances occupied by the offline service in the container cluster.

In an embodiment of the present invention, the scheduling policy has a first step size set for a capacity reduction operation and a second step size set for a capacity expansion operation;

the adjusting operation executing module 604 includes:

a step size reduction sub-module, configured to reduce, if the adjustment operation is a size reduction operation, a number of instances in the container cluster that is the first step size;

alternatively, the first and second electrodes may be,

and the step capacity expansion submodule is used for increasing the number of the instances with the second step length in the container cluster if the adjustment operation is the capacity expansion operation.

In an embodiment of the present invention, the adjusting operation performing module 604 includes:

an instance adjusting submodule, configured to reduce or increase an instance occupied by the offline service in the container cluster according to the adjusting operation;

a resource allocation information determining submodule, configured to determine resources allocated to the instance, and obtain resource allocation information;

a configuration file writing module, configured to write the resource allocation information into a configuration file of the instance, where the configuration file allocates resources to the instance according to the resource allocation information when monitored by a scheduling component, where the scheduling component is loaded in a node where the instance is located.

In one embodiment of the present invention, further comprising:

the resource allocation information query module is used for querying the resources allocated to the instance to obtain resource allocation information if the adjustment operation is completed;

a resource manager notifying module, configured to notify a resource manager of the resource allocation information, where the resource manager is configured to schedule the offline service in the container cluster;

and the index configuration writing module is used for writing the resource allocation information into preset index configuration, the index configuration is displayed by taking the node as a unit when being pulled by a monitor, and the event notifier is independent of the container cluster.

In one embodiment of the present invention, the resource manager notification module includes:

and the event notifier is used for sending the resource allocation information to the resource manager, and the event notifier is independent of the container cluster and the resource manager.

The resource scheduling device based on the container cluster provided by the embodiment of the invention can execute the resource scheduling method based on the container cluster provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.

EXAMPLE six

Fig. 7 is a schematic structural diagram of a computer device according to a sixth embodiment of the present invention. FIG. 7 illustrates a block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention. The computer device 12 shown in fig. 7 is only an example and should not bring any limitations to the functionality or scope of use of the embodiments of the present invention.

As shown in FIG. 7, computer device 12 is in the form of a general purpose computing device. The components of computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.

Bus 18 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, micro-channel architecture (MAC) bus, enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

Computer device 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer device 12 and includes both volatile and nonvolatile media, removable and non-removable media.

The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. Computer device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 7, and commonly referred to as a "hard drive"). Although not shown in FIG. 7, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.

A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.

Computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with computer device 12, and/or with any devices (e.g., network card, modem, etc.) that enable computer device 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, computer device 12 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with the other modules of computer device 12 via bus 18. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

The processing unit 16 executes various functional applications and data processing by executing programs stored in the system memory 28, for example, to implement the resource scheduling method based on container cluster provided by the embodiment of the present invention.

EXAMPLE seven

A seventh embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements each process of the resource scheduling method based on a container cluster, and can achieve the same technical effect, and in order to avoid repetition, the computer program is not described herein again.

A computer readable storage medium may include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A resource scheduling method based on a container cluster, wherein a controller is loaded at a node in the container cluster, and the method is applied to the controller, and comprises the following steps:

reading a currently effective scheduling strategy;

2. The method of claim 1, wherein reading the currently effective scheduling policy comprises:

reading a preset scheduling strategy, wherein the scheduling strategy has an effective time period;

and if the current time is within the effective time period, determining that the scheduling strategy is effective.

3. The method of claim 1, wherein the scheduling policy has a level, a type of resource;

the counting resources occupied by the offline service in the container cluster according to the scheduling policy to obtain a resource occupation value includes:

if the level is the first level, counting the resources occupied by the offline service in the node where the offline service is located under the type to obtain a resource occupation value;

and if the level is the second level, counting the resources occupied by the offline service in the type in the instance of the offline service, and obtaining a resource occupation value.

4. The method according to claim 1, wherein the adjustment operation includes a capacity reduction operation and a capacity expansion operation, and the scheduling policy has a threshold set for the resource;

the determining, according to the scheduling policy, an adjustment operation for the offline service to make the resource occupation value converge toward a specified resource occupation target includes:

setting a resource occupation target for the resource based on the threshold;

if the resource occupation value is larger than the resource occupation target, determining to execute capacity reduction operation on the offline service so as to reduce the resource occupation value to the resource occupation target;

and if the resource occupation value is less than or equal to the resource occupation target, determining to perform capacity expansion operation on the offline service so as to increase the resource occupation value to the resource occupation target.

5. The method of claim 4, wherein the targeting the resource occupancy based on the threshold comprises:

if the resource is a processor resource, setting the threshold value as a resource occupation target;

if the resource is a memory resource, reading the allocable value and the floppy drive gradual value of the memory resource in the node where the offline service is located;

and subtracting the difference value of the soft eviction value and the threshold value from the allocable value in sequence to serve as a resource occupation target.

6. The method according to claim 1, wherein the scheduling policy has a first time window set for a capacity reduction operation and a second time window set for a capacity expansion operation;

the adjusting, according to the adjustment operation, an instance occupied by the offline service in the container cluster includes:

if the adjusting operation is a capacity reduction operation, setting the first time window by taking the time for executing the capacity reduction operation for the last time as a starting point;

if the current time is outside the first time window, executing the capacity reduction operation to reduce the instances occupied by the offline service in the container cluster;

alternatively, the first and second electrodes may be,

if the adjustment operation is a capacity expansion operation, setting the second time window by taking the time for executing the capacity expansion operation for the last time as a starting point;

and if the current time is outside the second time window, executing the capacity expansion operation to increase the instances occupied by the offline service in the container cluster.

7. The method according to claim 1, wherein the scheduling policy has a delay time set for the capacity reduction operation;

and if the adjustment operation is a capacity reduction operation, executing the capacity reduction operation after the delay time is exceeded so as to reduce the instances occupied by the offline service in the container cluster.

8. The method according to claim 1, wherein the scheduling policy has a first step size set for a capacity reduction operation and a second step size set for a capacity expansion operation;

if the adjustment operation is a capacity reduction operation, reducing the number of instances in the container cluster to be the first step length;

alternatively, the first and second electrodes may be,

and if the adjustment operation is a capacity expansion operation, adding instances with the number of the second step length in the container cluster.

9. The method of claim 1, wherein the adjusting the instances occupied by the offline traffic in the container cluster according to the adjusting operation comprises:

reducing or increasing the instances occupied by the offline service in the container cluster according to the adjustment operation;

determining resources allocated to the instances, and obtaining resource allocation information;

and writing the resource allocation information into a configuration file of the instance, wherein the configuration file allocates resources for the instance according to the resource allocation information when monitored by a scheduling component, and the scheduling component is loaded in a node where the instance is located.

10. The method according to any one of claims 1-9, further comprising:

if the execution of the adjustment operation is completed, inquiring the resources allocated to the instance to obtain resource allocation information;

notifying a resource manager of the resource allocation information, the resource manager being configured to schedule the offline service in the container cluster;

and writing the resource allocation information into a preset index configuration, wherein the index configuration is displayed by taking the node as a unit when being pulled by a monitor, and the event notifier is independent of the container cluster.

11. The method of claim 10, wherein notifying a resource manager of the resource allocation information comprises:

and sending the resource allocation information to an event notifier, wherein the event notifier is used for sending the resource allocation information to a resource manager, and the event notifier is independent of the container cluster and the resource manager.

12. An apparatus for scheduling resources based on a container cluster, wherein a controller is loaded at a node in the container cluster, and the apparatus is located at the controller, comprising:

13. A computer device, characterized in that the computer device comprises:

one or more processors;

a memory for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the method for container cluster based resource scheduling according to any of claims 1-11.

14. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method for container cluster based resource scheduling according to any one of claims 1 to 11.