CN112202860B - Container flow adjustment method, device and system - Google Patents
Container flow adjustment method, device and system Download PDFInfo
- Publication number
- CN112202860B CN112202860B CN202011002099.0A CN202011002099A CN112202860B CN 112202860 B CN112202860 B CN 112202860B CN 202011002099 A CN202011002099 A CN 202011002099A CN 112202860 B CN112202860 B CN 112202860B
- Authority
- CN
- China
- Prior art keywords
- container
- group
- groups
- flow
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/12—Avoiding congestion; Recovering from congestion
- H04L47/125—Avoiding congestion; Recovering from congestion by balancing the load, e.g. traffic engineering
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a container flow adjusting method, a device, computer equipment and a storage medium, belonging to the technical field of cluster management, wherein the method comprises the following steps: determining a target node in an overload state currently in a plurality of nodes of the cluster; dividing all container groups into two groups according to the resource utilization rate of each container group running on a target node so as to minimize the difference value of the sum of the resource utilization rates of the two groups, wherein the number of container groups of a first group containing the container group with the highest resource utilization rate is not more than that of the container groups of a second group; the invention can reduce the load of the containers on the nodes by grouping the containers on the hot spot and adjusting the flow of the containers of the second group, thereby achieving the result of reducing the use of the container resources and finally achieving the purpose of avoiding the node failure.
Description
Technical Field
The present invention relates to the field of cluster management technologies, and in particular, to a method and apparatus for adjusting container flow, a computer device, and a storage medium.
Background
Kubernetes cluster (hereinafter K8s cluster) implements a specification of a complete set of cluster orchestrations based on container technology. Pod is an application load in a Kubernetes cluster, pod runs on Node nodes, pod is composed of one or more containers (e.g., container containers created by the Docker Container engine) that share Container storage, network and Container running configuration items. Node is a computing Node in K8s cluster, is used for bearing the operation of allocated Pod, and is a host computer of Pod operation. The operation of the container depends on different resources on each Node, and one K8s cluster comprises a plurality of Node nodes.
In the running process of the K8s cluster, the situation that the resource distribution estimation is wrong due to application or service change is avoided, so that hot spots are generated due to the fact that partial host machine loads are too high, the host machine hot spots can not normally meet the service use requirement due to insufficient machine resources, the problems of overtime of an application request and the like are caused, the user request fails slightly, the application crashes due to heavy user request, and the service is not available.
At present, in response to a host hotspot, a conventional manner can select a hard power and an equal-minute-level HPA (horizontal capacity expansion tool) to load a high-load application in a balanced manner by increasing the number of container instances, but because the HPA is minute-level, the creation of a container to be capable of providing service often needs a few minutes, the response is relatively slow, and meanwhile, the situation of loading a large amount of data and the like can be easily encountered, so that the load of a host is increased.
Disclosure of Invention
In order to solve the above-mentioned problems in the background art, the present invention provides a method, an apparatus, a computer device and a storage medium for adjusting a container flow.
In a first aspect, there is provided a method of regulating a flow of a container, the method comprising:
determining a target node in an overload state currently in a plurality of nodes of the cluster;
dividing all the container groups into two groups according to the resource utilization rate of each container group running on the target node so as to minimize the difference value of the sum of the resource utilization rates of the two groups, wherein the number of the container groups of the first group containing the container group with the highest resource utilization rate is not more than that of the container groups of the second group;
and adjusting the flow of each container in the second packet to the node in the normal state.
Further, the determining, from a plurality of nodes included in the cluster, the target node currently in the overload state includes:
acquiring the current resource utilization rate of each node in the cluster;
the target node in the overload state is determined among a plurality of nodes of the cluster.
Further, the grouping of all the container groups into two groups according to the resource usage of each container group running on the target node includes:
sequencing all the container groups on the target node according to the sequence of the resource utilization rate from high to low;
and sequentially traversing all the sequenced container groups, and dividing the currently traversed container group into one group with the smallest sum of the current resource utilization rates in the two groups when traversing to one container group, until finishing the grouping of the last container group.
Further, adjusting the traffic of each container in the second packet to a node in a normal state includes:
obtaining a maximum container flow value of each container in the second group from a preset container configuration information table;
and adjusting the acquired maximum container flow value of each container according to a preset proportion, determining the flow to be scheduled, and scheduling the flow to be scheduled to the container of the node in a normal state.
Further, after the step of determining the target node currently in the overload state from the plurality of nodes in the cluster, the method further includes:
and marking each container on the target node with a forbidden state to forbid adjustment of the resources of each container.
Further, the method further comprises:
when the target node is recovered from the overload state to the normal state, adjusting the flow of each container on the second packet to the maximum flow;
and releasing the forbidden state of each container mark on the target node, and correspondingly increasing or decreasing the current resource upper limit value of the container when the container with the forbidden state released needs to be expanded or contracted, wherein the current resource upper limit value is obtained after expanding and contracted the container resource last time.
In a second aspect, there is provided a container flow adjustment device, the device comprising:
the node determining module is used for determining a target node in an overload state currently among a plurality of nodes of the cluster;
a container grouping module, configured to divide all the container groups into two groups according to the resource usage rates of the respective container groups running on the target node, so that a difference value of a sum of the resource usage rates of the two groups is minimum, and the number of container groups of a first group including a container group with a highest resource usage rate does not exceed the number of container groups of a second group;
and the flow adjustment module is used for adjusting the flow of each container in the second packet to the node in the normal state.
Further, the container grouping module is specifically configured to:
sequencing all the container groups on the target node according to the sequence of the resource utilization rate from high to low;
and sequentially traversing all the sequenced container groups, and dividing the currently traversed container group into one group with the smallest sum of the current resource utilization rates in the two groups when traversing to one container group, until finishing the grouping of the last container group.
Further, the flow adjustment module is specifically configured to:
obtaining a maximum container flow value of each container in the second group from a preset container configuration information table;
and adjusting the acquired maximum container flow value of each container according to a preset proportion, determining the flow to be scheduled, and scheduling the flow to be scheduled to the container of the node in a normal state.
Further, the apparatus further comprises:
and the state marking module is used for marking a forbidden state for each container on the target node so as to forbid the adjustment of the resources of each container.
Further, the apparatus further comprises:
a flow recovery module, configured to recover, when the target node recovers from the overload state to the normal state, the flow of each container on the second packet to a maximum flow;
a state releasing module, configured to release a forbidden state of each container label on the target node;
and the resource adjusting module is used for correspondingly adjusting the current resource upper limit value of the container to be higher or lower when the container after the forbidden state is released needs to be expanded or contracted, wherein the current resource upper limit value is obtained after the container resource is expanded or contracted last time.
In a third aspect, there is provided a computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of when executing the computer program:
determining a target node in an overload state currently in a plurality of nodes of the cluster;
dividing all the container groups into two groups according to the resource utilization rate of each container group running on the target node so as to minimize the difference value of the sum of the resource utilization rates of the two groups, wherein the number of the container groups of the first group containing the container group with the highest resource utilization rate is not more than that of the container groups of the second group;
and adjusting the flow of each container in the second packet to the node in the normal state.
In a fourth aspect, there is provided a computer readable storage medium storing a computer program which when executed by a processor performs the steps of:
determining a target node in an overload state currently in a plurality of nodes of the cluster;
dividing all the container groups into two groups according to the resource utilization rate of each container group running on the target node so as to minimize the difference value of the sum of the resource utilization rates of the two groups, wherein the number of the container groups of the first group containing the container group with the highest resource utilization rate is not more than that of the container groups of the second group;
and adjusting the flow of each container in the second packet to the node in the normal state.
The invention provides a container flow adjusting method, a device, computer equipment and a storage medium, wherein when all container components on a target node currently in an overload state are divided into two groups, the difference of the sum of the resource utilization rates of the two groups is limited to be minimum, and the number of container groups of a first group containing the container group with the highest resource utilization rate is not more than that of the container groups of a second group.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow diagram of a method of regulating a container flow in one embodiment;
FIG. 2 is a schematic diagram of a grouping of containers in one embodiment;
FIG. 3 is a schematic view of a structure of a container flow rate adjusting device in one embodiment;
FIG. 4 is a schematic diagram of a computer device in one embodiment.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It is noted that, unless the context clearly requires otherwise, the words "comprise," "comprising," and the like throughout the specification and the claims should be construed in an inclusive sense rather than an exclusive or exhaustive sense; that is, it is the meaning of "including but not limited to".
Furthermore, in the description of the present invention, it should be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Furthermore, in the description of the present invention, unless otherwise indicated, the meaning of "a plurality" is two or more.
As described in the background, in response to the host hotspots in the K8s cluster, the conventional manner generally selects a hard reactance plus an equal-minute HPA (horizontal expansion tool) to load the application with high load in an even way by increasing the number of container instances, but since the HPA is in a minute level, the creation of the container often needs several minutes to be able to provide services, the response is relatively slow, and meanwhile, the situation of loading a large amount of data and the like is also easily encountered, which results in increasing the load of the host. Therefore, the embodiment of the invention provides a container flow adjusting method, which can avoid blindly adjusting the flow of all container groups on a target node, so as to avoid the condition that the flow of the container group with high resource utilization rate is scheduled to other nodes to possibly cause the overload of other nodes, and simultaneously can also quickly reduce the container resource utilization of the target node, thereby achieving the purpose of effectively avoiding node faults.
In one embodiment, a method of regulating a flow of a container is provided, as shown with reference to fig. 1, which may include the steps of:
step S1, determining a target node in an overload state currently among a plurality of nodes of the cluster.
Wherein the clusters are K8s clusters, the K8s clusters comprise a plurality of host nodes, each host node runs a plurality of Pod, each Pod (i.e. container group) is created by one or more containers through a Docker container engine, each container running on one host node depends on resources such as a CPU, a memory, a disk, a GPU and the like on the host node, each container can realize isolation of resource usage based on a cgroup (control group) subsystem of linux, and each subsystem of cgroup has own individual resource usage statistics.
In one example, the implementation procedure of the step S1 may include:
the method comprises the steps of obtaining the current resource utilization rate of each node in a cluster, comparing the current resource utilization rate of each node with a preset utilization rate threshold value, and determining a target node in an overload state currently in a plurality of nodes of the cluster according to comparison results.
The states of the nodes can be divided into a plurality of states according to the resource utilization rate of the nodes, including: the node is in an overload state when the resource utilization rate of the node exceeds a preset utilization rate threshold value; when the resource utilization rate of the node is not higher than a preset utilization rate threshold value and not lower than an alert threshold value, the node is in an alert state; when the resource utilization rate of the node is lower than the guard threshold, the node is in a normal state. Wherein, each threshold value can be set according to an actual experience value, for example, when the current resource utilization rate of a node exceeds 80%, the node is in an overload state (the node can also be called as a 'hot spot'); when the current resource utilization rate of the node is between 60% and 80%, the node is in an alert state, and when the current resource utilization rate of the node is lower than 60%, the node is in a normal state.
Specifically, the resource utilization rate at the current time point calculated by the cgroup at the uppermost layer can be read in a preset time period (for example, every 5 s), and the current resource utilization rate of each node summarizes the utilization rate of the resources commonly used by the node and all containers running on the node, so that the self-summarizing calculation is not needed. The current resource utilization rate of each node can be acquired through a linux/proc system, so that second-level performance index data can be acquired.
And S2, dividing all the container groups into two groups according to the resource utilization rate of each container group running on the target node, so that the difference value of the sum of the resource utilization rates of the two groups is minimum, and the number of the container groups of the first group containing the container group with the highest resource utilization rate is not more than that of the container groups of the second group.
Wherein the resource usage of each container group may be obtained by summing the current resource usage of one or more containers contained in the container group.
Wherein, the current resource utilization rate of each container can be obtained by the following way, comprising:
and acquiring the accumulated resource usage of the target container for the resources in a preset time period, and calculating the current resource usage of the target container by combining the upper limit value of the resources of the current time of the target container. Preferably, when the current CPU resource usage of the host node is lower than the preset alert threshold, the current CPU resource usage of the target container is obtained.
Taking a CPU subsystem as an example, the CPU file in each container group stores the accumulated CPU time of each container in the whole group, the CPU file in the container group can be periodically read in a cgroupfs mode to obtain the accumulated resource usage of each container to the CPU in a preset time period, and then the CPU resource usage of each container in the current time can be calculated by combining the resource upper limit value in the CPU_quota_us of each container in the current time. An example formula is as follows:
wherein, cpu use (t) is the current resource utilization of a target container, cpu use t2 The resource usage amount of the target container in the cpu_user file read at the time point t2 is cpu_user t1 And (3) taking the resource usage of the target container in the CPU.use file read at the time point t1, wherein CPU.cfs_quota_us is the upper limit value of the resource of the current time CPU of the target container.
Wherein the two groups include a first group and a second group, the difference between the sum of the resource usage rates of the container groups of the first group and the second group is the smallest, the first group includes the container group with the highest resource usage rate, and the number of the container groups of the first group does not exceed the number of the container groups of the second group, so that the sum of the resource usage rates of the first group and the second group is approximately the same.
In one example, the implementation procedure of the step S2 may include:
sequencing all container groups on the target node according to the sequence of the resource utilization rate from high to low;
and sequentially traversing all the sequenced container groups, and dividing the currently traversed container group into one group with the smallest sum of current resource utilization rates in the two groups when traversing to one container group, until finishing the grouping of the last container group.
To facilitate understanding of the grouping process, 5 container groups A-E are illustrated.
Firstly, 5 container groups are ordered according to the order of the resource utilization rate from high to low, and the ordering result is [ A, D, C, B and E ], and the corresponding resource utilization rate is [60%,40%,38%,35% and 30% ].
And then sequentially traversing the sequencing results [ A, D, C, B, E ], wherein the first packet and the second packet are empty before the traversing operation. The traversed first container group is A, and the A is divided into a first group; the traversed second container group is D, and D is divided into the second packets at the moment because the second packets are empty; the third container group traversed is C, and C is divided into the second group because the sum (40%) of the current resource utilization rates of the second group is minimum; the fourth traversed container group is B, and at the moment, B is divided into the first grouping because the sum (60%) of the current resource utilization rates of the first grouping is minimum; the fourth container group traversed is E, and E is divided into a second group because the sum (78%) of the current resource utilization rates of the second group is minimum, so that the first group obtained by dividing comprises a container group A and a container group B, and the second group comprises a container group C, a container group D and a container group E.
And step S3, adjusting the flow of all containers in the second packet to the node in the normal state.
The flow of all containers in the second packet can be scheduled to the node in the normal state according to the preset proportion.
The preset proportion can be set according to actual needs, and the same preset proportion can be set for all containers, for example, 50%, namely, half of the flow of all containers in the second packet is adjusted to the node with the lowest resource utilization rate. In addition, different preset proportions can be set according to preset priorities of the containers, which is not particularly limited in the embodiment of the present invention.
In one example, the implementation procedure of the step S3 may include:
obtaining a maximum container flow value of each container in the second group from a preset container configuration information table; and adjusting the acquired maximum container flow value of each container according to a preset proportion, determining the flow to be scheduled, and scheduling the flow to be scheduled to the container of the node in a normal state.
Each container running on each node in the cluster is configured with a corresponding maximum container flow value in the container configuration information table in advance, wherein the maximum container flow value is used for limiting the maximum flow of the container, namely the maximum value of the flow which can be provided by the container in unit time.
When the traffic to be scheduled is scheduled to the container of the node in the normal state, the resource utilization rate of at least one node in the normal state in the cluster can be obtained, and the traffic of all the containers in the second packet is scheduled to the node with the lowest resource utilization rate in the at least one node according to the preset proportion.
In this embodiment, the flow of all the containers in the second packet is scheduled to the node in the normal state according to the preset proportion, so that the flow control on the containers in the second packet is implemented, so that the stability and usability of the container service can be ensured, and the problem that the container resource is used in a limited manner due to the fact that the container is further used in a resource limitation reducing manner is effectively avoided, so that faults such as request timeout and the like occur more quickly; in addition, the flow among the nodes in the cluster can be effectively balanced, and the utilization rate of each node is improved.
In one embodiment, the method may further comprise:
when the target node is recovered to a normal state from an overload state, adjusting the flow of each container on the second packet to the maximum flow;
and releasing the forbidden state of each container mark on the target node, and correspondingly increasing or decreasing the current resource upper limit value of the container when the container with the forbidden state released needs to be expanded or contracted, wherein the current resource upper limit value is obtained after expanding and contracted the container resource last time.
When the container needs to be expanded or contracted, the current resource upper limit value of the container is correspondingly adjusted up or down, and the process can comprise the following steps:
obtaining a target utilization rate and a single resource adjustment allowable range which are preconfigured by a container from a preset resource configuration information table;
calculating the product of the ratio of the current resource utilization rate of the container to the target utilization rate and the current resource upper limit value of the container to serve as the target resource upper limit value of the container;
determining the latest resource upper limit value corresponding to the container based on the single resource adjustment allowable range and the target resource upper limit value of the container;
the current resource upper limit of the container is adjusted to the latest resource upper limit.
The single resource adjustment allowable range includes a single resource expansion allowable range, and determining, based on the single resource adjustment allowable range and a target resource upper limit value of the container, a latest resource upper limit value corresponding to the container includes:
when the container needs to be expanded, taking the maximum value from the upper limit value of the target resource of the container and the lower limit of the single resource expansion allowable range, and taking the minimum value from the maximum value and the upper limit of the single resource expansion allowable range as the latest resource upper limit value;
the single resource adjustment allowable range includes a single resource contraction allowable range, and determining the latest resource upper limit value corresponding to the container based on the single resource adjustment allowable range and the target resource upper limit value of the container includes:
when the container needs to shrink, the maximum value is taken from the upper limit value of the target resource of the container and the lower limit of the allowable range of the single resource shrink, and the minimum value is taken from the maximum value and the upper limit of the allowable range of the single resource shrink as the latest upper limit value of the resource.
In this embodiment, when the container after the forbidden state is released needs to be expanded or contracted, the current resource upper limit value of the container is correspondingly adjusted to be high or low.
In one embodiment, there is provided a container flow adjustment device, as shown with reference to fig. 3, comprising:
a node determining module 31, configured to determine a target node currently in an overload state from a plurality of nodes in the cluster;
a container grouping module 32, configured to group all container groups into two groups according to the resource usage rates of the respective container groups running on the target node, so that the difference between the sum of the resource usage rates of the two groups is minimized, and the number of container groups of the first group including the container group with the highest resource usage rate does not exceed the number of container groups of the second group;
the flow adjustment module 33 is configured to adjust the flow of each container in the second packet to the node in the normal state.
In one embodiment, the container grouping module 31 is specifically configured to:
sequencing all container groups on the target node according to the sequence of the resource utilization rate from high to low;
and sequentially traversing all the sequenced container groups, and dividing the currently traversed container group into one group with the smallest sum of current resource utilization rates in the two groups when traversing to one container group, until finishing the grouping of the last container group.
In one embodiment, the flow adjustment module 33 is specifically configured to:
obtaining a maximum container flow value of each container in the second group from a preset container configuration information table;
and adjusting the acquired maximum container flow value of each container according to a preset proportion, determining the flow to be scheduled, and scheduling the flow to be scheduled to the container of the node in a normal state.
In one embodiment, the apparatus further comprises:
and the state marking module is used for marking a forbidden state for each container on the target node so as to forbid the adjustment of the resources of each container.
In one embodiment, the apparatus further comprises:
the flow recovery module is used for recovering the flow of each container on the second packet to the maximum flow when the target node is recovered from the overload state to the normal state;
a state releasing module, configured to release the forbidden state of each container label on the target node;
and the resource adjusting module is used for correspondingly adjusting the current resource upper limit value of the container to be higher or lower when the container after the forbidden state is released needs to be expanded or contracted, wherein the current resource upper limit value is obtained after the container resource is expanded or contracted last time.
In one embodiment, when the container needs to be expanded or contracted, the resource adjustment module is specifically configured to:
obtaining a target utilization rate and a single resource adjustment allowable range which are preconfigured by a container from a preset resource configuration information table;
calculating the product of the ratio of the current resource utilization rate of the container to the target utilization rate and the current resource upper limit value of the container to serve as the target resource upper limit value of the container;
determining the latest resource upper limit value corresponding to the container based on the single resource adjustment allowable range and the target resource upper limit value of the container;
the current resource upper limit of the container is adjusted to the latest resource upper limit.
The single resource adjustment allowable range includes a single resource capacity expansion allowable range, and the resource adjustment module is specifically configured to:
when the container needs to be expanded, taking the maximum value from the upper limit value of the target resource of the container and the lower limit of the single resource expansion allowable range, and taking the minimum value from the maximum value and the upper limit of the single resource expansion allowable range as the latest resource upper limit value;
the single resource adjustment allowable range includes a single resource capacity reduction allowable range, and the resource adjustment module is specifically further configured to:
when the container needs to shrink, the maximum value is taken from the upper limit value of the target resource of the container and the lower limit of the allowable range of the single resource shrink, and the minimum value is taken from the maximum value and the upper limit of the allowable range of the single resource shrink as the latest upper limit value of the resource.
It should be noted that: in the container flow adjusting device provided in this embodiment, only the division of the above functional modules is used as an example, and in practical application, the above functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to perform all or part of the functions described above. In addition, the container flow adjusting device of the present embodiment belongs to the same concept as the container flow adjusting method embodiment in the foregoing embodiments, and specific implementation processes and beneficial effects thereof are detailed in the container resource adjusting method embodiment, which is not described herein again.
Fig. 4 is an internal structure diagram of a computer device according to an embodiment of the present invention. The computer device may be a server, the internal structure of which may be as shown in fig. 4. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a container flow adjustment method.
It will be appreciated by persons skilled in the art that the architecture shown in fig. 4 is merely a block diagram of some of the architecture relevant to the present inventive arrangements and is not limiting as to the computer device to which the present inventive arrangements are applicable, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In one embodiment, there is also provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the computer program to perform the steps of:
determining a target node in an overload state currently in a plurality of nodes of the cluster;
dividing all container groups into two groups according to the resource utilization rate of each container group running on a target node so as to minimize the difference value of the sum of the resource utilization rates of the two groups, wherein the number of container groups of a first group containing the container group with the highest resource utilization rate is not more than that of the container groups of a second group;
the traffic of each container in the second packet is regulated to the node in the normal state.
In one embodiment, there is also provided a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
determining a target node in an overload state currently in a plurality of nodes of the cluster;
dividing all container groups into two groups according to the resource utilization rate of each container group running on a target node so as to minimize the difference value of the sum of the resource utilization rates of the two groups, wherein the number of container groups of a first group containing the container group with the highest resource utilization rate is not more than that of the container groups of a second group;
the traffic of each container in the second packet is regulated to the node in the normal state.
Those skilled in the art will appreciate that implementing all or part of the above-described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples illustrate only a few embodiments of the invention, which are described in detail and are not to be construed as limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.
Claims (10)
1. A method of regulating a flow of a container, the method comprising:
determining a target node in an overload state currently in a plurality of nodes of the cluster;
dividing all container groups running on the target node into two groups according to the resource utilization rate of each container group running on the target node, so that the difference value of the sum of the resource utilization rates of the two groups is minimum, and the number of container groups of a first group containing the container group with the highest resource utilization rate does not exceed the number of container groups of a second group;
adjusting the traffic of each container in the second packet to a node in a normal state;
wherein the resource usage of each container group is obtained by summing the current resource usage of one or more containers contained in the container group.
2. The method of claim 1, wherein the grouping all container groups running on the target node into two groups according to resource usage of respective container groups running on the target node comprises:
sequencing all the container groups on the target node according to the sequence of the resource utilization rate from high to low;
and sequentially traversing all the sequenced container groups, and dividing the currently traversed container group into one group with the smallest sum of the current resource utilization rates in the two groups when traversing to one container group, until finishing the grouping of the last container group.
3. The method of claim 1, wherein adjusting the traffic of each container within the second packet to the node in the normal state comprises:
obtaining a maximum container flow value of each container in the second group from a preset container configuration information table;
and adjusting the acquired maximum container flow value of each container according to a preset proportion, determining the flow to be scheduled, and scheduling the flow to be scheduled to the container of the node in a normal state.
4. A method according to any one of claims 1 to 3, wherein after the step of determining a target node currently in an overload state among a plurality of nodes of the cluster, the method further comprises:
and marking each container on the target node with a forbidden state to forbid adjustment of the resources of each container.
5. The method according to claim 4, wherein the method further comprises:
when the target node is recovered from the overload state to the normal state, adjusting the flow of each container on the second packet to the maximum flow;
and releasing the forbidden state of each container mark on the target node, and correspondingly increasing or decreasing the current resource upper limit value of the container when the container with the forbidden state released needs to be expanded or contracted, wherein the current resource upper limit value is obtained after expanding and contracted the container resource last time.
6. A container flow adjustment device, the device comprising:
the node determining module is used for determining a target node in an overload state currently among a plurality of nodes of the cluster;
a container grouping module, configured to divide all container groups running on the target node into two groups according to resource usage rates of respective container groups running on the target node, so that a difference value of a sum of the resource usage rates of the two groups is minimum, and a number of container groups of a first group including a container group with a highest resource usage rate does not exceed a number of container groups of a second group;
a flow adjustment module for adjusting the flow of each container in the second packet to a node in a normal state;
wherein the resource usage of each container group is obtained by summing the current resource usage of one or more containers contained in the container group.
7. The apparatus of claim 6, wherein the container grouping module is specifically configured to:
sequencing all the container groups on the target node according to the sequence of the resource utilization rate from high to low;
and sequentially traversing all the sequenced container groups, and dividing the currently traversed container group into one group with the smallest sum of the current resource utilization rates in the two groups when traversing to one container group, until finishing the grouping of the last container group.
8. The apparatus of claim 6, wherein the flow adjustment module is specifically configured to:
obtaining a maximum container flow value of each container in the second group from a preset container configuration information table;
and adjusting the acquired maximum container flow value of each container according to a preset proportion, determining the flow to be scheduled, and scheduling the flow to be scheduled to the container of the node in a normal state.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the container flow adjustment method of any of claims 1 to 5 when executing the computer program.
10. A computer readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the container flow adjustment method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011002099.0A CN112202860B (en) | 2020-09-22 | 2020-09-22 | Container flow adjustment method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011002099.0A CN112202860B (en) | 2020-09-22 | 2020-09-22 | Container flow adjustment method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112202860A CN112202860A (en) | 2021-01-08 |
| CN112202860B true CN112202860B (en) | 2023-07-14 |
Family
ID=74015862
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011002099.0A Active CN112202860B (en) | 2020-09-22 | 2020-09-22 | Container flow adjustment method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112202860B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108228354A (en) * | 2017-12-29 | 2018-06-29 | 杭州朗和科技有限公司 | Dispatching method, system, computer equipment and medium |
| CN110134497A (en) * | 2019-06-21 | 2019-08-16 | 深圳市比一比网络科技有限公司 | A kind of integrated dispositions method and its system applied to container technique |
| CN111104227A (en) * | 2019-12-28 | 2020-05-05 | 北京浪潮数据技术有限公司 | Resource control method and device of K8s platform and related components |
| CN111522636A (en) * | 2020-04-03 | 2020-08-11 | 安超云软件有限公司 | Application container adjusting method, application container adjusting system, computer readable medium and terminal device |
| CN111666131A (en) * | 2020-07-03 | 2020-09-15 | 腾讯科技（深圳）有限公司 | Load balancing distribution method and device, computer equipment and storage medium |
-
2020
- 2020-09-22 CN CN202011002099.0A patent/CN112202860B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108228354A (en) * | 2017-12-29 | 2018-06-29 | 杭州朗和科技有限公司 | Dispatching method, system, computer equipment and medium |
| CN110134497A (en) * | 2019-06-21 | 2019-08-16 | 深圳市比一比网络科技有限公司 | A kind of integrated dispositions method and its system applied to container technique |
| CN111104227A (en) * | 2019-12-28 | 2020-05-05 | 北京浪潮数据技术有限公司 | Resource control method and device of K8s platform and related components |
| CN111522636A (en) * | 2020-04-03 | 2020-08-11 | 安超云软件有限公司 | Application container adjusting method, application container adjusting system, computer readable medium and terminal device |
| CN111666131A (en) * | 2020-07-03 | 2020-09-15 | 腾讯科技（深圳）有限公司 | Load balancing distribution method and device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112202860A (en) | 2021-01-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112181649B (en) | Container resource adjustment method and device, computer equipment and storage medium | |
| CN109194584B (en) | Flow monitoring method and device, computer equipment and storage medium | |
| CN111818159B (en) | Management method, device, equipment and storage medium of data processing node | |
| US9319281B2 (en) | Resource management method, resource management device, and program product | |
| CN108881512B (en) | CTDB virtual IP balance distribution method, device, equipment and medium | |
| CN112269641B (en) | Scheduling method, scheduling device, electronic equipment and storage medium | |
| US20130232504A1 (en) | Method and apparatus for managing processing resources in a distributed processing system | |
| CN106959820B (en) | Data extraction method and system | |
| CN112597202B (en) | Data query method and device, computer equipment and storage medium | |
| EP3843451B1 (en) | Method and apparatus for adjusting bandwidth, communication device and computer readable storage medium | |
| CN113655969B (en) | Data balanced storage method based on streaming distributed storage system | |
| CN108540315A (en) | Distributed memory system, method and apparatus | |
| CN113132176A (en) | Method for controlling edge node, node and edge computing system | |
| CN113821340A (en) | Dynamic balancing method, system, terminal and storage medium of distributed system | |
| CN112202860B (en) | Container flow adjustment method, device and system | |
| CN108459926B (en) | Data remote backup method and device and computer readable medium | |
| CN112667467A (en) | Cluster self-adaptive current limiting method and device, computer equipment and storage medium | |
| CN116643858A (en) | Service priority pod-based rescheduling method, device, equipment and medium | |
| CN116244061A (en) | Pod scheduling method, storage medium and device based on k8s | |
| CN115203177A (en) | Distributed data storage system and storage method | |
| CN117527591A (en) | Resource elastic expansion method, device and equipment | |
| CN113258679A (en) | Power grid monitoring system channel distribution method based on server instance capacity reduction | |
| Yokouchi et al. | Multiple-Backup Resource Allocation for Virtual Machines with Probabilistic Protection in Cloud | |
| CN111897656A (en) | Main queue load balancing method based on mirror image queue | |
| CN116112499B (en) | Construction method of data acquisition system and data acquisition method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |