CN112202860A - Container flow adjusting method, device and system - Google Patents

Container flow adjusting method, device and system Download PDF

Info

Publication number
CN112202860A
CN112202860A CN202011002099.0A CN202011002099A CN112202860A CN 112202860 A CN112202860 A CN 112202860A CN 202011002099 A CN202011002099 A CN 202011002099A CN 112202860 A CN112202860 A CN 112202860A
Authority
CN
China
Prior art keywords
container
groups
group
node
flow
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011002099.0A
Other languages
Chinese (zh)
Other versions
CN112202860B (en
Inventor
梁鸿斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Pinwei Software Co Ltd
Original Assignee
Guangzhou Pinwei Software Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Pinwei Software Co Ltd filed Critical Guangzhou Pinwei Software Co Ltd
Priority to CN202011002099.0A priority Critical patent/CN112202860B/en
Publication of CN112202860A publication Critical patent/CN112202860A/en
Application granted granted Critical
Publication of CN112202860B publication Critical patent/CN112202860B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083—Techniques for rebalancing the load in a distributed system
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/12—Avoiding congestion; Recovering from congestion
    • H04L47/125—Avoiding congestion; Recovering from congestion by balancing the load, e.g. traffic engineering
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention discloses a container flow adjusting method, a container flow adjusting device, computer equipment and a storage medium, belonging to the technical field of cluster management, wherein the method comprises the following steps: determining a target node in an overload state currently in a plurality of nodes of a cluster; dividing all container groups into two groups according to the resource utilization rate of each container group operated on a target node, so that the difference of the sum of the resource utilization rates of the two groups is minimum, and the number of the container groups of a first group containing the container group with the highest resource utilization rate is not more than the number of the container groups of a second group; the invention can reduce the load of the containers on the nodes by grouping the containers on the hot spots and adjusting the flow of the containers of the second group, thereby achieving the result of reducing the use of the container resources and finally achieving the purpose of avoiding the node failure.

Description

Container flow adjusting method, device and system
Technical Field
The present invention relates to the field of cluster management technologies, and in particular, to a method and an apparatus for adjusting container traffic, a computer device, and a storage medium.
Background
The Kubernetes cluster (hereinafter referred to as K8s cluster) implements a whole set of clustering specifications based on container technology. A Pod is an application load in a Kubernetes cluster, the Pod running on a Node, the Pod consisting of one or more containers (e.g., Container containers created by a Docker Container engine), which share Container storage, network, and Container running configuration items. Node is a computing Node in the K8s cluster, and is used to carry the operation of the allocated Pod, and is the host of the Pod operation. The operation of the container depends on different resources on each Node, and one K8s cluster comprises a plurality of Node nodes.
In the operation process of the K8s cluster, it is inevitable that a resource distribution estimation error occurs due to application or service change, and then a hot spot is generated due to too high load of part of host machines, the host machine hot spot may not normally meet the service use requirement due to insufficient machine resources, and thus the problems of application request timeout and the like may be caused, and if the user request fails, the application is crashed, and the service is unavailable.
At present, for host hotspots, a conventional method can select hard reactance plus an HPA (horizontal capacity expansion tool) of an upper minute level to balance load of a high-load application by increasing the number of container instances, but because the HPA is of a minute level, a container often takes several minutes to create to provide service, response is slow, and meanwhile, situations such as loading of a large amount of data are easily encountered, so that load of a host is increased.
Disclosure of Invention
In order to solve the problems mentioned in the background art, the invention provides a container flow adjusting method, a container flow adjusting device, a computer device and a storage medium.
In a first aspect, a method for regulating container flow is provided, the method comprising:
determining a target node in an overload state currently in a plurality of nodes of a cluster;
dividing all the container groups into two groups according to the resource utilization rate of each container group running on the target node, so that the difference of the sum of the resource utilization rates of the two groups is minimum, and the number of the container groups of a first group containing the container group with the highest resource utilization rate does not exceed the number of the container groups of a second group;
and adjusting the flow of each container in the second packet to the node in the normal state.
Further, the determining a target node currently in an overload state from a plurality of nodes included in the cluster includes:
acquiring the current resource utilization rate of each node in the cluster;
and determining a target node which is currently in an overload state in a plurality of nodes of the cluster.
Further, the grouping all the container groups into two groups according to the resource utilization of each container group operating on the target node includes:
sorting all the container groups on the target node according to the sequence of the resource utilization rate from high to low;
and sequentially traversing all the sorted container groups, and dividing the currently traversed container group into one of the two groups with the smallest sum of the current resource utilization rates when traversing to one container group until finishing the grouping of the last container group.
Further, adjusting the traffic of each container in the second packet to a node in a normal state includes:
acquiring the maximum container flow value of each container in the second grouping from a preset container configuration information table;
and adjusting the acquired maximum container flow value of each container according to a preset proportion, determining the flow to be scheduled, and scheduling the flow to be scheduled to the container of the node in a normal state.
Further, after the step of determining a target node currently in an overload state among the plurality of nodes in the cluster, the method further includes:
marking each container on the target node with a disabled state to disable adjusting resources of each of the containers.
Further, the method further comprises:
when the target node is recovered to a normal state from an overload state, adjusting the traffic of each container on the second packet to the maximum traffic;
and releasing the forbidden state of each container mark on the target node, and correspondingly increasing or decreasing the current resource upper limit value of the container when the container after the forbidden state is released needs to be expanded or contracted, wherein the current resource upper limit value is obtained by expanding and contracting the resource of the container last time.
In a second aspect, there is provided a container flow regulating device, the device comprising:
the node determining module is used for determining a target node which is currently in an overload state from a plurality of nodes of the cluster;
a container grouping module, configured to group all the container groups into two groups according to resource usage rates of the container groups running on the target node, so that a difference between a sum of the resource usage rates of the two groups is minimized, and a number of container groups of a first group including a container group with a highest resource usage rate does not exceed a number of container groups of a second group;
and the flow adjusting module is used for adjusting the flow of each container in the second grouping to the node in the normal state.
Further, the container grouping module is specifically configured to:
sorting all the container groups on the target node according to the sequence of the resource utilization rate from high to low;
and sequentially traversing all the sorted container groups, and dividing the currently traversed container group into one of the two groups with the smallest sum of the current resource utilization rates when traversing to one container group until finishing the grouping of the last container group.
Further, the flow rate adjustment module is specifically configured to:
acquiring the maximum container flow value of each container in the second grouping from a preset container configuration information table;
and adjusting the acquired maximum container flow value of each container according to a preset proportion, determining the flow to be scheduled, and scheduling the flow to be scheduled to the container of the node in a normal state.
Further, the apparatus further comprises:
and the state marking module is used for marking the forbidden state of each container on the target node so as to forbid the adjustment of the resource of each container.
Further, the apparatus further comprises:
the traffic recovery module is used for recovering the traffic of each container on the second packet to the maximum traffic when the target node is recovered to the normal state from the overload state;
the state releasing module is used for releasing the forbidden state of each container mark on the target node;
and the resource adjusting module is used for correspondingly increasing or decreasing the current resource upper limit value of the container when the container after the forbidden state is released needs to be expanded or contracted, wherein the current resource upper limit value is obtained by expanding and contracting the resource of the container last time.
In a third aspect, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the following steps when executing the computer program:
determining a target node in an overload state currently in a plurality of nodes of a cluster;
dividing all the container groups into two groups according to the resource utilization rate of each container group running on the target node, so that the difference of the sum of the resource utilization rates of the two groups is minimum, and the number of the container groups of a first group containing the container group with the highest resource utilization rate does not exceed the number of the container groups of a second group;
and adjusting the flow of each container in the second packet to the node in the normal state.
In a fourth aspect, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program which, when executed by a processor, performs the steps of:
determining a target node in an overload state currently in a plurality of nodes of a cluster;
dividing all the container groups into two groups according to the resource utilization rate of each container group running on the target node, so that the difference of the sum of the resource utilization rates of the two groups is minimum, and the number of the container groups of a first group containing the container group with the highest resource utilization rate does not exceed the number of the container groups of a second group;
and adjusting the flow of each container in the second packet to the node in the normal state.
The invention provides a container flow adjusting method, a device, computer equipment and a storage medium, since the difference of the sum of the resource utilization rates of the two groups is limited to be minimum when all the container groups on the target node which is currently in the overload state are divided into the two groups, and the number of container groups of the first group containing the container group with the highest resource utilization does not exceed the number of container groups of the second group, thus, the traffic of each container in the second grouping is scheduled to the node in the normal state, the blind adjustment of the traffic of all the container groups on the target node can be avoided, thereby avoiding the situation that the traffic of the container group with the too high resource utilization rate is scheduled to other nodes to cause the too high load of other nodes, meanwhile, the use of container resources of the target node can be rapidly reduced, and the purpose of effectively avoiding node faults is achieved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic flow chart of a method for regulating a flow rate of a container in one embodiment;
FIG. 2 is a schematic diagram of a grouping of containers in one embodiment;
FIG. 3 is a schematic diagram of a container flow control device in one embodiment;
FIG. 4 is a block diagram of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It is to be understood that, unless the context clearly requires otherwise, throughout the description and the claims, the words "comprise", "comprising", and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is, what is meant is "including, but not limited to".
Furthermore, in the description of the present invention, it is to be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In addition, in the description of the present invention, "a plurality" means two or more unless otherwise specified.
As described in the background, currently in response to a host hotspot in a K8s cluster, a hard reactance plus an HPA (horizontal capacity expansion tool) of an upper minute level is generally selected in a conventional manner to balance the load of a highly loaded application by increasing the number of container instances, but since the HPA is of a minute level, the container often takes several minutes to be created to provide service, the response is slow, and a situation of loading a large amount of data is easily encountered, which results in an increase in the load of the host. Therefore, embodiments of the present invention provide a container traffic adjustment method, which can avoid blindly adjusting traffic of all container groups on a target node, so as to avoid a situation that traffic of a container group with an excessively high resource utilization rate is scheduled to other nodes, which may cause an excessively high load on other nodes, and at the same time, can quickly reduce container resource usage of the target node, thereby achieving an objective of effectively avoiding node failure.
In one embodiment, a method for regulating a flow rate of a container is provided, and as shown in fig. 1, the method may include the steps of:
in step S1, a target node currently in an overload state is determined among the plurality of nodes in the cluster.
The cluster is a K8s cluster, the K8s cluster includes a plurality of host nodes, each host node runs a plurality of Pod(s), each Pod (i.e., container group) is composed of one or more containers created by a Docker container engine, each container running on a host node depends on resources on the host node, such as CPU, memory, disk, GPU, and the like, each container can implement resource usage isolation based on a c group (control group) subsystem of linux, and each subsystem of the c group has its own individual resource usage statistical data.
In one example, the implementation process of step S1 may include:
the method comprises the steps of obtaining the current resource utilization rate of each node in a cluster, comparing the current resource utilization rate of each node with a preset utilization rate threshold value, and determining a target node which is in an overload state currently in a plurality of nodes of the cluster according to a comparison result.
The state of the node can be divided into a plurality of states according to the resource utilization rate of the node, and the method comprises the following steps: the node comprises a normal state, an alert state and an overload state, wherein when the resource utilization rate of the node exceeds a preset utilization rate threshold value, the node is in the overload state; when the resource utilization rate of the node is not higher than the preset utilization rate threshold and not lower than the alert threshold, the node is in an alert state; and when the resource utilization rate of the node is lower than the warning threshold value, the node is in a normal state. Each threshold may be set according to an actual experience value, for example, when the current resource utilization rate of a node exceeds 80%, the node is in an overload state (the node may also be referred to as a "hot spot"); when the current resource utilization rate of the node is between 60% and 80%, the node is in an alert state, and when the current resource utilization rate of the node is lower than 60%, the node is in a normal state.
Specifically, the resource usage at the current time point calculated by reading the cgroup at the uppermost layer at preset time periods (for example, every 5s) by setting a timing task, and the current resource usage of each node summarizes its own usage and the usage of resources commonly used by all containers running on the node, and does not need to perform the summary calculation by itself. The current resource utilization rate of each node can be acquired through a linux/proc system, so that the acquisition of second-level performance index data can be realized.
Step S2, according to the resource utilization of each container group running on the target node, grouping all the container groups into two groups so as to minimize the difference of the sum of the resource utilization of the two groups, and the number of container groups of the first group including the container group with the highest resource utilization does not exceed the number of container groups of the second group.
The resource utilization rate of each container group can be obtained by summing up the current resource utilization rates of one or more containers included in the container group.
The current resource utilization rate of each container can be obtained by the following method, including:
and acquiring the accumulated resource usage amount of the target container to the resources within a preset time period, and calculating the current resource usage rate of the target container by combining the resource upper limit value of the current time of the target container. Preferably, when the current CPU resource usage rate of the host node is lower than a preset alert threshold, the current CPU resource usage rate of the target container is obtained.
Taking a CPU subsystem as an example, a cpu.use file in each container group stores the time of the CPU used accumulatively by each container in the whole group, the cpu.use file in the container group can be periodically read in a cgroupfs manner to obtain the accumulated resource usage amount of each container to the CPU in a preset time period, and then, the CPU resource usage rate of each container at the current time can be calculated by combining the resource upper limit value in the cpu.cfs _ quota _ us of each container at the current time. An example formula is as follows:
Figure BDA0002694691600000081
wherein, cpu usage (t) is the current resource utilization of a certain target container, cpu acct2Use file read at time point t2, resource usage of the target container in the cpu accesst1The resource usage amount of the target container in the cpu access.use file read at the time point t1, and cpu.cfs _ quota _ us is the resource upper limit value of the cpu at the current time of the target container.
The two groups comprise a first group and a second group, the difference between the sum of the resource utilization rates of the container groups of the first group and the sum of the resource utilization rates of the container groups of the second group is minimum, the first group comprises the container groups with the highest resource utilization rate, and the number of the container groups of the first group does not exceed the number of the container groups of the second group, so that the sum of the resource utilization rates of the first group and the second group is approximately the same.
In one example, the implementation process of step S2 may include:
sorting all container groups on the target node in the order of the resource utilization rate from high to low;
and traversing all the sorted container groups in sequence, and dividing the currently traversed container group into one of the two groups with the smallest sum of the current resource utilization rates when traversing to one container group until completing the grouping of the last container group.
In order to facilitate understanding of the grouping process, 5 container groups a to E are exemplified.
Firstly, 5 container groups are sorted according to the sequence of the resource utilization rates from high to low, the sorting result is obtained as [ A, D, C, B and E ], and the corresponding resource utilization rates are [ 60%, 40%, 38%, 35% and 30% ].
And then sequentially traversing the sequencing results [ A, D, C, B and E ], wherein the first grouping and the second grouping are both empty before the traversing operation. The traversed first container group is A, and the A is divided into first groups; the traversed second container group is D, and the second grouping is null, so that the D is divided into the second grouping; the third traversed container group is C, and at the moment, the C is divided into second groups due to the fact that the sum (40%) of the current resource utilization rates of the second groups is the minimum; the traversed fourth container group is B, and at this time, B is divided into the first group due to the fact that the sum (60%) of the current resource utilization rates of the first group is the minimum; the fourth traversed container group is E, at this time, since the sum (78%) of the current resource utilization rates of the second packet is minimum, E is divided into the second packet, the first packet obtained by the division includes a container group a and a container group B, and the second packet includes a container group C, a container group D, and a container group E.
In step S3, the traffic of all containers in the second packet is adjusted to the nodes in the normal state.
And scheduling the traffic of all containers in the second packet to the nodes in the normal state according to a preset proportion.
The preset proportion may be set according to actual needs, and the same preset proportion may be set for all the containers, for example, 50% is set, that is, half of the traffic of all the containers in the second packet is adjusted to the node with the lowest resource utilization rate. In addition, different preset proportions may be set according to preset priorities of the containers, which is not specifically limited in the embodiment of the present invention.
In one example, the implementation process of step S3 may include:
acquiring the maximum container flow value of each container in the second grouping from a preset container configuration information table; and adjusting the acquired maximum container flow value of each container according to a preset proportion, determining the flow to be scheduled, and scheduling the flow to be scheduled to the container of the node in a normal state.
Each container running on each node in the cluster is configured with a corresponding maximum container flow value in the container configuration information table in advance, and the maximum container flow value is used for limiting the maximum flow of the container, namely the maximum value of the flow which can be provided by the container in unit time.
When traffic needing to be scheduled is scheduled to a container of a node in a normal state, the resource utilization rate of at least one node in the normal state in the cluster can be acquired, and the traffic of all containers in the second packet is scheduled to the node with the lowest resource utilization rate in the at least one node according to a preset proportion.
In this embodiment, the flow of all the containers in the second packet is scheduled to the node in the normal state according to the preset proportion, so that the flow control of the containers in the second packet is realized, the stability and the availability of the container service can be ensured, and the condition that the use of the container resources is limited due to the limitation of the containers to further use the resources in a manner of reducing the resource quota, so that the occurrence of failures such as request timeout and the like is caused more quickly, is effectively avoided; in addition, the flow among all nodes in the cluster can be effectively balanced, and the utilization rate of each node is improved.
In one embodiment, the method may further comprise:
when the target node is recovered to a normal state from the overload state, adjusting the traffic of each container on the second packet to the maximum traffic;
and releasing the forbidden state of each container mark on the target node, and correspondingly increasing or decreasing the current resource upper limit value of the container when the container after the forbidden state is released needs to be expanded or contracted, wherein the current resource upper limit value is obtained by expanding and contracting the resource of the container last time.
When the container needs to expand or contract, the current resource upper limit value of the container is correspondingly increased or decreased, and the process may include:
acquiring a target utilization rate and a single resource adjustment allowable range which are configured in advance by a container from a preset resource configuration information table;
calculating the product of the ratio of the current resource utilization rate of the container to the target utilization rate and the current resource upper limit value of the container to serve as the target resource upper limit value of the container;
determining the latest resource upper limit value corresponding to the container based on the single resource adjustment allowable range and the target resource upper limit value of the container;
and adjusting the current resource upper limit value of the container to the latest resource upper limit value.
Wherein, the single resource adjustment allowable range includes a single resource expansion allowable range, and the latest resource upper limit value corresponding to the container is determined based on the single resource adjustment allowable range and the target resource upper limit value of the container, including:
when the container needs to be expanded, taking the maximum value from the upper limit value of the target resource of the container and the lower limit of the single resource expansion allowable range, and taking the minimum value from the maximum value and the upper limit of the single resource expansion allowable range as the latest resource upper limit value;
the single resource adjustment allowable range comprises a single resource contraction capacity allowable range, and the latest resource upper limit value corresponding to the container is determined based on the single resource adjustment allowable range and the target resource upper limit value of the container, and the method comprises the following steps:
when the container needs to be scaled down, taking the maximum value from the upper limit value of the target resource of the container and the lower limit of the single resource scaling allowable range, and taking the minimum value from the maximum value and the upper limit of the single resource scaling allowable range as the latest resource upper limit value.
In this embodiment, when the container after the prohibition state is released needs to be expanded or contracted, the current resource upper limit value of the container is correspondingly increased or decreased, and compared with the prior art that the old container needs to be closed and the container resource needs to be adjusted through manual re-creation, dynamic adjustment of the size of the container resource can be realized, and the resource utilization rate is improved.
In one embodiment, there is provided a container flow regulating device, as shown with reference to fig. 3, comprising:
a node determining module 31, configured to determine, among a plurality of nodes in a cluster, a target node currently in an overload state;
a container grouping module 32, configured to group all container groups into two groups according to resource usage rates of the container groups running on the target node, so that a difference between a sum of the resource usage rates of the two groups is minimum, and a number of container groups of a first group including a container group with a highest resource usage rate does not exceed a number of container groups of a second group;
and a traffic adjusting module 33, configured to adjust the traffic of each container in the second packet to the node in the normal state.
In one embodiment, the container grouping module 31 is specifically configured to:
sorting all container groups on the target node in the order of the resource utilization rate from high to low;
and traversing all the sorted container groups in sequence, and dividing the currently traversed container group into one of the two groups with the smallest sum of the current resource utilization rates when traversing to one container group until completing the grouping of the last container group.
In one embodiment, the flow rate adjustment module 33 is specifically configured to:
acquiring the maximum container flow value of each container in the second grouping from a preset container configuration information table;
and adjusting the acquired maximum container flow value of each container according to a preset proportion, determining the flow to be scheduled, and scheduling the flow to be scheduled to the container of the node in a normal state.
In one embodiment, the apparatus further comprises:
and the state marking module is used for marking the forbidden state of each container on the target node so as to forbid the adjustment of the resource of each container.
In one embodiment, the apparatus further comprises:
the traffic recovery module is used for recovering the traffic of each container on the second packet to the maximum traffic when the target node is recovered to the normal state from the overload state;
the state releasing module is used for releasing the forbidden state of each container mark on the target node;
and the resource adjusting module is used for correspondingly increasing or decreasing the current resource upper limit value of the container when the container after the forbidden state is released needs to be expanded or contracted, wherein the current resource upper limit value is obtained by expanding and contracting the resource of the container last time.
In one embodiment, when the container needs to be expanded or contracted, the resource adjusting module is specifically configured to:
acquiring a target utilization rate and a single resource adjustment allowable range which are configured in advance by a container from a preset resource configuration information table;
calculating the product of the ratio of the current resource utilization rate of the container to the target utilization rate and the current resource upper limit value of the container to serve as the target resource upper limit value of the container;
determining the latest resource upper limit value corresponding to the container based on the single resource adjustment allowable range and the target resource upper limit value of the container;
and adjusting the current resource upper limit value of the container to the latest resource upper limit value.
Wherein, the single resource adjustment allowable range includes a single resource expansion allowable range, and the resource adjustment module is specifically configured to:
when the container needs to be expanded, taking the maximum value from the upper limit value of the target resource of the container and the lower limit of the single resource expansion allowable range, and taking the minimum value from the maximum value and the upper limit of the single resource expansion allowable range as the latest resource upper limit value;
the single resource adjustment allowable range includes a single resource contraction capacity allowable range, and the resource adjustment module is further specifically configured to:
when the container needs to be scaled down, taking the maximum value from the upper limit value of the target resource of the container and the lower limit of the single resource scaling allowable range, and taking the minimum value from the maximum value and the upper limit of the single resource scaling allowable range as the latest resource upper limit value.
It should be noted that: in the container flow rate adjusting device provided in this embodiment, only the division of the functional modules is exemplified, and in practical applications, the functions may be distributed by different functional modules according to needs, that is, the internal structure of the device may be divided into different functional modules to complete all or part of the functions described above. In addition, the container flow adjusting apparatus of this embodiment and the container flow adjusting method embodiment in the above embodiments belong to the same concept, and specific implementation processes and beneficial effects thereof are described in detail in the container resource adjusting method embodiment, and are not described herein again.
Fig. 4 is an internal structural diagram of a computer device according to an embodiment of the present invention. The computer device may be a server, and its internal structure diagram may be as shown in fig. 4. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a container flow adjustment method.
Those skilled in the art will appreciate that the configuration shown in fig. 4 is a block diagram of only a portion of the configuration associated with aspects of the present invention and is not intended to limit the computing devices to which aspects of the present invention may be applied, and that a particular computing device may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, there is also provided a computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
determining a target node in an overload state currently in a plurality of nodes of a cluster;
dividing all container groups into two groups according to the resource utilization rate of each container group operated on a target node, so that the difference of the sum of the resource utilization rates of the two groups is minimum, and the number of the container groups of a first group containing the container group with the highest resource utilization rate is not more than the number of the container groups of a second group;
the traffic of each container in the second packet is adjusted to the node in the normal state.
In one embodiment, there is also provided a computer readable storage medium having a computer program stored thereon, the computer program when executed by a processor implementing the steps of:
determining a target node in an overload state currently in a plurality of nodes of a cluster;
dividing all container groups into two groups according to the resource utilization rate of each container group operated on a target node, so that the difference of the sum of the resource utilization rates of the two groups is minimum, and the number of the container groups of a first group containing the container group with the highest resource utilization rate is not more than the number of the container groups of a second group;
the traffic of each container in the second packet is adjusted to the node in the normal state.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, databases, or other media used in embodiments provided herein may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above examples only show some embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (10)

1. A method of regulating a flow rate of a vessel, the method comprising:
determining a target node in an overload state currently in a plurality of nodes of a cluster;
dividing all the container groups into two groups according to the resource utilization rate of each container group running on the target node, so that the difference of the sum of the resource utilization rates of the two groups is minimum, and the number of the container groups of a first group containing the container group with the highest resource utilization rate does not exceed the number of the container groups of a second group;
and adjusting the flow of each container in the second packet to the node in the normal state.
2. The method according to claim 1, wherein said grouping all of said container groups into two groups according to resource utilization of each container group running on said target node comprises:
sorting all the container groups on the target node according to the sequence of the resource utilization rate from high to low;
and sequentially traversing all the sorted container groups, and dividing the currently traversed container group into one of the two groups with the smallest sum of the current resource utilization rates when traversing to one container group until finishing the grouping of the last container group.
3. The method of claim 1, wherein adjusting traffic for each container in the second packet to a node in a normal state comprises:
acquiring the maximum container flow value of each container in the second grouping from a preset container configuration information table;
and adjusting the acquired maximum container flow value of each container according to a preset proportion, determining the flow to be scheduled, and scheduling the flow to be scheduled to the container of the node in a normal state.
4. The method according to any of claims 1 to 3, wherein after the step of determining a target node currently in an overload state among the plurality of nodes in the cluster, the method further comprises:
marking each container on the target node with a disabled state to disable adjusting resources of each of the containers.
5. The method of claim 4, further comprising:
when the target node is recovered to a normal state from an overload state, adjusting the traffic of each container on the second packet to the maximum traffic;
and releasing the forbidden state of each container mark on the target node, and correspondingly increasing or decreasing the current resource upper limit value of the container when the container after the forbidden state is released needs to be expanded or contracted, wherein the current resource upper limit value is obtained by expanding and contracting the resource of the container last time.
6. A container flow regulating device, said device comprising:
the node determining module is used for determining a target node which is currently in an overload state from a plurality of nodes of the cluster;
a container grouping module, configured to group all the container groups into two groups according to resource usage rates of the container groups running on the target node, so that a difference between a sum of the resource usage rates of the two groups is minimized, and a number of container groups of a first group including a container group with a highest resource usage rate does not exceed a number of container groups of a second group;
and the flow adjusting module is used for adjusting the flow of each container in the second grouping to the node in the normal state.
7. The apparatus of claim 6, wherein the container grouping module is specifically configured to:
sorting all the container groups on the target node according to the sequence of the resource utilization rate from high to low;
and sequentially traversing all the sorted container groups, and dividing the currently traversed container group into one of the two groups with the smallest sum of the current resource utilization rates when traversing to one container group until finishing the grouping of the last container group.
8. The apparatus of claim 6, wherein the flow adjustment module is specifically configured to:
acquiring the maximum container flow value of each container in the second grouping from a preset container configuration information table;
and adjusting the acquired maximum container flow value of each container according to a preset proportion, determining the flow to be scheduled, and scheduling the flow to be scheduled to the container of the node in a normal state.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the container flow adjustment method according to any one of claims 1 to 5 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, implements the container flow adjusting method according to any one of claims 1 to 5.
CN202011002099.0A 2020-09-22 2020-09-22 Container flow adjustment method, device and system Active CN112202860B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011002099.0A CN112202860B (en) 2020-09-22 2020-09-22 Container flow adjustment method, device and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011002099.0A CN112202860B (en) 2020-09-22 2020-09-22 Container flow adjustment method, device and system

Publications (2)

Publication Number Publication Date
CN112202860A true CN112202860A (en) 2021-01-08
CN112202860B CN112202860B (en) 2023-07-14

Family

ID=74015862

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011002099.0A Active CN112202860B (en) 2020-09-22 2020-09-22 Container flow adjustment method, device and system

Country Status (1)

Country Link
CN (1) CN112202860B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108228354A (en) * 2017-12-29 2018-06-29 杭州朗和科技有限公司 Dispatching method, system, computer equipment and medium
CN110134497A (en) * 2019-06-21 2019-08-16 深圳市比一比网络科技有限公司 A kind of integrated dispositions method and its system applied to container technique
CN111104227A (en) * 2019-12-28 2020-05-05 北京浪潮数据技术有限公司 Resource control method and device of K8s platform and related components
CN111522636A (en) * 2020-04-03 2020-08-11 安超云软件有限公司 Application container adjusting method, application container adjusting system, computer readable medium and terminal device
CN111666131A (en) * 2020-07-03 2020-09-15 腾讯科技（深圳）有限公司 Load balancing distribution method and device, computer equipment and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108228354A (en) * 2017-12-29 2018-06-29 杭州朗和科技有限公司 Dispatching method, system, computer equipment and medium
CN110134497A (en) * 2019-06-21 2019-08-16 深圳市比一比网络科技有限公司 A kind of integrated dispositions method and its system applied to container technique
CN111104227A (en) * 2019-12-28 2020-05-05 北京浪潮数据技术有限公司 Resource control method and device of K8s platform and related components
CN111522636A (en) * 2020-04-03 2020-08-11 安超云软件有限公司 Application container adjusting method, application container adjusting system, computer readable medium and terminal device
CN111666131A (en) * 2020-07-03 2020-09-15 腾讯科技（深圳）有限公司 Load balancing distribution method and device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN112202860B (en) 2023-07-14

Similar Documents

Publication Publication Date Title
CN112181649B (en) Container resource adjustment method and device, computer equipment and storage medium
CN109194584B (en) Flow monitoring method and device, computer equipment and storage medium
CN110996352B (en) Flow control method, device, computer equipment and storage medium
CN111818159B (en) Management method, device, equipment and storage medium of data processing node
CN108881512B (en) CTDB virtual IP balance distribution method, device, equipment and medium
CN110677459A (en) Resource adjusting method and device, computer equipment and computer storage medium
CN111309644B (en) Memory allocation method and device and computer readable storage medium
US9058166B2 (en) Method and apparatus for managing processing resources in a distributed processing system
CN107872517B (en) Data processing method and device
CN112689007B (en) Resource allocation method, device, computer equipment and storage medium
CN112597202B (en) Data query method and device, computer equipment and storage medium
CN111580959B (en) Data writing method, data writing device, server and storage medium
CN106959820A (en) A kind of data extraction method and system
CN112256433B (en) Partition migration method and device based on Kafka cluster
CN108763312B (en) Slave data node screening method based on load
CN108459926B (en) Data remote backup method and device and computer readable medium
CN112202860A (en) Container flow adjusting method, device and system
CN114064226A (en) Resource coordination method and device for container cluster and storage medium
CN115203177A (en) Distributed data storage system and storage method
CN112667467A (en) Cluster self-adaptive current limiting method and device, computer equipment and storage medium
CN116643858A (en) Service priority pod-based rescheduling method, device, equipment and medium
CN117666926A (en) Data storage method and device and electronic equipment
CN114564153A (en) Volume mapping removing method, device, equipment and storage medium
CN117707779B (en) Data processing method, device, computer equipment and storage medium
CN115454997B (en) Bitmap management method, bitmap management device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant