CN113791863A - Virtual container-based power internet of things agent resource scheduling method and related equipment - Google Patents
Virtual container-based power internet of things agent resource scheduling method and related equipment Download PDFInfo
- Publication number
- CN113791863A CN113791863A CN202110914882.2A CN202110914882A CN113791863A CN 113791863 A CN113791863 A CN 113791863A CN 202110914882 A CN202110914882 A CN 202110914882A CN 113791863 A CN113791863 A CN 113791863A
- Authority
- CN
- China
- Prior art keywords
- physical node
- resource
- container
- load
- physical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 230000004044 response Effects 0.000 claims abstract description 28
- 239000013598 vector Substances 0.000 claims description 38
- 239000003795 chemical substances by application Substances 0.000 claims description 17
- 238000012937 correction Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 230000004931 aggregating effect Effects 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 description 17
- 238000004422 calculation algorithm Methods 0.000 description 12
- 238000004891 communication Methods 0.000 description 10
- 230000008859 change Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000005457 optimization Methods 0.000 description 5
- 230000006872 improvement Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000010248 power generation Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45562—Creating, deleting, cloning virtual machine instances
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The utility model provides a virtual container-based power internet of things agent resource scheduling method and related equipment, wherein the method comprises the following steps: acquiring load information of each physical node; carrying out resource scheduling on the physical node according to the load information; load resource prediction is carried out on each running service, and expected value of the number of containers needed by the service is calculated; acquiring the real value of the number of the containers, judging the magnitude relation between the real value and the expected value, responding to the fact that the real value is smaller than the expected value, selecting a first optimal physical node from the physical nodes after the resource scheduling, and deploying a new container; and/or in response to determining that the true value is greater than the expected value, selecting a second optimal physical node from the physical nodes after the resource scheduling, and deleting the old container. The method and the device realize load balance of the container cluster and improve the resource utilization rate, and enhance the performance of the equipment in the power internet of things.
Description
Technical Field
The disclosure relates to the technical field of computer virtualization, in particular to a virtual container-based power internet of things agent resource scheduling method and related equipment.
Background
The wide application of the power internet of things generates a huge amount of data, the virtualization technology provides a solution for storage management of the data, and the virtualization technology becomes a widely accepted server resource sharing mode, wherein the container technology becomes a main deployment model of the virtualization technology due to the light weight property of the container technology.
Disclosure of Invention
In view of this, the present disclosure provides a virtual container-based power internet of things proxy resource scheduling method and related devices.
In a first aspect of the present disclosure, a virtual container-based power internet of things agent resource scheduling method is provided, including: acquiring load information of each physical node; carrying out resource scheduling on the physical node according to the load information; load resource prediction is carried out on each running service, and expected value of the number of containers needed by the service is calculated; acquiring the real value of the container quantity, judging the magnitude relation between the real value and the expected value, responding to the fact that the real value is smaller than the expected value, selecting a first optimal physical node from the physical nodes after the resource scheduling, and deploying a new container on the first optimal physical node; and/or in response to determining that the true value is greater than the expected value, selecting a second optimal physical node from the physical nodes after the resource scheduling, and deleting an old container on the second optimal physical node.
Further, the air conditioner is provided with a fan,the load information includes: CPU utilization of the physical node
Memory utilization
Network bandwidth utilization
IO Bandwidth utilization
Load resource utilization
Wherein the load resource utilization
For the CPU utilization
Memory utilization
Network bandwidth utilization
And IO Bandwidth utilization
Maximum value of (2).
Further, the performing resource scheduling on the physical node according to the load information specifically includes: screening the load resource utilization
Less than 40% of the physical nodes are divided into a first cluster and a second cluster with the same number; wherein the total amount of used load resources of the first cluster is lower than that of the second clusterThe total amount of used load resources; scheduling containers of physical nodes in the first cluster onto physical nodes of the second cluster.
Further, the load resource prediction for each running service specifically includes:
a. initializing a prediction updating coefficient vector w, a load resource prediction vector y and a load resource vector o at the time t;
b. predicting a predicted value of the resources required by the service at the moment t, and calculating a prediction error;
c. adjusting the prediction updating coefficient vector w according to the prediction error, and calculating the mean square error corresponding to the adjusted prediction updating coefficient vector w;
d. repeating the steps b and c until the mean square error is smaller than a given threshold or the repetition times is larger than the maximum correction times;
e. and updating the load resource prediction vector y and the load resource vector o, and predicting the load resources required by service at the moment of t + 1.
Further, the obtaining of the actual value of the number of the containers and the judging of the magnitude relationship between the actual value and the expected value specifically include: acquiring the real value of the number of the containers, and calculating the difference value flag between the real value of the number of the containers and the expected value;
in response to determining that the true value is smaller than the expected value, selecting a first optimal physical node from the physical nodes after the resource scheduling, and deploying a new container on the first optimal physical node, specifically including: in response to determining that the difference flag is <0, obtaining a first set and a second set of the physical nodes for deploying containers from the physical nodes after the resource scheduling; calculating the weight of each physical node according to the load condition of the physical nodes of the first set; selecting the physical node with the maximum weight as a first optimal physical node, deploying a new container on the first optimal physical node, and repeating the flag for | times;
in response to determining that the true value is greater than the expected value, selecting a second optimal physical node from the physical nodes after the resource scheduling, and deleting an old container on the second optimal physical node, specifically including: in response to determining that the difference flag >0, obtaining a third set of physical nodes of a container in which the service has been deployed from the physical nodes after the resource scheduling; calculating the weight of each physical node according to the load condition of the physical nodes of the third set; and selecting the physical node with the maximum weight as a second optimal physical node, deleting the old container on the second optimal physical node, and repeating the flag for | times.
Further, the first set includes post-deployment load resource utilization for deploying new containers
The second set comprising the post-deployment load resource utilization for deploying the new container
A physical node of (a); when there are no physical nodes in the first set, aggregating the physical nodes of the second set into the first set.
Further, the weight value represents the resource utilization rate of the physical node after the container is deployed or deleted
Including resource utilization within and between the physical nodes
The degree of equalization of (a) varies.
In a second aspect of the present disclosure, a virtual container-based power internet of things agent resource scheduling device is provided, including:
the information acquisition module is configured to acquire load information of each physical node;
a node scheduling module configured to perform resource scheduling on the physical node according to the load information;
the resource prediction module is configured to predict load resources of each running service and calculate the expected value of the number of containers needed by the service;
the container scheduling module is configured to acquire a real value of the number of the containers, judge the magnitude relation between the real value and the expected value, select a first optimal physical node from the physical nodes after the resource scheduling in response to determining that the real value is smaller than the expected value, and deploy a new container on the first optimal physical node; and/or in response to determining that the true value is greater than the expected value, selecting a second optimal physical node from the physical nodes after the resource scheduling, and deleting an old container on the second optimal physical node.
Further, the container scheduling module is specifically configured to obtain a true value of the number of containers, and calculate a difference flag between the true value and the expected value of the number of containers; in response to determining that the difference flag is <0, obtaining a first set and a second set of the physical nodes for deploying containers from the physical nodes after the resource scheduling; calculating the weight of each physical node according to the load condition of the physical nodes of the first set; selecting the physical node with the maximum weight as a first optimal physical node, deploying a new container on the first optimal physical node, and repeating the flag for | times; in response to determining that the difference flag >0, obtaining a third set of physical nodes of a container in which the service has been deployed from the physical nodes after the resource scheduling; calculating the weight of each physical node according to the load condition of the physical nodes of the third set; and selecting the physical node with the maximum weight as a second optimal physical node, deleting the old container on the second optimal physical node, and repeating the flag for | times.
In a third aspect of the disclosure, an electronic device is provided, which includes a memory, a processor, and a computer program stored on the memory and executed by the processor, and the processor implements the method as described above when executing the program.
From the above, the present disclosure provides a virtual container-based power internet of things agent resource scheduling method and related devices, which achieve load balancing and resource utilization improvement of a container cluster, and enhance performance of the devices in the power internet of things; according to the load resource use condition of each physical node, resource scheduling is carried out, the condition that the nodes are used too much is avoided, and the node resource utilization rate is improved; the optimal physical node is selected to deploy or delete the container, so that the resource utilization rate and the balance degree of each physical node are further improved; by predicting load resources in advance, the container dynamic scheduling is better carried out, and the performance of each physical node under the condition of flow burst can be improved.
Drawings
In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings needed to be used in the description of the embodiments or related technologies are briefly introduced below, and it is obvious that the drawings in the following description are only embodiments of the present disclosure, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a schematic flowchart of a virtual container-based power internet of things agent resource scheduling method according to an embodiment of the present disclosure;
FIG. 2 is a schematic flow chart of a load resource prediction algorithm according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of a virtual container-based power internet of things agent resource scheduling device according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure.
Detailed Description
For the purpose of promoting a better understanding of the objects, aspects and advantages of the present disclosure, reference is made to the following detailed description taken in conjunction with the accompanying drawings.
It is to be noted that technical terms or scientific terms used in the embodiments of the present disclosure should have a general meaning as understood by those having ordinary skill in the art to which the present disclosure belongs, unless otherwise defined. The use of "first," "second," and similar terms in the embodiments of the disclosure is not intended to indicate any order, quantity, or importance, but rather to distinguish one element from another. The word "comprising" or "comprises", and the like, means that the element or item listed before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", and the like are used merely to indicate relative positional relationships, and when the absolute position of the object being described is changed, the relative positional relationships may also be changed accordingly.
As described in the background art, the power internet of things is a network that is widely deployed with intelligent sensing devices in each link of power production, transmission, consumption and management to realize safe and reliable information transmission, cooperative processing, unified service and application integration. The power internet of things improves the utilization efficiency of power facilities and provides important technical support for links such as power generation, transmission, transformation, distribution and power utilization of a power grid. Currently, the development of the internet of things industry has risen to national strategy.
The internet of things generates a huge amount of data, and how to better store and manage the data to improve the performance of the device becomes a challenge. Cloud computing provides a solution to store and manage the data it produces and accumulates.
The cloud computing technology is a computing mode provided for users through a network, so that the users can conveniently and quickly acquire computing resources according to own requirements. The cloud computing technology regards the whole computer resource as a whole, divides all requests into smaller segments according to a certain rule, distributes the smaller segments to different servers for simultaneous processing, greatly reduces the computing amount of a single server, and improves the computing efficiency. In the process, the user only needs to send a request and receive a service result, and does not need to pay attention to the intermediate process.
The virtualization technology is one of core technologies of cloud computing, and computing resources are managed in a unified manner by logically isolating physical resources, so that the resource utilization rate is improved. Common virtualization technologies are virtual machine technology and container technology. The virtual machine technology needs to simulate the whole operating system and all hardware, occupies a large amount of resources, and is slow in starting speed. Compared with virtual machine technology, container technology is more lightweight. In addition, in the power internet of things scenario, the device is usually limited by resources, and a lightweight application program is also required to be adopted, so as to reduce unnecessary overhead.
Docker is an implementation engine for containers that can efficiently deploy, execute, and manage containers. Because the container granularity of the Docker is small, more containers can be deployed on the same physical node, and the management and scheduling of the containers are more complicated. The traditional container cluster management strategy is simple in structure, insufficient in stability, difficult to guarantee efficient resource utilization, low in resource utilization rate and poor in load balance degree of the container cluster.
In the process of implementing the present disclosure, the applicant finds that if an optimal node is selected to deploy or delete a container according to the use condition of load resources, the resource utilization rate and the balance of each physical node can be improved, and in order to avoid the situation that the original node is used too much, the container of the node can be scheduled according to the load information, so that the resource utilization rate is further improved, and the change trend of resources required by each service can be predicted in advance, so as to help better perform container scheduling.
Hereinafter, the technical means of the present disclosure will be described in detail by specific examples.
Referring to fig. 1, a schematic flowchart of a virtual container-based power internet of things agent resource scheduling method according to an embodiment of the present disclosure is shown, where the method includes the following steps:
and S1, acquiring the load information of each physical node to provide a basis for the subsequent steps.
Entire container cluster environment is available (N)s,Ss,Cs) Is represented by, wherein, NsRepresenting a set of physical nodes, one of which is used
Represents; ssRepresenting a collection of services, one of which is for
Represents; csRepresenting a collection of containers, one of which
Represents; by | NsI represents NsNumber of physical nodes in
To represent
The number of middle containers, i, j, k, represents the serial number.
Each service may contain multiple containers, each of which can only run on one physical node and be used
Presentation service
Each physical node comprises a certain CPU, memory, network bandwidth and IO bandwidth resource and uses the resources
Representing the CPU utilization rate, the memory utilization rate, the network bandwidth utilization rate, the IO bandwidth utilization rate, the load resource utilization rate and the load resource utilization rate of the physical node i
Get CPU utilization
Memory utilization
Network bandwidth utilization
And IO bandWide utilization ratio
Maximum value of (2).
And S2, performing resource scheduling on the physical nodes according to the load information to avoid starting more nodes.
The method specifically comprises the following steps: screening the load resource utilization
Less than 40% of the physical nodes are divided into a first cluster and a second cluster with the same number; wherein the total amount of load resource usage of the first cluster is lower than the total amount of load resource usage of the second cluster; and dispatching the container of the physical node in the first cluster to the physical node of the second cluster, thereby improving the utilization rate of node resources.
And S3, performing load resource prediction on each running service, calculating the expected value of the number of containers required by the service, and performing container dynamic scheduling better through advance prediction.
Referring to fig. 2, a flowchart of a load resource prediction algorithm according to an embodiment of the present disclosure is shown, where the load resource prediction for each running service includes:
a. initializing a prediction updating coefficient vector w, a load resource prediction vector y and a load resource vector o at the time t;
b. predicting a predicted value of the resources required by the service at the moment t, and calculating a prediction error;
c. adjusting the prediction updating coefficient vector w according to the prediction error, and calculating the mean square error corresponding to the adjusted prediction updating coefficient vector w;
d. repeating the steps b and c until the mean square error is smaller than a given threshold or the repetition times is larger than the maximum correction times;
e. and updating the load resource prediction vector y and the load resource vector o, and predicting the load resources required by service at the moment of t + 1.
By predicting load resources in advance, the container dynamic scheduling is better carried out, and the performance of each physical node under the condition of flow burst can be improved.
The code for the load resource prediction algorithm can be designed as follows:
inputting: a prediction update coefficient vector w (including the prediction update coefficient at the time t-q- > the time t-1), a load resource prediction vector y (including the prediction value at the time t-q- > the time t-1), a load resource vector o (including the true value at the time t-q- > the time t-1), and a resource yt required for service at the time t
And (3) outputting: predicted value at time t +1
In the code of the load resource prediction algorithm, w represents a prediction updating coefficient vector, a group of coefficients are maintained, and the prediction of the load resource at the next moment is facilitated; y represents a load resource prediction vector, and the prediction values of the load resources at the latest q moments are maintained; o represents a load resource vector, and real values of the load resources at the latest q moments are stored; q represents the length of the vector, is a constant, and predicts the load resource at the next moment through the real data of the load resource at the past q moments; p represents an adjustment rate, is a constant and is generally smaller than 1, the size of the value determines the size of the iteration step of the prediction updating coefficient vector, the convergence speed is influenced, if p is too small, the convergence is too slow, and if the adjustment rate p is too large, the algorithm is easy to be not converged; dif represents a prediction error and is a difference value between a true value and a predicted value; MSE represents mean square error, represents mean square error between predicted value and true value of load resource, is used for considering whether the prediction vector w of load resource is rational; expecter represents a given error threshold, if mean square error MSE is greater than expecter, it represents that load resource prediction vector w is unreasonable, and deviation needs to be corrected continuously; and representing the maximum correction times by maxTryCount, and stopping correcting the deviation if the deviation of the maxTryCount is corrected for the load resource prediction vector w, and the w is still unreasonable, so as to prevent excessive deviation correction.
Lines 1-4 of the code of the load resource prediction algorithm may be understood as checking whether there is sufficient data accumulation, and if not, updating the prediction update coefficient vector w, the load resource prediction vector y, and the load resource vector o; the 5 th to 12 th rows represent entering a cycle, the whole cycle is to adjust the prediction updating coefficient vector w so as to better predict the value at the t +1 moment, predict the predicted value predicted and the prediction error dif of the resource required by the service at the current t moment, adjust the prediction updating coefficient vector w by using the prediction error, finally calculate the mean square error MSE of the error between the predicted value and the true value, and jump out of the cycle if the prediction error is smaller than a given threshold expectErr or the cycle number is larger than the maximum correction number maxTryCount; lines 13-19 update the vectors y, o and predict the resources needed for service at time t + 1.
S4, obtaining a real value of the number of the containers, judging the magnitude relation between the real value and the expected value, responding to the fact that the real value is smaller than the expected value, selecting a first optimal physical node from the physical nodes after the resource scheduling, and deploying a new container on the first optimal physical node; and/or in response to determining that the true value is greater than the expected value, selecting a second optimal physical node from the physical nodes after resource scheduling, and deleting an old container on the second optimal physical node, so that load balancing of a container cluster and improvement of resource utilization rate are realized, and performance of the equipment in the power internet of things is enhanced.
Further, the obtaining of the actual value of the number of the containers and the judging of the magnitude relationship between the actual value and the expected value specifically include: acquiring the real value of the number of the containers, and calculating the difference value flag between the real value of the number of the containers and the expected value;
in response to determining that the true value is smaller than the expected value, selecting a first optimal physical node from the physical nodes after the resource scheduling, and deploying a new container on the first optimal physical node, specifically including: in response to determining that the difference flag is <0, obtaining a first set and a second set of the physical nodes for deploying containers from the physical nodes after the resource scheduling; calculating the weight of each physical node according to the load condition of the physical nodes of the first set; selecting the physical node with the maximum weight as a first optimal physical node, deploying a new container on the first optimal physical node, and repeating the flag for | times;
in response to determining that the true value is greater than the expected value, selecting a second optimal physical node from the physical nodes after the resource scheduling, and deleting an old container on the second optimal physical node, specifically including: in response to determining that the difference flag >0, obtaining a third set of physical nodes of a container in which the service has been deployed from the physical nodes after the resource scheduling; calculating the weight of each physical node according to the load condition of the physical nodes of the third set; and selecting the physical node with the maximum weight as a second optimal physical node, deleting the old container on the second optimal physical node, and repeating the flag for | times.
According to the difference value between the actual value and the expected value of the number of the containers, namely, according to the change trend of the required resources, the containers are scheduled, the containers are deleted under the condition of large number of the containers, and new containers are established under the condition of small number of the containers, so that the balance degree of the container cluster is further improved, and the resource utilization rate is improved; it should be noted that if the difference flag is equal to 0, the expected value representing the container is the same as the actual value, and container scheduling is not required.
For deploying or deleting the container, the operation may be performed based on manual allocation, based on resource usage and performance conditions when different loads are operated, based on a dynamic weighted scheduling algorithm, or based on historical conflicts, and the like, which is not limited specifically.
Further, the first set includes post-deployment load resource utilization for deploying new containers
The second set comprising the post-deployment load resource utilization for deploying the new container
A physical node of (a); when no physical node exists in the first set, the physical nodes of the second set are collected into the first set, so that the resource utilization rate can be improved and the operation efficiency can be improved when a container is deployed for a node with relatively low resource utilization rate.
In this step, the weight value represents the resource utilization rate of the physical node after the container is deployed or deleted
Including resource utilization within and between the physical nodes
The balance degree of each physical node is changed, so that the balance degree of each physical node can be represented more comprehensively, and the optimization and balance of resources are facilitated.
In a general view, the virtual container-based power internet of things agent resource scheduling method realizes load balancing and resource utilization rate improvement of a container cluster, and enhances performance of equipment in the power internet of things; according to the load resource use condition of each physical node, resource scheduling is carried out, the condition that the nodes are used too much is avoided, and the node resource utilization rate is improved; the optimal physical node is selected to deploy or delete the container, so that the resource utilization rate and the balance degree of each physical node are further improved; by predicting load resources in advance, the container dynamic scheduling is better carried out, and the performance of each physical node under the condition of flow burst can be improved.
In some embodiments, the code of the virtual container-based power internet of things agent resource scheduling algorithm may be designed as follows:
inputting: is free of
And (3) outputting: is free of
In the code of the virtual container-based power internet of things agent resource scheduling algorithm, num represents a container number expected value required by the service;
a true value representing the number of containers; filter represents a filter; work 1 represents the first set of physical nodes; work 2 represents the second set of physical nodes; work 3 represents the third set of physical nodes;
representing resource utilization of the physical node after the container is deployed
The degree of equalization of;
representing resource utilization of the physical node after deleting a container
The degree of equalization of (a) varies.
Lines 1-9 of the code of the virtual container-based power internet of things agent resource scheduling algorithm can be understood as that nodes with load resource utilization rate lower than 40% are checked, and if the number of the nodes is greater than or equal to 2, containers of half of the nodes with less node resources in the nodes are scheduled to half of the nodes with more resources, so that excessive nodes are prevented from being started; lines 10-11 can be understood as calling a load resource prediction algorithm for each running service, and obtaining and recording a predicted value of the load resource of the service k; lines 12-13 represent the calculation of the flag as the difference between the number of containers contained by the service and the number of containers required by the service, i.e. the difference between the real value and the expected value; lines 14-24 represent flag <0, indicating that insufficient containers are needed and new containers need to be added; lines 25-33 represent flag >0, indicating that the container is redundant and that redundant containers need to be deleted.
Further, the air conditioner is provided with a fan,
and
is calculated as follows:
in the formula, alpha and beta are weight coefficients and are used for adjusting the proportion of an optimization target;
representing the change of the balance of the utilization rate of various resources in the physical node after the container is deployed;
representing the change of the balance of the utilization rate of various resources in the physical node after the container is deleted;
change of balance degree of load resource utilization rate between physical nodes after the container is deployed;
representing the change in the balance of load resource utilization between physical nodes after the container is deleted.
Further, the air conditioner is provided with a fan,
and
is calculated as follows:
can use
Representing the standard deviation between the internal resource utilization of physical node i, then in the equation,
representing the standard deviation between the internal resource utilization before physical node i deploys or deletes a container;
representing the standard deviation between internal resource utilization after physical node i deploys the container;
representing the standard deviation between internal resource utilization after physical node i deletes the container.
Can use
Representing the standard deviation between the load resource utilization of all physical nodes, then in the equation,
representing the standard deviation between the load resource utilization of all physical nodes before the physical node i deploys or deletes the container;
representing the standard deviation between the load resource utilization of all physical nodes after physical node i deploys the container;
representing the standard deviation between the load resource utilization of all physical nodes after physical node i deletes the container.
Further, the air conditioner is provided with a fan,
and
is calculated as follows:
in the formula (I), the compound is shown in the specification,
representing an average value of the utilization rate of the internal resources of the physical nodes;
representing the average value of resource utilization between physical nodes.
The optimization target of the algorithm is the optimization degree of the internal balance of the physical nodes and the balance of the load among the physical nodes, and the final total optimization target is as follows:
that is to say
Or
The largest physical node of the values, where,
the physical node with the largest numerical value is the first optimal physical node, and a new container is deployed on the node;
the physical node with the largest numerical value is the second optimal physical node, and the old container is deleted from the node; therefore, the purpose of container dynamic scheduling is achieved, and the utilization rate and the balance degree of load resources are improved.
It should be noted that the method of one or more embodiments of the present disclosure may be performed by a single device, such as a computer or server. The method of the embodiment can also be applied to a distributed scene and completed by the mutual cooperation of a plurality of devices. In such a distributed scenario, one of the devices may perform only one or more steps of the method of one or more embodiments of the present disclosure, and the devices may interact with each other to complete the method.
The foregoing description of specific embodiments of the present disclosure has been described. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
Based on the same inventive concept, corresponding to any of the above embodiments, the present disclosure further provides an electric power internet of things agent resource scheduling device based on a virtual container, and with reference to fig. 3, the method includes:
an information acquisition module 301 configured to acquire load information of each physical node;
a node scheduling module 302 configured to perform resource scheduling on the physical node according to the load information;
a resource prediction module 303, configured to perform load resource prediction on each running service, and calculate an expected value of the number of containers required by the service;
a container scheduling module 304, configured to obtain a true value of the number of containers, determine a magnitude relationship between the true value and the expected value, select a first optimal physical node from the physical nodes after the resource scheduling in response to determining that the true value is smaller than the expected value, and deploy a new container on the first optimal physical node; and/or in response to determining that the true value is greater than the expected value, selecting a second optimal physical node from the physical nodes after the resource scheduling, and deleting an old container on the second optimal physical node.
For convenience of description, the above system is described with the functions divided into various modules, which are described separately. Of course, the functionality of the various modules may be implemented in the same one or more software and/or hardware implementations in implementing one or more embodiments of the present disclosure.
The container scheduling module is specifically configured to obtain a true value of the number of containers, and calculate a difference value flag between the true value and the expected value of the number of containers; in response to determining that the difference flag is <0, obtaining a first set and a second set of the physical nodes for deploying containers from the physical nodes after the resource scheduling; calculating the weight of each physical node according to the load condition of the physical nodes of the first set; selecting the physical node with the maximum weight as a first optimal physical node, deploying a new container on the first optimal physical node, and repeating the flag for | times; in response to determining that the difference flag >0, obtaining a third set of physical nodes of a container in which the service has been deployed from the physical nodes after the resource scheduling; calculating the weight of each physical node according to the load condition of the physical nodes of the third set; and selecting the physical node with the maximum weight as a second optimal physical node, deleting the old container on the second optimal physical node, and repeating the flag for | times.
The system of the foregoing embodiment is used to implement the corresponding method in the foregoing embodiment, and has the beneficial effects of the corresponding virtual container-based power internet of things proxy resource scheduling method embodiment, which are not described herein again.
Based on the same inventive concept, corresponding to the method of any embodiment described above, the present disclosure further provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory and capable of running on the processor, where when the processor executes the program, the virtual container based power internet of things agent resource scheduling method described in any embodiment above is implemented.
Fig. 4 is a schematic diagram illustrating a more specific hardware structure of an electronic device according to this embodiment, where the electronic device may include: a processor 1010, a memory 1020, an input/output interface 1030, a communication interface 1040, and a bus 1050. Wherein the processor 1010, memory 1020, input/output interface 1030, and communication interface 1040 are communicatively coupled to each other within the device via bus 1050.
The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits, and is configured to execute related programs to implement the technical solutions provided in the embodiments of the present disclosure.
The Memory 1020 may be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, or the like. The memory 1020 may store an operating system and other application programs, and when the technical solution provided by the embodiments of the present specification is implemented by software or firmware, the relevant program codes are stored in the memory 1020 and called to be executed by the processor 1010.
The input/output interface 1030 is used for connecting an input/output module to input and output information. The i/o module may be configured as a component in a device (not shown) or may be external to the device to provide a corresponding function. The input devices may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices may include a display, a speaker, a vibrator, an indicator light, etc.
The communication interface 1040 is used for connecting a communication module (not shown in the drawings) to implement communication interaction between the present apparatus and other apparatuses. The communication module can realize communication in a wired mode (such as USB, network cable and the like) and also can realize communication in a wireless mode (such as mobile network, WIFI, Bluetooth and the like).
It should be noted that although the above-mentioned device only shows the processor 1010, the memory 1020, the input/output interface 1030, the communication interface 1040 and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, those skilled in the art will appreciate that the above-described apparatus may also include only those components necessary to implement the embodiments of the present description, and not necessarily all of the components shown in the figures.
The electronic device of the foregoing embodiment is used to implement the virtual container-based power internet of things agent resource scheduling method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which are not described herein again.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, is limited to these examples; within the idea of the present disclosure, also technical features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of brevity.
In addition, well-known power/ground connections to Integrated Circuit (IC) chips and other components may or may not be shown in the provided figures for simplicity of illustration and discussion, and so as not to obscure the embodiments of the disclosure. Furthermore, devices may be shown in block diagram form in order to avoid obscuring embodiments of the present disclosure, and this also takes into account the fact that specifics with respect to implementation of such block diagram devices are highly dependent upon the platform within which the embodiments of the present disclosure are to be implemented (i.e., specifics should be well within purview of one skilled in the art). Where specific details (e.g., circuits) are set forth in order to describe example embodiments of the disclosure, it should be apparent to one skilled in the art that the embodiments of the disclosure can be practiced without, or with variation of, these specific details. Accordingly, the description is to be regarded as illustrative instead of restrictive.
While the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic ram (dram)) may use the discussed embodiments.
The disclosed embodiments are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalents, improvements, and the like that may be made within the spirit and principles of the embodiments of the disclosure are intended to be included within the scope of the disclosure.
Claims (10)
1. A virtual container-based power internet of things agent resource scheduling method comprises the following steps:
acquiring load information of each physical node;
carrying out resource scheduling on the physical node according to the load information;
load resource prediction is carried out on each running service, and expected value of the number of containers needed by the service is calculated;
acquiring the real value of the container quantity, judging the magnitude relation between the real value and the expected value, responding to the fact that the real value is smaller than the expected value, selecting a first optimal physical node from the physical nodes after the resource scheduling, and deploying a new container on the first optimal physical node; and/or in response to determining that the true value is greater than the expected value, selecting a second optimal physical node from the physical nodes after the resource scheduling, and deleting an old container on the second optimal physical node.
2. The method of claim 1, wherein the load information comprises: CPU utilization of the physical node
Memory utilization
Network bandwidth utilization
IO Bandwidth utilization
Load resource utilization
Wherein the load resource utilization
For the CPU utilization
Memory utilization
Network bandwidth utilization
And IO Bandwidth utilization
Maximum value of (2).
3. The method according to claim 2, wherein the performing resource scheduling on the physical node according to the load information specifically includes:
screening the load resource utilization
Less than 40% of the physical nodes are divided into a first cluster and a second cluster with the same number; wherein the total amount of load resource usage of the first cluster is lower than the total amount of load resource usage of the second cluster;
scheduling containers of physical nodes in the first cluster onto physical nodes of the second cluster.
4. The method according to claim 1, wherein the load resource prediction for each running service specifically comprises:
a. initializing a prediction updating coefficient vector w, a load resource prediction vector y and a load resource vector o at the time t;
b. predicting a predicted value of the resources required by the service at the moment t, and calculating a prediction error;
c. adjusting the prediction updating coefficient vector w according to the prediction error, and calculating the mean square error corresponding to the adjusted prediction updating coefficient vector w;
d. repeating the steps b and c until the mean square error is smaller than a given threshold or the repetition times is larger than the maximum correction times;
e. and updating the load resource prediction vector y and the load resource vector o, and predicting the load resources required by service at the moment of t + 1.
5. The method according to claim 2, wherein the obtaining of the actual value of the number of containers and the determining of the magnitude relationship between the actual value and the expected value specifically include: acquiring the real value of the number of the containers, and calculating the difference value flag between the real value of the number of the containers and the expected value;
in response to determining that the true value is smaller than the expected value, selecting a first optimal physical node from the physical nodes after the resource scheduling, and deploying a new container on the first optimal physical node, specifically including: in response to determining that the difference flag is <0, obtaining a first set and a second set of the physical nodes for deploying containers from the physical nodes after the resource scheduling; calculating the weight of each physical node according to the load condition of the physical nodes of the first set; selecting the physical node with the maximum weight as a first optimal physical node, deploying a new container on the first optimal physical node, and repeating the flag for | times;
in response to determining that the true value is greater than the expected value, selecting a second optimal physical node from the physical nodes after the resource scheduling, and deleting an old container on the second optimal physical node, specifically including: in response to determining that the difference flag >0, obtaining a third set of physical nodes of a container in which the service has been deployed from the physical nodes after the resource scheduling; calculating the weight of each physical node according to the load condition of the physical nodes of the third set; and selecting the physical node with the maximum weight as a second optimal physical node, deleting the old container on the second optimal physical node, and repeating the flag for | times.
6. The method of claim 5, the first set comprising post-deployment load resource utilization for deploying new containers
The second set comprising the post-deployment load resource utilization for deploying the new container
A physical node of (a); when there are no physical nodes in the first set, aggregating the physical nodes of the second set into the first set.
7. The method of claim 5, wherein the weight represents a resource utilization of the physical node after a container is deployed or deleted
Including resource utilization within and between the physical nodes
The degree of equalization of (a) varies.
8. A virtual container-based power Internet of things agent resource scheduling device comprises:
the information acquisition module is configured to acquire load information of each physical node;
a node scheduling module configured to perform resource scheduling on the physical node according to the load information;
the resource prediction module is configured to predict load resources of each running service and calculate the expected value of the number of containers needed by the service;
the container scheduling module is configured to acquire a real value of the number of the containers, judge the magnitude relation between the real value and the expected value, select a first optimal physical node from the physical nodes after the resource scheduling in response to determining that the real value is smaller than the expected value, and deploy a new container on the first optimal physical node; and/or in response to determining that the true value is greater than the expected value, selecting a second optimal physical node from the physical nodes after the resource scheduling, and deleting an old container on the second optimal physical node.
9. The apparatus according to claim 8, wherein the container scheduling module is specifically configured to obtain a true value of the number of containers, and calculate a difference flag between the true value of the number of containers and the expected value; in response to determining that the difference flag is <0, obtaining a first set and a second set of the physical nodes for deploying containers from the physical nodes after the resource scheduling; calculating the weight of each physical node according to the load condition of the physical nodes of the first set; selecting the physical node with the maximum weight as a first optimal physical node, deploying a new container on the first optimal physical node, and repeating the flag for | times; in response to determining that the difference flag >0, obtaining a third set of physical nodes of a container in which the service has been deployed from the physical nodes after the resource scheduling; calculating the weight of each physical node according to the load condition of the physical nodes of the third set; and selecting the physical node with the maximum weight as a second optimal physical node, deleting the old container on the second optimal physical node, and repeating the flag for | times.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executed by the processor, the processor implementing the method of any one of claims 1 to 7 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110914882.2A CN113791863B (en) | 2021-08-10 | 2021-08-10 | Virtual container-based power Internet of things proxy resource scheduling method and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110914882.2A CN113791863B (en) | 2021-08-10 | 2021-08-10 | Virtual container-based power Internet of things proxy resource scheduling method and related equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113791863A true CN113791863A (en) | 2021-12-14 |
| CN113791863B CN113791863B (en) | 2024-01-23 |
Family
ID=78875847
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110914882.2A Active CN113791863B (en) | 2021-08-10 | 2021-08-10 | Virtual container-based power Internet of things proxy resource scheduling method and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113791863B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105794099A (en) * | 2013-12-11 | 2016-07-20 | Abb 技术有限公司 | Pulse width modulation with faster capacitor balancing |
| CN107045455A (en) * | 2017-06-19 | 2017-08-15 | 华中科技大学 | A kind of Docker Swarm cluster resource method for optimizing scheduling based on load estimation |
| US20180167487A1 (en) * | 2016-12-13 | 2018-06-14 | Red Hat, Inc. | Container deployment scheduling with constant time rejection request filtering |
| CN108829494A (en) * | 2018-06-25 | 2018-11-16 | 杭州谐云科技有限公司 | Container cloud platform intelligence method for optimizing resources based on load estimation |
| EP3410368A1 (en) * | 2013-06-19 | 2018-12-05 | United States Postal Service | System and method for providing real-time tracking of items in a distribution network |
| CN109918198A (en) * | 2019-02-18 | 2019-06-21 | 中国空间技术研究院 | A kind of emulation cloud platform load dispatch system and method based on user characteristics prediction |
| US20190197178A1 (en) * | 2017-12-22 | 2019-06-27 | International Business Machines Corporation | Container structure |
| CN111857953A (en) * | 2020-07-17 | 2020-10-30 | 苏州浪潮智能科技有限公司 | Container cluster management method, device, equipment and readable storage medium |
| US20210141655A1 (en) * | 2019-11-13 | 2021-05-13 | Vmware, Inc. | Unified resource management for containers and virtual machines |
| CN113110914A (en) * | 2021-03-02 | 2021-07-13 | 西安电子科技大学 | Internet of things platform construction method based on micro-service architecture |
-
2021
- 2021-08-10 CN CN202110914882.2A patent/CN113791863B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3410368A1 (en) * | 2013-06-19 | 2018-12-05 | United States Postal Service | System and method for providing real-time tracking of items in a distribution network |
| CN105794099A (en) * | 2013-12-11 | 2016-07-20 | Abb 技术有限公司 | Pulse width modulation with faster capacitor balancing |
| US20180167487A1 (en) * | 2016-12-13 | 2018-06-14 | Red Hat, Inc. | Container deployment scheduling with constant time rejection request filtering |
| CN107045455A (en) * | 2017-06-19 | 2017-08-15 | 华中科技大学 | A kind of Docker Swarm cluster resource method for optimizing scheduling based on load estimation |
| US20190197178A1 (en) * | 2017-12-22 | 2019-06-27 | International Business Machines Corporation | Container structure |
| CN108829494A (en) * | 2018-06-25 | 2018-11-16 | 杭州谐云科技有限公司 | Container cloud platform intelligence method for optimizing resources based on load estimation |
| CN109918198A (en) * | 2019-02-18 | 2019-06-21 | 中国空间技术研究院 | A kind of emulation cloud platform load dispatch system and method based on user characteristics prediction |
| US20210141655A1 (en) * | 2019-11-13 | 2021-05-13 | Vmware, Inc. | Unified resource management for containers and virtual machines |
| CN111857953A (en) * | 2020-07-17 | 2020-10-30 | 苏州浪潮智能科技有限公司 | Container cluster management method, device, equipment and readable storage medium |
| CN113110914A (en) * | 2021-03-02 | 2021-07-13 | 西安电子科技大学 | Internet of things platform construction method based on micro-service architecture |
Non-Patent Citations (5)
| Title |
|---|
| KEJIANG YE ET.AL.: "Performance Tuning and Modeling for Big Data Applications in Docker Containers", 2017 INTERNATIONAL CONFERENCE ON NETWORKING, ARCHITECTURE, AND STORAGE (NAS) * |
| YUTONG JI ET.AL.: "Dynamic Resource Scheduling Of Container-based Edge IoT Agents", 2021 INTERNATIONAL WIRELESS COMMUNICATIONS AND MOBILE COMPUTING (IWCMC) * |
| 吴双艳: "基于Docker容器调度优化方法的研究", 中国优秀硕士学位论文全文数据库 (信息科技辑) * |
| 李植;陈莉君;: "一种适于Docker容器资源控制的分类预测方法", 计算机与数字工程, no. 12 * |
| 蔡轶斌: "基于服务质量的服务功能链编排和动态优化方法", 中国优秀硕士学位论文全文数据库 (信息科技辑) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113791863B (en) | 2024-01-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109032801B (en) | Request scheduling method, system, electronic equipment and storage medium | |
| CN112363813A (en) | Resource scheduling method and device, electronic equipment and computer readable medium | |
| CN111431996B (en) | Method, apparatus, device and medium for resource configuration | |
| CN115134371A (en) | Scheduling method, system, equipment and medium containing edge network computing resources | |
| CN109933610B (en) | Data processing method, device, computer equipment and storage medium | |
| CN111131486A (en) | Load adjustment method and device of execution node, server and storage medium | |
| CN113590307A (en) | Edge computing node optimal configuration method and device and cloud computing center | |
| CN109783236A (en) | Method and apparatus for output information | |
| CN114205317A (en) | Service function chain SFC resource allocation method based on SDN and NFV and electronic equipment | |
| CN113791863B (en) | Virtual container-based power Internet of things proxy resource scheduling method and related equipment | |
| CN111858029A (en) | Storm cluster load balancing method and system based on discrete particle swarm | |
| CN106874108A (en) | Thin cloud is minimized in mobile cloud computing use number technology | |
| CN114003238B (en) | Container deployment method, device, equipment and storage medium based on transcoding card | |
| CN116204321A (en) | Model acquisition and model deployment methods, devices, equipment and media | |
| CN115525394A (en) | Method and device for adjusting number of containers | |
| Guo et al. | PARA: Performability‐aware resource allocation on the edges for cloud‐native services | |
| CN113271606B (en) | Service scheduling method for ensuring stability of cloud native mobile network and electronic equipment | |
| CN114567637A (en) | Method and system for intelligently setting weight of load balancing back-end server | |
| CN114168439A (en) | Pressure measurement control method and device for service in cluster, storage medium and terminal | |
| CN112156453B (en) | Example adaptive adjustment method, apparatus, computer readable storage medium and device | |
| CN111538560B (en) | Virtual machine deployment method and device, electronic equipment and storage medium thereof | |
| CN110119300A (en) | The load-balancing method and device of dummy unit cluster | |
| CN113918306A (en) | Method, device, equipment, medium and product for scheduling running resources | |
| CN112039714B (en) | Method and device for minimizing cross-site data analysis cost based on SLA | |
| Yao et al. | An efficient virtual machine allocation algorithm for parallel and distributed simulation applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |