CN114138486A - Containerized micro-service arranging method, system and medium for cloud edge heterogeneous environment - Google Patents
Containerized micro-service arranging method, system and medium for cloud edge heterogeneous environment Download PDFInfo
- Publication number
- CN114138486A CN114138486A CN202111463151.7A CN202111463151A CN114138486A CN 114138486 A CN114138486 A CN 114138486A CN 202111463151 A CN202111463151 A CN 202111463151A CN 114138486 A CN114138486 A CN 114138486A
- Authority
- CN
- China
- Prior art keywords
- micro
- service
- scheduled
- node
- cpu
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 230000003993 interaction Effects 0.000 claims abstract description 45
- 238000001914 filtration Methods 0.000 claims abstract description 24
- 238000012163 sequencing technique Methods 0.000 claims abstract description 12
- 238000010586 diagram Methods 0.000 claims description 19
- 230000008569 process Effects 0.000 claims description 17
- 230000005540 biological transmission Effects 0.000 claims description 16
- 238000003860 storage Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 12
- 239000013256 coordination polymer Substances 0.000 claims description 8
- 230000002452 interceptive effect Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000008447 perception Effects 0.000 abstract description 13
- 238000013468 resource allocation Methods 0.000 abstract description 6
- 238000012545 processing Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 4
- 238000007726 management method Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a containerized micro-service arranging method and a containerized micro-service arranging system for a cloud edge heterogeneous environment, wherein the containerized micro-service arranging method for the cloud edge heterogeneous environment comprises the following steps: ordering the micro-service instances to be scheduled in the queue to be scheduled according to the priority; selecting a micro-service instance to be scheduled with the highest priority as a scheduled micro-service instance; and filtering the nodes with unmatched resources or capabilities in the cluster aiming at the scheduled micro-service instances, then sequencing, selecting the optimal node to deploy the scheduled micro-service instances and distributing the resources. Aiming at the problem that the existing container arrangement system does not support micro-service architecture application service quality perception in the cloud-edge heterogeneous environment, the invention adopts a scheduling and resource allocation method of micro-service interaction perception and heterogeneous node single-thread capability perception, expands the micro-service arrangement capability of the container arrangement system, improves the hardware resource utilization rate in the cloud-edge heterogeneous environment, reduces the whole service delay and improves the service quality.
Description
Technical Field
The invention relates to micro-service technology in the field of computers, in particular to a containerization micro-service arranging method, a containerization micro-service arranging system and a containerization micro-service arranging medium for cloud edge heterogeneous environments.
Background
Under the micro-service architecture, one service application is composed of a large number of micro-services with relatively simple functions, independence, light weight and mutual interaction. Some micro-services are designed to be flexible and scalable, and one micro-service may contain a plurality of same instances, so that higher performance, flexibility, safety, robustness and the like are achieved, and the micro-service is very suitable for a cloud computing paradigm. The micro-service architecture reduces the complexity of large-scale application development and maintenance, accelerates the application iteration speed, is widely applied by a large number of enterprises, and is adopted in domestic large-scale distributed business systems such as Taobao and the like.
Large applications can be generally split into multiple microservices according to functions, and different microservice instances can be deployed in different containers and have a natural distributed characteristic. One microservice is generally reused by multiple different applications, and therefore, complex calling relationships exist between microservices. A large application is divided into a plurality of small independent functional components for micro-servitization, so that although the application development efficiency can be accelerated, the number of the components can be very large, and additional deployment and management burden can be brought.
In order to realize efficient interaction of components, a micro service architecture must include support of API gateway, service discovery, load balancing, and the like, and if there is no efficient bearer, the management cost is huge. Kubernetes is an open source container orchestration system, platform independent, that shields the use details of containers, aiming to deploy container applications more conveniently. Kubernetes provides a large number of automation tools for deploying and managing micro services, improves the deployment and management efficiency of the micro services, and is suitable for bearing the application of a micro service architecture.
The service quality means that after a user request is sent out, the server returns a processing result to the user within a given time, and the shorter the time is, the higher the service quality is, the maximum processing delay has an important influence on the quality of service. Under the microservice architecture, there must be API calls between microservice components of an application. In kubernets, a micro-service component corresponds to a service, one service forwards a request to one or more back-end container units, and the service corresponds to an API gateway, a service discovery and a load balancing function of a micro-service architecture, and one back-end container unit is also a micro-service instance. The communication efficiency of two containers in a cloud platform is related to their relative positions in the cluster, and inter-container communication across nodes is generally inefficient compared to inter-container communication within a node, and the communication efficiency is generally limited by the bandwidth of a physical network or a virtual machine network. The nodes in the edge environment are widely distributed in space, the physical medium, architecture and kinds of networks are various, such as cellular mobile network, WLAN and wired network, and the network bandwidth of these different kinds of networks is greatly different, which results in higher correlation between the efficiency of communication between containers and their positions in the cluster. Under the micro-service architecture, interaction exists between different micro-services, and unreasonable micro-service example placement brings extra interaction overhead. Kubernets do not support interactive perception container scheduling at present, and the influence of a request forwarding strategy of Kubernets and a network topological structure in a cloud side environment on micro-service interaction is not fully considered in existing research, so that the problems that interaction efficiency among services is not high and service quality cannot be guaranteed when micro-services are deployed through the Kubernets are solved.
On the other hand, when deploying a micro-service, since the computing resources of a single node are limited, an important process is to allocate the computing resources to the micro-service instance and schedule the micro-service instance to an appropriate node in the cluster. A large application is composed of a plurality of different micro services and realizes the expected functions, API calls exist among the different micro services, and load relevance exists among the different micro services, namely the performance of one micro service is closely related to the performance of other micro services called by the micro service. When a micro-service contains multiple identical instances, kubernets will distribute the API call request of the micro-service evenly to all the instances, and therefore, there is also a load correlation between multiple instances of a micro-service. In kubernets, each micro-service instance is a container unit, and one or more containers are contained in one container unit, which is the minimum scheduling unit of kubernets, and each container is allocated with a certain amount of resources such as CPU, main memory and the like. Due to the heterogeneity of the clusters, the single-thread capabilities of different nodes are different, and the resource configuration of each micro-service instance is static. When the scheduling schemes of the microservice instances in the cluster are different, different application performances are shown. When the load reaches a certain level, the instance running on the low-performance node will reach the bottleneck first, and meanwhile, the instance load on the high-performance node will be lower, so that the situations of uneven load of the node and the micro-service instance and low resource utilization rate occur. Services are also more prone to additional processing delays since instances on low performance nodes are more prone to reaching performance bottlenecks. Little consideration has been given to the effects of single-threaded capability heterogeneity of nodes in existing research on microservice orchestration.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: aiming at the problem that the existing container arrangement system does not support micro-service architecture application service quality perception under the cloud-edge heterogeneous environment, the invention adopts a scheduling and resource allocation method of micro-service interaction perception and heterogeneous node single-thread capability perception, expands the micro-service arrangement capability of the container arrangement system, improves the hardware resource utilization rate in the cloud-edge heterogeneous environment, reduces the overall service delay and improves the service quality.
In order to solve the technical problems, the invention adopts the technical scheme that:
a containerized micro-service orchestration method for a cloud-edge heterogeneous environment comprises the following steps:
1) ordering the micro-service instances to be scheduled in the queue to be scheduled according to the priority;
2) selecting a micro-service instance to be scheduled with the highest priority as a scheduled micro-service instance;
3) filtering nodes with unmatched resources or capabilities in the cluster aiming at the scheduled microservice instances;
4) sequencing the rest nodes in the cluster according to the network topology structure of the cluster and the micro-service interaction relationship graph;
5) and selecting the sequenced optimal nodes, deploying the scheduled micro-service instances and allocating resources.
Optionally, when the micro-service instances to be scheduled in the queue to be scheduled in step 1) are sorted according to priority, the method includes determining whether there is a direct API call between the micro-service instances to be scheduled and existing loads of a cluster according to a preset micro-service interaction relationship graph, where the micro-service interaction relationship graph is a directed acyclic graph, and is determined in an application design stage, a node in the micro-service interaction relationship graph represents a micro-service, an edge represents an API call direction between the micro-services, and an edge weight refers to an average data size generated by API call, the existing loads of the cluster refer to container units that have completed scheduling, and for any two micro-service instances 1 to be scheduled and micro-service instances 2 to be scheduled: if the micro-service example 1 to be scheduled and the existing load of the cluster have API calls, and the micro-service example 2 to be scheduled and the existing load of the cluster have no API calls, the priority of the micro-service example 1 to be scheduled is greater than that of the micro-service example 2 to be scheduled; if the micro-service example 1 to be scheduled and the micro-service example 2 to be scheduled do not have API call with the existing load of the cluster, judging the size of the sum of the weights of all edges of the micro-service in the interactive relation graph, which are connected by the micro-service example 1 to be scheduled and the micro-service example 2 to be scheduled, and if the sum of the edges corresponding to the micro-service example 1 to be scheduled is greater than the sum of the edges corresponding to the micro-service example 2 to be scheduled, judging that the priority of the micro-service example 1 to be scheduled is greater than the priority of the micro-service example 2 to be scheduled; if API call exists between the micro-service example 1 to be scheduled and the micro-service example 2 to be scheduled and the existing load of the cluster, the sum of the weights of the sides, connected with the existing load of the cluster, of the micro-service example 1 to be scheduled and the micro-service example 2 to be scheduled in the micro-service interaction relation graph is judged, and if the sum of the weights corresponding to the micro-service example 1 to be scheduled is larger than the sum of the weights corresponding to the micro-service example 2 to be scheduled, the priority of the micro-service example 1 to be scheduled is larger than the priority of the micro-service example 2 to be scheduled.
Optionally, the method in step 2) selects the micro-service instance to be scheduled with the highest priority as the micro-service instance to be scheduled, and further includes determining whether the micro-service instance to be scheduled meets a scheduling condition, where the scheduling condition indicates whether there is a micro-service instance with a higher priority and no micro-service instance queue to be scheduled is input, determining whether there is a micro-service instance belonging to the micro-service interaction diagram in an existing load of the cluster according to a preset micro-service interaction diagram and the existing load of the cluster, and if so, and there is no API call relationship between the micro-service instance to be scheduled and the existing load in the cluster, indicating that there is a micro-service instance with a higher priority and no micro-service instance queue to be scheduled is input, and the scheduling condition is not satisfied, an error is returned, the scheduling process is ended, and the micro-service instance to be scheduled rejoins the scheduling queue; otherwise step 3) is entered.
Optionally, when filtering nodes in the cluster whose resources or capabilities are not matched in step 3), the method includes a step of filtering nodes in the cluster whose single thread capability is too low according to the CPU resource configuration values of the containers, and the condition of the nodes whose single thread capability is too low is as follows:
in the above formula, requests.cpu represents the minimum number of resources required for the operation of the container, and the unit is the number of CPUs, and when the numerical value of the requests.cpu is less than 1, the container uses the CPUs in a time-sharing manner according to the proportion represented by the numerical value; k is a radical ofbRepresenting the single-threaded capability of the reference node, kjIndicating the single-threaded capability of any jth node within the cluster,
a rounded-up value representing the minimum number of resources required for the container to run, requests.
Optionally, the single-thread capability k of any jth node in the clusterjAdding the single-thread capability k of any jth node in the cluster into the node information in a text formjThe estimating step comprises: firstly, selecting micro-services with multiple copies, wherein the copies execute the same operation, distribute the same load and are distributed in a plurality of different nodes; then reading the CPU occupancy rate C of the copy on the node jjThe CPU occupancy rate represents the percentage of time of using the CPU, and when the CPU occupancy rate exceeds 100%, the CPU occupancy rate represents that a plurality of CPU cores are used; one of the nodes corresponding to the copies is arbitrarily selected as a reference node, and the single-thread capability k of the reference node is designated0Single-threaded capability k of other nodes, 1 or any other specified valuejThe formula of the calculation function is:
kj＝C0/Cj·k0。
in the above formula, C0CPU occupancy for copies on reference nodes, CjIs the CPU occupancy, k, of the copy on node j0Is the single threaded capability of the reference node.
Optionally, when filtering nodes in the cluster with unmatched resources or capabilities in step 3), the method includes the step of filtering nodes with insufficient resources according to the CPU, the configuration value of the main memory resource, the node residual allocable resources and the node single thread capability:
3.1) traversing all nodes in the cluster, calculating the CPU resources and main memory resources allocated to the nodes, reading all container units running on each node aiming at each node, directly summing the main memory resources, reading the annotation information part in the container unit information aiming at the CPU resources, if the annotation information is not empty, directly accumulating the CPU resources, and if the annotation information is empty, reading the label k in the container unit informationbAnd single threaded capability k of the nodejCPU multiplies the minimum number of resources required for the container operation converted from CPU resources by the tag k in the container unit informationbDivided by the single thread capability k of the nodejThen accumulating to obtain the main memory and CPU resources distributed to the nodes;
3.2) reading CPU resource configuration requests, CPU, main memory resource configuration requests, mem, k of micro service instance to be scheduledbAnd k of the target nodejRespectively accumulating CPU and main memory resource values of each container, and then converting the CPU resource values into requestsb/kj；
3.3) reading the total amount of the CPU and main memory resources which can be distributed by the node, if any resource in the total amount is less than the distributed resource value plus the resource value of the micro service instance to be scheduled, indicating that the resource of the node is insufficient, and filtering the node.
Optionally, step 4) comprises:
4.1) judging whether a scheduled micro service instance and a scheduled micro service instance have an API call relationship in the cluster according to a preset micro service interaction relationship diagram, if not, executing the step 4.2), calculating data transmission delay L generated when the scheduled micro service instance performs API call with the existing micro service instance in the cluster at each node, wherein the deployed micro service instance set is represented as EPs, the scheduled micro service instance is represented as CP, and the maximum value of the data transmission delay of different elements in the EPs called by the CP is LAMaximum delay in data transfer of different elements in the EPs calling the CPA value of LBThe data transmission delay L is estimated to be equal to LAPlus LBSorting all available nodes according to the data transmission delay estimation values, wherein the lower the delay is, the higher the priority of the nodes is; skipping to execute the step 5);
4.2) sequencing the nodes according to the residual resource quantity of the nodes: calculating a coefficient A and a coefficient B, wherein the coefficient A is equal to a main memory resource requested by a scheduled micro-service instance divided by a node residual main memory resource, the coefficient B is equal to a CPU resource requested by a scheduled container divided by a node residual CPU resource, the node residual resource is equal to the total amount of the node resource minus the amount of the allocated resource, the coefficient C is equal to a larger value of the coefficient A and the coefficient B, all nodes are sorted in a descending order according to the size of the coefficient C, and the smaller the coefficient C is, the higher the node priority is.
Optionally, the method further includes that the Etcd database is used for storing the states of all scheduled and to-be-scheduled container units in the cluster, the states are a key-value-key type cache database, the key is a storage path of the container unit, the value is a specific storage object, and the step 5) of selecting the sorted optimal node and deploying the scheduled micro-service instance and allocating resources includes: 5.1) calculating a corresponding key according to the name of the scheduled container unit, reading a corresponding Etcd container unit object from an Etcd database through the key, decoding, and updating the CPU resource configuration in the container unit object: the minimum number of resources required for the container operation per container, requests, cpu, equals the original value of the minimum number of resources required for the container operation, requests, cpu, multiplied by the reference node single thread capability, kbDivided by the target node single thread capability kjThe limit CPU number per container, Limits. CPU, is equal to the original value of the limit CPU number, Limits. CPU, multiplied by the reference node single thread capability, kbDivided by the single-threaded capability k of the target nodej(ii) a 5.2) updating the node name field of the Etcd container unit object to be the node name of the target node, and updating the state field to be scheduled; 5.3) adding a comment to the Etcd container unit object to indicate that the container unit is scheduled by the scheduler; 5.4) encoding the Etcd container unit object, writing the Etcd container unit object into the Etcd database by using the same key, and ending the scheduling process.
In addition, the invention also provides a containerized micro-service orchestration system facing the cloud edge heterogeneous environment, which comprises a microprocessor and a memory which are connected with each other, wherein the microprocessor is programmed or configured to execute the steps of the containerized micro-service orchestration method facing the cloud edge heterogeneous environment.
In addition, the present invention also provides a computer readable storage medium, in which a computer program is stored, and the computer program is used for being executed by a computer device to implement the steps of the containerized micro-service orchestration method for the cloud edge heterogeneous environment.
Compared with the prior art, the invention has the following advantages: aiming at the problem that micro-service interaction cannot be sensed and single-thread capability heterogeneous CPU resources cannot be allocated when a micro-service application is deployed in a cloud-edge heterogeneous environment by the conventional container arrangement system Kubernets, a micro-service interaction relation graph and a network topology graph of a cluster are constructed; before scheduling, sequencing a micro service instance queue to be scheduled, ensuring that the micro service instances loaded with API call relations and micro service instances with high interaction requirements in a cluster are scheduled preferentially, and ensuring that the interaction between the micro service instances with the API call relations has lower data transmission delay; the method comprises the steps of scheduling micro service instances according to the single-thread capability of the nodes, avoiding scheduling the micro service instances to the nodes with low single-thread capability, dynamically allocating CPU resources according to the single-thread capability of the nodes, improving the performance of the micro service instances allocated to the nodes with low single-thread capability, avoiding the condition that the micro service instances show different performances when allocated to different nodes, improving the resource utilization rate of a cluster and the performance of the micro service in a cloud edge heterogeneous environment, and ensuring the service quality.
Drawings
Fig. 1 is a schematic diagram of a system topology oriented to a cloud-edge heterogeneous environment in the embodiment of the present invention.
Fig. 2 is a schematic flow chart of a micro-service orchestration method for a cloud-edge heterogeneous environment according to an embodiment of the present invention.
Fig. 3 is a flowchart of the step of comparing the queue priority of the micro-service instance to be scheduled according to the embodiment of the present invention.
Fig. 4 is a flowchart of a step of selecting a micro service instance to be scheduled from a micro service instance queue to be scheduled according to an embodiment of the present invention.
FIG. 5 is a flowchart of a step of filtering nodes with low single-thread capability in a cluster according to an embodiment of the present invention.
FIG. 6 is a flowchart of a step of filtering nodes with insufficient cluster resources according to an embodiment of the present invention.
FIG. 7 is a flowchart of the steps in sorting available nodes in an embodiment of the present invention.
FIG. 8 is a flowchart illustrating the sorting step according to the expected data transmission delay in an embodiment of the present invention.
FIG. 9 is a flowchart illustrating steps for sorting according to remaining resources of nodes in an embodiment of the present invention.
FIG. 10 is a flowchart of the steps for deploying a microservice instance in an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more clearly understood, the containerization micro-service orchestration method, system and medium oriented to cloud-edge heterogeneous environment of the present invention will be further described in detail below by taking kubernets as an example. As shown in fig. 1, the cloud-side heterogeneous environment in this embodiment includes:
API service: one of the control plane components, all of which interact in an HTTP REST (Representational State Transfer) API manner through an API service;
a controller manager: one of the control plane components comprises a plurality of resource controllers which are responsible for creating and deleting various resources, such as a container unit copy controller which creates a certain number of same container units;
other schedulers: one of the control plane components determining on which node the container unit is deployed, a cluster comprising one or more different schedulers, which scheduler is selected to schedule the container unit to be defined in the corresponding application orchestration script, here all schedulers except the micro service scheduler;
node agent: the system runs in a cloud working node and is responsible for creating and managing container unit instances;
edge proxy: an edge implementation of a node agent is operated in an edge working node, is responsible for communicating with a cloud control plane and creating and managing a container unit instance, and provides additional edge offline autonomy and special functions of an edge environment such as an edge service grid compared with a cloud node agent;
etcd data: one of the control plane components, a key-value type database, stores all the states of the cluster;
node state monitoring component: the CPU occupancy rate information of each node container in the cluster is collected and divided into a client and a server, the client is deployed in each working node, the server is deployed in the cloud, the client sends the collected CPU occupancy rate information of the containers to the server, and the server collects the information and writes the information into an external database;
an external database: storing running state monitoring data of all loads in the cluster, wherein the running state monitoring data comprises CPU occupancy rate, running positions and the like of each container at a certain moment;
a node capability analysis component: reading CPU utilization rate data of container running on each node from external database, and calculating single-thread capability k of node according to the datajThen k is addedjAdding the node label to the node information of the cluster;
the micro-service scheduler: one of the control plane components determines which node the micro-service instance is deployed on, and is only responsible for scheduling the container unit corresponding to the micro-service, wherein one micro-service instance refers to one container unit;
a microservice deployment component: and the user inputs the micro-service interaction relation graph and the cluster network topological graph through the micro-service component, and the micro-service deployment component writes the coded micro-service interaction relation graph and the cluster network topological graph into an Etcd database.
As shown in fig. 2, the containerization micro-service orchestration method for the cloud-edge heterogeneous environment in the embodiment includes:
1) ordering the micro-service instances to be scheduled in the queue to be scheduled according to the priority;
2) selecting a micro-service instance to be scheduled with the highest priority as a scheduled micro-service instance;
3) filtering nodes with unmatched resources or capabilities in the cluster aiming at the scheduled microservice instances;
4) sequencing the rest nodes in the cluster according to the network topology structure of the cluster and the micro-service interaction relationship graph;
5) and selecting the sequenced optimal nodes, deploying the scheduled micro-service instances and allocating resources.
The containerization micro-service arranging method facing the cloud edge heterogeneous environment comprises the steps of ordering micro-service examples to be scheduled in a queue to be scheduled according to priority; selecting a micro-service instance to be scheduled with the highest priority as a scheduled micro-service instance; and filtering the nodes with unmatched resources or capabilities in the cluster aiming at the scheduled micro-service instances, then sequencing, selecting the optimal node to deploy the scheduled micro-service instances and distributing the resources. Aiming at the problem that the existing container arrangement system does not support micro-service architecture application service quality perception in the cloud-edge heterogeneous environment, the invention adopts a scheduling and resource allocation method of micro-service interaction perception and heterogeneous node single-thread capability perception, expands the micro-service arrangement capability of the container arrangement system, improves the hardware resource utilization rate in the cloud-edge heterogeneous environment, reduces the whole service delay and improves the service quality.
In this embodiment, when the micro service instances to be scheduled in the queue to be scheduled are ranked according to priority in step 1), the method includes ranking according to the micro service scheduling relationship and the distribution state of existing micro service instances of the cluster, and selecting the micro service instance to be scheduled with the highest priority to enter a scheduling process, where the micro service instance to be scheduled indicates a container unit in a container cloud platform, the container unit includes one or more containers, and one micro service includes one or more micro service instances.
As shown in fig. 3, when ordering the micro service instances to be scheduled in the queue to be scheduled in step 1) according to priorities, the method includes determining whether there is a direct API call between the micro service instances to be scheduled and existing loads of a cluster according to a preset micro service interaction relationship diagram, where the micro service interaction relationship diagram is a directed acyclic graph, a node in the micro service interaction relationship diagram represents a micro service, an edge represents an API call direction between the micro services, and a weight of the edge indicates an average data size generated by the API call, the existing loads of the cluster indicate container units that have been scheduled, and for any two micro service instances 1 to be scheduled and micro service instances 2 to be scheduled:
if the micro-service example 1 to be scheduled and the existing load of the cluster have API calls, and the micro-service example 2 to be scheduled and the existing load of the cluster have no API calls, the priority of the micro-service example 1 to be scheduled is greater than that of the micro-service example 2 to be scheduled;
if the micro-service example 1 to be scheduled and the micro-service example 2 to be scheduled do not have API call with the existing load of the cluster, judging the size of the sum of the weights of all the edges of the micro-service example 1 to be scheduled and the micro-service example 2 to be scheduled, which are connected in an interactive relation graph of the micro-service, if the sum of the edges corresponding to the micro-service example 1 to be scheduled is greater than the sum of the edges corresponding to the micro-service example 2 to be scheduled, the priority of the micro-service example 1 to be scheduled is greater than the priority of the micro-service example 2 to be scheduled, and the edges in the micro-service interactive relation graph represent the API call direction between the micro-services and the size of data to be transmitted during call;
if API call exists between the micro-service example 1 to be scheduled and the micro-service example 2 to be scheduled and the existing load of the cluster, the sum of the weights of the sides, connected with the existing load of the cluster, of the micro-service example 1 to be scheduled and the micro-service example 2 to be scheduled in the micro-service interaction relation graph is judged, and if the sum of the weights corresponding to the micro-service example 1 to be scheduled is larger than the sum of the weights corresponding to the micro-service example 2 to be scheduled, the priority of the micro-service example 1 to be scheduled is larger than the priority of the micro-service example 2 to be scheduled.
And (3) sequencing the micro service instance queues to be scheduled by the step 1) to obtain the micro service instance with the highest priority.
In this embodiment, the method in step 2) selects the micro service instance to be scheduled with the highest priority as the micro service instance to be scheduled, and further includes determining whether the micro service instance to be scheduled meets a scheduling condition, where the scheduling condition indicates whether there is a micro service instance with a higher priority that is not input into the micro service instance queue to be scheduled, as shown in fig. 4, determining whether there is a micro service instance belonging to the micro service interaction diagram in an existing load of the cluster according to a preset micro service interaction diagram and the existing load of the cluster, and if so, and the micro service instance to be scheduled has no API call relation with the existing load in the cluster, indicating that there is a micro service instance with a higher priority that is not input into the micro service instance queue to be scheduled, and does not meet the scheduling condition, returning an error, ending the scheduling process, and rejoining the scheduling queue with the micro service instance to be scheduled; otherwise step 3) is entered.
In this embodiment, when filtering nodes in the cluster whose resources or capabilities are not matched in step 3), the method includes a step of filtering nodes in the cluster whose single thread capability is too low according to the CPU resource configuration values of the containers, and the condition of the nodes whose single thread capability is too low is as follows:
in the above formula, requests.cpu represents the minimum number of resources required for the operation of the container, and the unit is the number of CPUs, and when the numerical value of the requests.cpu is less than 1, the container uses the CPUs in a time-sharing manner according to the proportion represented by the numerical value; k is a radical ofbRepresenting the single-threaded capability of the reference node, kjIndicating the single-threaded capability of any jth node within the cluster,
a rounded-up value representing the minimum number of resources required for the container to run, requests. As shown in fig. 5, in this step, it is necessary to traverse through minimum requests of CPU resource configurations of all containers in the scheduled microservice instance and determine whether the above condition is satisfied, and if the above condition is satisfied, it indicates that the single thread capability of the node is too low to satisfy the operation requirement of the container, and the node is filtered. The CPU unit is the number of the CPU, and when the value is less than 1, the CPU is used by the container according to the proportion expressed by the value. The single-threaded capability k of a node is a relative value indicating that the single-threaded capability of a node is singleK times of node with thread capability of 1, kbThe single-thread capability of the reference node is shown, the CPU resource allocation basis of the micro-service instance is the reference node, the reference node can be any node in the cluster or an off-cluster test node, and if the single-thread capability of the reference node is not specified in the configuration, the default value is 1, kjRepresenting the single-threaded capability of the jth node within the cluster.
In this embodiment, the single-thread capability k of any jth node in the clusterjAdding the single-thread capability k of any jth node in the cluster into the node information in a text formjThe estimating step comprises: firstly, selecting micro-services with multiple copies, wherein the copies execute the same operation, distribute the same load and are distributed in a plurality of different nodes; then reading the CPU occupancy rate C of the copy on the node jjThe CPU occupancy rate represents the percentage of time of using the CPU, and when the CPU occupancy rate exceeds 100%, the CPU occupancy rate represents that a plurality of CPU cores are used; one of the nodes corresponding to the copies is arbitrarily selected as a reference node, and the single-thread capability k of the reference node is designated0Single-threaded capability k of other nodes, 1 or any other specified valuejThe formula of the calculation function is:
kj＝C0/Cj·k0。
in the above formula, C0CPU occupancy for copies on reference nodes, CjIs the CPU occupancy, k, of the copy on node j0Is the single threaded capability of the reference node.
As shown in fig. 6, when filtering nodes with unmatched resources or capabilities in the cluster in step 3) of this embodiment, the method includes the steps of filtering nodes with insufficient resources according to the CPU, the configuration value of the main storage resource, the remaining allocable resources of the node, and the single thread capability of the node:
3.1) traversing all nodes in the cluster, calculating the CPU resources and main memory resources allocated by the nodes, reading all container units running on each node aiming at each node, directly summing the main memory resources, reading the annotation information part in the container unit information aiming at the CPU resources, if the annotation information is not empty, directly accumulating the CPU resources, and if the annotation information is not empty, directly accumulating the CPU resourcesIf the information is empty, reading the label k in the container unit informationbAnd single threaded capability k of the nodejCPU multiplies the minimum number of resources required for the container operation converted from CPU resources by the tag k in the container unit informationbDivided by the single thread capability k of the nodejThen accumulating to obtain the main memory and CPU resources distributed to the nodes;
3.2) reading CPU resource configuration requests, CPU, main memory resource configuration requests, mem, k of micro service instance to be scheduledbAnd k of the target nodejRespectively accumulating CPU and main memory resource values of each container, and then converting the CPU resource values into requestsb/kj；
3.3) reading the total amount of the CPU and main memory resources which can be distributed by the node, if any resource in the total amount is less than the distributed resource value plus the resource value of the micro service instance to be scheduled, indicating that the resource of the node is insufficient, and filtering the node.
In this embodiment, all container unit information running on the node in step 3.1) is cached in the scheduler, after the optimal node is determined in the scheduling process, the process of binding the node and the container unit is always successful, and in order to improve the performance, before the process of binding the node and the container unit, the container unit successfully scheduled by the scheduler is directly added into the node information cache of the scheduler, so as to be used for scheduling steps related to the cluster state, including waiting for the queue ordering of the micro service instances to be scheduled in step 1), calculating the residual allocable resources of the node in step 3), obtaining the existing load distribution information of the cluster in step 4), and the process of binding the node and the container unit is performed asynchronously.
And 4) sequencing all available nodes according to the expected data transmission delay and the resource residual amount. As shown in fig. 7, step 4) in this embodiment includes:
4.1) judging whether the scheduled micro service instance has an API call relationship with the scheduled micro service instance in the cluster according to a preset micro service interaction relationship diagram, if not, executing the step 4.2), and calculating the scheduled micro service instance at each node and the scheduled micro service instance in the cluster as shown in FIG. 8The maximum value of the data transmission delay L of different elements in the deployed micro-service instance set is represented as EPs, the scheduled micro-service instance is represented as CP, and the CP calls the EPsAThe maximum value of the data transmission delay of different elements in the EPs calling the CP is LBThe data transmission delay L is estimated to be equal to LAPlus LBSorting all available nodes according to the data transmission delay estimation values, wherein the lower the delay is, the higher the priority of the nodes is; skipping to execute the step 5);
4.2) as shown in FIG. 9, the nodes are sorted according to the number of the nodes remaining resources: calculating a coefficient A and a coefficient B, wherein the coefficient A is equal to a main memory resource requested by a scheduled micro-service instance divided by a node residual main memory resource, the coefficient B is equal to a CPU resource requested by a scheduled container divided by a node residual CPU resource, the node residual resource is equal to the total amount of the node resource minus the amount of the allocated resource, the coefficient C is equal to a larger value of the coefficient A and the coefficient B, all nodes are sorted in a descending order according to the size of the coefficient C, and the smaller the coefficient C is, the higher the node priority is.
In this embodiment, the method further includes that the Etcd database is used to store the states of all scheduled and to-be-scheduled container units in the cluster, and the states are a key-value-key type cache database, the key is a storage path of the container unit, and the value is a specific storage object. Before the container unit is not scheduled, the corresponding container unit object node name field in the Etcd is empty, and the key step of binding the container unit and the node is to update the field by using the node name of the optimal node. And the node agent daemon running on the working node judges whether to create the container unit by monitoring the change of the node name field of the container unit object in the Etcd. The target node of the dispatched container unit is obtained only when the node binding step is carried out after the node sequencing step is finished, because the CPU resource is required to be distributed to the container unit according to the single thread capability of the target node, the binding process will update the CPU resource allocation values of the container units in the Etcd database, but the binding process cannot update the container unit information in the scheduler cache, the CPU resource configuration value of the scheduled container unit information added to the scheduler cache is still an un-updated value, in order to distinguish from the scheduled container unit obtained from the API service, adding annotation information into the object of the container unit to be scheduled in the binding process, wherein the annotation information does not influence the operation of the container unit, other container units not deployed by this scheduler are synchronized from the API service to the scheduler cache and these scheduled container unit information is consistent with the Etcd database.
And 5) deploying the micro service instances on the optimal nodes after the optimal nodes are obtained through the scheduling process. As shown in fig. 10, selecting the ranked optimal node in step 5) of this embodiment, deploying the scheduled microservice instance and allocating resources includes:
5.1) calculating a corresponding key according to the name of the scheduled container unit, reading a corresponding Etcd container unit object from an Etcd database through the key, decoding, and updating the CPU resource configuration in the container unit object: the minimum number of resources required for the container operation per container, requests, cpu, equals the original value of the minimum number of resources required for the container operation, requests, cpu, multiplied by the reference node single thread capability, kbDivided by the target node single thread capability kjThe limit CPU number per container, Limits. CPU, is equal to the original value of the limit CPU number, Limits. CPU, multiplied by the reference node single thread capability, kbDivided by the single-threaded capability k of the target nodej；
5.2) updating the node name field of the Etcd container unit object to be the node name of the target node, and updating the state field to be scheduled;
5.3) adding a comment to the Etcd container unit object to indicate that the container unit is scheduled by the scheduler;
5.4) encoding the Etcd container unit object, writing the Etcd container unit object into the Etcd database by using the same key, and ending the scheduling process.
In summary, the method of the embodiment includes constructing a micro-service interaction relationship graph, a cluster network topology graph and a node single-thread capability information base; carrying out priority ordering on the micro service instance queue to be scheduled according to a preset micro service interaction relation graph; filtering single-thread capacity overlow nodes in the cluster according to the CPU resource configuration of the scheduled micro-service instance; filtering the nodes with insufficient resources in the cluster according to the main memory of the scheduled micro-service instance, the CPU resource configuration value and the single-thread capability of the nodes; estimating the interaction delay of the micro-service instance according to the node network topological graph, the micro-service interaction relation graph and the cluster state, and sequencing all available nodes according to the interaction delay and the node residual resources to obtain an optimal node; and deploying the micro-service instance for the micro-service instance according to the single-thread capability of the optimal node, and allocating computing resources. Aiming at the problem that the Kubernets of the conventional container arrangement system does not support micro-service architecture application service quality perception in the cloud-side heterogeneous environment, the invention adopts a scheduling and resource allocation method of micro-service interaction perception and heterogeneous node single-thread capability perception, expands the micro-service arrangement capability of the Kubernets, improves the utilization rate of hardware resources in the cloud-side heterogeneous environment, reduces the whole service delay and improves the service quality.
In addition, the embodiment also provides a containerized micro-service orchestration system facing the cloud-edge heterogeneous environment, which includes a microprocessor and a memory connected to each other, where the microprocessor is programmed or configured to execute the steps of the containerized micro-service orchestration method facing the cloud-edge heterogeneous environment.
In addition, the present embodiment also provides a computer-readable storage medium, in which a computer program is stored, and the computer program is used for being executed by a computer device to implement the steps of the containerized micro-service orchestration method for a cloud-edge-oriented heterogeneous environment.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments, and all technical solutions belonging to the idea of the present invention belong to the protection scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may occur to those skilled in the art without departing from the principle of the invention, and are considered to be within the scope of the invention.
Claims (10)
1. A containerized micro-service orchestration method for a cloud-edge heterogeneous environment is characterized by comprising the following steps:
1) ordering the micro-service instances to be scheduled in the queue to be scheduled according to the priority;
2) selecting a micro-service instance to be scheduled with the highest priority as a scheduled micro-service instance;
3) filtering nodes with unmatched resources or capabilities in the cluster aiming at the scheduled microservice instances;
4) sequencing the rest nodes in the cluster according to the network topology structure of the cluster and the micro-service interaction relationship graph;
5) and selecting the sequenced optimal nodes, deploying the scheduled micro-service instances and allocating resources.
2. The cloud edge heterogeneous environment-oriented containerization micro-service arranging method according to claim 1, wherein, when ordering the micro-service instances to be scheduled in the queue to be scheduled according to the priority in step 1), the method comprises the step of judging whether direct API call exists between the micro-service instances to be scheduled and existing loads of a cluster according to a preset micro-service interaction relation graph, wherein the micro-service interaction relation graph is a directed acyclic graph, nodes in the micro-service interaction relation graph represent micro-services, edges represent API call directions between the micro-services, and weight of the edges indicates average data size generated by API call, the existing loads of the cluster indicate scheduled container units, and aiming at any two micro-service instances to be scheduled 1 and micro-service instances to be scheduled 2: if the micro-service example 1 to be scheduled and the existing load of the cluster have API calls, and the micro-service example 2 to be scheduled and the existing load of the cluster have no API calls, the priority of the micro-service example 1 to be scheduled is greater than that of the micro-service example 2 to be scheduled; if the micro-service example 1 to be scheduled and the micro-service example 2 to be scheduled do not have API call with the existing load of the cluster, judging the size of the sum of the weights of all edges of the micro-service in the interactive relation graph, which are connected by the micro-service example 1 to be scheduled and the micro-service example 2 to be scheduled, and if the sum of the edges corresponding to the micro-service example 1 to be scheduled is greater than the sum of the edges corresponding to the micro-service example 2 to be scheduled, judging that the priority of the micro-service example 1 to be scheduled is greater than the priority of the micro-service example 2 to be scheduled; if API call exists between the micro-service example 1 to be scheduled and the micro-service example 2 to be scheduled and the existing load of the cluster, the sum of the weights of the sides, connected with the existing load of the cluster, of the micro-service example 1 to be scheduled and the micro-service example 2 to be scheduled in the micro-service interaction relation graph is judged, and if the sum of the weights corresponding to the micro-service example 1 to be scheduled is larger than the sum of the weights corresponding to the micro-service example 2 to be scheduled, the priority of the micro-service example 1 to be scheduled is larger than the priority of the micro-service example 2 to be scheduled.
3. The containerized micro-service orchestration method for cloud-edge heterogeneous environments according to claim 2, it is characterized in that the micro-service instance to be scheduled with the highest priority is selected as the scheduled micro-service instance in the step 2), and the method also comprises the steps of judging whether the scheduled micro-service instance meets the scheduling condition, the scheduling condition refers to whether the micro service instances with higher priority are not input into the micro service instance queue to be scheduled, judging whether the micro service instance belonging to the micro service interaction relation graph exists in the existing cluster load according to the preset micro service interaction relation graph and the existing cluster load, if so, and the micro-service instances to be scheduled have no API call relation with the existing loads in the cluster, which shows that the micro-service instances with higher priority are not input into the micro-service instance queue to be scheduled, the scheduling conditions are not met, an error is returned, the scheduling process is ended, and the micro-service instances to be scheduled are added into the scheduling queue again; otherwise step 3) is entered.
4. The cloud-edge heterogeneous environment-oriented containerized micro-service orchestration method according to claim 3, wherein when filtering nodes in the cluster where resources or capabilities are not matched in step 3), the method comprises a step of filtering nodes with too low single-thread capability in the cluster according to the CPU resource configuration values of the containers, and the condition of the nodes with too low single-thread capability is as follows:
in the above formula, requests.cpu represents the minimum number of resources required for the operation of the container, and the unit is the number of CPUs, and when the numerical value of the requests.cpu is less than 1, the container uses the CPUs in a time-sharing manner according to the proportion represented by the numerical value; k is a radical ofbRepresenting the single-threaded capability of the reference node, kjIndicating the single-threaded capability of any jth node within the cluster,
a rounded-up value representing the minimum number of resources required for the container to run, requests.
5. The cloud edge heterogeneous environment-oriented containerized micro-service orchestration method according to claim 4, wherein the single-thread capability k of any jth node in the clusterjAdding the single-thread capability k of any jth node in the cluster into the node information in a text formjThe estimating step comprises: firstly, selecting micro-services with multiple copies, wherein the copies execute the same operation, distribute the same load and are distributed in a plurality of different nodes; then reading the CPU occupancy rate C of the copy on the node jjThe CPU occupancy rate represents the percentage of time of using the CPU, and when the CPU occupancy rate exceeds 100%, the CPU occupancy rate represents that a plurality of CPU cores are used; one of the nodes corresponding to the copies is arbitrarily selected as a reference node, and the single-thread capability k of the reference node is designated0Single-threaded capability k of other nodes, 1 or any other specified valuejThe formula of the calculation function is:
kj＝C0/Cj·k0。
in the above formula, C0CPU occupancy for copies on reference nodes, CjIs the CPU occupancy, k, of the copy on node j0Is the single threaded capability of the reference node.
6. The cloud edge heterogeneous environment-oriented containerized micro-service orchestration method according to claim 5, wherein when filtering nodes in the cluster where resources or capabilities are not matched in step 3), the method comprises the steps of filtering nodes with insufficient resources according to a CPU, a main memory resource configuration value, node residual allocable resources and node single-thread capabilities:
3.1) traversing all nodes in the cluster, computing the allocated CPU resources and main memory resources of the nodes, and aiming at each node,reading all container units running on the node, directly summing the main memory resources, reading the annotation information part in the container unit information for the CPU resources, if the annotation information is not empty, directly accumulating the CPU resources, and if the annotation information is empty, reading the label k in the container unit informationbAnd single threaded capability k of the nodejCPU multiplies the minimum number of resources required for the container operation converted from CPU resources by the tag k in the container unit informationbDivided by the single thread capability k of the nodejThen accumulating to obtain the main memory and CPU resources distributed to the nodes;
3.2) reading CPU resource configuration requests, CPU, main memory resource configuration requests, mem, k of micro service instance to be scheduledbAnd k of the target nodejRespectively accumulating CPU and main memory resource values of each container, and then converting the CPU resource values into requestsb/kj；
3.3) reading the total amount of the CPU and main memory resources which can be distributed by the node, if any resource in the total amount is less than the distributed resource value plus the resource value of the micro service instance to be scheduled, indicating that the resource of the node is insufficient, and filtering the node.
7. The cloud-edge heterogeneous environment-oriented containerized micro-service orchestration method according to claim 6, wherein step 4) comprises:
4.1) judging whether a scheduled micro service instance and a scheduled micro service instance have an API call relationship in the cluster according to a preset micro service interaction relationship diagram, if not, executing the step 4.2), calculating data transmission delay L generated when the scheduled micro service instance performs API call with the existing micro service instance in the cluster at each node, wherein the deployed micro service instance set is represented as EPs, the scheduled micro service instance is represented as CP, and the maximum value of the data transmission delay of different elements in the EPs called by the CP is LAThe maximum value of the data transmission delay of different elements in the EPs calling the CP is LBThe data transmission delay L is estimated to be equal to LAPlus LBAccording to the estimated value of data transmission delayAvailable nodes are sorted, and the lower the delay is, the higher the priority of the nodes is; skipping to execute the step 5);
4.2) sequencing the nodes according to the residual resource quantity of the nodes: calculating a coefficient A and a coefficient B, wherein the coefficient A is equal to a main memory resource requested by a scheduled micro-service instance divided by a node residual main memory resource, the coefficient B is equal to a CPU resource requested by a scheduled container divided by a node residual CPU resource, the node residual resource is equal to the total amount of the node resource minus the amount of the allocated resource, the coefficient C is equal to a larger value of the coefficient A and the coefficient B, all nodes are sorted in a descending order according to the size of the coefficient C, and the smaller the coefficient C is, the higher the node priority is.
8. The cloud-edge heterogeneous environment-oriented containerized micro-service orchestration method according to claim 7, further comprising the steps of storing the states of all scheduled and to-be-scheduled container units in the cluster by using an Etcd database, wherein the states are one key-value type cache databases, the key is a storage path of the container unit, the value is a specific storage object, selecting the sorted optimal node in step 5), deploying the scheduled micro-service instance, and allocating resources comprises:
5.1) calculating a corresponding key according to the name of the scheduled container unit, reading a corresponding Etcd container unit object from an Etcd database through the key, decoding, and updating the CPU resource configuration in the container unit object: the minimum number of resources required for the container operation per container, requests, cpu, equals the original value of the minimum number of resources required for the container operation, requests, cpu, multiplied by the reference node single thread capability, kbDivided by the target node single thread capability kjThe limit CPU number per container, Limits. CPU, is equal to the original value of the limit CPU number, Limits. CPU, multiplied by the reference node single thread capability, kbDivided by the single-threaded capability k of the target nodej；
5.2) updating the node name field of the Etcd container unit object to be the node name of the target node, and updating the state field to be scheduled;
5.3) adding a comment to the Etcd container unit object to indicate that the container unit is scheduled by the scheduler;
5.4) encoding the Etcd container unit object, writing the Etcd container unit object into the Etcd database by using the same key, and ending the scheduling process.
9. A containerized micro-service orchestration system for a cloud-edge-oriented heterogeneous environment, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to perform the steps of the containerized micro-service orchestration method for a cloud-edge-oriented heterogeneous environment according to any one of claims 1-8.
10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and the computer program is used for being executed by a computer device to implement the steps of the containerized micro-service orchestration method for a cloud-edge heterogeneous environment according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111463151.7A CN114138486B (en) | 2021-12-02 | 2021-12-02 | Method, system and medium for arranging containerized micro-services for cloud edge heterogeneous environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111463151.7A CN114138486B (en) | 2021-12-02 | 2021-12-02 | Method, system and medium for arranging containerized micro-services for cloud edge heterogeneous environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114138486A true CN114138486A (en) | 2022-03-04 |
| CN114138486B CN114138486B (en) | 2024-03-26 |
Family
ID=80387308
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111463151.7A Active CN114138486B (en) | 2021-12-02 | 2021-12-02 | Method, system and medium for arranging containerized micro-services for cloud edge heterogeneous environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114138486B (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114860401A (en) * | 2022-05-07 | 2022-08-05 | 中国电信股份有限公司 | Heterogeneous cloud desktop scheduling system, method, service system, device and medium |
| CN114866544A (en) * | 2022-04-02 | 2022-08-05 | 中国人民解放军国防科技大学 | Containerized micro-service load balancing method for CPU heterogeneous cluster in cloud edge environment |
| CN114915663A (en) * | 2022-03-21 | 2022-08-16 | 北京沃东天骏信息技术有限公司 | Request response method, device, system, electronic equipment and medium |
| CN115022928A (en) * | 2022-06-02 | 2022-09-06 | 上海中传网络技术股份有限公司 | Resource cooperative allocation method and device in mobile edge computing network |
| CN115114052A (en) * | 2022-08-30 | 2022-09-27 | 北京易诚互动网络技术股份有限公司 | Method and device for intelligently providing database microservice |
| CN115277863A (en) * | 2022-06-27 | 2022-11-01 | 中盈优创资讯科技有限公司 | Scheduling method of API (application program interface) route |
| CN115379000A (en) * | 2022-10-26 | 2022-11-22 | 小米汽车科技有限公司 | Service platform capacity expansion method and device, electronic equipment and storage medium |
| CN115426365A (en) * | 2022-08-17 | 2022-12-02 | 西安理工大学 | Cluster scheduling method based on generic computation architecture |
| CN115801694A (en) * | 2022-11-16 | 2023-03-14 | 浪潮云信息技术股份公司 | Kubernet scheduler design method and system based on network perception |
| CN115858168A (en) * | 2022-08-29 | 2023-03-28 | 苏州空天信息研究院 | Earth application model arrangement system and method based on importance ranking |
| CN116095177A (en) * | 2023-02-03 | 2023-05-09 | 西安交通大学 | Hierarchical clustering scheduling method, system, medium and equipment |
| CN116170518A (en) * | 2023-04-26 | 2023-05-26 | 北京太极信息系统技术有限公司 | Method and equipment for cloud cross-architecture management of domestic chip container |
| WO2024021467A1 (en) * | 2022-07-26 | 2024-02-01 | 天翼云科技有限公司 | Cluster resource planning method, device, apparatus, and medium |
| WO2024159734A1 (en) * | 2023-02-03 | 2024-08-08 | 华为技术有限公司 | Container scheduling method and related device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108829509A (en) * | 2018-05-03 | 2018-11-16 | 山东汇贸电子口岸有限公司 | Distributed container cluster framework resources management method based on domestic CPU and operating system |
| CN112835714A (en) * | 2021-01-29 | 2021-05-25 | 中国人民解放军国防科技大学 | Container arrangement method, system and medium for CPU heterogeneous cluster in cloud edge environment |
-
2021
- 2021-12-02 CN CN202111463151.7A patent/CN114138486B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108829509A (en) * | 2018-05-03 | 2018-11-16 | 山东汇贸电子口岸有限公司 | Distributed container cluster framework resources management method based on domestic CPU and operating system |
| CN112835714A (en) * | 2021-01-29 | 2021-05-25 | 中国人民解放军国防科技大学 | Container arrangement method, system and medium for CPU heterogeneous cluster in cloud edge environment |
Non-Patent Citations (1)
| Title |
|---|
| THOMAS RAUSCH等: "Optimized container scheduling for data-intensive serverless edge computing", FUTURE GENERATION COMPUTER SYSTEMS, vol. 114, pages 259 - 271, XP086285798, DOI: 10.1016/j.future.2020.07.017 * |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114915663A (en) * | 2022-03-21 | 2022-08-16 | 北京沃东天骏信息技术有限公司 | Request response method, device, system, electronic equipment and medium |
| CN114866544A (en) * | 2022-04-02 | 2022-08-05 | 中国人民解放军国防科技大学 | Containerized micro-service load balancing method for CPU heterogeneous cluster in cloud edge environment |
| CN114866544B (en) * | 2022-04-02 | 2023-10-03 | 中国人民解放军国防科技大学 | CPU heterogeneous cluster-oriented containerized micro-service load balancing method in cloud edge environment |
| CN114860401A (en) * | 2022-05-07 | 2022-08-05 | 中国电信股份有限公司 | Heterogeneous cloud desktop scheduling system, method, service system, device and medium |
| CN114860401B (en) * | 2022-05-07 | 2023-10-03 | 中国电信股份有限公司 | Heterogeneous cloud desktop scheduling system, heterogeneous cloud desktop scheduling method, heterogeneous cloud desktop scheduling service system, heterogeneous cloud desktop scheduling device and medium |
| CN115022928A (en) * | 2022-06-02 | 2022-09-06 | 上海中传网络技术股份有限公司 | Resource cooperative allocation method and device in mobile edge computing network |
| CN115277863A (en) * | 2022-06-27 | 2022-11-01 | 中盈优创资讯科技有限公司 | Scheduling method of API (application program interface) route |
| CN115277863B (en) * | 2022-06-27 | 2024-05-14 | 中盈优创资讯科技有限公司 | Scheduling method of API (application program interface) route |
| WO2024021467A1 (en) * | 2022-07-26 | 2024-02-01 | 天翼云科技有限公司 | Cluster resource planning method, device, apparatus, and medium |
| CN115426365A (en) * | 2022-08-17 | 2022-12-02 | 西安理工大学 | Cluster scheduling method based on generic computation architecture |
| CN115858168B (en) * | 2022-08-29 | 2023-10-13 | 苏州空天信息研究院 | Earth application model arrangement system and method based on importance ranking |
| CN115858168A (en) * | 2022-08-29 | 2023-03-28 | 苏州空天信息研究院 | Earth application model arrangement system and method based on importance ranking |
| CN115114052A (en) * | 2022-08-30 | 2022-09-27 | 北京易诚互动网络技术股份有限公司 | Method and device for intelligently providing database microservice |
| CN115379000A (en) * | 2022-10-26 | 2022-11-22 | 小米汽车科技有限公司 | Service platform capacity expansion method and device, electronic equipment and storage medium |
| CN115379000B (en) * | 2022-10-26 | 2023-03-24 | 小米汽车科技有限公司 | Service platform capacity expansion method and device, electronic equipment and storage medium |
| CN115801694A (en) * | 2022-11-16 | 2023-03-14 | 浪潮云信息技术股份公司 | Kubernet scheduler design method and system based on network perception |
| CN116095177A (en) * | 2023-02-03 | 2023-05-09 | 西安交通大学 | Hierarchical clustering scheduling method, system, medium and equipment |
| WO2024159734A1 (en) * | 2023-02-03 | 2024-08-08 | 华为技术有限公司 | Container scheduling method and related device |
| CN116170518A (en) * | 2023-04-26 | 2023-05-26 | 北京太极信息系统技术有限公司 | Method and equipment for cloud cross-architecture management of domestic chip container |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114138486B (en) | 2024-03-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114138486B (en) | Method, system and medium for arranging containerized micro-services for cloud edge heterogeneous environment | |
| CN107273185B (en) | Load balancing control method based on virtual machine | |
| CN110297699B (en) | Scheduling method, scheduler, storage medium and system | |
| Bicer et al. | Time and cost sensitive data-intensive computing on hybrid clouds | |
| Ajit et al. | VM level load balancing in cloud environment | |
| CN108762896A (en) | One kind being based on Hadoop cluster tasks dispatching method and computer equipment | |
| KR101471749B1 (en) | Virtual machine allcoation of cloud service for fuzzy logic driven virtual machine resource evaluation apparatus and method | |
| CN113454614A (en) | System and method for resource partitioning in distributed computing | |
| Ullah et al. | Task classification and scheduling based on K-means clustering for edge computing | |
| CN110221920B (en) | Deployment method, device, storage medium and system | |
| Li et al. | Enabling elastic stream processing in shared clusters | |
| CN115408100A (en) | Container cluster scheduling method, device, equipment and storage medium | |
| Sonkar et al. | A review on resource allocation and VM scheduling techniques and a model for efficient resource management in cloud computing environment | |
| US20210390405A1 (en) | Microservice-based training systems in heterogeneous graphic processor unit (gpu) cluster and operating method thereof | |
| Komarasamy et al. | A novel approach for Dynamic Load Balancing with effective Bin Packing and VM Reconfiguration in cloud | |
| CN116627661B (en) | Method and system for scheduling computing power resources | |
| CN107070965A (en) | A kind of Multi-workflow resource provision method virtualized under container resource | |
| Aarthee et al. | Energy-aware heuristic scheduling using bin packing mapreduce scheduler for heterogeneous workloads performance in big data | |
| Lin et al. | Joint deadline-constrained and influence-aware design for allocating MapReduce jobs in cloud computing systems | |
| Ru et al. | Providing fairer resource allocation for multi-tenant cloud-based systems | |
| Shubair | Enhancement of task scheduling technique of big data cloud computing | |
| Miranda et al. | Dynamic communication-aware scheduling with uncertainty of workflow applications in clouds | |
| Fernández-Cerero et al. | Quality of cloud services determined by the dynamic management of scheduling models for complex heterogeneous workloads | |
| Hanif et al. | Jargon of Hadoop MapReduce scheduling techniques: a scientific categorization | |
| US11233738B1 (en) | Dynamic performance configuration for data traffic workflows |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |