CN114398148A - Power industry K8S dynamic container arrangement method and storage medium - Google Patents
Power industry K8S dynamic container arrangement method and storage medium Download PDFInfo
- Publication number
- CN114398148A CN114398148A CN202111644965.0A CN202111644965A CN114398148A CN 114398148 A CN114398148 A CN 114398148A CN 202111644965 A CN202111644965 A CN 202111644965A CN 114398148 A CN114398148 A CN 114398148A
- Authority
- CN
- China
- Prior art keywords
- task
- resource
- container
- classification
- slt
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 16
- 238000007637 random forest analysis Methods 0.000 claims abstract description 16
- 238000013145 classification model Methods 0.000 claims description 7
- 238000012544 monitoring process Methods 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims description 3
- 238000004422 calculation algorithm Methods 0.000 claims description 2
- 238000004590 computer program Methods 0.000 claims 1
- 230000005611 electricity Effects 0.000 claims 1
- 238000004364 calculation method Methods 0.000 abstract description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/24323—Tree-organised classifiers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Economics (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- Evolutionary Computation (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a K8S dynamic container arranging method and a storage medium in the power industry, aiming at the application characteristics of high availability, high safety, high concurrency and intranet in the power industry, by researching the task characteristics and occupied resources of clusters and containers, according to the load of the clusters and the load condition of the similar containers, based on an improved random forest model, the tasks in the same category are uniformly distributed to proper resource nodes in a sub-cluster to be executed, and when the load of the sub-cluster reaches a threshold value, a new sub-cluster is restarted, so that the dynamic, efficient and rapid arrangement of the containers is realized. The invention realizes the maximum utilization of the existing resources for the containers and solves the problem of optimal arrangement of the calculation containers under the condition of limited resources.
Description
Technical Field
The invention relates to the field of power industry Internet application, in particular to a power industry K8S dynamic container arrangement method and a storage medium, and especially relates to a power industry K8S dynamic container arrangement method and a storage medium applying improved random forests.
Background
Generally, the power industry has a certain specificity, and the application is required to have high availability, high safety and high concurrency, and can only be applied in an intranet. Many of the K8S container allocation strategies that are available on the internet are not applicable to the power industry. Docker is used as a de facto standard of containerization, each container is allocated with corresponding resources as a basic resource task scheduling unit, the containers encapsulate runtime of the whole software, and DevOps integrated management can be realized. However, the load degree problem differential distribution of the current container arrangement on the resource nodes has problems, different task characteristics are ignored, the K8S container distribution is easy to unbalance, and the problems of low utilization of system resources, low execution speed, long execution time and the like are caused.
Aiming at the container distribution condition under the power grid K8S environment, the aspects of task attribute, load balance and the like need to be researched from the aspects of container engine management, performance monitoring, anomaly detection, network configuration, I/O performance isolation, Web deployment and the like,
disclosure of Invention
Aiming at the problems, the inventor provides a dynamic container arrangement method for the power industry K8S, and the inventor provides a dynamic weighting distribution algorithm for updating the weight of the resource nodes according to the utilization rate of each resource node and different service requests by analyzing the problems of unbalanced cluster load, low efficiency of resource utilization rate and the like, and carries out arrangement of the K8S container. The container arrangement is carried out by adopting an improved random forest mode, the container is carried out by utilizing the existing resources to the maximum extent, and the problem of the optimized arrangement of the calculation container under the condition of limited resources is solved.
The invention provides a dynamic container arrangement method for an electric power industry K8S, which comprises the following steps:
step S1: the Docker task is classified. Dividing the tasks with the execution duration between (0,50000) into 5 different levels, namely a Super Long Task (Super Long Task, SLT), a Long Task (Long Task, LT), a Normal Task (Normal Task, NT), a Short Task (Short Task, ST) and an ultra Short Task (Super Short Task, SST), taking the five levels as five class labels of a container Task set, wherein the binary labels of the five class labels are shown in the following table:
| binary labeling of tasks | [10000] | [01000] | [00100] | [00010] | [00001] |
| Task type | SLT | LT | NT | ST | SST |
Therefore, the Class attribute set of the container task set is denoted by Class { SLT, LT, NT, ST, SST }.
Step S2: and (5) taking the Class set of the step S1 as a classification condition, and classifying the tasks according to a task classification model trained by the improved random forest classifier, wherein the classification process is divided into two steps.
Step S21: and constructing a task set with class labels, creating a Docker task container, and putting the Docker task container into a task resource pool to form the task set.
Step S22: and taking the task set with the class label as an input parameter, and returning to the optimal random forest classification model. After training of a container cloud task classification model based on an improved random forest is completed, n base classifiers are represented, leaf nodes of each base classifier represent one classification result, if [10000] represents a first classification result, namely SLT, and [01000] represents a second classification result, namely LT, and so on, the result is the final classification of the task.
In step S3, the final result of step S2 is used by K8S. The Task set Task of the request is divided into subtask sets Docker _ Task of different execution time lengths { SLT, LT, NT, ST, SST }. The monitoring platform carried by the K8S acquires the Resource usage of each Resource node in the cluster, and forms a set Resource { R1, R2, R3R 4 }. K8S evaluates whether the assignment of the batch of tasks to the resource nodes would be overloaded, and if not, assigns as planned. If the resource nodes are overloaded, the resource nodes which are possibly overloaded are removed from the resource set, whether the task is equally divided to the rest resource nodes is overloaded or not is re-evaluated, and the steps are repeated until the resource set is empty or the task set is empty, and if the task set is empty.
The invention has the beneficial effects that:
by researching task characteristics and occupied resources of clusters and containers aiming at application characteristics of high availability, high safety, high concurrency and intranet in the power industry, the power industry K8S dynamic container arrangement method for improving random forests has remarkable effect in load balancing or minimum time span, can be widely applied to scenes of large-specification docker container arrangement, and improves the utilization rate of resources.
Drawings
FIG. 1 is a schematic diagram of a container cloud task classification model of an improved random forest.
FIG. 2 is a flow chart of the method of the present invention.
Detailed Description
Referring to fig. 1, the invention relates to a power industry K8S dynamic container arrangement method for improving random forests, which comprises the following steps:
step S1: the Docker task is classified. Dividing the tasks with the execution duration (ms) between (0,50000) into 5 different levels, namely a Super Long Task (Super Long Task, SLT), a Long Task (Long Task, LT), a Normal Task (Normal Task, NT), a Short Task (Short Task, ST) and an ultra Short Task (Super Short Task, SST), taking the five levels as five class labels of a container Task set, wherein the binary labels of the five class labels are shown in the following table:
| binary labeling of tasks | [10000] | [01000] | [00100] | [00010] | [00001] |
| Task type | SLT | LT | NT | ST | SST |
Therefore, the Class attribute set of the container task set is denoted by Class { SLT, LT, NT, ST, SST }.
Step S2: and (3) training a task classification model according to an improved random forest classifier to classify the tasks, wherein the classification process is divided into two steps.
Step S21: and constructing a Task set with class labels, creating a Docker Task container, and putting the Docker Task container into a Task resource pool to form an unordered Task set order _ Docker _ Task ═ { LT, NT, SLT, SST, ST }.
Step S22: and taking the task set with the class label as an input parameter, and returning the ordered task set of the optimal random forest classification. After training of a container cloud Task classification model based on an improved random forest is completed, n base classifiers are represented, leaf nodes of each base classifier represent one classification result, if [10000] represents a first classification result, namely SLT, and [01000] represents a second classification result, namely LT, and so on, the concrete correspondence is as shown in FIG. 1, the result is the final classification of the Task, and an ordered set order _ Docker _ Task ═ SLT, LT, NT, ST, SST } is obtained.
In step S3, K8S utilizes the final result. The Task set Task of the request is divided into subtask sets order _ Docker _ Task with different execution time lengths { SLT, LT, NT, ST, SST }. The monitoring platform carried by the K8S acquires the Resource usage of each Resource node in the cluster, and forms a set Resource { R1, R2, R3R 4 }. K8S evaluates whether the assignment of the batch of tasks to the resource nodes would be overloaded, and if not, assigns as planned. If the resource nodes are overloaded, the resource nodes which are possibly overloaded are removed from the resource set, whether the task is equally divided to the rest resource nodes is overloaded or not is re-evaluated, and the steps are repeated until the resource set is empty or the task set is empty, and if the task set is empty.
Claims (3)
1. A dynamic container arrangement method for the power industry K8S applies a random forest algorithm, and is characterized by comprising the following steps:
step S1, classifying the tasks with execution duration (0,50000) according to the execution time of the Docker container Task, dividing the tasks with execution duration (0,50000) into 5 different levels, which are respectively an ultra-Long Task (Super Long Task, SLT), a Long Task (Long Task, LT), a general Task (Normal Task, NT), a Short Task (Short Task, ST), and an ultra-Short Task (Super Short Task, SST), and using the five levels as five class labels of the container Task set, wherein the binary labels of the five class labels are shown in the following table:
Therefore, the Class attribute set of the container task set is denoted as Class { SLT, LT, NT, ST, SST };
step S2, taking the Class set of step S1 as a classification condition, and classifying the tasks according to an improved random forest classifier training task classification model, wherein the classification process comprises two steps:
step S21, constructing a task set with class labels, creating a Docker task container, and putting the Docker task container into a task resource pool to form the task set;
step S22, taking the task set with class labels as input parameters and returning to the optimal random forest classification model;
step S3, dividing the requested Task set Task into sub Task sets Docker _ Task ═ { SLT, LT, NT, ST, SST }, with different execution time lengths; acquiring Resource use conditions of Resource nodes in the cluster by a monitoring platform carried by K8S, and forming a set Resource { R1, R2, R3R 4 }; K8S evaluating whether the task is overloaded when being distributed to each resource node, if not, distributing according to plan; if the resource nodes are overloaded, the resource nodes which are possibly overloaded are removed from the resource set, whether the task is equally divided to the rest resource nodes is overloaded or not is re-evaluated, and the steps are repeated until the resource set is empty or the task set is empty, and if the task set is empty.
2. The method according to claim 1, wherein step S22 further comprises:
after training of a container cloud task classification model based on an improved random forest is completed, n base classifiers are represented, leaf nodes of each base classifier represent one classification result, if [10000] represents a first classification result, namely SLT, and [01000] represents a second classification result, namely LT, and so on, the result is the final classification of the task.
3. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method for dynamic container arrangement of the electricity industry K8S according to claim 1 or 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111644965.0A CN114398148A (en) | 2021-12-30 | 2021-12-30 | Power industry K8S dynamic container arrangement method and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111644965.0A CN114398148A (en) | 2021-12-30 | 2021-12-30 | Power industry K8S dynamic container arrangement method and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114398148A true CN114398148A (en) | 2022-04-26 |
Family
ID=81229713
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111644965.0A Pending CN114398148A (en) | 2021-12-30 | 2021-12-30 | Power industry K8S dynamic container arrangement method and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114398148A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115426365A (en) * | 2022-08-17 | 2022-12-02 | 西安理工大学 | Cluster scheduling method based on generic computation architecture |
-
2021
- 2021-12-30 CN CN202111644965.0A patent/CN114398148A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115426365A (en) * | 2022-08-17 | 2022-12-02 | 西安理工大学 | Cluster scheduling method based on generic computation architecture |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110413389B (en) | Task scheduling optimization method under resource imbalance Spark environment | |
| CN108829494A (en) | Container cloud platform intelligence method for optimizing resources based on load estimation | |
| CN107861796B (en) | Virtual machine scheduling method supporting energy consumption optimization of cloud data center | |
| CN103873569A (en) | Resource optimized deployment method based on IaaS (infrastructure as a service) cloud platform | |
| CN103475538A (en) | Multi-interface-based self-adaptive cloud service test method | |
| Domanal et al. | Load balancing in cloud environment using a novel hybrid scheduling algorithm | |
| CN107086929B (en) | Batch streaming computing system performance guarantee method based on queuing modeling | |
| Javadpour et al. | An intelligent energy-efficient approach for managing IoE tasks in cloud platforms | |
| CN104484222A (en) | Virtual machine dispatching method based on hybrid genetic algorithm | |
| CN109901932A (en) | A kind of Server Consolidation method based on virtual machine | |
| CN108446179A (en) | Priority weights Task method for scheduling task based on load-balancing mechanism | |
| Shukla et al. | FAT-ETO: Fuzzy-AHP-TOPSIS-Based efficient task offloading algorithm for scientific workflows in heterogeneous fog–cloud environment | |
| CN114398148A (en) | Power industry K8S dynamic container arrangement method and storage medium | |
| CN105872082B (en) | Fine granularity resource response system based on container cluster load-balancing algorithm | |
| Zhang et al. | An energy-aware host resource management framework for two-tier virtualized cloud data centers | |
| Prajapati et al. | QoS Based Virtual Machine Consolidation for Energy Efficient and Economic Utilization of Cloud Resources | |
| AlMansour et al. | A survey of scheduling algorithms in cloud computing | |
| Zhu et al. | A multi-resource scheduling scheme of Kubernetes for IIoT | |
| CN118193188A (en) | Dynamic load balancing system and method based on virtualization platform | |
| CN117687774A (en) | Task model training method for computing power scheduling and computing power scheduling method and system | |
| Xu et al. | Hybrid scheduling deadline-constrained multi-DAGs based on reverse HEFT | |
| CN114417577A (en) | Cross-platform resource scheduling and optimization control method | |
| CN115202810A (en) | Kubernetes working node distribution method and system | |
| CN113191534A (en) | Logistics resource allocation method, device, equipment and storage medium | |
| Chitgar et al. | DSCTS: Dynamic stochastic cloud task scheduling |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |