CN113742033A - Kubernetes cluster federal system and implementation method thereof - Google Patents
Kubernetes cluster federal system and implementation method thereof Download PDFInfo
- Publication number
- CN113742033A CN113742033A CN202111048076.8A CN202111048076A CN113742033A CN 113742033 A CN113742033 A CN 113742033A CN 202111048076 A CN202111048076 A CN 202111048076A CN 113742033 A CN113742033 A CN 113742033A
- Authority
- CN
- China
- Prior art keywords
- cluster
- controller
- resource
- slave
- federal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 230000001360 synchronised effect Effects 0.000 claims abstract description 49
- 238000004891 communication Methods 0.000 claims abstract description 9
- 230000008859 change Effects 0.000 claims description 13
- 230000036541 health Effects 0.000 claims description 13
- 238000012217 deletion Methods 0.000 claims description 5
- 230000037430 deletion Effects 0.000 claims description 5
- 230000002452 interceptive effect Effects 0.000 claims description 5
- 108010026228 mRNA guanylyltransferase Proteins 0.000 claims description 4
- 238000012544 monitoring process Methods 0.000 claims description 4
- 239000000284 extract Substances 0.000 claims description 3
- 238000011161 development Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45562—Creating, deleting, cloning virtual machine instances
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45595—Network integration; Enabling network access in virtual machine instances
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a kubernets cluster federation realization method, which relates to the technical field of network communication and solves the problem of service discovery and arrangement of kubernets across cluster resources, and the method comprises the following steps: setting a cluster controller, a state controller, a synchronous controller, a scheduler and a command controller; initializing a cluster controller, a state controller, a synchronous controller, a scheduler and a command controller to a kubernets system to enable the kubernets to become a main cluster of a federal cluster, wherein the cluster controller, the state controller, the synchronous controller, the scheduler and the command controller can run at least 1 copy. The invention also discloses a kubernets cluster federal system. The invention runs a cluster controller, a state controller, a synchronous controller, a scheduler and a command controller in a kubernets system, utilizes CRD resources and admission control of an administration webhook built in the kubernets to achieve the complete cluster federal function, realizes unified management of a plurality of kubernets clusters by a single cluster, and can effectively solve the problem of service discovery and arrangement of cross-cluster resources of the kubernets.
Description
Technical Field
The invention relates to the technical field of network communication, in particular to a kubernetes cluster federal system and an implementation method thereof.
Background
With the continuous development of cloud computing, the technical architecture of IT infrastructure is undergoing drastic change, kubernets reduces the development, management, deployment and operation and maintenance costs of application programs from a brand-new perspective, and the architecture thereof becomes a de facto standard for rapid development, testing and deployment of application programs, and is deployed on a large scale in a production environment and adopted by more and more companies.
For enterprise-level customers, the data is required to be in safety compliance in the cloud uploading process, so that the cloud uploading is generally carried out in a multi-cloud and mixed-cloud manner. Before Kubernetes and container technology came into existence, it was necessary to customize development efforts for each cloud facilitator to achieve a cloudy and mixed cloud. After the Kubernets and the containers are arranged, the local data center and the Kubernets cluster of the cloud service provider can provide consistent interfaces, so that the application program does not need to be directly bound with a specific cloud service provider under most conditions, the cloud service provider is unlocked, and the situation that the service is only dependent on a certain cloud service provider is avoided. Meanwhile, the service availability is improved, a scheme with lower cost can be selected according to the price of the cloud service provider, the elastic expansion of service burst is guaranteed, and the method is expanded to other cloud service providers when the capacity of a certain cloud service provider is insufficient.
Building the Kubernetes cluster federation, however, presents a service discovery and orchestration problem across cluster resources for Kubernetes.
Disclosure of Invention
The technical problem to be solved by the invention is to solve the defects in the prior art, and the invention aims to provide a kubernets cluster federation realization method.
The invention also aims to provide a kubernets cluster federal system.
In order to achieve the first purpose, the invention provides a kubernetes cluster federation realization method, which comprises the steps of setting a cluster controller, a state controller, a synchronous controller, a scheduler and a command controller;
initializing the cluster controller, the state controller, the synchronous controller, the scheduler and the command controller to a kubernets system to enable the kubernets to become a main cluster of a federal cluster, wherein the cluster controller, the state controller, the synchronous controller, the scheduler and the command controller can run at least 1 copy;
configuring a federal cluster through the command controller, and performing interactive operation with an administrator, wherein the interactive operation comprises initializing the federal cluster, adding a slave cluster and viewing information of the federal cluster;
managing, by the cluster controller, a lifecycle of the federated cluster and checking health of master and slave clusters;
sensing and caching the running state of the federated resource in the slave cluster through the state controller;
distributing, by the dispatcher, workload instances into different clusters of the federated cluster;
and monitoring the expected value and the current running state of the federated resource of the slave cluster through the synchronous controller, and when the expected value is inconsistent with the running state, the synchronous controller is responsible for operating the federated resource of the slave cluster so as to keep the running state of the resource consistent with the expected value.
As a further improvement, the command controller operates as follows:
step 1.1, the command controller is initialized and operated to the main cluster, communicates with a kube-apiserver of the main cluster, and acquires configuration information of the federal cluster; if the acquisition is successful, executing the step 1.3; otherwise, executing step 1.2;
step 1.2, the command controller initializes the configuration information of the federal cluster; if the initialization is successful, the command controller is started successfully, and the step 1.3 is executed; otherwise, re-executing step 1.1;
step 1.3, the command controller waits for a user to input a command;
step 1.4, when the command controller receives a slave cluster adding command, executing a slave cluster adding step;
step 1.5, when the command controller receives a command for deleting the slave cluster, executing the step of deleting the slave cluster;
step 1.6, when the command controller receives a request for displaying a slave cluster list, searching and displaying the slave cluster resources;
step 1.7, step 1.3 is performed.
Further, the step of performing an add slave cluster comprises:
step 1.4.1, the command controller receives configuration file information of a slave cluster;
step 1.4.2, the command controller generates CRD resources of kubernets according to the configuration file information of the slave cluster;
step 1.4.3, the command controller connects the kube-apiserver of the slave cluster according to the connection information of the CRD resource;
step 1.4.4. the command controller creates role resources on the slave cluster, and if the creation is successful, successful information is returned; otherwise, failure information is returned.
Further, the step of performing deletion slave clustering comprises:
step 1.5.1, the command controller receives a request for deleting the appointed slave cluster, and searches the connection information of the CRD resource in the resources in the slave cluster;
step 1.5.2, the command controller connects and appoints a slave cluster kube-apiserver according to the connection information in the step 1.5.1, and deletes relevant information appointed on the slave cluster;
step 1.5.3, the command controller deletes the CRD resource in the appointed slave cluster, and if the deletion is successful, the success information is returned; otherwise, failure information is returned.
Further, the cluster controller operates as follows:
step 2.1, the cluster controller initializes and runs to the main cluster, communicates with the kube-apiserver of the main cluster, and obtains configuration information of the federal cluster and a slave cluster list; if the acquisition is successful, executing the step 2.2; otherwise, continuing to execute the step;
step 2.2, the cluster controller records the current time T1, traverses the slave cluster list to obtain slave cluster access information, and then accesses the kube-apiserver of the slave cluster; when the request can be correctly returned and the return value is a set value, the slave cluster is in a healthy state, otherwise, the slave cluster is in an unhealthy state;
if the health state changes, writing new health information into the slave cluster list, and recording the current time T2 after traversing is completed;
and 2.3, the cluster controller acquires health check delay time T according to the configuration information of the federal cluster, if T < T2-T1, the step 2.2 is executed, and if not, the step is repeatedly executed.
Further, the state controller operates as follows:
step 3.1, the state controller initializes and runs to the main cluster, communicates with the kube-apiserver of the main cluster, and obtains configuration information and a cluster list of the federal cluster; if the acquisition is successful, executing the step 3.2; otherwise, continuing to execute the step;
step 3.2, the state controller acquires all running federatization resources in a traversing manner;
and 3.3, the state controller acquires the current single federal resource, uses a kubernetes Informer mode to subscribe the change of the current resource, caches the change to the main cluster, and circulates the step until all the currently running federal resources are monitored.
And 3.4, the state controller continuously monitors the change of all running federated resources, and when the federated resources change, the changed information is cached to the main cluster.
Further, the scheduler operates as follows:
step 4.1, the scheduler is initialized and operated to the main cluster, communicates with the kube-apiserver of the main cluster, and executes the step 4.2 if the communication is successful; otherwise, continuing to execute the step;
step 4.2, the scheduler records the current time T3, and acquires the currently running Deployment federal resource, replicase federal resource, Statefelset federal resource and Daemonset federal resource;
step 4.3, the scheduler acquires the current single federal resource mentioned in the step 4.2, compares the scheduling state of the current federal resource, and extracts newly created kubernets Pod, unscheduled kubernets Pod and failed kubernets Pod;
step 4.4, the scheduler calls a set scheduling algorithm to schedule the currently acquired resources to a proper cluster, the scheduling result is written into the resources, and the synchronous controller executes resource synchronization;
step 4.5. if the scheduler has traversed all the resources mentioned in step 4.2, then step 4.6 is performed; otherwise, the scheduler acquires the next resource and executes the step 4.3;
step 4.6. the scheduler obtains the current time T4, T5 ═ T4-T3, if T5> the set time, step 4.2 is performed; otherwise, step 4.2 is executed after waiting the set time-T5.
Further, the setting of the scheduling algorithm includes:
step 4.4.1, acquiring the current resource type, kubernets nodes, cluster information and the service condition of resources;
4.4.2, arranging the resources of kubernets to obtain a PSP file based on the total copy number and the definition strategy of the cluster;
4.4.3, performing priority ordering operation according to the PSP file, and performing priority ordering on nodes suitable for resource operation;
and 4.4.4, selecting one cluster with the highest final priority as a resource scheduling result.
Further, the synchronization controller operates as follows:
step 5.1, the synchronous controller is initialized to run to the main cluster, communicates with the kube-apiserver of the main cluster, and if the communication is successful, the step 5.2 is executed; otherwise, continuing to execute the step;
step 5.2, the synchronous controller records the current time T6 and traverses a resource list needing to be synchronized;
step 5.3, the synchronous controller acquires a single resource and the latest state of the resource, and if the expected state of the resource is inconsistent with the latest state, the step 5.4 is executed; otherwise, executing step 5.6;
step 5.4, the synchronous controller acquires the slave cluster name from the resource, inquires the connection information of the slave cluster from the master cluster according to the slave cluster name, communicates with the kube-apiserver of the slave cluster, and executes the step 5.5 if the communication is successful, otherwise executes the step 5.6;
step 5.5, the synchronous controller creates or modifies corresponding resource information on the slave cluster according to the acquired data of the resource, and executes step 5.6;
step 5.6, if the synchronous controller has traversed all resources, executing step 5.7; otherwise, executing step 5.3;
step 5.7. if the synchronization controller obtains the current time T7, T8 ═ T7-T6, if T8> set time, then step 5.2 is performed; otherwise, step 5.2 is executed after waiting the set time-T8.
In order to achieve the second purpose, the invention provides a kubernetes cluster federal system, which comprises a cluster controller, a state controller, a synchronous controller, a scheduler and a command controller, wherein the cluster controller is used for controlling the state controller to be in a state of being in a state of being in a being in a state of being in a state of being in a being in a state of being in a being in a;
the cluster controller, the state controller, the synchronous controller, the scheduler and the command controller are initialized to a kubernets system and operate according to the kubernets cluster federal implementation method.
Advantageous effects
Compared with the prior art, the invention has the advantages that:
the invention runs a cluster controller, a state controller, a synchronous controller, a scheduler and a command controller in a kubernets system, utilizes CRD resources and admission control of an administration webhook built in the kubernets to achieve the complete cluster federal function, realizes unified management of a plurality of kubernets clusters by a single cluster, and can effectively solve the problem of service discovery and arrangement of cross-cluster resources of the kubernets. The invention realizes the complete cluster federal function, is easy to maintain, and enables the invention to be unbound with a specific Kubernets version by skillfully utilizing Kubernets CRD resources, and to be backward compatible with a newer version.
Drawings
FIG. 1 is an architectural diagram of the present invention.
Detailed Description
The invention will be further described with reference to specific embodiments shown in the drawings.
Referring to fig. 1, a kubernets Cluster federation implementation method sets a Cluster Controller, a Status Controller, a synchronous Controller, a Scheduler, and a Command Controller;
initializing a Cluster Controller, a Status Controller, a synchronous Controller, a Scheduler, a Command Controller to a kubernets system, so that the kubernets becomes a main Cluster of a federal Cluster, and the Cluster Controller, the Status Controller, the synchronous Controller, the Scheduler, and the Command Controller can run at least 1 copy;
configuring a federal cluster through a Command Controller, and performing interactive operation with an administrator, wherein the operations comprise initializing the federal cluster, adding a slave cluster and viewing information of the federal cluster;
managing the life cycle of the federated Cluster through a Cluster Controller, and checking the health of a master Cluster and a slave Cluster;
sensing and caching the running state of the federated resource in the slave cluster through a state Controller Status Controller;
the method comprises the steps that a Scheduler disperses workload instances into different clusters of a federated cluster;
and monitoring the expected value and the current running state of the federated resource of the slave cluster through a synchronous Controller Sync Controller, and when the expected value is inconsistent with the running state, the synchronous Controller Sync Controller is responsible for operating the federated resource of the slave cluster so as to keep the running state of the resource consistent with the expected value.
The Command Controller operates as follows:
step 1.1, initializing and running a Command Controller to a main cluster, communicating with a kube-apiserver of the main cluster, and acquiring configuration information CaihFederalConfig of the federal cluster; if the acquisition is successful, executing the step 1.3; otherwise, executing step 1.2;
step 1.2, a Command Controller initializes configuration information of the federal cluster; if the initialization is successful, the Command Controller is successfully started, and the step 1.3 is executed; otherwise, re-executing step 1.1; the configuration information for initialization is as follows:
-Role, general Role information;
-ServiceAccount, service account information;
-clusterrollingbinding, cluster role and account binding information;
-RoleBinding, generic role and account binding information;
-a CRD resource of caihhfeldalcluster, kubernets, for holding connection information from the cluster, initialization declaring only this resource, containing no content;
-crid resources of caihhfeleralconfig, kubernets, describing federal cluster configuration information, including the following:
-a health check delay time;
-a failed retry time;
-whether domain name service is enabled;
-whether Scheduler is enabled;
-whether cross cluster service discovery is enabled;
-a ValidatingWebhookConfiguration and a muttatingwebhookconfiguration for unified format and associated calls when cluster federation generates resources;
-caihh federalcrds federate resources federatization kubernets resources, defamation of Namespace, Deployment, Service, ConfigMap, Secret, Ingress, Job, ReplicaSet, ClusterRole, Role, ServiceAccount, ClusterRole binding, rollering binding resources federatization, other resources may federate using command lines;
step 1.3, a Command Controller waits for a user to input a Command;
step 1.4, when the Command Controller receives the Command of adding the slave cluster, executing the step of adding the slave cluster;
step 1.5, when the Command Controller receives a Command for deleting the slave cluster, executing the step of deleting the slave cluster;
step 1.6, when the Command Controller receives a request for displaying a slave cluster list, searching and displaying the slave cluster resources; searching and displaying the resources of the slave cluster in the CaihFederalCluster resources;
step 1.7, step 1.3 is performed.
The step of performing an add slave cluster comprises:
step 1.4.1, the Command Controller receives the information of the yaml configuration file added to the slave cluster; the yaml profile information is as follows:
-federally clustered proprietary namespace: caih-feeder
Cluster role information
Cluster url, ca authentication information and cluster name from the cluster
-user configuration information of the slave cluster, including user name, authentication information cert, authentication key
-association configuration of usernames and clusters
The file format is referenced as follows:
step 1.4.2, the Command Controller generates CRD resources of kubernets according to the yaml configuration file information of the slave cluster, namely, generates CRD resources of CaihFederalCluster;
step 1.4.3, the Command Controller connects the kube-apiserver of the slave cluster according to the connection information of the CRD resource (namely according to the connection information of the CaihFederalCluster);
step 1.4.4, the Command Controller creates role resources on the slave cluster, and if the creation is successful, successful information is returned; otherwise, returning failure information;
role resources are as follows:
-creating namespace where the federated cluster is located: caih-feed;
-ServiceAccount, service account information;
-ClusterRole, cluster role information;
-clusterrollingbinding, cluster role and account binding information;
-Secret, account key;
-caihhfeleralcrds, federatnetes resources federally, identical to caihhfeleralcrds initialized by the master cluster in step 1.2.
The step of performing deletion slave clustering comprises:
step 1.5.1, the Command Controller receives a request for deleting the appointed slave cluster, and searches the connection information of the CRD resource in the slave cluster, namely the connection information of the slave cluster in the CaihFederalCluster resource;
step 1.5.2, the Command Controller connects and appoints the slave cluster kube-apiserver according to the connection information in step 1.5.1, and deletes the relevant information appointed on the slave cluster;
step 1.5.3, the Command Controller deletes the CRD resource in the appointed slave cluster, namely, deletes the CaihFederalCluster resource, if the deletion is successful, the success information is returned; otherwise, failure information is returned.
The Cluster Controller operates as follows:
step 2.1, the Cluster Controller initializes and runs to the main Cluster, communicates with the kube-apiserver of the main Cluster, and obtains the configuration information CaihFederalConfig of the federal Cluster and the CaihFederal Cluster from the Cluster list; if the acquisition is successful, executing the step 2.2; otherwise, continuing to execute the step;
step 2.2, the Cluster Controller records the current time T1, traverses the slave Cluster list CaihFederalCluster to obtain the slave Cluster access information, and then accesses the kube-apiserver of the slave Cluster; when the request can be correctly returned and the return value is a set value, in this embodiment, the set value is 200, which indicates that the slave cluster is in a healthy state, otherwise, indicates that the slave cluster is in an unhealthy state;
if the health state changes, writing new health information into a subordinate cluster list CaihFederalCluster, and recording the current time T2 after traversing is completed;
and 2.3, the Cluster Controller acquires the health check delay time T according to the configuration information CaihFederalConfig of the federal Cluster, if T is less than T2-T1, the step 2.2 is executed, otherwise, the step is repeatedly executed.
The state Controller Status Controller operates as follows:
step 3.1, initializing and running a Status Controller to a main cluster, communicating with a kube-apiserver of the main cluster, and acquiring configuration information CaihFederalConfig of the federated cluster and a cluster list CaihFederalCluster; if the acquisition is successful, executing the step 3.2; otherwise, continuing to execute the step;
step 3.2, the Status Controller traverses and obtains all running federated resources;
and 3.3, the Status Controller acquires the current single federal resource, subscribes the change of the current resource by using a kubernets inform mode, caches the change to the main cluster, and circulates the step until all the currently running federal resources are monitored.
And 3.4, continuously monitoring the change of all running federal resources by a Status Controller, and caching the changed information to the main cluster when the change of the federal resources occurs.
The Scheduler runs as follows:
step 4.1, the Scheduler is initialized and operated to the main cluster, communicates with the kube-apiserver of the main cluster, if successful, then step 4.2 is executed; otherwise, continuing to execute the step;
step 4.2, the Scheduler records the current time T3, and acquires the currently running Delploymet federal resource, replicase federal resource, Statedulet federal resource and Daemonset federal resource;
step 4.3, the Scheduler obtains the current single federal resource mentioned in the step 4.2, compares the scheduling state of the current federal resource, and extracts newly created kubernets Pod, unscheduled kubernets Pod and failed kubernets Pod;
step 4.4, the Scheduler calls a set scheduling algorithm to schedule the currently acquired resources to a proper cluster, the scheduling result is written into the resources, and a synchronous Controller (Sync Controller) executes resource synchronization;
step 4.5, if the Scheduler has traversed all the resources mentioned in step 4.2, then step 4.6 is executed; otherwise, the Scheduler acquires the next resource and executes the step 4.3;
step 4.6, the Scheduler acquires the current time T4, T5 is T4-T3, if T5> the set time, step 4.2 is executed; otherwise, step 4.2 is executed after waiting the set time-T5.
The set scheduling algorithm comprises the following steps:
step 4.4.1, acquiring the current resource type, kubernets nodes, cluster information and the service condition of resources; such as how many scheduled pots are on the cluster, the resources that have been applied for, and the resources that can also be allocated; acquiring kubernets nodes, namely acquiring slave cluster information of the current resource and all node information of the slave cluster, and determining which node a pod related to the Deploymet, the replicase, the Stateful set and the Daemonset should be dispatched to;
step 4.4.2, arranging the resources of kubernets to obtain a PSP (PodSchedulingPreference) file based on the total copy number and the definition strategy of the cluster;
4.4.3, performing priority ordering operation according to the PSP file, and performing priority ordering on the nodes suitable for resource operation;
and 4.4.4, selecting one cluster with the highest final priority as a resource scheduling result.
The Sync Controller operates as follows:
step 5.1, the synchronous Controller is initialized to operate to the main cluster, communicates with kube-apiserver of the main cluster, if successful, carry out step 5.2; otherwise, continuing to execute the step;
step 5.2, the synchronous Controller records the current time T6 and traverses the resource list to be synchronized;
step 5.3, the synchronous Controller obtains the single resource and the latest state of the resource, if the expected state of the resource is not consistent with the latest state, the step 5.4 is executed; otherwise, executing step 5.6;
step 5.4, the synchronous Controller Sync Controller acquires the names of the slave clusters from the resources, inquires the connection information of the slave clusters from the master cluster according to the names of the slave clusters, and communicates with the kube-apiserver of the slave clusters, if the communication is successful, the step 5.5 is executed, otherwise, the step 5.6 is executed;
step 5.5, the synchronous Controller creates or modifies the corresponding resource information on the slave cluster according to the acquired data of the resource, and executes step 5.6;
step 5.6, if the synchronous Controller Sync Controller has traversed all resources, then step 5.7 is executed; otherwise, executing step 5.3;
step 5.7, if the synchronization Controller obtains the current time T7, and T8 is T7-T6, if T8> the set time, step 5.2 is executed; otherwise, step 5.2 is executed after waiting the set time-T8.
A kubernets Cluster federation system comprises a Cluster Controller, a state Controller Status Controller, a synchronous Controller Sync Controller, a Scheduler and a Command Controller;
the Cluster Controller, the Status Controller, the synchronous Controller, the Scheduler and the Command Controller are initialized to the kubernets system and operate according to the kubernets Cluster federation realization method.
The above is only a preferred embodiment of the present invention, and it should be noted that it is obvious to those skilled in the art that several variations and modifications can be made without departing from the structure of the present invention, which will not affect the effect of the implementation of the present invention and the utility of the patent.
Claims (10)
1. A kubernetes cluster federation realization method is characterized in that a cluster controller, a state controller, a synchronous controller, a scheduler and a command controller are arranged;
initializing the cluster controller, the state controller, the synchronous controller, the scheduler and the command controller to a kubernets system to enable the kubernets to become a main cluster of a federal cluster, wherein the cluster controller, the state controller, the synchronous controller, the scheduler and the command controller can run at least 1 copy;
configuring a federal cluster through the command controller, and performing interactive operation with an administrator, wherein the interactive operation comprises initializing the federal cluster, adding a slave cluster and viewing information of the federal cluster;
managing, by the cluster controller, a lifecycle of the federated cluster and checking health of master and slave clusters;
sensing and caching the running state of the federated resource in the slave cluster through the state controller;
distributing, by the dispatcher, workload instances into different clusters of the federated cluster;
and monitoring the expected value and the current running state of the federated resource of the slave cluster through the synchronous controller, and when the expected value is inconsistent with the running state, the synchronous controller is responsible for operating the federated resource of the slave cluster so as to keep the running state of the resource consistent with the expected value.
2. The method of claim 1, wherein the command controller operates as follows:
step 1.1, the command controller is initialized and operated to the main cluster, communicates with a kube-apiserver of the main cluster, and acquires configuration information of the federal cluster; if the acquisition is successful, executing the step 1.3; otherwise, executing step 1.2;
step 1.2, the command controller initializes the configuration information of the federal cluster; if the initialization is successful, the command controller is started successfully, and the step 1.3 is executed; otherwise, re-executing step 1.1;
step 1.3, the command controller waits for a user to input a command;
step 1.4, when the command controller receives a slave cluster adding command, executing a slave cluster adding step;
step 1.5, when the command controller receives a command for deleting the slave cluster, executing the step of deleting the slave cluster;
step 1.6, when the command controller receives a request for displaying a slave cluster list, searching and displaying the slave cluster resources;
step 1.7, step 1.3 is performed.
3. The method of claim 2, wherein performing the add slave cluster step comprises:
step 1.4.1, the command controller receives configuration file information of a slave cluster;
step 1.4.2, the command controller generates CRD resources of kubernets according to the configuration file information of the slave cluster;
step 1.4.3, the command controller connects the kube-apiserver of the slave cluster according to the connection information of the CRD resource;
step 1.4.4. the command controller creates role resources on the slave cluster, and if the creation is successful, successful information is returned; otherwise, failure information is returned.
4. The method of claim 2, wherein the step of performing a delete slave cluster comprises:
step 1.5.1, the command controller receives a request for deleting the appointed slave cluster, and searches the connection information of the CRD resource in the resources in the slave cluster;
step 1.5.2, the command controller connects and appoints a slave cluster kube-apiserver according to the connection information in the step 1.5.1, and deletes relevant information appointed on the slave cluster;
step 1.5.3, the command controller deletes the CRD resource in the appointed slave cluster, and if the deletion is successful, the success information is returned; otherwise, failure information is returned.
5. The method of claim 1, wherein the cluster controller operates as follows:
step 2.1, the cluster controller initializes and runs to the main cluster, communicates with the kube-apiserver of the main cluster, and obtains configuration information of the federal cluster and a slave cluster list; if the acquisition is successful, executing the step 2.2; otherwise, continuing to execute the step;
step 2.2, the cluster controller records the current time T1, traverses the slave cluster list to obtain slave cluster access information, and then accesses the kube-apiserver of the slave cluster; when the request can be correctly returned and the return value is a set value, the slave cluster is in a healthy state, otherwise, the slave cluster is in an unhealthy state;
if the health state changes, writing new health information into the slave cluster list, and recording the current time T2 after traversing is completed;
and 2.3, the cluster controller acquires health check delay time T according to the configuration information of the federal cluster, if T < T2-T1, the step 2.2 is executed, and if not, the step is repeatedly executed.
6. The method of claim 1, wherein the state controller operates as follows:
step 3.1, the state controller initializes and runs to the main cluster, communicates with the kube-apiserver of the main cluster, and obtains configuration information and a cluster list of the federal cluster; if the acquisition is successful, executing the step 3.2; otherwise, continuing to execute the step;
step 3.2, the state controller acquires all running federatization resources in a traversing manner;
and 3.3, the state controller acquires the current single federal resource, uses a kubernetes Informer mode to subscribe the change of the current resource, caches the change to the main cluster, and circulates the step until all the currently running federal resources are monitored.
And 3.4, the state controller continuously monitors the change of all running federated resources, and when the federated resources change, the changed information is cached to the main cluster.
7. The method of claim 1, wherein the scheduler operates as follows:
step 4.1, the scheduler is initialized and operated to the main cluster, communicates with the kube-apiserver of the main cluster, and executes the step 4.2 if the communication is successful; otherwise, continuing to execute the step;
step 4.2, the scheduler records the current time T3, and acquires the currently running Deployment federal resource, replicase federal resource, Statefelset federal resource and Daemonset federal resource;
step 4.3, the scheduler acquires the current single federal resource mentioned in the step 4.2, compares the scheduling state of the current federal resource, and extracts newly created kubernets Pod, unscheduled kubernets Pod and failed kubernets Pod;
step 4.4, the scheduler calls a set scheduling algorithm to schedule the currently acquired resources to a proper cluster, the scheduling result is written into the resources, and the synchronous controller executes resource synchronization;
step 4.5. if the scheduler has traversed all the resources mentioned in step 4.2, then step 4.6 is performed; otherwise, the scheduler acquires the next resource and executes the step 4.3;
step 4.6. the scheduler obtains the current time T4, T5 ═ T4-T3, if T5> the set time, step 4.2 is performed; otherwise, step 4.2 is executed after waiting the set time-T5.
8. The method of claim 7, wherein setting a scheduling algorithm comprises:
step 4.4.1, acquiring the current resource type, kubernets nodes, cluster information and the service condition of resources;
4.4.2, arranging the resources of kubernets to obtain a PSP file based on the total copy number and the definition strategy of the cluster;
4.4.3, performing priority ordering operation according to the PSP file, and performing priority ordering on nodes suitable for resource operation;
and 4.4.4, selecting one cluster with the highest final priority as a resource scheduling result.
9. The method of claim 1, wherein the synchronization controller operates as follows:
step 5.1, the synchronous controller is initialized to run to the main cluster, communicates with the kube-apiserver of the main cluster, and if the communication is successful, the step 5.2 is executed; otherwise, continuing to execute the step;
step 5.2, the synchronous controller records the current time T6 and traverses a resource list needing to be synchronized;
step 5.3, the synchronous controller acquires a single resource and the latest state of the resource, and if the expected state of the resource is inconsistent with the latest state, the step 5.4 is executed; otherwise, executing step 5.6;
step 5.4, the synchronous controller acquires the slave cluster name from the resource, inquires the connection information of the slave cluster from the master cluster according to the slave cluster name, communicates with the kube-apiserver of the slave cluster, and executes the step 5.5 if the communication is successful, otherwise executes the step 5.6;
step 5.5, the synchronous controller creates or modifies corresponding resource information on the slave cluster according to the acquired data of the resource, and executes step 5.6;
step 5.6, if the synchronous controller has traversed all resources, executing step 5.7; otherwise, executing step 5.3;
step 5.7. if the synchronization controller obtains the current time T7, T8 ═ T7-T6, if T8> set time, then step 5.2 is performed; otherwise, step 5.2 is executed after waiting the set time-T8.
10. A kubernetes cluster federation system is characterized by comprising a cluster controller, a state controller, a synchronous controller, a scheduler and a command controller;
the cluster controller, the state controller, the synchronization controller, the scheduler and the command controller are initialized to a kubernets system and operate according to a kubernets cluster federation implementation method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111048076.8A CN113742033A (en) | 2021-09-08 | 2021-09-08 | Kubernetes cluster federal system and implementation method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111048076.8A CN113742033A (en) | 2021-09-08 | 2021-09-08 | Kubernetes cluster federal system and implementation method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113742033A true CN113742033A (en) | 2021-12-03 |
Family
ID=78737040
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111048076.8A Pending CN113742033A (en) | 2021-09-08 | 2021-09-08 | Kubernetes cluster federal system and implementation method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113742033A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114296637A (en) * | 2021-12-09 | 2022-04-08 | 广西东信数建信息科技有限公司 | Method and device for dynamically creating local storage volume |
| CN114584559A (en) * | 2022-03-07 | 2022-06-03 | 浪潮云信息技术股份公司 | Method and device for realizing declarative management of container cloud |
| CN115460089A (en) * | 2022-08-30 | 2022-12-09 | 兴业银行股份有限公司 | Method and system for realizing open, multi-cloud and multi-cluster federated |
| WO2024016624A1 (en) * | 2022-07-19 | 2024-01-25 | 京东科技信息技术有限公司 | Multi-cluster access method and system |
| CN117743181A (en) * | 2023-12-25 | 2024-03-22 | 杭州云掣科技有限公司 | System for constructing observable control surface |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111176697A (en) * | 2020-01-02 | 2020-05-19 | 广州虎牙科技有限公司 | Service instance deployment method, data processing method and cluster federation |
| CN111193782A (en) * | 2019-12-18 | 2020-05-22 | 北京航天智造科技发展有限公司 | PAAS cloud cluster construction method and device, electronic equipment and storage medium |
| CN111800303A (en) * | 2020-09-09 | 2020-10-20 | 杭州朗澈科技有限公司 | Method, device and system for guaranteeing number of available clusters in mixed cloud scene |
| CN112099989A (en) * | 2020-08-28 | 2020-12-18 | 中国—东盟信息港股份有限公司 | Disaster recovery, migration and recovery method for Kubernetes cloud native application |
| CN112965795A (en) * | 2021-02-23 | 2021-06-15 | 卓望数码技术（深圳）有限公司 | Cluster scheduling method, electronic device and storage medium |
| WO2021135545A1 (en) * | 2020-06-30 | 2021-07-08 | 平安科技（深圳）有限公司 | Kubernetes multi-cluster management system and method, terminal and storage medium |
| US20210240542A1 (en) * | 2020-01-31 | 2021-08-05 | EMC IP Holding Company LLC | Dynamic application management across multi-cloud computing environment |
-
2021
- 2021-09-08 CN CN202111048076.8A patent/CN113742033A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111193782A (en) * | 2019-12-18 | 2020-05-22 | 北京航天智造科技发展有限公司 | PAAS cloud cluster construction method and device, electronic equipment and storage medium |
| CN111176697A (en) * | 2020-01-02 | 2020-05-19 | 广州虎牙科技有限公司 | Service instance deployment method, data processing method and cluster federation |
| US20210240542A1 (en) * | 2020-01-31 | 2021-08-05 | EMC IP Holding Company LLC | Dynamic application management across multi-cloud computing environment |
| WO2021135545A1 (en) * | 2020-06-30 | 2021-07-08 | 平安科技（深圳）有限公司 | Kubernetes multi-cluster management system and method, terminal and storage medium |
| CN112099989A (en) * | 2020-08-28 | 2020-12-18 | 中国—东盟信息港股份有限公司 | Disaster recovery, migration and recovery method for Kubernetes cloud native application |
| CN111800303A (en) * | 2020-09-09 | 2020-10-20 | 杭州朗澈科技有限公司 | Method, device and system for guaranteeing number of available clusters in mixed cloud scene |
| CN112965795A (en) * | 2021-02-23 | 2021-06-15 | 卓望数码技术（深圳）有限公司 | Cluster scheduling method, electronic device and storage medium |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114296637A (en) * | 2021-12-09 | 2022-04-08 | 广西东信数建信息科技有限公司 | Method and device for dynamically creating local storage volume |
| CN114296637B (en) * | 2021-12-09 | 2024-05-17 | 广西东信数建信息科技有限公司 | Dynamic creation method and device for local storage volume |
| CN114584559A (en) * | 2022-03-07 | 2022-06-03 | 浪潮云信息技术股份公司 | Method and device for realizing declarative management of container cloud |
| WO2024016624A1 (en) * | 2022-07-19 | 2024-01-25 | 京东科技信息技术有限公司 | Multi-cluster access method and system |
| CN115460089A (en) * | 2022-08-30 | 2022-12-09 | 兴业银行股份有限公司 | Method and system for realizing open, multi-cloud and multi-cluster federated |
| CN117743181A (en) * | 2023-12-25 | 2024-03-22 | 杭州云掣科技有限公司 | System for constructing observable control surface |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113742033A (en) | Kubernetes cluster federal system and implementation method thereof | |
| CN106844198B (en) | Distributed dispatching automation test platform and method | |
| CN106126346B (en) | A kind of large-scale distributed data collection system and method | |
| CN103414761B (en) | Mobile terminal cloud resource scheduling method based on Hadoop framework | |
| CN104317610B (en) | Method and device for automatic installation and deployment of hadoop platform | |
| CN103092698B (en) | Cloud computing application automatic deployment system and method | |
| CN101252471B (en) | Distributed automatization test system and method | |
| CN107301048B (en) | Internal control management system of application response type shared application architecture | |
| CN108920153B (en) | Docker container dynamic scheduling method based on load prediction | |
| CN109150987B (en) | Two-layer container cluster elastic expansion method based on host layer and container layer | |
| CN111506412A (en) | Distributed asynchronous task construction and scheduling system and method based on Airflow | |
| CN103414712B (en) | A kind of distributed virtual desktop management system and method | |
| CN110245003A (en) | A kind of machine learning uniprocessor algorithm arranging system and method | |
| CN102880503A (en) | Data analysis system and data analysis method | |
| CN101815013B (en) | Method for monitoring operation of satellite application system based on Ajax and Web service technology | |
| CN109840166A (en) | Across the cluster object storage async backup methods, devices and systems of one kind | |
| CN105786603B (en) | Distributed high-concurrency service processing system and method | |
| CN104243617A (en) | Task scheduling method and system facing mixed load in heterogeneous cluster | |
| US6804710B1 (en) | Configuration information management system, method, program, and program storage device | |
| CN112804362B (en) | Dispersed data micro-service automation operation and maintenance system | |
| CN113672240A (en) | Container-based multi-machine-room batch automatic deployment application method and system | |
| CN112437129B (en) | Cluster management method and cluster management device | |
| CN110704465A (en) | Method, device and storage medium for processing service work order table | |
| CN109992373A (en) | Resource regulating method, approaches to IM and device and task deployment system | |
| CN114912255A (en) | On-line simulation experiment system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |