CN118540371A

CN118540371A - Cluster resource management method and device, storage medium and electronic equipment

Info

Publication number: CN118540371A
Application number: CN202410597304.4A
Authority: CN
Inventors: 张永曦
Original assignee: China Mobile Communications Group Co Ltd; China Mobile Suzhou Software Technology Co Ltd
Current assignee: China Mobile Communications Group Co Ltd; China Mobile Suzhou Software Technology Co Ltd
Priority date: 2024-05-14
Filing date: 2024-05-14
Publication date: 2024-08-23

Abstract

The application discloses a cluster resource management method, a cluster resource management device, a storage medium and electronic equipment, and relates to the technical field of computers, wherein the method comprises the following steps: firstly, establishing a Watch connection between a server and a plurality of sub-clusters to be managed based on LISTANDWATCH mechanisms, and collecting resource event information of the plurality of sub-clusters to be managed; uniformly coding the first resource version number corresponding to the resource event information to obtain a second resource version number corresponding to the resource event information; responding to a resource acquisition request and a target resource version number uploaded by a client, and searching in resource event information corresponding to a second resource version number based on the target resource version number; and sending the searched target resource event information to a client, wherein the client is used for managing a plurality of sub-clusters to be managed according to the target resource event information. Compared with the prior art, the application realizes LISTANDWATCH functions of the client to a plurality of clusters.

Description

Cluster resource management method and device, storage medium and electronic equipment

Technical Field

The present application relates to the field of computer technologies, and in particular, to a method and apparatus for managing cluster resources, a storage medium, and an electronic device.

Background

Kubernetes (K8 s) is an open source for managing containerized applications on multiple hosts in a cloud platform, and the goal of Kubernetes is to make deploying containerized applications simple and efficient, and Kubernetes provides a mechanism for application deployment, planning, updating, and maintenance.

At present, the server is mainly connected with the server through LISTANDWATCH mechanism, and then cluster resource information of a single cluster connected with the server is sent to the client through APISERVER, so that the client can acquire all resource information in the cluster in real time, and further manage cluster resources.

However, due to the large number of K8s clusters, only one cluster can be managed by one client using this management method, which results in low management efficiency of cluster resources.

Disclosure of Invention

In view of this, the present application provides a method, an apparatus, a storage medium and an electronic device for managing cluster resources, which are mainly aimed at improving the technical problem that in the prior art, due to the large number of K8s clusters, only one client can manage one cluster, which results in lower management efficiency of cluster resources.

In a first aspect, the present application provides a method for managing cluster resources, including:

Establishing a Watch connection between a server and a plurality of sub-clusters to be managed based on LISTANDWATCH mechanism, and collecting resource event information of the plurality of sub-clusters to be managed;

Uniformly coding a first resource version number corresponding to the resource event information to obtain a second resource version number corresponding to the resource event information, wherein the first resource version number is a resource version number obtained by coding based on a single cluster, and the second resource version number is a resource version number obtained by uniformly coding based on a plurality of sub-clusters to be managed;

responding to a resource acquisition request and a target resource version number uploaded by a client, and searching in the resource event information corresponding to the second resource version number based on the target resource version number;

and sending the searched target resource event information to the client, wherein the client is used for managing the plurality of sub-clusters to be managed according to the target resource event information.

In a second aspect, the present application provides a cluster resource management device, including:

The collection module is configured to establish a Watch connection between a server and a plurality of sub-clusters to be managed based on LISTANDWATCH mechanisms and collect resource event information of the plurality of sub-clusters to be managed;

The coding module is configured to uniformly code a first resource version number corresponding to the resource event information to obtain a second resource version number corresponding to the resource event information, wherein the first resource version number is a resource version number obtained by coding based on a single cluster, and the second resource version number is a resource version number obtained by uniformly coding based on a plurality of sub-clusters to be managed;

The searching module is configured to respond to the resource acquisition request and the target resource version number uploaded by the client and search in the resource event information corresponding to the second resource version number based on the target resource version number;

The sending module is configured to send the searched target resource event information to the client, and the client is used for managing the plurality of sub-clusters to be managed according to the target resource event information.

In a third aspect, the present application provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of managing cluster resources of the first aspect.

In a fourth aspect, the present application provides an electronic device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, where the processor implements the method for managing cluster resources of the first aspect when executing the computer program.

In a fifth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the method of clustering face images of the first aspect.

By means of the technical scheme, the cluster resource management method, the cluster resource management device, the storage medium and the electronic equipment provided by the application are characterized in that firstly, a service end is connected with a plurality of sub-clusters to be managed by a switch based on LISTANDWATCH mechanism, and resource event information of the plurality of sub-clusters to be managed is collected; uniformly coding a first resource version number corresponding to the resource event information to obtain a second resource version number corresponding to the resource event information, wherein the first resource version number is a resource version number obtained by coding based on a single cluster, and the second resource version number is a resource version number obtained by uniformly coding based on a plurality of sub-clusters to be managed; responding to a resource acquisition request and a target resource version number uploaded by a client, and searching in the resource event information corresponding to the second resource version number based on the target resource version number; and sending the searched target resource event information to the client, wherein the client is used for managing the plurality of sub-clusters to be managed according to the target resource event information. Compared with the prior art, the application establishes the Watch connection between the server and the plurality of sub-clusters to be managed, collects the resource event information of the plurality of sub-clusters to be managed, and realizes the aggregation of cluster resource events of the plurality of clusters, thereby realizing LISTANDWATCH functions of the server on the plurality of clusters; the first resource version number corresponding to the resource event information is uniformly coded to obtain the second resource version number corresponding to the resource event information, so that the multi-cluster resource can take the second resource version number as a unique identifier, the identifier and the sequence are realized through the uniform second resource version number, the uniform management of the multi-cluster is realized, the operation efficiency of the whole system is improved, the query can be directly carried out in the resource event information corresponding to the second resource version number based on the target resource version number, the resource event information corresponding to the second resource version number is not required to be stored in other positions, and the dependence on a third-party storage medium is avoided; the single-cluster application can monitor and use the multi-cluster capability without modification, and then the LISTANDWATCH function of the client to the multiple clusters is realized.

The foregoing description is only an overview of the present application, and is intended to be implemented in accordance with the teachings of the present application in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present application more readily apparent.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.

In order to more clearly illustrate the embodiments of the application or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.

Fig. 1 is a schematic flow chart of a cluster resource management method according to an embodiment of the present application;

Fig. 2 is a schematic flow chart of a cluster resource management method according to an embodiment of the present application;

FIG. 3 illustrates an exemplary flow diagram provided by an embodiment of the present application;

fig. 4 is a schematic structural diagram of a cluster resource management device according to an embodiment of the present application.

Detailed Description

Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other.

In the prior art, all components in the K8s, and the container application running on the K8s, interact with each other through APISERVER (APISERVER synchronizes the real-time state of all resources in the cluster for the components and container application), and APISERVER synchronizes the real-time state of all resources in the K8s cluster using LISTANDWATCH. When Informer client disconnects APISERVER for the first time, all K8s cluster resources are subjected to one-time List to local cache in a mode of Http short link, then, http long link is established through Watch, event events sent by APISERVER are continuously received, the local cache is updated in real time, and when the client needs to inquire a certain resource, the client can directly inquire in the local cache, so that the pressure of APISERVER is greatly reduced.

However, as the number of K8s clusters increases, cross-cluster applications have become unavoidable, and the ability to use multiple clusters imperceptibly without changing current business codes for cloud native applications has become an urgent need. To meet this requirement, the implementation of multiple clusters LISTANDWATCH is machined as the most critical ring. Currently for multi-cluster solutions, a truly open multi-cloud K8s can be realized mainly by using K8s native APIs and providing advanced scheduling functions Karmada. The state aggregation of K8s resources of the multi-cluster part only is achieved by Karmada at present, and a multi-cluster LISTANDWATCH mechanism is not supported; or, by aggregating multiple K8s cluster resources, a more powerful retrieval function is additionally provided on the basis of being compatible with K8s OpenAPI, so that a user can acquire any desired resources in multiple clusters more quickly and conveniently. At present Clusterpedia only performs multi-cluster resource retrieval, and a multi-cluster LISTANDWATCH mechanism is not realized.

In the computing power network scenario, after the cloud native application is scheduled by the computing network brain, a management module (e.g., an Operator, etc.) of the cloud native application needs to dynamically manage and reconcile the running state of the cloud native application above the multi-K8 s cluster, and the management module of the cloud native application cannot run above the multi-K8 s cluster without perceiving that the code is not modified. Specifically, the Karmada project of the CNCF foundation currently realizes the functions of multi-strategy-based resource scheduling of multi-K8 s clusters, multi-cluster state aggregation of cluster resources and the like, and well plays a role in multi-cluster management of the K8s resources. However Karmada does not support the multi-cluster LISTANDWATCH mechanism, and cloud native applications cannot use the multi-cluster running resources based on Karmada without awareness. Correspondingly, according to Clusterpedia items of the CNCF foundation, the bottom layer is based on MySql, the persistence of multi-cluster resource objects is realized, and on the basis, a multi-cluster retrieval function of a K8s OpenAPI mode is provided. However Clusterpedia currently only implements simple Get and List functions, and is not supported for more complex Watch functions. Clusterpedia, mainly operation, maintenance and research personnel, search the multi-cluster resources by means of Kubectl tools.

In order to improve the technical problem that in the prior art, due to the fact that a large number of K8s clusters exist, one client can only manage one cluster, and management efficiency of cluster resources is low. The embodiment provides a method for managing cluster resources, as shown in fig. 1, where the method includes:

Step 101, establishing a Watch connection between a server and a plurality of sub-clusters to be managed based on LISTANDWATCH mechanisms, and collecting resource event information of the plurality of sub-clusters to be managed.

In the embodiment of the application, LISTANDWATCH is a core mechanism for synchronously and asynchronously monitoring the state change of the resources in the Kubernetes, LISTANDWATCH is a core mode for synchronously and asynchronously monitoring the state of the resources in the Kubernetes, and each component is ensured to respond to the change of the resources in the cluster in time. It is widely used in the internal components of Kubernetes and in the client libraries that interact with Kubernetes API SERVER. In particular, in Kubernetes internal components such as Kube-apiserver, the cacher component also uses a similar mechanism to List and Watch objects, such as Secret objects, in order to maintain consistency of the local cache with the cluster state. If LISTANDWATCH encounters an error, such as failure to list resources from API SERVER or a snoop failure, it may cause the component to be abnormal or restarted.

In some examples, the plurality of sub-clusters to be managed may correspond to different Kubernetes clusters, and in particular, the Kubernetes (K8 s) cluster is an open source platform for automating deployment, extension, and management of containerized applications, and a Master node in the Kubernetes cluster may include: kube-apiserver: as a unified entry of the cluster, it is responsible for receiving and processing operation requests of all objects in the cluster. etcd: the distributed key value storage system stores all configuration data and state information of the cluster. kube-schedule: is responsible for scheduling the newly created Pod to run on the appropriate Worker node. kube-controller-manager: a series of Controller processes, such as Node controllers, REPLICASET CONTROLLER, service Account & Token Controllers, etc., are run that together maintain the state of the cluster consistent with the desired state.

Alternatively, the resource Event information may be a cluster Event, and in the Kubernetes cluster, an Event (Event) is a core concept, which is used to record detailed information about a state change of a resource object in the cluster. Each event represents a result of an operation or current state of a particular resource, such as container creation, restart, service endpoint modification, etc. Kubernetes components such as Kubelet, controller manager, and other custom controllers all generate and report events.

And 102, uniformly encoding the first resource version number corresponding to the resource event information to obtain a second resource version number corresponding to the resource event information.

The first resource version number is a resource version number obtained by coding based on a single cluster, and the second resource version number is a resource version number obtained by uniformly coding based on a plurality of sub-clusters to be managed.

For this embodiment, the resource version number is ResourceVersion, specifically in Kubernetes, the resource version number (ResourceVersion) is a string identifier that represents a specific version of each API object within the cluster. Each Kubernetes resource object (e.g., pod, deployment, service, etc.) has a ResourceVersion field in its metadata. In the embodiment of the application, the watch operation is an important characteristic of the Kubernetes API, and allows the client to monitor the change condition of the resource set. The client may monitor for resource change events from a particular version by specifying ResourceVersion parameters.

Correspondingly, the first resource version number is the resource version number of the Kubernetes cluster based on encoding, and the second resource version number is the resource version number obtained after recoding based on a plurality of sub-clusters to be managed.

And step 103, responding to the received resource acquisition request and the target resource version number uploaded by the client, and searching in the resource event information corresponding to the second resource version number based on the target resource version number.

In the embodiment of the application, the resource acquisition request may include a resource acquisition request corresponding to a List mechanism and a resource acquisition request corresponding to a Watch mechanism in a LISTANDWATCH mechanism; and the target resource version number is the resource version number required by the client for acquiring the resource event information.

And 104, sending the searched target resource event information to the client.

The client is used for managing the plurality of sub-clusters to be managed according to the target resource event information.

Compared with the prior art, the embodiment establishes the Watch connection between the server and the plurality of sub-clusters to be managed, collects the resource event information of the plurality of sub-clusters to be managed, and realizes the aggregation of cluster resource events of the plurality of clusters, thereby realizing LISTANDWATCH functions of the server on the plurality of clusters; the first resource version number corresponding to the resource event information is uniformly coded to obtain the second resource version number corresponding to the resource event information, so that the multi-cluster resource can take the second resource version number as a unique identifier, and the identifier and the sequence are realized through the uniform second resource version number, so that the uniform management of the multi-cluster is realized; thereby improving the operation efficiency of the whole system and getting rid of the dependence on a third party storage medium; the single-cluster application can monitor and use the multi-cluster capability without modification, and then the LISTANDWATCH function of the client to the multiple clusters is realized.

To further illustrate the implementation of the method of this embodiment, this embodiment provides a specific method as shown in fig. 2, which includes:

step 201, cluster access information and cluster configuration information respectively corresponding to a plurality of sub-clusters to be managed are obtained.

In the embodiment of the application, the cluster access information can be APISERVER address information, kubernetes API SERVER is one of core components of Kubernetes clusters, provides a RESTful API interface of the clusters, and serves as a unified entry point for all resource operations; accordingly, the cluster configuration information may be Kubeconfig, kubeconfig file is a Kubernetes client configuration file, which is used to store authentication information, cluster details, and context information required to access the Kubernetes cluster, and this file is commonly named config or kubeconfig.

Step 202, based on the cluster access information and the cluster configuration information, the server and the plurality of sub-clusters to be managed are connected according to a LISTANDWATCH mechanism.

In some examples ClusterResource is a CRD definition, where the most predominant field is ClusterSpec, in ClusterSpec the addresses of the sub-clusters Kubeconfig, APISERVER and all K8s cluster resources GVR to be monitored are defined, respectively, where ClusterSpec represents a detailed specification describing the Kubernetes cluster overall configuration.

As shown in fig. 3, the method specifically may include a multi-cluster management module, a subset group data acquisition module, a ResourceVersion transcoding module and a WATCHCACHE module, where the subset group management module may include a Watch information collection channel (result chan) and an event handler (EVENT HANDLER); resourceVersion the transcoding module may include Aggregator Channel, RV Generator, delta Fifo; the WATCHCACHE module includes DELTA HANDER, a preset sliding window, DISPATCHEVENT, STORES memory search engine, and result chan. The embodiment included in step 202 may be performed by a multi-cluster management module in the server, which is responsible for registration, modification and deletion of the sub-clusters. When a new sub-cluster needs to be managed, the multi-cluster management module needs to analyze a resource type list which needs to be managed in the sub-cluster, namely Group-Version-Resources (GVR) of K8 s; meanwhile, the resource list of the sub-cluster can support change and deletion. In order to effectively manage multiple clusters, the CustomResourceDefinition (CRD) method of K8s is adopted. The CRD is a K8s custom resource, the CRD is monitored in Informer mode, corresponding logic processing is carried out on the sub-cluster event by utilizing the Handler hook function of Informer architecture, and flexible management of the sub-cluster can be realized.

It should be noted that in Kubernetes, group-Version-Resources (GVR) is a common way to represent API objects. It consists of Group, version and Resources, where Group represents the Group or class to which the API object belongs. The Kubernetes API is made up of a number of different groups, each group representing a class of related resources and operations. Version specifies the Version number used by the API object. Kubernetes supports multi-version APIs for backward compatibility and functional evolution. Resource represents a specific API Resource type, such as Pod, service, deployment, namespace, etc. A resource is a specific object instance that a user can create, update, query, or delete, in such a way that Kubernetes can effectively organize and manage its large API set and allow clients (e.g., kubectl or other applications using Kubernetes client libraries) to accurately specify API resources to operate. At the same time, this also provides infrastructure support for extending the Kubernetes function, and developers can introduce custom resource types by defining new groups and versions.

Step 203, determining information collecting channels corresponding to the plurality of sub-groups to be managed respectively, and collecting resource event information through the information collecting channels.

Optionally, step 203 may specifically include: if the target resource information in the resource event information to be collected is judged to be error resource event information, determining an error category corresponding to the target resource event information; resetting a first resource version number corresponding to the target resource event information in response to determining that the error category corresponding to the target resource event information is the target error category, and; and refreshing the other resource event information except the target resource event information in the sub-cluster to be managed corresponding to the reset target resource event information, and collecting the resource event information based on the refreshing result. And in response to determining that the error category corresponding to the target resource event information is other error categories except the target error category, re-collecting the resource event information corresponding to the first resource version number based on the first resource version number corresponding to the target resource event information to obtain the resource event information.

In some examples, the information collecting Channel may be a Result Channel, which may be a Channel structure implemented at an application layer for delivering the Result of the Kubernetes related operation, and illustratively, in a Kubernetes custom controller written in Go language, a developer may create a Channel as a Result Channel for receiving notification or Result information after the API resource operation is completed. Illustratively, when an asynchronous operation is performed (such as listening for a resource event, asynchronously calling a Kubernetes API to update a resource state, etc.), the operation result may be returned to the main flow through a channel for subsequent processing; in the embodiment of the application, the Result Channel is used for receiving the resource event information of the sub-cluster to be managed.

Illustratively, based on step 202, the embodiment included in step 203 may be performed by a subset group data collection module and ResourceVersion transcoding module in the server, where the subset group data collection module functions to perform a Watch operation on APISERVER of the sub-cluster and plug in Aggregator Channel the received Event. The acquisition module is triggered by a Handler hook function of the multi-Cluster management module, and the corresponding relation between the subset group data acquisition module and the Cluster/GVR is 1:1. After the sub-cluster data acquisition module is started, kubeConfig of the sub-clusters are parsed, according to KubeConfig, CLIENTSET is initialized, and on the basis of CLIENTSET, the Watch is executed on APISERVER of the sub-clusters (ResourceVersion is 0 for the first time), and the Result Channel of the Watch is acquired.

Further, resourceVersion the transcoding module aggregates the start of a thread (or coroutine) collected by the subset data collection module, and monitors the Result Channel of the Watch. When a normal Event is received, the current subset group data collection module's latest ResourceVersion is updated and the Event is plugged Aggregator Channel. When an abnormal Event is received, executing the Watch again based on ResourceVersion currently maintained by the module, so that Event breakpoint continuous transmission can be realized; when the exception is Expired ERROR, module maintenance ResourceVersion is reset to 0 and an exception Event is plugged into Aggregator Channel to perform a Refresh operation on the own child cluster resource.

And 204, uniformly encoding the first resource version number corresponding to the resource event information to obtain a second resource version number corresponding to the resource event information.

Optionally, step 204 may specifically include: aggregating the resource event information in a preset aggregation channel, and determining the name of the subset group to be managed corresponding to the resource event information in the preset aggregation channel; carrying out hash coding based on the name of the subset group to be managed and the first resource version number to obtain a second resource version number corresponding to the resource event information; accordingly, after step 204, the method of this embodiment further includes: and respectively storing the resource event information in a preset sliding window and a preset memory search engine through a preset first-in first-out pipeline.

The preset memory search engine is used for storing all resource event information, and the sliding window is used for storing the resource event information in a preset time period.

In the embodiment of the present application, the preset aggregation channel may be Aggregator Channel, and specifically, aggregator Channel is used to aggregate resource event information; the First-In First-Out pipe may be Delta Fifo, which is a data structure for storing and processing event data, and is particularly widely used In Kubernetes environments. In Kubernetes' client library client-go DeltaFIFO is a custom double-ended queue implementation that is particularly well suited for caching and deduplicating change events of API objects.

Illustratively, deltaFifo may store the add-delete events (deltas) of the resource object observed from Kubernetes API SERVER in order. When multiple changes occur in a short time for the same resource object, deltaFIFO can identify and merge these successive change events, only the change of the final state is reserved, thus avoiding the problem of repeatedly processing the same event. Both synchronous and asynchronous modes may be supported to handle events. In the synchronous mode, the next event in the queue can be directly acquired and processed through a Pop () method; in the asynchronous mode, a processor function may be provided to automatically trigger processing when a new event is entered into the queue. DeltaFIFO also tracks the ResourceVersion of the resource object to ensure that data is not inconsistent or outdated due to concurrent operations. In Kubernetes controller, deltaFIFO is used in conjunction with a Reflector to monitor changes in the state of resources within the cluster and take corresponding action based on those changes.

Illustratively, based on step 203, resourceVersion of the Event is transcoded into a multi-cluster version ResourceVersion, and then the transcoded Event is stuffed into Delta Filo. The effect of ResourceVersion of the multi-cluster version is to gather the events of multiple sub-clusters, and the two ResourceVersion can be compared in size like a single K8s cluster, so that the sequence of the events can be identified according to ResourceVersion.

Multi-cluster version ResourceVersion maintains a Hash structure, the Key of which is ClusterName and Value is single cluster ResourceVersion. The algorithm for the larger multiple cluster version ResourceVersion is as follows (ResourceVersion, resourceVersion for two ResourceVersion respectively): traversing the Hash structure of ResourceVersion to obtain Key1 and Value1; taking out Value2 by using Key1 to ResourceVersion; comparing Value1 with Value2, if the values are different, returning a comparison result, otherwise, continuing to execute the step 1; if no return is made to the traversal ResourceVersion, the two are equal.

Step 205, in response to receiving the resource obtaining request and the target resource version number uploaded by the client, searching in the resource event information corresponding to the second resource version number based on the target resource version number.

Optionally, step 205 may specifically include: in response to receiving a first resource acquisition request sent by a client, sending all resource event information in a preset memory search engine to the client; responding to a second resource acquisition request and a target resource version number sent by a receiving client, and searching in resource event information in a preset time period in a preset sliding window based on the target resource version number; and determining target resource event information corresponding to the target resource version number in a preset sliding window.

The target resource version number is contained in all the resource event information, and the first resource acquisition request is sent by the client based on a List mechanism in a LISTANDWATCH mechanism; the second resource acquisition request is sent by the client based on a Watch mechanism in LISTANDWATCH mechanisms; the target resource event information is the resource event information stored in a preset sliding window after the resource event information corresponding to the target resource version number is stored.

In the embodiment of the application, the preset sliding window can store the latest Event with the upper capacity limit for the Event sliding window, and the Event sliding window keeps being refreshed continuously along with the addition of the Event. The sliding window is designed to configure a Watch mechanism, and when a Watch request of a client is received, a binary search can be performed in the sliding window based on ResourceVersion (the events are arranged in ResourceVersion ascending order in the sliding window), and the events meeting the conditions are returned to the client. The sliding window is realized based on an array and double pointers, wherein the capability is the upper limit of the capacity of the sliding window; the cache is an array itself, and each element stores an Event; startIndex and endIndex are double pointers, namely a starting position and an ending position of a window respectively, when an Event is added to the sliding window, endIndex can slide back by one position, and when the number of the events reaches the capability, startIndex can slide forward by one position in advance to cover the oldest Event of the window.

Correspondingly, the preset memory search engine can be Indexer memory search engine, the Indexer memory search engine is a place where the multi-cluster K8s resource object is finally stored, the memory search engine can convert the Event into the final K8s resource object, and the cluster resource stored by the Indexer memory search engine can be consistent with all APISERVER of the multi-cluster as long as all the Event can be received. The final storage of Informer also depends on Indexer in-memory search engine, which is a key stone that can be implemented by the Watch mechanism.

In some examples, in stores, there is an important Hash table indexers, indexers where Key is the Name of the sub-cluster and Value is the Cache data structure. It can be seen that the resource objects of different sub-clusters are stored separately, which has the advantage that when a problem occurs in one sub-cluster, the resource objects of the sub-cluster can be conveniently emptied.

The cache structure, which is the location cache structure where the multi-cluster resource object is finally stored in memory, contains two data structures, cacheStorage and keyFunc. cacheStorage is ThreadSafeStore, which is a memory search engine supporting multi-threaded concurrency, and keyFunc, which is a Key value generation function of a resource object. Before storing the resource object, keyFunc is required to generate a Key value, and the generated Key value and the resource object are stored in cacheStorage together.

This ResourceVersion counter, clusterRVStore in module three. In WATCHCACHE, the function of the ResourceVersion counter is to record the ResourceVersion of the most recent Event, which ResourceVersion represents the current WATCHCACHE state.

When an Event is sent DELTA HANDLER to WATCHCACHE, the Event is passed to the Event sliding window and Indexer in-memory search engine processes (Add, update, and Delete), respectively, and finally, the ResourceVersion counter is assigned ResourceVersion for the current Event.

And 206, sending the searched target resource event information to the client.

Optionally, after step 206, the method of this embodiment further includes: and in response to the establishment of the Watch connection between the server and the client, forwarding the resource event information to be forwarded to the client through a preset forwarding channel.

The resource event information to be forwarded is the resource event information received by the preset forwarding channel after the Watch connection is established.

In the embodiment of the present application, the preset forwarding channel DISPATCHEVENT, DISPATCHEVENT may transmit an event object to an appropriate processor or listener, so as to trigger corresponding service logic processing.

Illustratively, based on step 204, the list method is implemented based on Http short connections, returning all cluster resources to the client at once; while Watch requires the latest Event to be passed to the client in real time, the Watch approach is based on Http long connections. Both the List method and Watch Fang Fati are implemented in the Handler at APISERVER, and the present application proposes that only the internal implementation logic of the method be needed.

Specifically, the List method returns to the client as long as there are two parts, all current cluster resource objects, and the latest version ResourceVersion. In the proposal of the application, all cluster resource objects of a plurality of clusters are acquired, a Indexer memory search engine in WATCHCACHE is required to be traversed, as the resource objects of a plurality of sub-clusters are respectively stored in a cache, the condition that the sub-clusters are renamed exists in different subsets, and in the traversing process, the de-duplication is also required, and the de-duplication strategy can be based on configuration, for example, only the first cluster resource object is taken, etc.; the method is convenient to acquire the latest version ResourceVersion, and returns to the ResourceVersion counter of WATCHCACHE.

Further, the Watch method has mainly two stages. Stage one: when a client initiates a Watch request, a CACHEWATCHER data structure is required to be established for the client; and the method is characterized in that the method comprises the steps of finding all events with ResourceVersion more than RVx based on a ResourceVersion parameter RVx transmitted by a Watch request and based on a binary search method in an Event sliding window, and returning the events to a client Channel; finally, CACHEWATCHER protocol is started to continuously accept real-time events transmitted from DELTA HANDLER. Stage two: at this time, the Http long connection with the client is established, and the real-time Event transmitted by DELTA HANDLER needs to be continuously transmitted to the client. WatcherBuffer an array of CACHEWATCHERS is maintained, the elements of which are CACHEWATCHER data structures, one for each client. In CACHEWATCHER, input is responsible for receiving events sent from DELTA HANDLER, and the result Channel is connected to the Http long connection, and the Event stuffed in the result Channel is directly sent to the client. Whenever DELTA HANDLER receives a new Event, it loops through CACHEWATCHERS arrays, issues the Event to the input Channel of each CACHEWATCHER, and when the input Channel receives a new Event, the associated CACHEWATCHER routine fetches the Event and plugs in its result Channel.

The problem to be solved by the embodiment of the application is to realize the nanotube and aggregation of multiple cluster resources based on the current APISERVER architecture, and externally provide interfaces (including Create, update and the like) such as a List and a Watch for the external K8s OpenAPI mode, so that the cloud native application running on a single K8s cluster can use the capability of multiple clusters under the condition of no perception.

It should be noted that the embodiment of the present application is realized based on the modification of K8S APISERVER or class APISERVER (which may be KARMADA APISERVER or Clusterpedia ApiServer). APISERVER can be understood as a complex Webservice, and each K8s cluster resource (such as Pod, deployment) is a Rest resource, which provides Rest services to the outside. Within APISERVER, for each K8s cluster resource, a Webserver-Handler-Storage call chain is maintained, which together provide Rest services to the outside.

The present application proposes that the modified part is the handle-Storage part of APISERVER. Handler can be understood as Handler of Webservice, implementing List and Watch methods; the Storage layer stores cluster resource objects for the List and Watch methods of the Handler to call, wherein the Storage layer of K8S APISERVER is cache+ Etcd, and the Storage layer of Clusterpedia ApiServer is MySQL.

Compared with the prior art, the embodiment establishes the Watch connection between the server and the plurality of sub-clusters to be managed, collects the resource Event information of the plurality of sub-clusters to be managed, and realizes the aggregation of cluster resources and events of the plurality of clusters, thereby realizing LISTANDWATCH functions of the server on the plurality of clusters; the first resource version number corresponding to the resource event information is uniformly coded to obtain the second resource version number corresponding to the resource event information, so that the multi-cluster resource can take the second resource version number as a unique identifier, and the identifier and the sequence are realized through the uniform second resource version number, so that the uniform management of the multi-cluster is realized; the memory-based storage engine WATCHCACHE stores the latest Event through the Event sliding window, and stores cluster resources of a plurality of sub-clusters through the Indexer memory search engine, so that the caching and storage of the multi-cluster Event and the cluster resources are realized, the running efficiency of the whole system is improved, and the dependence on a third-party storage medium is eliminated; the single-cluster application can monitor and use the multi-cluster capability without modification, and then the LISTANDWATCH function of the client to the multiple clusters is realized.

To illustrate the specific implementation procedure of the present embodiment, the following specific application examples are given, but not limited thereto:

Firstly, collecting cluster resource Event events of a subset group in a mode of a plurality of sub K8s clusters of a Watch; then, collecting Event events of the plurality of sub-clusters to a Aggregator Channel aggregation pipeline, and after recoding ResourceVersion of the Event events, plugging the Event events into a Delta Fifo pipeline by a transcoding module; after receiving the Event transmitted from the Delta Fifo pipeline, DELTA HANDLER of WATCHCACHE module updates the Event to the Cache sliding window and the memory search engine respectively, and sends the Event to the client side in the Watch through the dispatchEvent method. When a client initiates a List and Watch request, the multi-cluster resource and cluster resource Event are taken out from the cache sliding window and the store memory search engine and returned to the client.

Further, as a specific implementation of the method shown in fig. 1 and fig. 2, the present embodiment provides a cluster resource management apparatus, as shown in fig. 4, where the apparatus includes: the device comprises a collecting module 31, an encoding module 32, a searching module 33 and a sending module 34.

The collection module 31 is configured to establish a Watch connection between the server and the plurality of sub-clusters to be managed based on LISTANDWATCH mechanism, and collect resource event information of the plurality of sub-clusters to be managed;

The encoding module 32 is configured to uniformly encode a first resource version number corresponding to the resource event information to obtain a second resource version number corresponding to the resource event information, where the first resource version number is a resource version number obtained by encoding based on a single cluster, and the second resource version number is a resource version number obtained by uniformly encoding based on the multiple sub-clusters to be managed;

a searching module 33, configured to respond to receiving a resource acquisition request and a target resource version number uploaded by a client, and search in the resource event information corresponding to the second resource version number based on the target resource version number;

The sending module 34 is configured to send the searched target resource event information to the client, where the client is configured to manage the multiple sub-clusters to be managed according to the target resource event information.

In some examples of this embodiment, the collecting module 31 is further configured to obtain cluster access information and cluster configuration information corresponding to the plurality of sub-clusters to be managed respectively; correspondingly, the collection module 31 is specifically configured to establish a Watch connection between the server and the plurality of sub-clusters to be managed according to a LISTANDWATCH mechanism based on the cluster access information and the cluster configuration information; and determining information collecting channels corresponding to the plurality of sub-groups to be managed respectively, and collecting the resource event information through the information collecting channels.

In some examples of this embodiment, the collecting module 31 is specifically further configured to determine, if it is determined that the target resource information in the resource event information to be collected is error resource event information, an error category corresponding to the target resource event information; resetting a first resource version number corresponding to the target resource event information in response to determining that the error category corresponding to the target resource event information is a target error category, and; and refreshing the other resource event information except the target resource event information in the sub-cluster to be managed corresponding to the reset target resource event information, and collecting the resource event information based on a refreshing processing result. And in response to determining that the error category corresponding to the target resource event information is other error categories except the target error category, re-collecting the resource event information corresponding to the first resource version number based on the first resource version number corresponding to the target resource event information to obtain the resource event information.

In some examples of this embodiment, the encoding module 32 is specifically configured to aggregate the resource event information in a preset aggregation channel, and determine a to-be-managed subset group name corresponding to the resource event information in the preset aggregation channel; performing hash coding based on the to-be-managed subset group name and the first resource version number to obtain a second resource version number corresponding to the resource event information; correspondingly, the encoding module 32 is further configured to store the resource event information in a preset sliding window and a preset memory search engine through a preset first-in first-out pipeline, where the preset memory search engine is used for storing all the resource event information, and the sliding window is used for storing the resource event information in a preset time period.

In some examples of this embodiment, the lookup module 33 is specifically configured to send, to the client, all the resource event information in the preset memory search engine in response to receiving a first resource acquisition request sent by the client, where the all the resource event information includes the target resource version number, and the first resource acquisition request is sent by the client based on a List mechanism in a LISTANDWATCH mechanism; responding to a second resource acquisition request sent by the client and the target resource version number, and searching in resource event information in the preset time period in the preset sliding window based on the target resource version number, wherein the second resource acquisition request is sent by the client based on a Watch mechanism in a LISTANDWATCH mechanism; and determining target resource event information corresponding to the target resource version number in the preset sliding window, wherein the target resource event information is the resource event information stored in the preset sliding window after the resource event information corresponding to the target resource version number is stored.

In some examples of this embodiment, the sending module 34 is further configured to, in response to establishing a Watch connection with the client at the server, forward, to the client, resource event information to be forwarded through a preset forwarding channel, where the resource event information to be forwarded is resource event information received by the preset forwarding channel after the Watch connection is established.

It should be noted that, other corresponding descriptions of each functional unit related to the cluster resource management device provided in this embodiment may refer to corresponding descriptions in fig. 1 and fig. 2, and are not described herein again.

Based on the above-described methods shown in fig. 1 to 2, correspondingly, the present embodiment further provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor, implements the above-described methods shown in fig. 1 to 2.

Based on such understanding, the technical solution of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.), and includes several instructions for causing a computer device (may be a personal computer, a server, or a network device, etc.) to execute the method of each implementation scenario of the present application.

Based on the method shown in fig. 1 to 2 and the virtual device embodiment shown in fig. 4, in order to achieve the above object, the embodiment of the present application further provides an electronic device, such as a personal computer, a server, a notebook computer, a smart phone, a smart robot, and other smart terminals, where the device includes a storage medium and a processor; a storage medium storing a computer program; a processor for executing a computer program to implement the method as described above and shown in fig. 1-2.

Optionally, the entity device may further include a user interface, a network interface, a camera, a Radio Frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, and so on. The user interface may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), etc.

It will be appreciated by those skilled in the art that the above-described physical device structure provided in this embodiment is not limited to this physical device, and may include more or fewer components, or may combine certain components, or may be a different arrangement of components.

The storage medium may also include an operating system, a network communication module. The operating system is a program that manages the physical device hardware and software resources described above, supporting the execution of information handling programs and other software and/or programs. The network communication module is used for realizing communication among all components in the storage medium and communication with other hardware and software in the information processing entity equipment.

Based on the above-mentioned methods shown in fig. 1 to 2, the embodiments of the present application further provide a computer program product, where the computer program product includes a computer program, and when the computer program is executed by a processor, the method shown in fig. 1 to 2 is implemented, and when the computer program is executed by the processor, the method implemented by the computer program is referred to in various embodiments of the present application and will not be repeated herein.

From the above description of the embodiments, it will be apparent to those skilled in the art that the present application may be implemented by means of software plus necessary general hardware platforms, or may be implemented by hardware. Compared with the prior art, the method and the device have the advantages that iteration training is carried out on the loop generation countermeasure network, the second interface characteristic data obtained through each iteration are used for the next training, so that training sets used in the training process can be richer and richer, models with higher recognition accuracy can be obtained through training the models once, the models with higher accuracy can be trained through fewer sample sets, more interface characteristic data of abnormal webpages can be recognized in the process of recognizing the interface characteristic data, abnormal requests can be accurately filtered, normal request information is generated, and the generation effect of a generator can be effectively enhanced and the conversion accuracy is improved by sending characteristic constraint conditions into the loop generation countermeasure network. The interface characteristic data of the abnormal webpage can be converted into the interface characteristic data of the normal webpage by circularly generating the countermeasure network, and the request information obviously with the abnormal characteristic is filtered to a certain extent, so that malicious attacks of the website are effectively prevented, modification of the user request by an interceptor is filtered, the safety of the website is protected, the cost in the safety risk protection of the webpage is reduced, and the instantaneity and the effectiveness are also ensured.

It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The foregoing is only a specific embodiment of the application to enable those skilled in the art to understand or practice the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for managing cluster resources, comprising:

2. The method of claim 1, wherein before the service end establishes a Watch connection with a plurality of sub-clusters to be managed based on the LISTANDWATCH mechanism, the method further comprises:

acquiring cluster access information and cluster configuration information respectively corresponding to the plurality of sub-clusters to be managed;

The establishing connection between the server and the plurality of sub-clusters to be managed based on LISTANDWATCH mechanism, collecting the resource event information of the plurality of sub-clusters to be managed, includes:

Based on the cluster access information and the cluster configuration information, the server and a plurality of sub-clusters to be managed are connected according to a LISTANDWATCH mechanism;

And determining information collecting channels corresponding to the plurality of sub-groups to be managed respectively, and collecting the resource event information through the information collecting channels.

3. The method according to claim 2, wherein determining an information collecting channel corresponding to each of the plurality of sub-groups to be managed, collecting the resource event information through the information collecting channel, comprises:

if the target resource information in the resource event information to be collected is judged to be error resource event information, determining an error category corresponding to the target resource event information;

resetting a first resource version number corresponding to the target resource event information in response to determining that the error category corresponding to the target resource event information is a target error category, and;

Refreshing the other resource event information except the target resource event information in the sub-cluster to be managed corresponding to the reset target resource event information, and collecting the resource event information based on a refreshing processing result;

And in response to determining that the error category corresponding to the target resource event information is other error categories except the target error category, re-collecting the resource event information corresponding to the first resource version number based on the first resource version number corresponding to the target resource event information to obtain the resource event information.

4. The method of claim 1, wherein the uniformly encoding the first resource version number corresponding to the resource event information to obtain the second resource version number corresponding to the resource event information comprises:

Aggregating the resource event information in a preset aggregation channel, and determining a to-be-managed subset group name corresponding to the resource event information in the preset aggregation channel;

Performing hash coding based on the to-be-managed subset group name and the first resource version number to obtain a second resource version number corresponding to the resource event information;

After uniformly encoding the first resource version number corresponding to the resource event information to obtain the second resource version number corresponding to the resource event information, the method further comprises:

And respectively storing the resource event information in a preset sliding window and a preset memory search engine through a preset first-in first-out pipeline, wherein the preset memory search engine is used for storing all the resource event information, and the sliding window is used for storing the resource event information in a preset time period.

5. The method of claim 1, wherein the searching in the resource event information corresponding to the second resource version number based on the target resource version number in response to receiving the resource acquisition request and the target resource version number uploaded by the client comprises:

The method comprises the steps of responding to a first resource acquisition request sent by a client, and sending all resource event information in the preset memory search engine to the client, wherein all resource event information comprises the target resource version number, and the first resource acquisition request is sent by the client based on a List mechanism in a LISTANDWATCH mechanism;

responding to a second resource acquisition request sent by the client and the target resource version number, and searching in resource event information in the preset time period in the preset sliding window based on the target resource version number, wherein the second resource acquisition request is sent by the client based on a Watch mechanism in a LISTANDWATCH mechanism;

And determining target resource event information corresponding to the target resource version number in the preset sliding window, wherein the target resource event information is the resource event information stored in the preset sliding window after the resource event information corresponding to the target resource version number is stored.

6. The method of claim 5, wherein after said sending the found target resource event information to the client, the method further comprises:

And in response to the establishment of the Watch connection between the server and the client, forwarding the resource event information to be forwarded to the client through a preset forwarding channel, wherein the resource event information to be forwarded is the resource event information received by the preset forwarding channel after the Watch connection is established.

7. A cluster resource management apparatus, comprising:

8. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the method of any one of claims 1 to 6.

9. An electronic device comprising a storage medium, a processor and a computer program stored on the storage medium and executable on the processor, characterized in that the processor implements the method of any one of claims 1 to 6 when executing the computer program.

10. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 6.