CN117130730A - Metadata management method for federal Kubernetes cluster - Google Patents

Metadata management method for federal Kubernetes cluster Download PDF

Info

Publication number
CN117130730A
CN117130730A CN202311102423.XA CN202311102423A CN117130730A CN 117130730 A CN117130730 A CN 117130730A CN 202311102423 A CN202311102423 A CN 202311102423A CN 117130730 A CN117130730 A CN 117130730A
Authority
CN
China
Prior art keywords
metadata
field
management
field information
event
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311102423.XA
Other languages
Chinese (zh)
Inventor
陈晨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Construction Bank Corp
CCB Finetech Co Ltd
Original Assignee
China Construction Bank Corp
CCB Finetech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Construction Bank Corp, CCB Finetech Co Ltd filed Critical China Construction Bank Corp
Priority to CN202311102423.XA priority Critical patent/CN117130730A/en
Publication of CN117130730A publication Critical patent/CN117130730A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/4557Distribution of virtual machine instances; Migration and load balancing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45595Network integration; Enabling network access in virtual machine instances

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the specification provides a metadata management method for a federal Kubernetes cluster, which relates to the technical field of cloud computing, and comprises the steps of monitoring metadata change events of the cluster, wherein the change events comprise creation events, update events and deletion events; extracting field information of metadata of a change event; and feeding the field information back to the management and control platform so that the management and control platform can synchronously operate the structural body data stored on the management and control platform according to the field information, wherein the structural body data is obtained by packing the management and control platform according to the field information of the metadata of the creation event and the custom call field information. According to the method, automatic monitoring of the metadata of each cluster and extraction of field information are realized through the data synchronization management module, so that the management and control platform can perform unified configuration on calling of the metadata, management of the field information of the metadata is realized, and calling consumption of data among applications is facilitated; and saves the storage space of the management and control platform by storing field information.

Description

Metadata management method for federal Kubernetes cluster
Technical Field
The embodiment of the specification relates to the technical field of cloud computing, in particular to a metadata management method oriented to a federal Kubernetes cluster.
Background
Kubernetes is an open-source container orchestration and management platform that aims to provide a simple, reliable, and extensible way to deploy, manage, and extend containerized applications.
With development of cloud native technology and continuous expansion of application scenes, enterprises and organizations gradually deploy application programs into a plurality of Kubernetes clusters so as to meet the requirements of high availability, load balancing, geographic position optimization and the like. However, as the number of clusters increases, so does the complexity of cluster management. Thus, federal Kubernetes clusters appear as a multi-cluster management solution. Federal Kubernetes cluster resource metadata analytics management refers to the process of analyzing and managing resource metadata in a federal Kubernetes cluster. Resource metadata refers to data related to a resource in a Kubernetes cluster, such as Pod, service, deployment, etc., in which information such as attributes, states, and behaviors of the resource are described. By collecting, analyzing and visually managing the resource metadata, the distribution and the use condition of the resources in the cluster can be better known, so that the performance and the resource utilization rate of the cluster are optimized.
In actual production practice, the data center of the enterprise often has its own resource management platform to perform topology display and management on the underlying resources, however, the native Kubernetes metadata information is stored in ETCD (a distributed key value pair storage) components of each service cluster. If the management and control platform actively collects metadata distributed in each cluster, the management and control center needs a great storage space and has great workload because the clusters are provided with a plurality of metadata, and the quantity of the metadata in the clusters is huge and the metadata are updated frequently.
In view of this, the present description embodiments are directed to a metadata management method, system, and apparatus for federal Kubernetes clusters.
Disclosure of Invention
In view of the foregoing problems in the prior art, an object of an embodiment of the present disclosure is to provide a metadata management method, system and device for federal Kubernetes clusters, so as to solve the problem in the prior art that it is difficult to manage metadata distributed in each cluster.
In order to solve the above technical problems, the specific technical solutions of the embodiments of the present specification are as follows:
in a first aspect, embodiments of the present disclosure provide a metadata management method for a federal Kubernetes cluster, where the method is applied to a data synchronization management module, and the method includes:
Monitoring metadata change events of a cluster, wherein the cluster is provided with a plurality of change events, and the change events comprise creation events, update events and deletion events;
extracting field information of metadata of a change event, wherein the fields of the metadata comprise necessary fields and unnecessary fields, and the necessary fields at least comprise a metadata name field, a unique identification code field, a cluster name field, an event time node field and a configuration field;
and feeding back the type of the change event and the field information to a management and control platform so that the management and control platform can synchronously operate the structural body data stored on the management and control platform and corresponding to the metadata according to the field information, wherein the structural body data is obtained by packing the management and control platform according to the field information of the metadata of the creation event and the custom call field information.
Specifically, when the change event is a create event, extracting field information of metadata of the change event, further comprising:
information of all necessary fields of metadata in which the creation event occurs is extracted.
Specifically, when the change event is an update event, extracting field information of metadata of the change event, further includes:
Judging whether an update field of metadata of an update event is the necessary field;
if yes, extracting information of an update field of metadata of the update event, and information of a name field, a unique identification code field and a cluster name field of the metadata of the update event;
if not, the update event is ignored.
Specifically, when the change event is a deletion event, extracting field information of metadata of the change event further includes:
the information of the name field, the unique identification code field, and the cluster name field of the metadata of the occurrence of the deletion event is extracted.
Further, the method further comprises:
judging whether communication between the cluster and the management and control platform has faults or not;
if yes, sending a clearing alarm to the control platform so that the control platform clears all the structural body data stored on the control platform, acquiring field information of all metadata in all clusters and feeding back the field information to the control platform.
In a second aspect, embodiments of the present disclosure further provide a metadata management method for a federal Kubernetes cluster, where the method is applied to a management platform, and the method includes:
receiving field information corresponding to metadata change events in a cluster and types of the change events, wherein the field information is sent by a data synchronization management module, the cluster is provided with a plurality of change events, the change events comprise creation events, update events and deletion events, the fields of the metadata comprise necessary fields and unnecessary fields, and the necessary fields at least comprise metadata name fields, unique identification code fields, cluster name fields, event time node fields and configuration fields;
And according to the field information and the type of the change event, carrying out synchronous operation on the locally stored structure body data corresponding to the metadata, wherein the structure body data is obtained by packing the management and control platform according to the field information and the custom call field information of the metadata of the creation event.
Specifically, when the change event is an update event, according to the field information and the type of the change event, performing a synchronization operation on locally stored structure data corresponding to metadata, and further including:
judging whether target structure body data corresponding to the information of the name field, the unique identification code field and the cluster name field in the field information exists locally according to the field information;
if so, updating a target field in the target structure body data by using the information of the updated field in the field information;
if not, sending alarm information.
Specifically, when the change event is a deletion event, according to the field information and the type of the change event, performing a synchronization operation on locally stored structure data corresponding to metadata, and further including:
Judging whether target structure body data corresponding to the information of the name field, the unique identification code field and the cluster name field in the field information exists locally according to the field information;
if so, deleting the target structure data;
if not, sending alarm information.
Further, the method further comprises:
receiving a clearing alarm sent by the data synchronization management module;
clearing all the structure data stored locally;
receiving field information of all metadata sent by the data synchronization management module;
and packing according to the field information of each metadata and the custom call field information to obtain the structural body data corresponding to each metadata and storing the structural body data.
In a third aspect, embodiments of the present disclosure further provide a metadata management system for a federal Kubernetes cluster, including:
a plurality of clusters are arranged, and metadata generated by an application running on the clusters are stored on the clusters;
the data synchronization management module is used for monitoring metadata change events of the clusters, extracting field information of monitored metadata of change events, and feeding back the types of the change events and the field information to the management and control platform, wherein the change events comprise creation events, update events and deletion events, the fields of the metadata comprise necessary fields and unnecessary fields, and the necessary fields at least comprise metadata name fields, unique identification code fields, cluster name fields, event time node fields and configuration fields;
And the management and control platform is used for synchronously operating the structural body data stored on the management and control platform and corresponding to the metadata according to the field information and the type of the change event, wherein the structural body data is obtained by packing the management and control platform according to the field information and the custom call field information of the metadata of the creation event.
Preferably, the data synchronization management module is further configured to:
judging whether communication between the cluster and the management and control platform has faults or not; when a fault occurs, sending a clearing alarm to the management and control platform, and acquiring field information of all metadata in all clusters and feeding back the field information to the management and control platform;
and the management and control platform is also used for clearing all the locally stored structural body data after receiving the clearing alarm sent by the data synchronization management module, and packaging the field information and the custom call field information of each metadata sent by the data synchronization management module to obtain the structural body data corresponding to each metadata and storing the structural body data.
In a fourth aspect, embodiments of the present disclosure further provide a metadata management apparatus for a federal Kubernetes cluster, including:
the monitoring unit is used for monitoring metadata change events of a cluster, wherein the cluster is provided with a plurality of change events, and the change events comprise creation events, update events and deletion events;
An extracting unit, configured to extract field information of metadata of a change event, where a field of the metadata includes a necessary field and an unnecessary field, and the necessary field includes at least a metadata name field, a unique identifier field, a cluster name field, an event time node field, and a configuration field;
and the feedback unit is used for feeding back the type of the change event and the field information to the management and control platform so that the management and control platform can synchronously operate the structural body data stored on the management and control platform and corresponding to the metadata according to the field information, wherein the structural body data is obtained by packing the management and control platform according to the field information of the metadata of the creation event and the custom call field information.
In a fifth aspect, embodiments of the present disclosure further provide a metadata management apparatus for a federal Kubernetes cluster, including:
the device comprises a receiving unit, a data synchronization management module and a processing unit, wherein the receiving unit is used for receiving field information corresponding to metadata change events in a cluster and the types of the change events, the field information is sent by the data synchronization management module, the cluster is provided with a plurality of change events, the change events comprise creation events, update events and deletion events, the fields of the metadata comprise necessary fields and unnecessary fields, and the necessary fields at least comprise metadata name fields, unique identification code fields, cluster name fields, event time node fields and configuration fields;
And the synchronization unit is used for performing synchronization operation on the locally stored structure body data corresponding to the metadata according to the field information and the type of the change event, wherein the structure body data is obtained by packing the management and control platform according to the field information and the custom call field information of the metadata of the creation event.
In a sixth aspect, embodiments of the present disclosure further provide a computer device, including a memory, a processor, and a computer program stored on the memory and capable of running on the processor, where the processor implements the method provided in the above technical solution when the processor executes the computer program.
In a seventh aspect, the embodiments of the present specification further provide a computer readable storage medium storing a computer program, which when executed by a processor implements a method as provided in the above technical solution.
In an eighth aspect, embodiments of the present disclosure further provide a computer program product comprising at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by a processor to implement a method as provided in the above-mentioned technical solutions.
By adopting the technical scheme, the metadata management method, system and device for the federal Kubernetes cluster, which are provided by the embodiment of the specification, automatically monitor metadata in each cluster through the data synchronization management module, and synchronize the monitored field information of the metadata with a change event to the management and control platform so as to uniformly configure the calling of the cluster metadata, thereby realizing convenient and reliable management of the field information of the metadata and facilitating the calling consumption of the data between applications; and saving the storage space of the management and control platform through storing the field information of the metadata.
The foregoing and other objects, features and advantages of the embodiments of the invention will be apparent from the following more particular description of the preferred embodiments, as illustrated in the accompanying drawings.
Drawings
In order to more clearly illustrate the embodiments of the present description or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present description, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 shows a schematic step diagram of a metadata management method for federal Kubernetes clusters according to an embodiment of the present disclosure;
FIG. 2 is a schematic diagram showing steps for extracting field information of metadata of an update event in an embodiment of the present disclosure;
FIG. 3 is a schematic diagram illustrating a further step of the metadata management method for federal Kubernetes clusters according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram showing steps for synchronizing structure data corresponding to metadata of an update event in an embodiment of the present disclosure;
FIG. 5 is a schematic diagram showing steps for synchronizing structure data corresponding to metadata of a deletion event in an embodiment of the present disclosure;
FIG. 6 is a schematic diagram of a metadata management system for a federal Kubernetes cluster according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of a metadata management device facing a federal Kubernetes cluster according to an embodiment of the present disclosure;
FIG. 8 is a schematic diagram of a metadata management apparatus for federal Kubernetes clusters according to an embodiment of the present disclosure;
fig. 9 shows a schematic structural diagram of a computer device according to an embodiment of the present disclosure.
Description of the drawings:
100. clustering;
200. a control platform;
300. a data synchronization management module;
71. a monitoring unit;
72. an extraction unit;
73. a feedback unit;
81. a receiving unit;
82. a synchronization unit;
902. a computer device;
904. a processor;
906. a memory;
908. a driving mechanism;
910. an input/output module;
912. an input device;
914. an output device;
916. a presentation device;
918. a graphical user interface;
920. a network interface;
922. a communication link;
924. a communication bus.
Detailed Description
The technical solutions of the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is apparent that the described embodiments are only some embodiments of the present specification, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the present disclosure.
It should be noted that the terms "first," "second," and the like in the description and the claims, and in the foregoing figures, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the present description described herein may be capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, apparatus, article, or device that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or device.
In order to solve the above problems, the embodiments of the present disclosure provide a metadata management method, system and device for a federal Kubernetes cluster, which can implement convenient and reliable management and control of metadata in the cluster by a management and control platform. FIG. 1 is a schematic diagram of the steps of a metadata management method for a federal Kubernetes cluster provided in an embodiment of the present specification, which provides the method operational steps described in the examples or flowcharts, but may include more or fewer operational steps based on conventional or non-inventive labor. The order of steps recited in the embodiments is merely one way of performing the order of steps and does not represent a unique order of execution. When a system or apparatus product in practice is executed, it may be executed sequentially or in parallel according to the method shown in the embodiments or the drawings. As shown in fig. 1, the method is applied to a data synchronization management module, and the method may include:
s110: monitoring metadata change events of a cluster, wherein the cluster is provided with a plurality of change events, and the change events comprise creation events, update events and deletion events.
Metadata in the embodiments of the present disclosure, which is also called intermediate data and relay data, is data describing data, and is mainly used for describing information of data attributes, and is used to support functions such as indicating storage locations, historical data, searching resources, and file recording.
The monitoring of the data synchronization management module can be realized through a monitoring (watch) mechanism of Kubernetes, and when the metadata of the underlying resource such as Alcorgroup, deployment, statefulSet, pod, PV, PVC, job is monitored to generate a change event, the change type (namely the creation event create, update event update and delete event delete) is pushed to the data synchronization management module in the form of an event (event).
S120: the method comprises the steps of extracting field information of metadata of a change event, wherein the fields of the metadata comprise necessary fields and unnecessary fields, and the necessary fields at least comprise a metadata name field, a unique identification code field, a cluster name field, an event time node field and a configuration field.
The necessary fields are fields required by the management and control platform to construct structural body data corresponding to the metadata. The configuration fields may further include affinity configuration fields, restart policy fields, memory configuration fields, CPU configuration fields, storage configuration fields, and the like. I.e. in the present embodiment, the necessary fields are fields required for consumption commonly used between applications.
S130: and feeding back the type of the change event and the field information to a management and control platform so that the management and control platform can synchronously operate the structural body data stored on the management and control platform and corresponding to the metadata according to the field information, wherein the structural body data is obtained by packing the management and control platform according to the field information of the metadata of the creation event and the custom call field information.
In the embodiment of the present disclosure, the management and control center functions as a transfer station or a data warehouse of metadata, which may store data of not only the Kubernetes platform, but also data pushed by other multiple platforms. The management and control platform encapsulates and packages the field information of the received metadata and the customized calling field information into structural body data, so that unified management of the field information of the metadata and unified configuration of the calling interface are realized. When a certain application a running on the cluster 1 needs to consume metadata of the application B, C, D (the application B, C, D may be running on the cluster 1 or on another cluster), the application a needs to call a corresponding query interface provided by the application B, C, D to obtain corresponding information; by the method provided by the embodiment of the specification, the data of the application B, C, D is stored on the management and control platform, and the management and control platform provides a universal call interface, so that the application A can acquire the corresponding data of the application B, C, D only by calling the interface of the management and control platform, thereby facilitating the access consumption of each application to the metadata.
Meanwhile, because the partial field information which is the metadata and is stored in the management and control center is not the metadata, the management and control center can meet the consumption requirement of each application on the metadata, and the convenient and reliable management of the metadata of each Kubernetes cluster is realized; but also saves the storage space of the management and control platform.
In summary, according to the metadata management method for federal Kubernetes clusters provided in the embodiments of the present disclosure, metadata in each cluster is automatically monitored through a data synchronization management module, and monitored metadata with a change event is synchronized to a management and control platform, so that convenient and reliable synchronization and management operation on cluster metadata are realized.
When the change event is a create event, step S120: extracting field information of metadata of the change event may include:
information of all necessary fields of metadata in which the creation event occurs is extracted.
And the management and control platform packages all necessary fields of the metadata of the occurrence creation event fed back by the data synchronization management component and the customized calling field information of the management and control platform to obtain the structural body data corresponding to the metadata, and stores the structural body data in a local place for consumption of each application.
It should be noted that, when the management and control platform obtains the structure data through packaging, besides the customized call field information, some other concepts of the management and control platform may be packaged in the structure, for example, an application system field and a deployment unit information field.
Further, as shown in fig. 2, in the embodiment of the present disclosure, when the change event is an update event, step S120: extracting field information of metadata of the change event may include:
S210: and judging whether an update field of metadata of the update event is the necessary field.
When metadata is changed, the name field, the unique identification code field, and the cluster name field are not changed. Therefore, it is necessary to determine whether the update field of the metadata of the update event is a necessary field other than the information of the name field, the unique identification code field, and the cluster name field.
S220: if so, extracting information of an update field of metadata of the update event, and information of a name field, a unique identification code field and a cluster name field of the metadata of the update event.
In the embodiment of the present specification, the information of the name field, the unique identification code field, and the cluster name field may be used to uniquely determine metadata. The management and control platform can be used for inquiring the target structure body data corresponding to the occurrence update event according to the name field, the unique identification code field and the cluster name field; and updating the corresponding field in the target structure body data by using the updated field.
In the embodiment of the present disclosure, the metadata of the update event may have one update field or a plurality of update fields. The update method may be to update the original information of the field, for example, to change the information of the affinity configuration field from a to b. The updating mode can also be to delete one or more fields, namely, the information of one or more fields is updated to be empty from the original information. It should be noted that when all necessary fields of a certain metadata are deleted, the event should be classified as a deletion event.
S230: if not, the update event is ignored.
For metadata of an update event, the event time node field includes not only a create event time node field and an update event time node field. The creation event time node field is the time node when the metadata is created, and the update event time node is the time node when the metadata generates the update event.
That is, when the changed field of the metadata of the update event is an unnecessary field, the data synchronization management module does not feed back the update event to the management platform. According to the metadata management method for the federal Kubernetes cluster, which is provided by the embodiment of the specification, the update fields of the metadata with the update event are selectively synchronized to the management and control platform, so that the management of the metadata which is frequently changed under a plurality of clusters is realized, and meanwhile, the workload of a data synchronization management module is greatly reduced.
Further, in the embodiment of the present disclosure, when the change event is a delete event, step S120: extracting field information of metadata of the change event may include:
the information of the name field, the unique identification code field, and the cluster name field of the metadata of the occurrence of the deletion event is extracted.
As described above, a deletion event occurs for metadata when information of all necessary fields of the metadata is deleted, or information of all necessary fields and at least a part of unnecessary fields of the metadata is deleted. At this time, only the information of the name field, the unique identification code field and the cluster name field of the metadata is extracted to the management and control platform, so that the management and control platform queries the target structure body data corresponding to the metadata according to the information of the name field, the unique identification code field and the cluster name field, and deletes the target structure body data from the management and control platform.
According to the metadata management method for the federal Kubernetes cluster, which is provided by the embodiment of the specification, the field information of metadata extracted by the data synchronization management component for different types of change events is different: for metadata of an occurrence creation event, the data synchronization management component extracts information of all necessary fields of the metadata; for metadata of an update event, the data synchronization management component only needs to extract a change field and uniquely determine field information of the metadata; and for the metadata of the deletion event, the data synchronization management component only needs to extract the field information which uniquely determines the metadata, thereby being beneficial to improving the response rate of the data synchronization management component to the change event and improving the extraction efficiency of the data synchronization management component to the metadata related field of the change event. Meanwhile, the management and control platform is convenient to synchronize the structure body data corresponding to the metadata of different change types according to the difference of the received field information.
Further, in an embodiment of the present disclosure, the method further includes:
and judging whether the communication between the cluster and the management and control platform has faults or not.
Specifically, the judgment can be performed according to the state of the communication interface; it may also be judged by the consumer's consumption results, as well as some other feedback mechanism. For example, the application A invokes structural body data corresponding to the metadata of the application B from the management and control platform and interacts with the application B based on the structural body data; when in interaction, the consumption failure of the application A and the application B caused by the misinformation of the structural body data can judge that the latest structural body data which is stored in the management and control platform and corresponds to the application B metadata, namely the metadata of the application B generates an update event and the management and control platform does not receive and update in time, so that the communication failure between the cluster and the management and control platform can be judged.
If yes, sending a clearing alarm to the control platform so that the control platform clears all the structural body data stored on the control platform, acquiring field information of all metadata in all clusters and feeding back the field information to the control platform.
When communication between the cluster and the management and control platform fails, the following non-corresponding situations may exist between the metadata in the cluster and the structure data stored on the management and control platform:
1. Some metadata exists in the cluster, and structural body data corresponding to the metadata does not exist on the management and control platform;
2. some metadata is deleted in the cluster, but structural body data corresponding to the metadata also exists on the management and control platform;
3. the information of the necessary field of a certain metadata in the cluster is inconsistent with the information of the corresponding field in the structural body data corresponding to the metadata on the management and control platform;
the metadata of the creation event when synchronizing to the management platform includes a cluster name field in the necessary fields, which is used to characterize which cluster the metadata originates from. In the embodiment of the present disclosure, when a communication between a certain cluster and a management and control platform fails, in order to avoid an application calling to incorrect structural body data, the structural body data corresponding to the metadata of the cluster may be deleted by using the cluster name field.
As shown in fig. 3, the embodiment of the present disclosure further provides a metadata management method for a federal Kubernetes cluster, where the method is applied to a management platform, and the method includes:
s310: receiving field information corresponding to metadata change events in a cluster and types of the change events, wherein the field information is sent by a data synchronization management module, the cluster is provided with a plurality of change events, the change events comprise creation events, update events and deletion events, the fields of the metadata comprise necessary fields and unnecessary fields, and the necessary fields at least comprise metadata name fields, unique identification code fields, cluster name fields, event time node fields and configuration fields.
S320: and according to the field information and the type of the change event, carrying out synchronous operation on the locally stored structure body data corresponding to the metadata, wherein the structure body data is obtained by packing the management and control platform according to the field information and the custom call field information of the metadata of the creation event.
The management and control platform encapsulates and packages the field information of the received metadata and the customized calling field information thereof into structure data, so that unified configuration of calling of the structure data is realized, and access and calling of each application to the structure data stored in the management and control platform are facilitated; unified management of field information of metadata from each cluster is achieved.
As shown in fig. 4, in the embodiment of the present disclosure, when the change event is an update event, in step S320, according to the field information and the type of the change event, performing a synchronization operation on locally stored structure body data corresponding to metadata may include:
s410: and judging whether target structural body data corresponding to the information of the name field, the unique identification code field and the cluster name field in the field information exists locally according to the field information.
Since the information of the name field, the unique identification code field, and the cluster name field may be used to uniquely determine metadata, the management and control platform may search for the structure body data corresponding to the metadata using the information of the name field, the unique identification code field, and the cluster name field.
S420: and if the field information exists, updating the target field in the target structure body data by using the information of the updated field in the field information.
S430: if not, sending alarm information.
It may be indicated that the structure data is not synchronously updated to the management and control platform, and communication between the management and control platform and the cluster may be failed, so that alarm information is sent.
Similarly, as shown in fig. 5, when the change event is a deletion event, in step S320, according to the field information and the type of the change event, performing a synchronization operation on the locally stored structure body data corresponding to the metadata may further include:
s510: and judging whether target structural body data corresponding to the information of the name field, the unique identification code field and the cluster name field in the field information exists locally according to the field information.
S520: and if so, deleting the target structure data.
S530: if not, sending alarm information.
When the deleting event occurs in the metadata, the information of all necessary fields is deleted, but in the embodiment of the present disclosure, the data synchronization management component only needs to extract the information for uniquely determining the name field, the unique identification code field and the cluster name field of the metadata, and does not need to extract all the information of all necessary fields, which is beneficial to improving the response rate of the data synchronization management component to the deleting event and the information extraction efficiency of the above fields of the metadata of the deleting event. And when the management and control platform does not inquire the structure body data corresponding to the metadata uniquely determined by the information of the name field, the unique identification code field and the cluster name field, the structure body data can be indicated to be deleted or not synchronously updated to the management and control platform, namely, the communication between the management and control platform and the cluster is failed, so that the alarm information is sent.
It should be noted that, in some preferred embodiments, when the metadata has an update event or a delete event, if the management platform queries, according to the field information sent by the data synchronization management module, more than one locally stored structure body data corresponding to the information of the name field, the unique identification code field and the cluster name field, it will also determine that a fault exists between the management platform and the cluster, and send alarm information.
In some preferred embodiments, the management and control platform may check the type of the change event according to the number of fields when receiving the field information and the type of the change type sent by the data synchronization management module. The sum of the number of all necessary fields is recorded as N, N >3, and an exemplary specific verification method can be as follows:
when the number of fields sent by the data synchronization management module is equal to 3 (namely, the sum of the number of fields of the name field, the unique identification code field and the cluster name field), checking whether the change type is a deletion event;
when the number of the transmitted fields is equal to N, checking whether the change event is a creation event; and when the number of the transmitted fields is more than 3 and less than N, checking whether the change event is an update event.
The data synchronization management module can also feed back the type of the change event to the management and control platform so that the management and control platform can synchronously operate the corresponding structure data according to the field information and the type of the change event.
Further, the method further comprises:
receiving a clearing alarm sent by the data synchronization management module;
clearing all the structure data stored locally;
Receiving field information of all metadata sent by the data synchronization management module;
and packing according to the field information of each metadata and the custom call field information to obtain the structural body data corresponding to each metadata and storing the structural body data.
When the data synchronization management module determines that the communication between the management and control platform and the cluster fails, a clearing alarm is sent to the management and control platform, and the management and control platform clears all the structural body data stored on the management and control platform; and according to the field information of each metadata sent by the structural body data, creating corresponding structural body data for each metadata in a mode of creating an event so as to ensure that each application calls the correct structural body data.
As shown in fig. 6, the embodiment of the present disclosure further provides a metadata management system facing to a federal Kubernetes cluster, including:
a plurality of clusters 100, said clusters 100 being provided, said clusters 100 having stored thereon metadata generated by an application running thereon.
The data synchronization management module 300 is configured to monitor a metadata update event of the cluster 100, extract field information of the monitored metadata of the event, and feed back the type of the event and the field information to the management platform 200, where the event includes a create event, an update event, and a delete event, and the fields of the metadata include a necessary field and an unnecessary field, and the necessary field includes at least a metadata name field, a unique identifier field, a cluster name field, an event time node field, and a configuration field.
And the management and control platform 200 is used for carrying out synchronous operation on the structural body data stored on the management and control platform and corresponding to the metadata according to the field information and the type of the change event, wherein the structural body data is obtained by packing the management and control platform according to the field information and the custom call field information of the metadata of the creation event.
Further, the data synchronization management module 300 is further configured to:
judging whether communication between the cluster 100 and the management and control platform 200 has a fault or not; sending a clearing alarm to the management and control platform 200 when a fault occurs, and acquiring field information of all metadata in all clusters 100 and feeding back the field information to the management and control platform 200;
the management and control platform 200 is further configured to, after receiving the clearing alert sent by the data synchronization management module 300, clear all the locally stored structure body data, package field information of each metadata sent by the data synchronization management module 300 and custom call field information to obtain structure body data corresponding to each metadata, and store the structure body data.
As shown in fig. 7, the embodiment of the present disclosure further provides a metadata management device facing to a federal Kubernetes cluster, including:
A listening unit 71, configured to listen to metadata change events of a cluster, where the cluster is provided with a plurality of change events, where the change events include a create event, an update event, and a delete event;
an extracting unit 72 for extracting field information of metadata of an occurrence of a change event, the fields of the metadata including a necessary field including at least a metadata name field, a unique identification code field, a cluster name field, an event time node field, and a configuration field, and an unnecessary field;
and the feedback unit 73 is configured to feed back the type of the change event and the field information to a management and control platform, so that the management and control platform performs synchronous operation on the structural body data corresponding to the metadata stored on the management and control platform according to the field information, where the structural body data is obtained by packing the management and control platform according to the field information of the metadata of the creation event and the custom call field information.
As shown in fig. 8, the embodiment of the present disclosure further provides a metadata management device facing to a federal Kubernetes cluster, including:
a receiving unit 81, configured to receive field information corresponding to a metadata change event in a cluster and a type of the change event, where the cluster is provided with a plurality of change events, where the change event includes a create event, an update event, and a delete event, and the metadata field includes a necessary field and an unnecessary field, and the necessary field includes at least a metadata name field, a unique identifier field, a cluster name field, an event time node field, and a configuration field;
And the synchronization unit 82 is configured to perform a synchronization operation on locally stored structure body data corresponding to the metadata according to the field information and the type of the change event, where the structure body data is obtained by packaging the management and control platform according to the field information and the custom call field information of the metadata where the creation event occurs.
The beneficial effects obtained by the device provided by the embodiment of the present disclosure are consistent with those obtained by the above method, and will not be described herein.
As shown in fig. 9, a metadata management device for a federal Kubernetes cluster in the present specification may be a computer device in the present embodiment, which is provided in the embodiments of the present specification, and performs the foregoing method of the present specification. The computer device 902 may include one or more processors 904, such as one or more Central Processing Units (CPUs), each of which may implement one or more hardware threads. The computer device 902 may also include any memory 906 for storing any kind of information, such as code, settings, data, etc. For example, and without limitation, the memory 906 may include any one or more of the following combinations: any type of RAM, any type of ROM, flash memory devices, hard disks, optical disks, etc. More generally, any memory may store information using any technique. Further, any memory may provide volatile or non-volatile retention of information. Further, any memory may represent fixed or removable components of computer device 902. In one case, when the processor 904 executes associated instructions stored in any memory or combination of memories, the computer device 902 can perform any of the operations of the associated instructions. The computer device 902 also includes one or more drive mechanisms 908 for interacting with any memory, such as a hard disk drive mechanism, optical disk drive mechanism, and the like.
The computer device 902 may also include an input/output module 910 (I/O) for receiving various inputs (via an input device 912) and for providing various outputs (via an output device 914). One particular output mechanism may include a presentation device 916 and an associated Graphical User Interface (GUI) 918. In other embodiments, input/output module 910 (I/O), input device 912, and output device 914 may not be included, but merely as a computer device in a network. The computer device 902 may also include one or more network interfaces 920 for exchanging data with other devices via one or more communication links 922. One or more communication buses 924 couple the above-described components together.
The communication link 922 may be implemented in any manner, for example, through a local area network, a wide area network (e.g., the internet), a point-to-point connection, etc., or any combination thereof. Communication link 922 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.
Corresponding to the method as shown in fig. 1 to 5, the present embodiment also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the above method.
The present description also provides computer-readable instructions, wherein the program therein causes the processor to perform the method as shown in fig. 1 to 5 when the processor executes the instructions.
The present description also provides a computer program product comprising at least one instruction or at least one program loaded into and executed by a processor to implement the method as shown in fig. 1-5.
It should be understood that, in various embodiments of the present disclosure, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation of the embodiments of the present disclosure.
It should also be understood that, in the embodiments of the present specification, the term "and/or" is merely one association relationship describing the association object, meaning that three relationships may exist. For example, a and/or B may represent: a exists alone, A and B exist together, and B exists alone. In the present specification, the character "/" generally indicates that the front and rear related objects are an or relationship.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the various example components and steps have been generally described in terms of function in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present specification.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
In the several embodiments provided in this specification, it should be understood that the disclosed systems, apparatuses, and methods may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. In addition, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices, or elements, or may be an electrical, mechanical, or other form of connection.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purposes of the embodiments of the present description.
In addition, each functional unit in each embodiment of the present specification may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present specification is essentially or a part contributing to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present specification. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The principles and embodiments of the present specification are explained in this specification using specific examples, the above examples being provided only to assist in understanding the method of the present specification and its core ideas; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope based on the ideas of the present specification, the present description should not be construed as limiting the present specification in view of the above.

Claims (16)

1. A metadata management method for a federal Kubernetes cluster, wherein the method is applied to a data synchronization management module, and the method comprises:
monitoring metadata change events of a cluster, wherein the cluster is provided with a plurality of change events, and the change events comprise creation events, update events and deletion events;
extracting field information of metadata of a change event, wherein the fields of the metadata comprise necessary fields and unnecessary fields, and the necessary fields at least comprise a metadata name field, a unique identification code field, a cluster name field, an event time node field and a configuration field;
and feeding back the type of the change event and the field information to a management and control platform so that the management and control platform can synchronously operate the structural body data stored on the management and control platform and corresponding to the metadata according to the field information, wherein the structural body data is obtained by packing the management and control platform according to the field information of the metadata of the creation event and the custom call field information.
2. The method of claim 1, wherein extracting field information of metadata of an occurrence of a change event when the change event is a create event, further comprises:
information of all necessary fields of metadata in which the creation event occurs is extracted.
3. The method of claim 1, wherein extracting field information of metadata of an occurrence of a change event when the change event is an update event, further comprises:
judging whether an update field of metadata of an update event is the necessary field;
if yes, extracting information of an update field of metadata of the update event, and information of a name field, a unique identification code field and a cluster name field of the metadata of the update event;
if not, the update event is ignored.
4. The method of claim 1, wherein when the change event is a delete event, extracting field information of metadata of the change event further comprises:
the information of the name field, the unique identification code field, and the cluster name field of the metadata of the occurrence of the deletion event is extracted.
5. The method according to claim 1, wherein the method further comprises:
Judging whether communication between the cluster and the management and control platform has faults or not;
if yes, sending a clearing alarm to the control platform so that the control platform clears all the structural body data stored on the control platform, acquiring field information of all metadata in all clusters and feeding back the field information to the control platform.
6. A metadata management method for a federal Kubernetes cluster, wherein the method is applied to a management and control platform, and the method comprises:
receiving field information corresponding to metadata change events in a cluster and types of the change events, wherein the field information is sent by a data synchronization management module, the cluster is provided with a plurality of change events, the change events comprise creation events, update events and deletion events, the fields of the metadata comprise necessary fields and unnecessary fields, and the necessary fields at least comprise metadata name fields, unique identification code fields, cluster name fields, event time node fields and configuration fields;
and according to the field information and the type of the change event, carrying out synchronous operation on the locally stored structure body data corresponding to the metadata, wherein the structure body data is obtained by packing the management and control platform according to the field information and the custom call field information of the metadata of the creation event.
7. The method of claim 6, wherein when the change event is an update event, synchronizing locally stored structure data corresponding to metadata according to the field information and the type of the change event, further comprising:
judging whether target structure body data corresponding to the information of the name field, the unique identification code field and the cluster name field in the field information exists locally according to the field information;
if so, updating a target field in the target structure body data by using the information of the updated field in the field information;
if not, sending alarm information.
8. The method of claim 6, wherein when the change event is a delete event, synchronizing locally stored structure data corresponding to metadata according to the field information and the type of the change event, further comprising:
judging whether target structure body data corresponding to the information of the name field, the unique identification code field and the cluster name field in the field information exists locally according to the field information;
If so, deleting the target structure data;
if not, sending alarm information.
9. The method of claim 6, wherein the method further comprises:
receiving a clearing alarm sent by the data synchronization management module;
clearing all the structure data stored locally;
receiving field information of all metadata sent by the data synchronization management module;
and packing according to the field information of each metadata and the custom call field information to obtain the structural body data corresponding to each metadata and storing the structural body data.
10. A federal Kubernetes cluster-oriented metadata management system, comprising:
a plurality of clusters are arranged, and metadata generated by an application running on the clusters are stored on the clusters;
the data synchronization management module is used for monitoring metadata change events of the clusters, extracting field information of monitored metadata of change events, and feeding back the types of the change events and the field information to the management and control platform, wherein the change events comprise creation events, update events and deletion events, the fields of the metadata comprise necessary fields and unnecessary fields, and the necessary fields at least comprise metadata name fields, unique identification code fields, cluster name fields, event time node fields and configuration fields;
And the management and control platform is used for synchronously operating the structural body data stored on the management and control platform and corresponding to the metadata according to the field information and the type of the change event, wherein the structural body data is obtained by packing the management and control platform according to the field information and the custom call field information of the metadata of the creation event.
11. The system of claim 10, wherein the data synchronization management module is further configured to:
judging whether communication between the cluster and the management and control platform has faults or not; when a fault occurs, sending a clearing alarm to the management and control platform, and acquiring field information of all metadata in all clusters and feeding back the field information to the management and control platform;
and the management and control platform is also used for clearing all the locally stored structural body data after receiving the clearing alarm sent by the data synchronization management module, and packaging the field information and the custom call field information of each metadata sent by the data synchronization management module to obtain the structural body data corresponding to each metadata and storing the structural body data.
12. A metadata management device for a federal Kubernetes cluster, comprising:
The monitoring unit is used for monitoring metadata change events of a cluster, wherein the cluster is provided with a plurality of change events, and the change events comprise creation events, update events and deletion events;
an extracting unit, configured to extract field information of metadata of a change event, where a field of the metadata includes a necessary field and an unnecessary field, and the necessary field includes at least a metadata name field, a unique identifier field, a cluster name field, an event time node field, and a configuration field;
and the feedback unit is used for feeding back the type of the change event and the field information to the management and control platform so that the management and control platform can synchronously operate the structural body data stored on the management and control platform and corresponding to the metadata according to the field information, wherein the structural body data is obtained by packing the management and control platform according to the field information of the metadata of the creation event and the custom call field information.
13. A metadata management device for a federal Kubernetes cluster, comprising:
the device comprises a receiving unit, a data synchronization management module and a processing unit, wherein the receiving unit is used for receiving field information corresponding to metadata change events in a cluster and the types of the change events, the field information is sent by the data synchronization management module, the cluster is provided with a plurality of change events, the change events comprise creation events, update events and deletion events, the fields of the metadata comprise necessary fields and unnecessary fields, and the necessary fields at least comprise metadata name fields, unique identification code fields, cluster name fields, event time node fields and configuration fields;
And the synchronization unit is used for performing synchronization operation on the locally stored structure body data corresponding to the metadata according to the field information and the type of the change event, wherein the structure body data is obtained by packing the management and control platform according to the field information and the custom call field information of the metadata of the creation event.
14. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 9 when executing the computer program.
15. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 9.
16. A computer program product comprising at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by a processor to implement the method of any one of claims 1 to 9.
CN202311102423.XA 2023-08-29 2023-08-29 Metadata management method for federal Kubernetes cluster Pending CN117130730A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311102423.XA CN117130730A (en) 2023-08-29 2023-08-29 Metadata management method for federal Kubernetes cluster

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311102423.XA CN117130730A (en) 2023-08-29 2023-08-29 Metadata management method for federal Kubernetes cluster

Publications (1)

Publication Number Publication Date
CN117130730A true CN117130730A (en) 2023-11-28

Family

ID=88852391

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311102423.XA Pending CN117130730A (en) 2023-08-29 2023-08-29 Metadata management method for federal Kubernetes cluster

Country Status (1)

Country Link
CN (1) CN117130730A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117573295A (en) * 2024-01-16 2024-02-20 之江实验室 Multi-cluster component management method and device based on k8s and computer equipment
CN117707794A (en) * 2024-02-05 2024-03-15 之江实验室 Heterogeneous federation-oriented multi-class job distribution management method and system
CN117573295B (en) * 2024-01-16 2024-05-28 之江实验室 Multi-cluster component management method and device based on k8s and computer equipment

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117573295A (en) * 2024-01-16 2024-02-20 之江实验室 Multi-cluster component management method and device based on k8s and computer equipment
CN117573295B (en) * 2024-01-16 2024-05-28 之江实验室 Multi-cluster component management method and device based on k8s and computer equipment
CN117707794A (en) * 2024-02-05 2024-03-15 之江实验室 Heterogeneous federation-oriented multi-class job distribution management method and system

Similar Documents

Publication Publication Date Title
CN106713487B (en) Data synchronization method and device
US10592521B2 (en) Method and system for implementing target model configuration metadata for a log analytics system
CN102291416B (en) A kind of method and system of client and server bi-directional synchronization
CN101065731B (en) System and method for runtime of redundant host pair for processing control network environment
CN101076736B (en) Device and method for configuring redundancy in a supervisory process control system
CN104408071A (en) Distributive database high-availability method and system based on cluster manager
CN109656742B (en) Node exception handling method and device and storage medium
CN107992537B (en) Service attribute transmission method, device, computer equipment and storage medium
CN112667362B (en) Method and system for deploying Kubernetes virtual machine cluster on Kubernetes
CN105653425A (en) Complicated event processing engine based monitoring system
CN112506702B (en) Disaster recovery method, device, equipment and storage medium for data center
CN111064626B (en) Configuration updating method, device, server and readable storage medium
CN110895488B (en) Task scheduling method and device
CN110543512B (en) Information synchronization method, device and system
CN109361777B (en) Synchronization method, synchronization system and related device for distributed cluster node states
CN117130730A (en) Metadata management method for federal Kubernetes cluster
CN111857998A (en) Configurable timing task scheduling method and system
CN113422692A (en) Method, device and storage medium for detecting and processing node faults in K8s cluster
CN114064217B (en) OpenStack-based node virtual machine migration method and device
US11544091B2 (en) Determining and implementing recovery actions for containers to recover the containers from failures
CN113765690A (en) Cluster switching method, system, device, terminal, server and storage medium
CN111008095A (en) State snapshot generation and recovery method facing edge cloud
CN115766715A (en) High-availability super-fusion cluster monitoring method and system
CN109062599A (en) The management method and device of code update under micro services framework
CN108055159A (en) A kind of clustered node operation synchronous method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination