KR20240077270A - Integrated monitoring system and method for multi-cluster - Google Patents

Abstract

An integrated monitoring system and method for multi-cluster are disclosed. An integrated monitoring system for multi-clusters according to one aspect of the present invention includes a plurality of clusters in which an agent that generates monitoring-related information and pushes it to the outside is installed; Temporary storage where monitoring metrics sent by clusters by push method are temporarily stored; and a monitoring server that distributes agents in a cluster, collects monitoring metrics, and manages monitoring information in an integrated manner, wherein the monitoring server includes: a metarepository for storing metadata necessary for checking the status of each deployed agent; a collector manager that uses metadata to check whether each agent is operating normally and creates each collector corresponding to a normal agent; A monitoring information storage for each collector to collect monitoring metrics stored in temporary storage and store them as monitoring information; and a metadata manager that modifies metadata information about abnormal agents and stores them in a metadata repository when there is a collector for which monitoring metrics are not collected.

Description

Integrated monitoring system and method for multi-cluster}

The present invention relates to an integrated monitoring system and method for multi-cluster.

As a limitation of the existing Kubernetes multi-cluster environment monitoring system, when building a multi-cluster, it is necessary to build a monitoring system for each cluster and link between monitoring systems. Additionally, public cloud service providers need to build separate software to link distributed monitoring systems. For example, the open source Prometheus is installed in each cluster and Thanos is installed on the central monitoring server. Development and installation of separate integration software to integrate monitoring information into each cluster. is needed. In other words, users of the public cloud service provider's monitoring system need to build an additional monitoring system for the Kubernetes service of another CSP (Cloud Service Provider).

Republic of Korea Patent No. 10-1987664 Method for monitoring multiple clusters and applications on a cloud platform

Therefore, the present invention was devised to solve the above-mentioned problems, and is intended to provide a flexible integrated monitoring system and method that is not dependent on CSP in a multi-cluster environment and does not require separate software linkage based on a centralized monitoring system. will be.

In addition, the present invention is to provide an integrated monitoring system and method for multi-clusters for integrated management of stable monitoring information according to the situation of the multi-clusters on a push basis.

Other objects of the present invention will become clearer through the preferred embodiments described below.

According to one aspect of the present invention, a plurality of clusters in which an agent that generates monitoring-related information and pushes it to the outside is installed; Temporary storage where monitoring metrics sent by clusters by push method are temporarily stored; and a monitoring server that distributes agents to the cluster, collects the monitoring metrics, and manages monitoring information in an integrated manner, wherein the monitoring server includes: a metarepository for storing metadata necessary for checking the status of each deployed agent; a collector manager that checks whether each agent is operating normally using the metadata and creates each collector corresponding to a normal agent; a monitoring data storage for each collector to collect and store monitoring metrics stored in the temporary storage as monitoring information; and a metadata manager that modifies metadata information about an abnormal agent and stores it in the metadata repository when there is a collector for which monitoring metrics are not collected.

Here, each agent that receives information about the agents of each cluster from the monitoring server shares status information including network environment and available process information with each other, and sets a push schedule for monitoring-related information based on the status information. You can.

Additionally, among each agent, the agent set as the master provides information about the push schedule to the monitoring server, and the monitoring server can determine the collector to be created and the creation order according to the push schedule.

Additionally, the master may be dynamically determined based on the status information.

Additionally, each agent can include the status information acquired from other agents as its own monitoring-related information and push it to the monitoring server.

Additionally, the metadata manager can refer to the status information to change metadata information for agents for which monitoring metrics have not been collected.

According to another aspect of the present invention, in an integrated monitoring method in a monitoring server that distributes agents that transmit monitoring-related information in a push method to a plurality of clusters, metadata necessary for checking the status of each deployed agent is used. Checking whether each agent is operating normally and creating each collector corresponding to a normal agent; Collecting, by the collector, monitoring metrics stored in a temporary storage where monitoring metrics transmitted by clusters in a push manner are temporarily stored and storing them as monitoring information; and a step of modifying and storing metadata information about an abnormal agent when there is a collector in which monitoring metrics are not collected. An integrated monitoring method for a multi-cluster and a computer program executing the method are provided.

Here, information can be provided to the agents of each cluster so that the agents installed in each cluster can set a push schedule for monitoring-related information by sharing status information including each network environment and available process information.

In addition, information about the push schedule can be obtained from the agent set as the master among each agent, and the collector to be created and the generation order can be determined according to the push schedule.

Additionally, as each agent pushes the status information acquired from other agents as its own monitoring-related information, the status information can be referred to when changing metadata information for agents for which monitoring metrics have not been collected.

Other aspects, features and advantages in addition to those described above will become apparent from the following drawings, claims and detailed description of the invention.

According to the present invention, it is possible to provide a flexible integrated monitoring system and method that is not dependent on CSP in a multi-cluster environment and does not require separate software linkage based on a centralized monitoring system.

In addition, according to the present invention, more stable integrated management of monitoring information is possible on a push basis due to scheduling tailored to each situation according to interconnection between multi-clusters.

1 is a diagram illustrating software installation for conventional multi-cluster integrated monitoring.
Figure 2 is an example diagram showing an integrated monitoring method for multi-cluster using multiple CSPs according to an embodiment of the present invention.
Figure 3 is a block diagram showing the configuration of an integrated monitoring system for multi-cluster according to an embodiment of the present invention.
4 and 5 are flowcharts and detailed illustrations showing a schematic monitoring process in an integrated monitoring system according to each embodiment of the present invention.
Figure 6 is a flowchart showing a process for pushing monitoring information by setting a push schedule performed in a cluster according to an embodiment of the present invention.
Figure 7 is a flowchart showing a multi-cluster monitoring process using a push schedule and status information performed in a monitoring server according to an embodiment of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, terms such as first threshold value and second threshold value, which will be described later, may be pre-designated as threshold values that are substantially different or partially the same, but may cause confusion when expressed with the same word threshold. Since there is room, for convenience of classification, terms such as first and second will be used together.

The terms used in this specification are merely used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

In addition, the components of the embodiments described with reference to each drawing are not limited to the corresponding embodiments, and may be implemented to be included in other embodiments within the scope of maintaining the technical spirit of the present invention, and may also be included in separate embodiments. Even if the description is omitted, it is natural that a plurality of embodiments may be re-implemented as a single integrated embodiment.

In addition, when describing with reference to the accompanying drawings, identical or related reference numbers will be assigned to identical or related elements regardless of the drawing symbols, and overlapping descriptions thereof will be omitted. In describing the present invention, if it is determined that a detailed description of related known technologies may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

FIG. 1 is a diagram illustrating the installation of software for conventional multi-cluster integrated monitoring, and FIG. 2 is an exemplary diagram illustrating an integrated monitoring method for multi-clusters using multiple CSPs according to an embodiment of the present invention.

First, referring to Figure 1, in order to build a conventional multi-cluster monitoring system using open source, Prometheus, an agent, is installed in the cluster and Thanos is installed in the monitoring server to collect monitoring. , Additionally, separate software must be installed on the monitoring server to integrate the collected monitoring information.

In contrast, referring to FIG. 2 according to an embodiment of the present invention, a flexible integrated monitoring system without separate software linkage is provided based on a centralized monitoring system in a multi-cluster environment.

Compared to Kubernetes monitoring based on the PULL monitoring mechanism, it supports large-scale monitoring based on the PUSH monitoring mechanism suitable for a multi-cluster environment. And, unlike monitoring solutions from public cloud service providers, it supports integrated monitoring functions for all Kubernetes clusters that are not dependent on CSP.

Hereinafter, the integrated monitoring system for multi-cluster of the present invention and its operating method will be described in more detail.

Figure 3 is a block diagram showing the configuration of an integrated monitoring system for multi-cluster according to an embodiment of the present invention.

Referring to FIG. 3, the entire system includes a plurality of clusters (10-1, ..., 10-n, hereinafter collectively referred to as 10), a temporary storage 30, and a monitoring server 50. The monitoring server 50 includes a meta storage 51, a meta data manager 52, a collector manager 53, and a monitoring data storage 54.

An agent provided by the monitoring server 50 is installed in each of the plurality of clusters 10. The agent generates monitoring information (e.g., CPU usage, available storage capacity, network environment, memory usage, execution process information, etc., which will be obvious to those skilled in the art, so detailed descriptions are omitted) and transmits it to temporary storage through the communication network in a push manner. do.

The temporary storage 30 may be, for example, a Kafka Broker. The cluster agent sends monitoring information to Kapuka Broker in push form. Kafka Broker refers to a system commonly referred to as 'Kafka' because the Producer and Consumer are composed of separate applications, while the Broker is Kafka itself. Therefore, in 'constructing Kafka' or 'delivering messages through Kafka', Kafka means a broker. The Kafka broker will be self-evident to those skilled in the art, so a more detailed explanation will be omitted.

The monitoring server 50 distributes agents to the cluster 10, collects monitoring metrics stored in the temporary storage 30, and stores and manages monitoring information in an integrated manner.

Metadata necessary for checking the status of each deployed agent is stored in the metadata location of the monitoring server 50. You can check the status of each agent (active or inactive, normal or abnormal operation, etc.) by referring to the metadata.

The metadata manager 52 manages the metadata of these agents. For example, if there is a collector for which monitoring metrics are not collected, metadata information about abnormal agents is modified and stored in the metadata repository.

The collector manager 53 uses metadata to check whether each agent is operating normally and creates each collector corresponding to a normal agent. The created collectors are pushed from the agent of the corresponding cluster 10 and collect monitoring metrics stored in the temporary storage 30.

The monitoring data storage 54 stores monitoring information according to the monitoring metrics collected by each collector from the temporary storage.

Figures 4 and 5 are flowcharts and detailed illustrations showing a schematic monitoring process in the integrated monitoring system according to each embodiment of the present invention.

Referring to FIG. 4, the monitoring server 50 distributes the agent to each cluster 10 to be managed (S410).

Then, the cluster 10 runs the installed agent to periodically generate monitoring information (S420), and the monitoring server 50 creates a collector according to each deployed agent (S430). At this time, a collector can be initially created for all clusters 10, and later metadata for each cluster 10 is created and managed according to the monitoring information collected from the clusters 10, and each agent according to the metadata Depending on the status, you can decide whether and when to create a collector.

When the monitoring information generated by the cluster 10 is transmitted by the push method (S440), the collector created in the monitoring server 50 collects the monitoring information and stores it in the monitoring data storage 54 (S450).

If there is a collector for which monitoring metrics are not collected, the corresponding agent is treated as an abnormal agent and related metadata information is modified (S460). And, for example, when a certain time or a certain number of monitoring collections have elapsed, a collector is created for an abnormal agent to attempt to collect monitoring information, and if collected, the metadata for the agent can be modified to a normal state.

Referring to Figure 5, which shows a detailed example of this process, the monitoring server 50 receives installation data such as CSP information, agent IP, and connection information required for agent deployment from the user, and distributes the agent accordingly. do.

Afterwards, the metadata required to check the agent status is stored, and the agent information (number of clusters) in the metadata storage is delivered to the collector manager 53.

The collector manager 53 calculates collector creation based on agent information (collector deletion for abnormal agents). Afterwards, the collector manager 53 creates a collector and delivers a Topic (agent information) to subscribe to.

The created collector collects monitoring metrics stored in the Kafka Broker based on the received Topic. If data collection is abnormal more than 5 times, the agent status of the metadata is changed.

Then, normally collected monitoring metrics are stored in the monitoring data storage, and metadata information about abnormal agents is modified and stored in the metadata storage.

According to this embodiment, monitoring information is collected in a push method using agents distributed to each cluster 10 in an integrated manner by the monitoring server 50, and stable monitoring is achieved by managing the agent status of each cluster 10. It becomes possible.

In addition, it becomes possible to build a Kubernetes cluster for each CSP (Cloud Service Provider), and build multi-cluster interconnection and management software.

Below, a more stable method of collecting monitoring information through interworking between agents of the cluster 10 will be described.

Figure 6 is a flowchart showing the process of pushing monitoring information by setting a push schedule performed in a cluster according to an embodiment of the present invention, and Figure 7 is a flowchart showing the process performed by the monitoring server 50 according to an embodiment of the present invention. This is a flowchart showing the multi-cluster monitoring process using push schedule and status information.

First, referring to FIG. 6, information (e.g., IP address, MAC address, communication method, network environment, etc.) on each agent of all clusters 10 is acquired from the monitoring server 50 (S610) ).

Then, the cluster 10 communicates with other clusters (i.e. agents) and shares status information including network environment and available process information (S620). According to one example, some of the monitoring information provided to the monitoring server 50 may be included as status information. Then, the shared status information can be provided to the monitoring server 50 by another cluster. If any cluster 10 is in an abnormal state, the monitoring information cannot be provided to the monitoring server 50. The monitoring server 50 can check some of the monitoring information based on the status information transmitted to the cluster.

Based on the shared status information of each cluster 10, a push schedule to push monitoring information is set. For example, the push cycle, turn number (or transmission time), etc. of each cluster 10 can be set as a push schedule. According to one example, one agent is set as the master agent, and the master agent can determine the push schedule of each agent based on the status information of each cluster 10. The master agent can be dynamically determined by referring to each status information. For example, the one with the best network environment and the smallest number of running processes can be selected. Of course, this is just an example and can be determined according to various rules. One may be dynamically set as the master agent based on status information.

The set push schedule is also shared with all agents, and each agent performs the push process of monitoring information according to the push schedule (S640).

According to this embodiment, by dynamically setting a push schedule according to the clusters 10's own status, more stable pushing of monitoring information can be performed.

In addition, information about this push schedule can be provided to the monitoring server 50. According to one example, the push schedule information is sent to the monitoring server 50 in a push form by the master agent separately from the monitoring information or together with the monitoring information. Can be transmitted.

Referring to FIG. 7, when the monitoring server 50 receives the push schedule (S710), it analyzes the push schedule and determines the collector to be created and the creation order (S720).

As a result, the monitoring server 50 can more accurately create the necessary collector by referring to the push schedule set by communication between agents as well as the meta information of the agents, and can also provide faster and more accurate monitoring information by the generation order. Collection becomes possible.

A collector is created with the determined contents to collect monitoring information (S730).

In addition, the status information of each agent collected together as monitoring information is used to refer to changes in metadata information (S740). For example, if the communication environment of a certain cluster is poor as a result of checking the status information, the meta information of the agent for that cluster is changed to an abnormal state.

Additionally, as described above, for clusters for which monitoring information has not been collected, status information collected from agents of other clusters can be checked and some of it can be used as monitoring information.

A computer program stored in a computer-readable medium may be provided to perform the integrated monitoring method for multi-clusters according to the present invention described above.

Additionally, the integrated monitoring method for multi-cluster described above can be implemented as computer-readable code on a computer-readable recording medium. Computer-readable recording media include all types of recording media storing data that can be deciphered by a computer system. For example, there may be Read Only Memory (ROM), Random Access Memory (RAM), magnetic tape, magnetic disk, flash memory, optical data storage device, etc. Additionally, the computer-readable recording medium can be distributed to computer systems connected through a computer communication network, and stored and executed as code that can be read in a distributed manner.

In addition, although the present invention has been described above with reference to preferred embodiments, those skilled in the art will understand the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that it can be modified and changed in various ways.

10: Cluster
30: Temporary storage
50: Monitoring server

Claims (11)

A plurality of clusters where agents that generate monitoring-related information and push it to the outside are installed;
Temporary storage where monitoring metrics sent by clusters by push method are temporarily stored; and
A monitoring server that distributes agents to the cluster, collects the monitoring metrics, and manages monitoring information in an integrated manner,
The monitoring server is,
Metarepository for storing metadata required for checking the status of each deployed agent;
a collector manager that checks whether each agent is operating normally using the metadata and creates each collector corresponding to a normal agent;
a monitoring data storage for each collector to collect and store monitoring metrics stored in the temporary storage as monitoring information; and
An integrated monitoring system for a multi-cluster, including a metadata manager that modifies metadata information about abnormal agents and stores it in the metadata repository when there is a collector for which monitoring metrics are not collected.
In claim 1,
Each agent that receives information about the agents of each cluster from the monitoring server shares status information including network environment and available process information with each other, and sets a push schedule for monitoring-related information based on the status information. Integrated monitoring system for multi-cluster.
In claim 2,
Among each agent, the agent set as the master provides information about the push schedule to the monitoring server,
An integrated monitoring system for multi-cluster where the monitoring server determines the collectors to be created and the creation order according to the push schedule.
In claim 3,
An integrated monitoring system for multi-cluster in which the master is dynamically determined based on the status information.
In claim 2,
An integrated monitoring system for a multi-cluster in which each agent includes the status information acquired from other agents as its own monitoring-related information and pushes it to the monitoring server.
In claim 5,
The metadata manager refers to the status information when changing metadata information for agents for which monitoring metrics have not been collected. An integrated monitoring system for multi-clusters.
In an integrated monitoring method in a monitoring server that distributes an agent that transmits monitoring-related information in a push method to a plurality of clusters,
Confirming whether each agent is operating normally using metadata required to check the status of each deployed agent and creating each collector corresponding to a normal agent;
Collecting, by the collector, monitoring metrics stored in a temporary storage where monitoring metrics transmitted by clusters in a push manner are temporarily stored and storing them as monitoring information; and
An integrated monitoring method for multi-cluster, including the step of modifying and storing metadata information about abnormal agents when there is a collector for which monitoring metrics are not collected.
In claim 7,
For multi-clusters, the agents installed in each cluster share status information, including each network environment and available process information, to set a push schedule for monitoring-related information and provide each other's information to the agents of each cluster. Integrated monitoring method.
In claim 8,
An integrated monitoring method for a multi-cluster that acquires information about the push schedule from the agent set as the master among each agent, and determines the collector to be created and the creation order according to the push schedule.
In claim 8,
As each agent pushes the status information acquired from other agents as its own monitoring-related information,
An integrated monitoring method for multi-cluster that refers to the status information when changing metadata information for agents for which monitoring metrics have not been collected.
A computer program stored on a computer-readable medium for performing an integrated monitoring method for a multi-cluster, wherein the computer program causes a computer to perform the following steps, the steps comprising:
Confirming whether each agent is operating normally using metadata required to check the status of each deployed agent and creating each collector corresponding to a normal agent;
Collecting, by the collector, monitoring metrics stored in a temporary storage where monitoring metrics transmitted by clusters in a push manner are temporarily stored and storing them as monitoring information; and
A computer program stored in a computer-readable medium, comprising the step of modifying and storing metadata information about the abnormal agent when there is a collector for which monitoring metrics are not collected.