CN114297023A - Prometheus dynamic alarm rule management method based on CONFD - Google Patents
Prometheus dynamic alarm rule management method based on CONFD Download PDFInfo
- Publication number
- CN114297023A CN114297023A CN202111609742.0A CN202111609742A CN114297023A CN 114297023 A CN114297023 A CN 114297023A CN 202111609742 A CN202111609742 A CN 202111609742A CN 114297023 A CN114297023 A CN 114297023A
- Authority
- CN
- China
- Prior art keywords
- confd
- service
- prometheus
- rules
- dynamic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Stored Programmes (AREA)
Abstract
The invention discloses a method for managing ProMETHEUS dynamic alarm rules based on CONFD, which is implemented by acquiring dynamic alarm rules from a centralized unified configuration management center CONFD and automatically assembling the dynamic alarm rules into an alarm file format of a PROMETHEUS standard; and the source of the alarm rule is configured through a custom module written by JAVA, the custom module is used as a back-end system of the confd, the confd can automatically inquire and dynamically monitor the custom module in real time, after the configuration change is detected, the prometheus alarm rule file is re-rendered into the latest version on the custom module, and the alarm rule is immediately validated through a hot loading instruction. The invention realizes the dynamic configuration of the alarm rule on the premise of no code invasion, is safe and reliable, has strong expansibility and greatly improves the operation and maintenance efficiency.
Description
Technical Field
The invention relates to a method dynamic alarm rule management based on CONFD, belonging to the technical field of system monitoring and operation and maintenance.
Background
With the rapid development of internet services, the requirements of users on the system are higher and higher, and the system is well monitored to be a system security pilot, so that the reliability, the availability and the user experience of the system can be effectively improved. The monitoring system is the most important ring in the whole operation and maintenance link, even in the whole project and product life cycle. And prometheus is an outstanding new generation monitoring solution due to the unique advantages in the field of cloud-originated and containerized monitoring.
The most core function of real-time monitoring is to find a problem and alarm to notify a user at the first time, like most systems, alarm rule configuration of prometheus is completed through a configuration file, and manually modifying the configuration file is required every time an alarm rule is newly added or modified, and then the prometheus service is restarted to enable the new configuration to take effect. This is clearly inefficient and does not facilitate the developer to customize the rules, all configurations are tied to prometheus operation and maintenance personnel, unnecessarily increasing communication costs.
While the traditional configuration management system is code intrusive, the application access configuration management system needs to use the corresponding SDK to query and monitor the change of data. For some mature systems, it is difficult to access the SDK to implement dynamic configuration management, especially for prometheus, which is a huge project.
How to realize alarm rule configuration of prometheus quickly and efficiently is a technical problem which needs to be solved by the technical personnel in the field urgently.
Disclosure of Invention
The purpose is as follows: in order to overcome the defects in the prior art, the invention provides a method for managing the dynamic alarm rules of the ProMETHEUS based on the CONFD, the CONFD service is a lightweight configuration management tool, the dynamic configuration management of a third-party system can be realized by inquiring a back-end storage system, and the system configuration is changed on the premise of no code invasion.
The technical scheme is as follows: in order to solve the technical problems, the technical scheme adopted by the invention is as follows:
a prometheus dynamic alarm rule management method based on confd comprises the following steps:
prometheus rules are added and issued from custom JAVA modules.
Configuring confd service core parameters, wherein the core parameters comprise: template files, dynamic target rendering file paths, rule key sets, and prometheus hot load instructions.
And starting the confd service, and setting the self-defined JAVA module as the back-end storage of the confd service in the starting command line.
When a prometheus rule is newly added/modified/deleted in a front-end UI page corresponding to the custom JAVA module, the confd service senses the change of the prometheus rule through the back-end storage, renders the changed prometheus rule stored in a rule key set into a template file, and writes the rendered template file into a dynamic target rendering file path.
The prometheus hot load instruction executes the template file under the dynamic object render file path.
Preferably, the adding and issuing of prometheus rules from the custom JAVA module includes the following steps:
and adding prometheus rules from the page, and filling the unique identification key and description of the current rules.
And the specific content of the rules is input by a key value pair mode.
And checking the specific content of the entered rules.
The concrete content of the rules is stored in a JSON array with the unique identification key as the name in the background of the custom JAVA module.
Preferably, the unique identifier key corresponds to a rule key set in the confd service, and one unique identifier key may correspond to a plurality of rules, which are equivalent to packet names of the same type of rules.
As a preferred scheme, the configuring of the confd service core parameter includes the following steps:
and creating a template file under a template folder of the conf service installation path/etc/conf, wherein attribute values in the template file are represented by a placeholder "{ } }", and the attribute values are derived from prometheus rules of the custom JAVA module.
Creating a core configuration file, prometheus _ rules.tom of a conf service, configuring four core parameters of src, dest, keys and load _ cmd under a default/etc/conf/conf.d folder of a directory, wherein the src represents a template file, the dest represents a dynamic target rendering file path, the keys represents a rule key set, and the load _ cmd represents a prometheus hot loading instruction.
As a preferred scheme, the starting of the confd service, setting the custom JAVA module as the backend storage of the confd service in the start command line, includes the following steps:
in the parameter of the start command line of the confd service, the backup parameter is set as the name of the back-end micro-service corresponding to the self-defined JAVA module, the node parameter is set as the back-end service address of the self-defined JAVA module, and the watch parameter is set to enable the confd service to support dynamic monitoring.
Preferably, the confd service is further customized as follows:
the background type in the confd service is added with the java-client type.
When the confd service is started, the command line node parameter designates the TCP service port of the self-defined JAVA module, and TCP long connection is established. And adding a TCP long connection message processing module in the conf service, and calling the self logic of the conf service to dynamically render the configured template file after the conf service receives the rules change message.
And modifying parameters of a confd service starting command line, when the back parameter in the starting command line is set as java-client type service, switching the logic processing mode from the confd service self-contained mode to TCP long connection monitoring, and converting the TCP address and port of the connection into node parameters in the starting command line.
Has the advantages that: the invention provides a method for managing ProMETHEUS dynamic alarm rules based on CONFD, which is implemented by acquiring the dynamic alarm rules from a centralized unified configuration management center CONFD and automatically assembling the dynamic alarm rules into an alarm file format of a PROMETHEUS standard; and the source of the alarm rule is configured through a custom module written by JAVA, the custom module is used as a back-end system of the confd, the confd can automatically inquire and dynamically monitor the custom module in real time, after the configuration change is detected, the prometheus alarm rule file is re-rendered into the latest version on the custom module, and the alarm rule is immediately validated through a hot loading instruction. The method realizes dynamic configuration of the alarm rule on the premise of no code invasion, is safe and reliable, has strong expansibility, and greatly improves the operation and maintenance efficiency. Its advantages are as follows:
1. the method has no code invasion and reduced development complexity.
2. Dynamic configuration of alarm rules is realized, and a prometheus service does not need to be manually restarted.
3. Simple use, flexible configuration and convenient deployment.
4. And a WEB UI is provided for developers and operation and maintenance personnel to use, so that the method is more convenient and reduces the communication cost.
Drawings
FIG. 1 is a schematic structural diagram of the present invention.
Detailed Description
The present invention will be further described with reference to the following examples.
When the method is applied, the method relates to three software, which are respectively as follows:
1. prometheus service: and the monitoring service is used for collecting program indexes and processing alarm.
2. confd service: the management service is dynamically configured.
3. Self-defining a JAVA module: and the java micro service is used for the user to enter the custom alarm rule.
A prometheus dynamic alarm rule management method based on confd comprises the following steps:
prometheus rules are added and issued from custom JAVA modules.
Configuring confd service core parameters, wherein the core parameters comprise: template files, dynamic target rendering file paths, rule key sets, and prometheus hot load instructions.
And starting the confd service, and setting the self-defined JAVA module as the back-end storage of the confd service in the starting command line.
When a prometheus rule is newly added/modified/deleted in a front-end UI page corresponding to the custom JAVA module, the confd service senses the change of the prometheus rule through the back-end storage, renders the changed prometheus rule stored in a rule key set into a template file, and writes the rendered template file into a dynamic target rendering file path.
The prometheus hot load instruction executes the template file under the path of the dynamic target rendering file, and the changed prometheus rules take effect.
Preferably, the adding and issuing of prometheus rules from the custom JAVA module includes the following steps:
and adding prometheus rules from the page, and filling the unique identification key and description of the current rules.
And the specific content of the rules is input by a key value pair mode.
The specific content of the entered rules is checked to prevent invalid alarm rules from being generated.
The concrete content of the rules is stored in a JSON array with the unique identification key as the name in the background of the custom JAVA module.
The unique identification key is a mandatory item, corresponds to a rule key set in the confd service, and one unique identification key can correspond to a plurality of rules and is equivalent to the packet names of the same type of rules.
Preferably, the configuring of the confd service core parameter includes the following steps:
and creating a template file under a template folder of the conf service installation path/etc/conf, wherein attribute values in the template file are represented by a placeholder "{ } }", and the attribute values are derived from prometheus rules of the custom JAVA module.
Creating a core configuration file, namely, a proto _ rules.tom of a conf service, configuring four core parameters, namely src, dest, keys and load _ cmd, of the file under a default/etc/conf/conf.d folder, wherein src represents a template file, dest represents a dynamic target rendering file path, namely, an alarm rule storage path configured in the proto service, keys represents a rule key set, and load _ cmd represents a promethus hot loading instruction.
Preferably, the starting of the confd service, setting the custom JAVA module as a backend storage of the confd service in the start command line, includes the following steps:
in the parameter of the start command line of the confd service, the backup parameter is set as the name of the back-end micro-service corresponding to the self-defined JAVA module, the node parameter is set as the back-end service address of the self-defined JAVA module, and the watch parameter is set to enable the confd service to support dynamic monitoring.
Preferably, the conf service is capable of dynamically reading prometheus rules stored in the JAVA custom module, and the conf service source code needs to be customized as follows:
1. the background type in the confd service is added with the java-client type.
2. The monitoring logic for the custom JAVA module is written, and the custom JAVA module in this example has 2 uses as a core, namely, the custom JAVA module mentioned above is used for storing prometheus rules entered by the user, and the custom JAVA module serves as a TCP server to connect to the client (in this example, the client is the confd service) that needs to communicate with the client. When the custom J AVA module is started, a port is opened for external TCP connection, and when the confd service is started, the command line node parameter specifies the TCP service port, and then the long TCP connection is established between the confd service and the custom JAVA module. When prometheus rules stored in the custom JAVA module change, the custom JAVA module sends a change message to the connection, a TCP long connection message processing module needs to be added to the conf service, and after the conf service receives the rule change message, the self logic of the conf service is called to dynamically render the configured template file.
3. And modifying parameters of a confd service starting command line, and when the back parameter in the starting command line is set as java-client type service, automatically switching the logic processing mode from the confd service self-contained mode to TCP long connection monitoring, and converting the TCP address and port of the connection into node parameters in the starting command line. Thereby supporting the custom JAVA module as a back-end storage unit of the confd service.
And adding/modifying/deleting prometheus rules on a front-end UI page corresponding to the custom JAVA module, changing the custom JAVA module into a confd service storage background, and dynamically updating a local configuration file by combining a configuration template engine to keep the consistency with the data of a back end.
Example 1:
as shown in fig. 1, when the method for managing dynamic alarm rules of PROMETHEUS implemented based on CONFD is used, the method includes the following steps:
step 1: and starting a promteus service, and setting a path where an alarm rule file is located in a core configuration file prometheus.
Step 2: write the confd service back-end storage (backend) core logic code.
And step 3: creating a confd service configuration comprising a template file, a dynamic target rendering file path, a set of rule keys, and prometheus hot load instructions.
And 4, step 4: and customizing the confd service source code to enable the confd service source code to support a customized JAVA module as a back-end storage (backup).
And 5: and adding alarm rule configuration content and key on a front-end UI page corresponding to the custom JAVA module.
Step 6: and starting the confd service, and setting the configuration to the parameters of the starting related command line.
And 7: logging in prometheus UI, checking that the configured alarm rule is effective, and meanwhile, modifying the alarm rule configuration on a WEB UI page at any time, wherein the new rule can also be immediately effective.
Example 2:
the invention innovatively introduces the confd service as a configuration management tool of the project. Background storage (backhaul) supported by the condf service at present is etcd, zookpper, nacos, consl, etc., in this example, dynamic configuration can also be realized by combining the above background modes, but the expansibility of later functions is greatly reduced, so the invention adopts a mode of accessing the custom background storage (backhaul) to the confd service, and although there is code development amount, there is great space and flexibility for the expansion and customization of the custom background in the later period.
The above description is only of the preferred embodiments of the present invention, and it should be noted that: it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the invention and these are intended to be within the scope of the invention.
Claims (6)
1. A method for managing a dynamic alarm rule of PROMETHEUS based on CONFD is characterized in that: the method comprises the following steps:
adding and issuing prometheus rules from a custom JAVA module;
configuring confd service core parameters, wherein the core parameters comprise: template files, dynamic target rendering file paths, rule key sets and prometheus hot loading instructions;
starting the confd service, and setting the self-defined JAVA module as the back-end storage of the confd service in a starting command line;
when a prometheus rule is newly added/modified/deleted in a front-end UI page corresponding to a custom JAVA module, a confd service senses the change of the prometheus rule through a back-end storage, renders the changed prometheus rule stored in a rule key set in a grouped manner to a template file, and writes the rendered template file into a dynamic target rendering file path;
the prometheus hot load instruction executes the template file under the dynamic object render file path.
2. The method for managing PROMETHEUS dynamic alarm rules based on CONFD implementation as claimed in claim 1, wherein: the method for adding and issuing prometheus rules from the custom JAVA module comprises the following steps:
adding prometheus rules from a page, and filling in a unique identifier key and description of the current rules;
the specific content of rules is input in a key value pair mode;
checking the specific content of the recorded rules;
the concrete content of the rules is stored in a JSON array with the unique identification key as the name in the background of the custom JAVA module.
3. A method for managing PROMETHEUS dynamic alarm rules based on CONFD implementation as claimed in claim 2, wherein: the unique identifier key corresponds to a set of rule keys in the confd service, and one unique identifier key may correspond to a plurality of rules, which are equivalent to packet names of the same type of rules.
4. The method for managing PROMETHEUS dynamic alarm rules based on CONFD implementation as claimed in claim 1, wherein: the method for configuring the confd service core parameters comprises the following steps:
creating a template file under a template folder of a conf service installation path/etc/conf, wherein attribute values in the template file are represented by placeholders "{ }", and the attribute values are derived from prometheus rules of a custom JAVA module;
creating a core configuration file, prometheus _ rules.tom of a conf service, configuring four core parameters of src, dest, keys and load _ cmd under a default/etc/conf/conf.d folder of a directory, wherein the src represents a template file, the dest represents a dynamic target rendering file path, the keys represents a rule key set, and the load _ cmd represents a prometheus hot loading instruction.
5. The method for managing PROMETHEUS dynamic alarm rules based on CONFD implementation as claimed in claim 1, wherein: the method for starting the confd service sets the custom JAVA module as the back-end storage of the confd service in the starting command line comprises the following steps:
in the parameter of the start command line of the confd service, the backup parameter is set as the name of the back-end micro-service corresponding to the self-defined JAVA module, the node parameter is set as the back-end service address of the self-defined JAVA module, and the watch parameter is set to enable the confd service to support dynamic monitoring.
6. The method for managing PROMETHEUS dynamic alarm rules based on CONFD implementation as claimed in claim 1, wherein: the confd service is also customized as follows:
adding a java-client type into the background type in the confd service;
when the confd service is started, the command line node parameter designates a TCP service port of a self-defined JAVA module, and TCP long connection is established; adding a TCP long connection message processing module in the conf service, and calling the self logic of the conf service to dynamically render the configured template file after the conf service receives the rules change message;
and modifying parameters of a confd service starting command line, when the back parameter in the starting command line is set as java-client type service, switching the logic processing mode from the confd service self-contained mode to TCP long connection monitoring, and converting the TCP address and port of the connection into node parameters in the starting command line.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111609742.0A CN114297023A (en) | 2021-12-27 | 2021-12-27 | Prometheus dynamic alarm rule management method based on CONFD |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111609742.0A CN114297023A (en) | 2021-12-27 | 2021-12-27 | Prometheus dynamic alarm rule management method based on CONFD |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114297023A true CN114297023A (en) | 2022-04-08 |
Family
ID=80970184
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111609742.0A Pending CN114297023A (en) | 2021-12-27 | 2021-12-27 | Prometheus dynamic alarm rule management method based on CONFD |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114297023A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115134270A (en) * | 2022-06-28 | 2022-09-30 | 北京奇艺世纪科技有限公司 | Code monitoring method, monitoring system, electronic device and storage medium |
-
2021
- 2021-12-27 CN CN202111609742.0A patent/CN114297023A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115134270A (en) * | 2022-06-28 | 2022-09-30 | 北京奇艺世纪科技有限公司 | Code monitoring method, monitoring system, electronic device and storage medium |
| CN115134270B (en) * | 2022-06-28 | 2023-09-08 | 北京奇艺世纪科技有限公司 | Code monitoring method, monitoring system, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101621405B (en) | Distributed type management monitoring system, monitoring method and establishing method thereof | |
| CN111290753B (en) | Method, device, equipment and storage medium for building front-end development environment | |
| CN1910854B (en) | Automatic update system and method for using a meta mib | |
| CN111556158B (en) | Access method, access device, access equipment and storage medium of Internet of things platform | |
| US10671463B2 (en) | Reconfigurable embedded rules engine for internet of things (IOT) devices | |
| US8589381B2 (en) | Resource management program, resource management process, and resource management apparatus | |
| CN111064626B (en) | Configuration updating method, device, server and readable storage medium | |
| CN112804362B (en) | Dispersed data micro-service automation operation and maintenance system | |
| CN114297023A (en) | Prometheus dynamic alarm rule management method based on CONFD | |
| CN103944814A (en) | Data exchange method and system and gateway server | |
| CN111966465A (en) | Method, system, equipment and medium for modifying configuration parameters of host machine in real time | |
| CN116340413A (en) | Internet of things edge data acquisition and storage method | |
| CN114221690A (en) | Semantic model construction method for cloud edge cooperative system | |
| CN110457132B (en) | Method and device for creating functional object and terminal equipment | |
| CN103475690A (en) | Memcached instance configuration method and Memcached instance configuration system | |
| CN114020572A (en) | Distributed alarm method and device | |
| CN105324728A (en) | An industrial control system for monitoring and controlling an automation plant | |
| CN109783253A (en) | Distributed message data/address bus based on kafka | |
| CN113835904A (en) | Remote procedure call control method, device, equipment and storage medium | |
| CN113486099A (en) | Memory computing service framework and implementation system | |
| CN116980475B (en) | Data pushing system based on binlog and double annular buffer areas | |
| CN115080152B (en) | Broker configuration method and device of message middleware, computer equipment and medium | |
| JP2006285666A (en) | Information providing system, information generation method, server, and information providing program | |
| KR20010027192A (en) | Method for processing message from communication system in pilot terminal | |
| CN114296809B (en) | Object model construction method based on operating system and system call interface thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |