CN115757063A - Integrated panoramic intelligent monitoring system - Google Patents

Abstract

The application discloses an integrated panoramic intelligent monitoring system, which comprises a snmp-ag module, an apisix module, a gateway module and an index module, wherein the snmp-ag module is connected with the gateway module, the gateway module is connected with the sky module, the sky module is connected with a transfer module, the transfer module is connected with a tsdb module, and the tsdb module is connected with the index module; the apisix module is connected with the gateway module, the gateway module is connected with the agent module, and the agent module is connected with the transfer module. The application has the advantages that the method has powerful Gateway: the push and pull modes are supported, and data can be captured and transmitted according to a black and white list; horizontal expansion of each component can be realized; the alarm setting is humanized: visual alarm configuration and management, support rich alarm rules, provide the configuration capabilities of shielding rules and subscription rules, support various reach channels for alarm, support alarm self-healing and alarm event management; and (3) monitoring trend prediction: and calculating a presenting trend prediction graph through the historical indexes.

Description

Integrated panoramic intelligent monitoring system

Technical Field

The application relates to the field of monitoring systems, in particular to an integrated panoramic intelligent monitoring system.

Background

With the rapid development of the internet, the monitored data is more and more, no matter the scale of the internet infrastructure or the scale of the software service provided for the user is larger and larger. With the technical innovation, higher and higher requirements are put on the operation monitoring of software, the monitoring is carried out on a server, an operating system, middleware, service, a link and K8s, the monitoring range is wider and wider, the monitoring system belongs to a bottom-layer service, the reliability and the stability of the monitoring system are very important, and components all support the distributed extensible capability. The value of the monitoring data is not only an alarm function, but also more important provides data query capability. The monitoring data can provide panoramic visual angles for development, operation and maintenance. The running condition of the service can be rapidly known, and the monitoring system can exert a greater value.

In the prior art, a subassembly in the middle lacks of distributed expansion capability, a single point fault affects partial data acquisition and alarm, a query subassembly depends on grafana and lacks of deep integration capability, distributed expansion capability and intelligent query function of monitoring graphs, and service health observable capability is not easy to realize. Therefore, an integrated panoramic intelligent monitoring system is provided for the above problems.

Disclosure of Invention

The embodiment provides an integrated panoramic intelligent monitoring system for solving the problems that in the prior art, communication with customers is not easy to carry out personalized service and accurate marketing are not easy to carry out.

According to one aspect of the application, an integrated panoramic intelligent monitoring system is provided, and comprises a snmp-ag module, an apisix module, a gateway module and an index module, wherein the snmp-ag module is connected with the gateway module, the gateway module is connected with the sky module, the sky module is connected with the transfer module, the transfer module is connected with the tsdb module, and the tsdb module is connected with the index module; the apisix module is connected with the gateway module, the gateway module is connected with the agent module, and the agent module is connected with the transfer module.

Furthermore, the gateway module is connected with the station monitoring module, and the gateway module is connected with the starsea module.

Further, the gateway module is connected with the first flink module, and the gateway module is also connected with the mysql module.

Further, the gateway module is connected with the es module, and the gateway module is connected with the second flink module.

Further, the sky module is connected with an alert-manager module, and the alert-manager module is connected with a judge module.

Further, the judge module is connected with the transfer module.

Further, the transfer module is linked to a clickhouse module.

Further, the transfer module is connected with a first m-module, and the first m-module is connected with a second m-module.

Further, the gateway module is connected with the transfer module.

Further, the index module is connected with the judge module.

Through the embodiment of the application, the problem that partial components lack distributed expansion capability, a single-point fault affects partial data acquisition and alarm is solved, the query component depends on the grafana and lacks deep integration capability, the distributed expansion capability is lacked, the intelligent query function of monitoring graphs is lacked, the observable capability of service health is difficult to realize, the monitoring of integrated panoramic intelligence is utilized, the system has the capabilities of acquisition, transmission, alarm and storage, various components all support the distributed expansion capability, the capability of querying data is classified and reconstructed, the scene of most users for monitoring data query is met, and the monitored panoramic intelligence is realized.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.

Fig. 1 is a schematic overall structure diagram of an embodiment of the present application.

In the figure: 1. a snmp-ag module; 2. a station monitoring module; 3. a star-sea module; 4. a first flink module; 5. a mysql module; 6. an apisix module; 7. an es module; 8. a second flink module; 9. a sky module; 10. a gateway module; 11. an agent module; 12. a transfer module; 13. an alert-manager module; 14. a judge module; 15. a tsdb module; 16. a first m3 module; 17. a clickhouse module; 18. an index module; 19. a second m3 module.

Detailed Description

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be used. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

In the present application, the terms "upper", "lower", "left", "right", "front", "rear", "top", "bottom", "inner", "outer", "middle", "vertical", "horizontal", "lateral", "longitudinal", and the like indicate an orientation or positional relationship based on the orientation or positional relationship shown in the drawings. These terms are used primarily to better describe the present application and its embodiments, and are not used to limit the indicated devices, elements or components to a particular orientation or to be constructed and operated in a particular orientation.

Moreover, some of the above terms may be used to indicate other meanings besides the orientation or positional relationship, for example, the term "on" may also be used to indicate some kind of attachment or connection relationship in some cases. The specific meaning of these terms in this application will be understood by those of ordinary skill in the art as appropriate.

Furthermore, the terms "mounted," "disposed," "provided," "connected," and "sleeved" are to be construed broadly. For example, it may be a fixed connection, a removable connection, or a unitary construction; can be a mechanical connection, or an electrical connection; may be directly connected, or indirectly connected through intervening media, or may be in internal communication between two devices, elements or components. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art as the case may be.

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

Please refer to fig. 1, an integrated panoramic intelligent monitoring system includes a snmp-ag module 1, an apisix module 6, a gateway module 10 and an index module 18, where the snmp-ag module 1 is connected to the gateway module 10, the gateway module 10 is connected to the sky module 9, the sky module 9 is connected to the transfer module 12, the transfer module 12 is connected to the tsdb module 15, and the tsdb module 15 is connected to the index module 18; the apisix module 6 is connected with a gateway module 10, the gateway module 10 is connected with an agent module 11, and the agent module 11 is connected with a transfer module 12.

The gateway module 10 is connected with the station monitoring module 2, and the gateway module 10 is connected with the starry-sea module 3.

The gateway module 10 is connected to the first flink module 4, and the gateway module 10 is also connected to the mysql module 5.

The gateway module 10 is connected with the es module 7, and the gateway module 10 is connected with the second flink module 8.

The sky module 9 is connected with an alert-manager module 13, and the alert-manager module 13 is connected with a judge module 14.

The judge module 14 is connected to the transfer module 12.

The transfer module 12 is linked to a clickhouse module 17.

The transfer module 12 is connected to a first m3 module 16, and the first m3 module 16 is connected to a second m3 module 19.

The gateway module 10 is connected to a transfer module 12.

The index module 18 is connected with the judge module 14.

The application provides a complete set of functions of distributed monitoring components, including acquisition, transmission, alarm and storage capabilities, and various components support distributed extensible capabilities. And the capability of inquiring data is classified and reconstructed, so that the condition that most users inquire the monitoring data is met.

The using method comprises the following steps: firstly, the gateway module 10 supports a data pushing mode, uses a domain name to perform load balancing, and an interface supports a Prometheus remote writing mode. The method supports a data pulling mode, a plurality of nodes dynamically expand and pull data of a designated url based on etcd distribution, and the data format supports the data types of a prometheus type and an OpenFalcon format. The Gateway pull data can configure indexes to discard black and white lists, and retain or discard specified indexes. The pull mode supports a common authentication mode, supports the pull of address data with authentication, and supports the visual display of the pull data volume of indexes;

the transfer module 12 forwards data, and delivers the data to the judge module 14, the tsdb module 15, the first m3 module 16 and other components, and the components are stateless components and mainly perform domain name load balancing. Regardless of the type of backend storage, data can be pushed to the specified backend storage as long as a pushable interface is implemented. The data push supports a topic mode, the back end can be configured to only accept specified data, and the data throughput capacity is improved;

the judge module 14 supports a common OpenFalcon determination method and also supports a Prometheus-based query statement to perform judge. Mainly depends on the dynamic registration of the etcd to carry out distributed dynamic capacity expansion. The Judge component only processes the Judge on the data, and pushes the event generated after the Judge to an alert-manager module 13 at the back end;

the alert-manager module 13 mainly realizes the pushing of the alarm, and the pushing mode is dynamically expandable. Alert-manager implements complex alerting approaches such as: advanced features such as alarm upgrading/alarm shielding/alarm aggregation/repeated transmission;

the tsdb module 15 stores index data in a short time, and is mainly used for query supplement of judge and query of index data;

the first m3 module 16 and the second m3 module 19 use the cyber time series storage of the Uber open source, 3 copies, the storage period is 365 days, and the persistent storage is performed.

Monitoring a graph: the monitoring panel is utilized to support not only Grafana-like configuration query statements, but also one-stop access to other infrastructures to automatically generate query monitoring graphs. Provides easy configuration for entry level personnel and also supports depth configuration of professionals; the customized middleware query component is mainly suitable for data display of monitoring data in each middleware, unifies/intelligentizes the middleware data, and supports data query in a manner of same data ratio and ring ratio; the meteorological graph can freely configure node topology, the nodes and lines can configure corresponding service configuration, corresponding highlight warning can be realized when corresponding strategy warning is abnormal or triggered, and service architecture and the health state of each node can be clearly seen in large-screen display, service topology display and other scenes.

The application has the advantages that: has a strong Gateway: the push and pull modes are supported, and data can be captured and transmitted according to a black and white list; the horizontal extension of each component can be realized: billions of times of data collection, alarming, storage and drawing are carried out in the production environment every minute, and the sustainable level expansion is realized; the alarm setting is humanized: visual alarm configuration and management, supporting rich alarm rules, providing shielding rules and subscription rule configuration capabilities, supporting multiple access channels for alarm, and supporting alarm self-healing and alarm event management; and (3) monitoring trend prediction: and calculating a presented trend prediction graph through the historical indexes.

The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (10)

1. The utility model provides an intelligent monitored control system of integration panorama which characterized in that: the sensor comprises a snmp-ag module (1), an apisix module (6), a gateway module (10) and an index module (18), wherein the snmp-ag module (1) is connected with the gateway module (10), the gateway module (10) is connected with a sky module (9), the sky module (9) is connected with a transfer module (12), the transfer module (12) is connected with a tsdb module (15), and the tsdb module (15) is connected with the index module (18); the apisix module (6) is connected with a gateway module (10), the gateway module (10) is connected with an agent module (11), and the agent module (11) is connected with a transfer module (12).

2. The integrated panoramic intelligent monitoring system of claim 1, characterized in that: the gateway module (10) is connected with the station monitoring module (2), and the gateway module (10) is connected with the starfish module (3).

3. The integrated panoramic intelligent monitoring system according to claim 1, characterized in that: the gateway module (10) is connected with the first flink module (4), and the gateway module (10) is also connected with the mysql module (5).

4. The integrated panoramic intelligent monitoring system according to claim 1, characterized in that: the gateway module (10) is connected with the es module (7), and the gateway module (10) is connected with the second flink module (8).

5. The integrated panoramic intelligent monitoring system according to claim 1, characterized in that: the sky module (9) is connected with an alert-manager module (13), and the alert-manager module (13) is connected with a judge module (14).

6. The integrated panoramic intelligent monitoring system according to claim 5, wherein: the judge module (14) is connected with the transfer module (12).

7. The integrated panoramic intelligent monitoring system of claim 1, characterized in that: the transfer module (12) is connected to a clickhouse module (17).

8. The integrated panoramic intelligent monitoring system according to claim 1, characterized in that: the transfer module (12) is connected to a first m3 module (16), and the first m3 module (16) is connected to a second m3 module (19).

9. The integrated panoramic intelligent monitoring system according to claim 1, characterized in that: the gateway module (10) is connected with the transfer module (12).

10. The integrated panoramic intelligent monitoring system according to claim 1, characterized in that: the index module (18) is connected with the judge module (14).

Images