CN117112348A - Non-invasive system service operation monitoring method, medium and electronic equipment - Google Patents

Abstract

The application provides a non-invasive system service operation monitoring method, a medium and electronic equipment. The method comprises the following steps: configuring an application to be monitored, a log analysis rule, a dynamic point burying rule and an alarm rule, and acquiring log data of the application to be monitored; processing the log data based on the log analysis rule and the dynamic embedded point rule to obtain monitoring index data; and carrying out monitoring alarm based on the alarm rule and the monitoring index data. According to the application, the embedded point is carried out based on log data analysis and calculation, so that the monitoring and visual checking and analysis of the service operation condition are realized through a non-invasive dynamic embedded point mode, the service operation condition is known in real time, and the potential problem is found and solved in time.

Description

Non-invasive system service operation monitoring method, medium and electronic equipment

Technical Field

The application belongs to the technical field of data monitoring, in particular to the technical field of business system monitoring, and particularly relates to a non-invasive system business operation monitoring method, medium and electronic equipment.

Background

In today's business environment, stability and efficiency of business operations are critical to the success of an enterprise. In order to ensure smooth operation of the service, it is necessary to know the operation condition of the service in real time and discover and solve the potential problem in time.

The prior art can only monitor the running state (such as QPS, RT, abnormity, etc.) of the system, and needs to embed points in the monitored application, so that a certain degree of invasion is caused to the monitored application, and the monitoring, alarming and visual viewing of the service running state of the user cannot be flexibly met.

Disclosure of Invention

The application provides a non-invasive system service operation monitoring method, medium and electronic equipment, which are used for realizing monitoring, visual checking and analysis of service operation conditions in a non-invasive dynamic point burying mode.

In a first aspect, an embodiment of the present application provides a method for monitoring operation of a non-invasive system service, including: configuring an application to be monitored, a log analysis rule, a dynamic point burying rule and an alarm rule, and acquiring log data of the application to be monitored; processing the log data based on the log analysis rule and the dynamic embedded point rule to obtain monitoring index data; and carrying out monitoring alarm based on the alarm rule and the monitoring index data.

In an implementation manner of the first aspect, a filebed log data collector is used to collect and acquire log data of the application to be monitored; the acquiring the log data of the application to be monitored by using the filebed log data collector comprises the following steps: downloading and decompressing a filebean program package from a server of an application to be monitored; and editing the filecoat.yml configuration file under the filecoat program package directory to respectively configure the reading information of the log data and the output target of the log data.

In an implementation manner of the first aspect, configuring the log parsing rule includes: inputting log sample data of an application to be monitored; identifying a log data format of the log sample data; and configuring a corresponding analysis mode and analysis expression for the application to be monitored based on the log data format.

In an implementation manner of the first aspect, configuring the dynamic buried point rule includes configuring: monitoring application name, buried point name, extraction value type, extraction source field, extraction method parameters and extraction value data type.

In an implementation manner of the first aspect, processing the log data based on the log parsing rule and the dynamic embedded point rule includes: acquiring log analysis data from the log data based on the log analysis rule; acquiring buried point monitoring data from the log analysis data based on the dynamic buried point rule; and extracting monitored event data and state data from the buried point monitoring data.

In an implementation manner of the first aspect, the log data includes log data collected by a fileshoe log data collector and broadcast stream data of a log parsing rule configuration table and a dynamic buried point configuration table.

In an implementation manner of the first aspect, the method further includes: acquiring real-time monitoring index data and storing the monitoring index data; the monitoring alarm based on the alarm rule and the monitoring index data comprises: monitoring stream type alarming based on the alarming rule and the real-time monitoring index data; and carrying out periodic monitoring alarm from the stored monitoring index data based on the periodic timing of the alarm rule.

In an implementation manner of the first aspect, the log data is processed based on an Apache link framework and a distributed processing engine, monitoring index data is obtained, and monitoring alarm is performed based on the alarm rule and the monitoring index data.

In a second aspect, an embodiment of the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the non-intrusive system traffic operation monitoring method of any of the first aspects of the present application.

In a third aspect, an embodiment of the present application provides an electronic device, including: a memory storing a computer program; and the processor is in communication connection with the memory and executes the non-invasive system service operation monitoring method according to any one of the first aspect of the application when the computer program is called.

According to the non-invasive system service operation monitoring method provided by the embodiment of the application, the embedded point is carried out based on log data analysis and calculation, the monitoring and visual checking and analysis of service operation conditions are realized through a non-invasive dynamic embedded point mode, the service operation condition is known in real time, and the potential problem is found and solved in time.

Drawings

Fig. 1 is a schematic diagram showing the distinction between the embedded point mode and the conventional embedded point mode in the non-invasive system service operation monitoring method according to an embodiment of the present application.

Fig. 2 is a schematic diagram of an application scenario of a non-invasive system service operation monitoring method according to an embodiment of the present application.

Fig. 3 is a schematic diagram illustrating the overall flow execution of a non-intrusive system operation monitoring method according to an embodiment of the present application.

Fig. 4 is a flow chart of a non-intrusive system traffic operation monitoring method according to an embodiment of the present application.

Fig. 5 is a schematic diagram of a service monitoring system in a non-intrusive system service operation monitoring method according to an embodiment of the present application.

Fig. 6 is a schematic diagram of a web front end configuration page of an application to be monitored in a non-intrusive system service operation monitoring method according to an embodiment of the present application.

Fig. 7 is a schematic diagram of a web front end configuration page of log parsing rules in a non-intrusive system service operation monitoring method according to an embodiment of the present application.

Fig. 8 is a flowchart of a log parsing rule editing and submitting process in a non-intrusive system service operation monitoring method according to an embodiment of the application.

Fig. 9 is a schematic diagram of a web front end configuration page of a dynamic embedded rule in a non-intrusive system service operation monitoring method according to an embodiment of the present application.

Fig. 10a is a schematic diagram of an event editing page element configuration page in a non-intrusive system service operation monitoring method according to an embodiment of the present application.

Fig. 10b is a schematic diagram of a configuration page of a status editing page element in a non-intrusive system service operation monitoring method according to an embodiment of the present application.

FIG. 11 is a schematic diagram of the Flink operation and status management in a non-intrusive system operation monitoring method according to an embodiment of the present application.

Fig. 12 is a data flow diagram of log parsing and buried points in a non-intrusive system service operation monitoring method according to an embodiment of the present application.

Fig. 13 is a timing chart showing log parsing in a non-intrusive system service operation monitoring method according to an embodiment of the present application.

FIG. 14 is a timing diagram illustrating the operation of the embedded data extraction and calculation in a non-intrusive system operation monitoring method according to an embodiment of the present application.

Fig. 15 shows a web front-end page for monitoring alarm rules in a non-intrusive system business operation monitoring method according to an embodiment of the present application.

Fig. 16 is a schematic diagram illustrating a flow alarm executing process in a non-intrusive system service operation monitoring method according to an embodiment of the present application.

Fig. 17 is a schematic structural diagram of an electronic device according to an embodiment of the application.

Detailed Description

Other advantages and effects of the present application will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present application with reference to specific examples. The application may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present application. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict.

The embodiment of the application provides a service monitoring scheme which is used for monitoring the operation condition of each service system and ensuring the stable and efficient operation of the service. The service monitoring scheme provided by the embodiment of the application has the following characteristics:

1) The access is quick and convenient, and the application can be accessed and monitored at any time without modification;

2) The buried point is carried out in the monitoring service, so that no invasion and low coupling are caused to the service system;

3) The response speed is high, and the real-time alarm can be given to the occurrence of the abnormality;

4) And the abnormal positioning is assisted, and data support is provided for abnormal positioning.

Specifically, an embodiment of the present application provides a non-invasive system service operation monitoring method, and fig. 1 is a schematic diagram showing a distinction comparison between a buried point mode and a conventional buried point mode in the non-invasive system service operation monitoring method of the present embodiment. As shown in fig. 1, the conventional point burying method is to perform point burying in the monitored application, which causes a certain degree of intrusion to the monitored application. The embedded point mode of the embodiment is different from other traditional embedded point modes, is not invaded into monitored application, is embedded point based on log data analysis and calculation, and realizes monitoring and visual checking and analysis of service running conditions through a non-invasive dynamic embedded point mode.

The embodiment of the application provides a non-invasive system service operation monitoring method. Fig. 2 is a schematic diagram of an application scenario of a non-invasive system service operation monitoring method according to an embodiment of the present application. As shown in fig. 2, in the non-invasive system service operation monitoring method of this embodiment, the service monitoring system outputs log data (or other data obtained by other modes) of an application to kafka through a filebean or other mode, and the Flink consumes the kafka data in real time, and performs processes such as parsing, format arrangement, data completion, aggregation calculation and the like on monitoring source data. The data processed by the Flink are classified into two types, namely monitoring index data, wherein the monitoring index data are aggregated according to time sequence and the dimension of events and states according to a rolling window with the size of 5 s. After the index data is output to the ES, pull Trigger pulls the data in real time, executes alarm rule calculation, and executes whether to send out alarm information according to the calculation result. And the other path of monitoring index data is output to kafka and Flow Trigger for real-time consumption, so that streaming alarm is realized. The Flow Trigger streaming alarm is different from Pull Trigger in that the streaming alarm does not depend on intermediate storage service, data in a short time is stored in application service, dependence on the storage service is avoided, and therefore the alarm in a short period is more real-time and reliable. Another type of data processed by the link is original data, and the link analyzes the ip, time, log level and other contents in the original data and then writes the analyzed contents into the ES according to a specified format. The original data can be subjected to query analysis through SQL at the application layer, and meanwhile, the original data can also be used as a data source of Pull Tirgger to execute alarm rules.

The following describes the technical solution in the embodiment of the present application in detail with reference to fig. 3 to 16 in the embodiment of the present application.

Fig. 3 is a schematic diagram illustrating the overall flow execution of a non-intrusive system operation monitoring method according to an embodiment of the present application. In this embodiment, as shown in fig. 3, by collecting logs for the monitored application, processing data, calculating indexes, executing alarm rules, sending an alarm, the monitoring and visual checking and analysis of service operation conditions are realized. Fig. 4 is a flowchart of a non-intrusive system service operation monitoring method according to an embodiment of the present application. As shown in fig. 4, specifically, the method for monitoring operation of a non-invasive system service according to the embodiment of the present application includes the following steps S100 to S300.

Step S100, configuring an application to be monitored, a log analysis rule, a dynamic point burying rule and an alarm rule, and acquiring log data of the application to be monitored;

step S200, processing the log data based on the log analysis rule and the dynamic embedded point rule to obtain monitoring index data;

and step S300, monitoring alarm is carried out based on the alarm rule and the monitoring index data.

Steps S100 to S300 of the non-intrusive system service operation monitoring method of the present embodiment will be specifically described with reference to fig. 2 to 8 in the embodiment of the present application.

Fig. 5 is a schematic diagram of a service monitoring system in a non-intrusive system service operation monitoring method according to an embodiment of the present application. As shown in fig. 5, the service monitoring system service formed when steps S100 to S300 of the non-invasive system service operation monitoring method of the present embodiment are executed includes: monitoring application management, monitoring index management, real-time alarm data calculation, monitoring alarm service and visual retrieval analysis service. The monitoring application management comprises application information management, application log data acquisition and log analysis rule management; the monitoring buried point management comprises dynamic buried point rule management and application event-state management, and the real-time alarm data calculation comprises log analysis, buried point data extraction and index calculation; the monitoring alarm service comprises monitoring alarm rule management, periodically executing alarm and real-time stream alarm.

The following describes the non-invasive system service operation monitoring method of the present embodiment in connection with the service monitoring system service formed when the steps S100 to S300 of the non-invasive system service operation monitoring method of the present embodiment are executed.

Step S100, configuring an application to be monitored, a log analysis rule, a dynamic point burying rule and an alarm rule, and acquiring log data of the application to be monitored.

And configuring the application to be monitored, namely managing the application information. The configuration items for configuring the application to be monitored comprise application names (namely, globally unique identification of the application), application descriptions and the like, and the information is a connection anchor point of the monitored application and the monitoring alarm. The monitored application information is recorded, so that the embodiment can perform subsequent data acquisition, log data analysis, monitoring of buried points, monitoring index calculation and monitoring alarm rule execution on the monitored application.

Specifically, the present embodiment provides a web front end configuration page of an application to be monitored, and fig. 6 is a schematic diagram of the web front end configuration page of the application to be monitored in the non-intrusive system service operation monitoring method according to an embodiment of the present application. Specifically, as shown in fig. 6, the web front-end configuration page of the application to be monitored includes 2 form items of displaying a chinese name and an application name, submitting the form after the user inputs/selects the form content, and persisting the application information into the MySQL database after the back-end service receives the form submitting request.

In one implementation manner of this embodiment, configuring the log parsing rule includes: inputting log sample data of an application to be monitored; identifying a log data format of the log sample data; and configuring a corresponding analysis mode and analysis expression for the application to be monitored based on the log data format.

In this embodiment, by inputting log sample data of an application to be monitored, identifying the log data format of the application, setting a corresponding analysis mode and an analysis expression, and executing the set analysis mode and analysis expression on the log information of the application, data with a specific meaning, such as an application server IP outputting a log, a level of the log, an event of printing the log, log content output by a user, and the like, can be analyzed from the log information, and the data obtained by analysis can be used as a data source of a buried point for performing service monitoring subsequently.

Specifically, the present embodiment provides a web front end configuration page of a log parsing rule, fig. 7 is a schematic diagram of the web front end configuration page of the log parsing rule in the non-intrusive system service operation monitoring method according to an embodiment of the present application, as shown in fig. 7, the web front end configuration page of the log parsing rule includes 5 form items including a monitoring application name, a log sample, a log parsing mode expression, and log parsing field information, a user submits a form after inputting/selecting the form content, and a back end service persistence the application log parsing rule information into a MySQL database after receiving a form submitting request. The parsing pattern options include separator patterns and regular extraction patterns.

The log content is a character string output by a vertical line ('|') dividing mode, and each section of content is respectively IP, log printing time, log level, thread name, link tracking mark, log output class name and user output log content according to the vertical line from the beginning to the end of the character string.

When the user selects the resolution mode to be the 'separator mode' after inputting the log sample, automatically generating a resolution expression of a json character string description in a resolution expression input box, wherein the expression comprises two items of contents: a separator and a segmented content name ("content key"), with the default separator being a vertical line ('|'), the user can get different content parsing results by modifying the separator. When editing the separator, the user displays the results obtained by analyzing the log sample data in real time according to the input separator in the table of analyzing the field information from top to bottom in sequence from top to bottom, and the user can modify the field name list content in the table and synchronously display the modified content on the segment content names in the expression in real time. The analytical expression of the log content in the separator mode in the above example is as follows:

JSON

{"separator":"|","contentKey":["ip","logTime","level","threadName","trace","loggerName","message"]}

The log content in the above example can also use a "regular extraction pattern" to achieve log parsing. When the user inputs a log sample, and selects an analysis mode as an extraction mode, the system automatically generates a named group regular expression sample in an analysis expression input box, after the user edits the named group regular expression matched with the log content, the system extracts data with specific meaning in the log data by executing the regular expression input by the user on the log sample data in real time, and the data are sequentially displayed in a table of analysis field information from top to bottom according to the sequence in the regular expression where the groups are located. Each naming capture packet in the regular expression corresponds to one extracted data, and the packet name is the field name corresponding to the content. The analytical expression of the regular extraction pattern of the log content in the above example is as follows:

JSON

^(？<ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\|(？<logTime>\d{4}-\d{2}-\d{2}\d{2}:\d{2}:\d{2}\.\d{3})\|(？<level>[A-Z]+)\|\[(？<threadName>[^\]]+)\]\|(？<trace>[^|]+)\|(？<logger Name>[^|]+)\|(？<message>.+)$

after a user submits a form, the back-end service receives a request message, firstly analyzes the regular expression, analyzes the corresponding group name and the sequence number corresponding to the group name, and then stores the group name and the sequence number in a MySQL database. The parse description json object obtained by parsing is as follows:

The pattern is a regular expression of a naming capture packet for extracting corresponding data in log data, and the namedGroups is a mapping of a corresponding packet name and a corresponding packet sequence number. After the user submits the form, the process flow is shown in FIG. 8.

In this embodiment, the monitoring scene of the service is abstracted into corresponding events and states, and an event or a group of events can form a monitoring point, and the index data monitored by the system is obtained by calculating the data of the monitoring points in real time. Such as the number of times of occurrence of an event in about 10 minutes, the number of times of occurrence of a failure state of an event in about 10 minutes, etc., can be used as an index for monitoring the running state of an application service.

The dynamic embedded point rule is used for describing that event and state data are extracted from log data of an application, and the event and state of each event correspond to each business operation in execution of the application and the result state of the action executed at the time of the operation. And configuring a buried point rule, namely correspondingly adding an event monitoring point to the application, and monitoring the whole running state of the application by adding a plurality of monitoring points needing to be concerned to the application.

According to the embodiment, based on the log analysis rule configured by the monitored application, the fields analyzed from the monitored application log are processed, so that data such as events and states representing the running condition of the application service are obtained, embedding of the monitoring application is realized, and data are provided for monitoring index calculation. In one implementation of this embodiment, configuring the dynamic buried point rule includes configuring: monitoring application name, buried point name, extraction value type, extraction source field, extraction method parameters and extraction value data type.

Specifically, the embodiment provides a web front end configuration page of a dynamic point burying rule, fig. 9 is a schematic diagram of the web front end configuration page of the dynamic point burying rule in the non-invasive system service operation monitoring method according to an embodiment of the present application, and as shown in fig. 9, the web front end configuration page of the dynamic point burying rule includes 7 form element contents including a monitoring application name, a point burying name, an extraction value type, an extraction source field, an extraction method parameter, and an extraction value data type, and a user submits a form after inputting/selecting the form content, and a back end service persists application dynamic point burying rule information into a MySQL database after receiving a form submitting request. The form content is shown in fig. 9.

The buried point name is the current buried point identifier so that a user can read and inquire; the monitoring application is a drop-down list, and the option is an application name recorded in application information management.

The data filtering block is used for configuring data filtering rules before event extraction, so that buried point operation is matched with accurate log data, the data amount of extraction operation is reduced, and resource consumption is reduced. The data filtering is to judge whether the application log analysis result data field is true or not through executing logic, so that the accuracy of embedded point data extraction is improved. In the filtering condition, the field option is a drop-down list, and the list item is a field obtained by analyzing the monitoring application log data; the field value data type is a drop-down list, and the list items comprise character strings, numbers and time character strings (yyyMMddHH: mm: ss).

The condition is selected as a drop-down list, selectable items are different for different selected field value data types, and when the selected field value data type is a character string, the selectable items include, not include, start with, end with, regular match, equal to, not equal to; when the selected field value data type is a number, the condition selectable items include equal to, not equal to, less than or equal to, greater than or equal to; when the selected field value data type is a time string (yyyyMMddHH: mm: ss), the condition selectable items include equal to, not equal to, less than or equal to, greater than or equal to; each field may have multiple matching conditions, and the multiple conditions are in a combined relationship, and the field filtering condition is true only if the field satisfies multiple conditions at the same time.

The filtering conditions can be multiple, the filtering conditions of the multiple fields are in a same relation, and only if the filtering conditions are true, the data can enter the event value operation and can be output to the next operation.

The extraction value type is content related to event execution by applying event, state, time consuming event execution, event occurrence time and the like, meanwhile, the extraction field of each embedded point regular configuration cannot be repeated, and each embedded point must contain an event extraction field and a state extraction field.

The source field is a drop-down list and corresponds to a field name resolved by the log resolution rule configured for application.

The extraction method provides three options for a drop-down selection box, namely direct value, regular value and constant. The direct value represents a value of an event or a state or the like in which a field value parsed from the application log is directly taken as a buried point; the regular value is a value of an event or a state, etc. of which the character string corresponding to the regular expression description is taken as a buried point from a certain field obtained by analyzing the log by using the regular expression, which is very useful when the event or the state data is contained in the log description content output by the system; the constant is a value that designates a constant as an event or state of a buried point, such as the state code 500 that characterizes a service anomaly.

When the extraction method is direct value, the parameters of the extraction method do not need to be input with contents; when the extraction method is a regular value, the extraction method parameter is a regular expression with a group named value; when the extraction method is constant, the extraction method parameter is a constant value. The extracted value data type is the data type of the extracted characterized event, state, event execution time consumption and other data, such as numbers, character strings, time date character strings and the like.

The present embodiment implements monitoring and alerting functions based on event-state data of the monitored application. The event and the execution state corresponding to the event are extracted from the log data of the monitored application. The event may be a fixed value, such as "login event", "modification event", etc., or a value that changes over time, such as an event beginning with "abc_", etc. When the monitoring alarm rule configuration is carried out, the corresponding event needs to be designated, so that the user is required to input the event information corresponding to the monitored application, and the selection during the monitoring alarm rule configuration is facilitated.

The embodiment provides a web front-end configuration page of an event-state of a monitored application, wherein the web front-end configuration page of the event-state of the monitored application comprises 5 list element contents of a monitoring application name, an event code, an event description, a state code and a state description, and simultaneously the web front-end configuration page also provides a sampling function, and the sampling function fills the event and the state which are not configured in the application into an event and state editing page by inquiring application log embedded point data so as to reduce manual input operation of a user. And after the user inputs/selects the form content, the form is submitted, and the back-end service persistence of the application dynamic embedded point rule information into the MySQL database is realized after receiving the form submitting request. The edit event and status information table element content is shown in fig. 10a and 10 b.

In one implementation manner of the embodiment, a filebed log data collector is adopted to collect and acquire log data of the application to be monitored; the acquiring the log data of the application to be monitored by using the filebed log data collector comprises the following steps: downloading and decompressing a filebean program package from a server of an application to be monitored; and editing the filecoat.yml configuration file under the filecoat program package directory to respectively configure the reading information of the log data and the output target of the log data.

Filebean is a lightweight, open source log data collector. It may collect and transmit log data from different sources and then send the data to a designated destination, such as Elasticsearch, kafka or output directly to a console. Using filebean, a variety of different log files, including system logs, application logs, and other related data sources, can be easily monitored.

Filebean will continually poll the specified log files, read the newly added content of those files, and send the content to the specified output destination. In this way, filebean can collect and process log data in real-time without waiting for the entire log file to be fully generated. In addition, the filecoat is very light and handy, occupies less system resources, supports multiple platforms, and is an excellent log collection tool.

The application log collection is to collect log data of the monitored application by using a filecoat, and provide data for later log analysis, buried point and monitoring index calculation.

After the monitored application server downloads and decompresses the filebox package, the filebox. Yml configuration file under the package directory is edited, and the read data and the output target are respectively configured, wherein the configuration content is exemplified as follows:

the file at.inputs part is input information for configuring collected data, the type log indicates that the collected data is a log file, paths are used for designating paths of application log files, fields are used for customizing output fields, and app_name is used for designating an application to which the output log data belongs. The specification app_name is necessary, and is an identification of an application to which data belongs, and information related to the application, such as a log analysis rule, a log embedded point rule and the like configured by a corresponding application, can be found through the specification app_name in log analysis, so that the data and the application are associated in monitoring. The output Kafka section is an output target Kafka configuring the acquisition data, hots are used to set the address and port of Kafka, and topic is used to which Kafka topic the configuration message is sent. The execution of the command sudo/filebean-e-c filebean.yml may initiate filebean, implement log data collection and output to the designated kafka topic.

And step 200, processing the log data based on the log analysis rule and the dynamic embedded point rule to obtain monitoring index data.

In one implementation of this embodiment, processing the log data based on the log parsing rule and the dynamic embedded point rule includes:

1) Acquiring log analysis data from the log data based on the log analysis rule;

2) Acquiring buried point monitoring data from the log analysis data based on the dynamic buried point rule;

3) And extracting monitored event data and state data from the buried point monitoring data.

In one implementation manner of this embodiment, the log data includes log data collected by a filebed log data collector, and broadcast stream data of a log parsing rule configuration table and a dynamic buried point configuration table.

Real-time alarm data calculation is finally aimed at calculating monitoring index data from monitored application log data in real time, and is used for calculating a monitoring alarm rule and giving an alarm, and the operation of the real-time alarm data calculation is mainly divided into three main parts: journal data analysis, buried data extraction and monitoring index calculation.

In this embodiment, first, file is consumed in real time by using a link job to collect monitored application log data output to kafka, and meanwhile, an application log analysis rule stored in MySQL is consumed through a link-CDC broadcast stream, the log data is analyzed according to the application configured log analysis rule to obtain data of a specified field, and then, format arrangement and data completion are performed, and then, kafka is output downwards.

And then, consuming the Flink operation of the log analysis result data, consuming buried point rule data stored in MySQL through the Flink-CDC broadcast stream, extracting corresponding event and state data according to the buried point rule configured by application, and outputting the event and state data to a data source calculated by using kafka as a monitoring index.

In the process of operation, a user modifies the applied log analysis rule and buried point rule through a page, the Flink-CDC captures the change of the log analysis rule and buried point rule data in the MySQL database, and then the change is read by the Flink operation and updates the rule for processing data in the Flink operation, and the data is processed according to the new rule.

And finally, calculating the data of information such as events, states and the like of the Flink operation consumption buried points of the monitoring indexes, calculating according to the dimension of the events and the states as a window according to 5 seconds, such as the occurrence quantity, average time consumption, maximum time consumption and the like of a certain state of a certain event of approximately 5 seconds, outputting one path of index results obtained by calculation to an ES for storage, and outputting the other path of index results to the kafka topic of the stream alarm.

In one implementation of this embodiment, the log data is processed based on an Apache link framework and a distributed processing engine, monitoring index data is obtained, and monitoring alarms are performed based on the alarm rules and the monitoring index data.

Apache Flink is a framework and distributed processing engine for stateful computation on borderless and bordered data streams. The flank can operate in all common cluster environments and can perform calculations at memory speed and on any scale. The Flink operation and state management is shown in fig. 11.

Flink-CDC (Change Data Capture) is a streaming data processing technique that may be used to capture and process data changes in a MySQL database, which may include operations such as adding, updating, and deleting. Compared with the traditional batch processing mode, the Flink-CDC is more real-time, accurate and efficient, because the latest data change can be acquired at any time and timely processed according to the requirement.

The basic working principle of the Flink-CDC is: the MySQL database table is monitored, and when the data in the table changes, the corresponding change record is recorded and then transmitted to other systems for processing and analysis. Compared with the traditional polling mode, the Flink-CDC can capture data change in real time, not only can improve the accuracy and instantaneity of data processing, but also can lighten the load of a database and improve the performance and expandability of the system.

As shown in fig. 12, the log parsing and the embedded point Flink have two data sources, one is the target kafka topic data output after the filedate collects log data, and the other is the broadcast stream data of the monitoring application log parsing rule configuration table and the dynamic embedded point configuration table read through MySQL-cdc connection.

And carrying out format arrangement, data complementation and content classification splitting on the original log data of the monitored application through the Flink operation to obtain formatted log data convenient for buried point extraction processing. When the journal analysis Flink operation is started, the application journal analysis rule stored in the MySQL table is read, and the rule is cached in the memory. After the Flink operation is started, real-time consumption filebat collects data output to kafka, an app_name field value in the consumed data is firstly obtained, application log analysis rule configuration information with an application name equal to the app_name field value is searched in a cache through the value, log data are analyzed to obtain corresponding field information according to the log analysis rule, and the analyzed data are output to downstream kafka. The log parsing timing diagram is shown in fig. 13.

And carrying out format arrangement, data complementation and content classification splitting on the original log data of the monitored application through the Flink operation to obtain formatted log data convenient for buried point extraction processing. When the embedded point data extraction Flink operation is started, the application dynamic embedded point rule stored in the MySQL table is read and cached in the memory. After the Flink operation is started, consuming the parsed log data, searching an embedded point rule configuration information list of an application with the same application name in a cache through an app_name field value in the data, distributing data streams according to rules, each rule is a data stream, executing respective embedded point rules, performing field extraction operation on the data meeting rule filtering conditions, extracting event, state and other data, and outputting the event, state and other data to a downstream kafka for subsequent monitoring index calculation. The buried data extraction computation timing diagram is shown in fig. 14.

The monitoring and alarming rules are used for carrying out scheduling configuration of conditions, combinations, thresholds and the like based on the indexes, then executing the rules according to index data and giving an alarm for the condition meeting the rule conditions. The buried point data is data obtained by extracting and sorting through the flank operation according to the buried point rule, and the fields contained in the data are fixed fields, and the buried point data comprise: an application name (app_name), an event name (event), a status code (status), a time consuming service execution (duration), and an event occurrence event (event_time) to which the buried data belongs.

And using the Flink operation to consume Kafka data output by the buried point, performing data calculation of monitoring indexes, calculating the atomic indexes according to the application, the event and the state, wherein the size of a dimension window is 5s, and then outputting the calculated atomic indexes to an ES storage service and a Kafka topic.

And step S300, monitoring alarm is carried out based on the alarm rule and the monitoring index data.

In one implementation of this embodiment, the method further includes: acquiring real-time monitoring index data and storing the monitoring index data; the monitoring alarm based on the alarm rule and the monitoring index data comprises: monitoring stream type alarming based on the alarming rule and the real-time monitoring index data; and carrying out periodic monitoring alarm from the stored monitoring index data based on the periodic timing of the alarm rule.

The monitoring alarm rule management of the embodiment is used for providing configuration application monitoring alarm rules and execution rule calculation for users and sending out alarms for monitoring trigger rule thresholds.

In this embodiment, the calculation of the monitoring alarm rule is divided into two parts, one part is real-time streaming alarm, and the calculation of the alarm rule is triggered by real-time streaming data (Flow Trigger); the other part is a timing period pulling and executing alarm rule (Pull Trigger), which periodically triggers the execution of a calculation task through timing task scheduling, and pulls monitoring index data to calculate the alarm rule. The streaming alarm Flink task consumes the monitoring alarm index data in real time, executes alarm rule calculation in real time, gives an alarm once the rule is triggered, has high instantaneity, does not increase the query load on the ES storage along with the increase of the number of the alarm rules, and is very suitable for monitoring alarm rules with shorter time windows.

The embodiment provides a web front-end page for monitoring alarm rules, which is used for providing monitoring rules for configuring monitoring scenes required by users for the users, the users submit form requests to the back-end service, and the back-end service receives the submitted form data and then persists the form data to the MySQL database.

The monitoring rule configuration page element comprises three types of large contents, namely rule basic information, rule definition and notification policy. The rule basic information comprises a monitoring rule name, a monitoring application and an alarm triggering calculation mode; the rule definition comprises monitoring index data query SQL, a query time range, SQL test, execution frequency, rule logic expression, duration and alarm level; the notification policy includes notification date, notification time period, notification mode, risk level, and alert recipient. A schematic diagram of the monitor alarm rule configuration edit page element is shown in fig. 15.

The monitoring application is a monitored application recorded in application management; the alarm triggering calculation mode is divided into a period Pull execution alarm (Pull Trigger) and a real-time Flow alarm (Flow Trigger), and because the real-time Flow alarm is not executed regularly, the execution frequency is not displayed in the rule definition when the alarm triggering calculation mode selects the real-time Flow alarm.

The data query column is SQL for index data aggregation query, through which monitoring index data is to be queried from the ES storage service. The query time range is used to limit the event range of the SQL query data, which includes a characteristic time period (e.g., the day, yesterday, etc.) and a relative time (e.g., approximately 5 minutes, etc.). The SQL test is used for testing and verifying whether the written SQL is correct or not and returning a desired result or not; there may be multiple SQL's for a data query, but the query result field of each SQL cannot be repeated. The execution frequency selectable items comprise fixed frequency and a Cron expression, and when the execution frequency selectable items are set to be fixed frequency, the monitoring alarm rule is executed at regular time according to the set frequency value and the period expressed by the unit; when the Cron expression is set, the monitoring alarm rule is executed according to the plan described by the Cron expression. The alarm rule threshold column is used for describing an alarm triggering rule, when the logic expression is hit, the logic expression is an expression conforming to the grammar of the java logic expression, each variable of the logic expression is a result field of the data query SQL, and the value is a corresponding query result value. And performing logic calculation by assigning a value corresponding to the query result to a variable with a corresponding name in the expression, wherein if the logic expression calculation result is true, the logic expression calculation result indicates that the execution rule hits. The arbitration condition is used for describing whether the hit rule triggers an alarm or not, and is divided into two types: and a window, wherein the window refers to how many times the alarm rule is hit in total in a specified window time, and once the hit condition of the current rule meets the decision condition, an alarm notification selected by a risk level drop-down list is sent.

The notification policy is used to set the policy and goal of sending the alert. The notification date is used to specify what date needs to send an alarm after the rule is triggered, the date is composed of monday through sunday, when the corresponding date is selected, the alarm notification will be sent when the alarm is penalized on the date, otherwise, the alarm notification will not be sent, for example: the notification date is from Monday to Friday, the rule is penalized only on weekdays to give an alarm, and the weekend does not send an alarm notification even if the rule is triggered. The notification time period is similar to the notification date and is used for limiting the time period for sending out the alarm, so that one alarm notification can be sent out, and the rule trigger needs to be met at the same time in the notification time period in the notification date.

The notification configuration is used to specify what type of notification is sent to the specified person in what manner, wherein the notification manner includes mail, instant messaging, short message, etc. The risk level indicates that the alarm rule for that risk level is penalized and is used to send a notification to the corresponding designated person. The alarm receiver is a target person for receiving the alarm, and the alarm receiver can set a plurality of targets, such as E-mai, mobile phone numbers and the like, with the values corresponding to the corresponding notification modes.

The embodiment can Pull and execute alarm calculation (Pull Trigger) in a timing period. According to the calculation period configured by the monitoring alarm rule, the calculation is triggered at fixed time, the monitoring index data of the corresponding event-state configured in the monitoring rule is pulled (Pull) from the ES storage service to calculate, and the alarm notification of the corresponding level is sent out under the condition that the specified threshold condition is met.

The monitoring rule timing scheduling execution service comprises two parts, namely polling timing task registration and timing task execution monitoring alarm rule calculation. The polling timing registration task scans the monitoring alarm rule with the calculation time window larger than 5 minutes in the MySQL table at regular time, inquires whether the task exists in the alarm execution timing task queue through the name of the monitoring alarm rule, and if not, registers a corresponding timing task according to the latest trigger time point of the execution period of the task, thereby ensuring that each monitoring alarm rule cannot be executed in a missing mode.

After the timing task for executing the calculation of the monitoring alarm rule is triggered and executed, the task firstly inquires the monitoring alarm index aggregation data from the ES storage service according to the filtering condition, the time window and the aggregation function of the monitoring alarm rule corresponding to the timing task, then carries out logic operation according to the alarm triggering condition rule, and if the alarm threshold of some grades under the current rule is reached, the alarm with the highest grade in the triggered alarms is sent out. The back-end timing scheduling execution service is used for executing the monitoring rule according to the execution period of the monitoring rule and giving an alarm to the event conforming to the rule.

The embodiment also performs real-time streaming alarm calculation (Flow Trigger), calculates index data output to kakfa by the Flink job by using the Flink job real-time consumption monitoring index, performs alarm rule calculation in real time according to window sizes, trigger conditions and the like configured by the monitoring alarm rule, and sends out alarm notification of corresponding level under the condition of meeting the specified threshold condition.

The streaming alarm is calculated by triggering alarm rules by real-time streaming data, the real-time performance is higher compared with the calculation of executing alarm by periodically triggering pull data, and the query load to the ES storage service is not increased due to the increase of monitoring alarm rules. Because the Flink uses the memory to store and calculate the state, the cost of data with overlarge storage is high, so that the alarm is executed by triggering the pulled data in the use period of the alarm rule with the window more than 5 minutes, and the window uses real-time streaming alarm within 5 minutes.

The stream alarm uses the Flink operation to consume the monitoring alarm index data in real time, monitors the consumption monitoring alarm rule storage table through the Flink-CDC, stores the monitoring alarm rule information in the memory, calculates and executes the consumed corresponding application monitoring alarm index data in a rule mode, and sends out alarm notification if the monitoring alarm rule threshold is met. The flow monitoring alarm execution process is shown in fig. 16.

Therefore, the non-invasive system service operation monitoring method provided by the embodiment performs embedded point based on log data analysis and calculation, realizes monitoring and visual checking and analysis of service operation conditions through a non-invasive dynamic embedded point mode, knows service operation conditions in real time, and timely discovers and solves potential problems.

The protection scope of the non-invasive system service operation monitoring method according to the embodiment of the present application is not limited to the execution sequence of the steps listed in the present embodiment, and all the schemes implemented by adding or removing steps and replacing steps according to the prior art made by the principles of the present application are included in the protection scope of the present application.

The embodiment of the application also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the non-invasive system service operation monitoring method provided by any embodiment of the application.

Any combination of one or more storage media may be employed in embodiments of the present application. The storage medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The embodiment of the application also provides electronic equipment. Fig. 8 is a schematic structural diagram of an electronic device 100 according to an embodiment of the application. In some embodiments, the electronic device may be a mobile phone, tablet, wearable device, in-vehicle device, augmented Reality (Augmented Reality, AR)/Virtual Reality (VR) device, notebook, ultra-Mobile Personal Computer (UMPC), netbook, personal digital assistant (Personal Digital Assistant, PDA), or other terminal device. In addition, the embodiment of the application does not limit the specific application scene of the non-invasive system service operation monitoring method.

As shown in fig. 17, an electronic device 100 provided in an embodiment of the present application includes a memory 101 and a processor 102.

The memory 101 is for storing a computer program; preferably, the memory 101 includes: various media capable of storing program codes, such as ROM, RAM, magnetic disk, U-disk, memory card, or optical disk.

In particular, memory 101 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) and/or cache memory. Electronic device 100 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. Memory 101 may include at least one program product having a set (e.g., at least one) of program modules configured to carry out the functions of embodiments of the application.

The processor 102 is connected to the memory 101 for executing a computer program stored in the memory 101 to cause the electronic device 100 to execute the non-intrusive system operation monitoring method provided in any of the embodiments of the present application.

Alternatively, the processor 102 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

Optionally, the electronic device 100 in this embodiment may further include a display 103. A display 103 is communicatively coupled to the memory 101 and the processor 102 for displaying a GUI interactive interface associated with the non-intrusive system business operation monitoring method.

In summary, the method for monitoring operation of non-invasive system service provided by the embodiment of the application performs embedding based on log data analysis and calculation, realizes monitoring and visual checking and analysis of operation condition of service by a non-invasive dynamic embedding method, knows operation condition of service in real time, and discovers and solves potential problems in time. Therefore, the application effectively overcomes various defects in the prior art and has high industrial utilization value.

The above embodiments are merely illustrative of the principles of the present application and its effectiveness, and are not intended to limit the application. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the application. Accordingly, it is intended that all equivalent modifications and variations of the application be covered by the claims, which are within the ordinary skill of the art, be within the spirit and scope of the present disclosure.

Claims (10)

1. A method for non-intrusive system service operation monitoring, comprising:

configuring an application to be monitored, a log analysis rule, a dynamic point burying rule and an alarm rule, and acquiring log data of the application to be monitored;

processing the log data based on the log analysis rule and the dynamic embedded point rule to obtain monitoring index data;

and carrying out monitoring alarm based on the alarm rule and the monitoring index data.

2. The non-invasive system service operation monitoring method according to claim 1, wherein a filecoat log data collector is used to collect and obtain log data of the application to be monitored;

the acquiring the log data of the application to be monitored by using the filebed log data collector comprises the following steps:

Downloading and decompressing a filebean program package from a server of an application to be monitored;

and editing the filecoat.yml configuration file under the filecoat program package directory to respectively configure the reading information of the log data and the output target of the log data.

3. The non-intrusive system business operation monitoring method of claim 1, wherein configuring the log parsing rule comprises:

inputting log sample data of an application to be monitored;

identifying a log data format of the log sample data;

and configuring a corresponding analysis mode and analysis expression for the application to be monitored based on the log data format.

4. The non-intrusive system business operation monitoring method of claim 1, wherein configuring the dynamic embedded point rule comprises configuring: monitoring application name, buried point name, extraction value type, extraction source field, extraction method parameters and extraction value data type.

5. The non-intrusive system business operation monitoring method of claim 2, wherein processing the log data based on the log parsing rules and the dynamic embedded point rules comprises:

Acquiring log analysis data from the log data based on the log analysis rule;

acquiring buried point monitoring data from the log analysis data based on the dynamic buried point rule;

and extracting monitored event data and state data from the buried point monitoring data.

6. The non-intrusive system operation monitoring method of claim 5, wherein the log data comprises log data collected by a filebed log data collector and broadcast stream data of a log parsing rule configuration table and a dynamic buried point configuration table.

7. The non-intrusive system operation monitoring method of claim 1, further comprising: acquiring real-time monitoring index data and storing the monitoring index data;

the monitoring alarm based on the alarm rule and the monitoring index data comprises:

monitoring stream type alarming based on the alarming rule and the real-time monitoring index data;

and carrying out periodic monitoring alarm from the stored monitoring index data based on the periodic timing of the alarm rule.

8. The non-intrusive system operation monitoring method according to any of claims 1 to 7, wherein the log data is processed based on an Apache link framework and a distributed processing engine to obtain monitoring index data, and monitoring alarms are performed based on the alarm rules and the monitoring index data.

9. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program, when executed by a processor, implements the non-intrusive system traffic operation monitoring method of any of claims 1 to 8.

10. An electronic device, the electronic device comprising:

a memory storing a computer program;

a processor, in communication with the memory, which when invoked performs the non-intrusive system traffic operation monitoring method of any one of claims 1 to 8.