CN116450467A

CN116450467A - Monitoring method and device for distributed system, computer equipment and storage medium

Info

Publication number: CN116450467A
Application number: CN202310246882.9A
Authority: CN
Inventors: 王鹏培; 王鑫; 张家宇; 张丹枫
Original assignee: Industrial and Commercial Bank of China Ltd ICBC
Current assignee: Industrial and Commercial Bank of China Ltd ICBC
Priority date: 2023-03-15
Filing date: 2023-03-15
Publication date: 2023-07-18

Abstract

The application relates to a monitoring method, a monitoring device, computer equipment, a storage medium and a computer program product of a distributed system, and relates to the technical field of cloud computing. The method comprises the following steps: receiving a configuration request of a target monitoring view; inquiring a target database corresponding to the target data type in a preset mapping relation between the data type and the database; the database is used for storing the monitoring data of the corresponding data type; inquiring a target data table corresponding to the target system dimension and the target system level in each data table contained in the target database according to a preset mapping relation among the distributed system dimension, the distributed system level and the data table; and configuring data source information of the target monitoring view based on the target database and the target data table, and constructing the target monitoring view of the target distributed system based on the data source information and the display information. The method can monitor the operation of the distributed system and customize the target monitoring view.

Description

Monitoring method and device for distributed system, computer equipment and storage medium

Technical Field

The present disclosure relates to the field of cloud computing technologies, and in particular, to a monitoring method, an apparatus, a computer device, a storage medium, and a computer program product for a distributed system.

Background

A distributed system is a software system built on a network, has high cohesiveness and transparency, and is a system which is colloquially composed of a plurality of single machines with the same performance specifications and code meanings. In order to ensure stable operation of the distributed system, the operation of the distributed system needs to be monitored.

Thus, there is a need for a monitoring method of a distributed system that can monitor the operation of the distributed system.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a monitoring method, apparatus, computer device, computer readable storage medium, and computer program product for a distributed system capable of monitoring the operation of the distributed system.

In a first aspect, the present application provides a method for monitoring a distributed system. The method comprises the following steps:

receiving a configuration request of a target monitoring view; the configuration request comprises a target data type, a target system dimension, a target system level and display information;

inquiring a target database corresponding to the target data type in a preset mapping relation between the data type and the database; the database is used for storing the monitoring data of the corresponding data type;

Inquiring a target data table corresponding to the target system dimension and the target system level in each data table contained in the target database according to a preset mapping relation among the distributed system dimension, the distributed system level and the data table;

configuring data source information of the target monitoring view based on the target database and the target data table, and constructing a target monitoring view of a target distributed system based on the data source information and the display information; the target monitoring view is used for monitoring the distributed system.

In one embodiment, the building the target monitoring view of the target distributed system based on the data source information and the presentation information includes:

acquiring monitoring data of a target distributed system from the target data table in the target database based on the data source information;

based on the presentation information, a target monitoring view of the target distributed system is generated that includes the monitoring data, as well as the target data type, the target system dimension, and the target system hierarchy.

In one embodiment, the presentation information includes data mapping information, threshold information, and presentation effect information; the generating a target monitoring view of the target distributed system including the monitoring data, the target data type, the target system dimension, and the target system hierarchy based on the presentation information includes:

Performing data mapping setting on the target monitoring view based on the data mapping information to obtain a data mapping setting result;

based on the threshold information, performing threshold setting on the target monitoring view to obtain a threshold setting result;

and generating a target monitoring view of the target distributed system comprising the monitoring data, the target data type, the target system dimension and the target system level based on the data mapping setting result, the threshold setting result and the display effect information.

In one embodiment, the display effect information includes data display form information, data display standard information and data display style information; the generating a target monitoring view of the target distributed system including the monitoring data, the target data type, the target system dimension, and the target system hierarchy based on the data mapping setting result, the threshold setting result, and the presentation effect information includes:

setting a data display form of the target monitoring view based on the data display form information to obtain a data display form setting result;

setting a data display standard of the target monitoring view based on the data display standard information to obtain a data display standard setting result;

Setting the data display style of the target monitoring view based on the data display style information to obtain a data display style setting result;

and generating a target monitoring view of the target distributed system comprising the monitoring data, the target data type, the target system dimension and the target system level based on the data mapping setting result, the threshold setting result, the data presentation form setting result, the data presentation standard setting result and the data presentation style setting result.

In one embodiment, the method further comprises:

acquiring monitoring data of a target distributed system, and determining the data type of the monitoring data according to a preset data classification rule;

and determining a target storage space of the monitoring data according to the data type of the monitoring data and a preset storage rule, and storing the monitoring data.

In one embodiment, the storing the monitoring data includes:

inquiring a gateway corresponding to the monitoring data according to the data type of the monitoring data in a preset mapping relation between the data type and the gateway;

And transmitting the monitoring data to the target storage space through the gateway corresponding to the monitoring data for storage processing.

In one embodiment, the transmitting, by the gateway corresponding to the monitoring data, the monitoring data to the target storage space for storage processing includes:

aiming at each piece of monitoring data, carrying out message analysis on the monitoring data through a gateway corresponding to the monitoring data to obtain analysis data of the monitoring data;

verifying the analysis data based on a message integrity strategy to obtain a verification result of the monitoring data;

if the verification result of the monitoring data shows that the verification is passed, message filtering is carried out on the analysis data to obtain a filtering result of the monitoring data;

and if the filtering result of the monitoring data indicates that the monitoring data is not filtered, carrying out message assembly on the analysis data of the monitoring data to obtain processed monitoring data, and transmitting the processed monitoring data to the target storage space for storage processing.

In one embodiment, the method further comprises:

determining alarm levels meeting preset alarm conditions according to the sequence of the distributed system levels from low to high, and taking the highest distributed system level in the alarm levels as a monitoring level; determining alarm dimensions meeting preset alarm conditions according to the sequence of the distributed system dimensions from low to high, and taking the highest distributed system dimension in the alarm dimensions as a monitoring dimension;

And generating alarm information based on the monitoring hierarchy and the monitoring dimension.

In a second aspect, the present application further provides a monitoring device of a distributed system. The device comprises:

the receiving module is used for receiving a configuration request of the target monitoring view; the configuration request comprises a target data type, a target system dimension, a target system level and display information;

the first query module is used for querying a target database corresponding to the target data type in a preset mapping relation between the data type and the database; the database is used for storing the monitoring data of the corresponding data type;

the second query module is used for querying a target data table corresponding to the target system dimension and the target system level in each data table contained in the target database according to the preset mapping relation among the distributed system dimension, the distributed system level and the data table;

the construction module is used for configuring data source information of the target monitoring view based on the target database and the target data table, and constructing a target monitoring view of a target distributed system based on the data source information and the display information; the target monitoring view is used for monitoring the distributed system.

In one embodiment, the construction module is specifically configured to:

In one embodiment, the presentation information includes data mapping information, threshold information, and presentation effect information; the construction module is specifically configured to:

In one embodiment, the display effect information includes data display form information, data display standard information and data display style information; the construction module is specifically configured to:

In one embodiment, the apparatus further comprises:

the first determining module is used for acquiring monitoring data of the target distributed system and determining the data type of the monitoring data according to a preset data classification rule;

and the storage processing module is used for determining a target storage space of the monitoring data according to the data type of the monitoring data and a preset storage rule and carrying out storage processing on the monitoring data.

In one embodiment, the storage processing module is specifically configured to:

In one embodiment, the apparatus further comprises:

the second determining module is used for determining alarm levels meeting preset alarm conditions according to the sequence from low distributed system levels to high distributed system levels, and taking the highest distributed system level in the alarm levels as a monitoring level; determining alarm dimensions meeting preset alarm conditions according to the sequence of the distributed system dimensions from low to high, and taking the highest distributed system dimension in the alarm dimensions as a monitoring dimension;

and the generation module is used for generating alarm information based on the monitoring hierarchy and the monitoring dimension.

In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the steps of the first aspect described above when the processor executes the computer program.

In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of the first aspect described above.

In a fifth aspect, the present application also provides a computer program product. The computer program product comprising a computer program which, when executed by a processor, carries out the steps of the first aspect described above.

According to the monitoring method, the device, the computer equipment, the storage medium and the computer program product of the distributed system, the database and the data table of the target monitoring view are determined and configured based on the target data type, the target system dimension and the target system level in the configuration request of the target monitoring view sent by the user, the target monitoring view of the target distributed system is constructed based on the configuration result and the display information in the configuration request, and the monitoring on the operation of the distributed system is realized through the target monitoring view. Furthermore, the method constructs the target monitoring view based on the target data type, the target system dimension and the target system level in the configuration request of the target monitoring view sent by the user by setting the data type, the system dimension and the system level, and can customize the monitoring view of the distributed system in real time.

Drawings

FIG. 1 is a flow diagram of a method of monitoring a distributed system in one embodiment;

FIG. 2 is a flow diagram of a process for building a target monitoring view of a target distributed system in one embodiment;

FIG. 3 is a flow chart of a process for generating a target monitor view based on presentation information in one embodiment;

FIG. 4 is a flowchart of a step of generating a target monitor view based on a data map setting result, a threshold setting result, and presentation effect information in one embodiment;

FIG. 5 is a flow chart of a monitoring method of a distributed system according to another embodiment;

FIG. 6 is a flow chart illustrating steps of storing monitoring data in one embodiment;

FIG. 7 is a flow chart of a storage processing step of transmitting monitoring data to a target storage space through a gateway corresponding to the monitoring data in one embodiment;

FIG. 8 is a flow chart of a method of monitoring a distributed system in another embodiment;

FIG. 9 is a block diagram of a monitoring device of a distributed system in one embodiment;

fig. 10 is an internal structural view of a computer device in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

In one embodiment, as shown in fig. 1, a monitoring method of a distributed system is provided, where the method is applied to a terminal to illustrate the method, it is understood that the method may also be applied to a server, and may also be applied to a system including the terminal and the server, and implemented through interaction between the terminal and the server. Wherein the distributed system may be implemented by a cloud computing platform. For example, the distributed system may be implemented by a platform as a service (Platform as a Service, paaS). The terminal can be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things equipment and portable wearable equipment, and the internet of things equipment can be smart speakers, smart televisions, smart air conditioners, smart vehicle-mounted equipment and the like. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server may be implemented as a stand-alone server or as a server cluster composed of a plurality of servers. In this embodiment, the method includes the steps of:

Step 101, receiving a configuration request of a target monitoring view.

Wherein the configuration request includes a target data type, a target system dimension, a target system hierarchy, and presentation information.

In the embodiment of the application, the terminal receives a configuration request of the target monitoring view sent by a user. The target monitoring view is a view for monitoring the operation of the distributed system, and is a monitoring view customized according to a configuration request of a user. The configuration request is a customization request for customizing the target monitoring view for the user. The target data type is a data type of the distributed system specified by the user customized target monitoring view. The target data type may be one or more data types. The data types may include an index class, a system class, and a link class. The monitoring data of the index class is the monitoring data related to the transaction action but not related to the transaction link, and is used for measuring whether a transaction is successful or not. The monitored data for the metrics includes, but is not limited to, transaction amount, time spent in transactions, whether transactions were successful, and type of transaction anomaly. The system class monitoring data is data of an operating system related to the transaction. The system class monitoring data includes, but is not limited to, monitoring data of the java virtual machine (jvm) of the operating system and monitoring data of the Operating System (OS). The link class monitoring data is the monitoring data of the transaction link related to the transaction action, and is used for representing the transaction link. A transaction link is a series of services that a transaction may involve. For example, a payment transaction involves a number of services, and a series of services such as a agreement inquiry service, a blacklist verification service, a customer authentication service, etc. are invoked prior to payment, with the entire link being a completed payment transaction. The monitoring data of the link class includes, but is not limited to, link links and call relations. The target system dimension is a distributed system dimension specified by the user custom target monitor view. The target system dimension may be one or more distributed system dimensions. The distributed system dimensions include: cities, campuses, units and businesses. The target system level customizes the distributed system level specified by the target monitor view for the user. The target system level may be one or more distributed system levels. The distributed system hierarchy includes: applications, groups, services, nodes, and standalone machines. The presentation information is information of how the target monitoring view presents monitoring of the distributed system.

Specifically, the terminal presents data types, system dimensions, system levels, and alternative presentation information to the user. The user selects a target data type from the data types. The user selects a target system dimension from the system dimensions. The user selects a target system level among the system levels. The user determines the presentation information among the alternative presentation information. Then, the terminal receives a configuration request including the target data type, the target system dimension, the target system level and the presentation information of the target monitoring view sent by the user.

Step 102, querying a target database corresponding to the target data type in a preset mapping relation between the data type and the database.

The database is used for storing the monitoring data of the corresponding data type.

In the embodiment of the application, the terminal queries a target database corresponding to the target data type in a preset mapping relation between the data type and the database. The mapping relationship between the data types and the databases may correspond to different databases for different data types. For example, the data type is an index class, and the corresponding database may be a column-type database, and in particular, the corresponding database may be a clickHouse database. Therefore, the query performance can be improved by fully utilizing the high-efficiency data convergence computing capability of the columnar database, so that the monitoring instantaneity is improved. For example, the data type is a link class, and the corresponding database may be an elastic search database. Therefore, the elastic search database with strong search engine capability is selected to store link monitoring data, and the accuracy of monitoring can be improved.

Step 103, inquiring a target data table corresponding to the target system dimension and the target system level in each data table contained in the target database according to the preset mapping relation among the distributed system dimension, the distributed system level and the data table.

In the embodiment of the application, the terminal presets the mapping relation among the dimension of the distributed system, the hierarchy of the distributed system and the data table. And then, the terminal queries the target data table corresponding to the dimension of the target system and the target system level in the data tables contained in the target database according to the mapping relation. The target data table is used for storing the monitoring data of the dimension of the target system and the hierarchy of the target system.

And 104, configuring data source information of the target monitoring view based on the target database and the target data table, and constructing the target monitoring view of the target distributed system based on the data source information and the display information.

The target monitoring view is used for monitoring the distributed system.

In the embodiment of the application, the terminal configures the target data table in the target database as the data source of the target monitoring view to obtain the data source information of the target monitoring view. And then, the terminal constructs a target monitoring view of the target distributed system based on the data source information and the display information.

In one example, after obtaining the data source information of the target monitoring view, the terminal configures data authority for the target monitoring view according to the configuration request, and obtains the data authority information of the target monitoring view. And then, the terminal constructs a target monitoring view of the target distributed system based on the data source information, the data authority information and the display information. Wherein, the data authority includes: query, modify, add, and delete. The default read-only user has only query rights. Therefore, the data table configuration can be performed only after the database configuration is completed, the data authority configuration can be performed only after the database configuration and the data table configuration are completed, the full-database full-table query is avoided through layer-by-layer control of the data range, the attack query can be prevented, and the monitoring safety and the query efficiency of the distributed system are improved.

In the monitoring method of the distributed system, the database and the data table of the target monitoring view are determined and configured based on the target data type, the target system dimension and the target system level in the configuration request of the target monitoring view sent by the user, the target monitoring view of the target distributed system is constructed based on the configuration result and the display information in the configuration request, and the monitoring on the running of the distributed system is realized through the target monitoring view. Furthermore, the method constructs the target monitoring view based on the target data type, the target system dimension and the target system level in the configuration request of the target monitoring view sent by the user by setting the data type, the system dimension and the system level, and can customize the monitoring view of the distributed system in real time. In addition, the distributed system is monitored through one monitoring view in multiple system dimensions, multiple system levels and multiple data types, so that the problem of the distributed system can be accurately positioned, the switching time among monitoring tools, among monitoring views and the like can be reduced, the monitoring efficiency of the distributed system is improved, and the production problem solving time is shortened.

In one embodiment, as shown in FIG. 2, the specific process of building a target monitoring view of a target distributed system based on data source information and presentation information includes the steps of:

step 201, based on the data source information, obtaining the monitoring data of the target distributed system from the target data table in the target database.

In the embodiment of the application, the terminal acquires the monitoring data of the target distributed system from the target data table in the target database based on the data source information.

Step 202, based on the presentation information, generating a target monitoring view of the target distributed system including the monitoring data, as well as the target data type, the target system dimension, and the target system hierarchy.

In the embodiment of the application, the terminal performs display configuration on the target monitoring view based on the display information to obtain a display configuration result. And then, the terminal generates a target monitoring view of the target distributed system comprising monitoring data, target data types, target system dimensions and target system levels according to the display configuration result.

According to the monitoring method of the distributed system, based on the data source information, the monitoring data of the target distributed system is obtained from the target data table in the target database, and the target monitoring view of the target distributed system comprising the monitoring data, the target data types, the target system dimension and the target system level is generated based on the display information, so that the distributed system can monitor the multiple system dimensions, the multiple system levels and the multiple data types through one monitoring view, the problem of the distributed system can be accurately positioned, the switching time among monitoring tools, among monitoring views and the like can be reduced, the monitoring efficiency of the distributed system is improved, and the production problem solving time is shortened.

In one embodiment, the presentation information includes data mapping information, threshold information, and presentation effect information. As shown in fig. 3, the specific process of generating a target monitoring view of a target distributed system including monitoring data, as well as target data types, target system dimensions, target system levels, based on presentation information, includes the steps of:

step 301, performing data mapping setting on the target monitoring view based on the data mapping information to obtain a data mapping setting result.

In the embodiment of the application, the terminal performs data mapping setting on the target monitoring view based on the data mapping information to obtain a data mapping setting result. The data mapping information is used for representing the mapping relation information between the original data of the monitoring data and the display data in the monitoring view. The data mapping information may include: data value mapping information, data range mapping information, and data type mapping information. The data value mapping information is used for representing the mapping relation between the specific value of the monitoring data and the display value in the monitoring view. The data range mapping information is information for representing a mapping relationship between a specific value of the monitor data and a presentation range of the presentation value in the monitor view. The data type mapping information is information for representing a mapping relationship of the data type of the monitor data and the data type of the display value in the monitor view. For example, the data types of the display values in the monitor view may be null and boolean types.

In one example, the terminal performs data value mapping setting on the target monitoring view based on the data value mapping information to obtain a data value mapping setting result. And then, the terminal performs data range mapping setting on the target monitoring view based on the data range mapping information to obtain a data range mapping setting result. And then, the terminal performs data type mapping setting on the target monitoring view based on the data type mapping information to obtain a data type mapping setting result. Then, the terminal constructs the data value map setting result, the data range map setting result, and the data type map setting result into a data map setting result.

And step 302, performing threshold setting on the target monitoring view based on the threshold information to obtain a threshold setting result.

In the embodiment of the application, the terminal sets the threshold value of the target monitoring view based on the threshold value information to obtain a threshold value setting result. Wherein the threshold information is used to represent information related to each index threshold of the monitoring data. The threshold information may include: threshold mode information, threshold value information, and threshold color information. The threshold pattern information is used to represent a threshold pattern of a threshold set by a user. The threshold pattern may include: absolute and percent values. The threshold value information is used to represent the size of a threshold value set by the user. The threshold value information is used to represent the presentation color of the threshold value set by the user.

In one example, the terminal sets a threshold mode for the target monitoring view based on the threshold mode information, and a threshold mode setting result is obtained. And then, the terminal sets the threshold value of the target monitoring view based on the threshold value information to obtain a threshold value setting result. And then, the terminal sets the threshold color of the target monitoring view based on the threshold color information to obtain a threshold color setting result. Then, the terminal composes the threshold mode setting result, the threshold value setting result, and the threshold color setting result into a threshold setting result.

Step 303, generating a target monitoring view of the target distributed system including the monitoring data, the target data type, the target system dimension and the target system level based on the data mapping setting result, the threshold setting result and the display effect information.

In the embodiment of the application, the terminal configures the display effect of the target monitoring view based on the display effect information, and a display effect configuration result is obtained. Then, the terminal generates a target monitoring view of the target distributed system including the monitoring data, the target data type, the target system dimension and the target system level based on the data mapping setting result, the threshold setting result and the presentation effect configuration result.

In the monitoring method of the distributed system, based on the data mapping information, the data mapping setting is carried out on the target monitoring view to obtain the data mapping setting result, the threshold setting is carried out on the target monitoring view based on the threshold information to obtain the threshold setting result, and the target monitoring view of the target distributed system comprising the monitoring data, the target data type, the target system dimension and the target system level is generated based on the data mapping setting result, the threshold setting result and the display effect information, so that the strict display of the monitoring data and the threshold can be improved, and the accuracy and the intuitiveness of the monitoring of the distributed system can be improved.

In one embodiment, the presentation effect information includes data presentation form information, data presentation standard information, and data presentation style information. As shown in fig. 4, the specific process of generating the target monitoring view of the target distributed system including the monitoring data, the target data type, the target system dimension, and the target system hierarchy based on the data mapping setting result, the threshold setting result, and the presentation effect information includes the following steps:

step 401, setting a data display form of the target monitoring view based on the data display form information, and obtaining a data display form setting result.

In the embodiment of the application, the terminal sets the data display form of the target monitoring view based on the data display form information, and a data display form setting result is obtained. The data display form information is used for representing a display form of the target monitoring view display monitoring data. The presentation form may include: statistics, time series, dashboards, tables, time status, stacks, historical status, text, alarm lists, file lists, news, notes, logs, and topology. The statistical graph comprises: bar, line and pie charts. The data presentation information may include one or more presentations.

Step 402, setting a data display standard of the target monitoring view based on the data display standard information, and obtaining a data display standard setting result.

In the embodiment of the application, the terminal sets the data display standard of the target monitoring view based on the data display standard information to obtain a data display standard setting result. The data display standard information is used for representing display standards of the monitoring data in the target monitoring view. The data presentation criteria information may include: units, minimums, decimal points, presentation aliases, data colors, and null presentation content. The null presentation may be "-", null, and null. Units include, but are not limited to, area, weight, number, data, humidity, angle, and storage.

And step 403, setting the data display style of the target monitoring view based on the data display style information, and obtaining a data display style setting result.

In the embodiment of the application, the terminal sets the data display style of the target monitoring view based on the data display style information, and a data display style setting result is obtained. The data display style information is used for representing the display style of the monitoring data in the target monitoring view.

In one example, in the case where the data presentation form information indicates that the presentation form is a statistical map of existence coordinate axes, the data presentation style information may include: and displaying style information and graphic display style information by the coordinate axes. The coordinate axis presentation style information may include: coordinate axis size, coordinate axis width, coordinate axis position, coordinate axis label and coordinate axis accuracy. The coordinate axis size includes: linear and logarithmic axes. The coordinate axis position is the coordinate axis placement position. The coordinate axis positions include: left, right and hidden. The coordinate axis precision may be 1, 10 and 100000. The graphic presentation style information may include: linear display style information, bar display style information, and dot display style information. The linear presentation style information may include: line type, line width, line fill, tilt angle, and null are shown. The bar presentation style information may include: data point location, line width, bar fill, and null values are shown. The dot-form presentation style information may include: the size of the dots.

Step 404, generating a target monitoring view of the target distributed system including the monitoring data, the target data type, the target system dimension, and the target system hierarchy based on the data mapping setting result, the threshold setting result, the data presentation form setting result, the data presentation standard setting result, and the data presentation style setting result.

In the embodiment of the application, the terminal generates the target monitoring view of the target distributed system including the monitoring data, the target data type, the target system dimension and the target system level based on the data mapping setting result, the threshold setting result, the data display form setting result, the data display standard setting result and the data display style setting result.

In one example, the presentation effect information further includes: data link information and panel information. The data link information includes: link title information and jump link information. The panel information includes: panel name and panel remarks. The panel name may be named for the function name, without format requirements, using the panel + number by default. The panel notes are used to further describe the panel content. The panel remarks are not forced to be displayed, and suspension display is supported, namely, when the mouse is placed at the name position of the panel, the panel remarks are suspended. In this way, when there are a plurality of panels, it is convenient to distinguish the respective panels. And the terminal sets the panel of the target monitoring view based on the panel information to obtain a panel setting result. And then, the terminal sets the data link of the target monitoring view based on the data link information to obtain a data link setting result. Then, the terminal generates a target monitoring view of the target distributed system including the monitoring data, as well as the target data type, the target system dimension, and the target system hierarchy, based on the data mapping setting result, the threshold setting result, the data presentation form setting result, the data presentation standard setting result, the data presentation style setting result, the panel setting result, and the data link setting result.

According to the monitoring method of the distributed system, the data display form, the data display standard and the data display style of the target monitoring view are set, and the target monitoring view of the target distributed system comprising the monitoring data, the target data type, the target system dimension and the target system level is generated based on the data mapping setting result, the threshold setting result, the data display form setting result, the data display standard setting result and the data display style setting result, so that the flexibility and the intuitiveness of monitoring of the distributed system can be improved.

In one embodiment, as shown in fig. 5, the monitoring method of the distributed system further includes the following steps:

step 501, obtaining monitoring data of a target distributed system, and determining a data type of the monitoring data according to a preset data classification rule.

In the embodiment of the application, the terminal acquires the monitoring data of the target distributed system. Then, the terminal determines the data type of the monitoring data according to a preset data classification rule.

In one example, the terminal defines and uniformly names the monitoring metadata of the target distributed system. And then, the terminal performs field expansion adaptation on the uniformly named monitoring metadata. And then, the terminal respectively performs various basic framework adaptations on the monitoring metadata subjected to field expansion adaptation.

Step 502, determining a target storage space of the monitoring data according to the data type of the monitoring data and a preset storage rule, and storing the monitoring data.

In the embodiment of the application, the terminal queries the target storage space of the monitoring data according to the data type of the monitoring data in the preset mapping relation between the data type and the storage space. Then, the terminal stores the monitoring data. The storage space is used for storing monitoring data of the distributed system. The storage space can be a database or a data table in the database.

In the monitoring method of the distributed system, the data type of the monitoring data is determined according to the preset data classification rule, and the target storage space of the monitoring data is determined according to the data type of the monitoring data. Therefore, the monitoring data of different data types are stored in a classified mode so as to support real-time customization of the target monitoring view, the efficiency of data query and convergence calculation is improved, the monitoring efficiency of the distributed system is improved, and the production problem solving time is shortened.

In one embodiment, as shown in fig. 6, the specific process of storing the monitoring data includes the following steps:

In step 601, in a preset mapping relationship between a data type and a gateway, according to the data type of the monitoring data, the gateway corresponding to the monitoring data is queried.

In the embodiment of the application, the terminal queries the gateway corresponding to the monitoring data according to the data type of the monitoring data in the preset mapping relation between the data type and the gateway. Wherein different data types correspond to different gateways.

Step 602, transmitting the monitoring data to the target storage space for storage processing through the gateway corresponding to the monitoring data.

In the embodiment of the application, the terminal transmits the monitoring data to the target storage space for storage processing through the gateway corresponding to the monitoring data.

In the monitoring method of the distributed system, in the preset mapping relation between the data type and the gateway, the gateway corresponding to the monitoring data is inquired according to the data type of the monitoring data, and the monitoring data is transmitted to the target storage space for storage processing through the gateway corresponding to the monitoring data. Therefore, the monitoring data of different data types are transmitted in a classified mode, transverse expansion and contraction can be conveniently carried out aiming at different classifications, mutual noninterference is achieved, abnormal data are reduced, and monitoring efficiency of a distributed system is improved.

In one embodiment, as shown in fig. 7, the specific process of transmitting the monitoring data to the target storage space for storage processing through the gateway corresponding to the monitoring data includes the following steps:

step 701, for each piece of monitoring data, performing message analysis on the monitoring data through a gateway corresponding to the monitoring data to obtain analysis data of the monitoring data.

In the embodiment of the application, for each piece of monitoring data, the gateway corresponding to the monitoring data obtains analysis data of the monitoring data by carrying out message analysis on the monitoring data according to a message format.

In one example, the message format of the monitoring data is json format. And the gateway corresponding to the monitoring data analyzes the actual content of the field according to the field corresponding to the json format to obtain the analysis data of the monitoring data.

In one example, if there are multiple fields in the fields corresponding to the message format, the gateway corresponding to the monitoring data completes parsing the multiple fields of the monitoring data one by one according to the multiple fields.

Step 702, checking the analysis data based on the message integrity policy to obtain a checking result of the monitoring data.

In the embodiment of the application, the gateway corresponding to the monitoring data checks the analysis data based on the message integrity policy to obtain a checking result of the monitoring data. The message integrity policy may include: message length rules and numerical type rules.

In step 703, if the verification result of the monitoring data indicates that the verification is passed, message filtering is performed on the parsed data, so as to obtain a filtering result of the monitoring data.

In the embodiment of the application, if the verification result of the monitoring data indicates that the verification is passed, the gateway corresponding to the monitoring data performs message filtering on the analysis data according to a preset list rule to obtain a filtering result of the monitoring data. The list rule is used for filtering monitoring data of a certain attribute. For example, the list rule may be that only the monitored data applied as a can pass filtering, and the remaining data is discarded.

If the check result of the monitoring data indicates that the check is not passed, the gateway corresponding to the monitoring data discards the monitoring data. Then, the gateway corresponding to the monitoring data records the operation of discarding the monitoring data in the log. In this way, it is convenient for the subsequent analysis why the verification is not passed.

And step 704, if the filtering result of the monitoring data indicates that the monitoring data is not filtered, performing message assembly on the analysis data of the monitoring data to obtain processed monitoring data, and transmitting the processed monitoring data to a target storage space for storage processing.

In the embodiment of the present application, if the filtering result of the monitoring data indicates that the monitoring data is not filtered, the gateway corresponding to the monitoring data performs message assembly on the analysis data of the monitoring data, so as to obtain the processed monitoring data. And then, the gateway corresponding to the monitoring data transmits the processed monitoring data to the target storage space for storage processing.

In one example, the message format of the monitoring data is json format. The gateway corresponding to the monitoring data reassembles the filtered analysis data of the monitoring data into the original json format to obtain the processed monitoring data.

In the monitoring method of the distributed system, in the data transmission process, the gateway corresponding to the monitoring data is used for sequentially carrying out message analysis, message verification, message filtration and message recombination on the monitoring data, and transmitting the processed monitoring data. Therefore, the monitoring data of different data types are processed in batches, useless and abnormal data are discarded, useless and abnormal data are further reduced, the situation that other monitoring data are influenced by reporting errors in the follow-up monitoring process is avoided, and the monitoring efficiency of the distributed system is further improved.

In one embodiment, as shown in fig. 8, the monitoring method of the distributed system further includes the following steps:

step 801, determining alarm levels meeting preset alarm conditions according to the order of the distributed system levels from low to high, and taking the highest distributed system level in the alarm levels as a monitoring level. And determining the alarm dimension meeting the preset alarm condition according to the sequence of the distributed system dimensions from low to high, and taking the highest distributed system dimension in the alarm dimension as the monitoring dimension.

In the embodiment of the application, the terminal determines the alarm level meeting the preset alarm condition according to the order of the distributed system level from low to high. The terminal then takes the highest distributed system level in the alarm levels as the monitoring level. Meanwhile, the terminal determines the alarm dimension meeting the preset alarm condition according to the sequence from low to high of the dimension of the distributed system. The terminal then takes the highest distributed system dimension of the alert dimensions as the monitoring dimension. The order of the distributed system hierarchy from low to high may be business, unit, campus, and city. The order of the distributed system dimensions from low to high may be stand-alone, node, service, group, and application. The monitoring hierarchy is a hierarchy that suggests the user to monitor. The monitoring dimension is the dimension that suggests the user to monitor.

Specifically, the terminal takes the lowest distributed system level as the current level. And then, the terminal calculates all the monitoring index values corresponding to all the current levels in the target distributed system in a converging way according to the current levels. Then, for each current level in the target distributed system, the terminal compares each monitoring index value corresponding to the current level with a threshold value corresponding to each monitoring index respectively to obtain a comparison result of the current level. And then, if the comparison result of the current level meets the preset alarm condition, taking the next level of the current level as the current level. And then, the terminal returns to the step of converging and calculating each monitoring index value corresponding to each current level in the target distributed system according to the current level. And when the comparison result of the current level does not meet the preset alarm condition, taking the previous level of the current level as an alarm level. It will be appreciated that the specific process of determining the alert dimension by the terminal is similar to the specific process of determining the alert level by the terminal described above. The threshold value corresponding to each monitoring index can be set by a user or can be determined by the terminal according to historical data.

In one example, the terminal inputs historical monitoring data of the target distributed system to a pre-trained machine learning model to obtain alternative thresholds corresponding to each monitoring index. And the terminal displays the alternative threshold value corresponding to each monitoring index to the user.

Step 802, generating alarm information based on the monitoring hierarchy and the monitoring dimension.

In the embodiment of the application, the terminal generates the alarm information based on the monitoring hierarchy and the monitoring dimension.

In one example, the terminal will monitor the hierarchy as a default target system hierarchy. Meanwhile, the terminal monitors the dimension as a default target system dimension. The default target system level is a default value of the target system level in the configuration request of the user. The default target system dimension is a default value of the target system dimension in the user's configuration request.

In the monitoring method of the distributed system, in the monitoring process of the distributed system, the total calculation is not carried out on all system dimensions and all system levels of the monitored data, but the monitoring level with the problem is calculated and determined step by step from the lowest system level, and the monitoring dimension with the problem is calculated and determined step by step from the lowest system dimension, so that useless calculation amount is avoided, and the accuracy and the efficiency of the positioning problem of the monitoring of the distributed system are improved.

It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides a monitoring device of the distributed system for realizing the monitoring method of the distributed system. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in the embodiments of the monitoring device of one or more distributed systems provided below may refer to the limitation of the monitoring method of the distributed system hereinabove, and will not be repeated herein.

In one embodiment, as shown in fig. 9, there is provided a monitoring apparatus 900 of a distributed system, including: a receiving module 910, a first querying module 920, a second querying module 930, and a constructing module 940, wherein:

a receiving module 910, configured to receive a configuration request of a target monitoring view; the configuration request comprises a target data type, a target system dimension, a target system level and display information;

the first query module 920 is configured to query a target database corresponding to the target data type in a mapping relationship between a preset data type and the database; the database is used for storing the monitoring data of the corresponding data type;

a second query module 930, configured to query, in each data table included in the target database, a target data table corresponding to the target system dimension and the target system level according to a preset mapping relationship between the distributed system dimension, the distributed system level, and the data table;

a building module 940, configured to configure data source information of the target monitoring view based on the target database and the target data table, and build a target monitoring view of a target distributed system based on the data source information and the display information; the target monitoring view is used for monitoring the distributed system.

Optionally, the building module 940 is specifically configured to:

Optionally, the display information includes data mapping information, threshold information and display effect information; the building module 940 is specifically configured to:

Optionally, the display effect information includes data display form information, data display standard information and data display style information; the building module 940 is specifically configured to:

Optionally, the apparatus 900 further includes:

Optionally, the storage processing module is specifically configured to:

Optionally, the apparatus 900 further includes:

The modules in the monitoring device of the distributed system may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a terminal, and an internal structure diagram thereof may be as shown in fig. 10. The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input means. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a method of monitoring a distributed system. The display unit of the computer device is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be a key, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the structure shown in fig. 10 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, implements the steps of the method embodiments described above.

In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

It should be noted that, the user information (including, but not limited to, user equipment information, user personal information, etc.) and the data (including, but not limited to, data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to comply with the related laws and regulations and standards of the related countries and regions.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.

Claims

1. A method of monitoring a distributed system, the method comprising:

2. The method of claim 1, wherein the constructing a target monitor view of a target distributed system based on the data source information and the presentation information comprises:

3. The method of claim 2, wherein the presentation information includes data mapping information, threshold information, and presentation effect information; the generating a target monitoring view of the target distributed system including the monitoring data, the target data type, the target system dimension, and the target system hierarchy based on the presentation information includes:

4. A method according to claim 3, wherein the presentation effect information comprises data presentation form information, data presentation standard information and data presentation style information; the generating a target monitoring view of the target distributed system including the monitoring data, the target data type, the target system dimension, and the target system hierarchy based on the data mapping setting result, the threshold setting result, and the presentation effect information includes:

5. The method according to claim 1, wherein the method further comprises:

6. The method of claim 5, wherein the storing the monitoring data comprises:

7. The method of claim 6, wherein the transmitting, by the gateway corresponding to the monitoring data, the monitoring data to the target storage space for storage processing includes:

8. The method according to claim 1, wherein the method further comprises:

9. A monitoring device for a distributed system, the device comprising:

10. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 8 when the computer program is executed.

11. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 8.

12. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the method of any one of claims 1 to 8.