CN112054913A - Data monitoring system and method - Google Patents
Data monitoring system and method Download PDFInfo
- Publication number
- CN112054913A CN112054913A CN201910486996.4A CN201910486996A CN112054913A CN 112054913 A CN112054913 A CN 112054913A CN 201910486996 A CN201910486996 A CN 201910486996A CN 112054913 A CN112054913 A CN 112054913A
- Authority
- CN
- China
- Prior art keywords
- alarm
- monitoring
- module
- variable
- basic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/16—Threshold monitoring
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Environmental & Geological Engineering (AREA)
- Testing And Monitoring For Control Systems (AREA)
- Alarm Systems (AREA)
Abstract
The invention discloses a data monitoring system and a method, wherein the system comprises: the basic alarm module is used for configuring monitoring rules in advance and used for acquiring data to be monitored and generating basic alarm items of the data to be monitored; the alarm index combination module is used for configuring monitoring tasks in advance, acquiring basic alarm items related to the monitoring tasks and combining the basic alarm items into alarm indexes; the alarm auditing module is used for acquiring the alarm indexes obtained by combination, judging whether the alarm indexes are alarm indexes needing to be notified or not, and if so, sending the acquired alarm indexes to the alarm notifying module; and the alarm notification module is used for sending the alarm indexes sent by the alarm auditing module to a corresponding alarm receiver. According to the technical scheme, data under various scenes can be flexibly monitored, and therefore the real-time performance of monitoring is improved.
Description
Technical Field
The invention relates to the technical field of internet, in particular to a data monitoring system and a data monitoring method.
Background
The monitoring service can make early warning and alarm for the event in time by analyzing and processing the data of the equipment to be monitored, thereby ensuring the normal operation of the equipment to be monitored and reporting errors in time.
Referring to fig. 1, in the prior art, different monitoring schemes may be respectively created for existing scenes in advance, and the schemes may acquire data to be monitored from a data source and analyze the acquired data, thereby generating alarm information.
However, in the existing data monitoring method, once a new monitoring scenario appears, it takes a considerable amount of time to design an independent monitoring scheme for the new monitoring scenario, which greatly affects the real-time performance of monitoring.
Disclosure of Invention
The purpose of the present application is to provide a data monitoring system and method, which can flexibly monitor data in various scenes, thereby improving the real-time performance of monitoring.
In order to achieve the above object, in one aspect, the present application provides a data monitoring system, which includes a basic alarm module, an alarm indicator combination module, an alarm auditing module, and an alarm notification module, wherein: the basic alarm module is configured with a monitoring rule in advance and used for acquiring data to be monitored, analyzing the monitoring data and generating a basic alarm item of the data to be monitored; the alarm index combination module is pre-configured with a monitoring task consisting of one or more monitoring rules and used for acquiring basic alarm items related to the monitoring task from one or more basic alarm modules and combining the acquired basic alarm items into alarm indexes according to the monitoring task; the alarm auditing module is used for acquiring alarm indexes obtained by combination from the alarm index combination module, judging whether the alarm indexes are alarm indexes needing to be notified or not, and if so, sending the acquired alarm indexes to the alarm notification module; and the alarm notification module is used for sending the alarm indexes sent by the alarm auditing module to the corresponding alarm receiving party.
In order to achieve the above object, another aspect of the present application further provides a data monitoring method, including: acquiring data to be monitored, reading monitoring variable values of monitoring variables corresponding to a monitoring rule configured in advance, and generating basic alarm items of the data to be monitored by comparing the data to be monitored with the read monitoring variable values; acquiring basic alarm items related to a pre-configured monitoring task, and combining the acquired basic alarm items into alarm indexes according to the monitoring task; and judging whether the alarm index is the alarm index needing to be notified, and if so, sending the alarm index to a corresponding alarm receiving party.
As can be seen from the above, the technical solution provided by the present application may include one or more basic alarm modules, and the basic alarm modules are respectively configured with monitoring rules. After a certain basic alarm module obtains the monitoring data, the monitoring data can be analyzed according to the monitoring rule of the basic alarm module, so that a basic alarm item is generated. Subsequently, when the monitoring task needs to be executed, the monitoring task may be composed of one or more of the monitoring rules, so that the alarm indicator combining module may respectively obtain corresponding basic alarm items from corresponding basic alarm modules according to the monitoring rules included in the monitoring task, and combine the basic alarm items into alarm indicators according to the monitoring task. After the alarm index is judged to be the alarm index needing to be notified by the alarm auditing module, the alarm index can be sent to a corresponding alarm receiving party by the alarm notifying module, so that the alarm process is completed.
From the above, the basic alarm modules may correspond to the monitoring rules one to one, and if a new monitoring scene is generated, only the new monitoring rules and the basic alarm modules need to be correspondingly added. The monitoring rule may be the smallest unit that constitutes a monitoring task, and if a new monitoring task occurs in a new scenario, only a plurality of monitoring rules need to be flexibly combined. Therefore, the technical scheme provided by the application can configure the monitoring rules and the monitoring tasks in the system according to the actual scene requirements, greatly improves the flexibility of the system, and can quickly adapt to the data monitoring of a new scene.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flow chart of a method of data monitoring in the prior art;
FIG. 2 is a schematic diagram of a data monitoring system according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of the internal flow of a basic alarm module in an embodiment of the present invention;
FIG. 4 is a schematic diagram of an internal flow of an alarm auditing module according to an embodiment of the present invention;
fig. 5 is a schematic step diagram of a data monitoring method according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
Referring to fig. 2, the data monitoring system may include a variable module, a basic alarm module, an alarm indicator combination module, an alarm auditing module, and an alarm notification module.
In this embodiment, the variable module may store monitoring variables, where the monitoring variables include device variables of the device to be monitored, variables of the system service quality, and other variables that need to be monitored. The monitoring variables may be pre-registered in the data monitoring system. In practical applications, the monitoring variables may include registrant identification, namespaces, variable groups, variable names, monitoring variable values, and value types. The registrant identifier may be a user name of a user who registers the monitoring variable in the data monitoring system. For example, the registrar identity may be user. A namespace can be a space in which monitoring variables reside, for example, a namespace can be represented in namespace. In practical applications, the name of the namespace may be defined by a user in the data monitoring system. For example, the common namespace may be common. And the variable groups can be used for dividing different types of monitoring variables, wherein the monitoring variables of the same type can be located in the same variable group. For example, the monitoring variable representing the memory usage may be located in the variable group of mem _ usage. For another example, the monitoring variable representing the CPU utilization may be located in the variable group of CPU _ usage. The variable name may be the name of the monitoring variable. In practical applications, the variable name may be represented by a name. In general, since the variable group defines the type of the monitoring variable, the variable name may be directly expressed as the device name of the device to be monitored. For example, for the device a to be monitored, the corresponding variable name may be a. Monitoring the variable values may generate thresholds for alarm information. For example, for memory usage, the monitoring variable value may be a threshold value of memory usage, and when the threshold value is exceeded, an alarm message may be triggered. The value type may indicate a value type used to monitor the variable value. In practical applications, the numeric type may be represented by type. The numerical value type may include int (integer), long (long integer), double (floating point), string (character), boolean, and the like, or a combination of these types.
In this embodiment, for the monitoring types of different devices to be monitored, the monitoring types can be registered in the same variable list. For example, currently there are 5 devices to be monitored A, B, C, D, E, and it is necessary to monitor the memory usage of these 5 devices, then the monitoring variables of the memory usage of these 5 devices can be defined as follows:
in the above example, the namespace is common. The variable group is a memory usage group mem _ usage. The numeric type is floating point type double. The monitoring variable is characterized by three parts: the first part is a default variable default.usage, the second part is a variable d.usage of the device D to be monitored, and the third part is a variable e.usage of the device E to be monitored. Use may represent devices to be monitored A, B and C, among others. The monitoring variable values correspond to the above three variables, and are respectively 0.85, 0.8, and 0.9, which indicate that the memory usage threshold of the devices to be monitored A, B and C is 85%, the memory usage threshold of the device to be monitored D is 80%, and the memory usage threshold of the device to be monitored E is 90%.
In this embodiment, if the monitoring variable of the device to be monitored needs to be added or deleted in the variable module, it is only necessary to add or delete the corresponding content in the variable list. In this way, the monitoring variables stored in the variable module can be defined by the above-mentioned name space, variable group, variable name and monitoring variable value. Subsequently, other modules in the data monitoring system can read the monitoring variable value of the specified monitoring variable from the variable module in a data query mode.
The data monitoring system can be provided with one or more basic alarm modules, and the basic alarm modules can acquire data to be monitored from a data center and generate corresponding basic alarm items after analyzing the acquired data to be monitored. The scene adopting the plurality of basic alarm modules is mainly used for responding to the scene of the multi-input condition through the plurality of basic alarm modules when one basic alarm module cannot meet the requirement of the monitoring scene, so that the alarm combination module can be reused. For example, each basic alarm module can only analyze the data to be monitored in one monitoring period. If the current monitoring scene only needs to analyze the monitoring data in one monitoring time period, only one basic alarm module is adopted. If the data of different monitoring time periods need to be analyzed currently, different basic alarm modules need to be adopted. At this time, the number of basic alarm modules is the same as the number of monitoring periods. For example, currently, the number of connections of a device to be monitored needs to be monitored, and a basic scheme of monitoring may be as follows:
scheme 1, average value less than 100 within 10 minutes of ligation
Protocol 2, average of ligation over 5 minutes greater than 130
Protocol 3 average of ligation number 3 min > 150
Then, these three basic solutions, since they cover three different monitoring periods (within 10 minutes, within 5 minutes and within 3 minutes), need to be monitored separately by three different basic alarm modules. In practical applications, different monitoring scenarios may need to combine the above three schemes. For example, in a certain monitoring scenario, it is necessary to simultaneously monitor that the average value over 10 minutes of the number of connections is less than 100 and the average value over 5 minutes of the number of connections is greater than 130, so it is necessary to combine scheme 1 and scheme 2. For another example, in another monitoring scenario, the average value of the connection number within 10 minutes needs to be monitored simultaneously to be less than 100, and the average value of the connection number within 3 minutes needs to be greater than 150, so that the scheme 1 and the scheme 3 need to be combined. Therefore, the monitoring scheme of the multi-input condition scene is divided into the scheme of a plurality of basic alarm modules, and the scheme of multiplexing the basic alarm modules can avoid repeatedly establishing the same scheme, thereby greatly reducing the overall overhead of the system.
In this embodiment, each basic alarm module may be preconfigured with a monitoring rule, and the monitoring rule may represent an object to be monitored currently and a trigger condition of a basic alarm item. For example, the three basic schemes described above may represent three different monitoring rules. Taking the scheme 1 as an example, the object required to be monitored currently is the number of connections, and if the average value of the number of connections in 10 minutes is less than 100, the basic alarm item is triggered. Of course, the threshold 100 for determining the number of connections in the scenario 1 may be stored in the variable block.
Therefore, the basic alarm module can acquire the data to be monitored in the monitoring time period from the data center, and simultaneously, the monitoring variable value corresponding to the monitoring rule can be read from the variable module. For example, for scenario 1 above, the monitoring variable value read from the variable module by the base alarm module is 100, while for scenarios 2 and 3, the monitoring variable value read from the variable module by the base alarm module is 130 and 150, respectively.
In this embodiment, when the basic alarm module reads the monitoring variable value from the variable module, it is usually necessary to generate a query statement having a fixed format and read the corresponding monitoring variable value through the query statement. Since the monitoring variables in the variable module are usually defined by a namespace, a variable group, a variable name, and a monitoring variable value, in order to read the monitoring variable value, the corresponding namespace, variable group, and variable name may be provided to the variable module. In practical application, the basic alarm module may generate, according to the monitoring rule, an inquiry statement for acquiring the monitoring variable value corresponding to the monitoring rule, where the inquiry statement includes a namespace, a variable group, and a variable name corresponding to the monitoring variable to be inquired. In one example application, the query statement may be represented in the following format:
${namespace.group:$name}
the name represents the name space of the monitored variable, the group represents the variable group to which the monitored variable belongs, and the name represents the name of the monitored variable.
Taking the above 5 devices to be monitored A, B, C, D, E as an example, the following query statements may be generated:
${common.mem_usage:${host}.usage|default(default.usage)}
mem _ use indicates that the query is currently performed for the memory usage rate in the public namespace, and { host } may fill any one of a-E according to an actual situation, indicating which memory usage rate threshold of the device to be monitored needs to be obtained currently. Since the variable list has a definition of default.use, the query statement may also be accompanied by a decision condition of default (default.use), which indicates that if a corresponding result cannot be queried in the variable module according to the { host }, then the query may be performed according to default.use. For example, if the content of { host } is A, B or C, in the variable module, since there are no a.usage, b.usage and c.usage, the variable module may feed back the monitoring variable value of default.usage to the base alarm module. That is, if the content of { host } is A, B or C, the read monitoring variable values are unified to 0.85; and if the content of { host } is D, the read monitoring variable value is 0.8; if the content of { host } is E, the value of the monitoring variable read is 0.9.
Therefore, in one embodiment, after the basic alarm module generates the query statement, the basic alarm module may read the monitoring variable value matched with the query statement from the variable module, and use the read monitoring variable value as the monitoring variable value corresponding to the monitoring rule. If the variable name carried in the query statement does not exist in the variable module (e.g., ABC described above), the basic alarm module may read the monitoring variable value of the default variable (default.use) from the variable module, and use the monitoring variable value of the default variable as the monitoring variable value corresponding to the monitoring rule.
In this embodiment, after the basic alarm module reads the monitoring variable value corresponding to the monitoring rule, the basic alarm module may compare the data to be monitored with the read monitoring variable value, so as to determine whether the current data to be monitored triggers the alarm condition in the monitoring rule. For example, for the device a to be monitored, the threshold of the memory usage rate is 0.85, and if the data to be monitored read by the basic alarm module shows that the current memory usage rate is 0.9, an alarm condition may be triggered. On the contrary, if the data to be monitored read by the basic alarm module shows that the current memory usage rate is 0.7, the alarm condition may not be triggered.
In this embodiment, the basic alarm module may generate different basic alarm items according to different determination results. For example, for data to be monitored which does not trigger an alarm condition, the generated basic alarm item may be null information, or the flag in the generated basic alarm item that indicates whether an alarm is false (indicating no alarm); for the data to be monitored that triggers the alarm condition, the generated basic alarm item may carry actual information of the alarm type, or the identifier that represents whether the alarm is true (represents the alarm) in the generated basic alarm item. In this way, each base alarm module may ultimately generate a respective base alarm item.
It should be noted that, in the same basic alarm module, a plurality of different monitoring rules may be set for the data to be monitored in the same monitoring time period. These monitoring rules may include rules for data filtering, mathematical operations, threshold comparisons, list comparisons, and the like. The data filtering may refer to filtering out data of the device to be monitored from the read data to be monitored, and filtering data of other non-monitoring devices. The mathematical operation may refer to mathematical operations such as summing, averaging, and the like. The threshold comparison may then be the comparison process with the monitored variable value described above. List comparison may refer to determining whether the data to be monitored is in a white list or a black list. Referring to fig. 3, after the data to be monitored is read from the data center, the data to be monitored may be sequentially processed according to the arrangement order of the monitoring rules, and finally, a basic alarm item is generated. Specifically, when the data to be monitored is sequentially processed, the output result processed by the previous monitoring rule may be used as the input data of the next monitoring rule. Therefore, the data to be monitored can be processed step by step through each monitoring rule arranged in sequence. For example, after data filtering is performed on data to be monitored, the data obtained by the data filtering is input into a monitoring rule of mathematical operation, so as to perform mathematical operations such as summing and averaging on the filtered data. Finally, the data output by the monitoring rule of the mathematical operation can be input into the monitoring rule of the threshold comparison, so that the data subjected to the mathematical operation is compared with the monitoring variable value, and finally, a basic alarm item is generated.
In this embodiment, the alarm indicator combining module may be configured with a monitoring task composed of one or more monitoring rules in advance. For example, the above-mentioned embodiments 1 to 3 are 3 basic monitoring rules, and in practical applications, some of the monitoring rules may be combined to form the final monitoring task. For example, a certain monitoring task is scheme 1& scheme 2, which represents a case where it is necessary to monitor that the average value over the number of connections 10 minutes is less than 100 and the average value over the number of connections 5 minutes is greater than 130. For another example, a certain monitoring task is scenario 1| scenario 2, which indicates a case where the average value of the number of connections in 10 minutes is required to be monitored to be less than 100 or the average value of the number of connections in 5 minutes is required to be monitored to be greater than 130.
Thus, since the monitoring task in the alarm indicator combining module may be composed of one or more monitoring rules, the alarm indicator combining module may be correspondingly connected to one or more basic alarm modules, so that the basic alarm items related to the monitoring task may be obtained from one or more basic alarm modules. Specifically, when the monitoring task of the alarm indicator combination module includes the monitoring rules of different monitoring time periods, the alarm indicator combination module may identify the basic alarm modules corresponding to the different monitoring time periods, and obtain the corresponding basic alarm items from the identified basic alarm modules. For example, the monitoring task is composed of a scheme 1 and a scheme 2, and the alarm indicator combination module can be connected with the two basic alarm modules of the scheme 1 and the scheme 2, so as to obtain two basic alarm items. Then, the alarm index combination module may combine the acquired basic alarm items into alarm indexes according to the monitoring task. Specifically, the monitoring task of the alarm indicator combination module may include one or more monitoring rules, and may also represent a composition manner of the one or more monitoring rules. For example, with respect to the above-described schemes 1& 2, it is indicated that the composition of the two schemes is in an and manner; and for scheme 1| scheme 2 above, it means that the constituent modes of the two schemes are an "or" mode. Therefore, after the alarm index combination module acquires each basic alarm item, the acquired basic alarm items can be combined into the alarm index according to the composition mode of the monitoring task representation. For example, the basic alarm item corresponding to the scheme 1 is alarm 1, the basic alarm item corresponding to the scheme 2 is alarm 2, the alarm indexes of the monitoring task schemes 1 and 2 are alarm 1& alarm 2, and the alarm index of the monitoring task scheme 1| scheme 2 is alarm 1| alarm 2.
The advantage of this is that the same monitoring rule can be reused by multiple different monitoring tasks, so that the same basic alarm item can also be reused by multiple alarm indicator combination modules. When the number of the monitoring tasks is large, the number of the multiplexed basic alarm items is relatively large. Therefore, the basic alarm item generated by one basic alarm module at one time can be simultaneously reused by a plurality of monitoring tasks, so that the utilization rate of the basic alarm item is greatly improved, and the load of the whole system is correspondingly reduced.
It should be noted that the alarm indexes obtained by combining the above manners may not trigger an alarm. The reason is that the basic alarm item generated by the basic alarm module may be null information or information with alarm identity false. Therefore, the alarm index needs to finally judge whether the alarm is triggered according to the composition logic of each basic alarm item. For example, in the monitoring task composed of the and relationship between the scheme 1 and the scheme 2, the final alarm index is represented as alarm 1& alarm 2. Then, in this case, the alarm indicator can trigger an alarm only if both alarm 1 and alarm 2 trigger an alarm condition. And for the monitoring task composed of the scheme 1 and the scheme 2 through the or relation, the final alarm index is represented as alarm 1| alarm 2. Then, in this case, the alarm indicator triggers an alarm whenever either of alarm 1 and alarm 2 triggers an alarm condition.
In this embodiment, after the alarm index combination module combines the alarm indexes to obtain the alarm index, the alarm index may be sent to the alarm auditing module. Of course, the alarm auditing module may also obtain the alarm indexes obtained by combination from the alarm index combination module according to the specified period. Referring to fig. 4, the alarm auditing module may determine whether the obtained alarm indicator is an alarm indicator that needs to be notified. The alarm auditing module can be configured with auditing rules in advance, and the auditing rules can be used for judging whether the currently acquired alarm indexes need to be notified or not. Specifically, the auditing rule may be, for example, that if the alarm indicator continuously appears for a specified number of times, or the alarm indicator appears for a preset number of times within a specified time period, the alarm auditing module determines that the alarm indicator is an alarm indicator that needs to be notified. For another example, if the device to be monitored to which the alarm indicator belongs is offline, the alarm indicator is determined to be an alarm indicator that does not need to be notified. The determination conditions that depend on the audit rule, such as the above-mentioned specified times, preset times, whether to go offline, and the like, may be stored in the variable module as the audit variable. In this way, the variable module can store monitoring variables and audit variables. For example, if the alarm indicator appears 3 times continuously, or the alarm indicator appears 3 times within 5 minutes, the alarm auditing module may determine that the alarm indicator is an alarm indicator that needs to be notified. The values of the variables can be used as the auditing variable values of the auditing variables after 3 times and 5 minutes, and are acquired from the variable module by the alarm auditing module. The manner of obtaining the monitoring variable value and the type of obtaining the monitoring variable value are not described herein again. Therefore, the variable module can also store the auditing variable of each device to be monitored, the auditing variable has the auditing variable value, and the alarm auditing module is pre-configured with the auditing rule. After the alarm checking module obtains the alarm indexes from the alarm index combination module, the checking variable value corresponding to the checking rule can be read from the variable module, and the obtained alarm indexes are compared with the read checking variable value to judge whether the obtained alarm indexes are the alarm indexes needing to be notified.
In one embodiment, for the child monitor and the parent monitor having a parent-child relationship, if the child monitor has an alarm indicator and the parent monitor also has an alarm indicator within a specified time, the alarm auditing module determines that the child monitor is an alarm indicator that does not need to be notified, and only uses the alarm indicator of the parent monitor as the alarm indicator that needs to be notified. In addition, aiming at the alarm indexes which have occurred currently, the alarm auditing module can judge that the alarm indexes which have occurred are alarm indexes which do not need to be notified within the specified time length. Thus, the same alarm index can be prevented from being repeatedly notified, and thus important alarm indexes are prevented from being submerged.
Specifically, referring to fig. 4, when comparing the obtained alarm indicator with the read audit variable value, the historical alarm indicator in the historical database needs to be used, so that the alarm audit module can read the audit variable value of the audit variable from the variable module on one hand, and can read the historical alarm indicator from the historical database on the other hand, and by combining the historical alarm indicator with the current alarm indicator and comparing the combined statistical result with the audit variable value, it can determine whether the current alarm indicator is the alarm indicator that needs to be notified.
Referring to FIG. 2, in one embodiment, the alarm indicators combined by the alarm indicator combination module are typically stored in a historical database. The history database may be a database with persistent storage. For example, the historical database may be a redis database, a mango database, or a rados database. In this embodiment, after the alarm indicator combination obtains the current alarm indicator, it may be determined whether the alarm indicator obtained by combination exists in the historical database, and if the alarm indicator exists in the historical database, the alarm indicator is generated before. If the current alarm index does not exist in the historical database, the current alarm index is not generated before, so that the alarm index obtained by combination can be sent to an alarm auditing module.
In addition, after the alarm checking module acquires the alarm indexes from the alarm index combination module, the acquired alarm indexes can be stored in the historical database, so that the unrepeated alarm indexes are backed up in the historical database.
In this embodiment, after receiving the alarm indicator sent by the alarm auditing module, the alarm notifying module may send the alarm indicator to the corresponding alarm receiver to notify the alarm receiver to process the abnormal condition corresponding to the alarm indicator.
The present application further provides a data monitoring method applied to the data monitoring system, please refer to fig. 5, the method includes:
s1: and acquiring data to be monitored, reading monitoring variable values of monitoring variables corresponding to the monitoring rules configured in advance, and generating basic alarm items of the data to be monitored by comparing the data to be monitored with the read monitoring variable values.
In this embodiment, the basic alarm module may be configured with a monitoring rule in advance, may obtain data to be monitored, reads a monitoring variable value corresponding to the monitoring rule from the variable module, and generates a basic alarm item of the data to be monitored by comparing the data to be monitored and the read monitoring variable value.
S3: and acquiring basic alarm items related to the pre-configured monitoring task, and combining the acquired basic alarm items into alarm indexes according to the monitoring task.
In this embodiment, the alarm indicator combining module may pre-configure a monitoring task composed of one or more monitoring rules, and may obtain basic alarm items related to the monitoring task from one or more basic alarm modules, and combine the obtained basic alarm items into the alarm indicator according to the monitoring task.
S5: and judging whether the alarm index is the alarm index needing to be notified, and if so, sending the alarm index to a corresponding alarm receiving party.
In this embodiment, the alarm auditing module may obtain the alarm index obtained by combination from the alarm index combination module, determine whether the alarm index is an alarm index that needs to be notified, and send the obtained alarm index to the alarm notification module if the alarm index is the alarm index that needs to be notified. The alarm notification module can send the alarm index sent by the alarm auditing module to the corresponding alarm receiver.
In one embodiment, the monitoring variables are defined by a namespace, a group of variables, a name of a variable, and monitoring variable values;
correspondingly, reading the monitoring variable value of the monitoring variable corresponding to the pre-configured monitoring rule includes:
generating an inquiry statement for acquiring a monitoring variable value corresponding to the monitoring rule according to the monitoring rule, wherein the inquiry statement comprises a name space, a variable group and a variable name corresponding to a monitoring variable to be inquired;
and reading the monitoring variable value matched with the query statement, and taking the read monitoring variable value as the monitoring variable value corresponding to the monitoring rule.
In one embodiment, each monitoring rule is provided with a respective monitoring period;
correspondingly, the method further comprises:
and when the monitoring tasks contain the monitoring rules in different monitoring periods, respectively obtaining the basic alarm items corresponding to the monitoring rules contained in the monitoring tasks.
In one embodiment, the determining whether the alarm indicator is an alarm indicator that needs to be notified includes:
if the alarm index continuously appears for a specified number of times or the alarm index appears for a preset number of times within a specified time period, judging the alarm index as the alarm index needing to be notified;
if the child monitoring with the parent-child relationship has an alarm index and the parent monitoring also has the alarm index within a specified time, the alarm auditing module judges the child monitoring as the alarm index which does not need to be notified; for example: and if the equipment to be monitored to which the alarm index belongs is offline, judging the alarm index as an alarm index which does not need to be notified.
Aiming at the alarm indexes which have occurred currently, the alarm auditing module judges that the alarm indexes which have occurred are alarm indexes which do not need to be notified within the specified time;
it can be seen from above that, the technical scheme that this application provided can deposit each and treat supervisory equipment's control variable through unified variable module, and these control variables all can possess the control variable value. In practical applications, the monitoring variable may be, for example, the number of connections, the memory usage rate, the CPU usage rate, and the like. The monitoring variable value may then be a threshold value at which alarm information is generated. The data monitoring system provided by the application can comprise one or more basic alarm modules, and the basic alarm modules are respectively configured with monitoring rules. After a certain basic alarm module obtains the monitoring data, the corresponding monitoring variable value can be obtained from the variable module according to the monitoring rule of the basic alarm module, the data to be monitored and the monitoring variable value are compared, and a basic alarm item can be generated according to the comparison result. Subsequently, when the monitoring task needs to be executed, the monitoring task may be composed of one or more of the monitoring rules, so that the alarm indicator combining module may respectively obtain corresponding basic alarm items from corresponding basic alarm modules according to the monitoring rules included in the monitoring task, and combine the basic alarm items into alarm indicators according to the monitoring task. After the alarm index is judged to be the alarm index needing to be notified by the alarm auditing module, the alarm index can be sent to a corresponding alarm receiving party by the alarm notifying module, so that the alarm process is completed.
Therefore, the monitoring schemes do not need to be created for each scene, the monitoring variables of each device to be monitored can be uniformly stored in the variable module, and if a new monitoring scene is generated, the corresponding monitoring variables only need to be added in the variable module. In addition, the basic alarm modules may correspond to the monitoring rules one to one, and if a new monitoring scene is generated, the new monitoring rules and the basic alarm modules are added correspondingly. The monitoring variable values required by the basic alarm module in the process of generating the basic alarm items can be uniformly read from the variable module. The monitoring rule may be the smallest unit that constitutes a monitoring task, and if a new monitoring task occurs in a new scenario, only a plurality of monitoring rules need to be flexibly combined. Therefore, the technical scheme provided by the application can configure the monitoring rules and the monitoring tasks in the system according to the actual scene requirements, and can uniformly manage different monitoring variables through the variable modules, so that the flexibility of the system is greatly improved, and the data monitoring of a new scene can be quickly adapted.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods of the various embodiments or some parts of the embodiments.
The present invention is not limited to the above preferred embodiments, and any modifications, equivalent replacements, improvements, etc. within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (15)
1. The utility model provides a data monitoring system, its characterized in that, the system includes basic warning module, warning index composite module, reports an emergency and asks for help or increased vigilance and examines module and report an emergency and ask for help or increased vigilance notice module, wherein:
the basic alarm module is configured with a monitoring rule in advance and used for acquiring data to be monitored, analyzing the monitoring data and generating a basic alarm item of the data to be monitored;
the alarm index combination module is configured with a monitoring task composed of one or more monitoring rules in advance, and is used for acquiring basic alarm items related to the monitoring task from one or more basic alarm modules and combining the acquired basic alarm items into alarm indexes according to the monitoring task;
the alarm auditing module is used for acquiring the alarm indexes obtained by combination from the alarm index combination module, judging whether the alarm indexes are alarm indexes needing to be notified or not, and if so, sending the acquired alarm indexes to the alarm notification module;
and the alarm notification module is used for sending the alarm indexes sent by the alarm checking module to a corresponding alarm receiver.
2. The system of claim 1, further comprising a variable module for storing a monitoring variable, the monitoring variable having a monitoring variable value; the monitoring variables stored in the variable module at least comprise variable groups, variable names and monitoring variable value definitions;
correspondingly, the basic alarm module is further configured to read a monitoring variable value corresponding to the monitoring rule from the variable module, and generate a basic alarm item of the data to be monitored by comparing the data to be monitored with the read monitoring variable value.
3. The system according to claim 2, wherein the basic alarm module is further configured to generate, according to the monitoring rule, an inquiry statement for obtaining a monitoring variable value corresponding to the monitoring rule, where the inquiry statement includes a variable group and a variable name corresponding to a monitoring variable to be inquired;
correspondingly, the basic alarm module reads the monitoring variable value matched with the query statement from the variable module, and takes the read monitoring variable value as the monitoring variable value corresponding to the monitoring rule.
4. The system according to claim 3, wherein if the monitoring variable value corresponding to the variable name carried in the query statement does not exist in the variable module, the basic alarm module reads the monitoring variable value of the default variable from the variable module, and uses the monitoring variable value of the default variable as the monitoring variable value corresponding to the monitoring rule.
5. The system according to claim 1, wherein when there are a plurality of said basic alarm modules, each of said basic alarm modules included in said system has its own monitoring period;
correspondingly, when the monitoring tasks of the alarm index combination module contain monitoring rules of different monitoring time periods, the alarm index combination module identifies the basic alarm modules corresponding to the different monitoring time periods respectively, and acquires corresponding basic alarm items from the identified basic alarm modules.
6. The system according to claim 1 or 5, wherein the monitoring task of the alarm indicator combination module is further configured to characterize a composition of one or more of the monitoring rules;
correspondingly, after the alarm index combination module acquires the corresponding basic alarm items from the identified basic alarm modules, the alarm index combination module combines the acquired basic alarm items into the alarm indexes according to the composition mode of the monitoring task representation.
7. The system according to claim 1, wherein the alarm indicator combination module, after combining to obtain an alarm indicator, is further configured to determine whether the combined alarm indicator exists in a historical database, and if the combined alarm indicator exists in the historical database, discard the combined alarm indicator; and if the alarm indexes do not exist in the historical database, sending the alarm indexes obtained by combination to the alarm auditing module.
8. The system according to claim 2, wherein the variable module further stores an audit variable of each device to be monitored, the audit variable has an audit variable value, and the alarm audit module is configured with an audit rule in advance;
correspondingly, after the alarm checking module acquires the alarm index from the alarm index combination module, the checking variable value corresponding to the checking rule is read from the variable module, and the acquired alarm index is compared with the read checking variable value to judge whether the acquired alarm index is the alarm index needing to be notified.
9. The system according to claim 8, wherein if the alarm indicator occurs continuously for a specified number of times or the alarm indicator occurs for a preset number of times within a specified time period, the alarm auditing module determines that the alarm indicator is an alarm indicator that needs to be notified;
if the child monitoring with the parent-child relationship has an alarm index and the parent monitoring also has the alarm index within a specified time, the alarm auditing module judges that the child monitoring is the alarm index which does not need to be notified;
aiming at the alarm indexes which have occurred currently, the alarm auditing module judges that the alarm indexes which have occurred are alarm indexes which do not need to be notified within the specified time length.
10. The system according to claim 1, wherein the alarm auditing module is further configured to store the acquired alarm indicators in a historical database.
11. The system according to claim 1, wherein a plurality of monitoring rules are provided in the basic alarm module, and the basic alarm module is further configured to sequentially process the data to be monitored according to an arrangement sequence of the monitoring rules, so as to generate a basic alarm item for the data to be monitored.
12. A data monitoring method applied to the data monitoring system according to any one of claims 1 to 11, the method comprising:
acquiring data to be monitored, reading monitoring variable values of monitoring variables corresponding to a monitoring rule configured in advance, and generating basic alarm items of the data to be monitored by comparing the data to be monitored with the read monitoring variable values;
acquiring basic alarm items related to a pre-configured monitoring task, and combining the acquired basic alarm items into alarm indexes according to the monitoring task;
and judging whether the alarm index is the alarm index needing to be notified, and if so, sending the alarm index to a corresponding alarm receiver.
13. The method of claim 12, wherein the monitoring variables are defined by a namespace, a group of variables, a name of a variable, and monitoring variable values;
correspondingly, reading the monitoring variable value of the monitoring variable corresponding to the pre-configured monitoring rule includes:
generating an inquiry statement for acquiring a monitoring variable value corresponding to the monitoring rule according to the monitoring rule, wherein the inquiry statement comprises a name space, a variable group and a variable name corresponding to a monitoring variable to be inquired;
and reading the monitoring variable value matched with the query statement, and taking the read monitoring variable value as the monitoring variable value corresponding to the monitoring rule.
14. The method according to claim 12, wherein each of the monitoring rules is provided with a respective monitoring period;
accordingly, the method further comprises:
and when the monitoring tasks contain the monitoring rules in different monitoring time periods, respectively obtaining the basic alarm items corresponding to the monitoring rules contained in the monitoring tasks.
15. The method of claim 12, wherein determining whether the alarm indicator is an alarm indicator that needs to be notified comprises:
if the alarm index continuously appears for a specified number of times or the alarm index appears for a preset number of times within a specified time period, judging the alarm index as the alarm index needing to be notified;
if the child monitoring with the parent-child relationship has an alarm index and the parent monitoring also has the alarm index within a specified time, the alarm auditing module judges that the child monitoring is the alarm index which does not need to be notified;
aiming at the current alarm indexes, the alarm auditing module judges that the alarm indexes which occur are alarm indexes which do not need to be notified within a specified time length;
and if the equipment to be monitored to which the alarm index belongs is offline, judging the alarm index as an alarm index which does not need to be notified.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910486996.4A CN112054913B (en) | 2019-06-05 | 2019-06-05 | Data monitoring system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910486996.4A CN112054913B (en) | 2019-06-05 | 2019-06-05 | Data monitoring system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112054913A true CN112054913A (en) | 2020-12-08 |
| CN112054913B CN112054913B (en) | 2023-07-18 |
Family
ID=73608869
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910486996.4A Active CN112054913B (en) | 2019-06-05 | 2019-06-05 | Data monitoring system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112054913B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114691455A (en) * | 2020-12-31 | 2022-07-01 | 科来网络技术股份有限公司 | Method and device for monitoring alarm data and dynamically pushing multiple platforms |
| CN115017023A (en) * | 2022-05-30 | 2022-09-06 | 北京高阳捷迅信息技术有限公司 | Embedded data index monitoring and alarming method and system based on data warehouse |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102034148A (en) * | 2010-12-08 | 2011-04-27 | 山东浪潮齐鲁软件产业股份有限公司 | Method for realizing event alert and storm-proof policy of monitoring system |
| WO2016029570A1 (en) * | 2014-08-28 | 2016-03-03 | 北京科东电力控制系统有限责任公司 | Intelligent alert analysis method for power grid scheduling |
| CN108206747A (en) * | 2016-12-16 | 2018-06-26 | 中国移动通信集团山西有限公司 | Method for generating alarm and system |
| CN108270618A (en) * | 2017-12-30 | 2018-07-10 | 杭州华为数字技术有限公司 | Alert the method, apparatus and warning system of judgement |
| CN109726072A (en) * | 2018-07-18 | 2019-05-07 | 平安科技（深圳）有限公司 | Monitoring alarm method, apparatus, system and the computer storage medium of weblogic server |
-
2019
- 2019-06-05 CN CN201910486996.4A patent/CN112054913B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102034148A (en) * | 2010-12-08 | 2011-04-27 | 山东浪潮齐鲁软件产业股份有限公司 | Method for realizing event alert and storm-proof policy of monitoring system |
| WO2016029570A1 (en) * | 2014-08-28 | 2016-03-03 | 北京科东电力控制系统有限责任公司 | Intelligent alert analysis method for power grid scheduling |
| CN108206747A (en) * | 2016-12-16 | 2018-06-26 | 中国移动通信集团山西有限公司 | Method for generating alarm and system |
| CN108270618A (en) * | 2017-12-30 | 2018-07-10 | 杭州华为数字技术有限公司 | Alert the method, apparatus and warning system of judgement |
| CN109726072A (en) * | 2018-07-18 | 2019-05-07 | 平安科技（深圳）有限公司 | Monitoring alarm method, apparatus, system and the computer storage medium of weblogic server |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114691455A (en) * | 2020-12-31 | 2022-07-01 | 科来网络技术股份有限公司 | Method and device for monitoring alarm data and dynamically pushing multiple platforms |
| CN115017023A (en) * | 2022-05-30 | 2022-09-06 | 北京高阳捷迅信息技术有限公司 | Embedded data index monitoring and alarming method and system based on data warehouse |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112054913B (en) | 2023-07-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107566163B (en) | Alarm method and device for user behavior analysis association | |
| CN113176978B (en) | Monitoring method, system, equipment and readable storage medium based on log file | |
| CN110851321B (en) | Service alarm method, equipment and storage medium | |
| CN110661659A (en) | Alarm method, device and system and electronic equipment | |
| EP2800024B1 (en) | System and methods for identifying applications in mobile networks | |
| CN111934920B (en) | Monitoring alarm method, device, equipment and storage medium | |
| CN112395156A (en) | Fault warning method and device, storage medium and electronic equipment | |
| CN111444067A (en) | Rule engine-based configuration system monitoring method, device and equipment | |
| CN112054913A (en) | Data monitoring system and method | |
| CN111752811A (en) | Abnormal alarm information processing method, electronic device and storage medium | |
| CN114036022A (en) | Monitoring alarm processing method, device, equipment and medium | |
| CN111756745B (en) | Alarm method, alarm device, terminal equipment and computer readable storage medium | |
| CN114553682B (en) | Real-time alarm method, system, computer equipment and storage medium | |
| CN114070711A (en) | Alarm information processing method and device, electronic equipment and storage medium | |
| CN114172785A (en) | Alarm information processing method, device, equipment and storage medium | |
| CN110677271B (en) | Big data alarm method, device, equipment and storage medium based on ELK | |
| CN116737765A (en) | Service alarm information processing method and device, electronic equipment and storage medium | |
| CN110737565B (en) | Data monitoring method and device, electronic equipment and storage medium | |
| CN105100216A (en) | Multi-service monitoring method and device | |
| CN113806045A (en) | Task allocation method, system, device and medium | |
| CN117271177A (en) | Root cause positioning method and device based on link data, electronic equipment and storage medium | |
| CN109753399B (en) | Automatic notification and response method for system unit state change | |
| CN113901153B (en) | Data processing method and related equipment | |
| CN115391141A (en) | Database flow analysis method, device, equipment and readable storage medium | |
| CN116416764A (en) | Alarm threshold generation method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |