CN111475370A - Operation and maintenance monitoring method, device and equipment based on data center and storage medium - Google Patents

Operation and maintenance monitoring method, device and equipment based on data center and storage medium Download PDF

Info

Publication number
CN111475370A
CN111475370A CN202010153280.5A CN202010153280A CN111475370A CN 111475370 A CN111475370 A CN 111475370A CN 202010153280 A CN202010153280 A CN 202010153280A CN 111475370 A CN111475370 A CN 111475370A
Authority
CN
China
Prior art keywords
monitoring
monitoring data
data
target
source
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010153280.5A
Other languages
Chinese (zh)
Inventor
朱仁宇
董超
许俊威
黄伟星
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Priority to CN202010153280.5A priority Critical patent/CN111475370A/en
Priority to PCT/CN2020/093315 priority patent/WO2021174694A1/en
Publication of CN111475370A publication Critical patent/CN111475370A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3003Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/3006Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3065Monitoring arrangements determined by the means or processing involved in reporting the monitored data
    • G06F11/3072Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/32Monitoring with visual or acoustical indication of the functioning of the machine
    • G06F11/324Display of status information
    • G06F11/327Alarm or error message display

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Physics (AREA)
  • Debugging And Monitoring (AREA)
  • Alarm Systems (AREA)

Abstract

The invention discloses an operation and maintenance monitoring method, device and equipment based on a data center and a storage medium. The method comprises the following steps: acquiring original monitoring data acquired by a related monitoring system, wherein each original monitoring data comprises an original alarm level; standardizing the original alarm level, acquiring a standard alarm level and determining to-be-processed monitoring data; performing source detection on monitoring data to be processed to obtain a target source; the method comprises the steps of carrying out validity detection on monitoring data to be processed, obtaining a validity detection result, and determining the monitoring data to be processed with the validity detection result being effective alarm as effective monitoring data; formatting the effective monitoring data and a target source corresponding to the effective monitoring data to obtain target monitoring data; and automatically monitoring the target monitoring data based on the target source to obtain an alarm monitoring result. The method can improve the efficiency of monitoring the target monitoring data acquired by the associated monitoring system by the data center.

Description

Operation and maintenance monitoring method, device and equipment based on data center and storage medium
Technical Field
The invention relates to the technical field of data monitoring, in particular to an operation and maintenance monitoring method, device and equipment based on a data center and a storage medium.
Background
Monitoring is an important link of operation and maintenance of the current service system, and operation and maintenance personnel find and position fault events occurring in the service system through monitoring so as to maintain and update the service system according to the fault events. Generally, an enterprise (especially a group enterprise) will configure a plurality of service systems, and operation and maintenance personnel respectively monitor the plurality of service systems by using different monitoring systems and respectively feed back monitoring results to a monitoring management center of the enterprise.
Disclosure of Invention
The embodiment of the invention provides an operation and maintenance monitoring method, device, equipment and storage medium based on a data center, and aims to solve the problems that monitoring results acquired by different monitoring systems are not compatible, so that effective monitoring and analysis of the monitoring results acquired by a plurality of monitoring systems cannot be realized, and the monitoring and analysis efficiency is low.
An operation and maintenance monitoring method based on a data center comprises the following steps:
acquiring original monitoring data corresponding to a service system acquired by a correlation monitoring system, wherein each original monitoring data comprises an original alarm level;
standardizing the original alarm level to obtain a standard alarm level, and determining original monitoring data with the standard alarm level as a target alarm level as to-be-processed monitoring data;
performing source detection on the monitoring data to be processed to obtain a target source corresponding to the monitoring data to be processed;
carrying out validity detection on the monitoring data to be processed to obtain a validity detection result, and determining the monitoring data to be processed with the validity detection result being effective for warning as effective monitoring data;
formatting the effective monitoring data and a target source corresponding to the effective monitoring data to obtain target monitoring data;
and automatically monitoring the target monitoring data based on the target source to obtain an alarm monitoring result.
An operation and maintenance monitoring device based on a data center comprises:
the system comprises an original monitoring data acquisition module, a data processing module and a data processing module, wherein the original monitoring data acquisition module is used for acquiring original monitoring data corresponding to a service system acquired by a related monitoring system, and each original monitoring data comprises an original alarm level;
the monitoring data to be processed acquisition module is used for carrying out standardization processing on the original alarm level, acquiring a standard alarm level and determining the original monitoring data of which the standard alarm level is a target alarm level as the monitoring data to be processed;
the target source acquisition module is used for carrying out source detection on the monitoring data to be processed and acquiring a target source corresponding to the monitoring data to be processed;
the effective monitoring data acquisition module is used for carrying out effectiveness detection on the monitoring data to be processed, acquiring an effectiveness detection result and determining the monitoring data to be processed with the effectiveness detection result as effective monitoring data;
the target monitoring data acquisition module is used for carrying out formatting processing based on the effective monitoring data and a target source corresponding to the effective monitoring data to acquire target monitoring data;
and the alarm monitoring result acquisition module is used for automatically monitoring the target monitoring data based on the target source and acquiring an alarm monitoring result.
A computer device comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to realize the operation and maintenance monitoring method based on the data center.
A computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the method for operation and maintenance monitoring based on a data center is implemented.
According to the operation and maintenance monitoring method, the operation and maintenance monitoring device, the operation and maintenance monitoring equipment and the storage medium based on the data center, the data center is networked with the associated monitoring system, so that the data center can acquire the original monitoring data corresponding to the service system acquired by the data center from the associated monitoring system, and unified monitoring and multi-level monitoring of all the original monitoring data acquired by the associated monitoring system are realized; the method comprises the steps of standardizing the original alarm level of original monitoring data to determine a standard alarm level, screening out monitoring data to be processed according to the standard alarm level, and ensuring the pertinence of screening of the monitoring data to be processed; performing source detection on the monitoring data to be processed to determine a target source so as to ensure the pertinence of subsequent automatic monitoring; screening effective monitoring data based on an effectiveness detection result of effectiveness detection on the monitoring data to be processed so as to ensure the pertinence and timeliness of subsequent automatic monitoring; formatting processing is carried out based on the effective monitoring data and the corresponding target source thereof so as to obtain target monitoring data with uniform format and guarantee the feasibility of the subsequent automatic monitoring process; and finally, automatically monitoring the formatted target monitoring data corresponding to any target source to obtain an alarm monitoring result output by the automatic monitoring program, so that the efficiency of monitoring the target monitoring data acquired by at least one associated monitoring system by the data center is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
FIG. 1 is a schematic diagram of an application environment of a data center-based operation and maintenance monitoring method according to an embodiment of the present invention;
FIG. 2 is a flow chart of a data center-based operation and maintenance monitoring method according to an embodiment of the present invention;
FIG. 3 is another flow chart of a data center-based operation and maintenance monitoring method according to an embodiment of the present invention;
FIG. 4 is another flow chart of a data center-based operation and maintenance monitoring method according to an embodiment of the present invention;
FIG. 5 is another flow chart of a data center-based operation and maintenance monitoring method according to an embodiment of the present invention;
FIG. 6 is another flow chart of a data center-based operation and maintenance monitoring method according to an embodiment of the present invention;
FIG. 7 is another flow chart of a data center-based operation and maintenance monitoring method according to an embodiment of the present invention;
FIG. 8 is another flow chart of a data center-based operation and maintenance monitoring method according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of a data center-based operation and maintenance monitoring device according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of a computer device according to an embodiment of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The operation and maintenance monitoring method based on the data center provided by the embodiment of the invention can be applied to the application environment shown in fig. 1. Specifically, the operation and maintenance monitoring method based on the data center is applied to an operation and maintenance monitoring system based on the data center, the operation and maintenance monitoring system based on the data center comprises the data center shown in fig. 1 and at least one associated monitoring system which is in communication connection with the data center, each associated monitoring system is connected with at least one service system, so that the associated monitoring system can collect original monitoring data corresponding to the service system connected with the associated monitoring system, the data center can obtain all the original monitoring data from the associated monitoring system and analyze and process all the original monitoring data to obtain an alarm monitoring result, so that the original monitoring data collected by all the associated monitoring systems are automatically monitored and analyzed and processed, real-time effective monitoring on a plurality of service systems is realized, and monitoring and analyzing efficiency is improved. The service system is a system that needs to be monitored and can implement a specific service, and the service system is a monitored object, and specifically may be a system of an application program or an application product. The related monitoring system is connected with the service system and is used for monitoring the service system so as to realize functions of judging whether a fault event exists, positioning faults and the like. The data center is a processing center which is communicated with at least one associated monitoring system and is used for realizing automatic monitoring and analysis processing of the original monitoring data acquired by all the associated monitoring systems and realizing real-time effective monitoring and analysis of a plurality of service systems.
The service system, the association monitoring system and the data center in this embodiment all include a server and a client communicating with the server through a network. The client is also called a client, and refers to a program corresponding to the server and providing local services for the client. The client may be installed on, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server may be implemented as a stand-alone server or as a server cluster consisting of a plurality of servers.
In an embodiment, as shown in fig. 2, a method for operation and maintenance monitoring based on a data center is provided, which is described by taking the method as an example of being applied to a server of the data center in fig. 1, and includes the following steps:
s201: and acquiring original monitoring data corresponding to the service system acquired by the associated monitoring system, wherein each original monitoring data comprises an original alarm level.
The original monitoring data is unprocessed data formed by the associated monitoring system in the operation process of the monitoring service system, and specifically is related data formed by the associated monitoring system in the operation process of the monitoring service system by determining that a fault event occurs, including but not limited to data content, event occurrence time, monitoring log, original alarm level and other information.
As an example, the monitoring service system can adopt alarm levels of Blocker, critic, Major, Minor, Info and the like and corresponding alarm level judgment standards with the preset severity weakened in sequence, or adopt alarm levels of L1, L2, L3, L4, L5 and the like and corresponding alarm level judgment standards, the correlation monitoring system monitors the running condition formed in the running process of the service system in real time, and if the running condition meets the corresponding alarm level judgment standards, the alarm level corresponding to the alarm level judgment standards is determined as the original alarm level.
In this example, the data center may send a data acquisition instruction to at least one associated monitoring system networked therewith in real time or at regular time, and receive original monitoring data acquired by each associated monitoring system to the at least one service system connected thereto based on the data acquisition instruction, so as to automatically monitor the original monitoring data acquired by all the associated monitoring systems in the data center, thereby implementing real-time effective monitoring on a plurality of service systems, improving monitoring efficiency, and saving monitoring cost.
S202: and standardizing the original alarm level to obtain a standard alarm level, and determining the original monitoring data with the standard alarm level as a target alarm level as to-be-processed monitoring data.
The step of standardizing the original alarm level refers to converting the original alarm level determined by adopting different alarm level judgment standards, which is acquired by all the associated monitoring systems, into the alarm level corresponding to the unified alarm level judgment standard. Accordingly, the standard alarm level is an alarm level determined after the original alarm level is standardized. The target alarm level refers to a preset alarm level which needs to be monitored. For example, the alarm level with a higher failure degree may be set as a target alarm level, so as to perform multi-level monitoring on the failure event corresponding to the target alarm level through the data center, so as to ensure monitoring efficiency.
As an example, after acquiring original monitoring data acquired by at least one associated monitoring system, a data center standardizes original alarm levels corresponding to all the original monitoring data to determine standard alarm levels corresponding to the original alarm levels; respectively judging whether the standard alarm level corresponding to each original monitoring data is a target alarm level; and if the standard alarm level is the target alarm level, determining the original monitoring data corresponding to the standard alarm level as the monitoring data to be processed so as to carry out subsequent automatic monitoring, thereby ensuring that the original monitoring data corresponding to the target alarm level is subjected to multi-level real-time monitoring and improving the monitoring pertinence.
S203: and performing source detection on the monitoring data to be processed to obtain a target source corresponding to the monitoring data to be processed.
The target source is to perform source detection on the monitoring data to be processed to determine a source corresponding to the monitoring data to be processed, specifically, a source of a professional company or a business team in the application business system, for example, a professional company a or an operation and maintenance responsibility group B.
As an example, after determining the monitoring data to be processed, the data center performs source detection on each monitoring data to be processed, so as to determine a target source corresponding to the monitoring data from the data content of the monitoring data to be processed, so that the monitoring analysis dimension of the subsequent target source performs data automatic monitoring analysis, thereby ensuring the effectiveness of uniformly monitoring the monitoring data corresponding to a certain target source in the service system, and facilitating real-time and effective tracking of a fault event.
S204: and carrying out validity detection on the monitoring data to be processed, acquiring a validity detection result, and determining the monitoring data to be processed with the validity detection result being effective alarm as effective monitoring data.
The validity detection of the monitoring data to be processed is a process for detecting whether a fault event corresponding to the monitoring data to be processed is valid or not in real time, so that a validity detection result is determined. The effectiveness detection result is a result determined after the effectiveness detection is carried out on the monitoring data to be processed. In this example, the validity detection result includes two cases, namely, an alarm valid case and an alarm invalid case.
As an example, the data center performs validity detection on the monitoring data to be processed according to a preset validity detection script to determine whether a validity detection result corresponding to the monitoring data to be processed is an alarm valid or an alarm invalid; and screening out the monitoring data to be processed with effective alarm to determine the monitoring data to be processed as effective monitoring data, so that the effective monitoring data can be automatically monitored and processed subsequently, and the pertinence and the timeliness of monitoring and processing are ensured.
S205: and formatting the effective monitoring data and the target source corresponding to the effective monitoring data to obtain the target monitoring data.
As an example, after obtaining the valid monitoring data, the data center performs formatting processing on the valid monitoring data by using the determined target source, so as to perform formatting processing on the valid monitoring data uploaded by all associated monitoring systems, that is, format conversion is performed on incompatible portions of the valid monitoring data uploaded by different associated monitoring systems, and the incompatible portions of the valid monitoring data are converted into data with a uniform format, which can be identified by a subsequent automatic monitoring program, so as to ensure feasibility of subsequent automatic monitoring processing, thereby contributing to improvement of monitoring efficiency.
S206: and automatically monitoring the target monitoring data based on the target source to obtain an alarm monitoring result.
As an example, the data center is provided with an automatic monitoring program in advance, and can perform automatic monitoring on formatted target monitoring data corresponding to any target source by executing the automatic monitoring program to obtain an alarm monitoring result output by the automatic monitoring program, so that the efficiency of the data center in monitoring the target monitoring data acquired by at least one associated monitoring system is improved, and the feasibility of an automatic monitoring process can be ensured because the target monitoring data is subjected to format conversion.
In the operation and maintenance monitoring method based on the data center provided by this embodiment, the data center is networked with the associated monitoring system, so that the data center can acquire the original monitoring data corresponding to the service system acquired by the data center from the associated monitoring system, and realize unified monitoring and multi-level monitoring on all the original monitoring data acquired by the associated monitoring system; the method comprises the steps of standardizing the original alarm level of original monitoring data to determine a standard alarm level, screening out monitoring data to be processed according to the standard alarm level, and ensuring the pertinence of screening of the monitoring data to be processed; performing source detection on the monitoring data to be processed to determine a target source so as to ensure the pertinence of subsequent automatic monitoring; screening effective monitoring data based on an effectiveness detection result of effectiveness detection on the monitoring data to be processed so as to ensure the pertinence and timeliness of subsequent automatic monitoring; formatting processing is carried out based on the effective monitoring data and the corresponding target source thereof so as to obtain target monitoring data with uniform format and guarantee the feasibility of the subsequent automatic monitoring process; and finally, automatically monitoring the formatted target monitoring data corresponding to any target source to obtain an alarm monitoring result output by the automatic monitoring program, so that the efficiency of monitoring the target monitoring data acquired by at least one associated monitoring system by the data center is improved.
As an example, after step S206, the operation and maintenance monitoring method based on the data center further includes the following steps: and executing a target monitoring reminding mechanism corresponding to the alarm monitoring result based on the alarm monitoring result.
In this example, a plurality of original monitoring reminding mechanisms are preset in the data center, and each original monitoring reminding mechanism corresponds to a reminding condition. The original monitoring reminding mechanism is a preset mechanism for reminding processing, and for example, the original monitoring reminding mechanism can be set as a processing mechanism for carrying out telephone reminding, mail reminding or other reminding on operation and maintenance personnel.
Specifically, after acquiring an alarm monitoring result corresponding to any target source, the data center judges which reminding condition the alarm monitoring result satisfies, determines an original monitoring reminding mechanism corresponding to the reminding condition that the alarm monitoring result satisfies as the target reminding mechanism, executes the target monitoring reminding mechanism corresponding to the alarm monitoring result, and sends reminding information to corresponding operation and maintenance personnel, so that timely response processing is performed on all target monitoring data corresponding to the target source, and response processing efficiency is improved.
As an example, after step S206, the operation and maintenance monitoring method based on the data center further includes the following steps: and displaying the alarm monitoring result corresponding to the target monitoring data according to a preset display interface. For example, an alarm monitoring result corresponding to the target monitoring data is displayed through a Web page; or displaying an alarm monitoring result corresponding to the target monitoring data through a regular report; or displaying the alarm monitoring result corresponding to the target monitoring data through an external interface. Accordingly, processing interfaces such as page query, periodic reports and data interfaces can be provided in the client of the data center, so that users can conveniently perform processing such as temporary query requirements, periodic inspection and inspection through the client.
In an embodiment, as shown in fig. 3, step S201, namely acquiring original monitoring data corresponding to a service system collected by an associated monitoring system, specifically includes the following steps:
s301: and monitoring the number of the monitoring systems corresponding to the associated monitoring systems in real time.
Wherein the number of monitoring systems is the number of associated monitoring systems networked with the data center. As an example, the data center may broadcast http requests to all associated monitoring systems, count http responses received within a preset time period and corresponding to the http requests, and determine the number of monitoring systems corresponding to the associated monitoring systems according to the number of the received http responses.
S302: and establishing data acquisition processes corresponding to the number of the monitoring systems, wherein each data acquisition process corresponds to one associated monitoring system.
The data acquisition process is a process which is created on a server corresponding to the data center and is used for acquiring data. In this example, each data acquisition process corresponds to one associated monitoring system, so that the data acquisition process is dedicated to acquiring original monitoring data corresponding to the corresponding associated monitoring system, and the pertinence of data acquisition is ensured. For example, the data collection process may be associated with a network address corresponding to the associated monitoring system, so that the data collection process may communicate with the associated monitoring system according to the network address, thereby obtaining the original monitoring data sent by the associated monitoring system.
S303: and executing all data acquisition processes in parallel, and acquiring original monitoring data corresponding to the service system acquired by each associated monitoring system in the current acquisition period.
The current acquisition period refers to a time period of the data acquisition, specifically, a time period from the acquisition time of the last data acquisition to the current time of a system in which a data acquisition instruction is formed in the data center.
As an example, the data center executes all data acquisition processes in parallel, so that each data acquisition process sends a data acquisition instruction to a corresponding associated monitoring system, and the data acquisition instruction carries a current acquisition cycle; after each associated monitoring system receives the data acquisition instruction, all original monitoring data of which the event occurrence time is in the data acquisition period are sent to the data center, so that the data center can acquire the original monitoring data corresponding to the service system acquired by each associated monitoring system in the current acquisition period, the continuity of the acquired original monitoring data in time is ensured, and all data acquisition processes are executed in parallel, thereby simultaneously acquiring the original monitoring data formed in the operation process of monitoring the corresponding service system by a plurality of associated monitoring systems, ensuring the acquisition efficiency and timeliness of the original monitoring data, and avoiding the problem that the data center cannot monitor and process the fault event in time due to the late time for acquiring the original monitoring data of different associated monitoring systems.
In the operation and maintenance monitoring method based on the data center provided by this embodiment, corresponding data acquisition processes are created based on the number of monitoring systems corresponding to the associated monitoring systems, so that each data acquisition process can respectively acquire original monitoring data acquired by the associated monitoring system corresponding to the data acquisition process, and the pertinence of the original monitoring data acquisition is ensured; all data acquisition processes are executed in parallel to acquire original monitoring data, so that the original monitoring data formed in the operation process of monitoring the corresponding business systems by a plurality of associated monitoring systems can be acquired simultaneously, and the acquisition efficiency and timeliness of the original monitoring data are guaranteed; each data acquisition process can realize acquisition of original monitoring data formed by monitoring the service system by the associated monitoring system in the current acquisition period, and ensure the continuity of the acquired original monitoring data in time.
In an embodiment, as shown in fig. 4, the step S203 of performing source detection on the monitoring data to be processed to obtain a target source corresponding to the monitoring data to be processed specifically includes the following steps:
s401: and identifying the monitoring data to be processed by adopting a keyword identification algorithm, and judging whether the monitoring data to be processed contains a source key field.
The keyword recognition algorithm is an algorithm for recognizing whether a certain text content includes a specific keyword, and the keyword recognition algorithm includes, but is not limited to, a regular matching algorithm, a string interception algorithm, and a hybrid matching algorithm. The source key field is a field that can directly reflect the data source, for example, a source field or a system attribute field is preset as the source key field.
S402: and if the to-be-processed monitoring data contains the source key field, determining the field content corresponding to the source key field as a target source corresponding to the to-be-processed monitoring data.
As an example, the data center identifies the monitoring data to be processed by using a keyword identification algorithm, and determines that the data content of the monitoring data to be processed includes source key fields such as a source field or a system attribute field, and the like, and then takes the field content corresponding to the source key field as the target source, thereby improving the determination efficiency and accuracy of the target source.
S403: and if the to-be-processed monitoring data does not contain the source key fields, identifying the to-be-processed monitoring data by adopting a key word identification algorithm, and judging whether the to-be-processed monitoring data contains the associated key fields.
The associated key field is a field which cannot directly reflect the data source of the associated key field, but is related to the data source, such as an eventName field or a parent field.
As an example, the data center identifies the monitoring data to be processed by using a keyword identification algorithm, and when it is determined that the data content of the monitoring data to be processed does not include the source key field, identifies whether the monitoring data to be processed includes the associated key field by using the keyword identification algorithm, so as to perform subsequent processing based on the determination result, thereby determining the target source corresponding to the monitoring data to be processed.
S404: and if the to-be-processed monitoring data contains the associated key field, performing data processing on the field content corresponding to the associated key field to obtain a target source corresponding to the to-be-processed monitoring data.
As an example, step S404, namely, performing data processing on the field content corresponding to the associated key field, and acquiring the target source corresponding to the to-be-processed monitoring data, specifically includes the following steps: and processing data of the field content corresponding to the associated key field by adopting a processing logic corresponding to the associated key field so as to determine a target source corresponding to the to-be-processed monitoring data. The processing logic corresponding to the associated key field is processing logic for processing the field content corresponding to the associated key field to obtain the target source. For example, data processing may be performed by splitting, merging or otherwise processing field contents corresponding to the associated key fields, so as to quickly and accurately determine the target source from the associated key fields, and ensure the determination efficiency of the target source.
As an example, step S404, namely, performing data processing on the field content corresponding to the associated key field, and acquiring the target source corresponding to the to-be-processed monitoring data, specifically includes the following steps: splitting and extracting field contents corresponding to the associated key fields to acquire associated information; and querying the association mapping table based on the association information, and determining the data source corresponding to the association information in the association mapping table as a target source corresponding to the to-be-processed monitoring data. The process of splitting and extracting the field content corresponding to the associated key field refers to a process of splitting the field content of the associated key field and extracting associated information related to a data source. The association mapping table is a preset data table for storing the association information and the corresponding data source. It can be understood that, in the present example, table look-up processing is performed on the association information extracted by splitting the association key field, so that the target source can be quickly determined, and the determination efficiency of the target source is ensured.
As another example, after step S403, that is, after the monitoring data to be processed is identified by using the keyword identification algorithm, and whether the monitoring data to be processed includes the associated key field is determined, if the monitoring data to be processed does not include the associated key field, other processing logic (for example, processing logic corresponding to steps S501 to S502) for obtaining the target source may be executed, and the monitoring data stream to be processed may also be transferred to a manual processing mechanism, and the target source is manually calibrated by an operation and maintenance person, so as to perform subsequent processing. The other processing logic herein may include invoking a data source interface, determining monitoring system information corresponding to an associated monitoring system that transmits the to-be-processed monitoring data, querying a preset source mapping table based on the monitoring system information, and determining a target source corresponding to the to-be-processed monitoring data.
In the operation and maintenance monitoring method based on the data center provided by this embodiment, a keyword recognition algorithm is sequentially adopted to recognize whether the monitored data to be processed includes the source key field and the associated key field, so that the target source is determined according to the field content corresponding to the source key field and the associated key field, and the accuracy and efficiency of determining the target source are ensured.
In an embodiment, as shown in fig. 5, the step S203 of performing source detection on the monitoring data to be processed to obtain a target source corresponding to the monitoring data to be processed specifically includes the following steps:
s501: and performing source detection on the monitoring data to be processed by adopting a preset source identification model to obtain at least one identification source and identification probability corresponding to each identification source.
The preset source identification model is a model which is trained in advance and is used for identifying the data source of the monitoring data. The source identification is to adopt a preset source identification model to carry out source detection on the monitoring data to be processed, and the data source is identified. The identification probability corresponding to the identification source refers to the probability that the preset source identification model identifies the monitored data to be processed and determines that the monitored data belongs to a certain data source.
As an example, before step S201, the operation and maintenance monitoring method based on the data center further includes a process of training a preset source identification model, and specifically includes the following steps:
(1) based on historical monitoring data, model training samples are obtained, and the model training samples are divided into a training set and a testing set. The historical monitoring data is the monitoring data collected before step S201 is executed, i.e. before the current time of the system at which the original monitoring data is acquired. The model training sample is a sample which is formed by labeling the historical monitoring data with corresponding data sources in advance and is used for model training.
(2) And training the neural network model by adopting the model training samples in the training set, updating model parameters in the neural network model, and acquiring an original source identification model. As an example, model training samples in the training set may be input into a CNN, RNN, or other neural network model for model training to update model parameters in the neural network model to form the original source recognition model.
(3) And testing the original source identification model by adopting the model training sample in the test set to obtain a model test result, and determining the original source identification model as the preset source identification model if the model test result reaches a preset standard. The model test result refers to that the original source identification model is tested by adopting the model training samples in the test set so as to determine the test accuracy of the model training samples in the test set. For example, each model training sample in the test set is input into an original source identification model for identification, and an identification result is obtained; if the recognition result is consistent with the data source marked by the model training sample, the recognition is determined to be accurate; if the recognition result is inconsistent with the data source marked by the model training sample, determining that the recognition is not accurate; and determining the testing accuracy rate based on the number of the model training samples which are accurately identified and the number of all the model training samples according to the identification result in the test set. The predetermined criterion is a preset criterion for evaluating whether the original source recognition model reaches a criterion that is regarded as high in accuracy, and may be set to 90%, for example.
S502: and comparing the maximum recognition probability with a preset probability threshold, and if the maximum recognition probability is greater than the preset probability threshold, determining the recognition source corresponding to the maximum recognition probability as the target source corresponding to the monitoring data to be processed.
The maximum recognition probability is the maximum value of the recognition probabilities corresponding to the recognition sources recognized by the preset source recognition model. Generally, the greater the recognition probability, the more likely the recognition source recognized by the preset source recognition model is to be the target source of the monitored data to be processed. The preset probability threshold is a preset probability threshold for evaluating whether the recognition probability reaches the target source.
As an example, the data center may perform source detection on the monitoring data to be processed by using a pre-trained preset source identification model, and obtain a detection result output by the preset source identification model, where the detection result includes at least one identification source and an identification probability corresponding to each identification source. Then, the identification probabilities corresponding to all the identification sources are sequenced, and then the maximum identification probability is compared with a preset probability threshold. And if the maximum recognition probability is greater than the preset probability threshold, determining the recognition source corresponding to the maximum recognition probability as the target source corresponding to the monitored data to be processed. If the maximum recognition probability is not greater than the preset probability threshold, other processing logic for obtaining the target source can be executed, or the monitoring data stream to be processed can be transferred to a manual processing mechanism, and the target source is calibrated manually by operation and maintenance personnel, so that subsequent processing can be performed.
In the operation and maintenance monitoring method based on the data center, the pre-trained preset source identification model is adopted to perform source detection on the monitored data to be processed, so that the target source of the monitored data to be processed is quickly and effectively determined according to the data content in the monitored data to be processed, and the accuracy and the efficiency of determining the target source are ensured.
In an embodiment, as shown in fig. 6, step S204, namely, performing validity detection on the monitoring data to be processed to obtain a validity detection result, specifically includes the following steps:
s601: and inquiring a legal source mapping table based on the target source of the monitoring data to be processed to obtain a legality checking result.
The legal source mapping table is a preset data table used for evaluating whether a data source is legal or not. The legality checking result is a result of legality checking the target source of the monitored data to be processed. The legality checking result comprises a source legality result and a source illegally result, wherein the source legality means that a target source of certain to-be-processed monitoring data is a legal data source which needs to be monitored by the data center; the source is illegal, namely that a target source of certain monitored data to be processed is an illegal data source which does not need to be monitored by the data center.
As an example, the source information of all legal sources may be stored in a legal source mapping table, the legal source mapping table is queried based on a target source identified by the to-be-processed monitoring data, and if the target source is in the legal source mapping table, a validity check result of the source validity is obtained; and if the target source is not in the legal source mapping table, acquiring a source illegal validity check result.
As another example, a plurality of data sources and source attributes corresponding to each data source may be stored in a legal source mapping table, where the source attributes include source legal and source illegal; and inquiring a legal source mapping table based on the target source identified by the monitoring data to be processed, and determining a corresponding validity check result according to the source attribute of the target source in the legal source mapping table.
S602: and inquiring an alarm state information table based on the monitoring data to be processed to obtain an alarm state result.
The alarm state information table is a preset data table for evaluating whether an alarm event in the monitoring data is valid. The alarm state result is the result of determining whether a certain monitored data to be processed is still in an effective state according to the alarm state information table.
As an example, since a time difference exists between the time when the associated monitoring system collects the original monitoring data and the time when the data center acquires the original monitoring data from the associated monitoring system, and the state of the monitoring data to be processed may change within the time difference, after the data center acquires the monitoring data to be processed, the data center needs to query an alarm state information table based on the monitoring log recorded in the monitoring data to be processed, and determine whether the monitoring log still identifies a fault event that needs to be monitored; and acquiring an alarm state result according to the judgment result. If the alarm state information table is inquired according to the monitoring log and the alarm state information table is determined to be a fault event which still needs to be monitored continuously, the obtained alarm state result is an effective state; and if the alarm state information table is inquired according to the monitoring log and is determined to be a fault event which does not need to be monitored continuously, the obtained alarm state result is in an invalid state.
S603: and if the source of the validity checking result is legal and the alarm state result is an effective state, acquiring an effective alarm validity detection result.
As an example, in the process of detecting validity of the to-be-processed monitoring data, if it is determined that the validity check result is valid and the alarm state result is valid, it indicates that the target source of the to-be-processed monitoring data is a valid data source that needs to be monitored by the data center, and the fault event corresponding to the to-be-processed monitoring data is a fault event that needs to be continuously monitored by the data center, at this time, the validity detection result of the alarm is obtained, so that the to-be-processed monitoring data that is valid for the alarm is continuously processed.
S604: and if the validity check result is that the source is illegal or the alarm state result is in an invalid state, acquiring the validity detection result of invalid alarm.
As an example, in the process of detecting the validity of the monitored data to be processed, if it is determined that the validity check result is illegal, or the alarm state result is valid, the data center indicates that the target source of the monitored data to be processed is not an illegal data source that needs to be monitored by the data center, or the fault data corresponding to the monitored data to be processed is not a fault event that needs to be continuously monitored by the data center, and at this time, the validity detection result that the alarm is invalid is obtained, so that the data center does not need to continuously monitor and analyze the monitored data to be processed.
In the operation and maintenance monitoring method based on the data center provided by this embodiment, a legal source mapping table and an alarm state information table are respectively queried based on to-be-processed monitoring data to obtain a validity check result and an alarm state result; and determining that the alarm is valid according to the validity detection result with the validity check result as a valid source and the alarm state result as a valid state, otherwise, determining that the alarm is invalid, and ensuring the accuracy and timeliness of the validity detection result.
In an embodiment, as shown in fig. 7, step S205, namely, performing formatting processing based on the valid monitoring data and the target source corresponding to the valid monitoring data to obtain the target monitoring data specifically includes the following steps:
s701: and coding and formatting the effective monitoring data to obtain standard monitoring data.
The code conversion of the effective monitoring data refers to a processing process of converting all effective monitoring data into unified codes. The standard monitoring data refers to monitoring data determined after effective monitoring data is coded and formatted.
The data center can uniformly convert effective monitoring data uploaded by different associated monitoring systems into a target coding format, such as a UTF-8 coding format, so that the feasibility of subsequent data monitoring and analysis processing is ensured, and the efficiency and the accuracy of monitoring and analysis are improved. As an example, the data center identifies a current encoding format of valid monitoring data; if the current coding format is a target coding format of the UTF-8 coding format, the coding conversion processing is not required; if the current coding format is not the target coding format UTF-8 coding format but the Unicode coding format, format conversion is carried out on the current coding format based on the format conversion rule corresponding to the current coding format and the target coding format so as to obtain the standard monitoring data matched with the target coding mode, thereby ensuring the consistency of the coding format of the standard monitoring data and facilitating the feasibility of subsequent automatic monitoring analysis.
S702: and identifying the standard monitoring data by adopting a keyword identification algorithm, and judging whether the standard monitoring data comprises a core format field.
The core format field refers to a field whose field content needs to adopt a specific format.
As an example, after acquiring the standard monitoring data, the data center may adopt a keyword recognition algorithm to recognize data content in the standard monitoring data, so as to determine whether all fields in the standard monitoring data include a core format field, so as to determine whether further format conversion is required according to a determination result, thereby ensuring pertinence of format conversion.
S703: if the standard monitoring data contains the core format field, adopting a format conversion script corresponding to the core format field to perform format conversion on the original field content corresponding to the core format field to obtain target field content, adopting the target field content to replace the original field content, newly adding a source field and a corresponding target source, and obtaining target monitoring data.
The format conversion script corresponding to the core format field is a preset script for realizing format conversion of field content corresponding to the core format field.
As an example, if the standard monitoring data includes a core format field of eventName (event name), a format conversion script corresponding to the core format field is used to uniformly convert the original field content corresponding to the core format field in the standard monitoring data into a target field content corresponding to a specific format of "ObjName (project name) -ObjType (project type) -Source-Desc (team)", and then the target field content is replaced with the original field content.
As an example, if the standard monitoring data includes a core format field, which is tag department, a format conversion script corresponding to the core format field is used to perform a character length check on the original field content corresponding to the core format field in the standard monitoring data, and if the character length meets the standard length, the format conversion is not required; and if the character length does not accord with the standard length, splicing new content according to a set rule to carry out addition and replacement so as to obtain the target field content of which the character length accords with the standard length, and replacing the target field content with the original field content.
As an example, if the standard monitoring data includes a core format field of duration (alarm duration), the format conversion script corresponding to the core format field is used to perform time unit conversion on the original field content corresponding to the core format field in the standard monitoring data, and the original field content is converted into a unified time unit, so that the monitoring data related to time is automatically monitored in the following process, and the monitoring efficiency is improved.
In this example, when the standard monitoring data includes a core format field, format conversion is performed on original field content corresponding to the core format field by using a format conversion script, and after the original field content is replaced by target field content, a source field and a target source corresponding to the source field need to be added to the standard monitoring data, so that the finally obtained target monitoring field adopts a uniform format to ensure feasibility of subsequent automatic monitoring analysis; and each target monitoring data carries information such as a target source and the like, so that all the target monitoring data can be automatically monitored and analyzed based on the target source in the following process, and the monitoring effectiveness is ensured.
S704: and if the standard monitoring data does not contain the core format field, adding a source field and a corresponding target source, and acquiring the target monitoring data.
In this example, when the standard monitoring data does not include the core format field, format conversion is not required to be performed on the original field content corresponding to the core format field, and the source field and the target source corresponding to the source field are directly added to the standard monitoring data, so that each target monitoring data carries information such as a target source, and therefore, all target monitoring data can be automatically monitored and analyzed based on the target source in the following process, and monitoring effectiveness is guaranteed.
In the operation and maintenance monitoring method based on the data center provided by this embodiment, the valid monitoring data is encoded and formatted to ensure the identifiability of the standard monitoring data obtained by encoding and formatting, so that an automatic monitoring program of the data center can accurately and effectively identify the data content in the standard monitoring data; when the standard monitoring data contains a core format field, format conversion is carried out by adopting a format conversion script, and the original field content is replaced by the target field content, so that the target field content corresponding to the core format field adopts a uniform format, and the feasibility of subsequent automatic monitoring analysis is ensured; and adding a source field and a target source corresponding to the source field in the standard format field so that each target monitoring data carries information such as a target source and the like, and performing automatic monitoring analysis on all target monitoring data based on the target source in the following process to ensure the monitoring effectiveness.
In an embodiment, as shown in fig. 8, step S206, namely, based on the target source, performing automatic monitoring on the target monitoring data to obtain the alarm monitoring result, specifically includes the following steps:
s801: and starting an automatic monitoring task, wherein the automatic monitoring task comprises a monitoring period and a target monitoring index.
The automatic monitoring task is a preset computer executable task for realizing automatic monitoring. The monitoring data cycle refers to a time interval of monitoring data collected from business data that needs to be monitored in a data center. The target monitoring index refers to an index which needs to be monitored by the automatic monitoring task, and includes but is not limited to an alarm processing rate, an alarm timeout rate, an alarm processing time limit, a user utilization rate and the like.
S802: dividing all target monitoring data of the event occurrence time in the monitoring period according to the target sources, and determining the monitoring data to be analyzed corresponding to each target source.
As an example, the data center may compare the event occurrence time of the target monitoring data with the monitoring period to determine whether the event occurrence time is within the monitoring period, so as to screen out all target monitoring data of which the event occurrence time is within the monitoring period as data that needs to be automatically monitored; then, all target monitoring data of the event occurrence time in the monitoring period are divided according to the target sources in the target monitoring data, so that all the target monitoring data are divided into different data sets to be monitored according to the number of the target sources, each data set to be monitored corresponds to one target source, each data set to be monitored specifically comprises monitoring data to be analyzed corresponding to one target source, and the monitoring data to be analyzed is specifically target monitoring data of all the event occurrence time corresponding to the target source in the monitoring period.
S803: and calling an index operation script corresponding to the target monitoring index, reading a variable factor value corresponding to the target variable factor from the monitoring data to be analyzed, calculating the variable factor value by adopting index operation logic, and acquiring an alarm monitoring result corresponding to each target source.
Generally, the index operation formula includes index variable factors and index operation logic, wherein the index variable factors are variable factors used for calculating a target monitoring index, and the index operation logic is processing logic used for performing mathematical operation on all index variable factors.
As an example, the data center may execute an index operation script corresponding to a target monitoring index set in an automatic monitoring task, and each monitored data to be analyzed is a formatted target monitoring index, so that the index operation script may quickly obtain a variable factor value corresponding to a target variable factor in an index operation formula from the monitored data to be analyzed, and then calculate the variable factor value by using an index operation logic in the index operation formula, thereby obtaining an alarm monitoring result corresponding to each target source, so as to implement automatic monitoring of all target monitoring data corresponding to each target source, improve the efficiency of data monitoring, and reduce the cost of data monitoring.
In the operation and maintenance monitoring method based on the data center provided by this embodiment, since the formatted target monitoring data all include the event occurrence time and the target source, it is feasible to divide the processing process of the monitoring data to be analyzed based on the event occurrence time and the target source, so that all the target monitoring data can be conveniently processed based on the analysis dimension of the target source; the index operation script corresponding to the target monitoring index is called, so that the corresponding variable factor value obtained from the monitoring data to be analyzed can be automatically and quickly calculated by using the index operation logic, the alarm monitoring result corresponding to each target source is obtained, all the target monitoring data corresponding to each target source can be automatically monitored, the data monitoring efficiency is improved, and the data monitoring cost is reduced.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
In an embodiment, an operation and maintenance monitoring device based on a data center is provided, and the operation and maintenance monitoring device based on the data center corresponds to the operation and maintenance monitoring method based on the data center in the above embodiment one to one. As shown in fig. 9, the operation and maintenance monitoring apparatus based on the data center includes an original monitoring data obtaining module 901, a to-be-processed monitoring data obtaining module 902, a target source obtaining module 903, an effective monitoring data obtaining module 904, a target monitoring data obtaining module 905, and an alarm monitoring result obtaining module 906. The functional modules are explained in detail as follows:
an original monitoring data obtaining module 901, configured to obtain original monitoring data corresponding to a service system collected by an associated monitoring system, where each original monitoring data includes an original alarm level.
The to-be-processed monitoring data obtaining module 902 is configured to perform standardization processing on the original alarm level, obtain a standard alarm level, and determine the original monitoring data of which the standard alarm level is the target alarm level as the to-be-processed monitoring data.
The target source obtaining module 903 is configured to perform source detection on the to-be-processed monitoring data, and obtain a target source corresponding to the to-be-processed monitoring data.
The effective monitoring data obtaining module 904 is configured to perform validity detection on the to-be-processed monitoring data, obtain a validity detection result, and determine the to-be-processed monitoring data with the validity detection result being effective for warning as effective monitoring data.
The target monitoring data obtaining module 905 is configured to perform formatting processing based on the effective monitoring data and a target source corresponding to the effective monitoring data, and obtain target monitoring data.
And an alarm monitoring result obtaining module 906, configured to perform automatic monitoring on the target monitoring data based on the target source, and obtain an alarm monitoring result.
Preferably, the original monitoring data acquisition module includes a monitoring system quantity acquisition unit, a data acquisition process creation unit, and an acquisition process parallel execution unit.
And the monitoring system quantity obtaining unit is used for monitoring the quantity of the monitoring systems corresponding to the associated monitoring systems in real time.
And the data acquisition process creating unit is used for creating data acquisition processes corresponding to the number of the monitoring systems, and each data acquisition process corresponds to one associated monitoring system.
And the acquisition process parallel execution unit is used for executing all data acquisition processes in parallel and acquiring the original monitoring data corresponding to the service system acquired by each associated monitoring system in the current acquisition period.
Preferably, the target source obtaining module includes a first keyword recognition unit, a first source determining unit, a second keyword recognition unit and a second source determining unit.
The first keyword identification unit is used for identifying the monitoring data to be processed by adopting a keyword identification algorithm and judging whether the monitoring data to be processed contains a source key field.
The first source determining unit is configured to determine, if the to-be-processed monitoring data includes a source key field, a field content corresponding to the source key field as a target source corresponding to the to-be-processed monitoring data.
And the second keyword identification unit is used for identifying the to-be-processed monitoring data by adopting a keyword identification algorithm if the to-be-processed monitoring data does not contain the source key field, and judging whether the to-be-processed monitoring data contains the associated key field.
And the second source determining unit is used for processing the data of the field content corresponding to the associated key field and acquiring the target source corresponding to the to-be-processed monitoring data if the to-be-processed monitoring data contains the associated key field.
Preferably, the target source obtaining module includes a model identification processing unit and a third source determining unit.
And the model identification processing unit is used for detecting the source of the monitoring data to be processed by adopting a preset source identification model and acquiring at least one identification source and the identification probability corresponding to each identification source.
And the third source determining unit is used for comparing the maximum recognition probability with a preset probability threshold, and if the maximum recognition probability is greater than the preset probability threshold, determining the recognition source corresponding to the maximum recognition probability as the target source corresponding to the to-be-processed monitoring data.
Preferably, the effective monitoring data obtaining module includes a validity check result obtaining unit, an alarm state result obtaining unit, a legal source result obtaining unit and an illegal source result obtaining unit.
And the legality checking result obtaining unit is used for inquiring the legal source mapping table based on the target source of the monitoring data to be processed to obtain the legality checking result.
And the alarm state result acquisition unit is used for inquiring the alarm state information table based on the monitoring data to be processed and acquiring the alarm state result.
And the legal source result obtaining unit is used for obtaining the effective validity detection result of the alarm if the validity check result is that the source is legal and the alarm state result is in an effective state.
And the illegal source result acquisition unit is used for acquiring the validity detection result of invalid alarm if the validity check result is that the source is illegal or the alarm state result is in an invalid state.
Preferably, the target monitoring data obtaining module includes a standard monitoring data obtaining unit, a core format field judging unit, a first target data obtaining unit, and a second target data obtaining unit.
And the standard monitoring data acquisition unit is used for coding and formatting the effective monitoring data to acquire the standard monitoring data.
And the core format field judging unit is used for identifying the standard monitoring data by adopting a keyword identification algorithm and judging whether the standard monitoring data comprises the core format field.
And the first target data acquisition unit is used for performing format conversion on original field content corresponding to the core format field by adopting a format conversion script corresponding to the core format field to acquire target field content, replacing the original field content by the target field content, newly adding a source field and a corresponding target source, and acquiring target monitoring data if the standard monitoring data contains the core format field.
And the second target data acquisition unit is used for adding a source field and a corresponding target source to acquire the target monitoring data if the standard monitoring data does not contain the core format field.
Preferably, the alarm monitoring result obtaining module includes a monitoring task starting unit, a to-be-analyzed data determining unit and an alarm monitoring processing unit.
And the monitoring task starting unit is used for starting an automatic monitoring task, and the automatic monitoring task comprises a monitoring period and a target monitoring index.
And the data to be analyzed determining unit is used for dividing all target monitoring data of which the event occurrence time is in the monitoring period according to the target sources and determining the monitoring data to be analyzed corresponding to each target source.
And the alarm monitoring processing unit is used for calling an index operation script corresponding to the target monitoring index, reading a variable factor value corresponding to the target variable factor from the monitoring data to be analyzed, calculating the variable factor value by adopting index operation logic and acquiring an alarm monitoring result corresponding to each target source.
For specific limitations of the operation and maintenance monitoring device based on the data center, reference may be made to the above limitations of the operation and maintenance monitoring method based on the data center, and details are not described here. All or part of the modules in the operation and maintenance monitoring device based on the data center can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 10. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer equipment is used for storing data adopted or generated in the process of executing the operation and maintenance monitoring method based on the data center. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to realize a data center-based operation and maintenance monitoring method.
In an embodiment, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the processor executes the computer program, the method for operation and maintenance monitoring based on a data center in the foregoing embodiments is implemented, for example, S201 to S206 shown in fig. 2, or shown in fig. 3 to fig. 8, which is not described herein again to avoid repetition. Alternatively, when executing the computer program, the processor implements the functions of each module/unit in the data center-based operation and maintenance monitoring apparatus in this embodiment, for example, the functions of the original monitoring data obtaining module 901, the to-be-processed monitoring data obtaining module 902, the target source obtaining module 903, the effective monitoring data obtaining module 904, the target monitoring data obtaining module 905, and the alarm monitoring result obtaining module 906 shown in fig. 9, and are not described herein again to avoid repetition.
In an embodiment, a computer-readable storage medium is provided, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the operation and maintenance monitoring method based on a data center in the foregoing embodiments is implemented, for example, S201 to S206 shown in fig. 2, or shown in fig. 3 to fig. 8, which is not described herein again to avoid repetition. Alternatively, when being executed by the processor, the computer program implements the functions of each module/unit in the above operation and maintenance monitoring apparatus based on the data center, for example, the functions of the original monitoring data obtaining module 901, the to-be-processed monitoring data obtaining module 902, the target source obtaining module 903, the effective monitoring data obtaining module 904, the target monitoring data obtaining module 905, and the alarm monitoring result obtaining module 906 shown in fig. 9, and are not described herein again to avoid repetition.
It will be understood by those of ordinary skill in the art that all or a portion of the processes of the methods of the embodiments described above may be implemented by a computer program that may be stored on a non-volatile computer-readable storage medium, which when executed, may include the processes of the embodiments of the methods described above, wherein any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.

Claims (10)

1. An operation and maintenance monitoring method based on a data center is characterized by comprising the following steps:
acquiring original monitoring data corresponding to a service system acquired by a correlation monitoring system, wherein each original monitoring data comprises an original alarm level;
standardizing the original alarm level to obtain a standard alarm level, and determining original monitoring data with the standard alarm level as a target alarm level as to-be-processed monitoring data;
performing source detection on the monitoring data to be processed to obtain a target source corresponding to the monitoring data to be processed;
carrying out validity detection on the monitoring data to be processed to obtain a validity detection result, and determining the monitoring data to be processed with the validity detection result being effective for warning as effective monitoring data;
formatting the effective monitoring data and a target source corresponding to the effective monitoring data to obtain target monitoring data;
and automatically monitoring the target monitoring data based on the target source to obtain an alarm monitoring result.
2. The operation and maintenance monitoring method based on the data center as claimed in claim 1, wherein the obtaining of the original monitoring data corresponding to the service system collected by the associated monitoring system comprises:
monitoring the number of monitoring systems corresponding to the associated monitoring systems in real time;
establishing data acquisition processes corresponding to the number of the monitoring systems, wherein each data acquisition process corresponds to one associated monitoring system;
and executing all the data acquisition processes in parallel, and acquiring original monitoring data corresponding to the service system acquired by each associated monitoring system in the current acquisition period.
3. The operation and maintenance monitoring method based on the data center as claimed in claim 1, wherein the performing source detection on the monitoring data to be processed to obtain a target source corresponding to the monitoring data to be processed comprises:
identifying the monitoring data to be processed by adopting a keyword identification algorithm, and judging whether the monitoring data to be processed contains a source key field;
if the to-be-processed monitoring data contains source key fields, determining field contents corresponding to the source key fields as target sources corresponding to the to-be-processed monitoring data;
if the to-be-processed monitoring data does not contain the source key field, identifying the to-be-processed monitoring data by adopting a key word identification algorithm, and judging whether the to-be-processed monitoring data contains the associated key field;
and if the to-be-processed monitoring data contains the associated key field, performing data processing on the field content corresponding to the associated key field to obtain a target source corresponding to the to-be-processed monitoring data.
4. The operation and maintenance monitoring method based on the data center as claimed in claim 1, wherein the performing source detection on the monitoring data to be processed to obtain a target source corresponding to the monitoring data to be processed comprises:
performing source detection on the monitoring data to be processed by adopting a preset source identification model, and acquiring at least one identification source and identification probability corresponding to each identification source;
and comparing the maximum recognition probability with a preset probability threshold, and if the maximum recognition probability is greater than the preset probability threshold, determining the recognition source corresponding to the maximum recognition probability as the target source corresponding to the to-be-processed monitoring data.
5. The operation and maintenance monitoring method based on the data center as claimed in claim 1, wherein the performing validity detection on the monitoring data to be processed to obtain a validity detection result comprises:
inquiring a legal source mapping table based on the target source of the monitoring data to be processed to obtain a legality checking result;
inquiring an alarm state information table based on the monitoring data to be processed to obtain an alarm state result;
if the validity check result is that the source is legal and the alarm state result is in a valid state, obtaining an effective alarm validity detection result;
and if the validity check result is that the source is illegal or the alarm state result is in an invalid state, acquiring the validity detection result of invalid alarm.
6. The operation and maintenance monitoring method based on the data center according to claim 1, wherein the step of performing formatting processing based on the effective monitoring data and the target source corresponding to the effective monitoring data to obtain the target monitoring data comprises:
coding and formatting the effective monitoring data to obtain standard monitoring data;
identifying the standard monitoring data by adopting a keyword identification algorithm, and judging whether the standard monitoring data comprises a core format field;
if the standard monitoring data contains a core format field, carrying out format conversion on original field content corresponding to the core format field by adopting a format conversion script corresponding to the core format field to obtain target field content, replacing the original field content with the target field content, adding a source field and a corresponding target source, and obtaining target monitoring data;
and if the standard monitoring data does not contain the core format field, adding a source field and a corresponding target source, and acquiring target monitoring data.
7. The operation and maintenance monitoring method based on the data center as claimed in claim 1, wherein the automatically monitoring the target monitoring data based on the target source and obtaining the alarm monitoring result comprises:
starting an automatic monitoring task, wherein the automatic monitoring task comprises a monitoring period and a target monitoring index;
dividing all the target monitoring data of which the event occurrence time is in the monitoring period according to the target sources, and determining the monitoring data to be analyzed corresponding to each target source;
and calling an index operation script corresponding to the target monitoring index, reading a variable factor value corresponding to a target variable factor from the monitoring data to be analyzed, and calculating the variable factor value by adopting index operation logic to obtain an alarm monitoring result corresponding to each target source.
8. An operation and maintenance monitoring device based on a data center is characterized by comprising:
the system comprises an original monitoring data acquisition module, a data processing module and a data processing module, wherein the original monitoring data acquisition module is used for acquiring original monitoring data corresponding to a service system acquired by a related monitoring system, and each original monitoring data comprises an original alarm level;
the monitoring data to be processed acquisition module is used for carrying out standardization processing on the original alarm level, acquiring a standard alarm level and determining the original monitoring data of which the standard alarm level is a target alarm level as the monitoring data to be processed;
the target source acquisition module is used for carrying out source detection on the monitoring data to be processed and acquiring a target source corresponding to the monitoring data to be processed;
the effective monitoring data acquisition module is used for carrying out effectiveness detection on the monitoring data to be processed, acquiring an effectiveness detection result and determining the monitoring data to be processed with the effectiveness detection result as effective monitoring data;
the target monitoring data acquisition module is used for carrying out formatting processing based on the effective monitoring data and a target source corresponding to the effective monitoring data to acquire target monitoring data;
and the alarm monitoring result acquisition module is used for automatically monitoring the target monitoring data based on the target source and acquiring an alarm monitoring result.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data center-based operation and maintenance monitoring method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the data center-based operation and maintenance monitoring method according to any one of claims 1 to 7.
CN202010153280.5A 2020-03-06 2020-03-06 Operation and maintenance monitoring method, device and equipment based on data center and storage medium Pending CN111475370A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202010153280.5A CN111475370A (en) 2020-03-06 2020-03-06 Operation and maintenance monitoring method, device and equipment based on data center and storage medium
PCT/CN2020/093315 WO2021174694A1 (en) 2020-03-06 2020-05-29 Operation and maintenance monitoring method and apparatus based on data center, device, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010153280.5A CN111475370A (en) 2020-03-06 2020-03-06 Operation and maintenance monitoring method, device and equipment based on data center and storage medium

Publications (1)

Publication Number Publication Date
CN111475370A true CN111475370A (en) 2020-07-31

Family

ID=71748061

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010153280.5A Pending CN111475370A (en) 2020-03-06 2020-03-06 Operation and maintenance monitoring method, device and equipment based on data center and storage medium

Country Status (2)

Country Link
CN (1) CN111475370A (en)
WO (1) WO2021174694A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112181967A (en) * 2020-09-29 2021-01-05 中国平安人寿保险股份有限公司 Method and device for monitoring source data quality, computer equipment and medium
CN113515603A (en) * 2021-04-27 2021-10-19 深圳力维智联技术有限公司 Method, system and equipment for processing operation and maintenance monitoring data of machine room
WO2022262013A1 (en) * 2021-06-15 2022-12-22 合肥维天运通信息科技股份有限公司 Method and system for collecting and monitoring vehicle state information

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113641567B (en) * 2021-10-13 2022-03-25 北京易真学思教育科技有限公司 Database inspection method and device, electronic equipment and storage medium
CN114161410B (en) * 2021-11-16 2024-01-09 中电信数智科技有限公司 Operation and maintenance method, device, electronic equipment and storage medium
CN114363934A (en) * 2021-12-30 2022-04-15 中国电信股份有限公司 Base station health degree evaluation method, device, equipment and medium
CN114140032A (en) * 2022-01-29 2022-03-04 北京优特捷信息技术有限公司 Facility running state monitoring method, device, equipment and storage medium
CN115118572A (en) * 2022-07-28 2022-09-27 济南浪潮数据技术有限公司 Server alarm adaptation method, device, equipment and storage medium
CN115629915B (en) * 2022-12-07 2023-03-14 深圳华声医疗技术股份有限公司 Medical equipment data processing method and device, medical equipment and storage medium
CN116380149B (en) * 2023-04-07 2024-02-02 深圳市兴源智能仪表股份有限公司 Method and system for testing rotation of instrument code wheel
CN116708506B (en) * 2023-06-28 2023-10-27 广州豪特节能环保科技股份有限公司 Intelligent monitoring method, system and storage medium for data center
CN116610664B (en) * 2023-07-19 2024-01-16 深圳高灯计算机科技有限公司 Data monitoring method, device, computer equipment, storage medium and product
CN116881526B (en) * 2023-09-07 2023-12-15 埃睿迪信息技术(北京)有限公司 Data processing method, device and equipment

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105306272B (en) * 2015-11-10 2019-01-25 中国建设银行股份有限公司 Information system fault scenes formation gathering method and system
CN107679713A (en) * 2017-09-16 2018-02-09 广西电网有限责任公司电力科学研究院 A kind of power transmission and transformation equipment state alert processing method
CN108763038B (en) * 2018-08-08 2022-04-12 平安科技(深圳)有限公司 Alarm data management method and device, computer equipment and storage medium
CN109218097A (en) * 2018-09-19 2019-01-15 山东浪潮云投信息科技有限公司 A kind of warning system and alarm method of cloud platform configurable alert rule
CN110505102B (en) * 2019-09-11 2022-03-01 国网湖北省电力有限公司鄂州供电公司 Power information communication fusion monitoring and service standardization management platform system and method
CN110620690A (en) * 2019-09-19 2019-12-27 国网思极网安科技(北京)有限公司 Network attack event processing method and electronic equipment thereof

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112181967A (en) * 2020-09-29 2021-01-05 中国平安人寿保险股份有限公司 Method and device for monitoring source data quality, computer equipment and medium
CN112181967B (en) * 2020-09-29 2023-08-22 中国平安人寿保险股份有限公司 Method, device, computer equipment and medium for monitoring quality of source data
CN113515603A (en) * 2021-04-27 2021-10-19 深圳力维智联技术有限公司 Method, system and equipment for processing operation and maintenance monitoring data of machine room
WO2022262013A1 (en) * 2021-06-15 2022-12-22 合肥维天运通信息科技股份有限公司 Method and system for collecting and monitoring vehicle state information
US11954955B2 (en) 2021-06-15 2024-04-09 Hefei Weitianyuntong Information Technology Co., Ltd Method and system for collecting and monitoring vehicle status information

Also Published As

Publication number Publication date
WO2021174694A1 (en) 2021-09-10

Similar Documents

Publication Publication Date Title
CN111475370A (en) Operation and maintenance monitoring method, device and equipment based on data center and storage medium
CN112631913B (en) Method, device, equipment and storage medium for monitoring operation faults of application program
EP4099170B1 (en) Method and apparatus of auditing log, electronic device, and medium
CN111143163B (en) Data monitoring method, device, computer equipment and storage medium
CN109684052B (en) Transaction analysis method, device, equipment and storage medium
CN110674009B (en) Application server performance monitoring method and device, storage medium and electronic equipment
CN110647523B (en) Data quality analysis method and device, storage medium and electronic equipment
CN112800061B (en) Data storage method, device, server and storage medium
CN110851471A (en) Distributed log data processing method, device and system
CN113190426B (en) Stability monitoring method for big data scoring system
CN115529595A (en) Method, device, equipment and medium for detecting abnormity of log data
WO2016188334A1 (en) Method and device for processing application access data
CN116112194A (en) User behavior analysis method and device, electronic equipment and computer storage medium
CN110362478B (en) Application upgrade test method and device, computer equipment and storage medium
WO2023050967A1 (en) System abnormality detection processing method and apparatus
CN116074215A (en) Network quality detection method, device, equipment and storage medium
CN113535458B (en) Abnormal false alarm processing method and device, storage medium and terminal
CN111309585A (en) Log data testing method, device and system, electronic equipment and storage medium
CN115408236A (en) Log data auditing system, method, equipment and medium
CN114385398A (en) Request response state determination method, device, equipment and storage medium
CN113342619A (en) Log monitoring method and system, electronic device and readable medium
US9625613B2 (en) Weather determination system and method
CN114564853B (en) Evaluation report generation method based on FMEA data and electronic equipment
CN114721912B (en) Data analysis method, device, equipment and medium
CN111859384A (en) Abnormal event monitoring method and device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination