A kind of network management data based on meta-model topology processes method for visually monitoring
Technical field
The present invention relates to a kind of technical field of the computer network, specifically a kind of webmaster number based on meta-model topology
According to treatment method for visually monitoring.
Background technology
The treatment of network management system normal data includes gathering, parses, is put in storage, collecting, using etc. step.For expansion, mainly
Flow includes:Timing slave unit, OMC extract raw data file and are transferred to NM server first, by parsing generation standard
Enter library file, realize data loading;If equipment manufacturer's open database, directly by data mapping ruler, it is directly realized by
Data loading;Data enter each producer's original table by collection, parsing, storage this moment, collect into normalization for different type
Tables of data, further collects the business summary sheet into business personnel's concern, and line number is entered eventually through functional forms such as form, special topics
According to presentation with application.
From said process, data processing runs through whole webmaster, and treatment effeciency directly determines whole webmaster system with quality
System application effect.For network management system, because data source is numerous, interface mode is different, and the production of data is by the multistage
Realize, data source is in itself, network transmission, each step of data processing process go wrong and can influence webmaster application effect.Having must
Real-time detection is carried out to system overall data process situation, and processing condition data is examined with reference to relevant investigation rule
It is disconnected, note abnormalities, notify attendant.
The technical scheme closest with the application motion is patent document in the prior art《A kind of monitoring data collection converges
The method and system of total state》, application number:200610114116.3.The program describes a kind of monitoring data collection and collects shape
The method of state, including:The collection summary status information of each Data Migration point in A, detection collection aggregation process.B, according to each
The configuration information of Data Migration point, each Data Migration point is sorted out according to attribute, and generation includes leading for each Data Migration point
Boat tree;The Data Migration point of concern is found out from navigation tree, using the task in the configuration information and collection summary status of focus
Status information, data integrity information, generate the topological diagram information centered on focused data migration point, and show topological diagram.
4th, the shortcoming of prior art and the application motion technical problem to be solved
For being primarily present following shortcoming in the prior art described in the technical scheme of prior art " three, ":
1. monitoring content is not comprehensive, and whether prior art is only monitored from the complete angle of data, and data
The guarantee for the treatment of needs to consider whether data complete, the data factor such as whether whether consistent, data reasonable.
2. case study stationkeeping ability is weak, and prior art only finds the abnormal problem of data processing step, then
Presented by patterned means, can only be by positioning problems to certain data problem point, it is impossible to which the root to producing problem is carried out
Analyse in depth.
The content of the invention
Method for visually monitoring is processed it is an object of the invention to provide a kind of network management data based on meta-model topology.
The purpose of the present invention is realized in the following manner, including following content:
A, be related to object, the data model of task and associate in network management system data handling procedure by extracting, carded web
The information technoloy equipment model that guard system is related to be connected, the attaching relation with object, task and equipment as corresponding point, generate two-level nested
, data processing end to end topology;
B, by testing equipment operation conditions, network connectivty, task treatment progress, parsing task treatment daily record, analysis
Data validity means, export testing result and are attached to the types of objects of data processing topology in real time, realize that webmaster is counted in real time
According to directly perceived, the effective monitoring means of processing procedure, Network Management and Maintenance personnel are assisted, realize prevention of problem, afterwards influence property in advance
Assessment is traced to the source with reason, and implementation steps are as follows:
Step 1 data processing synthesis Topology g eneration
Step 2 data handling procedure information and satellite information are extracted
Be attached on topological diagram for data handling procedure and satellite information by step 3;
Step 1, data processing synthesis Topology g eneration
Firstly, it is necessary to be related to data processing object, the model of task object in extracting network management data processing procedure and associate
Relation, wherein data object include producer's file, manufacturer data table, original document, enter library file, producer's original table, webmaster and return
One changes table, service application table, Report form application;Task object includes that file acquisition tasks, file analysis storage task, database are straight
It is reappointed or reelected consecutively business, database and collects task, the incidence relation generation primary data treatment topology according to above-mentioned meta-model;
Secondly, the information technoloy equipment connection topology that combing network management system is related to;
Finally, as corresponding point, generation processes end to the affiliation with object, task and information technoloy equipment towards whole network management data
To end, two-level nested comprehensive topological diagram;
Based on comprehensive topological diagram, data processing object, base attribute, the exception rules of each step process task can be checked,
Network traffic and time delay early warning rule between each server basic configuration information, load early warning rule, server, for above-mentioned
It is related to the meta model information of the data processing of each side and information technoloy equipment resource voluntarily to complete combing or extraction by each webmaster;
Step 2, data handling procedure information and satellite information are comprehensively extracted, and are comprised the steps of:
A, by " process analysis instrument " passively obtain processing procedure information:
In task scheduling system serial or parallel data handling procedure, after the completion of task scheduling, by MQ to subsequent treatment
Process analysis procedure analysis module sends scheduling triggering message, and after receiving message, lexical analysis instrument reads data handling procedure daily record, extracts
Operation information, analyzes checking process quality of data situation, is incorporated to data quality information storehouse;
B, by " result detection instrument " active obtaining deal with objects information:
Active detecting data file quantity, size, the information of information bar number;This secondary data of correlation table place in Test database
Manage bar number, field are air situation condition, the validity of index;The accuracy of Testing index algorithm, the object information that active is checked is entered
Data quality information storehouse;
C, reception webmaster information technoloy equipment and network state information:
By the running status of message interface real-time reception webmaster relevant device, including each service load condition, O/I pressures
Power, internal memory utilization power, kernel process state;Database positioning, including database process, load condition, table space;Server
Between network state, the information of reception is entered data quality information storehouse by including network delay, network packet loss rate, shake;
D, reception auxiliary information:
The abnormal all kinds of auxiliary informations of data processing, including webmaster producer may result in by message interface real-time reception
Webmaster edition upgrading plan, network engineering cut-over plan, equipment manufacturer and software release upgrade plan, the information of reception is entered
Data quality information storehouse;
E, according to rule generation data processing problem:
By above step a to step d, realize being managed collectively network management data processing relevant information, contrast combing
For the exception rules that process task, process object, device network are all types of, data processing problem is generated simultaneously according to reference format
It is pushed in problem base, directly can be externally issued in the form of alerting or problem is attached on topological diagram, for above-mentioned exception
Rule voluntarily completes combing by each webmaster;
Step 3, data handling procedure and satellite information are attached on topological diagram
Testing result is exported in real time and is attached in the comprehensive topological diagram types of objects of step 1 generation, realize that webmaster is real-time
Directly perceived, the effective monitoring means of data handling procedure;
Can the following information of monitor in real time based on topological diagram:
(1) whether data processing object is abnormal, including whether rationally data file quantity, size, information bar number, tables of data
Number of data, field are air situation condition, and whether index accurate, and normal process object is presented with white, exception object in topology with
Red is marked;
(2) whether data processing task is abnormal, including whether whether whether task process start, be timely completed, hang, pin
To collection, parsing storage, each generic task for collecting, normal process task represents that abnormal task is in topology with red line with black line
Represent;
(3) webmaster information technoloy equipment, using whether abnormal with network state, including data-source device, acquisition server, database
The cpu load of server and other application servers, O/I pressure, internal memory are utilized, kernel process state, database process, load
Situation, table space, network delay, network packet loss rate, shake, for such monitoring, according to regular thresholding by state be divided into normally,
Abnormal, serious three-level, the equipment of different conditions is identified with green, yellow, red respectively on topological diagram.
It is of the invention compared with immediate prior art, with advantages below:
1st, the data processing of information that can be monitored is comprehensive.Due to using the analysis of processing procedure concatenation, the treatment knot that are driven with MQ
Two technologies of fruit active detecting, it would be preferable to support to each step object of the data processing, integrality of process task, promptness, accurate
Property carries out overall monitor;In addition by probe technique, equipment, the network state of influence data processing are obtained in real time, realize data
The required i.e. finding of processing relevant information.
2nd, data processing problem positioning analysis ability is strong.This programme is using Topology g eneration and nested skill based on model interaction
The comprehensive topology of data processing that art is realized, carries all information that can trigger data hidden danger.By positioning problems to certain data
After problem points, device network operation conditions, the auxiliary information that can be belonged to according to problem points find the root of generation problem.
Brief description of the drawings
Fig. 1 is the data processing topological structure schematic diagram of prior art;
Fig. 2 is meta-model incidence relation generation primary data treatment topological structure schematic diagram;
Fig. 3 is the information technoloy equipment connection topological structure schematic diagram that combing network management system is related to;
Fig. 4 be towards the treatment of whole network management data end to end, the comprehensive topological structure schematic diagram of two-level nested;
Fig. 5 is that general steps describe topological diagram.
Specific embodiment
It is described in detail below to of the invention with reference to Figure of description.
A kind of network management data based on meta-model topology of the invention processes method for visually monitoring, and main contents include:
On the one hand be related to object, the data model of task and associate in network management system data handling procedure by extracting, comb
The information technoloy equipment model that is related to of reason network management system be connected, the attaching relation with object, task and equipment as corresponding point, generation two-stage
Nesting, the topology of data processing end to end.
On the other hand, by testing equipment operation conditions, network connectivty, task treatment progress, parsing task treatment day
The means such as will, analyze data validity, export testing result and are attached to the types of objects of data processing topology in real time, realize net
Directly perceived, the effective monitoring means of pipe real time data processing process.Network Management and Maintenance personnel are assisted, prevention of problem, thing in advance is realized
Influence property assessment afterwards is traced to the source with reason.Comprise the following steps that:
Step 1, data processing synthesis Topology g eneration
Firstly, it is necessary to be related to data processing object, the model of task object in extracting network management data processing procedure and associate
Relation.Wherein data object includes producer's file, manufacturer data table, original document, enters library file, producer's original table, webmaster and return
One changes table, service application table, Report form application;Task object includes that file acquisition tasks, file analysis storage task, database are straight
It is reappointed or reelected consecutively business, database and collects task.Incidence relation generation primary data treatment topology according to above-mentioned meta-model, below figure 2:
Secondly, the information technoloy equipment connection topology that combing network management system is related to, as shown in Figure 3:
Finally, as corresponding point, generation processes end to the affiliation with object, task and information technoloy equipment towards whole network management data
To end, two-level nested comprehensive topological diagram.As shown in Figure 4
Based on comprehensive topological diagram, data processing object, base attribute, the exception rules of each step process task can be checked,
Network traffic and time delay early warning rule between each server basic configuration information, load early warning rule, server.For above-mentioned
It is related to the meta model information of the data processing of each side and information technoloy equipment resource voluntarily to complete combing or extraction by each webmaster, specific side
Method this programme is no longer elaborated.
Step 2, data handling procedure information and satellite information are comprehensively extracted, and are comprised the steps of:
A, by " process analysis instrument " passively obtain processing procedure information:
In task scheduling system serial or parallel data handling procedure, after the completion of task scheduling, by MQ to subsequent treatment
Process analysis procedure analysis module sends scheduling triggering message, and after receiving message, lexical analysis instrument reads data handling procedure daily record, extracts
Operation information, analyzes checking process quality of data situation, is incorporated to data quality information storehouse;
B, by " result detection instrument " active obtaining deal with objects information:
Active detecting data file quantity, size, the information of information bar number;This secondary data of correlation table place in Test database
Manage bar number, field are air situation condition, the validity of index;The accuracy of Testing index algorithm, the object information that active is checked is entered
Data quality information storehouse;
C, reception webmaster information technoloy equipment and network state information:
By the running status of message interface real-time reception webmaster relevant device, including each service load condition, O/I pressures
Power, internal memory utilization power, kernel process state;Database positioning, including database process, load condition, table space;Server
Between network state, the information of reception is entered data quality information storehouse by including network delay, network packet loss rate, shake;
D, reception auxiliary information:
The abnormal all kinds of auxiliary informations of data processing, including webmaster producer may result in by message interface real-time reception
Webmaster edition upgrading plan, network engineering cut-over plan, equipment manufacturer and software release upgrade plan, the information of reception is entered
Data quality information storehouse;
E, according to rule generation data processing problem:
By above step a to step d, realize being managed collectively network management data processing relevant information, contrast combing
For the exception rules that process task, process object, device network are all types of, data processing problem is generated simultaneously according to reference format
It is pushed in problem base, directly can be externally issued in the form of alerting or problem is attached on topological diagram, for above-mentioned exception
Rule voluntarily completes combing by each webmaster;Particular content is no longer elaborated.
General steps description is as shown in Figure 5:
Step 3, data handling procedure and satellite information are attached on topological diagram
Testing result is exported in real time and is attached in the comprehensive topological diagram types of objects of step 1 generation, realize that webmaster is real-time
Directly perceived, the effective monitoring means of data handling procedure.
Can the following information of monitor in real time based on topological diagram:
(1) whether data processing object is abnormal, including whether rationally data file quantity, size, information bar number, tables of data
Number of data, field are air situation condition, and whether index is accurate.Normal process object with white present, exception object in topology with
Red is marked.
(2) whether data processing task is abnormal, including whether whether whether task process start, be timely completed, hang, pin
Collection, parsing are put in storage, each generic task such as collect.Normal process task represents that abnormal task is in topology with red line with black line
Represent.
(3) webmaster information technoloy equipment, using whether abnormal with network state, including data-source device, acquisition server, database
The cpu load of server and other application servers, O/I pressure, internal memory are utilized, kernel process state, database process, load
Situation, table space, network delay, network packet loss rate, shake etc..For such monitoring, state is divided into just according to regular thresholding
Often, abnormal, serious three-level, the equipment of different conditions is identified with green, yellow, red respectively on topological diagram.
In addition to the technical characteristic described in specification, the known technology of those skilled in the art is.