CN108055152A - Communication network information system method for detecting abnormality based on Distributed Services daily record - Google Patents
Communication network information system method for detecting abnormality based on Distributed Services daily record Download PDFInfo
- Publication number
- CN108055152A CN108055152A CN201711309778.0A CN201711309778A CN108055152A CN 108055152 A CN108055152 A CN 108055152A CN 201711309778 A CN201711309778 A CN 201711309778A CN 108055152 A CN108055152 A CN 108055152A
- Authority
- CN
- China
- Prior art keywords
- mrow
- msub
- node
- time
- direct
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
Abstract
The invention discloses the communication network information system method for detecting abnormality based on Distributed Services daily record, by excavating the service reference in network, the case compared during automatic abnormality detection is found, can automatically compare real-time logs stream and exploitation template, achievees the purpose that detection is abnormal.The present invention proposes a kind of method for detecting abnormality of the controlling stream graph model based on time weighting, the good CFG of structure node is generated by the filtering scheme of setting, and split border using cycle or the interval randomness of intermediate node of thread this feature, further optimize structure；The time weighting on each side is calculated, generates TCFG.The TCFG can be compared with real-time TCFG streams, be noted abnormalities, and assessment shows that the method has good accurate rate and recall rate, in true environment, averagely there is 80% accurate rate and 65% recall rate.
Description
Technical field
The present invention relates to communication network information abnormality detection technical field, more particularly to based on Distributed Services daily record
Communication network information system method for detecting abnormality.
Background technology
With the development of network, service when most of network service systems are required for offer 7*24 small, system occurs different
Chang Shi, if test problems and can not in time repair, it will cause huge economic loss.When system exception, developer
It would generally find and handle by log analysis and is abnormal.Diary record system operation information, it is that system exception analysis is most normal
One of method seen.But since system scale is excessive, system log is distributed on a different server, and server
Distribution is extensive and scattered, when developer has found that manual analyzing daily record will expend substantial amounts of to detect abnormal method
Between.In order to reduce the workload of developer, it is meaningful to study the method for detecting abnormality based on automatic log analysis.
For the abnormality detection of Distributed Services, center of gravity is placed on controlling stream graph of the exploitation based on time weighting by this patent
(TCFG) on model.The model development firstly generates controlling stream graph model (CFG), then partitioning boundary optimization structure, afterwards using K
Mean algorithm calculates time weighting.Finally by online log stream compared with the TCFG models excavated, realize that difference is examined
It is disconnected.
To understand the state of development of the prior art, retrieval is carried out to existing paper and patent, has compared and analyzes, has been screened
Go out the higher technical information of following degree related to the present invention：
Technical solution 1：Patent No. CN106209496A's《A kind of communication abnormality detection device and method》Patent is related to
A kind of method of abnormal detector is mainly completed by three steps of device：First, communication unit.Pass through the logical of communication unit
Believe that interface receives multiple data frames.The unit need to include at least one communication interface and at least one communication interface connects including first
Mouthful and second interface to meet the needs of different piece abnormality detection.Second, detection unit includes technical unit and time span
Determination unit.Counting unit is used for data of the mark received to the communication unit for frame end position to labeled as a frame
Data amount check until the data of start bit is counted.Count value of the time length determination unit for according to counting unit is true
Surely the time span between two frame data received.3rd, determination unit, by the time span detected and default time zone
Between compared, when the time span is in the time interval, the communication of the communication unit is normal, otherwise described logical
Believe the communication abnormality of unit and management server is sent to by transmitting element.
Technical solution 2：Patent No. CN106713012A's《A kind of detection method and device of communication abnormality》Patent relates to
And a kind of detection method and device of ATM communication abnormalities, mainly completed by four steps of device：First, according between default transmission
It is interposed between inside ATM and sends heartbeat packet；Second, the real-time Communication for Power load parameter of the corresponding heartbeat response bag of the heartbeat packet is detected,
The traffic load parameter is retardation rate or packet loss；3rd, working as inside the ATM, is determined according to the traffic load parameter
Preceding communications status, the communications status take Pyatyi grading system；4th, export the definite Current Communications Status.The invention
Have the advantages that can detect during no business datum abnormal.
Technical solution 3：Patent No. CN105323017A's《Communication abnormality detection device, communication abnormality detection method with
And program》Patent is related to a kind of communication to being detected in master device and multiple exceptions from the communication carried out between device
Abnormal detector is mainly completed by three steps of device：First, the test section of communication abnormality detection device observes main dress always
It puts and the communication voltage from the order wire between device；Second, notification unit is when communication voltage is less than setting value, except notice should
Outside alarm, ID of the identification as the slave device of communication object is informed about to master device；3rd, master device shows different in display unit
It often shows the maintenance warning of privileged site, and is shown the display unit by the ID slave devices determined, protected to show
Support advance notice.
Technical solution 1 employs a kind of by communication unit, detection unit, these three units of determination unit communicate
The method of abnormality detection, first, communication unit, for receiving multiple data frames；Second, detection unit is described logical for detecting
Believe the time span between the adjacent data frames in multiple data frames that unit receives；3rd, determination unit, for according to detection
The time span that unit detects is compared with default time interval, when the time span is in the time interval
When, the communication of the communication unit is normal, otherwise the communication abnormality of the communication unit.But the abnormality detection be merely able to by
Extremely detection is made caused by time, scope applicatory is also limited.
Technical solution 2 is employed to be detected during no business by sending heartbeat packet and detecting the real-time Communication for Power of its response bag
Mode.Heartbeat packet is sent according to prefixed time interval during no business and detects the real-time Communication for Power of its corresponding heartbeat response bag
Load parameter.Traffic load parameter can reflect the real-time communications status of ATM.The display of state uses grading system, has respectively just
Often, slight congestion, moderate congestion, heavy congestion, Lungs from Non-Heart-Beating this five states.By by traffic load parameter with more than five shapes
The requirement parameter of state, which is compared, would know that Current Communications Status.Finally by communications status output module by the communications status
Output.This method is simply effective, but canonical form is single, also not comprehensive enough with reference to factor, for the scale of the environment used
And condition is also conditional.
Technical solution 3 employs the mode of master and slave devices communication check in a communications system.Detection unit to master device with
At least one communication voltage between device is detected, i.e. rectangular wave buck/boost between reading master and slave devices in order wire
Value detects voltage；Notification unit is detecting that communication voltage is below setting voltage or expects that communication voltage can become
When smaller than setting voltage, in addition to the warning message, master device is informed about to determine the communication object of the communication
Identification information；Master device possesses communication unit, for being notified of warning to the slave device determined by identification information
Information；Possess display unit from device, be notified be notified of warning message when, display be notified of warning message.This side
Method be capable of the decline failure to master device and from the communication voltage in the order wire between device always generation be detected or
Prediction, but detectable failure is only as caused by voltage, functionality is more single.
The content of the invention
For the above situation, to overcome the defect of the prior art, the present invention's is designed to provide based on Distributed Services
The communication network information system method for detecting abnormality of daily record by excavating the service reference in network, finds automatic abnormality detection
When the case that compares, can automatically compare real-time logs stream and exploitation template, achieve the purpose that detection is abnormal.
Its technical solution solved is to detect including daily record parsing, generation TCFG and automatically abnormal three steps, first,
By parsing daily record, mapping and the template of original log and template are obtained, when mapping daily record template, record log template rope
To draw, then timestamp and daily record template generate TCFG, finally by online log stream compared with the TCFG models excavated,
Complete differential diagnostic, which is characterized in that described generation three steps of below TCFG,
S1, CFG side are excavated, in CFG or TCFG, node on behalf daily record template；For node there are one subset, one directly sub
Collection, a time weighting collection；Subset is one group of subsequent node in time window；Direct subset is the direct successor of node
Set；Time weighting collection is time interval set of the node directly between child node；It is the Candidate Set of side right weight；
Wherein there are two key point, one is time t, it is selected across the value of time window；The long time can be with too
More noises, and the short time can miss some correct descendant nodes；On the other hand how to calculate the frequency of child node；
Rule setting is that a child node is only calculated in a time window；
When calculating the frequency of child node, filtering rule is used to determine the direct subsequent node of node；Filtering rule 1
Be intended to based on the frequency of child node come noise filtering；Formula is described as follows；
TnRepresent every template node；N is the occurrence number of daily record every template node；Represent child node, MiIt is the generation of child node
Number；The author sets the filtering threshold of a child node as θ；If It will be by from sub- concentration filter and deleting
It removes；
Filter rules 2 are intended to select direct subsequent node according to the frequency of direct child node；Formula describes such as
Under；
Represent direct child node, QiIt is the frequency of direct child node；The author sets the filtering threshold of direct subset
η；If It will be by from direct sub- concentration filter and deletion；
Afterwards, formula (3) can be used to obtain direct subsequent node；
Ss=Sd∩Sc (3)
Ss,Sd,ScDirect subsequent node collection, subset and direct subset are represented respectively；
For each every template node, once direct subsequent node is calculated, it is confirmed that the side of node；Then, Suo Youjie
Point and its side can form CFG；
S2, conversion stream boundary segmentation, determines whether the time interval between adjacent node is stablized by calculating variance；It is public
Formula (4) is as follows；
When the variance of time interval between two adjacent nodes is more than threshold value, just delete between the two adjacent nodes
Side；Threshold value beta should be very big at this time；
Afterwards check CFG in whether cycling；Method is the variance for the variance for calculating all sides for forming Xun Huan, and deletes
The side of variance maximum；
S3, side time weighting calculate, and the time weighting on each side is represented between two daily record every template nodes in TCFG
Perform the time；Time weighting can be used for detection abnormal, using the time weighting algorithm based on K averages, input as time interval collection D
={ t1,t2,……,tm, cluster numbers K=3, output is time weighting；
The distribution of the time interval list on side is concentrated very much；Most of elements concentrate on a field, and other element is then
It is dispersed in other areas；Therefore time interval list is divided into 3 groups by this patent using K mean algorithms；The most group of quantity is adopted
It receives；Maximum element is time weighting in the group；
The basis of k mean algorithms is the quadratic sum of minimal error；Cost function is：
Wherein, k represents that cluster divides k groups, ciRepresent i-th group of element set, also referred to as x is ciIn arbitrary element, refer to one
A time interval,Represent cluster ciAverage value, E is variance.
Due to the use of above technical scheme, the present invention has the following advantages that compared with prior art；
The technology is studied automatic method for detecting abnormality, it is proposed that be based on based on network service benchmark is excavated
The automatic method for detecting abnormality of TCFG：Original log is analyzed first to generate the template with timestamp；Then, it is every by analyzing
The daily record of a service generates TCFG (time weight control flow chart), and TCFG is responsible for turning for the system that record is run in service
Change state；In addition, according to comparison TCFG, real-time logs throat floater can be diagnosed；Assessment shows the automatic abnormal inspection based on TCFG
Survey method has good accurate rate and recall rate, in true environment, averagely there is 80% accurate rate and 65% recall rate.
Description of the drawings
Fig. 1 is the flow chart of the communication network information system method for detecting abnormality the present invention is based on Distributed Services daily record.
Fig. 2 is the Hadoop days of the communication network information system method for detecting abnormality the present invention is based on Distributed Services daily record
The TCFG of will generation excavates figure.
Fig. 3 is the communication network information system method for detecting abnormality the present invention is based on Distributed Services daily record
The TCFG of Zookeeper daily records generation excavates figure.
Fig. 4 is the TCFG excavations of the communication network information system method for detecting abnormality the present invention is based on Distributed Services daily record
Precision and recall rate figure.
Fig. 5 is the abnormality detection of the communication network information system method for detecting abnormality the present invention is based on Distributed Services daily record
Precision and recall rate figure.
Specific embodiment
For the present invention foregoing and other technology contents, feature and effect, in following cooperation with reference to figures 1 through attached drawing 5
To in the detailed description of embodiment, can clearly present.The structure content being previously mentioned in following embodiment is with specification
Attached drawing is reference.
Embodiment one, the communication network information system method for detecting abnormality based on Distributed Services daily record, including daily record solution
Analysis, generation TCFG and abnormal three steps of automatic detection, first, by parse daily record (daily record parsing be using existing method,
First, by pretreatment, often row daily record according to space is segmented, and rejects the number of the inside, it then will by the method for cluster
Original log be divided into it is multigroup, finally, to one group of daily record according to its similar extraction daily record masterplate, the mapping of production master and daily record,
Obtain mapping and the template of original log and template, when mapping daily record template, record log template index, timestamp and daily record
Then template generates TCFG, finally by online log stream compared with the TCFG models excavated, complete differential diagnostic,
It is characterized in that, described generation three steps of below TCFG,
S1, CFG side are excavated, in CFG or TCFG, node on behalf daily record template；For node there are one subset, one directly sub
Collection, a time weighting collection；Subset is one group of subsequent node in time window；Direct subset is the direct successor of node
Set；Time weighting collection is time interval set of the node directly between child node；It is the Candidate Set of side right weight；
Wherein there are two key point, one is time t, it is selected across the value of time window；The long time can be with too
More noises, and the short time can miss some correct descendant nodes；On the other hand how to calculate the frequency of child node；
Rule setting is that a child node is only calculated in a time window；
When calculating the frequency of child node, filtering rule is used to determine the direct subsequent node of node；Filtering rule 1
Be intended to based on the frequency of child node come noise filtering；Formula is described as follows；
TnRepresent every template node；N is the occurrence number of daily record every template node；Represent child node, MiIt is the generation of child node
Number；The author sets the filtering threshold of a child node as θ；If It will be by from sub- concentration filter and deleting
It removes；
Filter rules 2 are intended to select direct subsequent node according to the frequency of direct child node；Formula describes such as
Under；
Represent direct child node, QiIt is the frequency of direct child node；The author sets the filtering threshold of direct subset
Value η；If It will be by from direct sub- concentration filter and deletion；
Afterwards, formula (3) can be used to obtain direct subsequent node；
Ss=Sd∩Sc (3)
Ss,Sd,ScDirect subsequent node collection, subset and direct subset are represented respectively；
For each every template node, once direct subsequent node is calculated, it is confirmed that the side of node；Then, Suo Youjie
Point and its side can form CFG；
S2, conversion stream boundary segmentation, determines whether the time interval between adjacent node is stablized by calculating variance；It is public
Formula (4) is as follows；
When the variance of time interval between two adjacent nodes is more than threshold value, just delete between the two adjacent nodes
Side；Threshold value beta should be very big at this time；
Afterwards check CFG in whether cycling；Method is the variance for the variance for calculating all sides for forming Xun Huan, and deletes
The side of variance maximum；
S3, side time weighting calculate, and the time weighting on each side is represented between two daily record every template nodes in TCFG
Perform the time；Time weighting can be used for detection abnormal, using the time weighting algorithm based on K averages, input as time interval collection D
={ t1,t2,……,tm, cluster numbers K=3, output is time weighting；
The distribution of the time interval list on side is concentrated very much；Most of elements concentrate on a field, and other element is then
It is dispersed in other areas；Therefore time interval list is divided into 3 groups by this patent using K mean algorithms；The most group of quantity is adopted
It receives；Maximum element is time weighting in the group；
The basis of k mean algorithms is the quadratic sum of minimal error；Cost function is：
Wherein, k represents that cluster divides k groups, ciRepresent i-th group of element set, also referred to as x is ciIn arbitrary element, refer to one
A time interval,Represent cluster ciAverage value, E is variance.
Present invention tool is in use, this patent is assessed using laboratory environment log data set and real log data set
TCFG is excavated and abnormity diagnosis, 70% data are developed, and 30% data are tested for assessing, and data details are as follows：
Hadoop daily record collection：Laboratory environment is Hadoop2.0 versions, two micro services is run, from one in system
The original log of ten days about 1.6GB by a definite date has been selected in micro services as data set；
Zookeeper daily record collection：Zookeeper systems are based on real world, the daily daily record for exporting about 20G, from being
The original log of eight hours about 8GB by a definite date has been selected in micro services in system as data set；
This patent assesses TCFG excavations and abnormity diagnosis, accuracy and recall rate definition by accuracy and recall rate
It is as follows.
Really really (TP) shows correct case being considered as correctly, and false correct (FP) shows wrong case being considered as correctly,
False error misses (FN) and shows correct case being considered as mistake.
Design parameter sets as shown in table 1.Depth is the depth of blog search tree, and similar threshold value is the similar of daily record template
Threshold value.
1 parameter setting of table
Fig. 2 and Fig. 3 is that have bifurcated and linear sequence from the TCFG of the Web log mining of Hadoop and Zookeeper, figure,
TCFG and abnormality diagnostic efficiency are assessed using precision/recall rate index of standard afterwards, this patent first passes through source generation
Code manually generates a benchmark TCFG structure, some abnormal logs are randomly with the addition of into test set；
It is the abnormity diagnosis side based on the mixing graph model (HGM) with daily record with the algorithm compared with this patent algorithm
Method, in this patent, for partitioning boundary, the execution for each step in ratio task that the interval time between two tasks is set
Time is all much longer, and therefore, the author highlights its advantage using HGM in an experiment；
From fig. 4, it can be seen that the precision of both algorithms is all very high, and recall rate is less than precision.This is because segmentation
Strategy is unable to divided ownership border, in order to filter all noises, the threshold value in boundary segmentation strategy be set it is larger, on the contrary,
If the too small normal side of threshold value can be also filtered, this patent algorithm has recalled 80% in Hadoop, higher than HGM algorithms
75%.Meanwhile this patent algorithm has recalled 75% in Zookeeper, higher than the 65% of HGM algorithms, thus, it will be seen that
Boundary segmentation strategy is better than HGM algorithms, in addition, the superiority of strategy is also be reflected in abnormity diagnosis；As shown in figure 5, this patent
Algorithm has 70% to recall in Hadoop, higher than the 65% of HGM algorithms, in addition, the algorithm is recalled in Zookeeper
65%, higher than the 50% of HGM algorithms.It may be concluded that the time weighting algorithm based on K averages also plays certain work
With.
The above is to combine specific embodiment further description made for the present invention, it is impossible to assert the present invention
Specific implementation is limited only to this；For belonging to the present invention and for those skilled in the technology concerned, based on skill of the present invention
Under the premise of art scheme thinking, the expansion made and operating method, the replacement of data, should all fall the scope of the present invention it
It is interior.
Claims (1)
1. the communication network information system method for detecting abnormality based on Distributed Services daily record, including daily record parsing, generation TCFG
With abnormal three steps of automatic detection, first, by parsing daily record, mapping and the template of original log and template are obtained, is reflected
When penetrating daily record template, record log template index, then timestamp and daily record template generate TCFG, finally by online log stream
Compared with the TCFG models excavated, differential diagnostic is completed, which is characterized in that described generation three steps of below TCFG,
S1, CFG side are excavated, in CFG or TCFG, node on behalf daily record template；Subset that there are one nodes, a direct subset,
One time weighting collection；Subset is one group of subsequent node in time window；Direct subset is the direct successor of node
Set；Time weighting collection is time interval set of the node directly between child node；It is the Candidate Set of side right weight；
Wherein there are two key point, one is time t, it is selected across the value of time window；Long time can be with too many
Noise, and the short time can miss some correct descendant nodes；On the other hand how to calculate the frequency of child node；Rule
It is arranged in a time window only calculate a child node；
When calculating the frequency of child node, filtering rule is used to determine the direct subsequent node of node；Filtering rule 1 is intended to
Based on the frequency of child node come noise filtering；Formula is described as follows；
<mrow>
<mi>P</mi>
<mrow>
<mo>(</mo>
<msub>
<mi>T</mi>
<msub>
<mi>c</mi>
<mi>i</mi>
</msub>
</msub>
<mo>|</mo>
<msub>
<mi>T</mi>
<mi>n</mi>
</msub>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mfrac>
<msub>
<mi>M</mi>
<mi>i</mi>
</msub>
<mi>N</mi>
</mfrac>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
</mrow>
TnRepresent every template node；N is the occurrence number of daily record every template node；Represent child node, MiIt is the generation time of child node
Number；The author sets the filtering threshold of a child node as θ；If It will be by from sub- concentration filter and deleting
It removes；
Filter rules 2 are intended to select direct subsequent node according to the frequency of direct child node；Formula is described as follows；
<mrow>
<mi>P</mi>
<mrow>
<mo>(</mo>
<msub>
<mi>T</mi>
<msub>
<mi>d</mi>
<mi>i</mi>
</msub>
</msub>
<mo>|</mo>
<msub>
<mi>T</mi>
<mi>n</mi>
</msub>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mfrac>
<msub>
<mi>Q</mi>
<mi>i</mi>
</msub>
<mi>N</mi>
</mfrac>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</mrow>
Represent direct child node, QiIt is the frequency of direct child node；The author sets the filtering threshold η of direct subset；Such as
Fruit It will be by from direct sub- concentration filter and deletion；
Afterwards, formula (3) can be used to obtain direct subsequent node；
Ss=Sd∩Sc (3)
Ss,Sd,ScDirect subsequent node collection, subset and direct subset are represented respectively；
For each every template node, once direct subsequent node is calculated, it is confirmed that the side of node；Then, all nodes and
Its side can form CFG；
S2, conversion stream boundary segmentation, determines whether the time interval between adjacent node is stablized by calculating variance；Formula
(4) it is as follows；
<mrow>
<mi>M</mi>
<mo>=</mo>
<mfrac>
<mrow>
<msub>
<mi>t</mi>
<mn>1</mn>
</msub>
<mo>+</mo>
<msub>
<mi>t</mi>
<mn>2</mn>
</msub>
<mo>+</mo>
<msub>
<mi>t</mi>
<mn>3</mn>
</msub>
<mo>+</mo>
<mo>...</mo>
<mo>+</mo>
<msub>
<mi>t</mi>
<mi>n</mi>
</msub>
</mrow>
<mi>n</mi>
</mfrac>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>4</mn>
<mo>)</mo>
</mrow>
</mrow>
<mrow>
<mi>var</mi>
<mo>=</mo>
<mfrac>
<mrow>
<msup>
<mrow>
<mo>(</mo>
<mi>M</mi>
<mo>-</mo>
<msub>
<mi>t</mi>
<mn>1</mn>
</msub>
<mo>)</mo>
</mrow>
<mn>2</mn>
</msup>
<mo>+</mo>
<msup>
<mrow>
<mo>(</mo>
<mi>M</mi>
<mo>-</mo>
<msub>
<mi>t</mi>
<mn>2</mn>
</msub>
<mo>)</mo>
</mrow>
<mn>2</mn>
</msup>
<mo>+</mo>
<msup>
<mrow>
<mo>(</mo>
<mi>M</mi>
<mo>-</mo>
<msub>
<mi>t</mi>
<mn>3</mn>
</msub>
<mo>)</mo>
</mrow>
<mn>2</mn>
</msup>
<mo>+</mo>
<mo>...</mo>
<mo>+</mo>
<msup>
<mrow>
<mo>(</mo>
<mi>M</mi>
<mo>-</mo>
<msub>
<mi>t</mi>
<mi>n</mi>
</msub>
<mo>)</mo>
</mrow>
<mn>2</mn>
</msup>
</mrow>
<mrow>
<mi>n</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
</mfrac>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>5</mn>
<mo>)</mo>
</mrow>
</mrow>
When the variance of time interval between two adjacent nodes is more than threshold value, the side between the two adjacent nodes is just deleted；
Threshold value beta should be very big at this time；
Afterwards check CFG in whether cycling；Method is the variance for the variance for calculating all sides for forming Xun Huan, and deletes variance
Maximum side；
S3, side time weighting calculate, and the time weighting on each side represents the execution between two daily record every template nodes in TCFG
Time；Time weighting can be used for detection abnormal, using the time weighting algorithm based on K averages, input as time interval collection D=
{t1,t2,……,tm, cluster numbers K=3, output is time weighting；
The distribution of the time interval list on side is concentrated very much；Most of elements concentrate on a field, and other element is then disperseed
In other areas；Therefore time interval list is divided into 3 groups by this patent using K mean algorithms；The most group of quantity is adopted；It should
Maximum element is time weighting in group；
The basis of k mean algorithms is the quadratic sum of minimal error；Cost function is：
<mrow>
<mi>E</mi>
<mo>=</mo>
<munderover>
<mo>&Sigma;</mo>
<mrow>
<mi>i</mi>
<mo>=</mo>
<mn>1</mn>
</mrow>
<mi>k</mi>
</munderover>
<munder>
<mo>&Sigma;</mo>
<mrow>
<mi>x</mi>
<mo>&Element;</mo>
<msub>
<mi>c</mi>
<mi>i</mi>
</msub>
</mrow>
</munder>
<mo>|</mo>
<mo>|</mo>
<mi>x</mi>
<mo>-</mo>
<msub>
<mi>u</mi>
<mi>i</mi>
</msub>
<mo>|</mo>
<msubsup>
<mo>|</mo>
<mn>2</mn>
<mn>2</mn>
</msubsup>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>6</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, k represents that cluster divides k groups, ciRepresent i-th group of element set, also referred to as x is ciIn arbitrary element, when referring to one
Between be spaced,Represent cluster ciAverage value, E is variance.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711309778.0A CN108055152B (en) | 2017-12-11 | 2017-12-11 | Communication network information system abnormity detection method based on distributed service log |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711309778.0A CN108055152B (en) | 2017-12-11 | 2017-12-11 | Communication network information system abnormity detection method based on distributed service log |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108055152A true CN108055152A (en) | 2018-05-18 |
| CN108055152B CN108055152B (en) | 2020-11-06 |
Family
ID=62123932
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711309778.0A Active CN108055152B (en) | 2017-12-11 | 2017-12-11 | Communication network information system abnormity detection method based on distributed service log |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108055152B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109634699A (en) * | 2018-11-23 | 2019-04-16 | 中国人民解放军63789部队 | A kind of visualization of Spacecraft anomaly information cluster and exchange method based on Sang Jitu |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112000085B (en) * | 2020-09-08 | 2021-08-31 | 广州云也科技有限公司 | Network transmission performance diagnosis system based on 5G big data |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030117279A1 (en) * | 2001-12-25 | 2003-06-26 | Reiko Ueno | Device and system for detecting abnormality |
| CN103514398A (en) * | 2013-10-18 | 2014-01-15 | 中国科学院信息工程研究所 | Real-time online log detection method and system |
| CN106209405A (en) * | 2015-05-06 | 2016-12-07 | 中国移动通信集团内蒙古有限公司 | Method for diagnosing faults and device |
| CN107070726A (en) * | 2017-05-22 | 2017-08-18 | 郑州云海信息技术有限公司 | A kind of integrated management approach based on MDC |
| US20170302536A1 (en) * | 2016-04-15 | 2017-10-19 | Arista Networks, Inc. | Method and system for detecting abnormalities in network element operation |
-
2017
- 2017-12-11 CN CN201711309778.0A patent/CN108055152B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030117279A1 (en) * | 2001-12-25 | 2003-06-26 | Reiko Ueno | Device and system for detecting abnormality |
| CN103514398A (en) * | 2013-10-18 | 2014-01-15 | 中国科学院信息工程研究所 | Real-time online log detection method and system |
| CN106209405A (en) * | 2015-05-06 | 2016-12-07 | 中国移动通信集团内蒙古有限公司 | Method for diagnosing faults and device |
| US20170302536A1 (en) * | 2016-04-15 | 2017-10-19 | Arista Networks, Inc. | Method and system for detecting abnormalities in network element operation |
| CN107070726A (en) * | 2017-05-22 | 2017-08-18 | 郑州云海信息技术有限公司 | A kind of integrated management approach based on MDC |
Non-Patent Citations (1)
| Title |
|---|
| 李文萃: "《基于人工智能的电网企业信息安全防控体系》", 《电力信息与通信技术》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109634699A (en) * | 2018-11-23 | 2019-04-16 | 中国人民解放军63789部队 | A kind of visualization of Spacecraft anomaly information cluster and exchange method based on Sang Jitu |
| CN109634699B (en) * | 2018-11-23 | 2022-04-05 | 中国人民解放军63789部队 | Spacecraft abnormal information clustering visualization and interaction method based on mulberry-based map |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108055152B (en) | 2020-11-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Root-cause metric location for microservice systems via log anomaly detection | |
| CN107943668A (en) | Computer server cluster daily record monitoring method and monitor supervision platform | |
| US11748227B2 (en) | Proactive information technology infrastructure management | |
| US7079010B2 (en) | System and method for monitoring processes of an information technology system | |
| US20210051203A1 (en) | Systems and Methods for Operation Management and Monitoring of Bots | |
| US8533536B2 (en) | Monitoring data categorization and module-based health correlations | |
| CN105976120A (en) | Electric power operation monitoring data quality assessment system and method | |
| CN107707431A (en) | The data safety monitoring method and system of a kind of facing cloud platform | |
| EP4020218B1 (en) | Analyzing large-scale data processing jobs | |
| CN105930255B (en) | A kind of system health degree prediction technique and device | |
| CN111541559A (en) | Fault positioning method based on causal rule | |
| CN115756929B (en) | Abnormal root cause positioning method and system based on dynamic service dependency graph | |
| CN116166505B (en) | Monitoring platform, method, storage medium and equipment for dual-state IT architecture in financial industry | |
| CN105122733A (en) | Queue monitoring and visualization | |
| CN108055152A (en) | Communication network information system method for detecting abnormality based on Distributed Services daily record | |
| CN114281877A (en) | Data management system and method | |
| Mart et al. | Observability in kubernetes cluster: Automatic anomalies detection using prometheus | |
| Ai et al. | A software network model for software structure and faults distribution analysis | |
| US11815989B2 (en) | Automated methods and systems for identifying problems in data center objects | |
| Zhu et al. | A Performance Fault Diagnosis Method for SaaS Software Based on GBDT Algorithm. | |
| CN109634808A (en) | A kind of chain type monitor event root cause analysis method based on association analysis | |
| CN113469559A (en) | Quality bit design and display method and system based on data quality inspection | |
| CN101453383B (en) | Device, network status analysis method and apparatus | |
| CN110187997A (en) | A kind of disk method for monitoring abnormality, device, equipment and medium | |
| Horovitz et al. | Online Automatic Characteristics Discovery of Faulty Application Transactions in the Cloud. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Shu Xinjian Inventor after: Xu Siya Inventor after: Sheng Lei Inventor after: Yue Gangyi Inventor after: Zhang Ningning Inventor after: Dong Gangsong Inventor after: Guo Shaoyong Inventor after: Jin Yi Inventor after: Wang Shiwen Inventor after: An Zhiyuan Inventor after: Luo Zhen Inventor after: Zhang Xiaojun Inventor after: Li Wencui Inventor before: Shu Xinjian Inventor before: Wang Shiwen Inventor before: An Zhiyuan Inventor before: Luo Zhen Inventor before: Zhang Xiaojun Inventor before: Li Wencui Inventor before: Sheng Lei Inventor before: Yue Gangyi Inventor before: Zhang Ningning |
|
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20201014 Address after: 450000 building C, office area, 87 South Road, 27 District, Henan, Songshan, Zhengzhou Applicant after: State Grid Henan electric power company information communication Co. Applicant after: Beijing University of Posts and Telecommunications Applicant after: State Grid Corporation of China Address before: 450000 building C, office building, 87 South Road, Zhengzhou, Henan, Songshan Applicant before: State Grid Henan electric power company information communication Co. Applicant before: State Grid Corporation of China |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |