CN110032494A - A kind of double grains degree noise log filter method based on incidence relation - Google Patents

A kind of double grains degree noise log filter method based on incidence relation Download PDF

Info

Publication number
CN110032494A
CN110032494A CN201910218832.3A CN201910218832A CN110032494A CN 110032494 A CN110032494 A CN 110032494A CN 201910218832 A CN201910218832 A CN 201910218832A CN 110032494 A CN110032494 A CN 110032494A
Authority
CN
China
Prior art keywords
event
degree
log
noise
track
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910218832.3A
Other languages
Chinese (zh)
Other versions
CN110032494B (en
Inventor
孙笑笑
俞东进
侯文杰
潘建梁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Hangzhou Electronic Science and Technology University
Original Assignee
Hangzhou Electronic Science and Technology University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Electronic Science and Technology University filed Critical Hangzhou Electronic Science and Technology University
Priority to CN201910218832.3A priority Critical patent/CN110032494B/en
Publication of CN110032494A publication Critical patent/CN110032494A/en
Application granted granted Critical
Publication of CN110032494B publication Critical patent/CN110032494B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
    • G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention discloses a kind of double grains degree noise log filter method based on incidence relation.This method is based on local dependence degree and mixed dependence degree is calculated in global dependency degree, and the coarseness of the fine granularity filtering and noise track that can be achieved at the same time noise event in log by the method for the invention filters.Compared to traditional log filter method, the present invention has following income: 1, using double grains degree strobe utility, different strobe utilities is used for different noise situations, to realize outstanding filter effect in the case where retaining log data as far as possible；2, the precision of process discovery model can be greatly improved for digging flow using filtered journal file, enhance the comprehensibility of model.

Description

A kind of double grains degree noise log filter method based on incidence relation
Technical field
The present invention relates to digging flow field more particularly to a kind of double grains degree noise log filtering sides based on incidence relation Method.
Background technique
Digging flow is intended to extract useful information from the event log that process apperception information system records to help benefit Beneficial relative understands the practical executive condition of process.Process finds the pith as digging flow, and its role is to construct It can be with the procedural model of recurring events log recording behavior.High-precision model can intuitively show actually holding for operation flow Market condition.
In business process management system, the activity of operation flow is executed according to well-designed procedural model, this A little movable execution will be recorded in log, to help stakeholder participation and monitor the execution of process.In actual life In, most of operation flows all without standardized procedural model, or with the continuous evolution process mould of operation flow Type and current operation flow are there are biggish difference, therefore the log that people need to generate by process discovery technique from process The middle practical process performing for extracting process.But noise present in log can find that the quality of model generates negatively to process It influences.If process for using discovery technique carries out process to the log comprising noise and finds that will lead to it finds that model generation can not Task and non-free selection structure are seen, to increase the complexity and comprehensibility of mining model.Common log noise has Following several classes: deletion form noise event (certain events in process are not recorded in log for some reason), redundancy-type Noise event (certain events are repeated as many times as required record in process), dislocation type noise event (send out in flow path track by certain events Raw ordinal position is recorded mistake).
Noise filtering algorithm can effectively filter out the noise event in log, greatly improve the essence of process discovery model Degree.Current log noise filtering algorithm can be roughly divided into two classes, coarseness filtering and fine granularity filtering according to its grade of filtration. Wherein the track comprising noise event is directly removed original log by coarseness filtering, but daily record data lesser for scale For remove whole track and may generate biggish change to the model structure of excavation.Fine granularity filtering is then only by noise event It removes, retains other events on the track, but cannot be guaranteed that the behavior will not be track while removing noise event New noise is brought, while such algorithm can not also solve the problems, such as that deletion form noise event generates.
Summary of the invention
In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a kind of double grains degree noise log based on incidence relation Filter method can effectively solve the above problems.The present invention it is specific the technical solution adopted is as follows:
A kind of double grains degree noise log filter method based on incidence relation, comprising the following steps:
(1) raw log files are inputted, generate one be made of a plurality of flow path track σ after data prediction is carried out to it Log setEvery flow path track is by multiple flow events eiForm σ=< e1,…,en>, remember in all flow path tracks The collection of all flow events e is combined into ε, i.e. e ∈ ε；
(2) statistical log setIn the frequency dependency degree DFD between flow events two-by-two in all flow path tracks (ei,ej)；
(3) the local dependence degree Dep between event two-by-two is further calculated outlocal(ei,ej), global dependency degree Depglobal(ei,ej) and mixed dependence degree Depmixed(ei,ej)；
The local dependence degree Deplocal(ei,ej) calculation formula is as follows:
Wherein C1、C2For constant, Dsuc(ei) indicate subsequent density, i.e. event eiAll succeeding events occur average frequency It is secondary；Dpre(ej) indicate forerunner's density, for indicating event ejAll forerunner's events occur the average frequency；Subsequent density is with before The calculation formula for driving density is as follows:
Dpre(ek)=Npre(ek)/|Upre(ek)|
Dsuc(ek)=Nsuc(ek)/|Usuc(ek)
Wherein Dpre(ek) it is event ekForerunner's density, Dsuc(ek) it is event ekSubsequent density, Npre(ek) it is with event ekFor the quantity for following relationship of succeeding events, Nsuc(ek) it is event ekFor the quantity for following relationship of precursor event, Upre(ek) For event ekForerunner set, | Upre(ek) | it is event ekForerunner set in event number, Usuc(ek) it is event ek's Successor set, | Usuc(ek) | event ekSuccessor set in event number；
The overall situation dependency degree Depglobal(ei,ej) calculation formula is as follows:
θ=Max { DFD (ex,ey)}
Wherein ζ is the global noise factor, for dividing global noise event.
The mixed dependence degree Depmixed(ei,ej) calculation formula is as follows:
Depmixed(ei,ej)=α * Deplocal(ei,ej)+(1-α)*Depglobal(ei,ej)
Wherein α weighting factor, for balancing the occupation ratio of global dependency degree and local dependency degree.
(4) according to the mixed dependence degree building log set calculated in previous stepIn all flow events mixing Rely on matrix
(5) carry out log noise filtering, comprising the following steps:
51) an empty log collection is constructedFor storing filtered track；
52) log collection is taken outA track σ, by the abandonment value of σIt is initialized as 1；
53) the beginning event e of σ is obtainedstartAnd event e will be startedstartIt is added to an empty sequence of events σfilterIn；
54) current event e is taken out according to the sequence of events in σi；
55) the next event e of current event in track is taken outi+1；
56) existIn search eiAnd ei+1Mixed dependence degree Depmixed(ei,ei+1), the thin of event is first carried out Granularity filter operation, if Depmixed(ei,ei+1) value be not less than degree of mixing threshold value beta, event ei+1Normal event is judged as, It is added to track σfilter, ei+1As current event, subscript i=i+1, and return step 55)；If Depmixed(ei, ei+1) value be less than degree of mixing threshold value beta, event ei+1It is judged as noise event, uses the abandonment value of penalty modification track σPenalty formula is as follows:
WhereinFor penalty factor, the punishment dynamics of penalty are determined；
If revised abandonment value is not less than the abandonment threshold value of settingThen return step 55)；If revised Abandonment valueLower than abandonment threshold valueThen the coarseness filter operation of execution track, track σ are judged as making an uproar Soundtrack mark, return step 52)；
If 57) event ei+1For the End Event e of current track σend, then by filter footprint σfilterIt is added to filtering day Will collectionIn；
58) step 52)~step 57) is repeated, until all tracks that original log is concentrated are removed；
59) filtering log collection is exported
(6) according to the filtering log collection of outputRegenerate journal file.
Preferably, log set described in step (1)It contains the whole of operation flow and executes example, i.e., Every flow path track σ therein is corresponding with the primary execution example of operation flow, and the flow path track σ is by multiple processes The ordered sequence of event e composition, the flow events e are that operation flow executes movable primary record.
Preferably, frequency dependency degree DFD (e described in step (2)i,ej) indicate degree of following directly after, i.e., in whole streams Event e in journey examplejFollow event e closelyiThe total frequency occurred.
Preferably, global noise factor ζ described in step (3) takes 0.02.
Preferably, degree of mixing threshold value beta described in step (5) takes 0.5.
Preferably, weighting factor value α described in step (5) takes 0.5.
Preferably, penalty factor described in step (5)Take 0.8.
Preferably, abandonment threshold value described in step (5)Take 0.7.
Filter method proposed by the present invention considers the dependence between event from global and local two angles, and with this Judge whether event is noise event.Compared to traditional log filter method, the present invention has following income: 1, using double Granularity strobe utility uses different strobe utilities for different noise situations, thus retaining original log number as far as possible Outstanding filter effect is realized in the case where；2, stream can be greatly improved for digging flow using filtered journal file The precision of Cheng Faxian model enhances the comprehensibility of model.
Detailed description of the invention
Fig. 1 is that the present invention is based on the flow charts of the double grains degree noise log filter method of incidence relation；
Fig. 2 is the example schematic of noise filtering of the present invention.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and It is not used in the restriction present invention.
On the contrary, the present invention covers any substitution done on the essence and scope of the present invention being defined by the claims, repairs Change, equivalent method and scheme.Further, in order to make the public have a better understanding the present invention, below to of the invention thin It is detailed to describe some specific detail sections in section description.Part without these details for a person skilled in the art The present invention can also be understood completely in description.
As shown in Figure 1, a kind of double grains degree noise log filter method based on incidence relation of the invention, including following step It is rapid:
(1) raw log files are inputted, generate one be made of a plurality of flow path track σ after data prediction is carried out to it Log setEvery flow path track is by multiple flow events eiForm σ=< e1,…,en>, remember in all flow path tracks The collection of all flow events e is combined into ε, i.e. e ∈ ε；
Log setContain the whole of operation flow and execute examples, i.e., every flow path track σ therein all with The primary execution example of operation flow is corresponding, the ordered sequence that every flow path track σ is made of multiple flow events e, stream Journey event e is that operation flow executes movable primary record.
(2) statistical log setIn the frequency dependency degree DFD between flow events two-by-two in all flow path tracks (ei,ej)。
Frequency dependency degree DFD (ei,ej) indicate degree of following directly after, i.e., the event e in whole flow instancesjFollow event e closelyi The total frequency occurred.
(3) the local dependence degree Dep between event two-by-two is further calculated outlocal(ei,ej), global dependency degree Depglobal(ei,ej) and mixed dependence degree Depmixed(ei,ej)；
The local dependence degree Deplocal(ei,ej) calculation formula is as follows:
Wherein C1、C2For constant, Dsuc(ei) indicate subsequent density, i.e. event eiAll succeeding events occur average frequency It is secondary；Dpre(ej) indicate forerunner's density, for indicating event ejAll forerunner's events occur the average frequency；Subsequent density is with before The calculation formula for driving density is as follows:
Dpre(ek)=Npre(ek)/|Upre(ek)|
Dsuc(ek)=Nsuc(ek)/|Usuc(ek)|
Wherein Dpre(ek) it is event ekForerunner's density, Dsuc(ek) it is event ekSubsequent density, Npre(ek) it is with event ekFor the quantity for following relationship of succeeding events, Nsuc(ek) it is event ekFor the quantity for following relationship of precursor event, Upre(ek) For event ekForerunner set, | Upre(ek) | it is event ekForerunner set in event number, Usuc(ek) it is event ek's Successor set, | Usuc(ek) | event ekSuccessor set in event number；
The overall situation dependency degree Depglobal(ei,ej) calculation formula is as follows:
θ=Max { DFD (ex,ey)}
Wherein ζ takes 0.02 for dividing global noise event for the global noise factor.
The mixed dependence degree Depmixed(ei,ej) calculation formula is as follows:
Depmixed(ei,ej)=α * Deplocal(ei,ej)+(1-α)*Depglobal(ei,ej)
Wherein α weighting factor takes 0.5 for balancing the occupation ratio of global dependency degree and local dependency degree.
(4) according to the mixed dependence degree building log set calculated in previous stepIn all flow events mixing Rely on matrix
(5) carry out log noise filtering, comprising the following steps:
51) an empty log collection is constructedFor storing filtered track；
52) log collection is taken outA track σ, by the abandonment value of σIt is initialized as 1；
53) the beginning event e of σ is obtainedstartAnd event e will be startedstartIt is added to an empty sequence of events σfilterIn；
54) current event e is taken out according to the sequence of events in σi；
55) the next event e of current event in track is taken outi+1；
56) existIn search eiAnd ei+1Mixed dependence degree Depmixed(ei,ei+1), the thin of event is first carried out Granularity filter operation, if Depmixed(ei,ei+1) value be not less than degree of mixing threshold value beta (taking 0.5), event ei+1It is determined and is positive Ordinary affair part is added to track σfilter, ei+1As current event, subscript i=i+1, and return step 55)；If Depmixed(ei,ei+1) value be less than degree of mixing threshold value beta, event ei+1It is judged as noise event, modifies rail using penalty The abandonment value of mark σPenalty formula is as follows:
WhereinFor penalty factor, determines the punishment dynamics of penalty, take 0.8；
If revised abandonment value is not less than the abandonment threshold value of settingThen return step 55)；If revised Abandonment valueLower than abandonment threshold valueThen the coarseness filter operation of execution track, track σ are judged as making an uproar Soundtrack mark, return step 52).Abandon threshold valueTake 0.7.
If 57) event ei+1For the End Event e of current track σend, then by filter footprint σfilterIt is added to filtering day Will collectionIn；
58) step 52)~step 57) is repeated, until all tracks that original log is concentrated are removed；
59) filtering log collection is exported
(6) according to the filtering log collection of outputRegenerate journal file.
Below based on above method process, its technical effect is further shown by embodiment.
Embodiment
The present embodiment step is identical as specific embodiment abovementioned steps, is no longer repeated herein.Just part is real below It applies process and result of implementation is shown:
Data source obtains: raw log files used in the present embodiment read journal file using java kit JDOM, The root node root of log document is obtained, the child node element of entitled Process under root node is obtained, further obtains again Whole child node elements of entitled ProcessInstance under Process node.One ProcessInstance node includes Process once executes all information of example, it usually possesses the node elements of multiple entitled AuditTrailEntry, process The details of each event occurred in example are recorded in an AuditTrailEntry node elements, these AuditTrailEntry node contains many event attributes, such as timestamp attribute, event name attribute, Resource Properties etc..To this A little event informations are screened and are rejected the event name attribute that event is remained after redundancy therein, and by same instance Event finally saves as a flow path track σ=< e according to time started stamp attribute sequence1,…,en>, and the track is opposite The id attribute for the ProcessInstance node elements answered assigns the track as its track id, by track whole in log More collection, that is, original log the collection constitutedIt saves.
Fig. 2 is illustrated in detail carries out based on incidence relation two tracks (example 1 and example 2) with the method for the present invention The detailed process of double grains degree noise log filtering:
1 track σ of example1=<ABCDEFGH>
1) σ is obtained1Beginning event A and add it to sky track sets σfIn；
2) the next event B for taking out event A, calculates the mixed interconnection degree Dep of event ABmixed(A, B)=0.80 is greater than mixed Right threshold value 0.5, therefore event B is normal event (non-noise event), adds it to sequence σfIn；
3) the next event C for taking out event B, calculates the mixed interconnection degree Dep of event BCmixed(B, C)=0.75 is greater than mixed Right threshold value 0.5, therefore event C is normal event (non-noise event), adds it to sequence σfIn；
4) the next event D for taking out event C, calculates the mixed interconnection degree Dep of event CDmixed(C, D)=0.85 is greater than mixed Right threshold value 0.5, therefore event D is normal event (non-noise event), adds it to sequence σfIn；
5) the next event E for taking out event D, calculates the mixed interconnection degree Dep of event DEmixed(D, E)=0.87 is greater than mixed Right threshold value 0.5, therefore event E is normal event (non-noise event), adds it to sequence σfIn；
6) the next event F for taking out event E, calculates the mixed interconnection degree Dep of event EFmixed(E, F)=0.26, small mixing Threshold value 0.5 is spent, therefore event F is noise event, does not add it to sequence σfIn；Track σ is modified using penalty1Something lost Abandoning valueBeing calculated is 0.9, is greater than and abandons threshold value 0.7, therefore σ1For normal trace (non-noise track)；
7) the next event G for taking out event F, calculates the mixed interconnection degree Dep of event EGmixed(E, G)=0.87 is greater than mixed Right threshold value 0.5, therefore event G is normal event (non-noise event), adds it to sequence σfIn；
8) the next event H for taking out event G, calculates the mixed interconnection degree Dep of event GHmixed(G, H)=0.85 is greater than mixed Right threshold value 0.5, therefore event H is normal event (non-noise event), adds it to sequence σfIn；
9) event H is current track σ1End Event, then track after being filtered using this method is σf=< ABCDEGH >, add it to filtering log concentration.
2 track σ of example2=<ABCEGH>
1) σ is obtained2Beginning event A and add it to sky track sets σfIn；
2) the next event B for taking out event A, calculates the mixed interconnection degree Dep of event ABmixed(A, B)=0.80 is greater than mixed Right threshold value 0.5, therefore event B is normal event (non-noise event), adds it to sequence σfIn；
3) the next event C for taking out event B, calculates the mixed interconnection degree Dep of event BCmixed(B, C)=0.75 is greater than mixed Right threshold value 0.5, therefore event C is normal event (non-noise event), adds it to sequence σfIn；
4) the next event E for taking out event C, calculates the mixed interconnection degree Dep of event CEmixed(C, E)=0.26 is less than mixed Right threshold value 0.5, therefore event E is noise event, does not add it to sequence σfIn；Track σ is modified using penalty2's Abandonment valueBeing calculated is 0.9, is greater than and abandons threshold value 0.7, therefore σ2For normal trace (non-noise track)；
5) the next event G for taking out event E, calculates the mixed interconnection degree Dep of event CGmixed(C, G)=0.01 is less than mixed Right threshold value 0.5, therefore event G is noise event, does not add it to sequence σfIn；Track σ is modified using penalty2's Abandonment valueBeing calculated is 0.72, is greater than and abandons threshold value 0.7, therefore σ2For normal trace (non-noise track)；
The next event H of taking-up event G calculates the mixed interconnection degree Dep of event CHmixed(C, H)=0.01 is less than mixing Threshold value 0.5 is spent, therefore event H is noise event, does not add it to sequence σfIn；
Track σ is modified using penalty2Abandonment valueBeing calculated is 0.58, is less than and abandons threshold value 0.7, Therefore σ2For noise track, noise log concentration is not added it to.

Claims (8)

1. a kind of double grains degree noise log filter method based on incidence relation, it is characterised in that the following steps are included:
(1) raw log files are inputted, generate a log being made of a plurality of flow path track σ after data prediction is carried out to it SetEvery flow path track is by multiple flow events eiForm σ=< e1..., en>, remember in all flow path tracks and owns The collection of flow events e is combined into ε, i.e. e ∈ ε；
(2) statistical log setIn the frequency dependency degree DFD (e between flow events two-by-two in all flow path tracksi, ej)；
(3) the local dependence degree Dep between event two-by-two is further calculated outlocal(ei, ej), global dependency degree Depglobal(ei, ej) and mixed dependence degree Depmixed(ei, ej)；
The local dependence degree Deplocal(ei, ej) calculation formula is as follows:
Wherein C1、C2For constant, Dsuc(ei) indicate subsequent density, i.e. event eiAll succeeding events occur the average frequency； Dpre(ej) indicate forerunner's density, for indicating event ejAll forerunner's events occur the average frequency；Subsequent density and forerunner The calculation formula of density is as follows:
Dpre(ek)=Npre(ek)/|Upre(ek)|
Dsuc(ek)=Nsuc(ek)/|Usuc(ek)|
Wherein Dpre(ek) it is event ekForerunner's density, Dsuc(ek) it is event ekSubsequent density, Npre(ek) it is with event ekFor The quantity for following relationship of succeeding events, Nsuc(ek) it is event ekFor the quantity for following relationship of precursor event, Upre(ek) it is thing Part ekForerunner set, | Upre(ek) | it is event ekForerunner set in event number, Usuc(ek) it is event ekIt is subsequent Set, | Usuc(ek) | event ekSuccessor set in event number；
The overall situation dependency degree Depglobal(ei, ej) calculation formula is as follows:
θ=Max { DFD (ex, ey)}
Wherein ζ is the global noise factor, for dividing global noise event.
The mixed dependence degree Depmixed(ei, ej) calculation formula is as follows:
Depmixed(ei, ej)=α * Deplocal(ei, ej)+(1-α)*Depglobal(ei, ej)
Wherein α weighting factor, for balancing the occupation ratio of global dependency degree and local dependency degree.
(4) according to the mixed dependence degree building log set calculated in previous stepIn all flow events mixed dependence Matrix
(5) carry out log noise filtering, comprising the following steps:
51) an empty log collection is constructedFor storing filtered track；
52) log collection is taken outA track σ, by the abandonment value of σIt is initialized as 1；
53) the beginning event e of σ is obtainedstartAnd event e will be startedstartIt is added to an empty sequence of events σfilterIn；
54) current event e is taken out according to the sequence of events in σi；
55) the next event e of current event in track is taken outi+1；
56) existIn search eiAnd ei+1Mixed dependence degree Depmixed(ei, ei+1), the particulate that event is first carried out is spent Filter operation, if Depmixed(ei, ei+1) value be not less than degree of mixing threshold value beta, event ei+1It is judged as normal event, is added It is added to track σfilter, ei+1As current event, subscript i=i+1, and return step 55)；If Depmixed(ei, ei+1) value Less than degree of mixing threshold value beta, event ei+1It is judged as noise event, uses the abandonment value of penalty modification track σ Penalty formula is as follows:
WhereinFor penalty factor, the punishment dynamics of penalty are determined；
If revised abandonment value is not less than the abandonment threshold value of settingThen return step 55)；If revised abandonment valueLower than abandonment threshold valueThen the coarseness filter operation of execution track, track σ are judged as noise track, Return step 52)；
If 57) event ei+1For the End Event e of current track σend, then by filter footprint σfilterIt is added to filtering log collectionIn；
58) step 52)~step 57) is repeated, until all tracks that original log is concentrated are removed；
59) filtering log collection is exported
(6) according to the filtering log collection of outputRegenerate journal file.
2. a kind of double grains degree noise log filter method based on incidence relation according to claim 1, it is characterised in that Log set described in step (1)It contains the whole of operation flow and executes example, i.e., every process rail therein Mark σ is corresponding with the primary execution example of operation flow, and the flow path track σ is made of orderly multiple flow events e Sequence, the flow events e are that operation flow executes movable primary record.
3. a kind of double grains degree noise log filter method based on incidence relation according to claim 1, it is characterised in that Frequency dependency degree DFD (e described in step (2)i, ej) indicate degree of following directly after, i.e., the event e in whole flow instancesjIt follows closely Event eiThe total frequency occurred.
4. a kind of double grains degree noise log filter method based on incidence relation according to claim 1, it is characterised in that Global noise factor ζ described in step (3) takes 0.02.
5. a kind of double grains degree noise log filter method based on incidence relation according to claim 1, it is characterised in that Degree of mixing threshold value beta described in step (5) takes 0.5.
6. a kind of double grains degree noise log filter method based on incidence relation according to claim 1, it is characterised in that Weighting factor value α described in step (5) takes 0.5.
7. a kind of double grains degree noise log filter method based on incidence relation according to claim 1, it is characterised in that Penalty factor described in step (5)Take 0.8.
8. a kind of double grains degree noise log filter method based on incidence relation according to claim 1, it is characterised in that Abandonment threshold value described in step (5)Take 0.7.
CN201910218832.3A 2019-03-21 2019-03-21 Double-granularity noise log filtering method based on incidence relation Active CN110032494B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910218832.3A CN110032494B (en) 2019-03-21 2019-03-21 Double-granularity noise log filtering method based on incidence relation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910218832.3A CN110032494B (en) 2019-03-21 2019-03-21 Double-granularity noise log filtering method based on incidence relation

Publications (2)

Publication Number Publication Date
CN110032494A true CN110032494A (en) 2019-07-19
CN110032494B CN110032494B (en) 2020-05-26

Family

ID=67236382

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910218832.3A Active CN110032494B (en) 2019-03-21 2019-03-21 Double-granularity noise log filtering method based on incidence relation

Country Status (1)

Country Link
CN (1) CN110032494B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110597686A (en) * 2019-08-18 2019-12-20 南京理工大学 Noise-tolerant process mining method based on mixed event log
CN112052990A (en) * 2020-08-21 2020-12-08 杭州电子科技大学 CNN-BilSTM hybrid model-based next activity prediction method for multi-angle business process
CN113176977A (en) * 2021-04-27 2021-07-27 南开大学 Interleaved log analysis method for networking workflow of construction
CN114564473A (en) * 2022-04-28 2022-05-31 江苏益柏锐信息科技有限公司 Data processing method, equipment and medium based on ERP enterprise management system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103198146A (en) * 2013-04-19 2013-07-10 中国科学院计算技术研究所 Real-time event filtering method and real-time event filtering system oriented to network stream data
CN103593484A (en) * 2013-12-03 2014-02-19 南京安讯科技有限责任公司 Method for filtering garbage logs during mobile phone internet surfing
KR20170127876A (en) * 2016-05-13 2017-11-22 한국전자통신연구원 System and method for dealing with troubles through fault analysis of log
CN107909344A (en) * 2017-11-21 2018-04-13 杭州电子科技大学 Workflow logs iterative task recognition methods based on relational matrix

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103198146A (en) * 2013-04-19 2013-07-10 中国科学院计算技术研究所 Real-time event filtering method and real-time event filtering system oriented to network stream data
CN103593484A (en) * 2013-12-03 2014-02-19 南京安讯科技有限责任公司 Method for filtering garbage logs during mobile phone internet surfing
KR20170127876A (en) * 2016-05-13 2017-11-22 한국전자통신연구원 System and method for dealing with troubles through fault analysis of log
CN107909344A (en) * 2017-11-21 2018-04-13 杭州电子科技大学 Workflow logs iterative task recognition methods based on relational matrix

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LI YU ET AL.: "Filtering log data: Finding the needles in the Haystack", 《RESEARCHGATE》 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110597686A (en) * 2019-08-18 2019-12-20 南京理工大学 Noise-tolerant process mining method based on mixed event log
CN110597686B (en) * 2019-08-18 2022-10-18 南京理工大学 Noise-tolerant process mining method based on mixed event log
CN112052990A (en) * 2020-08-21 2020-12-08 杭州电子科技大学 CNN-BilSTM hybrid model-based next activity prediction method for multi-angle business process
CN113176977A (en) * 2021-04-27 2021-07-27 南开大学 Interleaved log analysis method for networking workflow of construction
CN114564473A (en) * 2022-04-28 2022-05-31 江苏益柏锐信息科技有限公司 Data processing method, equipment and medium based on ERP enterprise management system
CN114564473B (en) * 2022-04-28 2022-07-12 江苏益柏锐信息科技有限公司 Data processing method, equipment and medium based on ERP enterprise management system

Also Published As

Publication number Publication date
CN110032494B (en) 2020-05-26

Similar Documents

Publication Publication Date Title
CN110032494A (en) A kind of double grains degree noise log filter method based on incidence relation
CN113010506B (en) Multi-source heterogeneous water environment big data management system
CN106339274B (en) A kind of method and system that data snapshot obtains
CN111400326A (en) Smart city data management system and method thereof
CN105528367A (en) A method for storage and near-real time query of time-sensitive data based on open source big data
Takemura et al. Tweet classification based on their lifetime duration
CN112527886A (en) Data warehouse system based on urban brain
CN112732680A (en) Data warehouse design method
Mikolai et al. Union dissolution and housing trajectories in Britain
DE112012006132T5 (en) Information processing apparatus and information processing method
CN106682100A (en) Data statistical method and system based on Hbase database
CA2452562A1 (en) System and method for rapidly locating historical performance data
Chang et al. Korea confronts globalization
CN112822078B (en) Method for realizing raft heartbeat report of nodes in different network domains
CN103870510A (en) Social network friend filtering method on basis of distributive parallel processing mode
CN116579733A (en) Case management system
CN109145109A (en) User group&#39;s message propagation anomaly analysis method and device based on social networks
CN116993391A (en) Site type shared bicycle system use demand prediction method
CN111813833B (en) Real-time two-degree communication relation data mining method
KR20220016625A (en) Artificial intelligence-based meeting assistant device and method
CN106855877A (en) A kind of synchronous method for replicating of big data
CN112256418A (en) Big data task scheduling method
Wortmann et al. Global Crisis, Local Impact: A Comparative Approach to the Financial Crisis’ Impacts on European Local Levels
CN111258955A (en) File reading method and system, storage medium and computer equipment
CN115729936A (en) Buried point data processing method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB03 Change of inventor or designer information

Inventor after: Sun Xiaoxiao

Inventor after: Hou Wenjie

Inventor after: Yu Dongjin

Inventor after: Pan Jianliang

Inventor before: Sun Xiaoxiao

Inventor before: Yu Dongjin

Inventor before: Hou Wenjie

Inventor before: Pan Jianliang

CB03 Change of inventor or designer information
GR01 Patent grant
GR01 Patent grant