CN104123448A - Multi-data-stream anomaly detection method based on context - Google Patents

Multi-data-stream anomaly detection method based on context Download PDF

Info

Publication number
CN104123448A
CN104123448A CN201410335201.7A CN201410335201A CN104123448A CN 104123448 A CN104123448 A CN 104123448A CN 201410335201 A CN201410335201 A CN 201410335201A CN 104123448 A CN104123448 A CN 104123448A
Authority
CN
China
Prior art keywords
data stream
abnormal
snapshot
value
computing node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201410335201.7A
Other languages
Chinese (zh)
Other versions
CN104123448B (en
Inventor
徐建
李涛
张宏
张琨
朱保平
衷宜
陈龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Science and Technology
Original Assignee
Nanjing University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Science and Technology filed Critical Nanjing University of Science and Technology
Priority to CN201410335201.7A priority Critical patent/CN104123448B/en
Publication of CN104123448A publication Critical patent/CN104123448A/en
Application granted granted Critical
Publication of CN104123448B publication Critical patent/CN104123448B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

The invention provides a multi-data-stream anomaly detection method based on the context. The method comprises the following steps that 1, multiple data streams are obtained, and snapshots are generated; 2, the anomaly of the snapshots of the multiple data streams is quantified; 3, the anomaly of the data streams is quantified; 4, the anomaly of the data streams is recognized. The detection method aims to adopt the node anomaly detection of an isomorphism distributed computation system as the study background and adopt the data streams monitored by computational nodes as the study object, the anomaly detection method comprehensively considers the context information of the multiple data streams and the historical behavior information of a single data stream, and the detection rate is high.

Description

Based on contextual multiple data stream method for detecting abnormality
Technical field
The invention belongs to abnormality detection technology, particularly a kind of method for detecting abnormality that merges the contextual information of multiple data stream and the historical behavior information of single data stream.
Background technology
It is an important direction in data stream Research on Mining that data flow anomaly detects.Extremely refer in the distinguished data of data centralization, these data are not because random deviation produces, but result from diverse mechanism.Owing to searching extremely monitoring at network attack in data stream, the fields such as credit card swindle, computing system performance evaluation have application very widely, data flow anomaly detection method is one of focus of current research, and the detection to abnormal behaviour in data stream and the research of excavation have been subjected to the common concern of academia and industry member.
In the real world applications such as such as Distributed Calculation; data stream management system need to receive many data stream simultaneously; and often not completely independent between pieces of data stream; but there is correlativity; for example, in securities exchange system; stock in same market often there will be same or analogous lifting trend, and in Traffic Net, the vehicle flowrate of different sections of highway also will have certain relevance.For the data stream that is mutually related, once find that the correlativity between them is destroyed, can conclude in these data stream and have abnormal conditions.Based on this thinking, researcher inquires into by monitoring plurality and detects abnormal method according to correlativity between stream.The data flow anomaly that existing data flow anomaly detection method roughly can be divided into based on grid detects, the abnormality detection based on density and the abnormality detection based on distance.
It is that whole data space is partitioned into separate that data flow anomaly based on grid detects, a lot of grid of the same size, set artificially a support, in the time that the support of the data element comprising in grid exceedes or equaled the support size of prior setting, just from all dimensions, select one dimension, and according to this dimension, grid is divided into two completely independently sub-grids dynamically.In the time that the support of sub-grid also meets or exceeds threshold value, same cutting operation also can carry out on sub-grid.Park and Lee etc. are proposing a kind of real-time data flow anomaly detection method, this Grid Clustering method does not need the distance between computational data object, only need to, according to pre-determined sizing grid, directly data be put into corresponding grid, therefore can realize real-time increment cluster.After each cluster is complete, only need to preserve the characteristic information of each class, and calculate the abnormality degree of all classes, sort according to descending order, the class of Top-k abnormality degree maximum is divided into final exception class.(Park N H, Lee W S.Statistical grid-based clustering over data streams[J]. ACM SIGMOD Record, 2004,33 (1): 32-37.) above-mentioned method for detecting abnormality otherwise adopt top-p mode p the highest abnormal quantized value data stream as extremely, the data stream that abnormal quantized value is exceeded to predefine threshold value is as extremely, and these methods have problems in actual application: 1) threshold value is difficult to set.The reasonable setting of threshold value need to be familiar with the bottom mechanism of application program very much, and this is for general application person, and difficulty is too large; 2) abnormal number is changing always.May have sometime that to exceed p data stream be abnormal, adopt top-p mode can miss the abnormal of these necessary beings.Therefore, in the present invention, adopt the abnormality detection threshold value of a kind of guideless learning method automatic acquisition dynamic change, can adapt to better the abnormal scene frequently changing.
The basic thought of the abnormality detection based on density is to utilize the density of sample in a certain neighborhood to determine extremely.LOF algorithm is representative algorithm (the Breunig M M of the abnormality detection based on density, Kriegel H P, Ng R T, et al.LOF:identifying density-based local outliers[C] //ACM Sigmod Record.ACM, 2000,29 (2): 93-104.).This algorithm is a kind of Outlier Detection Algorithm based on local density, and data object comparatively accurately can note abnormalities in the inhomogeneous data acquisition of Density Distribution.But LOF algorithm is also not suitable for being directly used in the abnormality detection of data stream, because its time complexity is larger, all needs the abnormality degree of all data objects to re-start calculating if often obtain a new data object, and its cost is intolerable.Therefore, the people such as Pokrajac and Lazarevic has made improvement to existing static LOF algorithm, dynamic increment LOF algorithm (Pokrajac D has been proposed, Lazarevic A, Latecki L J.Incremental local outlier detection for data streams[C] //Computational Intelligence and Data Mining, 2007.CIDM2007.IEEE Symposium on.IEEE, 2007:504-515.).The core concept of increment LOF algorithm is exactly in the time that a new data object arrives, do not recalculate the value of all data object characteristic informations, but only each characteristic information value of that a part of data object that is subject to new input data object impact is upgraded.Increment LOF algorithm is in the time receiving the data object of a new input, and it mainly operates and is divided into two steps: for the data object of new input, calculate its required characteristic information value; Affect for being subject to new input object the neighbor node that density changes, upgrade in turn its characteristic information value, for there is no affected data object, do not recalculate.After adopting this strategy, dynamic increment LOF algorithm, in can reaching and repeat the suitable effect of static LOF algorithm, but greatly reduces the time complexity that algorithm is carried out, and makes it be applicable to the abnormality detection for data stream.But LOF algorithm is not considered the difference of different dimensions codomain, may cause the influence power of part dimension to be significantly greater than other dimensions; In addition, its time complexity is acceptable for offline inspection, but also impracticable concerning real-time detection.Above-mentioned two limitation that the present invention is directed to LOF algorithm, the time complexity of the algorithm of proposition is O (n), is linear increase relation with data stream number, can meet application needs in real time.
Abnormality detection based on distance proposes (Knorr E M by Knorr and Ng etc., Ng R T, Tucakov V.Distance-based outliers:algorithms and applications[J] .The VLDB Journal-The International Journal on Very Large Data Bases, 2000,8 (3-4): 237-253.).Abnormal definition based on distance: in data set S, an object O is called DB (p, D)-outlier, if it meets following character: in data set S, at least the object of p*100% and the distance of O are greater than distance B.Briefly, the abnormity point based on distance is exactly those objects that there is no the neighbours of " abundant ".The people such as Angiulli have also proposed the data flow anomaly detection algorithm Storm based on distance, comprise exact-storm and approx-Storm, the former is accurate algorithm, latter is approximate data (the Angiulli F taking central limit theorem as ensureing, Fassetti F.Detecting distance-based outliers in streams of data[C] //Proceedings of the sixteenth ACM conference on Conference on information and knowledge management.ACM, 2007:811-820.).Storm algorithm has adopted the sliding window model based on counting, and the time order and function order arriving according to data object in data stream, and the neighbours of certain certain data objects are divided into forerunner neighbours and follow-up neighbours.Two threshold k of predefined and R, represent respectively neighbours' number and distance, if certain input data object neighbours' number within the distance range of R is less than K in data stream, this object is just abnormal data.In algorithm, defined the special data object safe inliers of a class, no matter data stream is along with how time variation develops, and such data object can not become abnormal data object in whole data window, and neighbours' number is greater than K all the time.On the basis of this hypothesis, the neighbours that algorithm all adopts R-Tree to search each data object to all non-safe inliers data objects, to improve effectiveness of retrieval.Above-mentioned algorithm, only consider the historical information of data stream, ignore the contextual information of data stream, such as in the computing system of isomorphism, the data stream of computing node often shows similarity, historical information on contextual information and the individual traffic of the multiple data stream that the present invention considers determines that whether certain data stream is abnormal data stream, has higher accuracy.
Summary of the invention
The deficiency existing in order to overcome prior art, the invention provides a kind of node anomaly detection method that adopts isomorphism distributed computing system, the data stream of monitoring taking computing node is the method for detecting abnormality of the contextual information that considers multiple data stream of research object and the historical behavior information of single data stream.
A kind of based on contextual multiple data stream method for detecting abnormality, comprise the following steps:
Step 1, multiple data stream obtains with snapshot and generates, and process is as follows:
A given distributed computing system being made up of the computing node of n isomorphism, records moment t and synchronously obtains the observed reading of each computing node, calculates the data stream of each computing node according to observed reading, and all computing nodes are designated as snapshot in the data stream in t moment;
Step 2, multiple data stream snapshot quantizes extremely, and process is as follows:
The snapshot structural matrix M of moment t is used for describing to the transient behavior of each computing node of moment t; Adopt one by one 0-1 normalization mode to calculate normalized value to each column vector of matrix M, obtain new matrix M '; The average of calculating the each column vector after normalization, obtains deviation matrix M "; Calculate the abnormal quantized value of each computing node at the data stream snapshot of moment t generation;
Step 3, data flow anomaly quantizes, and process is as follows:
Given data stream arbitrarily, calculate the influence power of the moment t abnormal quantized value of all snapshots before for current time t, consider historical data in the abnormal quantized result of multiple data stream snapshot and individual traffic and, for the influence power of abnormal quantized result, draw the abnormal quantification end value of data stream;
Step 4, data flow anomaly identification, process is as follows:
The result of abnormal quantification gained is sorted according to mode from small to large; Calculate median, maximal value, minimum value and the maximum deflection difference value of the rear abnormal quantized value of sequence, and detect data stream.
Compared with prior art, its advantage is in the present invention: (1) considers the contextual information of multiple data stream and the historical information of single data stream quantizes the abnormal of data stream, improved accuracy of detection; (2) abnormal recognition threshold adopts guideless learning method automatic acquisition, without the abnormal domain knowledge of computing node, has reduced the difficulty that identification parameter is set; (3) algorithm has lower time complexity, and n is linear growth relation with data stream number.
Below in conjunction with accompanying drawing, the present invention is described in further detail.
Brief description of the drawings
Fig. 1 is a kind of process flow diagram based on contextual multiple data stream method for detecting abnormality of the present invention;
Fig. 2 is that context, the snapshot of multiple data stream generates schematic diagram;
Fig. 3 is the abnormal process flow diagram quantizing of data stream snapshot;
Fig. 4 is the process flow diagram that data flow anomaly detects;
Embodiment
In conjunction with Fig. 1, a kind of based on contextual multiple data stream method for detecting abnormality, comprise the following steps:
Step 1, multiple data stream obtains with snapshot and generates, and process is as follows:
Step 1.1, a given distributed computing system being formed by the computing node of n isomorphism;
Step 1.2, the observed reading of synchronous recording t moment i computing node wherein represent the component of i computing node in m observation dimension of the observed reading of moment t, 1≤i≤n, a kind of interested peer metric of each representation in components, the tolerance of the computing node of isomorphism is identical, the value of m is determined by total tolerance number;
Step 1.3, forms data stream S corresponding to computing node i i={ s i1, s i2, s i3..., s it, S iit is an orderly but unlimited data observation sequence;
Step 1.4, the data stream composition data adfluxion S={S of n computing node 1, S 2... S 3... S i..., S n;
Step 1.5, records the snapshot S of the data adfluxion S in t moment t=[s it| s it∈ S i, S i∈ S, 1≤i≤n], the snapshot S of data adfluxion S tthat the observed reading of being obtained on each computing node by moment t forms; Therefore according to the definition of snapshot, data adfluxion S can be expressed as again snapshot collection, i.e. S={S 1, S 2..., S t... }.
Step 2, multiple data stream snapshot quantizes extremely, and process is as follows:
Step 2.1, the snapshot S of given time t t, structure n × m matrix for describing the transient behavior of each computing node of moment t;
Step 2.2, to matrix M tany column vector O t d = O 1 t d O 2 t d . . . O nt d , 1≤d≤m, represent the observed reading of i computing node of moment t, adopt 0-1 normalization mode to calculate ( O t d ) ′ = ( O 1 t d ) ′ ( O 2 t d ) ′ . . . ( O nt d ) ′ , Wherein represent the minimum value of d observation tolerance in n observed reading, represent the maximal value of d observation tolerance in n observed reading, and then obtain new matrix
Step 2.3, for matrix M t' any column vector calculate any column vector average O t , avg d = Σ z = 1 n ( O zt d ) ′ / n , Obtain deviation matrix wherein ( o it d ) ′ ′ = ( o it d ) ′ - o t , avg d , ( O t d ) ′ ′ = ( O 1 t d ) ′ ′ ( O 2 t d ) ′ ′ . . . ( O nt d ) ′ ′ ;
Step 2.4, calculates the abnormal quantized value N of each computing node at the data stream snapshot of moment t generation it, concrete steps are:
Step 2.4.1, initialization column vector X t = x 1 t x 2 t . . . x m t , For observing arbitrarily dimension k, 1≤k≤m, calculation deviation square the variance of t moment k observation dimension
Step 2.4.2, initialization is used for the column vector of the abnormal quantized value of depositing each observation dimension d 1 t d 2 t . . . d m t = 0 m , Wherein represent the entropy of t moment k observation dimension, represent to get to stay a variance the entropy of k observation dimension under situation, σ fk t = n σ k t - ( M t ′ ′ ( f , k ) ) 2 n - 1 , 1 ≤ f ≤ n , M t" (f, k) is matrix M t" the value of the capable k row of f; Therefore have d k t = ln σ fk t σ k t = n × ( x k t - ( M t ′ ′ ( f , k ) ) 2 ) ( n - 1 ) × x k t ;
Step 2.4.3, the abnormal quantized value of data stream snapshot of i computing node of calculating moment t N it = Σ k = 1 m d k , 1 ≤ k ≤ m .
Step 3, data flow anomaly quantizes, due to ubiquity momentary fluctuation in data stream and stage transport phenomena, only abnormal by data stream snapshot identification computing node, can cause the alert rate of higher mistake; In order to alleviate this problem, on to the abnormal basis quantizing of data stream snapshot, consider that the collective behavior of showing in data stream historical data further quantizes candidate's abnormal data stream, and in the abnormal quantized result of integrated data stream snapshot and individual traffic historical data for the influence power of abnormal quantized result, produce final abnormal quantized result, detailed process is as follows:
Step 3.1, the given S of data stream arbitrarily i, record the moment t abnormal quantized value { N of all snapshots before i0, N i1..., N i (t-1), calculate the influence power I of the abnormal quantized value of this t snapshot for current time t it, wherein U trepresent influence power attenuation function, in the present invention, choose decaying exponential function control effect power attenuation degree, i.e. U t=e -λ kt, wherein λ >0 is decay speed threshold value, therefore has:
I it=N i(t-1)e +N i(t-2)e -2λ+N i(t-1)e -3λ+...
=e (N i( t-1)+e (N i(t-2)+e (N i(t-3)+...;
=e (N i(t-1)+I i(t-1))
Step 3.2, considers historical data in the abnormal quantized result of multiple data stream snapshot and individual traffic and, for the influence power of abnormal quantized result, draws data stream S iabnormal quantification end value N i=N it+ I it.
Step 4, data flow anomaly identification, process is as follows:
Step 4.1, data-oriented adfluxion S={S 1, S 2..., S ncorresponding abnormal quantized result sequence N={N 1, N 2..., N n, abnormal quantized result sequence N is sorted according to mode from small to large, obtain new sequence N'={N ' 1, N' 2..., N' n, and record subscript mapping relations v=R (u), and 1≤u, v≤n, represents that u abnormal quantized result of new sequence is corresponding to v abnormal quantized result of former sequence;
Step 4.2, establish abnormal quantized result median under be designated as mIdx, have calculate the median N of abnormal quantized result median=N' mIdx, the minimum value N of abnormal quantized result min=N ' 1, the maximal value N of abnormal quantized result max=N' nwith the abnormal quantized value component of maximum d upper;
Step 4.3, sets idx=mIdx+1;
Step 4.4, if N ' idx>max (2 (N median-N min), N min+ d upper), wherein max represents to ask the peaked function of 2 number, when data stream is when abnormal, and N median-N minshould be approximately equal to N max-N minhalf, so if be greater than 2 (N median-N min) time, represent data stream S r (idx)be an abnormal data stream, produce abnormality alarming; Otherwise, data stream S r (idx)it is a normal flow;
If be illustrated under the situation of data stream shake, have the abnormal maximum deflection difference value quantizing between shake data stream and non-jitter data stream, when data stream exists shake, the abnormal quantized result of data stream is greater than N min+ d uppertime, data stream S r (idx)be an abnormal data stream, produce abnormality alarming; Otherwise, data stream S r (idx)it is a normal flow;
Step 4.5, if idx<n, idx ← idx+1, jumps to step 4.4, otherwise step 4 finishes.
Utilize data that method of the present invention calculates and existing Storm algorithm and LOF algorithm to compare, can show that algorithm of the present invention is all better than contrasting algorithm on accuracy rate, recall rate and comprehensive evaluation index.Experimental data derives from the Monitoring Data of 16 computing nodes, each computing node observation dimension is 10 dimensions, respectively that CPU usage, memory usage, page change to swap out number of times, Disk of number of times, page and read number of times, Disk and write number of times, Disk and read byte number, Disk and write that byte number, network interface card receive byte number, network interface card sends byte number etc., in experiment, inject 2 types abnormal, be respectively RAM leakage and CPU and reveal result, the injection duration is 1000s, and experimental result is as shown in table 1.
The Comparison of experiment results of table 1 algorithms of different

Claims (5)

1. based on a contextual multiple data stream method for detecting abnormality, it is characterized in that, comprise the following steps:
Step 1, multiple data stream obtains with snapshot and generates, and process is as follows:
A given distributed computing system being made up of the computing node of n isomorphism, records moment t and synchronously obtains the observed reading of each computing node, calculates the data stream of each computing node according to observed reading, and all computing nodes are designated as snapshot in the data stream in t moment;
Step 2, multiple data stream snapshot quantizes extremely, and process is as follows:
The snapshot structural matrix M of moment t is used for describing to the transient behavior of each computing node of moment t; Adopt one by one 0-1 normalization mode to calculate normalized value to each column vector of matrix M, obtain new matrix M '; The average of calculating the each column vector after normalization, obtains deviation matrix M "; Calculate the abnormal quantized value of each computing node at the data stream snapshot of moment t generation;
Step 3, data flow anomaly quantizes, and process is as follows:
Given data stream arbitrarily, calculate the influence power of the moment t abnormal quantized value of all snapshots before for current time t, consider historical data in the abnormal quantized result of multiple data stream snapshot and individual traffic and, for the influence power of abnormal quantized result, draw the abnormal quantification end value of data stream;
Step 4, data flow anomaly identification, process is as follows:
The result of abnormal quantification gained is sorted according to mode from small to large; Calculate median, maximal value, minimum value and the maximum deflection difference value of the rear abnormal quantized value of sequence, and detect data stream.
2. according to claim 1 based on contextual multiple data stream method for detecting abnormality, it is characterized in that step 1 multiple data stream obtains the concrete steps that generate with snapshot as follows:
Step 1.1, a given distributed computing system being formed by the computing node of n isomorphism, each computing node has m identical tolerance;
Step 1.2, the observed reading of synchronous recording t moment i computing node wherein represent the component of i computing node in m observation dimension of the observed reading of moment t, 1≤i≤n;
Step 1.3, forms data stream S corresponding to computing node i i={ s i1, s i2, s i3..., s it;
Step 1.4, the data stream composition data adfluxion S={S of n computing node 1, S 2... S 3... S i..., S n;
Step 1.5, records the snapshot S of the data adfluxion S in t moment t=[s it| s it∈ S i, S i∈ S, 1≤i≤n].
3. according to claim 1 based on contextual multiple data stream method for detecting abnormality, it is characterized in that the abnormal concrete steps that quantize of step 2 multiple data stream snapshot are as follows:
Step 2.1, the snapshot S of given time t t, structure n × m matrix
Step 2.2, to matrix M tany column vector O t d = O 1 t d O 2 t d . . . O nt d , 1≤d≤m, adopts 0-1 normalization mode to calculate ( O t d ) &prime; = ( O 1 t d ) &prime; ( O 2 t d ) &prime; . . . ( O nt d ) &prime; , And then obtain new matrix
Step 2.3, calculates any column vector average obtain deviation matrix wherein ( O it d ) &prime; &prime; = ( O it d ) &prime; - O t , avg d ;
Step 2.4, calculates the abnormal quantized value N of each computing node at the data stream snapshot of moment t generation it, concrete steps are:
Step 2.4.1, initialization column vector X t = x 1 t x 2 t . . . x m t , Wherein 1≤k≤m, calculates k the variance of observing dimension
Step 2.4.2, calculate each computing node the abnormal quantized value component of data stream snapshot d k t = ln &sigma; fk t &sigma; k t = n &times; ( x k t - ( M t &prime; &prime; ( f , k ) ) 2 ) ( n - 1 ) &times; x k t , Wherein &sigma; fk t = n &sigma; k t - ( M t &prime; &prime; ( f , k ) ) 2 n - 1 , 1 &le; f &le; n , M tthe value of the capable k row of " (f, k) is matrix M " f;
Step 2.4.3, the abnormal quantized value of data stream snapshot of i computing node of calculating moment t N it = &Sigma; k = 1 m d k , 1 &le; k &le; m .
4. according to claim 1 based on contextual multiple data stream method for detecting abnormality, it is characterized in that the data flow anomaly of step 3 quantizes, step is as follows:
Step 3.1, the given S of data stream arbitrarily i, record the moment t abnormal quantized value { N of all snapshots before i0, N i1..., N i (t-1), calculate the influence power I of the abnormal quantized value of this t snapshot for current time t it, wherein U t=e -λ kt, wherein λ >0 is decay speed threshold value;
Step 3.2, considers historical data in the abnormal quantized result of multiple data stream snapshot and individual traffic and, for the influence power of abnormal quantized result, draws data stream S iabnormal quantification end value N i=N it+ I it.
5. according to claim 1 based on contextual multiple data stream method for detecting abnormality, it is characterized in that the data flow anomaly identification of step 4, step is as follows:
Step 4.1, data-oriented adfluxion S={S 1, S 2..., S ncorresponding abnormal quantized result sequence N 1, N 2..., N n, abnormal quantized result sequence is sorted according to mode from small to large, obtain new sequence N ' 1, N' 2..., N' n, and record subscript mapping relations v=R (u), and 1≤u, v≤n, represents that u abnormal quantized result of new sequence is corresponding to v abnormal quantized result of former sequence;
Step 4.2, establish abnormal quantized result median under be designated as mIdx, have calculate the median N of abnormal quantized result median=N' mIdx, the minimum value N of abnormal quantized result min=N ' 1, the maximal value N of abnormal quantized result max=N' nwith the abnormal quantized value component of maximum d upper;
Step 4.3, sets idx=mIdx+1;
Step 4.4, if N ' idx>max (2 (N median-N min), N min+ d upper), represent data stream S r (idx)be an abnormal data stream, produce abnormality alarming; If be illustrated under the situation of data stream shake, have the abnormal maximum deflection difference value quantizing between shake data stream and non-jitter data stream, when data stream exists shake, the abnormal quantized result of data stream is greater than N min+ d uppertime, data stream S r (idx)be an abnormal data stream, produce abnormality alarming;
Step 4.5, if idx<n, idx=idx+1, jumps to step 4.4, otherwise step 4 finishes.
CN201410335201.7A 2014-07-14 2014-07-14 Multi-data-stream anomaly detection method based on context Active CN104123448B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410335201.7A CN104123448B (en) 2014-07-14 2014-07-14 Multi-data-stream anomaly detection method based on context

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410335201.7A CN104123448B (en) 2014-07-14 2014-07-14 Multi-data-stream anomaly detection method based on context

Publications (2)

Publication Number Publication Date
CN104123448A true CN104123448A (en) 2014-10-29
CN104123448B CN104123448B (en) 2017-05-17

Family

ID=51768857

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410335201.7A Active CN104123448B (en) 2014-07-14 2014-07-14 Multi-data-stream anomaly detection method based on context

Country Status (1)

Country Link
CN (1) CN104123448B (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104536996A (en) * 2014-12-12 2015-04-22 南京理工大学 Computational node anomaly detection method in isomorphic environments
CN106254321A (en) * 2016-07-26 2016-12-21 中国人民解放军防空兵学院 A kind of whole network abnormal data stream sorting technique
CN108038044A (en) * 2017-12-26 2018-05-15 北京航空航天大学 A kind of method for detecting abnormality towards continuous monitored target
CN108108253A (en) * 2017-12-26 2018-06-01 北京航空航天大学 A kind of abnormal state detection method towards multiple data stream
CN108345574A (en) * 2017-01-23 2018-07-31 无锡市计量测试院 Related dual data stream abnormality detection and modified method
CN111563007A (en) * 2020-04-27 2020-08-21 平安医疗健康管理股份有限公司 Operation error repairing method, device, computer system and readable storage medium
CN112699113A (en) * 2021-01-12 2021-04-23 上海交通大学 Industrial manufacturing process operation monitoring system driven by time sequence data stream
CN113032824A (en) * 2021-03-01 2021-06-25 上海观安信息技术股份有限公司 Low-frequency data leakage detection method and system based on database flow log

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1809000A (en) * 2006-02-13 2006-07-26 成都三零盛安信息系统有限公司 Network intrusion detection method
CN101848160A (en) * 2010-05-26 2010-09-29 钱叶魁 Method for detecting and classifying all-network flow abnormity on line
US7970772B2 (en) * 2004-03-16 2011-06-28 International Business Machines Corporation Methods and apparatus for data stream clustering for abnormality monitoring
CN102945320A (en) * 2012-10-29 2013-02-27 河海大学 Time series data abnormity detection method and device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7970772B2 (en) * 2004-03-16 2011-06-28 International Business Machines Corporation Methods and apparatus for data stream clustering for abnormality monitoring
CN1809000A (en) * 2006-02-13 2006-07-26 成都三零盛安信息系统有限公司 Network intrusion detection method
CN101848160A (en) * 2010-05-26 2010-09-29 钱叶魁 Method for detecting and classifying all-network flow abnormity on line
CN102945320A (en) * 2012-10-29 2013-02-27 河海大学 Time series data abnormity detection method and device

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
DRAGOLJUB POKRAJAC ETAL: ""incremental local outlier detection for data streams "", 《IEEE SYMPOSIUM ON COMPUTATIONAL INTELLIGENCE AND DATA MINING》 *
FABRIZIO ANGIULLI ETAL: ""detecting distance-based outliers in streams of data"", 《THE 16TH ACM CONFERENCE ON INFORMATION& KNOELEDGE MANAGEMENT》 *
YINGYI BU ETAL: ""efficient anomaly monitoring over moving object trajectory streams"", 《ACM SIGKDD INTERNATIONAL CONFERENCE ON KNOWLEDGE DISCOVERY & DATA MINING》 *

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104536996B (en) * 2014-12-12 2017-12-12 南京理工大学 Calculate node method for detecting abnormality under a kind of homogeneous environment
CN104536996A (en) * 2014-12-12 2015-04-22 南京理工大学 Computational node anomaly detection method in isomorphic environments
CN106254321B (en) * 2016-07-26 2019-03-19 中国人民解放军防空兵学院 A kind of whole network abnormal data stream classification method
CN106254321A (en) * 2016-07-26 2016-12-21 中国人民解放军防空兵学院 A kind of whole network abnormal data stream sorting technique
CN108345574B (en) * 2017-01-23 2021-09-03 无锡市计量测试院 Method for detecting and correcting related double data stream abnormity
CN108345574A (en) * 2017-01-23 2018-07-31 无锡市计量测试院 Related dual data stream abnormality detection and modified method
CN108038044A (en) * 2017-12-26 2018-05-15 北京航空航天大学 A kind of method for detecting abnormality towards continuous monitored target
CN108038044B (en) * 2017-12-26 2021-01-08 北京航空航天大学 Anomaly detection method for continuous monitored object
CN108108253A (en) * 2017-12-26 2018-06-01 北京航空航天大学 A kind of abnormal state detection method towards multiple data stream
CN111563007A (en) * 2020-04-27 2020-08-21 平安医疗健康管理股份有限公司 Operation error repairing method, device, computer system and readable storage medium
CN111563007B (en) * 2020-04-27 2022-11-25 深圳平安医疗健康科技服务有限公司 Operation error repairing method, device, computer system and readable storage medium
CN112699113A (en) * 2021-01-12 2021-04-23 上海交通大学 Industrial manufacturing process operation monitoring system driven by time sequence data stream
CN113032824A (en) * 2021-03-01 2021-06-25 上海观安信息技术股份有限公司 Low-frequency data leakage detection method and system based on database flow log
CN113032824B (en) * 2021-03-01 2023-06-23 上海观安信息技术股份有限公司 Low-frequency data leakage detection method and system based on database flow logs

Also Published As

Publication number Publication date
CN104123448B (en) 2017-05-17

Similar Documents

Publication Publication Date Title
CN104123448A (en) Multi-data-stream anomaly detection method based on context
Zhang et al. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data
Wang et al. An improved K-Means clustering algorithm
Ding et al. An anomaly detection approach for multiple monitoring data series based on latent correlation probabilistic model
Chadha et al. Generalized dilation convolutional neural networks for remaining useful lifetime estimation
Rahmani et al. Graph-based approach for outlier detection in sequential data and its application on stock market and weather data
CN113139600A (en) Intelligent power grid equipment anomaly detection method and system based on federal learning
Jung et al. Analyzing future communities in growing citation networks
Wang et al. New developments in unsupervised outlier detection
Li et al. Vehicle counting and traffic flow parameter estimation for dense traffic scenes
CN104536996A (en) Computational node anomaly detection method in isomorphic environments
Mohamed et al. A review on big data management and decision-making in smart grid
Wang et al. Group pattern mining on moving objects’ uncertain trajectories
Chandore et al. Outlier detection techniques over streaming data in data mining: A research perspective
Son et al. Time series prediction using pattern matching
Wang et al. An adaptive sliding window for anomaly detection of time series in wireless sensor networks
Anikin et al. Identification of Vehicle Trajectory Anomalies on Streaming Video
DS et al. Comparative analysis of machine learning-based algorithms for detection of anomalies in IIoT
Shibuya et al. Anomaly detection method based on fast local subspace classifier
Huo et al. Traffic anomaly detection method based on improved GRU and EFMS-Kmeans clustering
Babaei et al. AEGR: a simple approach to gradient reversal in autoencoders for network anomaly detection
Shen et al. Unsupervised concept drift detectors: A survey
Kim et al. Sophy: a morphological framework for structuring geo-referenced social media
Azim et al. Deep graph stream SVDD: anomaly detection in cyber-physical systems
Wang et al. Unsupervised fraud detection in environmental time series data

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant