CN105847029B - A kind of information security events auto-associating and quick response system based on big data - Google Patents
A kind of information security events auto-associating and quick response system based on big data Download PDFInfo
- Publication number
- CN105847029B CN105847029B CN201610130328.4A CN201610130328A CN105847029B CN 105847029 B CN105847029 B CN 105847029B CN 201610130328 A CN201610130328 A CN 201610130328A CN 105847029 B CN105847029 B CN 105847029B
- Authority
- CN
- China
- Prior art keywords
- alarm
- meta
- alarms
- correlation
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000004044 response Effects 0.000 title claims abstract description 26
- 238000010219 correlation analysis Methods 0.000 claims abstract description 33
- 238000004458 analytical method Methods 0.000 claims abstract description 17
- 238000000034 method Methods 0.000 claims abstract description 15
- 238000005516 engineering process Methods 0.000 claims abstract description 10
- 230000008439 repair process Effects 0.000 claims abstract description 5
- 238000004422 calculation algorithm Methods 0.000 claims description 9
- 238000012217 deletion Methods 0.000 claims description 6
- 230000037430 deletion Effects 0.000 claims description 6
- 238000003780 insertion Methods 0.000 claims description 6
- 230000037431 insertion Effects 0.000 claims description 6
- 238000004445 quantitative analysis Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 claims description 5
- 241000764238 Isis Species 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 4
- 238000013024 troubleshooting Methods 0.000 claims description 3
- 230000002159 abnormal effect Effects 0.000 claims description 2
- 230000009471 action Effects 0.000 claims description 2
- 238000000691 measurement method Methods 0.000 claims description 2
- 238000006243 chemical reaction Methods 0.000 claims 1
- 230000002776 aggregation Effects 0.000 abstract description 2
- 238000004220 aggregation Methods 0.000 abstract description 2
- 238000007405 data analysis Methods 0.000 abstract 1
- 230000008030 elimination Effects 0.000 abstract 1
- 238000003379 elimination reaction Methods 0.000 abstract 1
- 230000007257 malfunction Effects 0.000 abstract 1
- 238000012098 association analyses Methods 0.000 description 6
- 238000005065 mining Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000001514 detection method Methods 0.000 description 5
- 238000012423 maintenance Methods 0.000 description 4
- 238000007726 management method Methods 0.000 description 4
- 239000000284 extract Substances 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000000875 corresponding effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000007621 cluster analysis Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1425—Traffic logging, e.g. anomaly detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1433—Vulnerability analysis
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Alarm Systems (AREA)
Abstract
The information security events auto-associating and quick response method and system that the invention discloses a kind of based on big data analysis, it include: offline relating module, online association module, member alarm comparison module, first alert priority module, member alarm cluster module, attack mode discovery module alert response system/work order module.Through the invention, using big data technology, the Alert aggregation that safety equipment is reported alerts at member and carries out correlation analysis, generates many member alarms.First alert priority analysis is carried out after alarm correlation analysis, is distributed member and is alerted corresponding alarm level；Alarm response system is according to the high/low timely notice of alarm level and related personnel is appointed to carry out malfunction elimination and repair, and shortens the alarm response time significantly, eliminates the wrong report of the information safety devices such as IDS generation.
Description
Technical Field
The invention relates to the technical field of information security and big data, in particular to a method and a system for event association and quick response of an information system.
Background
The English abbreviation contained in the invention is as follows:
IDS: intrusion Detection Systems.
LOF: local Outlier Factor Local anomaly Factor
TTL: the Time to Live field refers to the maximum number of segments allowed to pass before an IP packet is dropped by a router
TOS: type of Service Type
ACG: alert Correlation Graph
GED: graph Edit Distance
DMZ: a dematialized zone isolation zone, or demilitarized zone
App: application program
Recent research results show that almost all information security devices employ log files as evidence of being attacked (Verizon, 2014). This means that the task of successfully analyzing such massive logs and performing attack/threat detection is very daunting.
This patent provides a big data based framework for information security analysis. The alarm correlation analysis is a core module of the framework, and the main function of the alarm correlation analysis is to perform comprehensive correlation analysis on log information reported by various information security devices (such as a firewall, an intrusion detection system and a UTM). Intrusion detection systems, especially those based on digital signatures, generate a large number of false positives. And (4) adopting a big data technology to aggregate the alarms into meta-alarms with a high-level structure for correlation analysis. Through alarm correlation analysis, the system will generate a number of meta-alarms. The meta-alarm is composed of multiple alarms reported from the security devices, thus significantly reducing the number of alarms that the information security analyst needs to evaluate and effectively handling a large number of false alarms such as those generated by an IDS.
The module at the other core of the framework is to perform priority analysis on the meta-alarms after the alarm correlation analysis is completed. The alarm priority analysis is to assign an alarm level corresponding to each element alarm and indicating the severity of the element alarm, which is very helpful for an alarm response system/work order to determine the priority for processing the element alarms. And the alarm response system timely informs and appoints related personnel to carry out troubleshooting and repairing according to the high/low alarm level. The alarm notification mode mainly comprises various modes such as App, short message, mail and the like.
At present, the existing alarm correlation analysis technology mainly adopts a rule-based model. The technology extracts various characteristics of the network attack to form an attack characteristic description library, constructs an automaton model for analysis based on the characteristics of the description library from the beginning to the end of the whole process of the network attack, thereby obtaining an association analysis rule, and applies the association analysis rule to information security detection. However, the association analysis technology based on the rule model mainly adopts a state machine method, and the "state machine" type association analysis has strong real-time performance and timing, must be consistent with the real timing of the event occurrence, and has high requirements on the timing of the trigger event. In a complex network environment, under the influence of network transmission delay and front-end processing delay, the time sequence of a security event entering an engine may be reversed, so that a state machine cannot be triggered, and a correlation analysis engine has false report or false report. Therefore, the accuracy of the existing association analysis technology based on the rule model has defects, and the requirement of association analysis of large data volume under the environment of an information security operation and maintenance management cloud platform is difficult to meet.
Therefore, how to solve the problem of information security event correlation analysis in the information security operation and maintenance management cloud platform environment and design a scheme for automatic information security event correlation analysis based on the information security operation and maintenance management cloud platform become an important issue to be solved in the design of the information security operation and maintenance management platform.
Disclosure of Invention
In view of the above, the present invention provides a method and system for automatically associating and quickly responding to an information security event log. And (4) adopting a big data technology to aggregate the alarms reported by the safety equipment into meta-alarms for correlation analysis, and generating a plurality of meta-alarms. After the alarm correlation analysis, performing meta alarm priority analysis and distributing the alarm level corresponding to the meta alarm; and the alarm response system timely informs and appoints related personnel to carry out troubleshooting and modification according to the alarm level.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
the invention provides an information security incident automatic association and quick response system based on big data, which comprises: the system comprises an offline association module, an online association module, a meta alarm comparison module, a meta alarm priority analysis module, a meta alarm clustering module, an attack mode discovery module and an alarm response system/work order module.
In the above scheme, the offline association module is to construct an alarm correlation analysis model by using historical alarms.
In the above scheme, the online correlation module performs correlation analysis on each alarm received in real time to generate a large number of meta-alarms.
In the above scheme, the meta alarm comparison module measures the difference between each meta alarm by using a quantitative method.
In the above scheme, the meta alarm priority module assigns a priority to each meta alarm based on the difference between the meta alarm and other meta alarms.
In the above scheme, the meta alarm clustering module combines the meta alarms into clusters.
In the scheme, the attack pattern discovery module extracts some representative features of each cluster through frequent pattern mining.
In the above scheme, the alarm response system/work order module notifies the client of the alarm with high severity in time by means of short message, App, mail or the like, or appoints experts and technicians to repair the alarm in time by the work order.
Further, the alarm correlation analysis model is composed of two knowledge tables: a correlation strength table and a correlation constraint table. For any two of the types of alarms,andthe correlation strength L (T _ a, T _ b) is the conditional probability: l: (L:),) Is the conditional probability: l: (L:),)=P() The correlation constraint C is a rule for correlating two alarm types.
Further, the correlation analysis is performed on each alarm received in real time, and if two alarms are received, the correlation analysis is performedAndare related by title, then satisfyAndassociation strength of two alarm types，Andrespective constraints of two alarm types, at least one pairAndis true.
Further, the quantitative method for measuring the difference between each meta-alarm adopts the alarm association graph ACG to describe the data structure of each meta-alarm, and the graph edit distance GED is used as a measurement method for quantitatively calculating the difference between two meta-alarms.
Further, each meta-alarm is assigned a priority, which is divided into 4 levels, the meta-alarms highly similar to other meta-alarms are generally divided into 1 or 2 levels, and the meta-alarms greatly different from other meta-alarms are generally divided into 3 or 4 levels.
Further, the meta alarms are combined into clusters, and a DBSCAN algorithm is applied to the process of clustering the meta alarms.
Further, the priority of the meta-alarm difference is obtained through the following five steps of calculation:
(1) calculating k-neighbors and k-distances of the element alarm;
(2) calculating the reachable distance of the element alarm;
(3) calculating the local reachable density of the element alarm;
(4) calculating local abnormal factors of the element alarms;
(5) the priority of the degree of difference LOF of the meta alarm is calculated.
The method and the system provided by the invention greatly make up the deficiency of the existing alarm correlation quantification, remarkably shorten the alarm response time, eliminate the false alarm generated by information safety equipment such as IDS and the like, and improve the accuracy and credibility of alarm judgment.
Drawings
FIG. 1 is an alarm log format for an IDS security device in accordance with the present invention;
FIG. 2 (a) is the alarm information (time of occurrence, source IP, source port, destination IP, destination port, intrusion type) according to the present invention;
FIG. 2 (b) is a meta-alarm association diagram according to the present invention;
FIG. 3 is a big data based security event alarm correlation analysis framework in accordance with the present invention;
FIG. 4 (a) is an alarm correlation diagram according to the present invention;
FIG. 4 (b) is an attack pattern diagram of an alarm correlation diagram according to the present invention.
Detailed Description
The invention is described in further detail below with reference to the figures and examples:
figure 1 is an alarm log format for an IDS security device produced by a company. Line 1 indicates the alarm type, alarm category and alarm priority. Line 2 indicates the intrusion occurrence time, the IP addresses and port numbers of the sender and receiver, and the TTL, network protocol, traffic type and length.
The log of FIG. 1 employs 6 attributes, such that each alarm is represented as a 6-dimensional array: (，，，，，). The attributes of the 6-dimensional array are the time when the alarm occurred, the source IP, the source port, the target IP, the target port, and the alarm type, respectively. These attribute values are either text, IP address, time, or number. One alarm type isIs actually whenTake a value ofAn example of an alarm.
Typically, meta-alerts are used in describing relevance alerts. A meta-alarm comprises one or more alarms reported by a device, which are grouped together by an aggregation or association system into a high-level structure of alarms (or super-alarms). FIG. 2 (a) shows an intrusion alert: (，，，，，) After some form of association, as shown in fig. 2 (b), a logical relationship between them is established, and is called an "alarm association graph".
An "alarm association graph" is a weighted directed acyclic graph G = (V, E). Wherein V represents a node, and each node VV represents an alarm of a 6-dimensional array, each edgeIs two nodes、A connection between, which represents:
（1）andit is the case that it is a related,
（2）is shown inThe alarm that occurred before, and the weight of the edge represents the strength of the correlation between the two nodes.
FIG. 3 provides a framework for big data based security log correlation analysis, which consists of 7 modules. They are as follows:
1. an offline association module;
2. an online correlation module;
3. comparing the meta alarms;
4. analyzing the priority of the meta alarm;
5. clustering the meta alarms;
6. discovering an attack mode;
7. alarm response system/work order.
Which will be described separately below.
1. An offline association module:
the off-line correlation module is a model for constructing alarm correlation analysis by using historical alarms. The correlation analysis model is built by an offline correlation analysis component and is periodically used and updated by an online correlation component. The correlation model consists of two knowledge tables:
(i) correlation intensity table
(ii) Correlation constraint table
The correlation intensity table represents two alarm types respectivelyAndthe relative strength of (c). More specifically, it representsTake place inHow likely it is later. For any two of the types of alarms,andcorrelation intensity L: (,) Is the conditional probability:
L(,)=P() （1）
the correlation constraint C is a rule for correlating two alarm types. For example,that is, indicating that the association occurred within 20 secondsAnd(ii) a The other is thatIndicating two alarm typesAndare correlated, i.e., when their destination IP addresses are the same (in other words, the difference between their destination IP addresses is zero).
When there is n between two alarm types: n > 1, the correlation strength between the two alarm types is the minimum value of n constraints:
L(,)=min{P（） （2）
wherein,
P（）= （3）
in equation (3), for a given historical alarm, P: () It is meant that within the time window W,number of subsequent occurrences, and with respect to being within the same time windowThe number of occurrences. P: () Refer toProbability of occurrence later, and these two alarm typesAndall satisfy the relevant constraintAboutThe number or times of occurrences in the historical alert summary H. Finally, P: (︱) Is thatGiven two alarm typesAndprobability of occurrence within the same time window.
Algorithm 1 describes the process by which the offline association component computes the correlation strength table and the correlation constraint table.
Lines 1-5 initialize the offline association process. A represents the attributes of all alarms, and the alarm information comprises: time of occurrence, source IP, source port, destination IP, destination port, and intrusion type. H is historical alarm, used for training the model; t denotes the alarm type in all the historical alarms H.The representation is a paired alarm type, in whichIndicates the ith pair of alarm typesAnd。
for each pair of alarm types, GETCnstraints generate some possible k combination attributes by calculating all possible k combination attributesAnd (4) relevant constraint. First, when k =1, a correlation constraint of length 1 is generated, wherein C contains only one attribute a for each correlation constraintA. For each relevant constraint, we will calculate that under condition C,is sent atThe previous probability. If this probability does not exceedThen, then、Are considered to be irrelevant.
2. An online association module:
on the alarmBeforeAlarm S =occurring in second timeFor each alarm received in real timeAnd (5) carrying out correlation analysis. To determineAnd whether the alarms in S are related or not, their alarm types are extracted and used as discoveryCorrelation strength and constraints (stored in the knowledge base by the offline module). If both alarms satisfy the following condition, it means correlation:
（1）andassociation strength of two alarm types
（2）Andrespective constraints of two alarm types, at least one pairAndis true.
Each analyzed historical alarm is stored in a database node. If the alarm is givenAndcorrelation, thenIs added to. Thus, an edge is added to the meta-alert to describe its relevance.
3. Meta-alarm comparison
The process of correlation analysis typically generates a number of meta-alarms. Under the pressure of massive alarm analysis, thousands of meta-alarms will be generated in a short time. To clarify these generated meta-alarms, a quantitative method is used to measure the difference between each meta-alarm, namely the meta-alarm priority component and the meta-alarm clustering component described below.
The significance of such an analysis is self-evident. From among the large number of generated meta-alarms, two meta-alarms are optionally selected、。From two connectable nodes () Consists of a mixture of, in which,indicating a suspicious ping from the mail server to the Web server, and,a response from the Web server to the mail server is also indicated (which also triggers an intrusion).Also consisting of two connectable nodes, however, with different contexts.Indicating an intrusion which is routed from an external address to a data packet on a network DMZ mail serverTriggered, the DMZ server then routes the data packet to the internal mail server. This also triggers an alarm, referred to herein as. Although structurally similar, analysis through visualization enables security analysts to learn attack patterns and their grammars (and they are not intrusions of the same type). In addition, many generated meta-alarms are very similar to the two meta-alarms based on analysis and observation、But not limited to, these two. Since thousands of meta-alarms detect such attack patterns that may prove to be impossible to visualize. However, cluster analysis can solve this problem.
The alarm association graph ACG is a data structure of a meta-alarm. The graph edit distance GED serves as a distance measure to quantitatively calculate the difference between the alarm correlation graphs.
For any two alarm association graphsAndthe difference between them is the GED, which is caused by operations such as node deletion/node insertion, edge deletion/edge insertion, and node replacement/edge replacementIs converted intoThe minimum value of the operation action times. Algorithm 2 details the comparison of any two meta-alarms. The distances between all pairs of meta-alarms are calculated and stored in the knowledge base as in fig. 3.
Among algorithm 2, there are 3 key operations: an alternate path, a delete path, and an insert path. Each operation requires a cost function, which is used as a sort. A set of cost functions suitable for meta-alarm comparison is defined by using the domain knowledge base.
Insert and delete Path C) And C)
When comparing any two ACGs, node insertion or deletion operations may be used to convert one graph to another if one contains more nodes (e.g., alarms) than the other. For each intrusion type, we define a weightAs a cost of inserting or deleting a T-type node.
For an edge, the cost of the operation of insertion or deletion of the edge is equivalent to the weight of the edge.
(2) Alternate path
One node replacement operation replaces another alarm. Thus, the more similar alarms, the lower the cost of replacement. This patent then uses the following Euclidean formula to measure the similarity between alarms.
C)=d（，）=
It is noted thatIs an alarmN is the maximum value of the alarm attribute. For the classification attribute, adopting a character string editing distance; for IP attributes, a common prefix method is used. Two examples of IP address differences are shown in the following table:
the cost of edge replacement, which is the absolute value of the difference between the weights of two edges, is also defined in this patent as follows:
C（）=
4. meta-alarm priority
The meta-alarm priority component assigns a priority to each meta-alarm based on its differences from other meta-alarms. The priority is divided into 4 levels in total. Meta-alarms that are highly similar to other meta-alarms are generally classified in 1 or 2 levels, while meta-alarms that differ significantly from other meta-alarms are generally classified in 3 or 4 levels.
Each meta-alarm is mapped to a priority based on the different degrees of phase difference. The degree of difference of the meta alarm is calculated using the LOF. The mapping between the priority value and LOF of the meta-alarm is as follows:
P（g）=
in the above equation, g is a meta-alarm,is the weight of the LOF value.Using 5 steps of neighbors (of meta-alarms) of a point, reachable distance and local density.
(1) k-neighbors and k-distances: one element alarmK-neighbor adoption of() A representation, which is a collection of some other meta-alarms; wherein any one of the other elements alarms andthe difference between them is less than or equal to said k-distance. One isIs thatAnd k isThe distance between the nearest meta-alarms. k is a configurable parameter provided by the algorithm.
(2) The reachable distance is: this is the maximum of the distance between two meta-alarms and the latter k-distance, as shown in the following equation.
r(g,)=max
(3) Local accessible density: the local reachable density of a meta-alarm is the inverse of the average reachable distance between it and its k-neighbors (as shown in the equation below).
Ir=（
(4) Local abnormality factor: for each meta-alarm g, its LOF is calculated by the following formula:
=
(5) LOF priority: consider the range of LOF values in 0EMeanwhile, the method utilizes the weighting technology to map the LOF value to 0E(as shown in the following equation).
nLOF（g）=
Based on the calculation of the value of each meta-alarm priority, the meta-alarm priority component uses its filtering subcomponent to raise the threshold aboveAll of the meta-alarms in (a) are forwarded to the alarm response system/work order for further processing.
5. Meta-alarm clustering
The meta-alarm clustering component receives the meta-alarms G and groups them into clusters, if anyGreater than 2. For a given meta-alarm, if clustering is passedG is considered to belong to the cluster if the internal member density of (g) is reachable. Any one of the meta-alarms within a cluster,there are at least k meta-alarms similar to it (e.g.,andshould be less than the threshold). The clustering process using the DBSCAN algorithm is as follows:
6. attack pattern discovery
The attack pattern discovery component receives clusters of meta-alarms and extracts some representative features of each cluster by frequent pattern mining (frequency pattern mining). The pattern mining, attack pattern discovery component, as described previously, represents each meta-alert as a less complex graph structure. The graph structure is called a pattern graph. An attack pattern graph is a graph representing meta-alarms, where each node represents an alarm or an attribute of an alarm, and each edge represents an association between two meta-alarms or an association of attributes of alarms.
FIG. 4 illustrates mapping from a meta-alarm graph to a graph schema. In fig. 4 (a), the node labeled "HTTP IESecurity …" indicates an alarm. Each alarm may be represented as a multi-dimensional vector. Since graph mode mining is a scenario that does not fit multiple attribute nodes. Each node is compressed into a single attribute labeled node by representing each alarm attribute as a new node. To maintain all attributes, each attribute node is associated with an alarm node by using an edge (FIG. 4 (b)). By adopting different shapes, the attribute nodes are distinguished from the alarm nodes. A graph frequent pattern mining algorithm GSPAN is used to extract frequent patterns from each cluster. For given graphs and a minimum support thresholdSome frequent patterns are extracted. From any one clusterEach frequent pattern extracted is a subgraph and is usually included in the clusterAnd (5) individual element alarming. The GSPAN algorithm is too long and will not be described in detail here.
7. Alarm response system/work order
The alarm response system/work order informs the client of the alarm with high severity degree through short message, APP, mail and other modes, or appoints experts or technicians to repair the fault through the form of the work order.
The Web application of visual analysis provided by the module is widely used by analysts to explore each meta-alarm priority through a series of interactions. For example, he or she may rank and filter the meta-alarms by querying according to size, priority, distance, etc.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention; all equivalent changes and modifications made according to the present invention are considered to be covered by the scope of the present invention.
Claims (4)
1. A big data based information security incident automatic association and quick response system is characterized in that a big data technology is adopted, alarms are aggregated into meta-alarms with a high-level structure for correlation analysis, after the meta-alarm correlation analysis is completed, the meta-alarms are subjected to priority analysis, and the priority order of processing the meta-alarms is determined, the system further comprises: the system comprises an offline association module, an online association module, a meta alarm comparison module, a meta alarm priority analysis module, a meta alarm clustering module, an attack mode discovery module and an alarm response system or work order module;
the off-line correlation module is a model for constructing alarm correlation analysis by using historical alarms; the relevance analysis model is built by an offline relevance analysis component and is periodically used and updated by an online association component;
the alarm correlation analysis model is composed of two knowledge tables: a correlation strength table, a correlation constraint table, for any two alarm types,andcorrelation intensity L: (, ) Is the conditional probability: l: (L:), )=P(→| C), the correlation constraint C is the rule that associates two alarm types, each alarm is represented as a 6-dimensional array a1, a2, a3, a4, a5, a6, the attributes of the 6-dimensional array are the time when the alarm occurs, the source IP, the source port, the target IP, the target port and the alarm type, respectively, and one alarm type isIs whenAn alarm instance at a value of a 6;
the online correlation module performs correlation analysis on each alarm received in real time to generate a plurality of meta-alarms before the alarm a1Alarms S = { a1, a2.., an } occurring within seconds, for each alarm received in real timePerforming correlation analysis for determiningAnd whether the alarms in S are related or not, the alarm types of the alarms are extracted and used as the strength and the constraint of discovering the correlation, and each analyzed historical alarm is stored in a database node;
the correlation analysis is carried out on each alarm received in real time, and if two alarms are receivedAndis called correlation, then satisfiesAndthe strength of association between two alarm types is not less than，Andrespective constraints of two alarms, at least one pairAndis true, that is, for each pair of alarm types, all possible combination attributes are computed, generating a number of associated constraints, for each associated constraint C, there is at least one associated constraint C, under which,is sent atThe previous probability is more than or equal to theta;
the alarm response system or the work order module informs the client of the alarm with high element alarm priority in a short message, App and mail mode, or appoints an expert or a technician in time to repair the alarm through the work order.
2. The big data based information security event auto-correlation and rapid response system as claimed in claim 1, wherein said meta-alarm comparing module measures the difference between each meta-alarm using a quantitative method;
the quantitative method for measuring the difference between each element alarm adopts an alarm association graph ACG to describe the data structure of each element alarm, graph edit distance GED is used as a measurement method for quantitatively calculating the difference between two element alarms, and any two alarm association graphsAndthe difference between the two is GED, and the quantitative method for obtaining the difference GED is to enable the alarm correlation diagram to be operated through node deletion, node insertion, edge deletion, edge insertion, node replacement and edge replacementConversion into another alarm correlation schemeIs the GED, i.e., the minimum value of the number of operation actions of (a).
3. The big-data-based information security event automatic association and quick response system according to claim 1, wherein the meta-alarm priority analysis module assigns a meta-alarm priority to each meta-alarm, so as to notify and delegate the related personnel to perform troubleshooting and repair in time according to the meta-alarm priority, and divides the meta-alarm priority into the following 4 levels:
wherein, g is a meta-alarm,is the weight of the LOF value.
4. The big-data-based information security event auto-correlation and quick response system as claimed in claim 3, wherein said big-data-based information security event auto-correlation and quick response systemThe method is obtained by the following steps:
(1) calculating k-neighbors and k-distances of the element alarm:
one element alarmK-neighbor adoption of() A representation is a collection of some other meta-alarms, any one of which is associated withThe difference therebetween is less than or equal to said k-distance; one isIs thatThe distance from the kth nearest neighbor alarm, k being a configurable parameter provided by the algorithm;
(2) calculating the reachable distance of the element alarm:
this is the maximum of the distance between two element alarms and the k-distance, as shown in the following equation:
r(g,)=max
(3) calculating the local reachable density of the element alarm:
the local reachable density of a meta-alarm is the inverse of the average reachable distance between it and its k-neighbors, as shown by the following equation:
Ir=（
(4) calculating local abnormal factors of the element alarm:
for each meta-alarm g, its LOF is calculated by the following formula:=
(5) calculating the priority of the element alarm:
the priority of the meta-alarm is calculated based on the difference degree of the meta-alarm, and the LOF value range of the difference degree of the meta-alarm is considered to be 0 toMeanwhile, the method utilizes the weighting technology to map the LOF value to 0EThe following formula shows:
nLOF（g）=。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510565165 | 2015-09-08 | ||
| CN2015105651658 | 2015-09-08 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105847029A CN105847029A (en) | 2016-08-10 |
| CN105847029B true CN105847029B (en) | 2019-08-09 |
Family
ID=56586983
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610130328.4A Active CN105847029B (en) | 2015-09-08 | 2016-03-09 | A kind of information security events auto-associating and quick response system based on big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105847029B (en) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106254125A (en) * | 2016-08-18 | 2016-12-21 | 南京联成科技发展有限公司 | The method and system of security incident correlation analysiss based on big data |
| CN107332717A (en) * | 2017-08-16 | 2017-11-07 | 南京联成科技发展股份有限公司 | A kind of new type of safe wisdom platform realizes framework |
| CN107479518A (en) * | 2017-08-16 | 2017-12-15 | 南京联成科技发展股份有限公司 | A kind of method and system for automatically generating alarm association rule |
| CN109995561B (en) * | 2017-12-30 | 2022-03-29 | 中国移动通信集团福建有限公司 | Method, device, equipment and medium for positioning communication network fault |
| CN108829794B (en) * | 2018-06-04 | 2022-04-12 | 北京交通大学 | Alarm analysis method based on interval graph |
| CN109064179B (en) * | 2018-07-11 | 2022-05-20 | 成都理工大学 | Mobile payment security situation perception system |
| CN111126729A (en) * | 2018-10-30 | 2020-05-08 | 千寻位置网络有限公司 | Intelligent safety event closed-loop disposal system and method thereof |
| CN109660526A (en) * | 2018-12-05 | 2019-04-19 | 国网江西省电力有限公司信息通信分公司 | A kind of big data analysis method applied to information security field |
| CN110149230B (en) * | 2019-05-20 | 2021-03-02 | 拉扎斯网络科技（上海）有限公司 | Service maintenance method and device, electronic equipment and readable storage medium |
| CN110933101B (en) * | 2019-12-10 | 2022-11-04 | 腾讯科技（深圳）有限公司 | Security event log processing method, device and storage medium |
| CN111224973A (en) * | 2019-12-31 | 2020-06-02 | 南京联成科技发展股份有限公司 | Network attack rapid detection system based on industrial cloud |
| CN112118141B (en) * | 2020-09-21 | 2021-12-17 | 中山大学 | Communication network-oriented alarm event correlation compression method and device |
| CN113377623B (en) * | 2021-07-02 | 2024-05-28 | 华青融天(北京)软件股份有限公司 | Automatic generation method and device of alarm rules and electronic equipment |
| CN113901452B (en) * | 2021-09-30 | 2022-05-17 | 中国电子科技集团公司第十五研究所 | Sub-graph fuzzy matching security event identification method based on information entropy |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101610174A (en) * | 2009-07-24 | 2009-12-23 | 深圳市永达电子股份有限公司 | A kind of log correlation analysis system and method |
| CN101697545A (en) * | 2009-10-29 | 2010-04-21 | 成都市华为赛门铁克科技有限公司 | Security incident correlation method and device as well as network server |
| CN101296122B (en) * | 2008-06-23 | 2011-04-20 | 中兴通讯股份有限公司 | Analytical method and device for alarm relativity |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070289013A1 (en) * | 2006-06-08 | 2007-12-13 | Keng Leng Albert Lim | Method and system for anomaly detection using a collective set of unsupervised machine-learning algorithms |
-
2016
- 2016-03-09 CN CN201610130328.4A patent/CN105847029B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101296122B (en) * | 2008-06-23 | 2011-04-20 | 中兴通讯股份有限公司 | Analytical method and device for alarm relativity |
| CN101610174A (en) * | 2009-07-24 | 2009-12-23 | 深圳市永达电子股份有限公司 | A kind of log correlation analysis system and method |
| CN101697545A (en) * | 2009-10-29 | 2010-04-21 | 成都市华为赛门铁克科技有限公司 | Security incident correlation method and device as well as network server |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105847029A (en) | 2016-08-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105847029B (en) | A kind of information security events auto-associating and quick response system based on big data | |
| CN112651006B (en) | Power grid security situation sensing system | |
| US11606373B2 (en) | Cyber threat defense system protecting email networks with machine learning models | |
| CN103001811B (en) | Fault locating method and device | |
| CN111565390B (en) | Internet of things equipment risk control method and system based on equipment portrait | |
| US20230012220A1 (en) | Method for determining likely malicious behavior based on abnormal behavior pattern comparison | |
| US10476752B2 (en) | Blue print graphs for fusing of heterogeneous alerts | |
| CN105376193B (en) | The intelligent association analysis method and device of security incident | |
| CN103441982A (en) | Intrusion alarm analyzing method based on relative entropy | |
| CN111181971B (en) | System for automatically detecting industrial network attack | |
| CN109302396A (en) | A kind of network security situational awareness method based on risk assessment | |
| CN104468193A (en) | Method for monitoring service system based on module finding | |
| WO2014096761A1 (en) | Network security management | |
| Ren et al. | Captar: Causal-polytree-based anomaly reasoning for scada networks | |
| Sen et al. | Towards an approach to contextual detection of multi-stage cyber attacks in smart grids | |
| CN107479518A (en) | A kind of method and system for automatically generating alarm association rule | |
| CN117729047B (en) | Intelligent learning engine method and system for industrial control network flow audit | |
| CN117792733A (en) | Network threat detection method and related device | |
| WO2017176676A1 (en) | Graph-based fusing of heterogeneous alerts | |
| TWI744545B (en) | Decentralized network flow analysis approach and system for malicious behavior detection | |
| Protic et al. | WK-FNN design for detection of anomalies in the computer network traffic | |
| Yan et al. | Sim-watchdog: Leveraging temporal similarity for anomaly detection in dynamic graphs | |
| CN118200019B (en) | Network event safety monitoring method and system | |
| Guan et al. | A summary of research on the false alarm judgment methods | |
| CN107332717A (en) | A kind of new type of safe wisdom platform realizes framework |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 210012, Nanjing high tech Zone, Jiangsu, Nanjing Software Park, No. 99 unity Road, Eagle building, block A, 14 floor Applicant after: Nanjing Liancheng science and technology development Limited by Share Ltd Address before: The small road line road in Yuhuatai District of Nanjing City, Jiangsu province 210012 Building No. 158 Building 1 new ideal Applicant before: NANJING LIANCHENG TECHNOLOGY DEVELOPMENT CO., LTD. |
|
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 210000 14F, building A, Eagle building, 99 solidarity Road, Nanjing Software Park, Nanjing hi tech Zone, Jiangsu Applicant after: Nanjing Liancheng science and technology development Limited by Share Ltd Address before: 210000, Nanjing high tech Zone, Jiangsu, Nanjing Software Park, No. 99 unity Road, Eagle building, block A, 14 floor Applicant before: Nanjing Liancheng science and technology development Limited by Share Ltd |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Information security event automatic association and rapid response system based on big data Effective date of registration: 20200328 Granted publication date: 20190809 Pledgee: Bank of Jiangsu, Limited by Share Ltd, Nanjing Jiangning branch Pledgor: NANJING LIANCHENG TECHNOLOGY DEVELOPMENT Co.,Ltd. Registration number: Y2020980001149 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right |
Date of cancellation: 20210521 Granted publication date: 20190809 Pledgee: Bank of Jiangsu Limited by Share Ltd. Nanjing Jiangning branch Pledgor: NANJING LIANCHENG TECHNOLOGY DEVELOPMENT Co.,Ltd. Registration number: Y2020980001149 |
|
| PC01 | Cancellation of the registration of the contract for pledge of patent right |