CN111259204B

CN111259204B - APT detection correlation analysis method based on graph algorithm

Info

Publication number: CN111259204B
Application number: CN202010033729.4A
Authority: CN
Inventors: 郭景楠
Original assignee: Shenzhen Leagsoft Technology Co ltd
Current assignee: Shenzhen Leagsoft Technology Co ltd
Priority date: 2020-01-13
Filing date: 2020-01-13
Publication date: 2023-04-11
Anticipated expiration: 2040-01-13
Also published as: CN111259204A

Abstract

The APT detection correlation analysis method based on the graph algorithm, provided by the invention, is used for acquiring terminal user behavior data and kernel-level data generated by a detection system so as to obtain original audit log data; storing the original audit log data into a database with a graph algorithm; performing TTP rule matching on the original audit log data according to the ATT & CK knowledge base model to obtain an alarm event; and evaluating the strength of the dependency relationship between the alarm events by utilizing the path association degree, constructing an APT attack scene graph, and upgrading the original audit log data to an APT attack step. The method analyzes the correlation between suspicious information streams of an attacker in an attack stage, detects the APT activity with high accuracy and low false alarm rate, can effectively summarize and backtrack the ongoing attack activity in real time, helps to perform real-time network response activity, and realizes the visualization of an attack scene.

Description

APT detection correlation analysis method based on graph algorithm

Technical Field

The invention belongs to the technical field of information security, and particularly relates to an APT (advanced persistent threat) detection association analysis method based on a graph algorithm.

Background

Advanced Persistent Threat (APT) is a form of attack that exploits Advanced attack means to carry out long-term Persistent cyber attacks on a specific target. The principle of the APT attack is more advanced and advanced relative to other attack forms, and the advanced nature is mainly reflected in that the APT needs to accurately collect the business process and the target system of an attack object before the attack is started. In the process of collecting, the attack actively excavates the vulnerabilities of the trusted system and the application programs of the attacked objects, builds the network required by the attackers by utilizing the vulnerabilities, and attacks by utilizing the 0day vulnerability.

The APT attack is mainly implemented for important industries and departments such as governments, energy sources and finance, and the advanced attack mode, advanced attack technology, continuous attack period and definite attack targets of the APT attack enable the attack to achieve accurate attack, and damage and loss which are difficult to estimate are caused. Therefore, reasonable evaluation of security situation and threat influence for the APT attack is urgent to provide assistant decision-making information for network administrators or security authorities.

The prior art remains very challenging in this ability to detect APT attacks:

the detection technologies of traditional firewalls, intrusion detection, security gateways, antivirus software, anti-spam systems and the like mainly detect network boundaries and host boundaries, and both of the technologies lack the detection capability and correlation analysis capability for APT attacks, particularly 0day attacks.

Secondly, the existing rule-based APT threat detection engine cannot effectively correlate a large number of alarm events, or can only simply correlate the alarm events by using some existing indexes such as timestamps and the like, so that the understanding of the complex relation between the alarm and the actual intrusion is lacked, and the attack events which occur on different hosts within a long time cannot be correlated into the whole threat event. On the other hand, relatively weak correlation analysis capability also causes a high false alarm rate of detection.

Disclosure of Invention

Aiming at the defects in the prior art, the invention provides the APT detection correlation analysis method based on the graph algorithm, so that the accuracy of APT attack detection is improved, and the false alarm rate of APT attack detection is reduced.

An APT detection correlation analysis method based on a graph algorithm comprises the following steps:

acquiring terminal user behavior data and kernel-level data generated by a detection system to obtain original audit log data;

storing the original audit log data into a database with a graph algorithm;

performing TTP rule matching on the original audit log data according to the ATT & CK knowledge base model to obtain an alarm event;

and evaluating the strength of the dependency relationship between the alarm events by utilizing the path relevance, constructing an APT attack scene graph, and promoting the original audit log data to an APT attack step.

Preferably, the kernel-level data generated by the detection system includes real-time operation information of the process in a file or network dimension; the database with the graph algorithm comprises a graph database, wherein nodes in the database represent entities and comprise processes, files and networks; the relationships in the database represent relationships between entities.

Preferably, the method for calculating the path association degree includes:

selecting a path containing key nodes among a plurality of alarm events;

respectively acquiring ancestor nodes of each node on the path;

and counting the number of different ancestor nodes on the path, and defining the number as the path association degree of the path.

Preferably, after the TTP rule matching is performed on the raw audit log data according to the ATT & CK knowledge base model, the method further includes:

and filtering the TTP rule according to the normally running original audit log data.

Preferably, the filtering the TTP rule according to the normally running original audit log data specifically includes:

collecting original audit log data which normally runs to obtain training data;

learning the training data by using a TTP specification to obtain a TTP rule frequently matched with the training data in the TTP rule matching process, and defining the TTP rule as a noise rule;

and when the TTP rule matching is carried out on the original audit log data collected in real time according to the ATT & CK knowledge base model, detecting that the TTP rule matched with the noise rule exists in the original audit log data collected in real time, and filtering the TTP rule.

Preferably, after the creating the APT attack scenario diagram, the method further includes:

and perfecting the APT attack scene graph by using a preset algorithm rule and an experience rule.

Preferably, after the constructing the APT attack scenario diagram, the method further includes:

setting a threat seven-tuple of each APT attack scene graph; the threat seven-tuple comprises the severity level of each attack stage of the APT attack scenario diagram in the TTP specification;

converting a threat seven-element group of the APT attack scene graph into a threat score vector according to a preset threat rule; wherein the threat rules comprise a score corresponding to each severity level, and the threat score vector comprises seven scores;

calculating the total score T of each APT attack scene graph according to the following formula:

wherein w _i For the weight of the ith attack stage of the APT attack scene graph in the TTP specification, n =7,S _i The score of the ith attack stage of the APT attack scene graph in the TTP specification;

and sequencing the APT attack scene graphs according to the total score T of all the APT attack scene graphs.

Preferably, after the T sequences the APT attack scenario diagram, the method further includes:

operating the APT attack scene graph in benign activities, and defining the maximum value of the total score of the APT attack scene graph in operation as a benign score;

running the APT attack scene graph in a malignant activity, and defining the minimum value of the total score of the APT attack scene graph in the running process as a malignant score;

selecting a value between the benign score and the malignant score, defining an alarm threshold;

and when detecting that the total score of the APT attack scene graph in real-time operation is greater than the alarm threshold value, alarming.

According to the technical scheme, the APT detection correlation analysis method based on the graph algorithm mainly solves the problem of correlation analysis of alarm events in APT detection, detects APT activities with high accuracy and low false alarm rate by analyzing the correlation between suspicious information streams of an attacker in an attack stage, can effectively summarize and backtrack ongoing attack activities in real time, helps to perform real-time network response activities, and realizes attack scene visualization.

Drawings

In order to more clearly illustrate the detailed description of the invention or the technical solutions in the prior art, the drawings that are needed in the detailed description of the invention or the prior art will be briefly described below. Throughout the drawings, like elements or portions are generally identified by like reference numerals. In the drawings, elements or portions are not necessarily drawn to scale.

Fig. 1 is a flowchart of an APT detection association analysis method according to an embodiment of the present invention.

Fig. 2 is a flowchart of a method for calculating a path relevance according to an embodiment of the present invention.

Fig. 3 is a flowchart of a TTP rule filtering method according to an embodiment of the present invention.

Fig. 4 is a flowchart of an APT attack scene graph scoring method according to a second embodiment of the present invention.

Fig. 5 is a flowchart of an alarm method according to a second embodiment of the present invention.

Detailed Description

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and therefore are only examples, and the protection scope of the present invention is not limited thereby. It is to be noted that, unless otherwise specified, technical or scientific terms used herein shall have the ordinary meaning as understood by those skilled in the art to which the invention pertains.

It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should be further understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items and includes such combinations.

As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".

The first embodiment is as follows:

an APT detection correlation analysis method based on graph algorithm, referring to fig. 1, includes the following methods:

s1: acquiring terminal user behavior data and kernel-level data generated by a detection system to obtain original audit log data; specifically, the kernel-level data generated by the detection system includes real-time operation information of the process in a file or network dimension.

S2: storing the original audit log data into a database with a graph algorithm; the database with graph algorithm comprises a graph database, wherein nodes in the database represent entities, including processes, files (including PE files) and networks; the relationships in the database represent relationships between entities.

In particular, the method may store raw audit log data in a graph database or other graph structured database. In the database, a node may have its own attributes, such as a process name, a start parameter, a file size, and the like. The relationship in the database includes the relationship between nodes, for example, the process P1 creates a sub-process P2, or the process P1 opens a file F, or the process P1 sends data to the network S. The relationship may also have its own properties, such as how much Byte data the process P1 sends to the network S.

S3: performing TTP rule matching on the original audit log data according to the ATT & CK knowledge base model to obtain an alarm event;

in particular, reference MITER ATT&CK ^TM The TTP specification mainly divides the behavior of an attacker into 7 Tactics (Tactics) including initial attack, establishment of foothold, internal detection, internal network diffusion, authority improvement, persistence of backdoor and data stealing. These 7 tactics, although progressive in steps, do not strictly adhere to sequence and completeness. Each tactical scenario contains several technologies (technologies), such as untrusted download traffic in the initial attack, unsigned PE for process start, etc.

The TTP specification provides a mapping between low-level audit events and APT attack steps. Specifically, the TTP specification mainly adopts two methods to map the original audit log data to the attack step: first, a general rule mapping is made using expert experience. Second, flow (i.e., path relatedness) mapping between nodes involved in the TTPs is used. Table 1 provides an example of TTP rules.

Table 1 TTP rule examples

In table 1, the first column represents the APT attack phase; the second column represents the associated TTP name; the third column indicates the severity level associated with each TTP: l, M, H and C respectively represent low, medium, high and extremely high; the fourth column represents the TTP rule, where s.ip does not belong to { Trusted IP Addresses } and p0.name belongs to { Sensitive Commands } using the first mapping method described above (mapping using common rules formulated by expert experience). The second mapping method (using information flow mapping between nodes involved in TTPs) is adopted for the path correlation (P0, F) < = path thres, where the path correlation is a calculation function of path correlation, and the path thres is an empirical value and can be determined according to actual scenario test conditions. The last column is a description of the TTP rules.

S4: and evaluating the strength of the dependency relationship between the alarm events by utilizing the path relevance, constructing an APT attack scene graph, and promoting the original audit log data to an APT attack step.

According to the method, the alarm events related to the APT attack stage are detected, and the alarm events are connected by using the path relevance, so that the whole attack chain used by an APT attacker can be constructed, the method is a high-level scene graph in the graph concept, and a very compact and visual attack summary is provided. The alarms generated by the user behavior data stream or log are directly mapped into the attack chain, and the alarms closer to the action steps (tactics, strategies and processes (TTPs)) of the APT attacker are realized.

Referring to fig. 2, a method for calculating a path correlation includes:

s11: selecting a path containing key nodes among a plurality of alarm events;

s12: respectively acquiring ancestor nodes of each node on the path;

s13: and counting the number of different ancestor nodes on the path, and defining the number as the path association degree of the path.

Specifically, the path relevance measures the relevance of a path between two nodes (entities) in a graph database, and the smaller the value, the greater the relevance. The method is mainly realized by depending on the coverage of ancestor nodes of the nodes in the path.

Example 1: the path P1- > P2- > P3- > P4- > P5 represents the creation process of a series of processes. In the path, ancestors of all nodes are P1, so path correlation (P1, P5) =1, which means that the node of the path only includes one ancestor node, and from practical point of view, P1 to P5 have strong correlation in information.

Example 2: in a path P1- > F1- > P2- > P3- > F2- > P4, P1 writes data to F1, P2 reads data from F1, and then a sub-process P3 is created, P3 writes a file F2, and F2 starts a process P4. In the path, ancestor nodes of each node are P1, P2, and P4, respectively, and the number (coverage) of ancestor nodes is 3, so path correlation (P1, P4) =3, which means that the node of the path only includes three ancestor nodes, and from an actual view point, the information association between P1 and P4 is weak. If the path thres is 2 in the above example (P0, F) < = path thres), the path association degree of the path is 3 and greater than 2, which indicates that the node in the path does not satisfy the TTP rule.

In addition, since there may be multiple paths between two nodes in the graph database, the method preferentially selects the shortest path with the key node. The key nodes mainly refer to files, networks and the like operated by the processes. The addition of the key nodes can be more accurate, the information flow dependence of the two nodes can be more reasonably measured, and more valuable correlation analysis is generated. For example, an unorushed Read (P0, S0) is a first attack, and when the association degree of the paths between the node P1 and the node P0 in the CnC (P1, S1) is to be calculated, the path to be queried is the shortest path between P0 and P1 that includes the network node S0.

The method can eliminate the dependence on the activity of the attacker irrelevant by the path relevance degree. For example, in example 2, it is assumed that P1 is an attack activity process nginx, which writes into a log file F1/usr/log/nginx-error, and then P2 (cat process) reads the file, although there is a dependency relationship between the cat process and the log file F1, the cat process is unrelated to the attack activity process nginx, and the cat process is independently invoked through ssh (abbreviation of Secure Shell). The cat process can therefore be considered a benign activity, a dependency unrelated to the attacker's activity.

As a further optimization scheme of the method, after the TTP rule matching is performed on the original audit log data according to the ATT & CK knowledge base model, the method further includes:

filtering the TTP rule according to the normally running original audit log data, referring to fig. 3, specifically includes:

s21: collecting original audit log data which normally runs to obtain training data;

s22: learning the training data by using a TTP specification to obtain a TTP rule frequently matched with the training data in the TTP rule matching process, and defining the TTP rule as a noise rule;

s23: and when the TTP rule matching is carried out on the original audit log data collected in real time according to the ATT & CK knowledge base model, detecting that the TTP rule matched with the noise rule exists in the original audit log data collected in real time, and filtering the TTP rule.

In particular, another challenge faced when actually making TTP rule matching is the presence of a large amount of noise. For example, long-lived processes (such as browsers, web servers, or SSH daemons) can trigger TTP rule matching, but in essence these processes are most likely benign events. Thus to reduce these false positives, the method utilizes learning frequently triggered TTP rules in normal data for each process, which are ignored if a TTP rule matching the noise rule encountered during training occurs at actual runtime. For example, qq. Exe creates a sub-process ipconfig.exe, because qq is a communication tool, the received data is a condition which occurs relatively often, and therefore, sensitive Command (P, P0) can be easily matched, so that a large number of matches of the Sensitive Command (P, P0) can also occur in the learning process of the training data, and therefore, the Sensitive Command (P, P0) is defined as a noise rule, and thus, in the actual detection link, if the Sensitive Command (P, P0) is matched, the noise rule can be ignored. The method also provides noise reduction technology based on benign activity training learning, and further weakens the dependence related to the known benign activity.

Table 2 gives an example of TTP rules for detecting phishing mail.

Table 2:

/>

specifically, the construction of the attack scenario diagram is mainly generated by driving of a TTP rule, and if the TTP rule is satisfied, the TTP is matched and added into the attack scenario diagram. The attack scene graph is based on the original graph, for example, events of original audit log data are mapped into a TTP attack matrix by using TTP rules in table 2, and are associated into an attack scene graph by using path correlation. The nodes in the attack scene graph are the nodes (processes, files and networks) meeting the TTP rule, and the edges represent the TTP rule or the original relationship existing among the nodes.

After the creation of the APT attack scenario diagram, the method further comprises the following steps:

and perfecting the APT attack scene graph by using a preset algorithm rule and an empirical rule.

Specifically, the method can utilize algorithm rules and empirical rules to perfect the attack scenario diagram, for example, a primary attack related service SE, and according to the knowledge and experience established previously, the creation process of SE1 is considered as P, so that the service SE created by the process P can be added into the attack scenario diagram. Because this is a graph relationship created empirically using knowledge, different relationship labels can be added to distinguish from the true existing relationship.

In summary, the method uses the log and the low-level information flow (file behavior, process behavior, etc.) in the detection system as the basis to perform data analysis to realize alarm association. For example, the internal reconnaissance step in the APT attack phase depends on the initial compromise and successful establishment of the foothold, so there is a flow of information between the processes involved in these two steps. The invention utilizes the graph analysis technology to correspondingly analyze and calculate the information flow between nodes (processes, files and networks), provides the concept of the path association degree, is a quantifiable numerical value capable of measuring the association relationship between the nodes, can evaluate the strength of the dependency relationship between alarm events through the path association degree, and then can delete weak dependency items so as to eliminate a plurality of false alarms.

In addition, the prior art may cause many related data not to be captured in the data collection stage for various reasons, such as the creation process of Windows system service, the creation process of task plan. The invention can associate the most possible related data by using the algorithm rule and the experience rule, and adds the data into the attack scene graph (particularly the nodes in the initial attack trap), thereby increasing the richness and the integrity of the attack scene graph and providing more perfect attack context information for a security administrator.

The second embodiment:

the second embodiment provides the following contents on the basis of the first embodiment:

after the method constructs the APT attack scene graph according to the screened path, referring to fig. 4, the method further includes:

s31: setting a threat seven-tuple of each APT attack scene graph; the threat seven-tuple comprises the severity level of each attack stage of the APT attack scenario diagram in the TTP specification;

s32: converting a threat seven-element group of an APT attack scene graph into a threat score vector according to a preset threat rule; wherein the threat rules include a score corresponding to each severity level, and the threat score vector includes seven scores;

s33: calculating the total score T of each APT attack scene graph according to the following formula:

wherein, w _i For the weight of the ith attack stage of the APT attack scene graph in the TTP specification, n =7,S _i The score of the ith attack stage of the APT attack scene graph in the TTP specification;

s34: and sequencing the APT attack scene graphs according to the total score T of all the APT attack scene graphs.

In particular, the method may also distinguish between high-confidence attacks, since multiple attack scenario maps may be generated. First, each attack scenario diagram corresponds to a threat seven-tuple and represents the severity level of each attack stage (initial attack, establishment of foothold, internal detection, intranet diffusion, authority promotion, persistence of backdoor, data stealing). The highest severity level is selected if an attack phase has multiple TTPs of different severity levels.

Example 3: (L, M, M, -, M, H, C) is a threat seven-tuple of an attack scenario, wherein '-' indicates that the attack activity does not have an intranet diffusion stage, and the TTP with the most serious attack activity is C, which occurs in a data stealing stage. Next, a plurality of attack scene graphs need to be sorted, that is, threat heptads are converted into threat score vectors, and a threat rule is given in table 3.

Table 3:

severity level	Fractional range	Mean value of
			L	[0.1,4.0)	2.0
M	[4.0,7.0)	6.0
			H	[7.0,9.0)	8.0
C	[9.0,10.0)	10.0

If a certain attack phase does not occur, the corresponding score value is set to 1. To sum up, the threat score vector for the threat seven-tuple (L, M, -, M, H, C) transformation in example 3 is (2,6,6,1,6,8, 10). And finally, converting the threat score vector into a total score, wherein the calculation method comprises the following steps:

wherein, w _i Weight of ith attack stage in TTP rule for APT attack scenario diagram, w _i ＝(10+i)/10；n＝7，S _i And (4) the score of the ith attack stage in the TTP rule for the APT attack scene graph. The total score for the threat score vector (2,6,6,1,6,8, 10) is therefore T =21.1 × 61.2 × 61.3 × 11.4 × 61.5 × 81.6 × 101.7=3878662.362.

The method sequences the generated high-level scene graph to delete most nodes and edges which are irrelevant to the APT attack activity, and can effectively distinguish attack and benign scenes.

After the T sequences the APT attack scenario diagrams, referring to fig. 5, the method further includes:

s41: operating the APT attack scene graph in benign activities, and defining the maximum value of the total score of the APT attack scene graph in operation as a benign score;

s42: running the APT attack scene graph in a malignant activity, and defining the minimum value of the total score of the APT attack scene graph in the running process as a malignant score;

s43: selecting a value between the benign score and the malignant score, defining an alarm threshold;

s44: and when detecting that the total score of the APT attack scene graph in real-time operation is greater than the alarm threshold value, alarming.

In particular, the method is trained in a supervised environment, and a threshold range can be derived that distinguishes benign activity from malicious activity. For example, assuming that the TTP rule formulated in table 2 is operated in a benign event, the highest total score of the APT attack scenario diagram obtained by the above method is 480. When the APT attack scene graph is operated in a malicious environment, the lowest total score of the APT attack scene graph obtained by the method is 13200, so that a value can be selected from 480-13200 to serve as a threshold value for distinguishing benign activities and malicious activities. And when detecting that the total score of the APT attack scene graph is greater than the alarm threshold value, alarming. For example, the total score of the scenario diagram of the APT attack in example 3 is significantly larger than 13200, so that the event is considered as a phishing event.

For the sake of brief description, the method provided by the embodiment of the present invention may refer to the corresponding contents in the foregoing method embodiments.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the present invention, and they should be construed as being included in the following claims and description.

Claims

1. An APT detection correlation analysis method based on a graph algorithm is characterized by comprising the following steps:

storing the original audit log data into a database with a graph algorithm;

evaluating the strength of the dependency relationship between alarm events by utilizing the path correlation degree, constructing an APT attack scene graph, and promoting the original audit log data to an APT attack step;

the method for calculating the path relevance comprises the following steps:

selecting a path containing key nodes among a plurality of alarm events;

respectively acquiring ancestor nodes of each node on the path;

2. The APT detection correlation analysis method based on graph algorithm according to claim 1,

the kernel-level data generated by the detection system comprises real-time operation information of a process under a file or network dimension; the database with the graph algorithm comprises a graph database, wherein nodes in the database represent entities and comprise processes, files and networks; the relationships in the database represent relationships between entities.

3. The APT detection correlation analysis method based on graph algorithm according to claim 2, after the TTP rule matching is performed on the original audit log data according to the ATT & CK knowledge base model, the method further comprises:

4. The APT detection correlation analysis method based on the graph algorithm according to claim 3, wherein the filtering TTP rules according to the normal running original audit log data specifically comprises:

collecting original audit log data which normally runs to obtain training data;

5. The APT detection correlation analysis method based on graph algorithm according to claim 4, characterized in that after the APT attack scenario graph is constructed, the method further comprises:

6. The APT detection correlation analysis method based on graph algorithm according to claim 4, characterized in that after the APT attack scenario graph is constructed, the method further comprises:

setting a threat seven-tuple of each APT attack scene graph; the threat seven-tuple comprises the severity level of each attack stage of the APT attack scenario diagram in the TTP specification; the attack stage comprises primary attack trapping, establishment of foothold, internal detection, intranet diffusion, authority promotion, rear door persistence and data stealing;

converting a threat seven-element group of the APT attack scene graph into a threat score vector according to a preset threat rule; wherein the threat rules include a score corresponding to each severity level, and the threat score vector includes seven scores;

/>

7. The APT detection correlation analysis method based on graph algorithm according to claim 6,

after the T sequences the APT attack scene graph, the method further comprises the following steps:

selecting a value between the benign score and the malignant score, and defining the value as an alarm threshold value;