CN115309907B

CN115309907B - Alarm log association method and device

Info

Publication number: CN115309907B
Application number: CN202211223822.7A
Authority: CN
Inventors: 文洲; 朱金涛; 朱震
Original assignee: Beijing Shengxin Network Technology Co ltd
Current assignee: Beijing Shengxin Network Technology Co ltd
Priority date: 2022-10-08
Filing date: 2022-10-08
Publication date: 2022-12-27
Anticipated expiration: 2042-10-08
Also published as: CN115309907A

Abstract

The invention provides an alarm log association method and device, which comprises the steps of firstly obtaining entity elements corresponding to each attack alarm log; then based on the access relation among the entity elements, forming the entity elements corresponding to each attack alarm log into a corresponding first process tree; and then associating all the acquired first process trees into one or more attack events according to preset association rules (including process dimension rules and time dimension rules). The invention can efficiently complete the real-time association of various alarm logs, and has the advantages of accuracy, flexibility and good expansibility.

Description

Alarm log association method and device

Technical Field

The invention relates to the technical field of communication, in particular to an alarm log association method and device.

Background

The intrusion attack process is composed of a plurality of stages/a plurality of attack points and often comprises a plurality of alarm data, but the current various intrusion alarm systems generally judge whether a single action triggers an alarm or not based on rules, the generated alarm reporting data are all independent, and because a single alarm cannot depict the complete intrusion attack process, a user needs to manually and repeatedly compare the alarm to put the related alarms together, so that the investigation and backtracking costs are extremely high. In the attack alarm detection process, the same alarm may hit a plurality of detection strategies, an unreasonable rule can trigger a large number of false alarms, the sequential troubleshooting of the alarms is time-consuming and labor-consuming, and the true attack alarm can be missed.

The traditional alarm log association method mainly comprises the following steps: reducing the number of alarms by time period slice association may result in a large number of false associations. By calculating the correlation through vector similarity, the correlation with larger calculation amount has poor interpretability. The correlation is compared in a step-by-step matching and knowledge base mode, and the calculation amount is large in the mode. The rule comparison mode is used for matching and associating, and the mode has high expandability and large calculation amount. The method has the advantages that the network security event graph is constructed firstly, then the network security event graph is divided into a plurality of alarm clusters, and then the statistical characteristic and the topological structure characteristic corresponding to each alarm cluster are extracted to associate different alarms, so that the method has large calculated amount and is difficult to associate in real time. And generating corresponding correlation events based on the self-built graph database, wherein the correlation events comprise correlation logic of a process dimension and a time dimension, but the required storage cost is high.

Therefore, the traditional alarm log association mode has the following problems:

(1) Large calculation amount and insufficient real-time property: step-by-step matching or vector similarity calculation is involved, and the calculation amount and the storage resource occupation are large; the correlation is difficult to realize in real time due to large calculation amount, most of the correlation is batch processing task calculation, real correlated attacks are easily divided into different attack events by calculation correlation after pure time period division (such as division according to hours), and the correlation real-time performance is not enough.

(2) The correlation dimension is single: it relates to a step-by-step matching according to time slicing, comparison approximation or according to rules, not in accordance with the case where the real association needs to take into account the time range and (process) access relations.

(3) The correlation is not accurate enough: only the probability of association is considered, and false association is easy to occur.

(4) Poor expandability: for the new alarm types, corresponding to new rules or vector calculation modes are required, and a uniform and general association algorithm cannot be established.

Disclosure of Invention

In view of the above, an object of the present invention is to provide an alarm log association method and apparatus, so as to efficiently complete real-time association of multiple alarm logs, and the method is accurate, flexible and good in expansibility.

In a first aspect, an embodiment of the present invention provides an alarm log association method, where the method includes: acquiring entity elements corresponding to each attack alarm log; based on the access relation among the entity elements, forming the entity elements corresponding to each attack alarm log into a corresponding first process tree; wherein, the nodes in the first process tree represent entity elements, and the connecting lines between the nodes in the first process tree represent the access relations between the entity elements; according to a preset association rule, associating all the acquired first process trees into one or more attack events; the preset association rules comprise a process dimension rule and a time dimension rule.

In a second aspect, an embodiment of the present invention further provides an alarm log association apparatus, where the apparatus includes: the entity element acquisition module is used for acquiring entity elements corresponding to each attack alarm log; the attack graph generation module is used for forming entity elements corresponding to each attack alarm log into a corresponding first process tree based on the access relation among the entity elements; wherein, the nodes in the first process tree represent entity elements, and the connecting lines between the nodes in the first process tree represent access relations between the entity elements; the association module is used for associating all the acquired first process trees into one or more attack events according to a preset association rule; the preset association rules comprise a process dimension rule and a time dimension rule.

According to the alarm log association method and device provided by the embodiment of the invention, entity elements corresponding to each attack alarm log are obtained firstly; then based on the access relation among the entity elements, forming the entity elements corresponding to each attack alarm log into a corresponding first process tree; and then associating all the acquired first process trees into one or more attack events according to preset association rules (including process dimension rules and time dimension rules). By adopting the technology, the real-time association of various alarm logs can be efficiently completed, and the method is accurate, flexible and good in expansibility.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.

FIG. 1 is a flow chart illustrating a method for associating alarm logs according to an embodiment of the present invention;

FIG. 2 is a diagram illustrating an overall work flow of the system according to an embodiment of the present invention;

FIG. 3 is an exemplary diagram of a Detect tree in an embodiment of the present invention;

FIG. 4 is an exemplary diagram of completing a partial Detect tree in an embodiment of the present invention;

FIG. 5 is an exemplary diagram of tagging network access relationships with quadruplets in an embodiment of the present invention;

FIG. 6 is a diagram illustrating an exemplary marking of ip address access relationships, website access relationships, and file access relationships according to an embodiment of the present invention;

FIG. 7 is an exemplary diagram illustrating a scan relationship of markup files in an embodiment of the present invention;

FIG. 8 is an exemplary diagram of association according to process dimension rules in an embodiment of the invention;

FIG. 9 is a diagram illustrating an exemplary structure of a virtual table in an embodiment of the present invention;

FIG. 10 is a flowchart illustrating an exemplary step in process according to an embodiment of the present invention;

FIG. 11 is a diagram illustrating an example of a process dimension association procedure in an embodiment of the present invention;

FIG. 12 is a diagram illustrating an example of implementing codes for a default white list according to an embodiment of the present invention;

FIG. 13 is an exemplary diagram of process dimension association in an embodiment of the invention;

FIG. 14 is an exemplary diagram of a time dimension association in an embodiment of the invention;

fig. 15 is a schematic structural diagram of an alarm log association apparatus in an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

At present, the following problems exist in the traditional alarm log association mode:

(1) Large calculation amount and insufficient real-time property: step-by-step matching or vector similarity calculation is involved, and the calculation amount and the storage resource occupation are large; the real-time association is difficult to achieve due to the fact that the calculation amount is large, most of the association is batch processing task calculation, real associated attacks are easily divided into different attack events through calculation of association after simple time period division (for example, division according to hours), and association instantaneity is not enough.

(4) Poor expandability: for the newly added alarm types, new rules or vector calculation modes are required to be correspondingly added, and a unified and universal association algorithm cannot be established.

Based on the method and the device for associating the alarm logs, provided by the embodiment of the invention, the real-time association of various alarm logs can be efficiently completed, and the method and the device are accurate, flexible and good in expansibility.

To facilitate understanding of the embodiment, first, a detailed description is given to an alarm log association method disclosed in the embodiment of the present invention, referring to a flow diagram of an alarm log association method shown in fig. 1, where the method may include the following steps:

and step S102, acquiring entity elements corresponding to each attack alarm log.

Wherein, the entity elements may include: host, ip address, process, file, web address, container, etc.

Step S104, based on the access relation among the entity elements, forming the entity elements corresponding to each attack alarm log into a corresponding first process tree; the nodes in the first process tree represent entity elements, and the connecting lines among the nodes in the first process tree represent access relations among the entity elements.

The access relation can be a process access relation, a network access relation, an ip address access relation, a website access relation, a file access relation and the like, and can be customized according to actual conditions without limitation.

Step S106, according to a preset association rule, associating all the acquired first process trees into one or more attack events; the preset association rules comprise process dimension rules and time dimension rules.

After entity elements (such as ip addresses, processes, files, websites and the like) corresponding to each attack alarm log are combined into a corresponding first process tree, the first process trees with the association relation can be associated into the same attack event from the process dimension and the time dimension according to a preset association rule.

The embodiment of the invention provides a method for associating alarm logs, which comprises the steps of firstly obtaining entity elements corresponding to each attack alarm log; then based on the access relation among the entity elements, forming the entity elements corresponding to each attack alarm log into a corresponding first process tree; and then associating all the acquired first process trees into one or more attack events according to preset association rules (including process dimension rules and time dimension rules). By adopting the technology, the real-time association of various alarm logs can be efficiently completed, and the method is accurate, flexible and good in expansibility.

As a possible implementation manner, after the step S102 (i.e., obtaining the entity element corresponding to each attack alarm log), the alarm log associating method may further include the following operation manners: and storing the obtained entity elements corresponding to each attack alarm log and the relationship between the entity elements as an attack graph based on a preset attack graph storage model. Based on this, the step S106 (i.e. associating all the acquired first process trees into one or more attack events according to the preset association rule) may include the following operation modes: and for each newly obtained first process tree, associating the first process tree with the existing first process tree in the attack graph according to a preset association rule, and updating the attack graph by using the association result of the first process tree.

In order to facilitate operation, in the process of acquiring the attack alarm log in real time, when an attack alarm log is acquired newly, the newly acquired attack alarm log is used as a current attack alarm log, and the relationship between an entity element corresponding to the current attack alarm log and the entity element is presented in the form of an attack graph and stored through a pre-constructed attack graph storage model. In the attack graph storage model, an attack graph comprises points and edges, each point can uniquely represent one entity element (namely one node in a first process tree), and each edge (namely a connecting line between two points) can represent a relation (such as an access relation, a parent-child relation and the like) between one entity element. After the current first process tree corresponding to the current attack alarm log is obtained, the current first process tree is associated with the historical first process tree corresponding to the historical attack alarm log (namely the existing first process tree in the attack graph) according to a preset association rule, so that an association result is written into the attack graph through a pre-constructed attack graph storage model to update the attack graph, and the updated attack graph comprises the current first process tree and an association relation between the current first process tree and the historical first process tree.

As a possible implementation manner, the step S106 (i.e., associating all the acquired first process trees into one or more attack events according to a preset association rule) may include the following operation manners: determining a correlation point in each first process tree according to a process dimension rule, and correlating the first process trees with the same correlation point into the same sub-event to obtain one or more sub-events; wherein, the association point is a node which represents the beginning of the attack in the first process tree; and according to the time dimension rule, aggregating one or more sub-events into one or more attack events.

As a possible implementation manner, the step of determining the association point in each first process tree according to the process dimension rule may include the following operation manners: for each first process tree, matching the first process tree with one or more preset attack paths; if the matching is successful, the endmost node of the longest matching preset attack path is used as the association point in the first process tree; and if the matching fails, traversing the first process tree along the sequence from the root node to the child nodes based on a preset node white list, and taking the node which is traversed for the first time in the first process tree and is not in the preset node white list as an associated point in the first process tree.

The preset attack path is usually a tree structure (similar to the structure of the first process tree) formed by connecting a plurality of nodes, and a blacklist may be predefined and one or more preset attack paths may be stored in the blacklist; for a certain first process tree, the first process tree can be respectively matched with each preset attack path in the blacklist; if the first process tree is matched with one or more preset attack paths, judging that the matching is successful; if the plurality of preset paths are matched, determining the end node of the longest matched preset path as a correlation point; and if the blacklist does not have a preset attack path matched with the first process tree, judging that the matching fails, traversing the first process tree along the sequence from the root node to the child node, sequentially judging whether the nodes in the first process tree are in a preset node white list according to the traversing sequence, stopping traversing until the nodes which are not in the preset node white list are traversed for the first time in the first process tree, and determining the nodes which are not in the preset node white list in the first process tree at the moment as the associated points.

By adopting the operation mode, the accuracy and the efficiency of determining the association point can be further improved.

As a possible implementation manner, the alarm log association method may include the following operation manners:

(11) And if the association point in the first process tree is not in the attack graph, taking the association point as a new association point, and associating the first process tree in a new sub-event in the attack graph.

For the obtained current first process tree, after obtaining the node traversed in the first process tree and not in the preset node white list, if the node does not exist in the attack graph, the node can be regarded as a new association point, a new sub-event is established, and then the first process tree is associated in the established new sub-event in the attack graph.

(12) If the first process tree meets the preset condition, associating the first process tree in a new sub-event in the attack graph; the preset conditions comprise the number of nodes in the sub-event and the time information of the sub-event.

The time information of the sub-event may include a save time of each first process tree in the sub-event. For example, a node number threshold (e.g., 100 nodes, 200 nodes, etc.) and a plurality of different time ranges (e.g., 24 hours, 48 hours, etc.) are preset; for the current first process tree, when the current first process tree needs to be associated in an existing sub-event in the attack graph, if the time information of the sub-event after association is within the time range and the number of nodes in the sub-event after association does not exceed the threshold of the number of nodes, the current first process tree can be associated in the sub-event in the attack graph; if the time information of the sub-event where the first process tree is located after the association is out of the time range and/or the number of the nodes in the sub-event exceeds the threshold of the number of the nodes after the association, a new sub-event can be established, and the current first process tree is associated in the established new sub-event in the attack graph. The operation mode can avoid the occurrence of too long association and too many node association, thereby improving the presentation effect of the association result.

As a possible implementation manner, the step of aggregating one or more sub-events into one or more attack events according to the time dimension rule may include the following operation manners: and aggregating the time information and the sub-events corresponding to the same time range into the same attack event based on a plurality of preset different time ranges and the time information of each sub-event, so as to obtain one or more attack events.

For example, a plurality of different time ranges (e.g., 1 day, 2 days, etc.) are preset, and sub-events with time information within the same time range can be aggregated into one attack event, so that one or more attack events are obtained.

In the real attack process, the alarm in a short time is triggered by the same attack, so the way of aggregating the sub-events can reflect the time correlation of the real attack process, avoid the correlation of overlong time and the correlation of too many nodes, and improve the presentation effect of the correlation result.

As a possible implementation manner, after the step S102 (i.e. obtaining the entity element corresponding to each attack alarm log and obtaining the entity element corresponding to each attack alarm log), the alarm log associating method may further include the following operation manners:

(21) And respectively determining the type of each access relation based on a plurality of preset different virtual table structures.

The attack alarm log usually contains one or more virtual tables, and the different access relation types can cause different combination modes of the virtual tables, and the different combination modes of the virtual tables can form different virtual table structures, so the access relation type of the attack alarm log can be judged according to the virtual table structures.

(22) And respectively correspondingly allocating corresponding type identifications for each access relation based on the determined type of each access relation.

The type identification is used for representing the type of the access relation, and for a certain access relation, the type of the access relation can be uniquely determined according to the type identification. For example, for each network access relationship, a corresponding quadruplet is allocated to the network access relationship as a type identifier of the network access relationship; the quadruplet includes a source ip address, a source port, a destination port and a destination ip address, wherein the source port usually corresponds to a source process, and the destination port usually corresponds to a destination process. The above way of allocating type identifiers to the access relationships can improve the recognition degree of different types of access relationships in the attack graph, so as to facilitate the query of the attack graph.

By adopting the operation modes of (21) and (22), the efficiency of attack graph query can be improved.

As a possible implementation manner, after the step of determining the association point in each first process tree according to the process dimension rule, associating the first process trees having the same association point with the same sub-event to obtain one or more sub-events, the alarm log association method may further include the following operation manners: for each first process tree associated to the same sub-event, the sub-event identifier of the sub-event is used to represent the association relationship between the first process tree and the sub-event. By adopting the operation mode, the existing association relation of the historical process tree can be stored, and the association points of different sub-events can be marked, so that the efficiency of inquiring the attack graph is improved.

As a possible implementation manner, after the step of aggregating one or more sub-events into one or more attack events according to the time dimension rule, the alarm log association method may further include the following operation manners: for each sub-event aggregated into the same attack event, the event identification of the event is used for representing the aggregation relation between the sub-event and the event. By adopting the operation mode, the existing association relation of the historical process tree can be stored, and the sub-events correspondingly contained in different events can be marked, so that the efficiency of inquiring the attack graph is improved.

Aiming at the condition that the attack alarm log is a file scanning alarm log, the process corresponding to the attack alarm log cannot be obtained, namely the entity element corresponding to the attack alarm log does not comprise the process; based on this, the alarm log association method may further include the following operation modes: and associating the attack alarm logs with the same target entity elements into the same attack event based on the scanning relation among the target entity elements.

The target entity elements may be hosts, ip addresses, files, containers, and the like, and may be specifically defined according to actual situations, without limitation.

Because the file scanning alarm can not find the corresponding process, the file scanning alarm logs with the same target entity element can be regarded as the file scanning alarm logs with the association relation, and the file scanning alarm logs with the association relation are associated into the same attack event, so that the association of the file scanning alarm logs is realized. For example, the entity elements corresponding to the attack alarm logs comprise a host, an ip address and files, and the attack alarm logs with the same files can be associated into the same attack event based on the scanning relationship among the files. The operation mode can provide a feasible operation mode for the association of the file scanning alarm log, thereby efficiently finishing the real-time association of the file scanning alarm log, being accurate, flexible and good in expansibility, and further improving the universality of the alarm log association method to different application scenes.

For convenience of understanding, the alarm log association method is exemplarily described as follows by taking a specific application example as an example:

in order to facilitate the implementation of the alarm log association method, a system may be set up in advance, as shown in fig. 2, a system architecture mainly includes a host, a server and an association engine (part of a dashed-line frame in fig. 2) connected in sequence; the host is deployed on the client, and an operating system adopted by the host can adopt Linux, windows and the like, and can be selected according to requirements without limitation. The overall work flow of the system mainly comprises the following steps:

step 1, an Agent collector of the host is responsible for collecting alarm messages (Detect messages in fig. 2), and the collected Detect messages are uploaded to a server according to an agreed format.

And 2, carrying out intermediate processing (including a processing process of judgment and conversion) on the Detect message through the service end, and forwarding the Detect message after the intermediate processing to the correlation engine.

And step 3, converting the Detect message forwarded by the server into a corresponding Detect tree (such as the first process tree) through the association engine, and performing graph entry and association of the Detect tree.

The association engine is integrally composed of a graph input part and an association part. Constructing an attack graph based on an input Detect tree through an graph input part of an association engine, sending the input Detect tree to an association part of the association engine, and synchronizing composition information (such as a tree structure, a graph structure classification and the like) of each Detect tree corresponding to the attack graph to the Detect tree currently processed by the association part; on the basis, carrying out process dimension association on the input Detect tree through an association part of an association engine, namely, merging the corresponding Detect process tree into a sub-event (SubIncident); carrying out time dimension association on the sub-events through an association part of an association engine, namely aggregating the sub-events into events (Incident); in addition, the association engine can also write the Incident relationship generated in the process dimension association process into the attack graph to realize real-time update of the attack graph, and query the attack graph in the process dimension association process to determine corresponding Incident association information.

Step 4, outputting the Incident message corresponding to each event to a server by the correlation engine in a segmented transmission mode; each of the included messages includes all relevant information (e.g., a tree structure, a graph structure classification, the event identifier, the sub-event identifier, etc.) of the corresponding event.

Because the time dimension association may generate a large tree structure, the message transmission limitation and the rendering presentation of the front end of the distributed message system such as kakfa are influenced. Therefore, each time updated or newly-built sub-content is output during output, and each sub-content marks the corresponding content.

And 5, receiving the Incident message output by the association engine through the server, merging the Incident message and performing rendering display of the event through the front end.

In order to facilitate the implementation of the above steps 1 to 5, the above attack graph storage model may be constructed in advance, where table 1 shows the meaning of the point in the attack graph, and table 2 shows the meaning of the edge (i.e., the relationship between two points) in the attack graph.

Table 1 list of meanings of points in attack graph

Name(s)	Description of the preferred embodiment
		Ip	Ip address
Agent	Main unit
		Process	Process
Dns	Website address
		Connect	Four-tuple (source ip-source port-destination ip) to uniquely mark one-time network access relationship
File	File path
		Detect	The reported original alarms are represented as a Detect process tree after each original alarm enters a graph
SubIncident	Aggregating a plurality of Detect process trees by each sub-event tree after association according to process dimension
		Incident	Correlating generated eventsAggregating multiple SubIncident trees per Incident in a time dimension

Table 2 list of meanings of edges in attack graph

Each original alarm log is represented as a Detect tree (such as the first process tree) in the attack graph, and the relationship between points can be divided into multiple types of graph relationships according to different entity elements and different alarm types. The action source (generally, process, file, etc.) and the action object (generally, process, web address, ip address, file, etc.) of the current alarm (DETECT) are respectively marked by DETECT _ POINT and DETECT _ ENDPOINT.

Illustratively, for example, in fig. 3, the current alarm is an alarm (such as a malicious command execution) including a process action, the alarm carries complete process tree information, and is represented as a multi-level Detect tree in fig. 3, entity elements involved in the alarm include a host, an ip address corresponding to the host, a parent process (i.e., process 1) serving as a root node of the process tree, and a child process (i.e., process 2) of the parent process, and both a source and an object of the current alarm are process 2.

In a few scenarios, such as a situation where a complete PROCESS tree cannot be obtained, for example, as shown in fig. 4, there is a complete PROCESS tree (i.e., alarm 1) in an existing attack graph, at this time, an alarm 2 is newly reported, but the alarm 2 can only be obtained from parent and child two-level processes (i.e., PROCESS 2 and PROCESS 4), at this time, a HOST _ PROCESS (i.e., HOST to PROCESS) can be used to mark a relationship between a HOST and the PROCESS 2, so as to obtain an incomplete PROCESS tree (i.e., alarm 2). For the case that the process action reports that the process tree is incomplete, before associating the alarm log, whether a node in the current incomplete process tree has a superior node or not can be inquired, and then the process tree relationship in the existing attack graph is used for completing the current incomplete process tree, for example, as shown in fig. 4, a parent-child relationship between a process 1 and a process 2 in an alarm 1 can be used for completing an alarm 2, that is: and marking the relation between the host and the PROCESS 1 by using PROCESS _ ROOT (namely the host to the ROOT node of the PROCESS tree), and marking the relation between the PROCESS 1 and the PROCESS 2 by using Process _ CREATE (namely the parent-child PROCESS relation), thereby realizing the completion of the alarm 2. And other alarms with incomplete process trees can complete incomplete Detect trees in a similar completion mode.

For example, in fig. 5, the network access relationship describes an access relationship between two hosts, and at this time, a quadruplet [ source ip address-source port-destination ip address ] may be introduced as an intermediate element to uniquely mark a primary network access relationship, where the source port corresponds to a source process and the destination port corresponds to a destination process. The port of each host corresponds to a process, a quadruple [ Ip address 1-port 2-port 4-Ip address 2] can be used as a middle element in fig. 5 to uniquely mark a primary network connection, the port 2 corresponds to the process 2, the port 4 corresponds to the process 4, corresponding edges are respectively established between each entity element and the quadruple, and the dotted part in fig. 5 is a Detect tree on another host. The automatic association of the access relation among the multiple hosts can be completed by constructing the quadruple and establishing the operation mode of the corresponding edges between the entity elements and the quadruple. In addition, the network access relationship between the containers may also adopt a similar operation mode, which is not described in detail herein.

Illustratively, the ip address access relationship, the website access relationship and the file access relationship can be hung at the tail end of the process tree. For example, in fig. 6, an IP address ACCESS relationship may be marked with IP ACCESS (i.e., process to IP ACCESS), a website ACCESS relationship may be marked with DNS ACCESS (i.e., process to foreign url), and a FILE ACCESS relationship may be marked with FILE ACCESS (i.e., process to FILE ACCESS/execution).

For example, for a part of scenarios, such as a case of a file scan alarm, reporting the alarm without a process tree, and thus failing to acquire a corresponding process, at this time, a corresponding file may be directly acquired, and a degradation process may be performed on the corresponding alarm (i.e., a Detect tree), that is, a host (or a container) having the same file is associated with the Detect tree. For example, in fig. 7, a FILE scan relationship may be marked with HAS _ FILE (i.e., host to FILE) so as to form a Detect tree with the host, the IP (i.e., IP address) corresponding to the host, and the FILE (i.e., FILE path) owned by the host.

For example, in order to record the association relationship between different alarms, an auxiliary entity may be further introduced to characterize the association relationship between the alarms on the basis of adopting the Detect tree to characterize the relationship between entity elements corresponding to each alarm (i.e., detect). For example, one or more Detect trees can be associated into a sub-event (subencident) according to the process dimension rule, and a corresponding edge is respectively established between each Detect tree and the subencident; one or more sub-content are aggregated into an attack event (content) according to the time dimension rule, and a corresponding edge is respectively established between each sub-content and the content. For example, fig. 8, alarm 1 is associated into a sub-event (i.e., sub-event 1) according to the above process dimension rule, and the association between alarm 1 and sub-event 1 is marked by using basic _ DETECT (i.e., sub-event points to alarm), and the association between process 2, which is an association point of alarm 1, and sub-event 1 is marked by using basic _ ROOT (i.e., association point relationship); and then aggregating the sub-events 1 into an attack event (namely, event) according to the time dimension rule, and marking the association relationship between the sub-event 1 and the event by using SUBINCIDENT _ INCIDENT (namely, sub-event pointing to event).

Exemplarily, as shown in fig. 9, the details in the Detect includes various virtual tables (process _ tree)/network access (process _ open _ sockets)/file scan (file)/script execution (script), and the corresponding graph structure classification may be determined according to different virtual table combinations actually reported. For example, if only the virtual table process tree (process _ tree) is included, the process _ tree is determined as the process access relationship; meanwhile, if the virtual table process tree (process _ tree)/network access (process _ open _ sockets) is included, the network access relation is determined; a file containing only a virtual table file scan (file) is determined to be a file scan relationship, and so on.

Exemplarily, since various behavior logic dependency relationships of a host (or a container) are recorded by a reported alarm (Detect), and different detects are different from each other corresponding to points and edges in an attack graph, in order to avoid that a Detect needs to define an entry graph flow (i.e., a flow for generating an attack graph) and a point edge composition of the Detect every new Detect, the Detect can be classified into different graph structures in advance according to different structures by referring to the attack graph storage model, each graph structure classification corresponds to one relationship type (such as a process access relationship, a network access relationship, an ip address access relationship, a website address access relationship, a file scanning relationship, and the like), and the entry graph flow and the point edge composition of the same graph structure classification are the same; the mapping relationship between the label (i.e., type identifier) and the edge of each graph structure classification is then written into the corresponding configuration file. Based on this, for example, in fig. 10, the mapping part of the association engine may perform the mapping process according to the following operation modes: receiving each Detect (i.e., detect tree) sent by the server in JSON format (or other format); respectively judging the graph structure classification of each Detect based on a plurality of preset different virtual table structures; respectively analyzing configuration files corresponding to the graph structure classifications of each Detect to obtain mapping relations between labels and edges of the corresponding graph structure classifications; constructing a corresponding graph structure classification according to the configuration; and processing the received Detect tree into an attack graph and writing the attack graph into a graph database, and synchronizing an Incident message generated in the association process to kafka.

Illustratively, the virtual table structure may actually be one virtual table or a combination of multiple virtual tables, each virtual table having corresponding configuration parameters. For example, as shown in fig. 10, the detail of Detect includes various virtual tables, such as process tree (process _ tree), network access (process _ open _ sockets), and file scan (files), and the corresponding graph structure classification can be determined according to combinations of different virtual tables in Detect actually reported. For example, if the details of the Detect only includes a virtual table of a process tree (process _ tree), it is determined that the graph structure classification of the Detect corresponds to the process access relationship; if the detail of the Detect comprises two virtual tables of a process tree (process _ tree) and a network access (process _ open _ sockets), judging that the graph structure classification of the Detect corresponds to the network access relation; if the details of the Detect only contains a virtual table of file scanning (files), judging that the graph structure classification of the Detect corresponds to the file scanning relation; if the details of the Detect only comprises two virtual tables of a process tree (process _ tree) and an Ip access (Ip _ access), judging that the graph structure classification of the Detect corresponds to the Ip access relation; if the details of the Detect only comprises two virtual tables of a process tree (process _ tree) and a Dns access (Dns _ events), judging that the graph structure classification of the Detect corresponds to the DNS access relation; if the details of the Detect only comprises two virtual tables of a process tree (process _ tree) and a file access (process _ open _ files), judging that the graph structure classification of the Detect corresponds to the file access relation.

The graph entry flow is mainly used for converting reported alarms into corresponding graph structure classes to be added into the attack graph, and subsequent alarms can be automatically converted into corresponding graph structure classes to be added into the attack graph as long as the subsequent alarms meet the agreed virtual table format and combination, so that the graph entry flow has high expandability.

In order to facilitate the association part of the association engine to sequentially perform process dimension association and time dimension association on the Detect tree, a graph association algorithm may be defined in advance, so that the Detect tree is firstly merged into the subordinate through the graph association algorithm, and then the subordinate is merged into the included through the graph association algorithm.

The association of the alarm needs to accurately control the association process, so that one attack process can be truly embodied. For example, process trees on a linux host are all started from a root process, all processes cannot be merged into one process tree, and different bash entries or ssh login entries can be divided into different associated events; in addition, different attack processes and process trees generated by attack users need to be merged into different sub-clients as much as possible.

For example, the association point determination rule of Detect may be predefined and then written into the corresponding configuration file. For example, in fig. 11, for Detect that needs to perform a graph entry flow, it is determined how to handle the Detect according to a graph structure classification of the Detect; analyzing the corresponding configuration file to obtain a correlation point determination rule; for different graph structure classifications, calculating the correlation points of Detect according to the correlation point determination rule; searching for a subconcent corresponding to the SUBICIDENT _ ROOT from the current graph database, if the searching is successful, associating the Detect with the existing subconcent (namely, marking the association relationship between the Detect tree corresponding to the Detect and the subconcent searched from the current graph database by using the SUBICIDENT _ DETECT), if the searching is failed, establishing a subconcent corresponding relationship, and pointing to the current association point by using the SUBICIDENT _ ROOT (namely, marking the association relationship between the association point and the subconcent by using the SUBICIDENT _ ROOT).

Illustratively, the association point determination rule of Detect may be predefined to include the following three rules in turn:

and (I) carrying out bottom holding rules. For example, a node number threshold and a plurality of different time intervals are preset; for a certain SubIncident, if the storage time of each Detect tree in the SubIncident is within a time interval and the number of nodes in the SubIncident does not exceed the node number threshold, not splitting the SubIncident; if the saving time of each Detect tree in the subordinate is already outside any one time interval and/or the number of nodes in the subordinate exceeds the node number threshold, the subordinate can be divided into a plurality of new subordinate, namely: and associating the Detect trees stored in the SubIncident at the moment in the same time interval into a new SubIncident, and/or associating the Detect trees of which the sum of the number of nodes does not exceed the threshold of the number of nodes in the SubIncident into a new SubIncident. The bottom-pocketing rule can avoid the condition that the number of nodes in a single sub-agent is too large, and avoid the occurrence of too long association and too many node associations, so that the presentation effect of the association result is improved.

And (II) blacklist rules. Presetting a common attack path; and sequentially matching each node in the current Detect tree with a preset attack path, and if the matching is successful, taking the last node (namely the endmost node) which can be matched in the current Detect tree as an association point. For example, the blacklist rule includes rule 1 (i.e., a complete matching rule) and rule 2 (i.e., a regular matching rule), the Detect tree is a process chain "systemd" - > "sshd" - > "sshd" - > "bash" - > "my-service" - > "nginx" - > "php-fpm", and the process chain is matched with a preset attack path by using the rule 1 and the rule 2; the operation of rule 1 is: traversing the process chain, comparing complete process names, wherein the matched processes are 'systemd', 'sshd', 'bash' and 'my-service', and the last matched process is my-service, so that the association point calculated by the rule 1 is my-service; the operation of rule 2 is: traversing the process chain, and comparing the process names according to a normal rule sequence, wherein the regular expressions of the process names are 'systemd', '(sshd)' 2 ',' (bash) 'n', '/' service. + 'and' + ', respectively, wherein,' sshd) '2' indicates that 2 continuous processes with the names of sshd are matched on a path, '/regular expression/' indicates that any process meeting the regular expression is matched, '+' indicates that any process is matched, and the last matching process is nginx, so that the association point calculated by the rule 2 is nginx; comparing the matching result of rule 1 with the matching result of rule 2, the matching path corresponding to rule 2 is longer, so the finally calculated association point is nginx.

And (III) white list rules. And when the blacklist rule does not calculate a correlation point, namely when no node in the current Detect tree is matched with a preset attack path, traversing the Detect tree along the sequence from the root node to the child node, sequentially judging whether the node in the Detect tree is in a preset white list according to the traversing sequence, and then taking the node which is traversed for the first time from the Detect tree and is not in the white list as the correlation point. For example, fig. 12 shows a code implementation of a preset white list, that is, a code implementation of a white list process is to construct a hashmap; for a process chain (the Detect tree at this time), the process chain is traversed from a parent process to a child process according to a white list rule, and the first traversed non-white list process is set as an association point. White listing rules can be typically used to filter common system processes (root processes), thereby improving the efficiency of association point determination.

The existing Detect association scheme is generally to merge directly according to a process tree, but the complexity is high, for the association of n Detect trees with the number of nodes being m, each newly entered Detect tree needs to be compared with the existing Detect tree, the complexity of the association is O (m ^ n), and the association efficiency is low.

The association point is introduced here as a node on the Detect tree representing the start of the attack. With the help of a graph database, the association algorithm of the process dimension association in the embodiment of the invention is as follows: when each Detect tree is added into the graph (namely, added into the attack graph), calculating a corresponding association point (for example, a bash process node marked by a hollow short arrow in fig. 13) according to an association point determination rule; each sub-event only has a unique association point, and the association point is marked by using SUBINCIDENT _ ROOT (namely association point relation) in the attack graph; for the calculated association points, searching corresponding marked sub-events in the attack graph according to the association point relation; if the search is successful, the search is automatically associated with the previous subevents, namely, a SUBINCIDENT _ DETECT relationship is created (namely, a relationship that the subevents point to an alarm is created), if the search is failed, the subevents are created, and association points of the current subevents are marked by using SUBINCIDENT _ ROOT (namely, an association point relationship), so that the search association of a subsequent Detect tree in a graph is facilitated. The core of the algorithm is as follows: different Detect trees with the same association point are represented as the same graph node in the graph database, the alarm association process is converted into a process of traversing the Detect trees to calculate the corresponding association point according to the association point determination rule, and the alarm association cost is the cost for calculating the association point.

Referring to fig. 13, there are sub-events in the attack graph, the sub-event _ DETECT (i.e., the sub-event points to the alarm) is used to mark two alarms, namely alarm 1 and alarm 2, and the sub-event _ ROOT (i.e., the association point relationship) is used to mark the current association point (i.e., the bash process); at this time, for the newly reported alarm 4, firstly traversing the process chain to calculate the corresponding association point, calculating that the association point is also a bash process, then using the bash process to query the attack graph, finding that the sub-event in the attack graph already points to the bash process through the SUBINDENT _ ROOT relation (namely the association point relation), and marking the relation between the sub-event and the alarm 4 by using SUBINDENT _ DETECT (namely the sub-event points to the alarm), namely establishing the association between the sub-event and the alarm 4. For the association of n Detect trees with the number of nodes being m, the newly entered Detect tree each time only needs to calculate the own association point and compare the calculated association point with the existing association point in the attack graph, the association cost is only to calculate the association point according to the association point determination rule, the complexity of each association is O (m), and the calculation amount is greatly reduced. Compared with the mode that each newly entered Detect tree in the existing Detect association scheme needs to be compared with the previous n-1 Detect trees one by one, the newly entered Detect tree in the current association scheme only needs to calculate the corresponding association point and search the association point in the attack graph, and the whole Detect tree before comparison does not need to be traversed.

Illustratively, after the process dimension association is performed on the alarms, the time dimension association is also required. Reporting Detects in a certain time range are aggregated on the basis of process dimension association through time dimension association, and logic association according to a Detect tree is preferred to ensure association accuracy.

Referring to fig. 14, firstly, according to the process dimension rule, the alarm 1 is aggregated (i.e. associated) into the sub-event 1, the alarm 2 is aggregated (i.e. associated) into the sub-event 2, and the alarm 3 is aggregated (i.e. associated) into the sub-event 3; and then aggregating the sub-event 1, the sub-event 2 and the sub-event 3 into the same event according to the storage starting time of each sub-event (namely the storage time of the first entered Detect tree) and the time dimension rule.

By adopting the time dimension association mode, the alarms which are merged into the previous sub-events can be ensured not to be merged into the sub-events in the current time range. The time dimension aggregation can control the time span size (namely, the time range size, which supports the precision to millisecond, such as 2 days) by using the configuration parameters, and prevent the sub-event which is possibly too large due to too large time span, thereby reducing the data transmission cost and improving the front-end display efficiency.

Based on the above alarm log association method, an embodiment of the present invention further provides an alarm log association apparatus, as shown in fig. 15, where the apparatus may include the following modules:

an entity element obtaining module 1502 is configured to obtain an entity element corresponding to each attack alarm log.

An attack graph generating module 1504, configured to combine entity elements corresponding to each of the attack warning logs into a corresponding first process tree based on an access relationship between the entity elements; and the nodes in the first process tree represent entity elements, and the connecting lines among the nodes in the first process tree represent access relations among the entity elements.

The association module 1506 is configured to associate all the acquired first process trees into one or more attack events according to a preset association rule; the preset association rules comprise process dimension rules and time dimension rules.

The alarm log association device provided by the embodiment of the invention firstly obtains entity elements corresponding to each attack alarm log; then based on the access relation among the entity elements, forming the entity elements corresponding to each attack alarm log into a corresponding first process tree; and then associating all the acquired first process trees into one or more attack events according to preset association rules (including process dimension rules and time dimension rules). By adopting the technology, the real-time association of various alarm logs can be efficiently completed, and the method is accurate, flexible and good in expansibility.

Referring to fig. 15, the apparatus may further include:

and a map entering module 1508, configured to store, as an attack map, the entity elements corresponding to each obtained attack alarm log and the relationship between the entity elements based on a preset attack map storage model.

Based on this, the association module 1506 may further be configured to: and for each newly obtained first process tree, associating the first process tree with the existing first process tree in the attack graph according to a preset association rule, and updating the attack graph by using the association result of the first process tree.

The association module 1506 may also be configured to: determining a correlation point in each first process tree according to a process dimension rule, and correlating the first process trees with the same correlation point into the same sub-event to obtain one or more sub-events; wherein, the association point is a node which represents the beginning of the attack in the first process tree; and aggregating the one or more sub-events into one or more attack events according to the time dimension rule.

The association module 1506 may also be configured to: for each first process tree, matching the first process tree with one or more preset attack paths; if the matching is successful, taking the end node of the longest matching preset attack path as the association point in the first process tree; and if the matching fails, traversing the first process tree along the sequence from the root node to the child node based on a preset node white list, and taking the node which is traversed for the first time in the first process tree and is not in the preset node white list as an associated point in the first process tree.

The association module 1506 may also be configured to: if the association point in the first process tree is not in the attack graph, taking the association point as a new association point, and associating the first process tree in a new sub-event in the attack graph; if the first process tree meets the preset condition, the first process tree is associated in a new sub-event in the attack graph; the preset conditions comprise the number of nodes in the sub-event and the time information of the sub-event.

The association module 1506 may also be configured to: and aggregating the time information and the sub-events corresponding to the same time range into the same attack event based on a plurality of preset different time ranges and the time information of each sub-event, so as to obtain one or more attack events.

Referring to fig. 15, the apparatus may further include:

a determining module 1510, configured to determine a type of each access relationship based on a plurality of different preset virtual table structures.

The first identification module 1512 is configured to, based on the determined type of each access relationship, respectively assign a corresponding type identification to each access relationship.

The first identification module 1512 may be further configured to: for each network access relation, distributing a corresponding quadruplet for the network access relation as a type identifier of the network access relation; wherein the quad includes a source ip address, a source port, a destination port, and a destination ip address.

Referring to fig. 15, the apparatus may further include:

a second identifying module 1514, configured to, for each first process tree associated with the same sub-event, characterize an association relationship between the first process tree and the sub-event by using a sub-event identifier of the sub-event.

And a third identifying module 1516, configured to, for each sub-event aggregated into the same attack event, characterize the aggregation relationship between the sub-event and the event by using the event identifier of the event.

The attack alarm log can be a file scanning alarm log, and at the moment, entity elements corresponding to the attack alarm log do not comprise processes; based on this, the association module 1506 may further be configured to: and associating the attack alarm logs with the same target entity elements into the same attack event based on the scanning relation among the target entity elements.

The implementation principle and the generated technical effect of the alarm log association device provided by the embodiment of the invention are the same as those of the alarm log association method embodiment, and for brief description, corresponding contents in the alarm log association method embodiment can be referred to where the alarm log association device embodiment is not mentioned.

Unless specifically stated otherwise, the relative steps, numerical expressions and values of the components and steps set forth in these embodiments do not limit the scope of the present invention.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.

Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An alarm log association method, the method comprising:

acquiring entity elements corresponding to each attack alarm log;

based on the access relation among the entity elements, forming the entity elements corresponding to each attack alarm log into a corresponding first process tree; wherein, the nodes in the first process tree represent entity elements, and the connecting lines between the nodes in the first process tree represent access relations between the entity elements;

according to a preset association rule, associating all the acquired first process trees into one or more attack events; the preset association rules comprise a process dimension rule and a time dimension rule;

after the step of obtaining the entity element corresponding to each attack alarm log, the method further comprises:

based on a preset attack graph storage model, storing the entity elements corresponding to each obtained attack alarm log and the relationship between the entity elements as an attack graph;

the method comprises the following steps of associating all acquired first process trees into one or more attack events according to a preset association rule, and comprises the following steps:

for each newly obtained first process tree, associating the first process tree with the existing first process tree in the attack graph according to a preset association rule, and updating the attack graph by using the association result of the first process tree;

according to a preset association rule, associating all the acquired first process trees into one or more attack events, further comprising:

determining a correlation point in each first process tree according to a process dimension rule, and correlating the first process trees with the same correlation point into the same sub-event to obtain one or more sub-events; wherein, the association point is a node for representing the start of attack in the first process tree;

and aggregating the one or more sub-events into one or more attack events according to the time dimension rule.

2. The alarm log association method of claim 1, wherein the step of determining the association point in each first process tree according to the process dimension rule comprises:

for each first process tree, matching the first process tree with one or more preset attack paths; if the matching is successful, taking the end node of the longest matching preset attack path as the association point in the first process tree; and if the matching fails, traversing the first process tree along the sequence from the root node to the child node based on a preset node white list, and taking the node which is traversed for the first time in the first process tree and is not in the preset node white list as an associated point in the first process tree.

3. The alarm log association method of claim 2, wherein the method further comprises:

if the association point in the first process tree is not in the attack graph, taking the association point as a new association point, and associating the first process tree in a new sub-event in the attack graph;

if the first process tree meets the preset condition, the first process tree is associated in a new sub-event in the attack graph; the preset conditions comprise the number of nodes in the sub-event and the time information of the sub-event.

4. The alarm log association method of claim 1, wherein the step of aggregating the one or more sub-events into one or more attack events according to a time dimension rule comprises:

and aggregating the time information and the sub-events corresponding to the same time range into the same attack event based on a plurality of preset different time ranges and the time information of each sub-event, so as to obtain one or more attack events.

5. The alarm log association method according to claim 1, wherein after the step of obtaining the entity element corresponding to each attack alarm log, the method further comprises:

respectively determining the type of each access relation based on a plurality of preset different virtual table structures;

and respectively correspondingly allocating corresponding type identifications for each access relation based on the determined type of each access relation.

6. The alarm log association method according to claim 1, wherein for each network access relationship, a corresponding quadruple is allocated to the network access relationship as a type identifier of the network access relationship; wherein the quad includes a source ip address, a source port, a destination port, and a destination ip address.

7. The alarm log association method according to claim 1, wherein the entity elements corresponding to the attack alarm log do not include a process; the method further comprises the following steps:

and associating the attack alarm logs with the same target entity elements into the same attack event based on the scanning relation among the target entity elements.

8. An alarm log association apparatus, the apparatus comprising:

the entity element acquisition module is used for acquiring entity elements corresponding to each attack alarm log;

the attack graph generation module is used for forming entity elements corresponding to each attack alarm log into a corresponding first process tree based on the access relation among the entity elements; wherein, the nodes in the first process tree represent entity elements, and the connecting lines between the nodes in the first process tree represent access relations between the entity elements;

the association module is used for associating all the acquired first process trees into one or more attack events according to a preset association rule; the preset association rules comprise process dimension rules and time dimension rules;

the device further comprises:

the map entry module is used for storing the obtained entity elements corresponding to each attack alarm log and the relationship between the entity elements as an attack map based on a preset attack map storage model;

the association module is further configured to: for each newly obtained first process tree, associating the first process tree with the existing first process tree in the attack graph according to a preset association rule, and updating the attack graph by using an association result of the first process tree;

the association module is further configured to: determining a correlation point in each first process tree according to a process dimension rule, and correlating the first process trees with the same correlation point into the same sub-event to obtain one or more sub-events; wherein, the association point is a node which represents the beginning of the attack in the first process tree; and aggregating the one or more sub-events into one or more attack events according to the time dimension rule.