CN116015823A - Event detection method and device, electronic equipment and storage medium - Google Patents

Event detection method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN116015823A
CN116015823A CN202211634738.4A CN202211634738A CN116015823A CN 116015823 A CN116015823 A CN 116015823A CN 202211634738 A CN202211634738 A CN 202211634738A CN 116015823 A CN116015823 A CN 116015823A
Authority
CN
China
Prior art keywords
flow
information
index
collision
backtracking
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211634738.4A
Other languages
Chinese (zh)
Inventor
郭兰杰
丁杰
赵粤征
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nsfocus Technologies Inc
Nsfocus Technologies Group Co Ltd
Original Assignee
Nsfocus Technologies Inc
Nsfocus Technologies Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nsfocus Technologies Inc, Nsfocus Technologies Group Co Ltd filed Critical Nsfocus Technologies Inc
Priority to CN202211634738.4A priority Critical patent/CN116015823A/en
Publication of CN116015823A publication Critical patent/CN116015823A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Debugging And Monitoring (AREA)

Abstract

The embodiment of the application provides an event detection method, an event detection device, electronic equipment and a storage medium, which are used for responding to set collision time, acquiring an information index set, and acquiring the flow index set aiming at the collision time from a flow index library, wherein each information index is related to one increment information in a local information library, the information recording time of each increment information is not earlier than the set historical detection time, each flow index is related to one backtracking flow in a flow message queue, each backtracking flow is obtained through historical log analysis, and further, collision and analysis are carried out on the information index set and the flow index set to obtain each backtracking threat event, so that backtracking detection is carried out on the attack behavior before information generation based on the mode, and the integrity of a network attack chain of the backtracking is improved.

Description

Event detection method and device, electronic equipment and storage medium
Technical Field
The present invention relates to the field of information security technologies, and in particular, to an event detection method, an event detection device, an electronic device, and a storage medium.
Background
The information collision is a detection method for searching and comparing threat indexes in the discovered attacker information with threat indexes in the flow logs of the target network, and is used for detecting threat events in the target network and ensuring early warning.
In the prior art, threat information generated in real time is generally adopted to collide with the current flow, and related alarms are carried out after threat events are detected, and due to hysteresis generated by the information, the mode is easy to miss the attack behavior of an attacker before the information is generated, so that a network attack chain traced out is incomplete, and the safety of a network space is influenced.
Disclosure of Invention
The embodiment of the application provides an event detection method, an event detection device, electronic equipment and a storage medium, which are used for improving the integrity of a network attack chain.
In a first aspect, an embodiment of the present application provides an event detection method, including:
and responding to the set collision time, acquiring an information index set, wherein each information index is associated with one increment information in the local information base, and the information recording time of each increment information is not earlier than the set historical detection time.
And acquiring a flow index set aiming at the collision time from a flow index library, wherein each flow index is associated with one retrospective flow in a flow message queue, and each retrospective flow is acquired through historical log analysis.
And carrying out collision and analysis on the information index set and the flow index set to obtain each backtracking threat event.
And sending each retrospective threat event to an event message queue, and triggering analysis and alarm for each retrospective threat event.
In an optional embodiment, before the acquiring the information index set in response to the set collision opportunity, the method further includes:
and acquiring each backtracking flow aiming at the collision time in the flow message queue according to the started consumption process.
And extracting indexes of the backtracking flows to obtain candidate indexes, and filtering the candidate indexes by adopting a filtering rule to obtain flow indexes.
And sending each flow index to the flow index library, triggering the flow index library, merging and storing each flow index according to a set period, and merging and storing the local flow library by adopting persistence.
In an optional implementation manner, the triggering the flow index library, according to a set period, performs merging storage on the flow indexes, and includes:
triggering the flow index library, and merging and storing each first flow value in each flow index according to a set period, wherein the first flow value comprises: address information and hash values extracted by corresponding backtracking flow; the method comprises the steps of,
Triggering the flow index library, compressing each second flow value in each flow index by adopting a compression algorithm, and merging and storing each obtained second compression value according to the set period, wherein the second flow value comprises: and correspondingly backtracking the uniform resource locator extracted by the flow.
In an alternative embodiment, the acquiring the information index set in response to the set collision opportunity includes:
and responding to the local information library triggering update information, and acquiring an information index set associated with a set index type and/or an information type.
Or alternatively, the process may be performed,
and responding to the set detection period, and acquiring an information index set associated with the set index type and/or the information type.
In an optional embodiment, the acquiring, from a flow index library, a set of flow indexes for the collision opportunity includes:
and acquiring flow record time of corresponding backtracking flow from a flow index library, wherein the flow record time belongs to a plurality of flow indexes in a set backtracking period and is a flow index set aiming at the collision time.
And/or the number of the groups of groups,
and acquiring the log type of the corresponding backtracking flow correlation history log from the flow index library, wherein the log type is a set of a plurality of flow indexes of the backtracking type and is a set of flow indexes aiming at the collision time.
In an alternative embodiment, the flow index set is obtained in batches according to set backtracking data amounts,
and performing collision and analysis on the information index set and the flow index set to obtain each backtracking threat event, wherein the collision and analysis comprises the following steps:
for a plurality of batches for which the flow index set is acquired, the following operations are respectively executed:
aiming at a flow index subset of corresponding backtracking data volume, which is acquired from one batch, collision is carried out on the information index set and the flow index subset to obtain subset collision data;
and analyzing the subset collision data by adopting an event model to obtain each retrospective threat event corresponding to the information index set in the flow index subset.
In an alternative embodiment, the information index set adopts a plurality of threads to collide with a plurality of sub-set thread blocks of the flow index sub-set simultaneously,
and analyzing the index collision data by adopting an event model to obtain each backtracking threat event in the flow index subset, wherein the method comprises the following steps:
and in the multiple threads, concurrently adopting the event model, analyzing multiple block collision data obtained by block collision of the information index set and the multiple subset threads, and obtaining each retrospective threat event in the flow index subset.
In a second aspect, an embodiment of the present application provides an event detection apparatus, including:
the first acquisition module is used for responding to the set collision time and acquiring an information index set, wherein each information index is associated with one increment information in the local information base, and the information recording time of each increment information is not earlier than the set historical detection time.
The second obtaining module is used for obtaining the flow index set aiming at the collision time from the flow index library, wherein each flow index is associated with one retrospective flow in the flow message queue, and each retrospective flow is obtained through historical log analysis.
And the event generation module is used for carrying out collision and analysis on the information index set and the flow index set to obtain each backtracking threat event.
And the transmission module is used for sending each retrospective threat event to an event message queue and triggering analysis and alarm for each retrospective threat event.
In an optional embodiment, before the acquiring the information index set in response to the set collision opportunity, the first acquiring module is further configured to:
and acquiring each backtracking flow aiming at the collision time in the flow message queue according to the started consumption process.
And extracting indexes of the backtracking flows to obtain candidate indexes, and filtering the candidate indexes by adopting a filtering rule to obtain flow indexes.
And sending each flow index to the flow index library, triggering the flow index library, merging and storing each flow index according to a set period, and merging and storing the local flow library by adopting persistence.
In an optional implementation manner, the triggering the flow index library is used for merging and storing the flow indexes according to a set period, and the first acquisition module is specifically used for:
triggering the flow index library, and merging and storing each first flow value in each flow index according to a set period, wherein the first flow value comprises: address information and hash values extracted by corresponding backtracking flow; the method comprises the steps of,
triggering the flow index library, compressing each second flow value in each flow index by adopting a compression algorithm, and merging and storing each obtained second compression value according to the set period, wherein the second flow value comprises: and correspondingly backtracking the uniform resource locator extracted by the flow.
In an optional implementation manner, the acquiring an information index set in response to the set collision opportunity, and the first acquiring module is specifically configured to:
and responding to the local information library triggering update information, and acquiring an information index set associated with a set index type and/or an information type.
Or alternatively, the process may be performed,
and responding to the set detection period, and acquiring an information index set associated with the set index type and/or the information type.
In an optional implementation manner, the acquiring, from a flow index library, a flow index set for the collision opportunity, and the second acquiring module is specifically configured to:
and acquiring flow record time of corresponding backtracking flow from a flow index library, wherein the flow record time belongs to a plurality of flow indexes in a set backtracking period and is a flow index set aiming at the collision time.
And/or the number of the groups of groups,
and acquiring the log type of the corresponding backtracking flow correlation history log from the flow index library, wherein the log type is a set of a plurality of flow indexes of the backtracking type and is a set of flow indexes aiming at the collision time.
In an alternative embodiment, the flow index set is obtained in batches according to set backtracking data amounts,
The information index set collides with and analyzes the flow index set to obtain each backtracking threat event, and the event generating module is specifically configured to:
for a plurality of batches for which the flow index set is acquired, the following operations are respectively executed:
aiming at a flow index subset of corresponding backtracking data volume, which is acquired from one batch, collision is carried out on the information index set and the flow index subset to obtain subset collision data;
and analyzing the subset collision data by adopting an event model to obtain each retrospective threat event corresponding to the information index set in the flow index subset.
In an alternative embodiment, the information index set adopts a plurality of threads to collide with a plurality of sub-set thread blocks of the flow index sub-set simultaneously,
and analyzing the index collision data by adopting an event model to obtain each retrospective threat event in the flow index subset, wherein the event generation module is specifically configured to:
and in the multiple threads, concurrently adopting the event model, analyzing multiple block collision data obtained by block collision of the information index set and the multiple subset threads, and obtaining each retrospective threat event in the flow index subset.
In a third aspect, an electronic device is provided, comprising a processor and a memory, wherein the memory stores program code that, when executed by the processor, causes the processor to perform the steps of the event detection method described in the first aspect.
In a fourth aspect, a computer readable storage medium is proposed, comprising program code for causing an electronic device to perform the steps of the event detection method as described in the first aspect above, when said program code is run on said electronic device.
The technical effects of the embodiment of the application are as follows:
the embodiment of the application provides an event detection method, an event detection device, electronic equipment and a storage medium, which are used for responding to set collision time, acquiring an information index set, and acquiring the flow index set aiming at the collision time from a flow index library, wherein each information index is related to one increment information in a local information library, the information recording time of each increment information is not earlier than the set historical detection time, each flow index is related to one backtracking flow in a flow message queue, each backtracking flow is obtained through historical log analysis, and further, collision and analysis are carried out on the information index set and the flow index set to obtain each backtracking threat event, so that backtracking detection is carried out on the attack behavior before the information generation based on the mode, and the traceability integrity of a network attack chain is improved.
On the other hand, the historical detection time is adopted to configure each increment information for collision, and each backtracking flow for collision is obtained through a flow message queue, so that repeated collision of data is avoided, the total data quantity of buffering needed during collision is reduced, and the detection efficiency for threat events is improved.
Drawings
Fig. 1 is a schematic diagram of a possible application scenario provided in an embodiment of the present application;
FIG. 2 is a schematic diagram of a possible platform according to an embodiment of the present application;
fig. 3 is a flowchart of an event detection method according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an event detection device according to an embodiment of the present application;
fig. 5 is a schematic diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present invention based on the embodiments herein.
It should be noted that "a plurality of" is understood as "at least two" in the description of the present application. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. A is connected with B, and can be represented as follows: both cases of direct connection of A and B and connection of A and B through C. In addition, in the description of the present application, the words "first," "second," and the like are used merely for distinguishing between the descriptions and not be construed as indicating or implying a relative importance or order.
In addition, in the technical scheme, the data are collected, transmitted, used and the like, and all meet the requirements of national related laws and regulations.
First, for convenience of understanding, the following explanation is made on partial terms and terms related to the embodiments of the present application:
intelligence: evidence-based knowledge, including context, mechanisms, logos, meaning, and actionable advice, provides information support for response or processing decisions of asset-related principals to threats or hazards, and in particular, to the present application, known attack data published in the internet may be used as incremental intelligence required for event detection.
Threat metrics (Indicator of Compromise, IOC for short), refer to various attack metrics for intrusion detection, including: a network co-address IP, a uniform resource locator URL, a DOMAIN name DOMAIN, a HASH value HASH, etc.
Information collision: the method comprises the steps of finding a black IP/domain name/URL/HASH value in threat information, and collecting the same IP/domain name/URL/HASH value in a flow log, so as to find out network attacks and trace back attack events, and particularly in the application, matching the increment information IOC with the trace back flow IOC so as to detect the attack events occurring before the current information is generated.
The design thought of the application is as follows:
in the prior art, threat information generated in real time is generally adopted to collide with the current flow, and related alarms are carried out after threat events are detected, and due to hysteresis generated by the information, the mode is easy to miss the attack behavior of an attacker before the information is generated, so that a network attack chain traced out is incomplete, and the safety of a network space is influenced.
In order to improve the integrity of a network attack chain, the embodiment of the application provides an event detection method, an event detection device, an electronic device and a storage medium, which are used for responding to a set collision time to obtain an information index set, and obtaining the flow index set aiming at the collision time from a flow index library, wherein each information index is associated with one increment information in a local information library, the information recording time of each increment information is not earlier than the set historical detection time, each flow index is associated with one backtracking flow in a flow message queue, each backtracking flow is obtained through historical log analysis, and further, collision and analysis are carried out on the information index set and the flow index set to obtain each backtracking threat event, so that the backtracking detection is carried out on the attack behavior before the information generation based on the mode, and the integrity of the network attack chain obtained by backtracking is improved.
On the other hand, the historical detection time is adopted to configure each increment information for collision, and each backtracking flow for collision is obtained through a flow message queue, so that repeated collision of data is avoided, the total data quantity of buffering needed during collision is reduced, and the detection efficiency for threat events is improved.
Referring to fig. 1, a schematic diagram of a possible application scenario provided in this embodiment of the present application is provided, where the application scenario includes a platform 1, a cloud information source 2, and a full-flow probe 3, where the platform 1 may perform information interaction with the cloud information source 2 and the full-flow probe 3 through a communication connection manner, where the communication connection manner includes a wired connection manner (such as a hard wire) and/or a wireless connection manner (such as Wi-Fi).
Referring to fig. 2, in a specific embodiment, the intelligence component 11 in the platform 1 periodically obtains threat intelligence generated by the cloud intelligence source 2, where the threat intelligence includes, but is not limited to, advanced persistent threat (Advanced Persistent Threat, APT for short), mining, lux, remote control (C2 for short), and the like, and stores the threat intelligence in a local intelligence library of the platform 1 through intelligence analysis.
Further, the data access component 12 in the platform 1 acquires traffic data from the target network collected by the full-traffic probe 3 in real time, where the traffic data includes, but is not limited to, session logs, web access logs, domain name resolution logs, and the like. Optionally, the data access component 12 normalizes the obtained traffic data, writes it into a traffic message queue and stores it in a local traffic library.
Based on the above embodiment, the platform 1 further includes an information offline backtracking engine 13 for executing the method flow provided in the present application, where the information offline backtracking engine 13 includes a backtracking configuration module 131, an information loading and buffering module 132, a threat index collection module 133 and an information collision module 134, and the respective functions of the modules are as follows.
The backtracking configuration module 131 is configured to configure the collision opportunity, the information range, and the backtracking range.
In a specific embodiment, the collision opportunity is used to indicate the execution time of event detection, and includes: triggering collisions [ automatically triggering collisions in response to updated information ], periodic collisions [ fixed periodic execution collisions ], and the like.
The information range is used for indicating information indexes which are needed to participate in event detection, and can be configured according to index types [ such as IP, DOMAIN, URL, HASH ] or information types [ such as mining, halyard, APT, C2, black IP, and the like ].
The backtracking range is used for indicating traffic indexes of the event detection, and can be configured according to backtracking time periods [ e.g., before one day, before three days to before one day, near three days, etc. ] and log types [ session logs, web access logs, domain name resolution logs, malicious sample logs ], etc.
The information loading and buffering module 132 is configured to obtain incremental information according to the configured information range.
In a specific embodiment, the incremental information refers to newly added information data with information recording time not earlier than the set historical detection time, and the incremental information is adopted to detect events when each collision time occurs, so that repeated collision of the data is avoided, and data redundancy is reduced.
Alternatively, the historical detection time may be characterized as a time of collision record of the last event detection that is closest to the current time, or may be any specified time set after the last time of collision detection, which is not limited.
The threat index collection module 133 is configured to obtain the traceback traffic and the traffic index according to the configured traceback range.
In a specific embodiment, the threat index collection module 133 may consume the trace-back traffic in the traffic message queue connected to the local traffic pool according to the configured trace-back range, extract threat indexes in the trace-back traffic, and send the threat indexes to the traffic index pool. Optionally, the threat indicator collection module 133 filters each candidate indicator extracted by the traceback traffic in combination with a set filtering rule, where the filtering rule includes, but is not limited to: and filtering the intranet IOC and the white list IOC in each candidate index.
It should be noted that, in the above embodiment, the traffic message queue performs, according to a set filtering rule, pre-filtering on invalid indexes in each candidate index, so as to reduce the data volume of a traffic index set to be collided subsequently, and avoid additional resource consumption; the traffic index library may be a cache database [ e.g., remote dictionary service dis, etc. ], and according to a set period, each traffic index transmitted by the threat index collection module 133 is integrated and stored, and persistence is adopted to store in the local traffic library, so as to reduce the read-write times of the database and improve the query performance for the traffic IOC data.
The intelligence collision module 134 is configured to trigger event detection according to the configured collision opportunity, and obtain a backtracking threat event between the incremental intelligence and the history log.
In a specific embodiment, the information collision module 134 may obtain the flow index set in the flow index library in batches according to the set backtracking data amounts, and in one batch of obtaining the flow index set, use multiple threads to concurrently collide with the corresponding multiple sub-set threads in blocks, so as to improve the collision speed of the information index against the flow index, and ensure the event detection performance.
Based on the above application scenario, the event detection method provided in the present application will be further described and illustrated with reference to the accompanying drawings, and referring to fig. 3, the event detection method includes:
s301: and responding to the set collision time, and acquiring an information index set.
Specifically, in the event detection method provided by the application, an information index set extracted from incremental information in a local information library is adopted to collide with and analyze a flow index set extracted from retrospective flow in a flow message queue, the retrospective flow is obtained through historical log analysis, so that a preamble attack behavior related to the information index set is detected before collision time, and a network attack chain is assisted to complete retrospection; furthermore, the information data with the information recording time not earlier than the set historical detection time is used as incremental information, so that the data redundancy caused by repeated collision is avoided, and the detection efficiency for threat events is ensured.
In an alternative embodiment, before the collection of informative indicators is obtained, the following steps are further adopted for processing, including:
step 1: and acquiring each backtracking flow aiming at the collision time in the flow message queue according to the started consumption process.
Step 2: and extracting indexes of each backtracking flow to obtain each candidate index, and filtering each candidate index by adopting a filtering rule to obtain each flow index.
Step 3: and sending each flow index to a flow index library, triggering the flow index library, merging and storing each flow index according to a set period, and merging and storing the local flow library by adopting persistence.
Specifically, in the above embodiment, the consumption process is adopted to consume the trace-back traffic in the traffic message queue in real time, and extract candidate indexes of each trace-back traffic, and further, the set filtering rule is adopted to filter each candidate index, so as to reduce the resource consumption.
For example, for each backtracking flow obtained by consumption, the IOC value extracted from each backtracking flow is used as a candidate index, and the internal network IOC and/or the white list IOC are/is used for filtering each candidate index, so that the resource consumption caused by subsequent collision analysis is reduced.
Furthermore, for each obtained flow index, the flow index is periodically merged and stored in a flow index library, and the local flow library is subjected to persistence, so that the read-write times of the database are reduced, and the resource consumption is reduced.
In an alternative embodiment, the first flow values in each flow index are integrated and stored according to a set period, a compression algorithm is adopted to compress each second flow value in each flow index, and then the obtained second compression values are integrated and stored according to the set period, wherein the first flow values comprise address information and hash values extracted by corresponding retrospective flow, the second flow values comprise uniform resource locators extracted by corresponding retrospective flow, so that index data with a length which is larger than that of the index data are compressed and stored, and the storage efficiency of the flow index is ensured.
Illustratively, in a specific embodiment, address information and hash values [ e.g., IP, DOMAIN, HASH, etc. ] contained in each flow index after filtering are stored in a merging manner by day, and a compression algorithm is adopted to compress the contained uniform resource locator URL, store corresponding second compression values in a merging manner by day, and perform persistence by day to a local flow library, wherein the compression algorithm includes, but is not limited to, an information summarization algorithm MD5, and the local flow library includes, but is not limited to, a columnar database or an ES database.
In an alternative embodiment, the set of intelligence indicators is obtained in response to a set collision opportunity, including any one of the following:
1) And responding to the local information library triggering update information, and acquiring the set index type and/or the information index set associated with the information type.
2) And responding to the set detection period, and acquiring an information index set associated with the set index type and/or the information type.
Specifically, when the local information library has increment information, acquiring a corresponding information index set, or acquiring the information index set associated with the local information library at the triggering time of the detection period according to the set detection period; the informative index set can be associated with a set index type and/or an informative type; the index types include, but are not limited to IP, DOMAIN, URL, HASH, etc.; the intelligence types include, but are not limited to, mine dig, lux, APT, C2, black IP, etc.
Illustratively, at every other day, the IP set extracted from the enhanced intelligence is obtained.
S302: and acquiring a flow index set aiming at the collision time from the flow index library.
In an alternative embodiment, the flow index set for the collision opportunity is obtained from a flow index library, including any one of the following ways:
1) And acquiring flow record time of corresponding backtracking flow from a flow index library, wherein the flow record time belongs to a plurality of flow indexes in a set backtracking period and is a flow index set aiming at collision time.
2) And acquiring the log type of the corresponding backtracking flow correlation history log from the flow index library, wherein the log type is a set of a plurality of flow indexes of the backtracking type and is a set of flow indexes aiming at collision time.
Specifically, a corresponding flow index set is obtained by setting a backtracking period, or a corresponding flow index set is obtained by setting a backtracking type; the backtracking types include, but are not limited to, session logs, web access logs, domain name resolution logs, malicious sample logs, and the like.
Illustratively, a set of traffic indicators extracted from a session log of the last three days is obtained from a traffic indicator library.
S303: and (3) collision and analysis are carried out on the information index set and the flow index set, so that each backtracking threat event is obtained.
In an alternative embodiment, the flow index set is obtained in batches according to the set backtracking data amounts, and the following steps are respectively executed for a plurality of batches for obtaining the flow index set:
step 1: and aiming at the flow index subset of the corresponding backtracking data quantity, which is acquired by one batch, the information index set collides with the flow index subset to acquire subset collision data.
Step 2: and analyzing the sub-collision data by adopting an event model to obtain each retrospective threat event in the flow index subset, which corresponds to the information index set.
Specifically, the flow index set in the flow index library is obtained in batches, so that the query performance of the database is brought into full play, the event generation efficiency is improved, and optionally, the corresponding backtracking data volume of each batch is manually specified, or the batch number and the single backtracking data volume required by collision are set according to the size of the information index set.
For example, 100 or 1000 flow indexes are adopted as the corresponding flow index subsets, collision is performed with the obtained information index sets respectively, the same data result is detected, and an event model is adopted to generate corresponding backtracking threat events, wherein the event model can be shown in the following table 1:
TABLE 1
Event name "backtracking _ { IOC value } hits { information type } information }" information "
Event type Mine digging type, APT type, lesu type, etc
Event ranking Corresponds to the threat level of information
The IOC values in the event names are characterized as the same IOC values in the information collision, and the event names contain backtracking identifiers for event discrimination, and the exemplary event name of a backtracking threat event is: "backtracking_2.2.2.2 hits mine excavation intelligence".
In an alternative implementation mode, a plurality of threads are adopted, and the information index set and a plurality of subset threads of the flow index subset are collided in a blocking mode, so that event models are respectively adopted in the plurality of threads for analysis, backtracking threat events are generated in parallel without conflict, and event detection performance is improved.
For example, in the process of searching 1000 flow indexes by adopting the information index set, the flow indexes can be further segmented into 10 sub-set thread blocks of 100 each, and corresponding threads are started to be processed in parallel.
S304: and sending each backtracking threat event to an event message queue, and triggering analysis and alarm for each backtracking threat event.
Based on the mode, the incremental information and the historical flow are adopted for collision and analysis, so that the preface attack behavior of real-time collision omission is compensated, the network attack chain is assisted to carry out complete tracing, the data configuration is carried out by adopting the mode, and the high-efficiency performance of event detection is ensured.
Further, based on the same technical concept, the embodiment of the application also provides an event detection device, which is used for realizing the above-mentioned method flow of the embodiment of the application. Referring to fig. 4, the apparatus includes: a first acquisition module 401, a second acquisition module 402, an event generation module 403, and a delivery module 404, wherein:
The first obtaining module 401 is configured to obtain, in response to a set collision opportunity, a set of information indicators, where each information indicator is associated with one incremental information in the local information base, and an information recording time of each incremental information is not earlier than a set historical detection time.
A second obtaining module 402, configured to obtain, from a traffic index library, a traffic index set for the collision opportunity, where each traffic index is associated with a trace traffic in a traffic message queue, and each trace traffic is obtained through historical log analysis.
The event generating module 403 is configured to collide and analyze the information index set with the flow index set, so as to obtain each backtracking threat event.
And the transmission module 404 is configured to send the trace-back threat events to an event message queue, and trigger analysis and alarm for the trace-back threat events.
In an alternative embodiment, before the acquiring the information index set in response to the set collision opportunity, the first acquiring module 401 is further configured to:
and acquiring each backtracking flow aiming at the collision time in the flow message queue according to the started consumption process.
And extracting indexes of the backtracking flows to obtain candidate indexes, and filtering the candidate indexes by adopting a filtering rule to obtain flow indexes.
And sending each flow index to the flow index library, triggering the flow index library, merging and storing each flow index according to a set period, and merging and storing the local flow library by adopting persistence.
In an optional implementation manner, the triggering the flow index library is used for merging and storing the flow indexes according to a set period, and the first obtaining module 401 is specifically configured to:
triggering the flow index library, and merging and storing each first flow value in each flow index according to a set period, wherein the first flow value comprises: address information and hash values extracted by corresponding backtracking flow; the method comprises the steps of,
triggering the flow index library, compressing each second flow value in each flow index by adopting a compression algorithm, and merging and storing each obtained second compression value according to the set period, wherein the second flow value comprises: and correspondingly backtracking the uniform resource locator extracted by the flow.
In an optional embodiment, the acquiring an information index set in response to the set collision opportunity, and the first acquiring module 401 is specifically configured to:
and responding to the local information library triggering update information, and acquiring an information index set associated with a set index type and/or an information type.
Or alternatively, the process may be performed,
and responding to the set detection period, and acquiring an information index set associated with the set index type and/or the information type.
In an optional embodiment, the acquiring a set of flow indicators for the collision opportunity from the flow indicator library, the second acquiring module 402 is specifically configured to:
and acquiring flow record time of corresponding backtracking flow from a flow index library, wherein the flow record time belongs to a plurality of flow indexes in a set backtracking period and is a flow index set aiming at the collision time.
And/or the number of the groups of groups,
and acquiring the log type of the corresponding backtracking flow correlation history log from the flow index library, wherein the log type is a set of a plurality of flow indexes of the backtracking type and is a set of flow indexes aiming at the collision time.
In an alternative embodiment, the flow index set is obtained in batches according to set backtracking data amounts,
The collision and analysis are performed on the information index set and the flow index set to obtain each backtracking threat event, and the event generation module 403 is specifically configured to:
for a plurality of batches for which the flow index set is acquired, the following operations are respectively executed:
aiming at a flow index subset of corresponding backtracking data volume, which is acquired from one batch, collision is carried out on the information index set and the flow index subset to obtain subset collision data;
and analyzing the subset collision data by adopting an event model to obtain each retrospective threat event corresponding to the information index set in the flow index subset.
In an alternative embodiment, the information index set adopts a plurality of threads to collide with a plurality of sub-set thread blocks of the flow index sub-set simultaneously,
the event model is adopted to analyze the index collision data to obtain each retrospective threat event in the traffic index subset, and the event generation module 403 is specifically configured to:
and in the multiple threads, concurrently adopting the event model, analyzing multiple block collision data obtained by block collision of the information index set and the multiple subset threads, and obtaining each retrospective threat event in the flow index subset.
Based on the same inventive concept as the above-mentioned application embodiments, an electronic device is also provided in the application embodiments, and the electronic device may be used for event detection. In one embodiment, the electronic device may be a server, a terminal device, or other electronic device. In this embodiment, the electronic device may be configured as shown in fig. 5, including a memory 501, a communication interface 503, and one or more processors 502.
A memory 501 for storing a computer program for execution by the processor 502. The memory 501 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, a program required for running an instant messaging function, and the like; the storage data area can store various instant messaging information, operation instruction sets and the like.
The memory 501 may be a volatile memory (RAM), such as a random-access memory (RAM); the memory 501 may also be a non-volatile memory (non-volatile memory), such as a read-only memory, a flash memory (flash memory), a Hard Disk Drive (HDD) or a Solid State Drive (SSD), or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto. Memory 501 may be a combination of the above.
The processor 502 may include one or more central processing units (Central Processing Unit, CPU) or digital processing units, etc. A processor 502 for implementing the above-described event detection method when calling a computer program stored in the memory 501.
The communication interface 503 is used to communicate with terminal devices and other servers.
The specific connection medium between the memory 501, the communication interface 503, and the processor 502 is not limited in the embodiments of the present application. In the embodiment of the present application, the memory 501 and the processor 502 are connected by the bus 504 in fig. 5, the bus 504 is shown by a thick line in fig. 5, and the connection manner between other components is only schematically illustrated, but not limited to. The bus 504 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in fig. 5, but not only one bus or one type of bus.
Based on the same inventive concept, the embodiments of the present application also provide a storage medium storing computer instructions that, when executed on a computer, cause the computer to perform an event detection method as previously discussed.
It should be noted that although several units or sub-units of the apparatus are mentioned in the above detailed description, such a division is merely exemplary and not mandatory. Indeed, the features and functions of two or more of the elements described above may be embodied in one element in accordance with embodiments of the present application. Conversely, the features and functions of one unit described above may be further divided into a plurality of units to be embodied.
Furthermore, although the operations of the methods of the present application are depicted in the drawings in a particular order, this is not required to or suggested that these operations must be performed in this particular order or that all of the illustrated operations must be performed in order to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform.
The embodiment of the application provides an event detection method, an event detection device, electronic equipment and a storage medium, which are used for responding to set collision time, acquiring an information index set, and acquiring the flow index set aiming at the collision time from a flow index library, wherein each information index is related to one increment information in a local information library, the information recording time of each increment information is not earlier than the set historical detection time, each flow index is related to one backtracking flow in a flow message queue, each backtracking flow is obtained through historical log analysis, and further, collision and analysis are carried out on the information index set and the flow index set to obtain each backtracking threat event, so that backtracking detection is carried out on the attack behavior before the information generation based on the mode, and the traceability integrity of a network attack chain is improved.
On the other hand, the historical detection time is adopted to configure each increment information for collision, and each backtracking flow for collision is obtained through a flow message queue, so that repeated collision of data is avoided, the total data quantity of buffering needed during collision is reduced, and the detection efficiency for threat events is improved.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a server, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's equipment, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server.
In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected over the Internet using an Internet service provider).
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various modifications and variations can be made in the present application without departing from the spirit or scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims and the equivalents thereof, the present application is intended to cover such modifications and variations.

Claims (10)

1. An event detection method, comprising:
responding to the set collision time to obtain an information index set, wherein each information index is associated with one increment information in a local information base, and the information recording time of each increment information is not earlier than the set historical detection time;
acquiring a flow index set aiming at the collision time from a flow index library, wherein each flow index is associated with one retrospective flow in a flow message queue, and each retrospective flow is acquired through historical log analysis;
Collision and analysis are carried out on the information index set and the flow index set, so that each backtracking threat event is obtained;
and sending each retrospective threat event to an event message queue, and triggering analysis and alarm for each retrospective threat event.
2. The method of claim 1, wherein prior to obtaining the set of informative indicators in response to the set collision opportunity, further comprising:
acquiring each backtracking flow of the collision opportunity in the flow message queue according to the started consumption process;
extracting indexes of the backtracking flows to obtain candidate indexes, and filtering the candidate indexes by adopting a filtering rule to obtain flow indexes;
and sending each flow index to the flow index library, triggering the flow index library, merging and storing each flow index according to a set period, and merging and storing the local flow library by adopting persistence.
3. The method of claim 2, wherein triggering the flow index library to merge and store the flow indexes according to a set period comprises:
triggering the flow index library, and merging and storing each first flow value in each flow index according to a set period, wherein the first flow value comprises: address information and hash values extracted by corresponding backtracking flow; the method comprises the steps of,
Triggering the flow index library, compressing each second flow value in each flow index by adopting a compression algorithm, and merging and storing each obtained second compression value according to the set period, wherein the second flow value comprises: and correspondingly backtracking the uniform resource locator extracted by the flow.
4. A method according to any one of claims 1-3, wherein said obtaining a set of informative indicators in response to a set collision opportunity comprises:
responding to the local information library triggering update information, and acquiring an information index set associated with a set index type and/or an information type;
or alternatively, the process may be performed,
and responding to the set detection period, and acquiring an information index set associated with the set index type and/or the information type.
5. A method according to any one of claims 1-3, wherein said obtaining a set of traffic indicators for said collision occasion from a traffic indicator library comprises:
acquiring flow record time of corresponding backtracking flow from a flow index library, wherein the flow record time belongs to a plurality of flow indexes in a set backtracking period and is a flow index set aiming at the collision time;
and/or the number of the groups of groups,
and acquiring the log type of the corresponding backtracking flow correlation history log from the flow index library, wherein the log type is a set of a plurality of flow indexes of the backtracking type and is a set of flow indexes aiming at the collision time.
6. The method according to claim 1 to 3, wherein the set of flow indicators is obtained batchwise according to a set amount of each trace data,
and performing collision and analysis on the information index set and the flow index set to obtain each backtracking threat event, wherein the collision and analysis comprises the following steps:
for a plurality of batches for which the flow index set is acquired, the following operations are respectively executed:
aiming at a flow index subset of corresponding backtracking data volume, which is acquired from one batch, collision is carried out on the information index set and the flow index subset to obtain subset collision data;
and analyzing the subset collision data by adopting an event model to obtain each retrospective threat event corresponding to the information index set in the flow index subset.
7. The method of claim 5, wherein the set of informative indicators employs multiple threads to concurrently collide with multiple subset thread blocks of the subset of traffic indicators,
and analyzing the index collision data by adopting an event model to obtain each backtracking threat event in the flow index subset, wherein the method comprises the following steps:
and in the multiple threads, concurrently adopting the event model, analyzing multiple block collision data obtained by block collision of the information index set and the multiple subset threads, and obtaining each retrospective threat event in the flow index subset.
8. An event detection apparatus, comprising:
the first acquisition module is used for responding to the set collision time to acquire an information index set, wherein each information index is associated with one increment information in the local information base, and the information recording time of each increment information is not earlier than the set historical detection time;
the second acquisition module is used for acquiring a flow index set aiming at the collision time from a flow index library, wherein each flow index is associated with one retrospective flow in a flow message queue, and each retrospective flow is obtained through historical log analysis;
the event generation module is used for carrying out collision and analysis on the information index set and the flow index set to obtain each backtracking threat event;
and the transmission module is used for sending each retrospective threat event to an event message queue and triggering analysis and alarm for each retrospective threat event.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any of claims 1-7 when executing the computer program.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any of claims 1-7.
CN202211634738.4A 2022-12-19 2022-12-19 Event detection method and device, electronic equipment and storage medium Pending CN116015823A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211634738.4A CN116015823A (en) 2022-12-19 2022-12-19 Event detection method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211634738.4A CN116015823A (en) 2022-12-19 2022-12-19 Event detection method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116015823A true CN116015823A (en) 2023-04-25

Family

ID=86032730

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211634738.4A Pending CN116015823A (en) 2022-12-19 2022-12-19 Event detection method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116015823A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117520313A (en) * 2024-01-02 2024-02-06 北京淇瑀信息科技有限公司 Data backtracking method and device based on multidimensional associated data warehouse slice table

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117520313A (en) * 2024-01-02 2024-02-06 北京淇瑀信息科技有限公司 Data backtracking method and device based on multidimensional associated data warehouse slice table
CN117520313B (en) * 2024-01-02 2024-03-26 北京淇瑀信息科技有限公司 Data backtracking method and device based on multidimensional associated data warehouse slice table

Similar Documents

Publication Publication Date Title
CN108763031B (en) Log-based threat information detection method and device
CN106657057B (en) Anti-crawler system and method
CN112114995B (en) Terminal abnormality analysis method, device, equipment and storage medium based on process
US9661003B2 (en) System and method for forensic cyber adversary profiling, attribution and attack identification
CN107992415B (en) Fault positioning and analyzing method of transaction system and related server
US11809406B2 (en) Event records in a log file
CN107426196B (en) Method and system for identifying WEB invasion
CN108073499B (en) Application program testing method and device
KR101266930B1 (en) A visualization system for Forensics audit data
CN116015823A (en) Event detection method and device, electronic equipment and storage medium
CN110908957A (en) Network security log audit analysis method in power industry
CN111859399A (en) Vulnerability detection method and device based on oval
CN112784268A (en) Method, device, equipment and storage medium for analyzing host behavior data
CN115001753A (en) Method and device for analyzing associated alarm, electronic equipment and storage medium
Fatemi et al. Threat hunting in windows using big security log data
CN110650137A (en) Coal mine network abnormal behavior early warning method, system, equipment and readable storage medium
CN114461864A (en) Alarm tracing method and device
CN113987492A (en) Method and device for determining alarm event
CN110769003B (en) Network security early warning method, system, equipment and readable storage medium
CN115794479B (en) Log data processing method and device, electronic equipment and storage medium
WO2020017000A1 (en) Cyber attack information analyzing program, cyber attack information analyzing method, and information processing device
CN115509851A (en) Page monitoring method, device and equipment
CN116155519A (en) Threat alert information processing method, threat alert information processing device, computer equipment and storage medium
CN110990830A (en) Terminal evidence obtaining and tracing system and method
CN115086071B (en) Data stealing detection method, system and equipment based on cause and effect tracing of logs

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination