CN117978450A

CN117978450A - Security detection method, device, equipment and storage medium

Info

Publication number: CN117978450A
Application number: CN202410001976.4A
Authority: CN
Inventors: 韩颖; 高钦晴; 倪鹏; 王文轩
Original assignee: Tianyi Safety Technology Co Ltd
Current assignee: Tianyi Safety Technology Co Ltd
Priority date: 2024-01-02
Filing date: 2024-01-02
Publication date: 2024-05-03

Abstract

The application discloses a security detection method, a security detection device, security detection equipment and a security detection storage medium, which relate to the technical field of network and information security. According to the sequence of the detection priority from high to low, each log feature is matched with a preset detection data set, the detection priority of each log feature is updated according to a matching result and a preset priority updating strategy, and the updated detection priority is used for carrying out the safety detection processing on the next safety log so as to improve the safety detection efficiency.

Description

Security detection method, device, equipment and storage medium

Technical Field

The application relates to the technical field of network and information security, and provides a security detection method, a security detection device, security detection equipment and a storage medium.

Background

Along with the high-speed development of network information technology, the information security field faces threats such as complex and changeable network attacks, malicious software, abnormal traffic and the like, and it is important to timely and effectively detect relevant threat behaviors.

At present, the existing security detection method needs to detect all fields in a security log to identify possible threat behaviors, the data volume to be detected is large, and the security detection efficiency is low.

Therefore, how to improve the safety detection efficiency is a problem to be solved.

Disclosure of Invention

The embodiment of the application provides a security detection method, a security detection device, security detection equipment and a storage medium, which are used for improving security detection efficiency.

In one aspect, a security detection method is provided, the method comprising:

Iteratively executing security detection processing on each security log to be detected until a detection result of each security log is obtained; wherein each security detection process includes:

Acquiring a log feature set of a target security log;

Determining the detection priority of each log feature based on the confidence level of the log feature; the confidence level characterizes abnormal association degrees between the corresponding log features and other log features in the log feature set;

based on a preset detection data set, matching the log features according to the sequence of the detection priority from high to low;

Based on the obtained matching result and a preset priority updating strategy, the detection priority of each log feature is updated, and the next security log is used as a target security log of the next security detection processing.

In one aspect, an embodiment of the present application provides a security detection device, including:

the processing unit is used for iteratively executing security detection processing on each security log to be detected until a detection result of each security log is obtained;

the processing unit comprises an acquisition subunit, a determination subunit and a matching subunit, wherein:

The acquisition subunit is used for acquiring a log feature set of the target security log;

the determining subunit is used for determining the detection priority of each log feature based on the confidence level of the log feature; the confidence level characterizes abnormal association degrees between the corresponding log features and other log features in the log feature set;

The matching subunit is used for matching the log features according to the sequence from high to low of the detection priority based on a preset detection data set;

And the updating unit is used for updating the detection priority of each log characteristic based on the obtained matching result and a preset priority updating strategy, and taking the next security log as a target security log of the next security detection process.

Optionally, the acquiring subunit is specifically configured to:

Extracting features of the target security log to obtain a corresponding candidate feature set;

Based on the support degree of each candidate feature, screening out log features with the support degree larger than a preset support degree threshold value from the candidate feature sets, and obtaining the log feature sets; and the support represents the ratio of the number of abnormal records of the corresponding candidate feature to the total number of abnormal records of the target security log.

Optionally, the determining subunit is specifically configured to:

Determining a front item feature from each log feature based on a preset confidence threshold; the confidence of the foretell feature is greater than the confidence threshold;

determining a detection weight value of each foretell feature according to a preset weight distribution strategy based on the confidence level of the foretell feature; wherein the detection weight value of each previous feature is larger than a preset value;

and obtaining the detection priority of each log feature based on the relative magnitude between the detection weight values.

Optionally, the determining subunit is specifically configured to:

Determining a maximum confidence coefficient and a minimum confidence coefficient from the confidence coefficient of each previous feature; determining a first weight parameter based on the difference between the maximum confidence and the minimum confidence;

determining, for each antecedent feature, a second weight parameter based on a difference between a confidence level of the antecedent feature and the minimum confidence level;

and obtaining the detection weight value of the front characteristic based on the ratio between the second weight parameter and the first weight parameter.

Optionally, after determining the antecedent feature from the log features based on a preset confidence threshold, the determining subunit is further configured to:

determining postamble features in the log features; the confidence of the postamble feature is smaller than the confidence threshold;

and determining the detection weight value of each postive feature as a preset value according to a preset weight distribution strategy.

In one aspect, an embodiment of the present application provides a computer device including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the steps of the security detection method described above when the processor executes the program.

In one aspect, embodiments of the present application provide a computer-readable storage medium storing a computer program executable by a computer device, which when run on the computer device, causes the computer device to perform the steps of the security detection method described above.

In one aspect, embodiments of the present application provide a computer program product comprising a computer program stored on a computer readable storage medium, the computer program comprising program instructions which, when executed by a computer device, cause the computer device to perform the steps of the security detection method described above.

In the embodiment of the application, when safety detection processing is carried out on each safety log to be detected, a log feature set of the safety log is obtained, and the detection priority of each log feature is determined according to the abnormal association degree between each log feature and other log features in the log feature set, namely the confidence level of the log feature, so as to identify the importance degree of each log feature in the safety detection processing process, thereby flexibly adjusting the detection sequence of different log features, effectively distributing computing resources, preferentially carrying out key detection on high risk features, reducing unnecessary detection on low risk features and improving the overall safety detection efficiency. According to the sequence of the detection priority from high to low, each log feature is matched with a preset detection data set, when the log feature is successfully matched with the detection data set, namely the potential security threat of the log feature is detected, the detection priority of each log feature is updated through a preset priority updating strategy, so that the updated detection priority is used for carrying out the security detection processing on the next security log, the purpose of fully utilizing real-time log data information and a circulating feedback mechanism is achieved, and the next security detection processing is optimized according to the security detection result of each time, so that the security detection processing can adapt to the security threat environment of the latest data change and the height change, and the accuracy and the sharpness of the security detection are improved.

Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application. The objectives and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it will be apparent that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of an application scenario provided in an embodiment of the present application;

FIG. 2 is a schematic diagram of a security detection system according to an embodiment of the present application;

Fig. 3 is a schematic flow chart of a security detection method according to an embodiment of the present application;

FIG. 4 is a schematic diagram of a transaction database according to an embodiment of the present application;

FIG. 5 is a flowchart of another security detection method according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of a security detection device according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application. Embodiments of the application and features of the embodiments may be combined with one another arbitrarily without conflict. Also, while a logical order of illustration is depicted in the flowchart, in some cases the steps shown or described may be performed in a different order than presented.

The terms first and second in the description and claims of the application and in the above-mentioned figures are used for distinguishing between different objects and not for describing a particular sequential order. Furthermore, the term "include" and any variations thereof is intended to cover non-exclusive protection. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus. The term "plurality" in the present application may mean at least two, for example, may be two, three or more, and embodiments of the present application are not limited.

The term "and/or" in the embodiment of the present application is merely an association relationship describing the association object, and indicates that three relationships may exist, for example, a and/or B may indicate: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.

In the technical scheme of the application, the data is collected, transmitted, used and the like, and all meet the requirements of national relevant laws and regulations. It will be appreciated that in the following detailed description of the application, reference is made to the collection of subject-related data and related data, and that when embodiments of the application are applied to a particular product or technology, related permissions or consents need to be obtained, and that the collection, use and processing of related data is required to comply with relevant national and regional laws and regulations and standards.

In order to facilitate understanding of the technical solution provided by the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

Information collision: a security detection method for discovering potential threats and attack activities by comparing and matching security event information in a security log with predefined information in a threat intelligence library includes cross-verifying information from different data sources to identify patterns or features associated with known threats.

Threat intelligence library: a collection of information containing information about computer network threats, attacks, vulnerabilities, and malicious activities. The information in the library is typically summarized and maintained by security professionals, security companies, government agencies, or other security organizations. The data in the threat intelligence library may include known attack patterns, malware fingerprints, malicious IP addresses, etc.

Security log: the information recorded by the system or network device about the security events, operation and operational status contains information about various aspects of user activity, system access, anomalies, etc., for auditing, monitoring and investigation of the security events.

Transaction database form: the method is characterized in that original data are arranged, cleaned and de-duplicated according to a certain specification and structure to form a database format suitable for transaction processing. In the security field, it is common to store cleaned security log data.

Advanced persistent threat (ADVANCED PERSISTENT THREA, APT): a long-term, organized network attack directed to a specific target. APT attacks typically take a hidden, persistent and sophisticated form to survive in the target network for a long period of time.

Engine x (nmginx): a high performance open source Web server may also be used as a reverse proxy server, load balancer, hypertext transfer protocol (Hypertext Transfer Protocol, HTTP) cache, and as an email proxy server. Many websites and applications choose to use ng nx as part of their infrastructure because of their high performance, stability, and low resource consumption.

Apache flame: a distributed, reliable, and useful service for high capacity data streams supports reliable collection, aggregation, and movement of data in large-scale data streams, such that the process of data from source to storage is simplified and reliable.

APACHE FLINK: a streaming processing engine and a distributed processing framework are provided for processing large-scale streaming data. High performance, fault tolerance, and accurate one-time processing semantics may be provided such that a user may process unbounded and bounded data streams in a reliable and efficient manner. The method is characterized by comprising a flexible stream processing model, scalability, fault tolerance and support for event time, so that the method becomes a powerful tool for processing real-time data and is widely applied to stream processing and batch processing scenes.

Denial of service provisioning/distributed denial of service attack (Denial of Service/Distributed Denial of Service, dos/DDos) traffic attack: a network attack mode that a large number of legal distributed servers send requests to a target so as to lead normal legal users to fail to obtain service is mainly characterized in that a large number of attack messages with fake source addresses are continuously sent to a network service port, so that a half-open connection queue in the target server is occupied, the bandwidth and host resources of the target server are maliciously occupied, and the network or the system is prevented from being overloaded so as to stop providing normal network service.

The following briefly describes the design concept of the embodiment of the present application:

Along with the high-speed development of network information technology, the information security field faces threats such as complex and changeable network attacks, malicious software, abnormal traffic and the like, and it is important to timely and effectively detect relevant threat behaviors. Existing security detection methods require detection of all fields in the security log to identify possible threat actions. For example, a global information collision method is commonly adopted at present, each field information in the security log is matched with a corresponding field in the threat information base until the matching processing of all field information is completed, and a security detection result corresponding to the security log is obtained.

However, the existing global information collision mode needs to perform indiscriminate information collision operation on the required fields in the security log, and has the problems that the data volume to be processed is large, analysis and processing requirements of mass data are difficult to deal with, the time consumption of security detection is long, and the security detection efficiency is low. Timeliness is critical in the field of data security, effective information has decision value in a specific time period, for example, for the treatment of network security threat, necessary measures are needed in time to reduce potential damage, and delay of security decision may lead to serious information leakage and other consequences.

In view of the above technical problems, an embodiment of the present application provides a security detection method, when performing security detection processing on each security log to be detected, a log feature set of a target security log is obtained, and a detection priority of each log feature is determined by an abnormal association degree between each log feature and other log features in the log feature set, that is, a confidence level of the log feature, so as to identify an importance degree of each log feature in a security detection processing process, thereby flexibly adjusting detection sequences of different log features, effectively allocating computing resources, preferentially performing key detection on high risk features, reducing unnecessary detection on low risk features, and improving overall security detection efficiency. According to the sequence of the detection priority from high to low, each log feature is matched with a preset detection data set, when the log feature is successfully matched with the detection data set, the potential security threat of the log feature is detected, the detection priority of each log feature is updated through a preset priority updating strategy, the updated detection priority is used for carrying out the security detection processing on the next security log, the real-time log data information and a circulating feedback mechanism are fully utilized, and the next security detection processing is optimized according to the security detection result of each time, so that the security detection processing can adapt to the security threat environment of the latest data change and the height change, and the accuracy and the sharpness of the security detection are improved

After the design idea of the embodiment of the present application is introduced, some simple descriptions are made below for application scenarios applicable to the technical solution of the embodiment of the present application, and it should be noted that the application scenarios described below are only used for illustrating the embodiment of the present application and are not limiting. In the specific implementation process, the technical scheme provided by the embodiment of the application can be flexibly applied according to actual needs.

The scheme provided by the embodiment of the application can be applied to any business scene related to safety detection, including but not limited to application scenes such as information collision. As shown in fig. 1, an application scenario is schematically provided in an embodiment of the present application, where the scenario may include a security detection device 100, a device to be detected 110, and a network 120.

The security detection device 100 may be a computer device with a certain processing capability, such as a mobile phone, a personal computer (personal computer, PC), a server, etc. that can be configured to perform any one of the methods provided in the embodiments of the present application, which are not illustrated here. For convenience of description, hereinafter, embodiments of the method will be described taking an execution subject of the method as a server capable of executing the method as an example. It will be appreciated that the subject matter of the method being performed by the server is merely an exemplary illustration and should not be construed as limiting the method. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and artificial intelligence platforms, but is not limited thereto.

The device to be detected 110 is a computer device to be detected, such as a server, a router, a gateway device, etc., for which the security detection method provided by the embodiment of the present application is directed, where the server may be an independent physical server, may be a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server that provides cloud services, a cloud database, cloud computing, a cloud function, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and an artificial intelligent platform, but is not limited thereto.

The security detection device 100 can acquire log data generated by various systems and network devices waiting for the detection device 110, wherein the log data contains key information such as system running state, security event and operation record, and based on the security detection method provided by the embodiment of the application, security detection functions such as information collision of the device 110 to be detected are realized.

The security detection device 100 and the device to be detected 110 may be connected through a network 120, where the network 120 may be a wired network, or may be a Wireless network, for example, a Wireless network may be a mobile cellular network, for example, a fourth generation mobile communication (4 g) network, a fifth generation mobile communication (5 g) network, or a New Radio (NR) network, or may be a Wireless-Fidelity (WIFI) network, or may be other possible networks, which may not be limited in this embodiment of the present invention.

In one possible implementation manner, in order to continuously monitor network security, real-time security threat information is provided, and the security detection device 100 may be an intrusion detection system (Intrusion Detection System, IDS) and/or an intrusion protection system (Intrusion Prevention System, IPS), based on the security detection method provided by the embodiment of the present application, monitor network traffic and network intrusion behavior recorded in a security log of a device to be detected in real time, analyze relevant features in log data, identify potential network intrusion behavior, dynamically adjust security detection policies such as an information collision policy according to a real-time analysis result, and improve identification efficiency and identification accuracy of network intrusion behavior.

In one possible implementation manner, the security detection device 100 may be applied to abnormal traffic detection and defense, based on the security detection method provided by the embodiment of the present application, monitors network traffic and traffic patterns, identifies abnormal traffic behavior, adjusts security detection policies such as information collision in real time according to normal traffic characteristics, implements traffic filtering and defense measures, and implements differentiated defense policies for different types of network attacks, so as to ensure network security and stability.

In a possible implementation manner, the security detection device 100 may be applied to malware detection and prevention, and based on the security detection method provided by the embodiment of the present application, by analyzing the malicious activity and the malicious software behavior in the log data, and combining with implementing a threat information base and a blacklist database, the known characteristics of the malicious software are detected quickly and accurately, and security detection policies such as information collision and the like are dynamically adjusted, so that the detection efficiency of the malicious software is improved, and the system defense policy is updated in time, and the isolation and the removal of the malicious software are implemented, so that the security and the integrity of the network system are improved.

In one possible implementation manner, the security detection device 100 may be applied to real-time response and handling of network security events, and based on the security detection method provided by the embodiment of the present application, security logs and alarm information are monitored in real time, and security detection policies are dynamically adjusted according to the urgency and threat level of the security events, so as to optimize event response flow and resource allocation manner, and implement faster handling of security events such as isolating affected systems, repairing vulnerabilities, reinforcing security defenses, and the like.

In one possible implementation, the security detection device 100 may analyze the security log in real time based on the security detection method provided by the embodiment of the present application, and continuously learn new network attack patterns and abnormal behaviors to automatically adjust the access control policy to ensure that only authorized users or authorized devices have authority to access the sensitive data and critical systems.

It should be noted that, the method shown in fig. 1 is only illustrative, and the number of the safety detection devices and the devices to be detected is not limited in practice, and the embodiment of the present application is not particularly limited.

As shown in fig. 2, a system architecture diagram of a security detection device according to an embodiment of the present application is shown, where the security detection device specifically includes the following modules:

And an analysis module: the system comprises a data acquisition module and a real-time calculation module. The data acquisition module acquires the security logs generated by the detection equipment such as each network system and each network equipment in real time from the application module, and transmits the security logs to the real-time calculation module. The real-time calculation module comprises a preprocessing sub-module, a priority calculation sub-module, a detection updating sub-module and the like, wherein the preprocessing sub-module performs preprocessing such as data cleaning, de-duplication, data format conversion and the like on the safety log, extracts log features in the safety log, and transmits the log features to the priority calculation sub-module. The priority calculating submodule calculates corresponding detection weight values of all log features according to the confidence degrees of all log features, and distributes different detection priorities for all log features according to the relative sizes of the detection weight values. The detection updating sub-module updates the detection weight value and the detection priority of the log characteristics according to the safety detection results of each safety log, and sends the updated results to the offline test module.

Specifically, the data acquisition module can transmit the safety log data through Ngnix, so that the data can be ensured to be smoothly sent to the subsequent processing module. Once the log data is transmitted to the thumb service, the thumb service will be responsible for transmitting the log data to the real-time computing module. In order to ensure that data is processed and updated on time and on demand and real-time requirements are guaranteed, the real-time calculation module in the embodiment of the application can efficiently process large-scale data streams through the Flink technology, can convert message queue data transmitted by the Flume service into a transaction data set, and ensures that the data can be effectively processed and analyzed. The detection updating sub-module can adopt a dynamically adjusted time window, perform accurate data analysis and updating according to a preset time span, ensure that data processing is performed within a specific time period, and optimize data processing efficiency, thereby providing a flexible strategy adjustment function, enabling related personnel to adjust the size and updating frequency of the time window according to specific requirements, and improving the self-adaptability and applicability of the system.

And an offline test module: and evaluating and testing the updating result sent by the detection updating sub-module through the prestored historical data, and generating an uploading file according to the latest detection priority and sending the uploading file to the application module when the quality and the accuracy of the detection weight value and the detection priority of each log feature pass through the test, so that the follow-up data processing and updating can be carried out according to the set standard.

And an application module: the system comprises an online updating module and a detection module. And the online updating module receives the uploading file sent by the offline testing module and updates the security detection strategy. And the detection module carries out detection processing such as information collision and the like on the data to be detected according to the updated security detection strategy, for example, carries out information collision operation on the data to be detected and a threat information base, and generates a new log file.

It should be noted that the components and structures of the functional block diagram shown in fig. 2 are merely exemplary and not limiting, and that other components and structures may be provided as desired in a practical scenario.

The security detection method provided by the exemplary embodiments of the present application will be described below with reference to the accompanying drawings in conjunction with the application scenarios described above, and it should be noted that the application scenarios described above are only shown for the convenience of understanding the spirit and principles of the present application, and embodiments of the present application are not limited in this respect.

Referring to fig. 3, a flow chart of a security detection method provided by an embodiment of the present application is illustrated by taking a security detection device as an execution body, and since in the embodiment of the present application, in performing security detection processing on each security log iteration to obtain a detection result of each security log, the security detection processing process of each iteration is similar, and thus, the description is given here taking an iteration as an example, and a specific implementation flow of the method is as follows:

step 301: a log feature set of the security log is obtained.

In the embodiment of the application, the security log data generated by each device is respectively acquired from each device to be detected, and the related log characteristics in the security log are extracted for subsequent security detection processing. The security log records key security detection information such as security events, running states, operation records and the like related to the equipment to be detected, and whether the equipment has security risks can be detected through the security log.

In one possible implementation manner, in the process of acquiring the security logs in the network device, the server or other devices to be detected, the data screening can be performed on the security logs, so that each acquired security log contains key security events and operation records, and the comprehensiveness and accuracy of subsequent security detection are ensured.

Specifically, the data screening can be performed on the security log through presetting keywords, regular expressions, character string matching and other modes, only log data containing the preset keywords are reserved, subsequent related processing on irrelevant log data is avoided, calculation resources are wasted, calculation redundancy is increased, and security detection efficiency is reduced.

In a possible implementation manner, the embodiment of the application further performs data preprocessing on the security log before extracting the log features of the security log, including but not limited to processing missing values, abnormal values and repeated records, and data cleaning and the like, so as to ensure the integrity and consistency of the data. And the preprocessed log data is converted into a consistent transaction database form (the abstract expression is adopted in the embodiment and does not represent the final specific machine processing format), so that the problem of inconsistent or nonstandard formats is avoided, and the subsequent safety detection processing for the log features is influenced.

Specifically, the transaction database stores the preprocessed log data in the form of tables, each table corresponds to a security log, one table contains a plurality of rows, each row corresponds to a security event or behavior record, and is divided into a plurality of columns, and each column represents field information of different data types. Referring to fig. 4, a security log table is shown in which each row corresponds to a record of a security event, and each row contains a plurality of columns of time stamps, event types, source internet protocol (Internet Protocol, IP) addresses, destination IP addresses, event descriptions, etc. The transaction database structure provides a highly structured data storage mode, supports complex data retrieval and processing operations through query languages such as (Structured Query Language, SQL) and the like, and is convenient for query, association and analysis.

In a possible implementation manner, in order to further improve the security detection efficiency, the embodiment of the present application may further screen out, through the support degree of each candidate feature, a log feature with a support degree greater than a preset support degree threshold value, for executing subsequent security detection processing. The support represents the ratio of the number of abnormal records of the corresponding candidate feature to the total number of abnormal records of the log data, so that the candidate feature with the support smaller than the preset support threshold represents the feature that the candidate feature is unlikely to be abnormal, possibly noise data or abnormal, and screening the feature is helpful for reducing the calculation complexity and improving the efficiency and accuracy of safety detection.

Specifically, taking a candidate feature set extracted from a security log as { x ₁,x₂,…,x_n},x_i, where i is a positive integer greater than 0 and less than n, and n is the total number of log features contained in the log feature set as an example. And calculating the support degree of each candidate feature through a preset support degree calculation formula, and screening out log features larger than the value through a support degree threshold value. The support calculation formula is as follows:

Wherein S _i represents the support corresponding to the i-th candidate feature x _i;

n represents the total number of abnormal records in the security log;

num _i represents the number of abnormal records corresponding to the i-th candidate feature x _i.

In one possible implementation manner, in order to avoid repetition of candidate features in the candidate feature set and ensure that the candidate feature set contains all data information of the security log, the embodiment of the application obtains a single feature set by scanning security log data and identifying frequently occurring single features therefrom, and generates a possible double feature set by combining each single feature set. Removing a double feature set containing infrequent features through pruning operation, generating N feature sets through the double feature set, pruning until the N feature sets cannot be generated, and obtaining a candidate feature set through synthesis of the obtained N Xiang Te feature sets.

Specifically, taking a simplified security log as an example, the security log includes the following records: { user a, operation: logging in and accessing files: file 1} { user B, operation: logging off, file access: file 2} { user A, operation: logging off, file access: file 1} { user C, operation: logging in and accessing files: file 3).

Step 1: by scanning the security log, identifying single characteristics frequently appearing in the whole security log: { user a }, { user B }, { user C }, { operation: login }, operation {: logoff } { file access: file 1} { File Access: file 2} { File Access: file 3).

Step 2: a possible set of bijective features is generated from the above-mentioned single features. Taking the user feature as an example, the method comprises the following steps: { user a, user B }, { user a, user C }, { user B, user C }, { user a, operation: login }, { user a, operation: logout } { user a, file access: file 1} { user B, operation: login }, { user B, operation: logout } { user B, file access: file 2} { user C, operation: login }, { user C, operation: logout } { user C, file access: file 3} { operation: logging in, and operating: logoff }, { operation: logging in and accessing files: file 1} { operation: logging off, file access: file 1} { File Access: file 1, file access: file 2} { File Access: file 1, file access: file 3} { File Access: file 2, file access: file 3).

Step 3: the dual feature set containing the infrequent features is removed by pruning operations. Specifically, this can be achieved by querying a single feature. For example, if { user A, user B } is not a frequently occurring feature in the subset { user B } biterm feature set, it may be determined that { user A, user B } is a bifeature set that contains non-frequent features, and a removal operation is performed to ensure the completeness of the candidate feature set.

Step 4: and repeatedly generating possible three feature sets through the double features, repeating the operation of generating a new N Xiang Te feature set through the (N-1) Xiang Te feature set, and removing the feature set containing the infrequent features through pruning operation until more N feature sets cannot be generated. Taking the example of generating three feature sets by using a dual feature set, wherein the dual feature set comprises { A, B }, { A, C }, { B, C }, and generating possible three feature sets as { A, B, C }, { A, B, D }, { A, C, D }, { B, C, D }, checking whether a subset of each three feature set is frequent feature, so that after pruning, the following three feature sets { A, B, C }, { A, C, D } arereserved.

Step 5: the N feature sets obtained in the process are integrated to generate a candidate feature set of the safety log, namely a feature set possibly formed by all features frequently appearing in the safety log, and the candidate feature set can be used for mining association relations among log features, carrying out safety event analysis and the like.

Step 302: and determining the detection priority of each log feature based on the confidence level of the log feature.

In the embodiment of the application, after each log feature is extracted from the security log, the detection priority of each log feature is respectively determined by representing the confidence coefficient of the abnormal association degree between the corresponding log feature and other log features, and the detection priority is used for realizing the security detection processing of each log feature according to the detection priority.

In a possible implementation manner, the embodiment of the application can determine the front item features with higher confidence coefficient than the confidence coefficient threshold value from the log features based on the preset confidence coefficient threshold value, filter the log features with lower confidence coefficient, and because the front item features have strong relevance with other log features, namely, when the front item features are abnormal, the other log features are influenced and have abnormal phenomena with high probability, the front item features with higher confidence coefficient have higher security threat, and the attention of security detection needs to be focused on the front item features with higher confidence coefficient, so that the features with higher security threat are focused preferentially, and the accuracy and efficiency of detection are improved. For each forefront feature, determining a detection weight value of each forefront feature according to a preset weight distribution strategy and the respective confidence coefficient of each forefront feature, wherein the detection weight value of each forefront feature is larger than a preset value. Corresponding detection priorities are distributed to the log features according to the relative sizes of the detection weight values of the front features, so that the detection priorities of the log features are adjusted in real time according to the change of the detection weight values, the security detection operation can be quickly adapted to new risk changes, and the network security coping capacity is improved.

Specifically, taking the example that the log feature set extracted from one security log is { x ₁,x₂,…,x_n},x_i for representing the ith log feature, the confidence coefficient T _ij can be obtained by the following formula:

Wherein x _i represents the i-th log feature in the set of log features, and x _j represents the j-th log feature in the set of log features; i and j are positive integers greater than 0 and less than n, n being the total number of log features contained in the log feature set;

t _ij represents the probability of an abnormality in log feature x _j when an abnormality occurs in log feature x _i.

Num _i represents the number of abnormal records corresponding to the ith log feature in the log feature set;

num _j represents the number of abnormal records corresponding to the j-th log feature in the log feature set.

Further, since the candidate feature xi may have a high abnormal correlation effect on a plurality of other log features, in order to determine whether the candidate feature is a previous feature and calculate a corresponding detection weight value through the confidence degrees, it is necessary to perform accumulation calculation on all the confidence degrees of the candidate feature xi to obtain a confidence degree accumulation sum Ai of the candidate feature xi, and the calculation formula is as follows:

A _i represents the confidence accumulated sum corresponding to the ith log feature in the log feature set;

T _ij represents the probability of abnormality of the log feature x _j when abnormality occurs in the log feature x _i;

n represents the total number of log features of the log feature set corresponding to the security log.

Therefore, the front-item features with the confidence coefficient accumulation sum Ai larger than the confidence coefficient threshold value are screened from the log features through the confidence coefficient threshold value, so that the other log features with abnormal association influence of each front-item feature in the log feature set can be ensured, namely, when each front-item feature is abnormal, at least one other log feature with high abnormal probability exists in the log feature set.

In one possible implementation, in order to conveniently and accurately allocate different detection weight values to each previous feature, it is necessary to ensure that each previous feature has the same dimension and dimension unit. Therefore, after the obtained confidence coefficient accumulation sum of each front characteristic is processed by data normalization, the dimensional influence among the front characteristics is eliminated by further processing the characteristic data in a characteristic scaling way, so that the front characteristics have comparability, the comprehensive comparison evaluation of the front characteristics is facilitated, and the detection weight value of each front characteristic is determined according to a preset weight distribution strategy and the respective confidence coefficient of each front characteristic.

Specifically, the embodiment of the application determines the maximum confidence coefficient and the minimum confidence coefficient from the confidence coefficient of each precursor feature, and determines the first weight parameter through the difference between the maximum confidence coefficient and the minimum confidence coefficient. And determining a second weight parameter corresponding to each foretell feature according to the difference between the confidence coefficient of the foretell feature and the minimum confidence coefficient. And obtaining the detection weight value of the previous feature through the ratio between the respective second weight parameter and the first weight parameter of each previous feature.

Specifically, the calculation formula of the detection weight value is as follows:

wherein W _i represents a detection weight value of the i-th previous feature in the log feature set, i is a positive integer greater than 0 and less than n, and n is the total number of previous features contained in the log feature set;

a _i represents the confidence accumulated sum corresponding to the ith previous feature in the log feature set;

a _min represents the minimum value of the confidence coefficient accumulation sum in the confidence coefficient accumulation sums of the various previous features;

A _max represents the maximum value of the confidence accumulated sums in the confidence accumulated sums for each of the antecedent features.

In one possible implementation manner, after the front features with the confidence degrees being greater than the confidence degree threshold are determined, for the rear features with the determined confidence degrees being less than the confidence degree threshold, the detection weight value of each rear feature may be set to a preset value according to a preset weight distribution policy, and since the detection weight values of the front features are all greater than the preset value, the detection priority of the rear features may be ensured to be less than that of all the front features.

Specifically, when it is determined that the post-term features have no higher abnormal association influence on other logs according to the confidence coefficient, the detection weight of each post-term feature can be set to be 0, and the possibility of detecting the post-term features is not completely eliminated, so that in the subsequent safety detection process, if all the pre-term features are not successfully matched with the detection data set, safety detection processing can be performed on the post-term features, and safety detection errors caused by special conditions are prevented.

Step 303: and based on a preset detection data set, matching the log features according to the sequence of the detection priority from high to low.

In the embodiment of the application, after the detection priority of each log feature of the security log is determined, each log feature is sequentially matched with corresponding data in the detection data set according to the sequence from high to low of the detection priority, and whether the log feature is successfully matched with field information in the detection data set is determined so as to identify the possible security threat of the security log.

Specifically, taking the example that the detection data set stores known malicious IP address, malicious port, protocol type and other information, according to the detection priority order, log features such as the IP address, port number, protocol type and the like of the security log are sequentially matched with corresponding field information in the detection data set, so that a matching result is output, wherein the matching result may include the matched threat type, related threat description, suggested response measures and other information.

Step 304: and updating the detection priority of each log feature based on the obtained matching result and a preset priority updating strategy.

In the embodiment of the application, in the process of matching each log feature according to the sequence from high to low of the detection priority, the detection priority of each log feature can be updated according to the matching result of each time, so that real-time log data information and a circulating feedback mechanism are fully utilized, the next safety detection processing is optimized according to the safety detection result of each time, the safety detection processing can adapt to the latest data change and the safety threat environment of high change, and the accuracy and the acuity of the safety detection are improved.

Specifically, when the matching result indicates that the matching between the corresponding log feature and the detection data set is successful, the detection weight value of each log feature is updated through a preset weight updating formula, and the weight updating formula is as follows:

W_i＝α*(W_i+k*C)

Wherein W _i represents a detection weight value of an i-th log feature in the log feature set, i is a positive integer greater than 0 and less than n, and n is a total number of log features contained in the log feature set.

And alpha is a weighting coefficient used for weighting each log feature, and ensuring that the sum of detection weight values of the weighted log features is still 1. Alpha can be a constant or adjusted according to actual conditions, namely the original weight of each log characteristic is W= { W ₁,W₂,…,W_n }, and the updated weight W ^· after primary matching is W^·＝{W₁ ^·,W₂ ^·,…,W_n ^·}＝{αW₁,αW₂,…,αW_n}.

K is an adjustable parameter used for adjusting the influence degree of the confidence coefficient on the log characteristics in the current weight updating process, and the k value needs to be set according to actual conditions so as to ensure that the update influence of the confidence coefficient on the detection weight value of the log characteristics meets actual requirements.

Step 305: and carrying out security detection processing on the next security log according to the latest detection priority.

In the embodiment of the application, the priority of each log characteristic is updated according to the security detection result of each security log for the next security detection processing.

In one possible implementation manner, the termination conditions of the security detection process that is iteratively performed in the embodiment of the present application include: in the safety detection processing process, all log features contained in the safety log are subjected to matching processing, and if all log features are failed to be matched, an empty set is returned, and the detection weight value of the log features does not need to be updated. Or, once the log features are successfully matched, the updated detection weight value and the current matching result are obtained and returned according to the weight updating operation in the step 304, so that threat information or a security analysis report can be generated according to the matching result, thereby helping related personnel to know the security threat state of the current equipment to be detected and to take corresponding security protection measures and coping strategies, and improving the security of the equipment.

The following describes the scheme of the embodiment of the present application with reference to a specific example in an APT scenario, and referring to fig. 5, a specific implementation flow of the method is as follows:

step 501: and acquiring the security log data of the APT.

Specifically, the feature data of the APT security log is subjected to preprocessing such as data cleaning, missing value processing, abnormal value processing, repeated recording and the like, so that the accuracy and the integrity of the data are ensured, and the original feature data information is converted into a transaction database format suitable for subsequent analysis (in this embodiment, abstract expression is adopted, and the abstract expression is not representative of a final specific machine processing format).

Step 502: and extracting the characteristics of the safety log data to obtain a candidate characteristic set.

Specifically, the candidate feature set may include, but is not limited to: { { Injoining }, …, { Access time }, …, { Injoining, exjoining }, …, { File Access record, access time }, …, { Injoining, exjoining, link tracking number }, …, { Process record, file Access record, access time }, …, { Injoining, exjoining, link tracking number, user request }, …, { user request, process record, file Access record, access time }.

Step 503: and screening the log feature set from the candidate feature sets based on the support degree of the log features.

Specifically, the support threshold is set to be 0.4, and a support calculation formula is adoptedAnd calculating the support degree of each log feature, and screening each log feature with the support degree larger than 0.4 from the candidate feature set as { { input parameter, output parameter, link tracking number, user request }, { input parameter, output parameter, link tracking number, process activity }, { input parameter, link tracking number, user request, process activity }, { link tracking number, user request, process activity, file access record }.

Step 504: and determining the detection priority of each log feature based on the confidence accumulation sum of the log features.

Specifically, a confidence coefficient threshold value is set to be 0.6, front-term features { parameter in } { link tracking number } with confidence coefficient larger than 0.6 are screened out from a log feature set through a confidence coefficient calculation formula, the detection weight value of each front-term feature is determined according to a calculation formula of the detection weight value, and meanwhile, 0 weight is allocated to each rear-term feature. And according to the detection weight value, corresponding detection priority is allocated to each log feature.

Step 505: and matching the log features according to the sequence of the detection priority from high to low.

Step 506: and when the matching is successful, updating the detection priority of each log feature.

Specifically, when it is determined that a certain log feature is successfully matched in sequence, the detection weight value of each log feature is recalculated according to a weight updating formula, and the detection priority of each log feature is dynamically updated through a cyclic feedback mechanism.

Step 506: and carrying out security detection processing on the next security log according to the latest detection priority.

Referring to fig. 6, based on the same inventive concept, an embodiment of the present application further provides a security detection device 60, which includes:

a processing unit 601, configured to iteratively perform security detection processing on each security log to be detected until a detection result of each security log is obtained;

The processing units include an acquisition subunit 6011, a determination subunit 6012, a matching subunit 6013, wherein:

an acquisition subunit 6011 configured to acquire a log feature set of the target security log;

A determining sub-unit 6012 for determining detection priorities of the respective log features based on the confidence levels of the log features; the confidence level characterizes abnormal association degree between the corresponding log feature and other log features in the log feature set;

A matching subunit 6013, configured to match each log feature according to the order of the detection priorities from high to low based on a preset detection data set;

An updating unit 602, configured to update the detection priority of each log feature based on the obtained matching result and a preset priority update policy, and take the next security log as the target security log of the next security detection process.

Optionally, the acquiring subunit 6011 is specifically configured to:

based on the support degree of each candidate feature, screening out log features with the support degree larger than a preset support degree threshold value from the candidate feature sets to obtain log feature sets; the support represents the ratio of the number of abnormal records of the corresponding candidate feature to the total number of abnormal records of the target security log.

Optionally, the determining subunit 6012 is specifically configured to:

Determining a front item feature from each log feature based on a preset confidence threshold; the confidence level of the forefront feature is greater than a confidence level threshold;

Optionally, the determining subunit 6012 is specifically configured to:

determining a maximum confidence coefficient and a minimum confidence coefficient from the confidence coefficient of each previous feature; determining a first weight parameter based on a difference between the maximum confidence and the minimum confidence;

Determining, for each of the antecedent features, a second weight parameter based on a difference between the confidence level of the antecedent feature and the minimum confidence level;

And obtaining the detection weight value of the previous feature based on the ratio between the second weight parameter and the first weight parameter.

Optionally, after determining the antecedent feature from the respective log features based on a preset confidence threshold, the determining subunit 6012 is further configured to:

determining postamble features in each log feature; the confidence of the postamble feature is less than a confidence threshold;

Through the device, when safety detection processing is carried out on each safety log to be detected, the log feature set of the target safety log is obtained, the detection priority of each log feature is determined through the abnormal association degree between each log feature and other log features in the log feature set, namely the confidence level of the log feature, so that the importance degree of each log feature in the safety detection processing process is identified, the detection sequence of different log features is flexibly adjusted, computing resources are effectively allocated, important detection is carried out on high-risk features preferentially, unnecessary detection on low-risk features is reduced, and the overall safety detection efficiency is improved. According to the sequence of the detection priority from high to low, each log feature is matched with a preset detection data set, when the log feature is successfully matched with the detection data set, namely the potential security threat of the log feature is detected, the detection priority of each log feature is updated through a preset priority updating strategy, so that the updated detection priority is used for carrying out the security detection processing on the next security log, the purpose of fully utilizing real-time log data information and a circulating feedback mechanism is achieved, and the next security detection processing is optimized according to the security detection result of each time, so that the security detection processing can adapt to the security threat environment of the latest data change and the height change, and the accuracy and the sharpness of the security detection are improved.

For convenience of description, the above parts are respectively described as being functionally divided into unit modules (or modules). Of course, the functions of each unit (or module) may be implemented in the same piece or pieces of software or hardware when implementing the present application. The apparatus may be used to perform the methods shown in the embodiments of the present application, and therefore, the description of the foregoing embodiments may be referred to for the functions that can be implemented by each functional module of the apparatus, and the like, which are not repeated.

Referring to fig. 7, based on the same technical concept, the embodiment of the application further provides a computer device. In one embodiment, the computer device may be, for example, the security detection device shown in FIG. 1 or the security detection system shown in FIG. 2. The computer device, as shown in fig. 7, may include a memory 701, a communication module 703, and one or more processors 702.

Memory 701 for storing a computer program for execution by processor 702. The memory 701 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system; the storage data area may store various sets of operation instructions, etc.

The memory 701 may be a volatile memory (RAM), such as a random-access memory (RAM); the memory 701 may also be a nonvolatile memory (non-volatile memory), such as a read-only memory, a flash memory (flash memory), a hard disk (HARD DISK DRIVE, HDD) or a solid state disk (solid-state disk) (STATE DRIVE, SSD); or memory 701 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. Memory 701 may be a combination of the above.

The processor 702 may include one or more central processing units (central processing unit, CPUs) or digital processing units, or the like. A processor 702 for implementing the above-described security detection method when calling a computer program stored in the memory 701.

The communication module 703 is used for communicating with a data transmitting device, a data receiving device or other network devices.

The specific connection medium between the memory 701, the communication module 703 and the processor 702 is not limited in the embodiment of the present application. The embodiment of the present application is shown in fig. 7, where the memory 701 and the processor 702 are connected by a bus 704, where the bus 704 is shown in bold in fig. 7, and the connection between other components is merely illustrative, and not limiting. The bus 704 may be divided into an address bus, a data bus, a control bus, and the like. For ease of description, only one thick line is depicted in fig. 7, but only one bus or one type of bus is not depicted.

The memory 701 stores a computer storage medium in which computer executable instructions for implementing the security detection method of the embodiment of the present application are stored. The processor 702 is configured to perform the security detection methods of the embodiments described above.

Based on the same inventive concept, embodiments of the present application also provide a storage medium having stored thereon a computer program which, when executed on a computer, causes a computer processor to perform the steps in the security detection method according to the various embodiments of the present application described above in the present specification.

In some possible embodiments, aspects of the security detection method provided by the present application may also be implemented in the form of a program product comprising program code for causing a computer device to carry out the steps of the security detection method according to the various exemplary embodiments of the application described herein above, when the program product is run on a computer device, e.g. the computer device may carry out the steps of the various embodiments.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The program product of embodiments of the present application may employ a portable compact disc read only memory (CD-ROM) and include program code and may run on a computing device. However, the program product of the present application is not limited thereto, and in the present application, the readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with a command execution system, apparatus, or device.

The readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with a command execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's equipment, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

It should be noted that although several units or sub-units of the apparatus are mentioned in the above detailed description, such a division is merely exemplary and not mandatory. Indeed, the features and functions of two or more of the elements described above may be embodied in one element in accordance with embodiments of the present application. Conversely, the features and functions of one unit described above may be further divided into a plurality of units to be embodied.

Furthermore, although the operations of the methods of the present application are depicted in the drawings in a particular order, this is not required or suggested that these operations must be performed in this particular order or that all of the illustrated operations must be performed in order to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims

1. A security detection method, the method comprising:

Acquiring a log feature set of a target security log;

2. The method of claim 1, wherein the obtaining the log feature set of the target security log comprises:

3. The method of claim 1, wherein determining the detection priority of each log feature based on the confidence level of the log feature comprises:

4. The method of claim 3, wherein determining the detection weight value for each of the antecedent features according to a preset weight distribution strategy based on the confidence level of the antecedent features comprises:

5. The method of claim 3, wherein after determining the antecedent features from the respective log features based on a preset confidence threshold, the method further comprises:

6. A security detection device, the device comprising:

The processing unit comprises an acquisition subunit, a determination subunit, a matching subunit and an updating subunit, wherein:

The updating subunit is configured to update the detection priority of each log feature based on the obtained matching result and a preset priority updating policy, and take the next security log as a target security log of the next security detection process.

7. The apparatus of claim 6, wherein the determination subunit is specifically configured to:

8. A computer device, comprising:

At least one processor, and

A memory coupled to the at least one processor;

Wherein the memory stores instructions executable by the at least one processor, the at least one processor performing the method of any of claims 1-5 by executing the instructions stored by the memory.

9. A computer storage medium, characterized in that the computer readable storage medium is for storing a computer program which, when run on a computer, causes the computer to perform the method according to any one of claims 1-5.

10. A computer program product comprising computer program instructions, characterized in that,

Which computer program instructions, when executed by a processor, carry out the steps of the method according to any one of claims 1-5.