CN111988341A - Data processing method, apparatus, computer system and storage medium - Google Patents

Abstract

The present disclosure provides a data processing method, including: acquiring security-related original data; extracting multiple entity objects from the original data; processing multiple entity objects by using a threat intelligence data set to obtain each of the multiple entity objects. tag features of each entity object, wherein the tag feature is used to represent the security attribute and/or malicious attribute of the entity object; and the threat information of each entity object is determined according to the tag feature of each entity object. The present disclosure provides a data processing apparatus, a computer system and a storage medium.

Description

Data processing method, apparatus, computer system and storage medium

technical field

The present disclosure relates to the technical field of network security, and more particularly, to a data processing method, apparatus, computer system and storage medium.

Background technique

With the rapid development of computer and Internet technology, network security has increasingly become the focus of attention. Attackers' attack methods are more and more diverse, and malicious information may exist in network data streams in various forms. Therefore, it is particularly important to determine and mark malicious information in network data.

At present, the information of malicious families and attack gangs is not fully grasped in the related art, and the judgment and marking of malicious information in network data need to rely on manual review, which results in low judgment efficiency and misjudgment.

SUMMARY OF THE INVENTION

In view of this, the present disclosure provides a data processing method, apparatus, computer system and storage medium.

One aspect of the present disclosure provides a data processing method, including: acquiring security-related raw data; extracting a plurality of entity objects from the raw data; processing the plurality of entity objects by using a threat intelligence data set, Obtaining the respective label features of each entity object in the plurality of entity objects, wherein the label features are used to characterize the security attributes and/or malicious properties of the entity objects; and according to the label features of each entity object , and determine the threat information of each entity object.

According to an embodiment of the present disclosure, the threat intelligence data set includes multiple knowledge bases; using the threat intelligence data set to process the multiple entity objects includes: for each entity object in the multiple entity objects, using At least one knowledge base in the plurality of knowledge bases processes the entity object to obtain a processing result of each knowledge base in the at least one knowledge base, wherein each of the knowledge bases includes multiple knowledge bases marked with label features. and determining the label feature of the entity object according to the processing result of each of the knowledge bases.

According to an embodiment of the present disclosure, using at least one knowledge base in a plurality of knowledge bases to process the entity object includes: for each knowledge base in the at least one knowledge base, determining whether the knowledge base includes a The entity object is the same target entity object; and if it is determined that the knowledge base includes the same target entity object as the entity object, marking the entity object with the tag feature of the target entity object.

According to an embodiment of the present disclosure, determining the threat information of each entity object according to the tag feature of each entity object includes: for each entity object, performing a network model on the tag feature of the entity object. processing to obtain the threat information of the entity object.

According to an embodiment of the present disclosure, the entity object includes a file; the method further includes: performing feature extraction on the file to obtain static features and dynamic features of the file; running the file with a sandbox to obtain the file behavior characteristics of the file; and determining threat information of the file according to at least one of the static characteristics, dynamic characteristics, behavior characteristics and tag characteristics of the file.

According to an embodiment of the present disclosure, the method further includes: integrating the entity objects processed by using the threat intelligence data set; and associating the integrated entity objects according to the label characteristics of the entity objects to obtain the entity objects relational datasets.

According to an embodiment of the present disclosure, the method further includes: acquiring a new entity object; processing the new entity object by using the entity-object relationship data set to obtain a label feature of the new entity object; and according to the new entity object The label feature of the entity object determines the threat information of the entity object.

According to an embodiment of the present disclosure, the entity object includes at least one of a file, a domain name, an IP, and a webpage address; the at least one knowledge base includes a whitelist database, a blacklist database, a domain name database for filing, a reputation file At least one of a host library and a reputation IP library; the threat information includes at least one of malicious type, attacker information and attack means.

Another aspect of the present disclosure provides a data processing apparatus, comprising: a first acquisition module for acquiring security-related original data; a first extraction module for extracting a plurality of entity objects from the original data; The first processing module is configured to process the plurality of entity objects by using the threat intelligence data set to obtain the respective label features of each entity object in the plurality of entity objects, wherein the label features are used to characterize the a security attribute and/or malicious attribute of an entity object; and a determining module, configured to determine threat information of each entity object according to the tag feature of each entity object.

Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which when executed, are used to implement the method as described above.

Another aspect of the present disclosure provides a computer program comprising computer-executable instructions, which when executed, are used to implement the method as described above.

Another aspect of the present disclosure provides a computer system including: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more programs When executed by a plurality of processors, the one or more processors are caused to implement the method as described above.

According to the embodiment of the present disclosure, multiple entity objects are extracted from the original data, and the multiple entity objects are processed by using the threat intelligence data set to obtain the respective label characteristics of each entity object. The tag feature of the object, the technical means to determine the threat information of each entity object. Since the threat intelligence data set is used to process the entity objects and label the malicious information of the entity objects, it at least partially overcomes the low judgment efficiency and low accuracy caused by relying on manual review for the judgment and marking of malicious information in the related art. Therefore, the technical effect of improving the efficiency and accuracy of malicious information determination and marking is achieved.

Description of drawings

The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure with reference to the accompanying drawings, in which:

FIG. 1 schematically shows an exemplary system architecture to which the data processing method of the embodiment of the present disclosure can be applied;

FIG. 2 schematically shows a flowchart of a data processing method according to an embodiment of the present disclosure;

3 schematically shows a flowchart of a method for processing multiple entity objects by using a threat intelligence data set according to an embodiment of the present disclosure;

FIG. 4 schematically shows a flowchart of a method for processing an entity object by using at least one knowledge base in a plurality of knowledge bases according to an embodiment of the present disclosure;

FIG. 5 schematically shows a flowchart of a data processing method according to another embodiment of the present disclosure;

FIG. 6 schematically shows a flowchart of a data processing method according to another embodiment of the present disclosure;

FIG. 7 schematically shows a flowchart of a data processing method according to another embodiment of the present disclosure;

FIG. 8 schematically shows a block diagram of a data processing apparatus according to an embodiment of the present disclosure; and

9 schematically illustrates a block diagram of a computer system according to an embodiment of the present disclosure.

Detailed ways

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood, however, that these descriptions are exemplary only, and are not intended to limit the scope of the present disclosure. In the following detailed description, for convenience of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It will be apparent, however, that one or more embodiments may be practiced without these specific details. Also, in the following description, descriptions of well-known structures and techniques are omitted to avoid unnecessarily obscuring the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the present disclosure. The terms "comprising", "comprising" and the like as used herein indicate the presence of stated features, steps, operations and/or components, but do not preclude the presence or addition of one or more other features, steps, operations or components.

All terms (including technical and scientific terms) used herein have the meaning as commonly understood by one of ordinary skill in the art, unless otherwise defined. It should be noted that terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly rigid manner.

Where expressions like "at least one of A, B, and C, etc.," are used, they should generally be interpreted in accordance with the meaning of the expression as commonly understood by those skilled in the art (eg, "has A, B, and C") At least one of the "systems" shall include, but not be limited to, systems with A alone, B alone, C alone, A and B, A and C, B and C, and/or A, B, C, etc. ). Where expressions like "at least one of A, B, or C, etc.," are used, they should generally be interpreted in accordance with the meaning of the expression as commonly understood by those skilled in the art (eg, "has A, B, or C, etc." At least one of the "systems" shall include, but not be limited to, systems with A alone, B alone, C alone, A and B, A and C, B and C, and/or A, B, C, etc. ).

APT (Advanced Persistent Threat, APT, Advanced Persistent Threat for short) is also called "targeted" attack, which is a new type of attack that is organized, has a specific target and has a very long duration. APT attacks continue, and malicious information emerges in an endless stream. Security vendors are constantly using various tools to detect network attacks in order to track and locate malicious information.

Security vendors generally set up an operation platform for network attack detection. This platform can run a variety of tools to detect network attacks. For example, use sandbox to detect network attacks, use web crawler to detect network attacks, and use sandbox to run logs, Web crawler logs, etc. to detect network attacks. However, all of this requires manual participation, such as manually setting sandbox operating conditions and manually writing crawler code for different situations. In addition, the test results need to be further analyzed and reviewed manually, which is inefficient and may cause misjudgment or omission.

Based on this, embodiments of the present disclosure provide a data processing method and apparatus. The method includes: acquiring security-related original data; extracting multiple entity objects from the original data; processing the multiple entity objects by using a threat intelligence data set to obtain the respective label features of each entity object in the multiple entity objects, The tag feature is used to represent the security attribute and/or malicious attribute of the entity object; and the threat information of each entity object is determined according to the tag feature of each entity object.

FIG. 1 schematically illustrates an exemplary system architecture 100 to which data processing methods of embodiments of the present disclosure may be applied. It should be noted that FIG. 1 is only an example of a system architecture to which the embodiments of the present disclosure can be applied, so as to help those skilled in the art to understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used for other A device, system, environment or scene.

As shown in FIG. 1 , the system architecture 100 according to this embodiment may include an electronic device 101 , a detection platform 102 based on a threat intelligence data set, and a manual operation marking platform 103 .

After obtaining the original security-related data, the electronic device 101 may first perform data extraction to obtain multiple IOC (Indicators of Compromise, IOC for short) entity objects. IOC entity objects can be of various types, such as file, IP, domain name, HOST, web page address, etc. Among them, the file can be represented by the MD5 (Message Digest Algorithm, message digest algorithm) of the file, and the web page address can include the main domain name HOST, URL (UniformResource Locator, Uniform Resource Locator), URI (Uniform Resource Identifier, Uniform Resource Identifier). Then the IOC data can be detected by the detection platform 102 and the manual operation marking platform 103 to detect malicious information. Specifically, the original data may be a network data stream, a file, a message, or the like.

The detection platform 102 may include a plurality of knowledge bases, and each knowledge base may be a database storing threat intelligence data of different business types, such as a whitelist database, a blacklist database, a compromised host database, a reputation file database, a reputation IP database, and a filing Domain name database, etc.

The threat intelligence data may be data obtained by processing, analyzing, judging and/or marking source data of multiple different dimensions. Among them, the source data of different dimensions can include open source intelligence data, business intelligence data (such as attack events released by various security vendors), operation data of security products (such as attack alarm information), and data output by the file depth analysis engine (such as file types). ), the judgment results of multi-virus engine scans, and manual operation data (such as crawler data, sandbox logs), etc.

Based on these source data, secure entity objects, such as IP, network address, domain name, file, etc., are extracted. After field comparison and extraction of entity objects, useful attribute information, such as security attributes, malicious attributes, and context attributes, is extracted. The attribute of the object determines whether the object is malicious information, so as to carry out the calibration of malicious information. After malicious determination and marking of the entity object, it can be stored in the database to form a threat intelligence data set.

The manual operation marking platform 103 runs a web crawler tool, a sandbox tool, etc., which can be used to assist the detection of the detection platform 102 , and can also adjust and correct the detection results of the detection platform 102 . The manual operation marking platform 103 mainly uses web crawler data, detection logs, network activity information obtained by sandbox operation, etc., to analyze malicious information and perform manual judgment and marking.

The IOC entity object is detected by the detection platform 102 and/or the manual operation labeling platform 103, and each detection link will be labeled according to the detection result, and the label can represent the security attribute and/or malicious attribute of the IOC object. For example, if a certain file is detected by the whitelist database, blacklist library and/or reputation file library in the detection platform 102, if the file is a malicious file, it can be marked with the file type, attack method, etc. The tag is used in a vulnerability attack. After the file is detected by the manual operation marking platform 103, if it is detected that the file is a malicious sample generated by a malicious family, the attacker can be marked as a tag of a known malicious family.

It should be noted that, the data processing method provided by the embodiments of the present disclosure may generally be executed by the electronic device 101 . Correspondingly, the data processing apparatus provided by the embodiments of the present disclosure may generally be provided in the electronic device 101 . The data processing method provided by the embodiment of the present disclosure may also be executed by a server or a server cluster that is different from the electronic device 101 and can communicate with the electronic device 101 . Correspondingly, the data processing apparatus provided by the embodiment of the present disclosure may also be provided in a server or a server cluster that is different from the electronic device 101 and can communicate with the electronic device 101 .

It should be understood that the numbers of the electronic device 101 , the detection platform 102 , the knowledge base in the detection platform 102 , the manual operation marking platform 103 and the detection tools in the manual operation marking platform 103 in FIG. 1 are only illustrative. According to implementation needs, there may be any number of electronic devices 101, detection platforms 102, knowledge bases, manual operation marking platforms 103, and detection tools.

FIG. 2 schematically shows a flowchart of a data processing method according to an embodiment of the present disclosure.

As shown in FIG. 2, the method includes operations S201-S204.

In operation S201, security-related raw data is acquired.

In operation S202, a plurality of entity objects are extracted from the original data.

According to an embodiment of the present disclosure, the original data may be, for example, a network data stream, a service data packet, or the like. The IOC entity object is extracted from the original data, and the IOC entity object can include MD5, IP, domain name, HOST, URL, etc. The IOC data can also be normalized, denoised, deduplicated, field filled, etc. to normalize the data.

In operation S203, multiple entity objects are processed by using the threat intelligence data set to obtain respective tag features of each entity object in the multiple entity objects, wherein the tag features are used to represent the security attribute and/or malicious attribute of the entity object.

According to an embodiment of the present disclosure, the threat intelligence data set includes multiple knowledge bases, such as a whitelist database, a blacklist database, a compromised host database, a reputation file database, a reputation IP database, a record domain name database, and the like.

Threat intelligence data is obtained by processing multi-dimensional source data, and its form can be a compound expression of some attributes. Attributes involved in threat intelligence data can include: observable behavior characteristics: such as network congestion, system damage, etc.; threat characteristic indicators: by looking at the characteristics, it can be determined whether the asset information is really attacked by this threat; security event description : Including malicious attack behavior, victim target, exploited weaknesses, impact and consequences, kill chain information, etc.; Attack intent description: Why the attack was launched, including attacker characteristics, intent, organization, etc.; Vulnerability characteristics description : Actions taken against the attacked system; traceability information: specific information of the attack initiator, including characteristics such as organization, national mailbox, and account number.

Among the above attribute features, the attribute representing the malicious feature or security feature of the entity object can be used as the label feature, and the feature representing the contextual relationship of the entity object can be used as the context attribute.

According to the embodiment of the present disclosure, the multi-dimensional source data may come from open source intelligence data, business intelligence data, operation data of security products, data output by a file in-depth parsing engine, determination results of multi-virus engine scanning, web crawler data, detection logs, sand It covers the source data of multiple links, and automatically obtains data in full-link scenarios to ensure the multi-dimensionality, breadth and depth of data sources.

According to the embodiment of the present disclosure, the threat intelligence data set is used to detect the IOC entity object, because the threat intelligence data set is obtained by processing multi-dimensional source data, compared with the traditional detection of the IOC entity object using the single source data, It ensures the breadth and depth of the data, thus ensuring the continuity and freshness of the source data, and improving the comprehensiveness and accuracy of the analysis and detection of IOC entity objects.

According to the embodiment of the present disclosure, at least one knowledge base in the threat intelligence data set is used to process each entity object. For example, the entity object can be compared with the entity object in the knowledge base. If the matching is successful, the knowledge base can be The tag property of the entity object that matches the current entity object is assigned to the current entity object.

According to the embodiment of the present disclosure, the context attribute of the entity object in the knowledge base that matches the current entity object can also be assigned to the current entity object, so as to enrich the attribute information of the current entity object.

In operation S204, threat information of each entity object is determined according to the tag feature of each entity object.

According to the embodiment of the present disclosure, each entity object will be marked with a plurality of tag attributes after being judged by at least one knowledge base and a manually operated tagging platform. For example, an IOC object is marked with tag 1, tag 2, and tag 3 after being processed by knowledge base 1, marked with tag 4, tag 5, and tag 6 after being processed by knowledge base 3, and marked with tag 7 after being processed by the manual operation marking platform. , label 8. The tags may include some attribute tags, and may also include malicious tags.

According to the embodiment of the present disclosure, malicious tags can be mainly concerned, and the most representative malicious tags can be selected to mark the entity object. For example, label 1, label 4, and label 7 can be selected to mark the entity object, and threat information such as attacker information, attack method, and malicious type can be determined according to these malicious labels. Vulnerability information exploited by attackers, attack tools used by attackers, etc.

According to an embodiment of the present disclosure, operation S204 may include: for each entity object, using a network model to process the label feature of the entity object to obtain threat information of the entity object. The network model may be a classification model trained based on a machine learning clustering algorithm, the input of the classification model may be the label feature of the entity object, and the output may be threat information such as the attack group to which the label feature belongs, the attack method, and the malicious type.

According to the embodiment of the present disclosure, after clustering the tags returned by different knowledge bases through a machine learning clustering algorithm, the attribution information and family gang information of the data tags can be automatically determined.

Among them, the machine learning clustering algorithm can be the random forest algorithm or the IBk (k nearest neighbor classification) algorithm is introduced on the basis of the random forest algorithm, which can make up for the weakness of the random forest falling into the local optimal solution resulting in false positives. The training set used by the forest algorithm is trained to generate an IBk model that supports an unknown sample to find the three most similar samples from the training set. By combining random forest algorithm and IBk algorithm, high-precision and high-accuracy APT samples and malicious family samples can be detected. Other machine learning classification and clustering algorithms may also be used, and the present disclosure does not limit the types of machine learning algorithms.

According to the embodiment of the present disclosure, multiple entity objects are extracted from the original data, and the threat intelligence data set is used to process the multiple entity objects to obtain the respective label features of each entity object. Threat information for an entity object. Since the threat intelligence data set is used to process the entity objects and label the malicious information of the entity objects, the efficiency and accuracy of the determination and labeling of malicious information can be improved.

Further, according to the label characteristics of the entity object, it can quickly locate the threat information such as malicious information such as attack group information, malicious family information, and attack methods. For attacks on organizations such as APT, fast tracking and accurate positioning can be achieved.

FIG. 3 schematically shows a flowchart of a method for processing a plurality of entity objects by using a threat intelligence data set according to an embodiment of the present disclosure.

As shown in FIG. 3 , operation S203 includes operations S301 to S302.

In operation S301, the entity object is processed by using at least one knowledge base in the plurality of knowledge bases, and a processing result of each knowledge base in the at least one knowledge base is obtained, wherein each knowledge base includes a plurality of entity object.

According to the embodiments of the present disclosure, different research and judgment processes need to be performed for different IOC objects. For example, if the IOC object is a file, a whitelist library, a reputation file library, and a manual operation research and judgment process are required. If the IOC object is an IP, it is necessary to conduct research and judgment on the whitelist database, IP reputation database, and manual operation marking process. If the IOC object is a domain name, a whitelist database, a compromised host database, a record domain name database, and a manual operation research and judgment process are required. If the IOC object is a URL, a whitelist library and a manual operation research and judgment process are required.

According to the embodiment of the present disclosure, for each type of IOC, the research and judgment results in different knowledge bases may be the same or different. After at least one knowledge base is researched and judged, the manual operation marking platform can be used to make adjustments to improve the marking accuracy. .

For example, for a URL object, after the detection by the whitelist library, no malicious label is marked, but after the detection by the manual operation labeling platform, the malicious download label is marked, then it is determined that the URL is malicious.

In operation S302, the label feature of the entity object is determined according to the processing result of each knowledge base.

According to the embodiments of the present disclosure, for each IOC, each knowledge base can be marked with rich tag features, and the same or different tags can be obtained through different knowledge base processing, the same tags can be deduplicated, and different The tags are integrated to obtain the complete tag features of the entity object.

FIG. 4 schematically shows a flowchart of a method for processing an entity object by using at least one knowledge base among a plurality of knowledge bases according to an embodiment of the present disclosure.

As shown in FIG. 4 , operation S301 includes operations S401 to S402.

In operation S401, for each knowledge base in the at least one knowledge base, it is determined whether the knowledge base includes the same target entity object as the entity object.

In operation S402, if it is determined that the knowledge base includes the same target entity object as the entity object, the entity object is marked with the tag feature of the target entity object.

For example, for a domain name, compare it with the host domain name in the compromised host database, if the domain name exists in the compromised host database, mark the label attribute of the domain name in the compromised host database to the current domain name.

According to an embodiment of the present disclosure, operation S301 further includes: for each knowledge base in the at least one knowledge base, performing information completion on the contextual feature of the current entity object according to the contextual feature of the entity object in the knowledge base, so as to generate an enriched mark The label information ensures the stability and accuracy of label information output.

FIG. 5 schematically shows a flowchart of a data processing method according to another embodiment of the present disclosure.

As shown in FIG. 5 , operations S501 to S503 are included.

In operation S501, feature extraction is performed on the file to obtain static features and dynamic features of the file.

In operation S502, the file is run using the sandbox to obtain behavior characteristics of the file.

According to the embodiment of the present disclosure, the IOC object of the file type can also be processed by using a deep file parsing engine and sandbox operation.

Specifically, through the deep file parsing engine, static analysis and dynamic analysis of the file can be performed to obtain the static and dynamic features of the file. Static features can include file name, file size, time of first occurrence of detection, etc. Dynamic features can be some information about file operation and information mapped to it, including maximum and minimum streams, stream types, compiler types, PDB (Program Database File) length, string length, number of dictionary elements, and array size , associated file information, etc.

Specifically, by running the file in the sandbox, features such as the submission area, sandbox network information, and release file information can be obtained.

In operation S503, threat information of the file is determined according to at least one of a static feature, a dynamic feature, a behavior feature, and a tag feature of the file.

According to the embodiments of the present disclosure, comprehensively determine the static features and dynamic features obtained based on the deep file parsing engine, the behavior features obtained based on the sandbox operation, and the label features obtained based on the threat intelligence data set. All the context information required for file calibration can be obtained, and after multi-dimensional statistical association, the final label attributes and context attributes can be obtained.

FIG. 6 schematically shows a flowchart of a data processing method according to another embodiment of the present disclosure.

As shown in FIG. 6 , operations S601 to S602 are included.

In operation S601, the entity objects processed using the threat intelligence data set are integrated.

In operation S602, the integrated entity objects are associated according to the label features of the entity objects to obtain an entity-object relationship data set.

According to the embodiment of the present disclosure, after the IOC data is detected and tagged by the detection platform 102 and/or the manual operation tagging platform 103, the IOC data can be integrated. Integrate the same IOC data together. The associated IOCs can also be aggregated according to the tag characteristics, for example, the IOC data belonging to the same malicious family are associated together, for example, an IP can be associated with a file or URL, etc., and finally a new associated IOC dataset is formed.

FIG. 7 schematically shows a flowchart of a data processing method according to another embodiment of the present disclosure.

As shown in FIG. 7 , operations S701 to S703 are included.

In operation S701, a new entity object is acquired.

In operation S702, the new entity object is processed by using the entity object relationship data set to obtain the label feature of the new entity object.

In operation S703, the threat information of the entity object is determined according to the tag feature of the new entity object.

According to the embodiment of the present disclosure, a unified API interface can be provided by using the IOC data set with an association relationship, and different service analysis systems can be associated, and different query processing services can be provided. In order to obtain a new IOC entity object, you can query and obtain threat information such as the malicious type, attack method, and malicious family of the IOC entity object through the API interface.

FIG. 8 schematically shows a block diagram of a data processing apparatus according to an embodiment of the present disclosure.

As shown in FIG. 8 , the data processing apparatus 800 includes a first acquisition module 801 , a first extraction module 802 , a first processing module 803 , and a determination module 804 .

The first acquisition module 801 is used to acquire security-related raw data.

The first extraction module 802 is used to extract a plurality of entity objects from the original data.

The first processing module 803 is configured to process the multiple entity objects by using the threat intelligence data set to obtain the respective tag features of each entity object in the multiple entity objects, wherein the tag features are used to represent the security attributes and/or the entity objects. malicious properties.

The first determining module 804 is configured to determine threat information of each entity object according to the tag feature of each entity object.

According to an embodiment of the present disclosure, the first processing module 803 includes a processing unit and a determination unit.

The processing unit is configured to process the entity object by using at least one knowledge base in the plurality of knowledge bases, and obtain the processing result of each knowledge base in the at least one knowledge base, wherein each knowledge base includes a plurality of entity object.

The determining unit is used for determining the label feature of the entity object according to the processing result of each knowledge base.

According to an embodiment of the present disclosure, the processing unit includes a determination subunit and a marking subunit.

The determining subunit is used for determining, for each knowledge base in the at least one knowledge base, whether the knowledge base includes the same target entity object as the entity object.

The marking subunit is used to mark the entity object with the tag feature of the target entity object if it is determined that the knowledge base includes the same target entity object as the entity object.

According to the embodiment of the present disclosure, the first determination module 804 is configured to process, for each entity object, the tag feature of the entity object by using the network model, and obtain threat information of the entity object.

According to an embodiment of the present disclosure, the data processing apparatus 800 further includes a second extraction module, a second processing module, and a second determination module.

The second extraction module is used to perform feature extraction on the file to obtain static features and dynamic features of the file.

The second processing module is used to run the file by using the sandbox to obtain the behavior characteristics of the file.

The second determining module is configured to determine threat information of the file according to at least one of the static feature, dynamic feature, behavior feature and tag feature of the file.

According to an embodiment of the present disclosure, the data processing apparatus 800 further includes an integration module and an association module.

The integration module is used to integrate the entity objects processed by the threat intelligence data set.

The association module is used to associate the integrated entity objects according to the label characteristics of the entity objects to obtain an entity object relationship data set.

According to an embodiment of the present disclosure, the data processing apparatus 800 further includes a second acquisition module, a third processing module, and a third determination module.

The second acquisition module is used to acquire the new entity object.

The third processing module is used to process the new entity object by using the entity-object relationship data set to obtain the label feature of the new entity object.

The third determining module is used for determining the threat information of the entity object according to the label feature of the new entity object.

According to an embodiment of the present disclosure, the entity object includes at least one of a file, a domain name, an IP, and a web page address. The at least one knowledge base includes at least one of a whitelist base, a blacklist base, a registered domain name base, a reputation file base, a compromised host base, and a reputation IP base. The threat information includes at least one of malicious type, attacker information, and attack means.

Any of the modules, sub-modules, units, sub-units, or at least part of the functions of any of them according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be divided into multiple modules for implementation. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented at least in part as hardware circuits, such as field programmable gate arrays (FPGA), programmable logic arrays (PLA), A system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or any other reasonable means of hardware or firmware that integrates or packages circuits, or can be implemented in software, hardware, and firmware Any one of these implementations or an appropriate combination of any of them is implemented. Alternatively, one or more of the modules, sub-modules, units, and sub-units according to embodiments of the present disclosure may be implemented at least in part as computer program modules that, when executed, may perform corresponding functions.

For example, any one of the first acquisition module 801, the first extraction module 802, the first processing module 803, and the determination module 804 may be combined in one module/unit/subunit, or any one of the modules/units. /subunits can be split into multiple modules/units/subunits. Alternatively, at least part of the functionality of one or more of these modules/units/subunits may be combined with at least part of the functionality of other modules/units/subunits and combined in one module/unit/subunit realized in. According to an embodiment of the present disclosure, at least one of the first acquisition module 801, the first extraction module 802, the first processing module 803, and the determination module 804 may be at least partially implemented as a hardware circuit, such as a field programmable gate array ( FPGA), Programmable Logic Array (PLA), System-on-Chip, System-on-Substrate, System-on-Package, Application-Specific Integrated Circuit (ASIC), or any other reasonable means by which circuits can be integrated or packaged such as hardware or firmware It can be realized by any one of the three implementation manners of software, hardware and firmware, or by any suitable combination of any of them. Alternatively, at least one of the first acquisition module 801, the first extraction module 802, the first processing module 803, and the determination module 804 may be at least partially implemented as a computer program module that, when executed, may execute corresponding function.

It should be noted that the part of the data processing apparatus in the embodiment of the present disclosure corresponds to the part of the data processing method in the embodiment of the present disclosure, and the description of the part of the data processing apparatus refers to the part of the data processing method, which is not repeated here.

Figure 9 schematically illustrates a block diagram of a computer system suitable for implementing the methods described above, according to an embodiment of the present disclosure. The computer system shown in FIG. 9 is only an example, and should not impose any limitation on the function and scope of use of the embodiments of the present disclosure.

As shown in FIG. 9 , a computer system 900 according to an embodiment of the present disclosure includes a processor 901 that can be loaded into a random access memory (RAM) 903 according to a program stored in a read only memory (ROM) 902 or from a storage section 908 program to perform various appropriate actions and processes. The processor 901 may include, for example, a general-purpose microprocessor (eg, a CPU), an instruction set processor and/or a related chipset, and/or a special-purpose microprocessor (eg, an application-specific integrated circuit (ASIC)), among others. The processor 901 may also include on-board memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiments of the present disclosure.

In the RAM 903, various programs and data necessary for the operation of the system 900 are stored. The processor 901 , the ROM 902 and the RAM 903 are connected to each other through a bus 904 . The processor 901 performs various operations of the method flow according to the embodiment of the present disclosure by executing the programs in the ROM 902 and/or the RAM 903 . Note that the program may also be stored in one or more memories other than the ROM 902 and the RAM 903 . The processor 901 may also perform various operations of the method flow according to the embodiments of the present disclosure by executing programs stored in the one or more memories.

According to embodiments of the present disclosure, the system 900 may also include an input/output (I/O) interface 905 that is also connected to the bus 904 . System 900 may also include one or more of the following components connected to I/O interface 905: input portion 906 including keyboard, mouse, etc.; including components such as cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers An output section 907 including a hard disk, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, and the like. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I/O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 910 as needed so that a computer program read therefrom is installed into the storage section 908 as needed.

According to the embodiments of the present disclosure, the method flow according to the embodiments of the present disclosure may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network via the communication portion 909, and/or installed from the removable medium 911. When the computer program is executed by the processor 901, the above-described functions defined in the system of the embodiment of the present disclosure are performed. According to embodiments of the present disclosure, the above-described systems, apparatuses, apparatuses, modules, units, etc. may be implemented by computer program modules.

The present disclosure also provides a computer-readable storage medium. The computer-readable storage medium may be included in the device/apparatus/system described in the above embodiments; it may also exist alone without being assembled into the device/system. device/system. The above-mentioned computer-readable storage medium carries one or more programs, and when the above-mentioned one or more programs are executed, implement the method according to the embodiment of the present disclosure.

According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples may include, but are not limited to, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), portable compact disk read only memory (CD- ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

For example, according to embodiments of the present disclosure, a computer-readable storage medium may include one or more memories other than ROM 902 and/or RAM 903 and/or ROM 902 and RAM 903 described above.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams or flowchart illustrations, and combinations of blocks in the block diagrams or flowchart illustrations, can be implemented in special purpose hardware-based systems that perform the specified functions or operations, or can be implemented using A combination of dedicated hardware and computer instructions is implemented. Those skilled in the art will appreciate that various combinations and/or combinations of features recited in various embodiments and/or claims of the present disclosure are possible, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments of the present disclosure and/or in the claims may be made without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of this disclosure.

Embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only, and are not intended to limit the scope of the present disclosure. Although the various embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and these substitutions and modifications should all fall within the scope of the present disclosure.

Claims (11)

1. A method of data processing, comprising:

acquiring original data related to safety;

extracting a plurality of entity objects from the raw data;

processing the entity objects by using a threat intelligence data set to obtain respective label characteristics of each entity object in the entity objects, wherein the label characteristics are used for representing the security attributes and/or malicious attributes of the entity objects; and

and determining the threat information of each entity object according to the label characteristics of each entity object.

2. The method of claim 1, wherein the threat intelligence data set comprises a plurality of knowledge bases;

processing the plurality of physical objects with a threat intelligence dataset comprises: for each of the plurality of entity objects,

processing the entity object by using at least one knowledge base in a plurality of knowledge bases to obtain a processing result of each knowledge base in the at least one knowledge base, wherein each knowledge base comprises a plurality of entity objects marked with tag characteristics; and

and determining the label characteristics of the entity object according to the processing result of each knowledge base.

3. The method of claim 2, wherein processing the entity object with at least one of a plurality of repositories includes: for each of the at least one knowledge base,

determining whether a target entity object identical to the entity object is included in the knowledge base; and

and if the knowledge base comprises the target entity object which is the same as the entity object, marking the entity object by using the label characteristic of the target entity object.

4. The method of claim 1, wherein determining threat information for the each physical object based on the tag characteristics of the each physical object comprises: for each of the entity objects, a function is performed,

and processing the label characteristics of the entity object by using a network model to obtain threat information of the entity object.

5. The method of claim 1 or 2, wherein the entity object comprises a file;

the method further comprises the following steps:

extracting the characteristics of the file to obtain the static characteristics and the dynamic characteristics of the file;

running the file by using a sandbox to obtain the behavior characteristics of the file; and

and determining threat information of the file according to at least one of the static characteristic, the dynamic characteristic, the behavior characteristic and the label characteristic of the file.

6. The method of claim 1, further comprising:

integrating the entity objects processed by the threat intelligence data set; and

and associating the integrated entity objects according to the label characteristics of the entity objects to obtain an entity object relation data set.

7. The method of claim 6, further comprising:

acquiring a new entity object;

processing the new entity object by using the entity object relation data set to obtain the label characteristic of the new entity object; and

and determining the threat information of the entity object according to the label characteristics of the new entity object.

8. The method of any one of claims 1-7, wherein:

the entity object comprises at least one of a file, a domain name, an IP and a webpage address;

the at least one knowledge base comprises at least one of a white name list base, a black name list base, a recorded domain name base, a credit file base, a lost host base and a credit IP base;

the threat information includes at least one of a malicious type, attacker information, and means of attack.

9. A data processing apparatus comprising:

the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring original data related to safety;

a first extraction module, configured to extract a plurality of entity objects from the raw data;

the first processing module is used for processing the entity objects by utilizing a threat intelligence data set to obtain respective label characteristics of each entity object in the entity objects, wherein the label characteristics are used for representing the security attributes and/or malicious attributes of the entity objects; and

and the determining module is used for determining the threat information of each entity object according to the label characteristics of each entity object.

10. A computer system, comprising:

one or more processors;

a memory for storing one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-8.

11. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to carry out the method of any one of claims 1 to 8.

Images