CN109688091B - Multi-source threat intelligence quality evaluation method and device - Google Patents
Multi-source threat intelligence quality evaluation method and device Download PDFInfo
- Publication number
- CN109688091B CN109688091B CN201810379261.7A CN201810379261A CN109688091B CN 109688091 B CN109688091 B CN 109688091B CN 201810379261 A CN201810379261 A CN 201810379261A CN 109688091 B CN109688091 B CN 109688091B
- Authority
- CN
- China
- Prior art keywords
- intelligence
- information
- threat
- source
- sources
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/30—Network architectures or network communication protocols for network security for supporting lawful interception, monitoring or retaining of communications or communication related information
- H04L63/302—Network architectures or network communication protocols for network security for supporting lawful interception, monitoring or retaining of communications or communication related information gathering intelligence information for situation awareness or reconnaissance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/55—Detecting local intrusion or implementing counter-measures
Abstract
The invention discloses a method and a device for evaluating the quality of multisource threat intelligence, wherein the method comprises the following steps: step 1, threat intelligence from multiple sources to be evaluated is obtained; step 2, normalizing a plurality of threat intelligence corresponding to different intelligence sources; and 3, respectively carrying out quality evaluation on the threat intelligence based on a plurality of preset characteristics, and generating an intelligence source sequence according to an evaluation result. The method can accurately evaluate different information sources, on one hand, help enterprises to make better investment decisions and purchase and use higher-quality information sources; on the other hand, the evaluation result can help better realize the standardized processing of the intelligence, for example, for the intelligence source with lower evaluation accuracy, the credibility score can be adjusted for the threat intelligence sent by the intelligence source.
Description
Technical Field
The invention relates to the technical field of internet security, in particular to a multi-source threat intelligence quality assessment method and device.
Background
With the high-speed development of the internet, particularly the mobile internet, the network environment is more complex, different attack behaviors are more industrialized and ganged, and the intrusion methods are more diversified and complicated, so that the traditional security solution is continuously challenged. Threat information obtained based on big data correlation analysis can promote enterprises and organizations to quickly know internal threat information, so that the enterprises are helped to make safety precaution in advance, attack detection and response are carried out more quickly, and after-the-fact attack tracing is carried out more efficiently. Under the background, threat intelligence is getting more and more attention of enterprises, and more enterprises add threat intelligence into the security protection network of the enterprise by self-production or purchase.
In 2013, 5, 16, Gartner has defined threat intelligence, which is evidence-based knowledge including context, mechanism, indicators, implications, and practical suggestions. Threat intelligence describes an existing, or imminent, threat or danger to an asset and may be used to notify a subject to take some response to the relevant threat or danger. The IOC (indicator of compliance) is a technical indicator defined by MANDIANT in long-term digital forensic practice that may reflect host or network behavior. The scheme provides a comprehensive evaluation scheme of multi-source information quality so as to help enterprises select threat information products suitable for the enterprises.
The main method for selecting threat information products in the industry is to test the information quality of each manufacturer through a threat sample, and the method comprises the following steps: 1. the number of intelligence products; 2. number of hits for intelligence products; 3. false reporting of intelligence products.
However, the conventional method has the following disadvantages: 1. through past samples, the timeliness of the information product cannot be judged, and only threat information has the value in time. 2. The value of the information product cannot be fully reflected by the quantity of the information product, and the quality of missed information cannot be judged. 3. The information richness is not evaluated, and the index can evaluate the comprehensive strength of information manufacturers. 4. The difference between the information sources is not evaluated, and the timeliness and difference performance of the information reflect whether the information manufacturer obtains the information through the open source information or not and the information produced by the manufacturer.
Disclosure of Invention
In view of the above mentioned situation mentioned in the background, the embodiments of the present invention provide a method and an apparatus for quality assessment of multi-source threat intelligence, the method includes: step 1, threat intelligence from multiple sources to be evaluated is obtained;
step 2, normalizing a plurality of threat intelligence corresponding to different intelligence sources;
and 3, respectively carrying out quality evaluation on the threat intelligence based on a plurality of preset characteristics, and generating an intelligence source sequence according to an evaluation result.
Preferably, the preset features include source ratio information, hit ratio information, richness information, difference information, and/or timeliness information of the threat intelligence.
Preferably, the method further comprises:
and counting the ratio of the threat situation report number recorded by each information source to the total recorded number in the past period of time to obtain the source ratio information.
Preferably, the method further comprises:
counting the hit threat intelligence in a past period of time, and analyzing the source information of the threat intelligence;
if the same threat intelligence is recorded by a plurality of intelligence sources, the plurality of intelligence sources are all recorded;
and counting the proportion of the number of hits of each intelligence source in all the numbers of hits according to the recorded information so as to obtain the hit information.
Preferably, the method further comprises: acquiring the threat intelligence appearing in a past period of time and acquiring source information of the threat intelligence;
if the same threat information is recorded by N information sources, adding N to a corresponding richness numerical value of the information source, if the threat information also has at least one historical threat information, adding 2 to the corresponding richness numerical value, and if M current effective threat information labels exist, adding 4 to the corresponding richness numerical value of the threat information;
and counting the information source distribution of which the richness value exceeds a preset value in a period of time so as to generate the richness information.
Preferably, the method further comprises:
counting the threat intelligence from a single intelligence source over a period of time;
analyzing source information of the threat intelligence;
if the same threat intelligence is input by a plurality of intelligence sources, the counting value corresponding to the intelligence source is updated, so that the quantity of the intelligence input by the intelligence source independently and the quantity of the intelligence input by the intelligence source not independently are obtained, and the ratio information of the two is generated to obtain the difference information.
Preferably, the method further comprises: counting the threat intelligence recorded in a past period of time;
acquiring source information of the threat intelligence;
if the same threat intelligence is recorded by a plurality of intelligence sources, only the first recorded intelligence source is confirmed according to the recording time information, and the proportion of all the intelligence sources is counted to obtain the timeliness information.
The embodiment of the invention also provides a multi-source threat intelligence quality evaluation device, which comprises: the device comprises an acquisition module, a normalization module and a processing module;
the acquisition module is configured to acquire threat intelligence from multiple sources to be evaluated;
the normalization module is configured to normalize the threat intelligence corresponding to different intelligence sources;
the processing module is configured to perform quality evaluation on the threat intelligence respectively based on a plurality of preset characteristics, and generate an intelligence source sequence according to an evaluation result.
Preferably, the preset features include source proportion information, hit information, richness information, difference information and/or timeliness information of the threat intelligence.
The invention can accurately evaluate different information sources and has important value. On one hand, the system helps enterprises to make better investment decisions, and purchase and use higher-quality information sources; on the other hand, the evaluation result can help better realize the standardized processing of the intelligence, for example, for the intelligence source with lower evaluation accuracy, the credibility score can be adjusted for the threat intelligence sent by the intelligence source.
Drawings
FIG. 1 is a flow chart of a method for quality assessment of multi-sourced threat intelligence in an embodiment of the invention.
Fig. 2 is a flow chart of an embodiment of a quality assessment method of an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
The threat information data is used as a non-entity object of a special industry, the data quality is not convenient to evaluate in practical production application, and a multi-angle systematic effective evaluation mode is summarized according to the accumulation of years in the field of threat information. As shown in fig. 1, the present embodiment performs the following steps:
threat intelligence from multiple sources to be evaluated is obtained;
normalizing a plurality of threat intelligence corresponding to different intelligence sources;
and respectively carrying out quality evaluation on the threat intelligence based on a plurality of preset characteristics, and generating an intelligence source sequence according to an evaluation result.
Threat information in this embodiment can be stored in the high in the clouds, when acquireing threat information, can regularly follow the high in the clouds and acquire the multisource threat information that needs the aassessment to convenient data processing carries out.
The preset features in this embodiment at least include source ratio information, hit ratio information, richness information, difference information, and/or timeliness information of threat intelligence. Namely, the comprehensive quality evaluation is carried out on the threat information from five angles of source proportion information, hit proportion information, richness information, difference information, timeliness information and the like.
The source proportion information is used for evaluating the proportion of the push threat information of each information source in the total threat information quantity, the index is used for measuring the delivery number of the information sources, and the index can be used for evaluating the rare delivery information number of some information sources and low cost performance; the hit proportion information is used for evaluating the proportion of information which is hit by a user query and distributed in each information source, the index is used for measuring the applicability of the information sources, and the index can be used for evaluating the condition that certain information sources are only used for pursuing quantity advantages, but a large amount of invalid information exists; the timeliness information comprises the proportion of first-sent information of each information source, the index can measure the output efficiency of the information sources, and the index can be used for checking the timeliness of information delivery of the information sources; the richness information is used for evaluating the proportion of threat intelligence rich in auxiliary information in each intelligence source, and the index is used for measuring the richness of the intelligence sources. The auxiliary information can be technical information such as WHOIS information, international public opinion certificate information, related sample information, rdns/pdns/port/service/host name/SSL digital certificate/executable file digital signature and the like, professional analysis reports and other current information and continuous historical information support, and the information can be used as important basis for subsequent traceability expansion, review, research and judgment tracking. The index can be used for examining the comprehensive strength of a production mechanism of an information source.
The difference information is used for evaluating the mutual information overlapping proportion of the information sources, and the index is used for measuring the independence of the output capacity of the information sources. In practical application, the multi-source threat information should have reasonable difference so as to be convenient for mutual defect detection and leakage repair, and highly overlapped information sources have no credibility superposition effect and generally have poor quality. The index can be used for checking the overlapping rate between information sources and the technical characteristics and the regionality of production organizations of the information sources, and effectively evaluating plagiarism imitation behaviors.
In the following, the evaluation purpose and the algorithm adopted based on the above five preset features are described in detail:
(1) the intelligence source proportion algorithm:
in order to realize statistics of the information delivery ratio of each threat information source (the higher the ratio is, the better the ratio is), the ratio of the information input by each information source to the total input number in the past period is counted.
(2) The intelligence hit statistical algorithm:
in order to realize the source distribution of hit threat intelligence in a past period of time (the higher the hit rate is, the better), the threat intelligence in the past period of time is counted, the source information of the threat intelligence is taken, if a threat intelligence is input by a plurality of sources, the plurality of intelligence sources can be recorded, then the proportion of the number of hits of each intelligence source to the total number of hits is counted, and due to the multi-source input condition, the data can be equal to or more than the number of hits in numerical value, but the evaluation effect is not influenced.
(3) Timeliness of information:
in order to realize the purpose of counting the source distribution (the higher the timeliness is, the better the source distribution) of the threat intelligence recorded by the first hand in the past period of time, the threat intelligence recorded in the past period of time is counted to obtain the source information of the threat intelligence, if one threat intelligence is recorded by a plurality of sources, only the first recorded intelligence source is taken according to the recording time, and then the proportion of each intelligence source in all sources is counted.
(4) Intelligence difference algorithm:
in order to realize the source distribution (the higher the difference is better) of the independent threat intelligence in the past period of time, the threat intelligence of a single intelligence source in the past period of time is counted, the source information of the threat intelligence is inquired one by one, if the threat intelligence is input by a plurality of intelligence sources, the counting is increased by one, the quantity of the intelligence input by the independent source and the quantity of the independent intelligence are obtained, and the quantity is finally displayed in a proportional mode.
(5) Intelligence richness algorithm:
in order to realize statistics of auxiliary information richness (the richness is higher and better) of information in a past period of time, threat information of a certain information source in the past period of time is obtained, source information of the threat information is obtained, if a threat information is input by N sources, the richness of the information source is added with N (the richness is defaulted to be a numerical value 1), and if the threat information also has a plurality of historical threat information (not according to the number integral, only according to the existence or not of the integral), the richness is added with 2; if there are M current valid threat intelligence labels, then richness is added with 4M, and finally, the distribution of the threat intelligence sources of which the richness exceeds a certain preset value in the past period is counted.
As shown in fig. 2, the multi-source threat information to be evaluated is updated from the cloud periodically, normalization processing is performed on the format of the threat information, and then the information source sequence is generated finally through each algorithm model.
The invention can accurately evaluate different information sources and has important value. On one hand, the system helps enterprises to make better investment decisions, and purchase and use higher-quality information sources; on the other hand, the evaluation result can help better realize the standardized processing of threat intelligence, for example, for an intelligence source with lower evaluation accuracy, the credibility score can be adjusted for the intelligence on the intelligence source as a whole.
The above embodiments are only exemplary embodiments of the present invention, and are not intended to limit the present invention, and the scope of the present invention is defined by the claims. Various modifications and equivalents may be made by those skilled in the art within the spirit and scope of the present invention, and such modifications and equivalents should also be considered as falling within the scope of the present invention.
Claims (6)
1. A multi-source threat intelligence quality assessment method is characterized by comprising the following steps:
step 1, threat intelligence from multiple sources to be evaluated is obtained;
step 2, normalizing a plurality of threat intelligence corresponding to different intelligence sources;
step 3, respectively carrying out quality evaluation on the threat intelligence based on a plurality of preset characteristics, and generating an intelligence source sequence according to an evaluation result;
the preset characteristics comprise source proportion information of the threat intelligence, wherein the source proportion information is used for evaluating the proportion of all threat intelligence quantity of push threat intelligence of each intelligence source, hit proportion information is used for evaluating the proportion of intelligence which is hit by a user query and distributed in each intelligence source, richness information is used for evaluating the proportion of threat intelligence which is rich in auxiliary information in each intelligence source, difference information is used for evaluating the overlapping proportion of intelligence among the intelligence sources, and/or timeliness information comprises the proportion of first-sent intelligence of each intelligence source;
wherein: the method for acquiring the difference information comprises the following steps:
counting the threat intelligence from a single intelligence source over a period of time;
analyzing source information of the threat intelligence;
if the same threat intelligence is input by a plurality of intelligence sources, the counting value corresponding to the intelligence source is updated, so that the quantity of the intelligence input by the intelligence source independently and the quantity of the intelligence input by the intelligence source not independently are obtained, and the ratio information of the two is generated to obtain the difference information.
2. The method of claim 1, further comprising:
and counting the ratio of the threat situation report number recorded by each information source to the total recorded number in the past period of time to obtain the source ratio information.
3. The method of claim 1, further comprising:
counting the hit threat intelligence in a past period of time, and analyzing the source information of the threat intelligence;
if the same threat intelligence is recorded by a plurality of intelligence sources, the plurality of intelligence sources are all recorded;
and counting the proportion of the number of hits of each intelligence source in all the numbers of hits according to the recorded information to obtain the hit proportion information.
4. The method of claim 1, further comprising: acquiring the threat intelligence appearing in a past period of time and acquiring source information of the threat intelligence;
if the same threat information is recorded by N information sources, adding N to a corresponding richness numerical value of the information source, if the threat information also has at least one historical threat information, adding 2 to the corresponding richness numerical value, and if M current effective threat information labels exist, adding 4 to the corresponding richness numerical value of the threat information;
and counting the information source distribution of which the richness value exceeds a preset value in a period of time so as to generate the richness information.
5. The method of claim 1, further comprising: counting the threat intelligence recorded in a past period of time;
acquiring source information of the threat intelligence;
if the same threat intelligence is recorded by a plurality of intelligence sources, only the first recorded intelligence source is confirmed according to the recording time information, and the proportion of all the intelligence sources is counted to obtain the timeliness information.
6. A multisource threat intelligence quality assessment apparatus based on the method of claim 1, characterized in that the apparatus comprises: the device comprises an acquisition module, a normalization module and a processing module;
the acquisition module is configured to acquire threat intelligence from multiple sources to be evaluated;
the normalization module is configured to normalize the threat intelligence corresponding to different intelligence sources;
the processing module is configured to perform quality evaluation on the threat intelligence respectively based on the preset characteristics, and generate an intelligence source sequence according to an evaluation result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810379261.7A CN109688091B (en) | 2018-04-25 | 2018-04-25 | Multi-source threat intelligence quality evaluation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810379261.7A CN109688091B (en) | 2018-04-25 | 2018-04-25 | Multi-source threat intelligence quality evaluation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109688091A CN109688091A (en) | 2019-04-26 |
| CN109688091B true CN109688091B (en) | 2021-10-08 |
Family
ID=66184411
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810379261.7A Active CN109688091B (en) | 2018-04-25 | 2018-04-25 | Multi-source threat intelligence quality evaluation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109688091B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111160749B (en) * | 2019-12-23 | 2023-07-21 | 绿盟科技集团股份有限公司 | Information quality assessment and information fusion method and device |
| CN111212049B (en) * | 2019-12-27 | 2022-04-12 | 杭州安恒信息技术股份有限公司 | Method for analyzing reputation of threat intelligence IOC |
| CN113627698A (en) * | 2020-05-07 | 2021-11-09 | 中国电信股份有限公司 | Threat information processing method, device and storage medium |
| CN111800439B (en) * | 2020-09-08 | 2020-12-22 | 江苏苏宁银行股份有限公司 | Application method and system of threat information in bank |
| CN112836038A (en) * | 2021-01-21 | 2021-05-25 | 中国科学院沈阳自动化研究所 | Intelligent recommendation system based on multi-source data credibility |
| CN113139025A (en) * | 2021-05-14 | 2021-07-20 | 恒安嘉新（北京）科技股份公司 | Evaluation method, device, equipment and storage medium of threat information |
| CN113360739A (en) * | 2021-06-02 | 2021-09-07 | 北京天融信网络安全技术有限公司 | Information source quality analysis method and device, electronic equipment and storage medium |
| CN113468384B (en) * | 2021-07-20 | 2023-11-03 | 山石网科通信技术股份有限公司 | Processing method, device, storage medium and processor for network information source information |
| CN114666144B (en) * | 2022-03-29 | 2024-03-12 | 杭州安恒信息技术股份有限公司 | Information source quality detection method, device, equipment and storage medium |
| CN114757790B (en) * | 2022-04-06 | 2022-10-11 | 山东新潮信息技术有限公司 | Method for evaluating multi-source information risk by using neural network |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012164336A1 (en) * | 2011-05-31 | 2012-12-06 | Bce Inc. | Distribution and processing of cyber threat intelligence data in a communications network |
| CN105743877A (en) * | 2015-11-02 | 2016-07-06 | 哈尔滨安天科技股份有限公司 | Network security threat information processing method and system |
| CN106060018A (en) * | 2016-05-19 | 2016-10-26 | 中国电子科技网络信息安全有限公司 | Network threat information sharing model |
| CN107729519A (en) * | 2017-10-27 | 2018-02-23 | 上海数据交易中心有限公司 | Appraisal procedure and device, terminal based on multi-source multidimensional data |
| CN107730096A (en) * | 2017-09-29 | 2018-02-23 | 北京神州绿盟信息安全科技股份有限公司 | A kind of method for evaluating quality and device in information data source |
-
2018
- 2018-04-25 CN CN201810379261.7A patent/CN109688091B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012164336A1 (en) * | 2011-05-31 | 2012-12-06 | Bce Inc. | Distribution and processing of cyber threat intelligence data in a communications network |
| CN105743877A (en) * | 2015-11-02 | 2016-07-06 | 哈尔滨安天科技股份有限公司 | Network security threat information processing method and system |
| CN106060018A (en) * | 2016-05-19 | 2016-10-26 | 中国电子科技网络信息安全有限公司 | Network threat information sharing model |
| CN107730096A (en) * | 2017-09-29 | 2018-02-23 | 北京神州绿盟信息安全科技股份有限公司 | A kind of method for evaluating quality and device in information data source |
| CN107729519A (en) * | 2017-10-27 | 2018-02-23 | 上海数据交易中心有限公司 | Appraisal procedure and device, terminal based on multi-source multidimensional data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109688091A (en) | 2019-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109688091B (en) | Multi-source threat intelligence quality evaluation method and device | |
| CN105657003B (en) | Information processing method and server | |
| CN112491611B (en) | Fault location system, method, apparatus, electronic device, and computer readable medium | |
| US20030236652A1 (en) | System and method for anomaly detection | |
| CN106815125A (en) | A kind of log audit method and platform | |
| Morbey et al. | The application of a novel ‘rising activity, multi-level mixed effects, indicator emphasis’(RAMMIE) method for syndromic surveillance in England | |
| WO2018184304A1 (en) | Method and device for detecting health state of network element | |
| CN110489314A (en) | Model method for detecting abnormality, device, computer equipment and storage medium | |
| CN106530121A (en) | Power monitoring system security protection compliance detection method and system | |
| CN106600303A (en) | Method and device for assessment of advertisement putting rationality | |
| KR101953558B1 (en) | Apparatus and Method for Fault Management of Smart Devices | |
| CN103577660B (en) | Gray scale experiment system and method | |
| CN109359234B (en) | Multi-dimensional network security event grading device | |
| CN110570097A (en) | business personnel risk identification method and device based on big data and storage medium | |
| CN113360566A (en) | Information content monitoring method and system | |
| CN113139025A (en) | Evaluation method, device, equipment and storage medium of threat information | |
| CN106649765A (en) | Smart power grid panoramic data analysis method based on big data technology | |
| CN102841922B (en) | Collecting method and device | |
| CN109145109B (en) | User group message propagation abnormity analysis method and device based on social network | |
| CN116051185A (en) | Advertisement position data abnormality detection and screening method | |
| WO2016187504A1 (en) | Crowd-based sentiment indices | |
| CN112671952B (en) | IP detection method, device, equipment and storage medium | |
| CN111813922B (en) | High-temperature event detection method and system based on microblog text data | |
| CN111427874B (en) | Quality control method and device for medical data production and electronic equipment | |
| CN113360376A (en) | Buried point testing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |