KR102592868B1 - Methods and electronic devices for analyzing cybersecurity threats to organizations - Google Patents

Abstract

This technology relates to methods and electronic devices for analyzing cybersecurity threats to organizations. A method for analyzing cybersecurity threats to an organization of the present technology includes preparing a data mart regarding a predetermined number of indicators; generating a univariate cybersecurity threat index as an anomaly score by combining the predetermined number of indicators for the data mart; Detecting abnormal index points by performing a statistics-based abnormal point detection algorithm and a machine learning-based abnormal point detection algorithm for the threat index; An anomaly score is generated for each of the predetermined number of indicators, a statistical anomaly detection algorithm is performed to detect indicator anomaly points, and the indicator anomaly points are located at the top N anomaly points among the index anomaly points. Verifying existence; And a step of performing time series cluster analysis on the index anomaly time points to cluster similar indexes and providing the results to the step of preparing the data mart.

Description

{Methods and electronic devices for analyzing cybersecurity threats to organizations}

The present invention relates to a method and electronic device for analyzing cybersecurity threats to an organization, and more specifically, to a method and electronic device for analyzing cybersecurity threats to an organization centered on anomaly detection.

Analysis is needed to respond to recent cyber security threats. In other words, it is time for companies and organizations to effectively recognize cyber security threats and propose ways to respond.

The main characteristics of cyber security threats that are occurring recently are as follows.

First, due to cybersecurity threats, problems in cyberspace are affecting the physical environment. As a result, an environment is being created where, if problems occur in the infrastructure that maintains society, the social system may collapse or large-scale damage may occur. These problems can cause greater damage in countries where the digitalization of major industrial facilities is well established.

Second, due to the development of information and communication technology and the spread of COVID-19, it is expected that the digitalization of the entire social system, including the government and businesses, will occur rapidly. In this process, the proportion of use of M2M devices and IoT devices is expected to increase, and it is necessary to prepare cyber threat response measures in a wide range of areas.

Third, as artificial intelligence technology is applied throughout society, it is becoming difficult to determine who is at fault when unexpected actions occur. Most artificial intelligence technologies operate in the form of a black box where it is difficult to interpret the causes of the results of actions, so as artificial intelligence technology develops, the subject of action will become more dependent on the software's own learning rules, which will become a bigger problem. This problem is expected to become an even bigger problem because it is difficult to easily determine the cause of the problem when a system (or service) using artificial intelligence technology operates abnormally due to a cyber threat (or attack).

Fourth, cyber attacks can cause problems over a wide range in a short period of time by utilizing the Internet space. When cyber security problems occur simultaneously around the world, the resulting ripple effect is expected to far exceed the scale of damage from existing physical disasters such as droughts and floods.

Fifth, since cyber attacks are carried out through virtual space, the attacker has the characteristic of being able to achieve his goal while hiding his identity through a bypass route. Because of this, problems that occur in cyberspace are difficult to identify who is acting.

Sixth, suspicions continue to be raised about intentional hacking attempts by manufacturers supplying equipment and at the national level. In relation to this, the United States is revising the National Defense Authorization Act in 2021 to specify products from Chinese manufacturers such as Huawei and ZTE as targets of security threats.

Due to these changes in cyber security threats, general companies and individuals are facing a situation where it is very difficult to respond effectively. In relation to this, the government is organizing policies to effectively respond to these cybersecurity threats and is preemptively developing key security technologies required for new technologies such as 5G. In addition, we have established and provided a system to effectively share cyber threat information. In the case of private companies, high-performance security equipment is being introduced and efforts are being made to establish an end-to-end security surveillance system. In detail, we are introducing various types of security equipment such as firewall, IDS (Intrusion Detection System), IPS (Intrusion Prevention System), APT response solution, and Content Filter solution, and we are implementing cyber security by introducing SIEM to effectively monitor and manage it. We are building a system to respond to threats. In addition, countries and organizations are conducting activities to improve the level of information protection of countries and companies by evaluating the cybersecurity index, and companies such as Broadcom and Splunk in the United States are applying AI technology to their security operation system to effectively respond to cybersecurity threats. We are continuously researching technologies such as AISecOps.

However, as cybersecurity threats develop into complex aspects such as strategizing, anonymization, and large-scale, there are limits to organically managing overall cybersecurity threats using the existing security surveillance system.

The inventor of the present invention completed the present invention after a long period of research and trial and error in order to solve these problems.

In order to complement the above-mentioned problems, the embodiment of the present invention discovers surveillance indicators used in various areas such as the security field, IT infrastructure field, and service field, and analyzes them comprehensively to monitor cyber security threats in near real-time. It provides a 'cyber security threat index framework' to develop a 'cyber security threat index' and operate it effectively.

Meanwhile, other unspecified purposes of the present invention will be additionally considered within the scope that can be easily inferred from the following detailed description and its effects.

A method for analyzing cybersecurity threats to an organization according to an embodiment of the present invention includes preparing a data mart about a predetermined number of indicators; generating a univariate cybersecurity threat index as an anomaly score by combining the predetermined number of indicators for the data mart; Detecting abnormal index points by performing a statistics-based abnormal point detection algorithm and a machine learning-based abnormal point detection algorithm for the threat index; An anomaly score is generated for each of the predetermined number of indicators, a statistical anomaly detection algorithm is performed to detect indicator anomaly points, and the indicator anomaly points are located at the top N anomaly points among the index anomaly points. Verifying existence; And it may include performing time series cluster analysis on the index anomaly time points to cluster similar indicators and providing the results to the step of preparing the data mart.

Preparing the data mart includes collecting information for each cyber security threat-related surveillance indicator from different collection target areas of the organization for a certain period of time; Converting the information for each monitoring indicator into individual time series statistical indicators in a certain time unit; and performing time series cluster analysis on the individual time series statistical indicators and assigning weights to each indicator.

After the weighting step, the method may further include compensating for missing data and applying log transformation and normalization to the individual time series statistical indicators.

The providing step may provide the clustering result to the providing step so that the weight for each indicator is modified.

Identifying key indicators that have made a significant contribution to detecting the index abnormality points among the predetermined number of indicators using a relative importance calculation algorithm and a variable selection regression analysis algorithm, further comprising: the identified key indicators; may be provided in the step of preparing the data mart.

The relative importance calculation algorithm may be chi-square-based, and the variable selection regression analysis algorithm may be LASSO-based.

The statistics-based anomaly detection algorithm may be a 3-sigma method that detects an anomaly based on a fixed threshold.

The machine learning-based abnormal point detection algorithm may be Prophet, which detects an abnormal point based on a dynamic threshold.

In the detecting step, anomaly detection may be performed by analyzing cases in which anomalies are detected at the same time in the 3 Sigma algorithm and the Profit algorithm.

The above different collection target areas include a security area that collects security equipment detection events and security measure history related to one or more of firewall, IPS, harmful site blocking, personal information leak detection, APT response solution, and spam blocking solution, and user groupware. It may include an IT infrastructure area that collects items related to one or more of connection history, PC status information, and PC connection information, and a service area that collects items related to one or more of network traffic information, equipment alarm occurrence information, and failure maintenance time information. You can.

Additionally, an electronic device for analyzing cybersecurity threats to an organization according to an embodiment of the present invention includes a preparation unit that prepares a data mart regarding a predetermined number of indicators; a generator that generates a univariate cybersecurity threat index as an anomaly score by combining the predetermined number of indicators for the data mart; a detection unit that detects index abnormality points by performing a statistics-based abnormal point detection algorithm and a machine learning-based abnormal point detection algorithm for the threat index; An anomaly score is generated for each of the predetermined number of indicators, a statistical anomaly detection algorithm is performed to detect indicator anomaly points, and the indicator anomaly points are located at the top N anomaly points among the index anomaly points. a verification unit that verifies existence; and a provision unit that performs time-series cluster analysis on the index anomaly points to cluster similar indices and provides the results to a preparation unit that prepares the data mart.

The preparation unit includes a collection unit that collects information for each cybersecurity threat-related surveillance indicator in different areas of the organization for a certain period of time; a conversion unit that converts the information for each monitoring indicator into individual time series statistical indicators in a certain time unit; and an assigning unit that performs cluster analysis on the individual time series statistical indicators and assigns weights to each indicator.

It may further include a correction unit that compensates for missing data and applies log transformation and normalization to the weighted individual time series statistical indicators.

It further includes an identification unit that identifies key indicators that have made a significant contribution to detecting the index abnormality points among the predetermined number of indicators using a relative importance calculation algorithm and a variable selection regression analysis algorithm, and the identified results. may be provided to a preparation unit that prepares the data mart.

In addition, a computer-readable recording medium on which a program for implementing a method of operating an electronic device according to an embodiment of the present invention is recorded includes the steps of preparing a data mart regarding a predetermined number of indicators; generating a univariate cybersecurity threat index as an anomaly score by combining the predetermined number of indicators for the data mart; Detecting abnormal index points by performing a statistics-based abnormal point detection algorithm and a machine learning-based abnormal point detection algorithm for the threat index; An anomaly score is generated for each of the predetermined number of indicators, a statistical anomaly detection algorithm is performed to detect indicator anomaly points, and the indicator anomaly points are located at the top N anomaly points among the index anomaly points. Verifying existence; and performing time-series cluster analysis on the index anomaly time points to cluster similar indicators and providing the result to the step of preparing the data mart.

This technology can provide a method of utilizing the index as a comprehensive indicator/meta-knowledge that provides a way to determine common trends by combining various individual indicators.

Figure 1 shows the detailed configuration of a cybersecurity threat analysis device according to an embodiment of the present invention.
Figure 2 is a diagram schematically showing the overall operation of a cybersecurity threat analysis device according to an embodiment of the present invention.
Figure 3 shows the detailed configuration of the preparation unit according to an embodiment of the present invention.
Figure 4 is a diagram showing the overall operation of the cybersecurity threat analysis device according to an embodiment of the present invention in more detail.
Figure 5 is a chart confirming the time series flow of 56 monitoring indicators selected according to an embodiment of the present invention.
Figure 6 shows the results of applying logarithms to individual indicators according to an embodiment of the present invention.
Figure 7 is a cluster satellite diagram showing the relationship between time series indicators in close proximity according to an embodiment of the present invention.
Figure 8 is an illustration of a swarm satellite diagram for cybersecurity threat indicators converted into a tree diagram according to an embodiment of the present invention.
Figure 9 is a diagram showing the results of distance map analysis according to an embodiment of the present invention.
Figure 10 shows a screen of an electronic device that generates one cybersecurity threat index by analyzing 56 indicators according to an embodiment of the present invention.
Figure 11 is a histogram showing variables that significantly contributed to anomaly detection in the cybersecurity threat index according to an embodiment of the present invention.
Figure 12 is an example of generating an anomaly score for an individual indicator according to an embodiment of the present invention.
Figure 13 shows an inter-surface swarm satellite diagram for individual indices according to an embodiment of the present invention.
Figure 14 shows the results of examining correlation coefficients between indicators for individual indicators according to an embodiment of the present invention.
Figure 15 shows a cyclical structure designed as a cybersecurity threat index framework according to an embodiment of the present invention.
Figure 16 is a flowchart showing a method for analyzing cybersecurity threats to an organization according to an embodiment of the present invention over time.
The attached drawings are intended as reference for understanding the technical idea of the present invention, and are not intended to limit the scope of the present invention.

The above objects, other objects, features and advantages of the present invention will be easily understood through the following preferred embodiments related to the attached drawings. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided to enable the disclosed content to be more thorough and complete and to sufficiently convey the spirit of the present invention to those skilled in the art, without any intention other than to provide convenience of understanding.

Figure 1 shows the detailed configuration of a cybersecurity threat analysis device 100 according to an embodiment of the present invention. As shown in Figure 1, the cybersecurity threat analysis device 100 (hereinafter simply referred to as the 'analysis device') includes a preparation unit 110, a generation unit 120, a detection unit 130, and a verification unit 140. and a provision unit 150.

Figure 2 is a diagram schematically showing the overall operation of the cybersecurity threat analysis device 100 according to an embodiment of the present invention. Looking at the operation of each component of the analysis device 100 of FIG. 1 with reference to FIG. 2, the preparation unit 110 prepares data (data preparation). Data prepared by the preparation unit may be referred to as a data mart in the present invention, but the term is not necessarily limited and is a concept that includes databases constructed in various forms. The generating unit 120 generates a cybersecurity threat index from the prepared data (cyber security threat index generation). Index is a comprehensive indicator or meta-knowledge that provides a way to determine common trends by combining various individual indicators. An indicator can be said to be a variable that helps measure change, and an index can be defined as a single value calculated by combining various indicators. The cybersecurity index can be defined as a 'comprehensive indicator that allows you to identify the status or level of the organization, resources, processes, and structural aggregates used to protect objects in cyberspace.' The cybersecurity threat index can be defined as a 'comprehensive index that provides the level of cybersecurity threats' and can be thought of as a detailed concept of the cybersecurity index. Note that in the present invention, the index is used separately from the indicator. The detection unit 130 detects an abnormal point in time from the generated cyber security threat index (anomaly detection). Several algorithms may be applied to detect abnormal points, but in the present invention, two algorithms with different characteristics are used in combination. This will be described later. The verification unit 140 verifies the abnormality detection results (detection result analysis (w/ abnormality detection result verification)). For this purpose, the concepts of overall integrated monitoring and individual indicator monitoring are introduced, as described later. And the provision unit 150 provides the detection result analysis results to be circulated through the data preparation process (reflection of indicator selection and weight optimization (Closed Loop)).

As a result, the present invention presents a framework for proposing and verifying a new cybersecurity threat index and continuously utilizing it.

Figure 3 shows the detailed configuration of the preparation unit 110 according to an embodiment of the present invention. As shown in FIG. 3, the preparation unit 110 includes a collection unit 112, a conversion unit 114, a correction unit 116, and an assignment unit 118. And Figure 4 is a diagram showing the overall operation of the cybersecurity threat analysis device according to an embodiment of the present invention in more detail. Hereinafter, the operation of each component of the analysis device will be examined in detail with reference to FIGS. 1 to 4.

The preparation unit 110 prepares a data mart about a predetermined number of indicators. The collection unit 112 of the preparation unit 110 collects information for each cybersecurity threat-related surveillance indicator from different collection target areas of the organization for a certain period of time. As shown in Figure 4, monitoring indicators ① to ③ are collected from the first collection target area, monitoring indicators A to C are collected from the second collection target area, and detection indicators ⓐ to ⓒ are collected from the third collection target area. It can be collected.

The first collection target area, the second collection target area, and the third collection target area may be different areas (eg, departments) within the organization. It is noted that the present invention analyzes cybersecurity threats by integrating surveillance indicators collected from different areas. For example, the first collection target area is the organization's security department (i.e., a department related to security within the organization), and the second collection target area is the organization's IT infrastructure department (i.e., a department related to IT infrastructure within the organization). , and the third collection target area may be the organization's service field department (i.e., department related to service operation within the organization). In analyzing cyber security threats, the present invention utilizes not only indicators collected from security departments directly related to security, but also indicators collected from indirectly related IT infrastructure departments and indicators collected from service provision departments.

In this case, detection indicators ① to ③ shown in the drawing are in the security field and may include various indicators related to security equipment detection, security measure history, external threat information, etc. Detection indicators A to C are in the IT infrastructure field and may include several indicators related to groupware access, PC status information, and PC connection information. Detection indicators ⓐ~ⓒ are a service provision field and may include various indicators related to service traffic, equipment failure alarms, and failure maintenance time. It is not limited to the examples listed, and various collection target areas and monitoring indicators may exist depending on the characteristics and type of the organization.

The conversion unit 114 converts the information for each monitoring indicator collected by the collection unit 112 into individual time series statistical indicators in a certain time unit. Time series refers to a set of continuous data arranged at regular time intervals.

The assignment unit 116 performs time series cluster analysis on the individual time series statistical indicators converted by the conversion unit 114 and assigns weights to each indicator. Time series cluster analysis is a technique that analyzes the characteristics of given data and groups groups of individuals with similar characteristics. Representative time series clustering algorithms include the Euclidean method and the Dynamic Time Wrapping (DTW) method. Weights can be assigned through the AHP (Analytic Hierarchy Process) technique. The AHP technique is a technique that helps evaluate the optimal alternative for complex decision-making problems by classifying multiple attributes hierarchically and identifying the importance through pair-wise comparison between attributes.

The operations of the collection unit 112, conversion unit 114, and grant unit 116 described above are referred to by reference numeral DTM in FIG. 4. That is, the collection unit 112 collects monitoring indicators from the first to third collection target areas, the conversion unit 114 converts the collected information into individual time series statistical indicators, and the granting unit 116 provides individual Perform time series cluster analysis on time series statistical indicators and assign weights to each indicator.

Meanwhile, although not shown in the drawing, the process of compensating for missing data through data interpolation for individual indicators and reducing the deviation for each indicator by applying log and normalization to ultimately create a data mart. This can be further performed by a correction unit (not shown).

The generation unit 120 compiles a predetermined number of indicators for the data mart prepared by the preparation unit 110 and generates a univariate cybersecurity threat index as an anomaly score. In FIG. 4, refer to the part where the anomaly score of the entire integrated surveillance is generated on the left side of the box indicated by IDX/IDR. The anomaly score is an indicator of the level of unexpected patterns that deviate from normal behavior in the system. An anomaly score value can be generated by changing the sigma (σ) level, which is the anomaly detection standard. For example, 3 Sigma assumes that 99.7% of the data is normal, so the value of 99.7 is defined as the anomaly score when detected according to the corresponding surveillance standard. An anomaly score can be generated by analyzing it with AWS Company's RRCF (Robust Random Cut Forest) algorithm. In the case of the AWS company's RRCF algorithm, it guarantees high speed and effectively expresses the volatility of multivariate variables as a single indicator, making it suitable for application to the present invention.

In the above example, data collected in the security field, service field, and IT field can be converted into time series data based on 1-hour statistics, and analyzed using the RRCF algorithm to generate a single anomaly score. By defining this as a cybersecurity threat index, ideals in a wide range of areas can be effectively expressed from an organizational governance perspective.

The detection unit 130 performs a statistical-based anomaly detection algorithm and a machine learning-based anomaly detection algorithm on the cybersecurity threat index (hereinafter simply referred to as 'threat index') generated by the generation unit 120. Detect exponential anomaly points. In FIG. 4, refer to the part labeled “Anomaly detection of overall integrated surveillance” on the left side of the box marked IDX/IDR. Anomaly detection is an activity that detects unexpected patterns that deviate from normal behavior in a system. Anomaly detection methods include statistical process control and machine learning-based anomaly detection methods generally used in the manufacturing field. A variety of methods are used in statistical process management, but in general, an abnormality is detected when the threshold at the 3 Sigma level is exceeded. This standard is conventionally used as a method of detecting items exceeding 0.27% as abnormal, assuming that the analysis data is normally distributed. The anomaly detection method utilizing this sigma (σ) level uses a fixed threshold, so detection accuracy may be low, but it has the advantage of being easy to interpret the detection results as it has been used for a long time. In the field of anomaly detection using machine learning, there are supervised learning with correct and incorrect answers, semi-supervised learning with only correct answers, and unsupervised learning with no correct and incorrect answers, and appropriate methods are used depending on the characteristics of the data to be analyzed. There are various detailed analysis methods, including deep learning-based, graphics-based, reinforcement learning-based, NLP-based, and time series prediction-based. Facebook's Prophet algorithm allows detecting anomalies in time series data using a curve-fitting method. The Profit algorithm has the advantage of reflecting holiday effects in the learning model so that irregularly occurring events are not detected.

The detection unit 130 defines the AI-based anomaly detection result as the AI-based anomaly detection result when the statistical-based anomaly detection result and the machine learning-based anomaly detection result are detected to be the same with respect to the cybersecurity threat index. In other words, statistical interpretation of detection results is secured using the 3 sigma (σ) technique, and false positives in abnormal detection results can be minimized by combining the advantages of the dynamic threshold of the profit algorithm and the holiday effect.

The verification unit 140 generates an anomaly score for each of a predetermined number of indicators and performs a statistical-based abnormal point detection algorithm to detect indicator abnormal points. In Figure 4, refer to the part called anomaly score generation of individual indicator monitoring and the part called anomaly detection on the right side of the box indicated by IDX/IDR. Then, it is verified whether index abnormality times exist in the top N abnormality times among the index abnormality times detected by the detection unit 130. The verification process by the verification unit has the meaning of verifying whether an abnormality has been detected in each of a predetermined number of indices with respect to the index abnormality points detected by the detection unit.

The operations of the above-described generation unit 120, detection unit 130, and verification unit 140 are referred to by reference numerals IDX/IDR in FIG. 4.

The provision unit 150 performs time-series cluster analysis on abnormal indicator points, clusters similar indicators, and provides the results to the preparation unit that prepares the data mart. In Figure 4, it is referred to by reference numeral ANA. This allows the clustering result by the provision unit 150 to be provided to the grant unit 116 of the preparation unit 110 so that the weight for each indicator is modified. It provides feedback to calculate the index in relation to various indicators that vary in various collection target areas, enabling the monitoring of cyber security threats in near real time.

Note that the time series cluster analysis performed on indicator anomaly time points by the provision unit 150 is different from the time series cluster analysis performed by the grant unit 116 of the preparation unit 110 described above. do. The time series cluster analysis by the granting department is performed before the point at which anomalies are detected in the individual indicators mentioned above, and the time series cluster analysis by the providing department is performed after that. Accordingly, in distinction from the time series cluster analysis by the granting department, the time series cluster analysis by the providing department is referred to as anomaly detection cluster analysis (see 'Anomaly detection cluster analysis' in the ANA box in Figure 4).

The weight for each indicator can have a value in a certain range, but it can also have a value of 0. This allows not only to assign a certain range of weight to individual indicators, but also to exclude individual indicators by assigning a weight of 0. .

Meanwhile, the analysis device 100 may further include an identification unit (not shown), where the identification unit (not shown) is a major component that has made an important contribution to detecting the index abnormality points among a predetermined number of indicators. Indicators are identified using a relative importance calculation algorithm and a variable selection regression analysis algorithm (see 'Selection of key indicators' in the ANA box in Figure 4). For example, the relative importance calculation algorithm may be chi-square based, and the variable selection regression analysis algorithm may be LASSO based. The present invention is not limited to the type of algorithm disclosed. Key indicators identified by the identification unit (not shown) may be provided to the assignment unit 116 of the preparation unit 110 so that the weights for each indicator can be modified. For example, indicators identified as key indicators may be modified to have higher weights.

Examples will be described below. It is assumed that the above-described analysis device is performed in an electronic device consisting of a memory, a communication interface, and a processor.

<Example>

A total of 56 items were selected for key monitoring elements in the security, IT infrastructure, and service fields through prior research and discussions with domain experts. The relevant monitoring component can be selected autonomously, taking into account the characteristics of the organization to which it is applied and whether or not it holds data. Table 1 is a table summarizing cyber threat monitoring components. For reference, the security indicators (16 Netflow, 22 services) discovered during the research process in Chapter 5 were excluded from this development process because the data at the time of analysis to be used for analysis could not be secured.

Cyber Threat Surveillance Component Daegu division Category subdivision Count The details Unique number Security detection security equipment firewall allow 6 full packet CNT7 total bytes CNT6 transmission packet CNT11 transfer byte CNT10 Received packet CNT9 Received bytes CNT8 firewall blocking 6
full packet CNT13 total bytes CNT12 transmission packet CNT17 transfer byte CNT16 Received packet CNT15 Received bytes CNT14 Monitoring harmful sites Three
port 443 CNT48 80 port CNT49 Other ports CNT50 Spam Security 12
Total number of files sent CNT36 Total number of shipments CNT38 Total sent mail size CNT37 Number of Chinese files CNT39 Number of shipments to China CNT40 Chinese mail size CNT41 Number of Korean files CNT45 Number of shipments to Korea CNT46 Korean mail size CNT47 Number of other files CNT42 Number of other shipments CNT43 Other mail sizes CNT44 APT solution 2
Email detection CNT34 Network detection CNT35 PC security 2 Personal information detection CNT51 Number of PC vulnerabilities CNT28 DBMS 5 Execution time CNT21 response time CNT22 Refusal to perform CNT20 response size CNT23 log in CNT33 security measures Security level 4 Critical CNT53 Major CNT54 Minor CNT56 Warning CNT55 external threat
information
Cyber threat information 5
C&C server IP CNT1 Attack attempt IP CNT2 Information leak IP CNT3 Phishing site CNT4 Smishing IP CNT5 IT
infra
groupware
Access
Groupware access 2 General access CNT52 Secure connection CNT32 PC status information One Number of SW execution times CNT24 PC connection information Three
Full-time employee CNT29 Partner company CNT30 etc CNT31 service
operate
service
watch
Service utilization Three
Received packet CNT26 transmission packet CNT27 Occupancy time CNT25 Equipment monitoring One Alarm count CNT18 Equipment monitoring One 1 alarm maintenance time CNT19 sum 56

First, 53 days of information was collected on 56 surveillance indicators needed to develop the 'Cyber Security Threat Index', and hourly time series statistical indicators were created. Figure 5 is a chart confirming the time series flow of 56 selected monitoring indicators. It can be seen that there is a large deviation in the units expressed for each indicator. In order to reduce the deviation between relevant time series indicators, log was applied to individual indicators. Log is generally used to make large numbers smaller and to easily perform complex calculations. For example, if you apply common logarithm to the number 100, it becomes the number 2, and is expressed as log ₁₀ 2. Figure 6 is a screen where logs are applied to individual indicators.

Log transformation was performed on individual indicators, and normalization was additionally applied to individual indicators. In addition, using the time series cluster analysis tool provided by SAS EM (Enterprise Miner), the relationship between time series indicators in the short distance was expressed as a cluster satellite diagram as shown in Figure 7. As a result of the cluster analysis, it was largely divided into two groups. Group 1 on the right side of the figure had 7 indicators clustered. In detail, it includes the Korea Internet & Security Agency's cyber threat information (C&C distribution IP, information leak IP, phishing site), firewall blocking policy (received bytes, number of packets), harmful site blocking (other ports), and security ticket (caution). and other indicators were classified into 2 groups. If the indicators included in group 1 are interpreted as domain knowledge, security tickets have the lowest level of action, and blocking harmful sites is a communication port other than 80 and 443, which is very infrequently used. Even in the case of blocking packets received from a firewall, it is judged to have little correlation with other indicators because it is an event that occurs irregularly from the outside. In addition, the Korea Internet & Security Agency's cyber threat information is presumed to have little correlation in time series because the information is shared after a security incident occurs. In other words, the indicators included in group 1 can be assumed to be indicators with low relevance to cybersecurity threats. The indicators in the other two groups are judged to be highly related to each other when examining their time series patterns.

The following Figure 8 is an illustration of a swarm satellite diagram for cybersecurity threat indicators converted into a tree diagram.

By analyzing the correlation between indicators through distance map analysis, useful information can be found in selecting important and unnecessary indicators (Figure 9). In distance map analysis, the less correlation between indicators, the less correlation is indicated in red, and the higher the correlation, the more it is indicated in blue. The indicators that showed the lowest correlation between indicators were security tickets (Warning, CNT55), threat information (phishing site, CNT04), threat information (information leak IP, CNT03), threat information (C&C server IP, CNT01), and firewall blocking ( These are in that order: received bytes, CNT14), and firewall blocking (received packets, CNT15), and were confirmed to be the same target as analyzed in the satellite diagram of the 2 clusters of threat indicators classified previously. Looking at the results of this analysis, it seems that it would be a good idea to consider excluding indicators with little correlation from the process of creating a cyber threat index. Other items with high correlation between indicators include firewall allow (all bytes, CNT06), firewall allow (received bytes, CNT08), groupware (PC connection, CNT31), firewall allow (received packets, CNT09), firewall allow (all packets) , CNT07), followed by service (DBMS login, CNT36). In the case of indicators with a high correlation between indicators, actions performed by users during work hours, groupware login, and an increase in network usage are judged to be highly correlated.

Indicators showing similar patterns were clustered through time series cluster analysis. The results can be used as a reference to select important indicators for organizations wishing to create a cybersecurity threat index. For example, a department that handles security tickets can look at indicators that are clustered in a time-series manner similar to the results of security measures as the main analysis target. Additionally, an organization that responds to APT attacks can look at patterns that are time-serially similar to APT attacks. can be judged as an important indicator in creating a cybersecurity index. Table 2 expresses the 56 analysis indicators along with their identification codes.

Time series cluster analysis results association code Column name association code Column name One cnt1 External threat information (C&C server IP) 7 cnt18 Equipment monitoring (number of alarms) One cnt4 External threat information (phishing site) 7 cnt19 Equipment monitoring (alarm maintenance time) 2 cnt2 External threat information (attack attempt IP) 7 cnt41 Spam security (Chinese mail size) 2 cnt34 APT solution (email detection) 7 cnt56 Security level (Minor) 2 cnt35 APT solution (network detection) 8 cnt20 DBMS (refusal to perform) 2 cnt51 PC security (personal information detection) 8 cnt31 PC connection information (other) 3 cnt3 External threat information (information leak IP) 8 cnt33 DBMS (login) 3 cnt14 Firewall blocking (received bytes) 8 cnt36 Spam security (total number of sent files) 3 cnt15 Firewall blocking (received packets) 8 cnt37 Spam security (total sent email size) 3 cnt55 Security measures (Warning) 8 cnt38 Spam security (total number of sendings) 4 cnt5 Cyber threat information (Smishing IP) 8 cnt40 Spam security (number of shipments to China) 4 cnt39 Spam security (number of Chinese attachments) 8 cnt42 Spam security (other files) 4 cnt45 Spam Security (Korea Attachment Count) 8 cnt43 Spam security (other sending frequency) 4 cnt53 Security measures (Major) 8 cnt44 Spam security (other email sizes) 4 cnt54 Security measures (Critical) 8 cnt46 Spam security (Korea sending frequency) 5 cnt6 Allow firewall (all bytes) 8 cnt47 Spam security (Korean mail size) 5 cnt7 Allow firewall (all packets) 8 cnt48 Harmful site monitoring (port 443) 5 cnt8 Allow firewall (received bytes) 8 cnt49 Harmful site monitoring (port 80) 5 cnt9 Allow firewall (received packet) 9 cnt24 PC status information (number of SW execution times) 5 cnt10 Allow firewall (transfer bytes) 9 cnt25 Service utilization (occupancy time) 5 cnt11 Allow firewall (transmission packet) 9 cnt26 Service utilization (received packets) 5 cnt21 DBMS (execution time) 9 cnt27 Service utilization (transmission packet) 5 cnt22 DBMS (execution response time) 9 cnt28 PC security (number of PC vulnerabilities) 5 cnt23 DBMS (response size) 9 cnt29 PC connection information (full-time employees) 6 cnt12 Firewall blocking (all bytes) 9 cnt30 PC connection information (partner) 6 cnt13 Firewall blocking (all packets) 9 cnt32 Groupware access (secure access) 6 cnt16 Firewall blocking (transmitted bytes) 9 cnt52 Groupware access (general access) 6 cnt17 Firewall blocking (transmission packet) 10 cnt50 Monitoring harmful sites (other ports)

Before creating a cybersecurity threat index, the importance of individual indicators for each domain expert can be selected and weights for each indicator can be applied. To do this, you can conduct a survey with experts in the field in which you want to use the index and apply weights for each indicator using the AHP technique. In addition, the final weight for each indicator can be adjusted by additionally multiplying the α coefficient in the weight calculation process. (Meanwhile, in this embodiment, it was difficult to specify the department that utilized the index, so weighting through the AHP technique was not applied.) After that, data correction (Interpolation) for 56 indicators was performed to create a 'Cyber Security Threat Index'. ), missing data, etc. were supplemented, and log and normalization were applied to reduce deviations by indicator to create the final data mart needed for analysis. Afterwards, 56 individual indicators were synthesized using the Robust Random Cut Forest algorithm developed by AWS to create a univariate 'Cyber Security Threat Index (Anomaly Score)'. The higher the cyber security threat index number, the more likely there is a problem, and the lower the number, the more likely it is to be within the normal range. Figure 10 is a screen showing one 'Cyber Security Threat Index (Anomaly Score)' created by analyzing 56 indicators. The 3 Sigma and Prophet algorithms were performed on the 'Cyber Security Threat Score' results proposed in this study to create an index. A total of 26 abnormal points (1.8% of the total) were detected. 26 abnormalities were detected in 3 Sigma and 15 abnormalities in Prophet, and a total of 15 abnormalities were detected simultaneously by 3 Sigma and Prophet. For reference, 3 Sigma detects anomalies based on a fixed threshold, and Prophet detects anomalies based on a dynamic threshold. In relation to this, the standards for monitoring can be defined by adjusting the sigma level and the level of Prophet's hyperparameters depending on the importance of the monitoring target. For example, in the case of very important services, the hyperparameter levels of Sigma and Prophet can be set narrowly, while unimportant services can be monitored by setting them broadly. Table 3 is a table summarizing the abnormality detection results for the cyber threat index. The detection time point is expressed as P1~P26, and the score is the degree of outlier. Rank is a ranking of the level of outliers for the score, and in 3 Sigma and Prophet, O indicates detected and X indicates not detected.

Cyber threat index abnormality detection results hour P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 Score 0.49 0.45 0.49 0.53 0.46 0.46 0.56 0.52 0.46 0.56 0.54 0.53 0.52 Rank 19 24 18 11 21 22 5 12 20 6 8 10 14 3 sigma O O O O O O O O O O O O O Prophet X X X O X X O O X O O O O The same time X X X O X X O O X O O O O hour P14 P15 P16 P17 P18 P19 P20 P21 P22 P23 P24 P25 P26 Score 0.45 0.74 0.44 0.49 0.49 0.52 0.67 0.82 0.54 0.45 0.55 1.00 0.50 Rank 23 3 26 17 16 13 4 2 9 25 7 One 15 3 sigma O O O O O O O O O O O O O Prophet X O X O X O O O O O X O O The same time X O X O X O O O O O X O O

The importance of variables was analyzed to identify individual indicators that had the greatest impact on the anomaly detection results of the ‘Cybersecurity Threat Index’. The process of analyzing important indicators can be useful information in interpreting the causal relationship between detection results and selecting important indicators. Generally, variable importance calculation methods are predicted based on relative importance (or chi-square score), and the formula used to calculate variable importance in this study is as follows. Variables used to effectively divide anomaly detection results are classified as having relatively high importance. Basically, if the value is 0.05 or higher, it can be defined as a major variable.

A(sv, T): Consistency of v usage at node T (1 for Sv separation rule, 0 for non-separation rule)

SSE(τ): Sum of predicted value errors

SSE(τ) means the reduction of the sum of squared errors for the predicted value, and can be expressed in the formula below.

Variables that significantly contributed to anomaly detection in the cybersecurity threat index were displayed in a histogram as shown in Figure 11.

Important anomaly detection indicators that affect the cyber threat index were identified as firewall traffic, spam sending, DBMS inquiry response, and APT response solution detection. Identification of key variables was performed using two methods, chi-square-based and LASSO-based, as described above. LASSO is a regression analysis method that can be used to select variables to improve prediction accuracy and interpretability in statistical models. Table 4 shows the variables that had a major impact on the detected cyber threats.

Cyber Threat Index Anomaly Detection Key Variables # Indicators Unique number Chi-square based LASSO
technique R-Square F value P-value Error Mean
Square One Firewall blocking (received packets) CNT15 0.179 278.721 <.0001 4.552 O 2 Firewall blocking (received bytes) CNT14 0.042 68.270 <.0001 1.059 O 3 Spam security (other sending frequency) CNT48 0.035 60.675 <.0001 0.900 O 4 Spam Security (Korea Attachment Count) CNT45 0.020 34.700 <.0001 0.501 O 5 Allow firewall (transfer bytes) CNT10 0.011 20.227 <.0001 0.288 X 6 DBMS (execution response time) CNT22 0.010 18.787 <.0001 0.264 O 7 Spam security (number of shipments to China) CNT40 0.005 10.036 0.002 0.140 O 8 Allow firewall (all bytes) CNT6 0.004 8.177 0.004 0.113 X 9 APT solution (network detection) CNT35 0.004 8.123 0.004 0.112 O 10 DBMS (refusal to perform) CNT20 0.003 5.201 0.023 0.071 X 11 Spam security (total sent email size) CNT37 0.003 6.116 0.014 0.084 X 12 Spam security (other email sizes) CNT44 0.003 4.703 0.030 0.064 X 13 Spam security (Korea sending frequency) CNT46 0.002 3.936 0.048 0.054 X 14 Spam security (total sent email size) CNT37 0.002 4.171 0.041 0.057 X 15 Firewall blocking (received packets) CNT15 0.002 4.378 0.037 0.059 O 16 DBMS (login) CNT33 0.001 2.391 0.122 0.032 X 17 PC connection information (partner) CNT30 0.002 2.834 0.093 0.038 X 18 Harmful site monitoring (port 443) CNT48 0.003 5.835 0.016 0.079 X 19 Service utilization (received packets) CNT26 0.001 2.017 0.156 0.027 X 20 Groupware access (secure access) CNT32 0.002 3.008 0.083 0.040 X 21 PC connection information (partner) CNT30 0.002 3.355 0.067 0.045 X 22 Service utilization (transmission packet) CNT27 0.002 2.911 0.088 0.039 X 23 Spam security (Chinese mail size) CNT41 0.001 1.743 0.187 0.023 X 24 Allow firewall (received bytes) CNT8 0.001 1.726 0.189 0.023 X

In addition, a procedure was conducted to verify in detail which indicator had the problem at the time of detecting an abnormality in the cybersecurity threat index. Anomaly scores were created for a total of 56 individual sub-indicators and anomaly points were detected based on 3 Sigma standards. Figure 12 is an example of generating an Anomaly Score for individual indicators. In the case of source data, the data values range from 0 to 18,000, and time series data has a downward trend. Regarding this, when an Anomaly Score was created through normalization for each indicator, it was converted into data that could be used for abnormality detection by generating an indicator (or index) between 0 and 1. Anomaly Scores were generated for a total of 56 individual indicators, and 3 As a result of detecting abnormal timings based on sigma, abnormal timings were detected in 1,415 (4%) of a total of 33,320 analysis subjects. In relation to this, all anomaly detection results of individual indicators were present at the top five anomaly points detected in the overall cybersecurity threat index (Anomaly Score), so the detailed detection contents were analyzed (Table 5). In the case of P25, when we estimate that the number of incoming communications blocked by the firewall has increased, it is assumed that the firewall blocked the wrong service or took security measures due to excessive traffic. In the case of P21, it can be seen that the reception of spam emails, including those from China, is increasing, and the number of communications blocked by the firewall is increasing. In the case of P15, since the traffic allowed or blocked by the firewall has increased overall, it can be assumed that excessive communication concentration is occurring on the network. In the case of P20, the traffic allowed or blocked by the firewall increased, and it can be assumed that DBMS access and spam from Korea increased. In the case of P7, spam from Korea has increased, and it can be assumed that program execution on PC terminals has increased.

Cyber security threat index anomaly detection correlation analysis Anomaly detection time Individual indicator detection (Anomay Score) Interpretation of detection results (estimate) number Detection number Score One P25 1.00 Firewall blocking (received bytes) (1.00)
Firewall blocking (received packets) (1.00)
Firewall blocking (total bytes) (0.93)
Firewall blocking (transmission packet) (0.97)
Firewall blocking (transmitted bytes) (0.85)
Firewall blocking (all packets) (0.81)
Allow firewall (received bytes) (0.36) Increase in firewall-blocked communications 2 P21 0.82 Firewall blocking (total bytes) (1.00) Firewall blocking (received bytes) (1.00)
Firewall blocking (received packets) (1.00)
Firewall blocking (transmitted bytes) (1.00)
Firewall blocking (transmission packet) (1.00)
Firewall blocking (all packets) (0.99)
Spam security (total number of sent files) (0.55)
Spam security (total sent email size) (0.55)
Spam security (Chinese mail size) (0.50)
Spam security (total number of sendings) (0.37) Increase in firewall-blocked communications
Receive spam from China 3 P15 0.74 Allow firewall (total bytes) (1.00) Allow firewall (total packets) (0.87)
Allowed traffic increase
Increased blocked traffic
Allow firewall (received packets) (0.86)
Allow firewall (transmitted bytes) (0.86)
Allow firewall (received bytes) (0.70)
Allow firewall (transmitted packet) (0.65)
Firewall blocking (received bytes) (1.00)
Firewall blocking (received packets) (1.00)
Firewall blocking (all packets) (0.67)
Firewall blocking (total bytes) (0.41)
Firewall blocking (transmitted bytes) (0.51)
Firewall blocking (transmission packet) (0.42)
DBMS (execution response time) (0.47) DBMS response delay phenomenon
4 P20 0.67 Allow firewall (transmitted bytes) (0.83) Allow firewall (total packets) (0.82)
Allow firewall (transmitted packet) (0.71)
Allow firewall (received packet) (0.63)
Allow firewall (total bytes) (0.61)
Allow firewall (received bytes) (0.47)
Firewall blocking (received bytes) (0.99)
Firewall blocking (received packets) (0.83)
Spam Security (Korea Number of Attachments) (0.88)
DBMS (login) (0.25) Increased firewall blocking communication
Increase in Korean spam 5 P7 0.56 Spam security (number of attachments in Korea) (0.63) PC status information (number of SW execution times) (0.37) Increase in spam from Korea
Increased program performance

In order to select and manage individual indicators used to monitor cyber security threats, time series cluster analysis between indicators can be performed at the time anomalies are detected in individual indicators to cluster indicators with a high relationship. To accomplish this, anomalies were detected using 3 Sigma for the Anomaly Score generated from individual indicators, and time series cluster analysis was performed on a total of 1,415 anomaly detection results detected from a total of 56 indicators. It can be seen that it is divided into two clusters, and the cluster satellite diagram between indicators is shown in Figure 13. Next, Figure 14 shows the correlation coefficient between indicators. Blue indicates high correlation between indicators, and red indicates low correlation between indicators.

Table 6 shows the results of time-series cluster analysis performed on 52 indicators that were detected more than once over a total of 53 days. In the case of cluster 1, it is assumed that groupware login, increased service traffic, increased attack attempt detection, and increased PC inspection were similarly clustered according to the user's work cycle. In Cluster 4, the increase in firewall traffic and the number of spam emails is estimated to be high for security operations. In the case of Cluster 7, it is presumed that the user logs in and receives spam emails from China and Korea or performs tasks such as DBMS inquiry. In the case of Cluster 8, it can be assumed that the security measures (Critical) operate similarly to the email detection results of the APT response solution. It is believed that the results can be used as reference material in the selection process by domain experts and key indicators that manage the cybersecurity threat index.

Individual indicator anomaly detection time series cluster analysis results association Column name association Column name
One External threat information (attack attempt IP) 5 Firewall blocking (received bytes) One Service utilization (occupancy time) 5 Firewall blocking (received packets) One PC security (number of PC vulnerabilities) 5 Harmful site monitoring (port 443) One Spam security (Korean mail size) 5 PC security (personal information detection) One Groupware access (general access) 6 Firewall blocking (all bytes) 2 Cyber threat information (Smishing IP) 6 Firewall blocking (transmission packet) 2 Firewall blocking (all packets) 6 Service utilization (transmission packet) 2 Firewall blocking (transmitted bytes) 6 DBMS (login) 2 Service utilization (received packets) 6 Harmful site monitoring (port 80) 3 Allow firewall (all bytes) 6 Security level (Minor) 3 Allow firewall (received bytes) 7 Equipment monitoring (alarm maintenance time) 3 Equipment monitoring (number of alarms) 7 DBMS (response size) 3 DBMS (refusal to perform) 7 PC connection information (full-time employees) 3 DBMS (execution time) 7 PC connection information (partner) 3 DBMS (execution response time) 7 Groupware access (secure access) 3 PC status information (number of SW execution times) 7 Spam security (total sent email size) 3 PC connection information (full-time employees) 7 Spam security (number of shipments to China) 3 Spam security (total number of sent files) 7 Spam security (Chinese mail size) 3 Spam security (total number of sendings) 7 Spam Security (Korea Attachment Count) 3 Spam security (other sending frequency) 7 Spam security (Korea sending frequency) 3 Spam security (other email sizes) 8 APT solution (email detection) 4 Allow firewall (all packets) 8 Security measures (Critical) 4 Allow firewall (transmission packet) 9 APT solution (network detection) 4 Spam security (number of other files) 10 Spam security (number of Chinese attachments) 4 Security measures (Major) 4 Security measures (Warning) 5 Allow firewall (received packet) 5 Allow firewall (transfer bytes)

Lastly, in order to continuously utilize the cybersecurity threat index, a circular structure such as the 'Cybersecurity Threat Index Framework' shown in Figure 15 was designed. The execution procedure described in the picture can be summarized as follows. First, data necessary for analysis is collected to generate time series data, and statistical-based similar group information is generated and provided through time series cluster analysis of individual indicators. Second, weights for each indicator are applied using the AHP technique through a survey of domain experts who want to utilize the cybersecurity threat index. Third, a univariate anomaly score is generated for the multivariate analysis indicators, and this is defined as the cybersecurity threat index. Fifth, abnormalities in the cybersecurity threat index are detected using the statistics-based 3-sigma and machine learning-based Prophet algorithms. Sixth, with regard to anomalies detected in the cybersecurity threat index, a detailed check is conducted to see whether anomalies are also detected in individual indicators. Seventh, key variables are selected through cluster analysis for individual indicators detected at abnormal points in the cybersecurity threat index, and the results are reflected in the data preparation process. Through this circular procedure, it is believed that it will be possible to manage a 'cyber security threat index' that can continuously respond to cyber security threats for heterogeneous services.

The present invention proposes a method for generating a 'cyber security threat index' that can manage the level of cyber security threats in the security field, IT infrastructure field, and overall service field in near real time, and can continuously utilize the 'cyber security threat index'. We propose an ‘index framework’. According to the present invention, first, we propose a 'cyber security threat index framework' that can be flexibly and intuitively used to monitor cyber threats in a wide range of fields. Second, through prior research, we select 56 security indicators in the security field, IT infrastructure field, and service field, and propose an anomaly detection analysis methodology. Third, at a time when the cyber security threat index is high, we check for abnormalities in sub-indicators and conduct a drill down type analysis that can estimate the cause of the abnormality. Fourth, it provides reference indicators that can be used to reanalyze detected results to select important and unimportant indicators.

conclusion

This experimental example is a method of generating a 'cyber security threat index' that can effectively detect cyber security threats that may occur complexly in an organization, and a 'cyber security threat index frame' that can continuously improve detection accuracy and expand the scope of surveillance. 'Work' was proposed. Through this, it is expected that by complementing the existing security surveillance system, it will be possible to build a system that can effectively recognize and manage cyber security threats that may occur in a wide range of areas. The main achievements of this study are summarized step by step as follows. First, by identifying trends in cyber threats, we defined six types of cyber security problems that may occur now and in the future, and mentioned the need to establish security measures for these problems. The details are as follows. ①Problems in cyberspace will also affect the physical environment. ②The digitalization of the entire social system, including governments and businesses, is rapidly occurring, and as a result, cyber damage may occur in all areas of society. ③With the spread of artificial intelligence technology, responsibility for unexpected actions (or cyber attacks) is becoming unclear. ④Cyber attacks can cause problems on a wide scale in a short period of time by utilizing Internet space. ⑤Cyber attacks utilize virtual space, so attackers can sufficiently conceal themselves. ⑥Intentional hacking attempts by manufacturers supplying equipment or at the national level will increase. Second, data from Netflow, a network communication protocol, was analyzed using machine learning to discover 16 candidate indicators of cyber security threats. This indicator is an indicator that can detect whether the service port of the operating equipment is open or information leakage. Specifically, it measures important factors such as foreign inflow (Not Korea), connection attempt (TCP Syn Flag), and connection termination (TCP Fin-Ack). It was identified as a variable. Third, using Netflow data, we proposed a method to identify whether service ports for operating equipment are open through network packets without directly checking service ports. Through this, it is believed that it will be possible to shorten the inspection time for operating equipment and prevent failures through direct connection to aging equipment. Fourth, a study was conducted to verify the efficiency of the operating department by combining data from the service operation field and information security field. In this process, we verified the AHP technique, which combines service operation information and security event information to link and analyze data and evaluate the importance of each indicator. In addition, 22 candidates for major indicators that can be used to develop a cybersecurity threat index were discovered. General efficiency evaluations are conducted individually based on operational data or information security events, but in this study, various information such as operational events, security events, and failure history are combined in a complex manner and used to evaluate the efficiency of each organization. Fifth, in the development of an AIOps-based anomaly detection framework (mobile communication field), a univariate Anomaly Score was created using various types of alarms and performance data, and a model that could detect anomalies using the 3 Sigma and Prophet algorithms was proposed. . In addition, we proposed a closed loop-type framework that can re-analyze the detected results to identify important indicators and reflect them back into the model. Sixth, we proposed a 'Cyber Security Threat Index' that can utilize cyber security threats in a data-based, objective and quasi-real-time manner for a wide range of fields, and a framework that can utilize it. The existing cybersecurity index creates an index based on a questionnaire, so it is subjective and has limitations that make it difficult to utilize in near real time. Additionally, there are cases where the data collection path for index creation is unclear and the index creation method is not disclosed. In these cases, there are limitations in using it as an index suitable for companies and organizations. The cyber security threat index proposed in this study proposes a quasi-real-time monitoring method, so it can be used to quickly detect threats, and because it generates a security index based on data, the intervention of subjective opinions can be minimized. It is believed that if an organization uses this cyber threat index, it will be able to detect comprehensive security abnormalities in each service field. In addition, it is believed that detection accuracy can be continuously improved by re-analyzing the detected results to select key indicators or optimize importance.

Figure 16 is a flowchart showing a method for analyzing cybersecurity threats to an organization according to an embodiment of the present invention over time. As shown in Figure 16, the method for analyzing cybersecurity threats to an organization includes preparing a data mart (S110), generating a cybersecurity threat index (S120), and detecting index abnormalities (S120). It may include steps S130), verifying whether indicator abnormalities exist (S140), and providing the results to the data mart preparation step (S150). The description of the preparation unit, generation unit, detection unit, verification unit, and provision unit of the cybersecurity threat analysis device previously described in FIGS. 1 to 4 can be applied in the same manner. That is, step S110 may be performed by a preparation unit, step S120 may be performed by a generation unit, step S130 may be performed by a detection unit, step S140 may be performed by a verification unit, and step S150 may be performed by a provision unit. Since the description of the components of the analysis device described above can be applied equally to each step, a more detailed description will be omitted.

The term “unit” used in this document may mean, for example, a unit including one or a combination of two or more of hardware, software, or firmware. “Part” can be used interchangeably with terms such as, for example, module, unit, logic, logical block, component, or circuit. ) can be. A “part” may be the minimum unit of an integrated part or a part thereof. “Part” may be the minimum unit or part of one or more functions. The “part” may be implemented mechanically or electronically. For example, a “part” may be an application-specific integrated circuit (ASIC) chip, field-programmable gate arrays (FPGAs), or programmable-logic device, known or to be developed in the future, that performs certain operations. It can contain at least one.

At least a portion of the device (e.g., modules or functions thereof) or method (e.g., operations) according to various embodiments may be stored in, for example, a computer-readable storage media in the form of a program module. It can be implemented with instructions stored in . When the instruction is executed by a processor (eg, processor), the one or more processors may perform the function corresponding to the instruction. A computer-readable storage medium may be, for example, memory.

Computer-readable recording media include hard disks, floppy disks, magnetic media (e.g., magnetic tape), and optical media (e.g., CD-ROM, DVD, magneto-optical media). may include optical media (e.g., floptical disk), hardware devices (e.g., ROM, RAM, or flash memory, etc.), etc. Additionally, program instructions may include machine code, such as that generated by a compiler. In addition, it may include high-level language code that can be executed by a computer using an interpreter, etc. The above-described hardware devices may be configured to operate as one or more software modules to perform the operations of various embodiments, and vice versa. .

A module or program module according to various embodiments may include at least one of the above-described components, some of them may be omitted, or may further include other additional components. Operations performed by modules, program modules, or other components according to various embodiments may be executed sequentially, in parallel, iteratively, or in a heuristic manner. Additionally, some operations may be executed in a different order, omitted, or other operations may be added.

100: Cybersecurity threat analysis device
110: Preparation department
112: collection department
114: conversion unit
116: Buyeobu
120: generation unit
130: detection unit
140: verification unit
150: Provider

Claims (15)

As a method for analyzing cybersecurity threats to an organization,
Preparing a data mart about a predetermined number of indicators;
generating a univariate cybersecurity threat index (index) as an anomaly score by combining the predetermined number of indicators for the data mart;
Detecting index anomaly points by performing a statistics-based anomaly point detection algorithm and a machine learning-based anomaly point detection algorithm on the index;
An anomaly score is generated for each of the predetermined number of indicators, a statistical anomaly detection algorithm is performed to detect indicator anomaly points, and the indicator anomaly points are located at the top N anomaly points among the index anomaly points. Verifying existence; and
A step of performing time-series cluster analysis on the indicator anomaly time points to cluster similar indicators and providing the results to the step of preparing the data mart;
The steps to prepare the data mart are:
collecting information for each cybersecurity threat-related surveillance indicator in different collection target areas of the organization for a certain period of time;
Converting the information for each monitoring indicator into individual time series statistical indicators in a certain time unit; and
Comprising: performing time series cluster analysis on the individual time series statistical indicators and assigning weights to each indicator,
After the weighting step,
Compensating for missing data for the individual time series statistical indicators and correcting them by applying log transformation and normalization. delete delete As a method for analyzing cybersecurity threats to an organization,
Preparing a data mart about a predetermined number of indicators;
generating a univariate cybersecurity threat index (index) as an anomaly score by combining the predetermined number of indicators for the data mart;
Detecting index anomaly points by performing a statistics-based anomaly point detection algorithm and a machine learning-based anomaly point detection algorithm on the index;
An anomaly score is generated for each of the predetermined number of indicators, a statistical anomaly detection algorithm is performed to detect indicator anomaly points, and the indicator anomaly points are located at the top N anomaly points among the index anomaly points. Verifying existence; and
A step of performing time-series cluster analysis on the indicator anomaly time points to cluster similar indicators and providing the results to the step of preparing the data mart;
The steps to prepare the data mart are:
collecting information for each cybersecurity threat-related surveillance indicator in different collection target areas of the organization for a certain period of time;
Converting the information for each monitoring indicator into individual time series statistical indicators in a certain time unit; and
Comprising: performing time series cluster analysis on the individual time series statistical indicators and assigning weights to each indicator,
The providing step provides the clustering result to the providing step so that the weight for each indicator is modified. As a method for analyzing cybersecurity threats to an organization,
Preparing a data mart about a predetermined number of indicators;
generating a univariate cybersecurity threat index (index) as an anomaly score by combining the predetermined number of indicators for the data mart;
Detecting index anomaly points by performing a statistics-based anomaly point detection algorithm and a machine learning-based anomaly point detection algorithm on the index;
An anomaly score is generated for each of the predetermined number of indicators, a statistical anomaly detection algorithm is performed to detect indicator anomaly points, and the indicator anomaly points are located at the top N anomaly points among the index anomaly points. Verifying existence; and
A step of performing time-series cluster analysis on the indicator anomaly time points to cluster similar indicators and providing the results to the step of preparing the data mart;
It further includes the step of identifying key indicators that have made a significant contribution to detecting the index abnormality points among the predetermined number of indicators using a relative importance calculation algorithm and a variable selection regression analysis algorithm,
The method of claim 1, wherein the identified key indicators are provided in the step of preparing the data mart. According to clause 5,
The method for calculating relative importance is based on chi-square, and the variable selection regression analysis algorithm is based on LASSO. As a method for analyzing cybersecurity threats to an organization,
Preparing a data mart about a predetermined number of indicators;
generating a univariate cybersecurity threat index (index) as an anomaly score by combining the predetermined number of indicators for the data mart;
Detecting index anomaly points by performing a statistics-based anomaly point detection algorithm and a machine learning-based anomaly point detection algorithm on the index;
An anomaly score is generated for each of the predetermined number of indicators, a statistical anomaly detection algorithm is performed to detect indicator anomaly points, and the indicator anomaly points are located at the top N anomaly points among the index anomaly points. Verifying existence; and
A step of performing time-series cluster analysis on the indicator anomaly time points to cluster similar indicators and providing the results to the step of preparing the data mart;
The statistical-based abnormal point detection algorithm is a 3-sigma method that detects an abnormal point based on a fixed threshold. In clause 7,
The machine learning-based abnormal point detection algorithm is Prophet, which detects an abnormal point based on a dynamic threshold. According to clause 8,
The method of detecting an anomaly by analyzing cases where an anomaly is detected simultaneously in the 3-sigma algorithm and the profit algorithm. According to any one of paragraphs 1, 4, 5 and 7,
The different collection target areas are
A security area that collects security device detection events and security measure history related to one or more of firewall, IPS, harmful site blocking, personal information leak detection, APT response solution, and spam blocking solution;
IT infrastructure area that collects items related to one or more of the user's groupware access history, PC status information, and PC connection information, and
A method comprising a service area that collects items related to one or more of network traffic information, equipment alarm occurrence information, and failure maintenance time information. An electronic device for analyzing cybersecurity threats to an organization, comprising:
A preparation department that prepares a data mart about a predetermined number of indicators;
a generator for generating a univariate cybersecurity threat index (index) as an anomaly score by combining the predetermined number of indicators for the data mart;
a detection unit that detects index anomaly points by performing a statistics-based anomaly point detection algorithm and a machine learning-based anomaly point detection algorithm on the index;
An anomaly score is generated for each of the predetermined number of indicators, a statistical anomaly detection algorithm is performed to detect indicator anomaly points, and the indicator anomaly points are located at the top N anomaly points among the index anomaly points. a verification unit that verifies existence; and
A provision unit that performs time-series cluster analysis on the indicator anomaly points to cluster similar indicators and provides the results to a preparation unit that prepares the data mart,
The preparation department
a collection unit that collects information by surveillance indicators related to cyber security threats in different areas of the organization for a certain period of time;
a conversion unit that converts the information for each monitoring indicator into individual time series statistical indicators in a certain time unit; and
A granting unit that performs cluster analysis on the individual time series statistical indicators and assigns weights to each indicator;
An electronic device further comprising: a correction unit that compensates for missing data and applies log transformation and normalization to the weighted individual time series statistical indicators. delete delete An electronic device for analyzing cybersecurity threats to an organization, comprising:
A preparation department that prepares a data mart about a predetermined number of indicators;
a generator for generating a univariate cybersecurity threat index (index) as an anomaly score by combining the predetermined number of indicators for the data mart;
a detection unit that detects index anomaly points by performing a statistics-based anomaly point detection algorithm and a machine learning-based anomaly point detection algorithm on the index;
An anomaly score is generated for each of the predetermined number of indicators, a statistical anomaly detection algorithm is performed to detect indicator anomaly points, and the indicator anomaly points are located at the top N anomaly points among the index anomaly points. a verification unit that verifies existence; and
A provision unit that performs time-series cluster analysis on the indicator anomaly points to cluster similar indicators and provides the results to a preparation unit that prepares the data mart,
It further includes an identification unit that identifies key indicators that have made a significant contribution to detecting the index abnormality points among the predetermined number of indicators using a relative importance calculation algorithm and a variable selection regression analysis algorithm,
The identified result is provided to a preparation unit that prepares the data mart. A recording medium containing a method of operating an electronic device for analyzing cybersecurity threats to an organization,
Preparing a data mart about a predetermined number of indicators;
generating a univariate cybersecurity threat index (index) as an anomaly score by combining the predetermined number of indicators for the data mart;
Detecting index anomaly points by performing a statistics-based anomaly point detection algorithm and a machine learning-based anomaly point detection algorithm on the index;
An anomaly score is generated for each of the predetermined number of indicators, a statistical anomaly detection algorithm is performed to detect indicator anomaly points, and the indicator anomaly points are located at the top N anomaly points among the index anomaly points. Verifying existence; and
A step of performing time-series cluster analysis on the indicator anomaly time points to cluster similar indicators and providing the results to the step of preparing the data mart;
The steps to prepare the data mart are:
collecting information for each cybersecurity threat-related surveillance indicator in different collection target areas of the organization for a certain period of time;
Converting the information for each monitoring indicator into individual time series statistical indicators in a certain time unit; and
Comprising: performing time series cluster analysis on the individual time series statistical indicators and assigning weights to each indicator,
The providing step provides the clustering result to the assigning step so that the weight for each indicator is modified. A computer-readable recording medium having a program recorded thereon for implementing a method of operating an electronic device.