JP2022024277A - Analysis system, analysis device, and analysis method - Google Patents

Abstract

To enable assisting a human to determine explanatory variables that are not required for a prediction model expression.SOLUTION: An analysis system comprises: an input data creation section which creates a plurality of explanatory variables using information pertaining to events, for creating input data including the plurality of explanatory variables and results of the events; a prediction section which creates a prediction model expression predicting values for objective variables representing the results of the events using the plurality of explanatory variables; an explanatory variable evaluation section which calculates exclusion candidate variables representing candidates of explanatory variables which are not required to be used in the prediction model expression; and an output section which outputs the exclusion candidate variables.SELECTED DRAWING: Figure 5

Description

The present invention relates to an analysis system, an analysis device, and an analysis method.

There is an increasing need to respond to cyberspace threats such as targeted attacks and other cyber attacks, "DDoS" (Denial of Service) attacks, and virus spread. In cyberspace, the attacker has a structural advantage, and the attacks are becoming more sophisticated, increasing, and changing day by day. Moreover, the targets of attacks are expanding beyond financial service providers and IT service providers to infrastructure providers and the like. On the other hand, IT systems in companies are becoming more complicated and diversified due to the expansion of the use of cloud services, the spread of mobile devices such as smartphones and tablets, and remote work. In such a situation, it is unknown when and where the network is compromised and important information for the enterprise is leaked to the outside.

Until now, engineers in the IT department who are familiar with the server and network fields have responded to security threats, but in addition to multi-layered security measures, a security operations center (SOC) that is constantly monitored by specialized staff is constantly monitored. ) Is increasing. However, there was concern that the required number of security specialists could not be secured, which would hinder the work of monitoring the occurrence of security incidents in information systems and control systems. Especially in the social infrastructure business, since the monitoring target covers the entire system, it has been desired to significantly improve the operational capacity of the SOC.

The most important thing in the operation work at SOC is to evaluate the importance of security alerts notified from "FW" (Firewall) / "IPS" (Intrusion Detection System) and the like. In other words, a security expert assesses whether an alert is an incident or something that does not require attention, such as false positives. SOC security experts refer to each device log of the monitored system, external threat information, malware risk assessment, etc., and judge the importance of alerts based on their own knowledge and experience.

In order to keep SOC operations stable in the future in response to the ever-increasing number of cyber attacks and the scale of monitored systems, automate or support the work of assessing the risk of security alerts. Is desired. Such a problem is not limited to the security aspect, but is the same even if it is an alert due to a failure of a device or the like.

Failure monitoring of the entire information system including server equipment and network equipment is performed by an operation center (information system department, network operation center (NOC: Network Operation Center), etc.) where specialized staff are stationed. Here, the log information generated from each device is linked with the alert information due to performance degradation, etc., and it is determined whether the alert is temporary, due to a service change, or due to a device failure. There must be. As in the case of SOC, the failure monitoring specialists judge the importance of alerts based on their own knowledge and experience.

With the progress of business automation, various services are being added to information systems more than ever, so the above log information and alert information are increasing. Therefore, it is desired to significantly improve the operational capacity of the operation center, and in order to continue the operation stably in the future, the task of evaluating the importance of alerts should be automated or this should be done in the same way as the SOC task. Is desired to support.

Against this background, statistical methods and machine learning techniques are used, with log statistics calculated from past alert importance judgment results as objective variables, past logs of each device, and externally disclosed information as explanatory variables. There is known a method of extracting factors that have influenced the importance judgment, generating a prediction model formula, and automatically predicting the importance of a newly generated alert using the prediction model formula. Considering that the system to be monitored is becoming larger and the methods of cyber attacks are changing and increasing day by day in the case of security, the log items of each device to be referred to and the items of external public information are extremely diverse. And change.

Patent Document 1 is an analyzer comprising a processor and a storage device for storing a prediction model formula for predicting a result for a factor of the event group, wherein the processor is a first event in the event group. The prediction error of the first predicted value is calculated based on the first predicted value obtained by giving the first appearance frequency to the factor of the above to the prediction model formula and the result corresponding to the first appearance frequency. The factor of the first event is based on the correlation between the prediction error calculation process, the second appearance frequency for the factor of the second event in the event group, and the prediction error calculated by the prediction error calculation process. An analyzer characterized by executing an error factor extraction process for extracting an error factor of the prediction error from the above is disclosed.

Japanese Unexamined Patent Publication No. 2019-144970

The technique disclosed in Patent Document 1 may include explanatory variables that do not contribute to prediction.

The analysis system according to the first aspect of the present invention includes an input data creation unit that creates a plurality of explanatory variables using information about an event and creates input data including the plurality of explanatory variables and the result of the event. A prediction unit that creates a prediction model formula that predicts the value of the objective variable that is the result of the event using the plurality of explanatory variables, and an exclusion candidate that is a candidate for the explanatory variable that does not need to be used in the prediction model formula. It includes an explanatory variable evaluation unit for calculating variables and an output unit for outputting the exclusion candidate variables.
The analyzer according to the second aspect of the present invention includes an input data creation unit that creates a plurality of explanatory variables using information about an event and creates input data including the plurality of explanatory variables and the result of the event. A prediction unit that creates a prediction model formula that predicts the value of the objective variable that is the result of the event using the plurality of explanatory variables, and an exclusion candidate that is a candidate for the explanatory variable that does not need to be used in the prediction model formula. It includes an explanatory variable evaluation unit for calculating variables and an output unit for outputting the exclusion candidate variables.
The analysis method according to the third aspect of the present invention is an analysis method executed by an analyzer, in which a plurality of explanatory variables are created using information about an event, and the plurality of explanatory variables and the result of the event are included. Creating input data, creating a prediction model formula that predicts the value of the objective variable that is the result of the event using the plurality of explanatory variables, and the explanation that does not need to be used in the prediction model formula. It includes calculating an exclusion candidate variable that is a candidate for a variable and outputting the exclusion candidate variable.

According to the present invention, it is possible to assist human beings in determining explanatory variables that are not used in the prediction model formula.

A block diagram showing a system configuration example of the analysis system according to the first embodiment. A block diagram showing hardware configuration examples of various computers shown in FIG. A block diagram showing a functional example of the integrated management device in the first embodiment. A block diagram showing a functional configuration example of the alert analyzer in the first embodiment. Sequence diagram showing an operation example of input data creation Explanatory diagram showing an example of an alert collection table Explanatory diagram showing an example of a log collection table Explanatory diagram showing an example of a calculated resource amount table Explanatory diagram showing an example of an input data table Explanatory diagram showing an example of the setting screen display of the alert analyzer Sequence diagram showing an example of the operation to extract the explanatory variables to be excluded from the creation target Explanatory diagram showing an example of the correlation degree evaluation table Explanatory diagram showing an example of an explanatory variable evaluation table Explanatory diagram showing an example of a cross-correlation degree evaluation table Explanatory diagram showing an example of an evaluation table in a group of explanatory variables Explanatory diagram showing an example of the output screen display of the integrated management device or the alert analysis device. Flow chart showing an example of the predictive impact evaluation procedure of the alert analyzer A block diagram showing a system configuration example of the analysis system according to the second embodiment. A block diagram showing a functional configuration example of the alert analyzer according to the second embodiment.

-First embodiment-
Hereinafter, the first embodiment of the analysis system will be described with reference to FIGS. 1 to 17.

FIG. 1 is a diagram showing a configuration of an analysis system 1 according to the first embodiment. The analysis system 1 includes an SOC 130 and an alert analyzer 135. The analysis system 1 analyzes one or more monitoring targets to protect them from security threats such as cyber attacks. In this embodiment, a case where the analysis system 1 targets the first monitoring target system 100a as a monitoring target will be described. Although security monitoring will be described below, the analysis system 1 may monitor other alerts such as hardware failure and system malfunction. The SOC 130 includes an alert management device 131, a log management device 132, an integrated management device 133, and a second network 134.

The first monitoring target system 100a includes a first network 110, one or more client terminals 111, a business server 112, a network monitoring device 113, a firewall 114, and a proxy server 115. The firewall 114 may have an intrusion prevention function. The processing target of the first monitored system 100a is not limited in the present invention, and the first monitored system 100a is, for example, a computer system in the financial field or a computer system in the IT field. However, the first monitored system 100a may be related to social infrastructure such as a power supply system and a water and sewage management system. The first monitoring target system 100a monitors hardware and software, and when it detects an attack on a server or fraud such as a virus, it creates an alert and notifies the SOC 130.

The first network 110 is, for example, a bus, and the client terminal 111, the business server 112, the network monitoring device 113, the firewall 114, the proxy server 115, and the SOC 130 are connected to each other so as to be able to communicate with each other. The firewall 114 is connected to the external network 116. The external network 116 is, for example, a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet.

The second network 134 is, for example, a bus, to which an alert management device 131, a log management device 132, an integrated management device 133, and an external network 116 are connected. Further, the alert analyzer 135 and the external threat information database 136 are connected to the SOC 130 via the external network 116. The alert analyzer 135 and the SOC 130 are generally connected by secure communication such as a VPN (Virtual Private Network). The external threat information database 136 is a Web server that provides information on threats.

The alert management device 131 identifies an alert when it receives an event such as virus detection, abnormal system behavior detection, connection detection with an unregistered device, failure, or failure from the first monitored system 100a as an alert. Register the management information of. The alert management information includes the date and time when the alert occurred, the type of the alert target, and the address of the alert target. The "type of alert target" can also be rephrased as the "type of alert source".

SOC security experts (hereinafter referred to as experts) refer to the external threat information database 136 and evaluate each alert. Then, the expert attaches the alert classification, which is the classification information, to each alert based on the evaluation result. The alert classification is, for example, one of flag 0 and flag 1. An alert with flag 0 has a high risk of leaking information and leading to a security incident such as a system runaway. The alert to which the flag 1 is attached is not high in risk, for example, the countermeasure has been taken or the alert is only a false alarm, and it is not necessary to pay much attention. The alert classification given by the expert is used in the process described later.

The log management device 132 acquires an access log, an authentication log, and the like from the first monitored system 100a, and records and manages the history of past logs. The log is historical information indicating information such as when, which computer in the first monitored system 100a sends and receives what data to which communication partner.

The integrated management device 133 includes an alert collection table 600, a log collection table 700, a calculation resource amount table 800, and an input data table 900. The integrated management device 133 collects alert management information from the alert management device 131 and registers it in the alert collection table 600, which will be described later. Further, the integrated management device 133 collects log information from the log management device 132 and registers it in the log collection table 700, which will be described later. Further, the integrated management device 133 creates an input data table 900, which will be described later, based on the information described in the alert collection table 600 and the log collection table 700, and transmits the input data table 900 to the alert analysis device 135. The calculated resource amount table 800 will be described later.

The alert analyzer 135 learns the combination of the content of the alert and the alert classification determined by the expert, and creates a model for evaluating another alert based on the learning result. The alert analyzer 135 analyzes the evaluation target alert which is a newly generated alert based on this model, calculates a predicted value for determining the alert classification, and notifies the integrated management device 133 of this. The expert who confirmed the display of the integrated management device 133 can confirm this predicted value and determine the formal alert classification of the alert to be evaluated.

The external threat information database 136 discloses threat information on the Internet. Threat information may include malware, program vulnerabilities, spam, and malicious URLs (Uniform Resource Identifiers). However, it is not essential that the external threat information database 136 exists outside the SOC 130, and the external threat information database 136 may be included in the SOC 130.

FIG. 2 is a hardware block diagram of the computer 200 applied to the system shown in FIG. The computer 200 is applied to each of the client terminal 111, the business server 112, the network monitoring device 113, the firewall 114, the proxy server 115, the alert management device 131, the log management device 132, the integrated management device 133, and the alert analysis device 135. To. The computer 200 includes a processor 201, a storage device 202, an input device 203, an output device 204, and a communication interface 205. The processors 201 and the like are connected to each other by the bus 206. The storage device 202 may be a non-temporary or temporary recording means for storing programs and data.

The configuration shown in FIG. 2 is an example, and for example, a rewritable logic circuit or an integrated circuit for a specific application may be used instead of the processor 201. Further, the client terminal 111, the business server 112, the network monitoring device 113, the firewall 114, the proxy server 115, the alert management device 131, the log management device 132, the integrated management device 133, and the alert analysis device 135 do not have the same configuration. May be good.

The storage device 202 may be provided with information such as an operation program for realizing each process from the outside via the storage medium 291. Further, the computer 200 may have a function of connecting to an external server 292 via the communication interface 205. The external server 292 is a server computer that provides information such as an operation program for realizing each process, and stores the information in a recording medium such as a storage device 293. The communication line 299 is a communication line such as the Internet and personal computer communication, or a dedicated communication line.

The external server 292 reads information such as an operation program from the storage device 293 and transmits it to the computer 200 via the communication line 299. That is, the program is transmitted as a data signal via a carrier wave and via a communication line 299. As described above, the program for operating the computer 200 can be supplied as a computer-readable computer program product in various forms such as a recording medium and a data signal (carrier wave).

FIG. 3 is a block diagram showing a functional configuration example of the integrated management device 133 according to the first embodiment. The integrated management device 133 includes an alert collection unit 301, a log collection unit 302, an input data creation unit 303, and an output data display unit 304. Each block of the alert collecting unit 301 and the like is a functional module realized by the processor 201 executing a program stored in the storage device 202. Therefore, each of the alert collecting unit 301 and the like may be paraphrased as an alert collecting module, an alert collecting means, an alert collecting circuit, an alert collecting unit, an alert collecting element, and the like. In addition, each functional module may be realized by hardware.

The alert collection unit 301 receives the alert management information from the alert management device 131 and registers it in the alert collection table 600, which will be described later. The log collection unit 302 refers to the information registered in the alert collection table 600, and obtains log statistics as explanatory variables from the log information received from the log management device 132 and the external threat information received from the external threat information database 136. It is calculated and registered in the log collection table 700, which will be described later. Further, the log collecting unit 302 registers the calculation resource amount required for the calculation of the log statistics, that is, the calculation time in the calculation resource amount table 800 described later.

The input data creation unit 303 creates the input data table 900 with reference to the alert collection table 600 and the log collection table 700, and transmits the input data table 900 to the alert analyzer 135. The output data display unit 304 displays the output data received from the alert analyzer 135 together with the contents of the calculation resource amount table 800. The security expert 310 can check the display screen of the output data display unit 304 and arbitrarily set the explanatory variable to be created, that is, the type of log statistics.

FIG. 4 is a block showing a functional configuration example of the alert analyzer 135 according to the first embodiment. The alert analyzer 135 first creates an alert evaluation model using, for example, machine learning by artificial intelligence, for example, predictive coding. Next, the alert analyzer 135 updates the evaluation model by continuing the learning. The alert analyzer 135 further calculates the predicted value of the classification for the alert based on the evaluation model. The alert analyzer 135 configures an alert and learns the association between the data elements that characterize the alert and the classification flags. Data elements are log information related to alerts. The evaluation model is, for example, a weight in relation to a flag for each of a plurality of data elements, or a model of a mathematical formula.

The alert analyzer 135 includes a model unit 400, a prediction unit 401, an explanatory variable evaluation unit 402, a prediction impact evaluation unit 403, and an output unit 404. Each block of the prediction unit 401 and the like is a functional module realized by the processor 201 executing a program stored in the storage device 202. Therefore, each of the prediction unit 401 and the like may be paraphrased as a prediction module, a prediction means, a prediction circuit, a prediction unit, a prediction element, and the like. The functional module may be realized by hardware.

The prediction unit 401 calculates the predicted value of the new alert based on the prediction model formula. This predicted value is transmitted to the integrated management device 133 via the output unit 404. Since the expert 310 can refer to the predicted value before actually processing the new alert, the alert can be processed efficiently, and the alert of high importance can be eliminated.

The explanatory variable evaluation unit 402 evaluates the explanatory variables using one or a plurality of predetermined extraction methods, and calculates exclusion candidate variables that are candidates for the explanatory variables that do not need to be used in the prediction model formula. However, in the present embodiment, a specific explanatory variable is not evaluated, but in other words, all the explanatory variables are evaluated without limiting the target, and the explanatory variables that meet the predetermined conditions are output as exclusion candidate variables. do.

The predictive impact evaluation unit 403 evaluates the predictive impact when the explanatory variables extracted by the explanatory variable evaluation unit 402 are excluded from the creation target. The predictive impact is the accuracy when no particular explanatory variable is used in the predictive model formula. For example, the correct answer rate of a test alert using a predictive model formula that does not use a specific explanatory variable is defined as the predictive impact. More specifically, when 9 out of 10 test alerts using the prediction model formula that does not use a specific explanatory variable are answered correctly, the prediction influence degree is "90%".

The evaluation results of the explanatory variable evaluation unit 402 and the prediction impact evaluation unit 403 are transmitted to the integrated management device 133 via the output unit 404. The expert 310 can confirm the display screen of the evaluation result and arbitrarily set the explanatory variable to be created, that is, the type of log statistics. The output unit 404 collectively transmits the output results of the prediction unit 401, the explanatory variable evaluation unit 402, and the prediction impact evaluation unit 403 to the integrated management device 133.

Next, the operation of the analysis system 1 will be described. FIG. 5 is a sequence diagram showing an operation example in which the integrated management device 133 creates input data. The alert collecting unit 301 determines the range in which the alert management information is collected from the alert management device 131 for learning the alert (step S501). The alert management device 131 accumulates alert management information each time an alert is received from the first monitoring target system 100a.

The alert collection unit 301 collects alert management information from the alert management device 131 (S502). The alert collection unit 301 collects, for example, management information of alerts that have occurred in the range of the collection period, for example, 30 days, from the alert with the oldest occurrence date and time among the alerts that have not been captured. The alert collection unit 301 registers the collected alert management information in the alert collection table 600 (step S503). Here, the alert collection table 600 will be described.

FIG. 6 is an explanatory diagram showing an example of the alert collection table 600. The alert collection table 600 is stored in a predetermined storage area of the integrated management device 133. The alert collection table 600 includes fields of an alert identifier 601, an occurrence date and time 602, a system 603, an alert target 604, a communication partner 605, a determination result 606, and an alert classification 607. Of the information stored in these fields, the determination result 606 and the information stored in the fields other than the alert classification 607 are obtained by analyzing the alert acquired by the alert management device 131 from the first monitored system 100a. ..

The alert identifier 601 stores identification information that uniquely identifies the alert. Information on the date and time when the alert occurred is stored in the occurrence date and time 602. The system 603 stores the identification information of the monitored system in which the alert has occurred. The alert target 604 stores information that identifies the source of the alert, for example, information that identifies the device. That is, the information stored in the alert target 604 shows more details of the information stored in the system 603. The communication partner 605 stores information indicating the destination of the data transmitted by the target specified by the alert target 604 or the transmission source of the data transmitted to the target specified by the alert target 604.

The determination result 606 stores information on the response taken by the expert to the alert. The determination result is, for example, one of "false positive and determination", "unaddressed attack and determination", "addressed attack and determination", and "unprocessed". The alert classification 607 stores information indicating the importance of the alert input by the expert based on the processing result 606. "1" or "0" is stored in the alert classification 607, "1" indicates that the alert is important and the risk is high, and "0" indicates that the alert is not important and not dangerous, so do not pay attention. Indicates that it may be. The latter is selected, for example, when the alert is a false alarm or when the alert has been addressed.

The explanation will be continued by returning to FIG. When the registration to the alert collection table 600 is completed, the alert collection unit 301 notifies the log collection unit 302 of the alert information (step S504). The log collection unit 302 accesses the log management device 132, refers to the log information group, and extracts log information related to the alert (S505). Further, the log collecting unit 302 accesses the external threat information database 136 and extracts the external threat information related to the alert (S506). The log collection unit 302 calculates log statistics related to alerts, that is, explanatory variables, based on log information and external threat information, and registers them in the log collection table 700. Then, the log collecting unit 302 stores the calculation time required for calculating the log statistics in the calculation resource amount table 800 (S508).

FIG. 7 is an explanatory diagram showing an example of the log collection table 700. The log collection table 700 is recorded in a predetermined storage area of the integrated management device 133. Each entry in the log collection table 700 has fields for an alert identifier 601, an aggregation date and time 702, a proxy server log 703, a business server log 704, and external threat information 705. The log collection table 700 may include logs related to other computers in the first monitored system 100a, such as the client terminal 111, the firewall 114, and the network monitoring device 113.

The log statistics, which are the statistical information of the logs included in the log collection table 700, described below correspond to the explanatory variables. In the example shown in FIG. 7, the number of cache misses 731, the number of abnormal responses 732, the number of abnormal responses 741, the number of accesses 742, the IP address risk 751, and the URL risk 752 are explanatory variables.

The log collection table 700 stores statistics for one or more logs corresponding to each alert, as detailed below. In other words, the log collection table 700 does not store one log corresponding to one alert, but stores information that aggregates a plurality of logs near the time when the alert is issued. Specifically, statistical information such as a log that goes back by a predetermined time from the time when the alert is issued and aggregated at a predetermined time interval is stored in the log collection table 700. In the example shown in FIG. 7, the predetermined time is "1 hour" and the time interval is "10 minutes", and seven statistical information corresponding to one alert is stored.

The alert identifier 601 stores information that identifies the alert corresponding to the statistics of the entry, that is, the identifier of the alert. The aggregation date and time 702 stores information on the last time of the log used to create the statistics shown in the entry. As mentioned above, in the example of FIG. 7, the time interval is set to "10 minutes", so in the example shown in the first entry, the aggregation date and time 702 is "2018/10/10 12:57", so "October 10, 2018" It is a total of the logs received in 10 minutes from "12:48" to "12:57".

The data elements that make up the alert may be defined as a range of logs prior to the date and time the alert occurred. That is, the log information 703 and 704 of the aggregation date and time of the alert having the same identifier are the log statistics related to the alert, that is, the data element constituting the alert. Further, the external threat information 705 may also be used as the alert configuration information.

As shown in the example of FIG. 6, the alert occurrence date and time 602 in which the alert identifier 601 is “Alert_001” is “2018/10/10 13:57”. In the following description of FIG. 7, the date is omitted and only the time is described. The aggregated date and time 702 of the alert of the entry whose alert identifier 601 is "Alert_001" is "12:57" which is one hour back from "13:57" of the occurrence date and time 602 and "12:57" in 10-minute increments. 13:07 ”,“ 13:17 ”,“ 13:27 ”,“ 13:37 ”,“ 13:47 ”, and“ 13:57 ”.

The proxy server log 703 has a cache miss count 731 and an abnormal response count 732 as subfields. The number of cache misses 731 stores the number of cache misses by the proxy server 115 at the aggregation date and time 702. The number of abnormal responses 732 stores the number of times that the proxy server 115 received the abnormal response at the aggregation date and time 702. The subfield of the proxy server log 703 is not limited to the number of cache misses 731 and the number of abnormal responses 732, and may include, for example, the number of communication bytes.

The business server log 704 has an abnormal response count 741 and an access count 742 as subfields. The number of abnormal responses 741 stores the number of times that the business server 112 received the abnormal response at the aggregation date and time 702. The number of accesses 742 includes the number of times that the business server 112 is accessed by another computer 200 during the aggregation period specified by the aggregation date and time 702. The subfield of the business server log 704 is not limited to the number of abnormal responses 741 and the number of accesses 742, and may include, for example, the number of authentication failures.

The external threat information 705 has an IP address risk level 751 and a URL risk level 752 as subfields. The IP address risk level 751 includes an index value indicating the risk level of the IP address in the external threat information database 136 when the communication partner 605 of the alert target 604 at the aggregation date and time 702 is specified by the IP address. Stored. This index value is, for example, 6 levels of "0 to 5", and "5" means that the risk is the highest.

In the URL risk level 752, when the communication partner 605 of the alert target 604 at the aggregation date and time 702 is specified by the URL, an index value indicating the risk level of the URL in the external threat information database 136 is stored. .. This index value is, for example, 6 levels of "0 to 5", and "5" means that the risk is the highest. The subfield of the external threat information 705 is not limited to the IP address risk level 751 and the URL risk level 752, and may include, for example, the vulnerability level of the terminal.

FIG. 8 is an explanatory diagram showing an example of the calculation resource amount table 800. The calculated resource amount table 800 is recorded in a predetermined storage area of the integrated management device 133. The calculation resource amount table 800 has a field of the explanatory variable name 801 and the calculation time 802 required for the calculation.

The explanatory variable name 801 stores the item names of the respective explanatory variables such as the cache miss count 731 of the proxy server log 703 shown in the log collection table 700 of FIG. The calculation time 802 required for the calculation stores the calculation time required to calculate the log statistics from the log information and the like. As the information stored in the calculation time 802 required for the calculation, for example, the time required for the calculation at the time of calculation is stored, and the average value thereof is stored. The entry in which the explanatory variable name 801 is "the number of proxy server cache misses" indicates that the calculation time 802 required for the calculation was "50.3 seconds".

Return to FIG. The input data creation unit 303 creates input data by referring to the alert collection table 600 and the log collection table 700 registered by the alert collection unit 301 and the log collection unit 302, and registers the input data in the input data table 900 (S511). By the above procedure, the input data to the alert analyzer 135 is created.

FIG. 9 is an explanatory diagram showing an example of the input data table 900. The input data table 900 is recorded in a predetermined storage area of the integrated management device 133. The input data table 900 has fields of an alert identifier 601, a data type 902, a judgment result classification 903, an aggregation date and time 904, a system 905, a proxy server log 906, a business server log 907, and an external threat information 908. ..

In the data type 902, one of "learning" and "test" indicating the type of handling as data for the captured alert is stored. In the following, an alert for which "learning" is set is referred to as a "learning alert", and an alert for which "test" is set is referred to as a "test alert". Learning alerts are used to evaluate the association between alert data elements and alert classification (classification) and to create and update predictive model formulas for predicting alert classification. Test alerts are used to test the created predictive model formulas.

It is optional to set the received alert to "learn" or "test". The input data creation unit 303 may determine an alert in which "learning" is set and an alert in which "test" is set, based on a predetermined rule. The ratio of "learning" and "test" is not particularly limited, and may be appropriately set by the input data creation unit 303.

The determination result classification 903 corresponds to the classification 607 in the alert collection table 600 illustrated in FIG. 6, and the same value is stored. The information stored in the determination result classification 903 corresponds to the objective variable in the input data. The aggregation date and time 904 corresponds to the aggregation date and time 702 in the log collection table 700 illustrated in FIG. 7, and the same value is stored. The system 905 corresponds to the system 603 of the alert collection table 600 illustrated in FIG. 6, and the same value is stored and recorded for each aggregation date and time 904.

The proxy server log 906, the business server log 907, and the external threat information 908 correspond to the proxy server log 703, the business server log 704, and the external threat information 705 in the log collection table 700, and their subfields also correspond to them. , The same value is stored. The system 905, the proxy server log 906, the business server log 907, and the external threat information 908 correspond to explanatory variables.

Next, the setting items for the alert analyzer 135 will be described. FIG. 10 is an explanatory diagram showing an example of a setting screen display of the alert analysis device 135. The screen 1000 shows two setting items of the extraction method 1010 and the threshold value 1020 in the setting tab 1001. The extraction method 1010 sets an extraction method for explanatory variables to be excluded from the creation target. The threshold value 1020 sets various threshold values.

In the extraction method 1010, the necessity of use can be independently set for each of the first method 1011 and the second method 1012, and the third method 1013. Since this item is the setting of the extraction method of the explanatory variables to be excluded from the creation target, the explanatory variables extracted by the selected extraction method are excluded from the creation target. The first method 1011 is a method of extracting an explanatory variable having a variance = 0. The second method 1012 is a method of extracting explanatory variables whose degree of correlation with the objective variable is equal to or less than the threshold value. The third method 1013 is a method of extracting from an explanatory variable group in which the degree of correlation between explanatory variables (hereinafter referred to as “cross-correlation degree”) is equal to or greater than a threshold value.

At the threshold value 1020, the threshold values of the uncorrelated threshold value 1021, the p threshold value 1022, and the strong correlation threshold value 1023 can be set numerically. The uncorrelated threshold value 1021 is a threshold value of the degree of correlation with no correlation, and when the degree of correlation is smaller than the value of the uncorrelated threshold value 1021, it is determined that there is no correlation. The p-threshold value 1022 is a threshold value of the p-value as the significance level. The p-value is an index generally used in statistics, and is the probability that the test statistic will be the value under the null hypothesis. The strong correlation threshold value 1023 is a threshold value of the degree of cross-correlation that makes a strong correlation, and when the degree of cross-correlation is larger than the value set in the strong correlation threshold value 1023, it is determined that the strong correlation has a strong correlation.

Next, the operation of the alert analyzer 135 will be described. FIG. 11 is a sequence diagram showing an example of an operation of extracting explanatory variables to be excluded from the creation target. The input data creation unit 303 of the integrated management device 133 transmits the registered input data table 900 to the alert analysis device 135. The input data table 900 is shared by the prediction unit 401, the explanatory variable evaluation unit 402, and the prediction impact evaluation unit 403 (S1101, S1102, S1103). The prediction unit 401 selects an alert identifier to which "learning" is assigned to the data type 902 of the input data table 900, and extracts items corresponding to the objective variable and the explanatory variable from those entries. Further, the prediction unit 401 calculates the degree of correlation between the appearance frequency of the objective variable and the explanatory variable and registers it in the correlation degree evaluation table 1200 (S1104). However, the prediction unit 401 may more simply calculate the degree of correlation between the objective variable and the explanatory variable. The process of S1104 will be described in detail with reference to FIG.

FIG. 12 is an explanatory diagram showing an example of the correlation degree evaluation table 1200. The correlation degree evaluation table 1200 is registered in a predetermined storage area of the alert analyzer 135. The correlation degree evaluation table 1200 has an explanatory variable name 1201, a range 1202, a correlation degree 1203, and a p-value 1204 as fields. The explanatory variable name 1201 stores the name of the explanatory variable related to the alert to which "learning" is given. This corresponds to the proxy server cache miss count 961 and the like in the input data table 900.

The range 1202 stores a range of values that the explanatory variable name 1201 can take. For example, a predetermined value is stored in the range 1202. The data element of the alert is composed of a combination of the explanatory variable name 1201 and the range 1202. The correlation degree 1203 stores an index for evaluating the correlation between the data element and the judgment result classification 903 corresponding to the objective variable, that is, a value corresponding to the correlation degree. This will be described in detail below.

The values stored in the correlation degree 1203 are, for example, the appearance frequency p of the range 1202 obtained by dividing the number of appearances of the range 1202 in the feature amount (data element) of the learning alert by the total number of times of the feature amount, and the classification of the judgment result classification 903. The correlation coefficient R1 with the degree q. The correlation coefficient R1 is calculated by the following (Equation 1) using the standard deviation σp of the appearance frequency p, the standard deviation σq of the classification degree q, and the covariance Spq of the appearance frequency p and the classification degree q. ..

R1 = Spq / (σp × σq) ... (Formula 1)

Hereinafter, a method for calculating the appearance frequency p and the classification degree q will be specifically described using the example of the input data table 900 shown in FIG.

It is as follows when the alert identifier 601 is "Alert_001" and the explanatory variable name 1201 explains "the number of proxy server cache misses". According to FIG. 9, the number of cache misses 961 of the proxy server log 906 in which the alert identifier 601 is “Alert_001” is “3” at “12:57”, “4” at “13:07”, and “13:57”. Is "4" in "". Here, the number of cache misses 961 in which the aggregation date and time 904 is "13:17" to "13:47", which is omitted in FIG. 9, is set to a value other than "3" and "4". In this case, the number of occurrences of the range 1202 "3" to "4" in the cache miss count 961 of the proxy server log 906 in which the alert identifier 601 is "Alert_001" is three times.

As described above, the total number of cache misses 961 of the proxy server log 906 in which the alert identifier 601 is "Alert_001" is 7 times. Therefore, the appearance frequency p of the range 1202 of “3” to “4” in the cache miss count 961 of the proxy server log 906 in which the alert identifier 601 is “Alert_001” is “3/7”. Further, the classification degree q in which the alert identifier 501 is “Alert_005” is “0” in the judgment result classification 903. The above is a specific calculation method of the appearance frequency p and the classification degree q.

The prediction unit 401 obtains a combination of the appearance frequency p and the classification degree q for each alert identifier 601 whose data type 902 is “learning”. Next, the prediction unit 401 obtains the standard deviation σp of the appearance frequency p from the frequency p at which the phenomenon of each item appears. Then, the prediction unit 401 obtains the standard deviation σq from the classification degree q of each of the plurality of alerts, and obtains the covariance Spq. Further, according to the above (formula 1), the prediction unit 401 changes the value range 1202 of the explanatory variable name 1201 “proxy server cache miss count” for the alert identifier 601 whose data type 902 is “learning” to “3” to “4”. The corresponding correlation coefficient (R1 = −0.54) is calculated.

When the correlation coefficient R1 is positive (R1> 0), it means that the alert is correct, that is, the alert indicates a danger such as a virus attack. Further, the larger the value of the correlation coefficient R1, the higher the degree of danger. On the other hand, the fact that the correlation coefficient R1 is negative (R1 <0) indicates that the alert is not so dangerous, such as that the alert is based on a false alarm. Further, the smaller the correlation coefficient R1, that is, the larger the negative absolute value, the higher the false alarm to some extent. In this way, since the degree of correlation is calculated for each of multiple data elements of the alert, it is possible to evaluate the importance of the alert itself based on the score of the alert by integrating the correlation coefficient of each data element. can. The term "integration" here may be paraphrased as fusion, combination, unification, synthesis, and the like.

The p-value 1204 is an index for testing uncorrelatedness. It is an index generally used in statistics, and is the probability that the test statistic will be its value under the null hypothesis. The statistical test statistic t is calculated by the following (Equation formula 2) by the correlation coefficient R1 and the number of sample samples m.

t = R1 × √ (m-2) / √ (1-R1 ^ 2) ・ ・ ・ (Formula 2)

The p-value 1204 is calculated as the probability that a value equal to or greater than the absolute value of the statistical test statistic 5 will occur in the t distribution. When the p-value reaches a certain threshold, for example 0.05 or more, the result of the correlation coefficient R1 calculated in advance is rejected at the significance level of 5%, and it can be said that the two variables are uncorrelated.

Return to FIG. The prediction unit 401 creates and stores a prediction model formula for calculating the predicted value of the classification degree based on the appearance frequency p and the classification degree q (S1106). In this prediction model formula, the objective variable Y is the judgment result classification 903, and the feature quantity Xi (i = 1, 2, ..., N) is "P (explanatory variable name 1201, range 1202)". However, P (Z) represents the appearance frequency p of the event Z. For the feature quantity Xi, refer to the correlation degree evaluation table 1200, X1 = P (proxy server cache miss count, “0” to “3”), X2 = P (proxy server cache miss count, “3” to “4”). "), Etc.

The prediction unit 401 creates a multiple regression equation as shown below (Formula 3) as an example of the prediction model equation. “N” is the total number of combinations with the explanatory variable name 1201 and the range 1202, that is, the total number of events Z.

Here, the value of the objective variable Y for the entry j of the alert which is the input data is set to "y_j". The entry j is, for example, a combination of the objective variable Y and the feature amount Xi regarding "Alert_001". Further, the value of the feature amount X1 is set to "x1_j", the value of the feature amount X2 is set to "x2_j", ..., And the value of the feature amount Xn is set to "xn_j". At this time, each coefficient "b0", "b1", "b2", ..., "Bn" of the above (formula 3) can be obtained by a determinant as shown below (formula 4) as an example. In the following (Formula 4), "j" indicates any entry of the alert entry "1" to "k" which is input data.

The prediction model formula based on the above (formula 3) is an example of a prediction model. This method of creating a predictive model formula is an example, and may be derived using a known method such as regularization, decision tree, ensemble learning, neural network, Bayesian network, or the like. The alert classification is set to "0" or "1" as described above. The predicted value of classification is set as a value of "0" or more and "1" or less. The prediction unit 401 may determine the degree of classification based on this prediction value and allow an expert to refer to it. The description of FIG. 11 will be continued.

The prediction unit 401 calculates the prediction value and the accuracy based on the learning alert and the test alert of the input data table 900 by using the above-mentioned prediction model formula (S1107). Then, the prediction unit 401 transmits the calculated predicted value and the system to the output unit 404 (S1108). The prediction unit 401 calculates the prediction value “y_j” by giving the feature quantities “x1_j”, “x2_j”, ..., “Xn_j” of this alert to the prediction model formula.

The prediction unit 401 determines the prediction threshold value based on a human designation or a predetermined rule. The prediction threshold value is an index for classifying the calculated predicted value into either "0" or "1". The prediction unit 401 determines "0" if the predicted value is smaller than the predicted threshold value and "1" if the predicted value is equal to or higher than the threshold value. The prediction unit 401 refers to the judgment result classification 903 of the input data table 900, compares it with the judgment result of the predicted value, and calculates the accuracy of the prediction. In the case of security monitoring, from the viewpoint of preventing dangerous alerts from being overlooked, the prediction threshold is adjusted so that the hit rate of alerts classified as "1" is 100%, and at that time, it is predicted to be "0". The accuracy rate of the classified alerts may be used as the accuracy.

Next, the explanatory variable evaluation unit 402 evaluates the explanatory variables according to the settings in the extraction method 1010 of FIG. The explanatory variable evaluation unit 402 first refers to the item of each explanatory variable in the input data table 900, for example, the value of the proxy server log cache miss count 961, and calculates the distribution of the explanatory variable and the degree of correlation with the objective variable. , Registered in the explanatory variable evaluation table 1300 (S1109).

FIG. 13 is an explanatory diagram showing an example of the explanatory variable evaluation table 1300. The explanatory variable evaluation table 1300 is registered in a predetermined storage area of the alert analyzer 135. The explanatory variable evaluation table 1300 has fields for the explanatory variable name 1301, the variance 1302, and the degree of correlation 1303 with the objective variable. The explanatory variable name 1301 stores the name of each explanatory variable in the input data table 900. The variance 1302 is obtained by referring to the items of each explanatory variable in the input data table 900, for example, the value of the proxy server log cache miss count 961, and calculating the mean value and the variance on the assumption that a normal distribution is followed. Value is stored.

The correlation degree 1303 with the objective variable refers to the correlation degree evaluation table 1200, and stores the value having the maximum absolute value of the correlation degree 1203 among the explanatory variable names 1201 having the same name. For example, for an entry in which the explanatory variable name 1301 is the “number of proxy server cache misses”, the variance 1302 is calculated to be “54.2” and the correlation degree 1303 with the objective variable is calculated to be “−0.54”.

However, the value of the correlation degree 1203 that is equal to or less than the value of the uncorrelated threshold value 1021 in the threshold value 1020 of FIG. 10 is assumed to have no correlation and the correlation degree is 0. Further, since the reliability value having a p-value 1204 equal to or higher than the value of the p-threshold value 1022 at the threshold value 1020 has low reliability, the value of the correlation degree 1203 is similarly set to the correlation degree = 0. For example, an entry whose explanatory variable name 1301 is “external threat information IP address risk” is calculated to have a correlation degree 1303 with the objective variable “0” in relation to the above threshold value.

Return to FIG. The explanatory variable evaluation unit 402 refers to the variance 1302 of the explanatory variable evaluation table 1300, extracts the explanatory variable having a variance of zero, and transmits the explanatory variable name to the output unit 404 (S1110). A zero variance means that the explanatory variables do not change and always take a constant value. Therefore, there is no correlation with the objective variable.

Next, the explanatory variable evaluation unit 402 extracts an explanatory variable whose degree of correlation with the objective variable is equal to or less than the threshold value (S1111), and transmits the name of the extracted explanatory variable to the prediction impact evaluation unit 403 (S1112). The explanatory variable evaluation unit 402 refers to the correlation degree 1303 with the objective variable in the explanatory variable evaluation table 1300, and extracts the explanatory variables having the correlation degree 1303 of zero. Since the value of the correlation degree 1203 having the value of the uncorrelated threshold 1021 or less at the threshold 1020 is already set to zero, extracting the explanatory variable having the correlation degree 1203 of zero correlates with the objective variable. It is equivalent to extracting explanatory variables whose degree is less than or equal to the threshold.

Next, the predictive impact evaluation unit 403 calculates the predictive impact when the explanatory variable name received in S1112 is excluded (S1113), and outputs the predictive impact together with the explanatory variable name to the output unit 404. (S1114). The prediction influence degree calculated in S1113 is the prediction accuracy calculated by the same means as described in S1106 and S1107, excluding the explanatory variables notified in S1112 from the input data table 900. Then, the explanatory variable evaluation unit 402 calculates the cross-correlation degree, which is the degree of correlation between the explanatory variables, and registers it in the cross-correlation degree evaluation table 1400 (S1115).

FIG. 14 is an explanatory diagram showing an example of the cross-correlation degree evaluation table 1400. The cross-correlation degree evaluation table 1400 is registered in a predetermined storage area of the alert analyzer 135. The cross-correlation degree evaluation table 1400 has fields in which explanatory variable names are listed in the row direction 1401 and the column direction 1402, respectively. The row direction 1401 and the column direction 1402 of the explanatory variable names store the names of the explanatory variables in the input data table 900, respectively. In each cell of the cross-correlation degree evaluation table 1400, the correlation coefficient R2 between the explanatory variables is registered. For example, the correlation coefficient R2 includes a standard deviation σα of the value α of the explanatory variable A, a standard deviation σβ of the value β of the explanatory variable B, and a covariance Sαβ of the value α of the explanatory variable A and the value β of the explanatory variable B. Is calculated by the following (Equation 5).

R2 = Sαβ / (σα × σβ) ... (Formula 5)

In the mutual correlation degree evaluation table 1400, for example, the correlation coefficient between the explanatory variable name “proxy server cache miss count” and the explanatory variable name “proxy server abnormal response count” is “0.87”.

Return to FIG. The explanatory variable evaluation unit 402 refers to the cross-correlation degree evaluation table 1400 and extracts a set of explanatory variables of the cross-correlation degree having a value equal to or higher than the value of the strong correlation threshold value 1023 in the threshold value 1020 of FIG. Next, the explanatory variable evaluation unit 402 creates a group of explanatory variables and registers it in the evaluation table 1500 within the group (S1116). Then, the explanatory variable evaluation unit 402 transmits the contents of the evaluation table 1500 in the group to the prediction impact evaluation unit 403 (S1117).

FIG. 15 is an explanatory diagram showing an example of the evaluation table 1500 in the group of explanatory variables. The in-group evaluation table 1500 is registered in a predetermined storage area of the alert analyzer 135. The in-group evaluation table 1500 has fields for the group 1501, the explanatory variable name 1502, and the degree of correlation 1503 with the objective variable. The group 1501 stores the name of the group, for example, a serial number from 1. The explanatory variable name 1502 stores the names of the explanatory variables belonging to the same group, that is, the explanatory variable names 1401 of the mutual correlation degree evaluation table 1400 or the values of the explanatory variables 1402. The degree of correlation with the objective variable 1503 stores the degree of correlation with the objective variable for each explanatory variable. This value is, for example, the value of the degree of correlation 1303 with the objective variable for the corresponding explanatory variable name 1301 in the explanatory variable evaluation table 1300.

The in-group evaluation table 1500 is created by referring to the cross-correlation evaluation table 1400 and extracting a set of explanatory variables of the cross-correlation degree having a cross-correlation degree threshold of 1023 or more, which is a strong correlation at the threshold value 1020. .. For example, when the explanatory variables A and B, and the explanatory variables B and C are extracted, the explanatory variables A, B, and C become one group.

Return to FIG. The predictive impact evaluation unit 403 refers to the in-group evaluation table 1500, selects one explanatory variable name 1502 for each group 1501, and calculates the predictive impact when the others in the group are excluded (). S1118), the predicted influence degree is transmitted to the output unit 404 together with the evaluation table 1500 in the group (S1119). The prediction influence degree is the prediction accuracy calculated by the same means as described in S1106 and S1107 by excluding the explanatory variables other than one selected in the group in S1118 from the input data table 900.

Finally, the output unit 404 transmits the predicted value and the evaluation result received in S1108, S1110, S1114, and S1119 to the output data display unit 304 of the integrated management device 133 (S1120). The above is the description of FIG.

FIG. 16 is an explanatory diagram showing an example of an output screen display of the integrated management device 133. The screen 1600 displays the training and test data prediction result 1610, the exclusion candidate variable, and the prediction influence degree 1620 on the tab of the explanatory variable evaluation 1601. The exclusion candidate variable is an explanatory variable that is a candidate to be excluded from the prediction model formula as described above. The training and test data prediction result 1610 displays a list of predicted values 1611 for the alert identifier and a prediction accuracy 1614 for each data type 1612. The prediction accuracy 1614 is the value notified in S1108 of FIG.

The exclusion candidate variable and the prediction influence degree 1620 display three explanatory variables 1621 having a variance = 0, an explanatory variable 1630 having a correlation with the objective variable below the threshold value, and a grouped explanatory variable 1640. .. These three correspond to the three extraction methods set in the extraction method 1010 of FIG. 10, and since all three methods are selected in the example shown in FIG. 10, all three are displayed in the example of FIG. ing. For example, when the second method 1012 is not selected in the extraction method 1010, in other words, when it is not selected, the explanatory variable 1630 whose correlation with the objective variable is equal to or less than the threshold value is not displayed in FIG.

The explanatory variable 1621 in which the variance = 0 displays the explanatory variable name 1622 notified in S1110 of FIG. 11 and the calculation resource reduction amount 1623 when the explanatory variable is excluded from the creation target. The calculation resource reduction amount 1623 displays the value of the calculation time 802 required for the calculation for the corresponding explanatory variable name in the calculation resource amount table 800 of FIG.

The explanatory variable 1630 whose correlation with the objective variable is equal to or less than the threshold value displays the explanatory variable name 1631 notified in S1114 of FIG. 11 and the predicted influence degree 1632. The calculation resource reduction amount 1623 displays the value of the calculation time 802 required for the calculation for the corresponding explanatory variable name in the calculation resource amount table 800 of FIG. It can be said that the value of the predicted influence degree 1632 represents the accuracy.

The grouped explanatory variables 1640 are the group 1641 which is the registered content of the in-group evaluation table 1500 notified in S1119 of FIG. 11, the explanatory variable name 1642, the degree of correlation 1643 with the objective variable, and the prediction at the time of selection. The influence degree 1644 and the calculation resource reduction amount 1645 at the time of selection are displayed. The predicted influence degree 1644 at the time of election displays the predicted influence degree at the time of selection, that is, when the other explanatory variables are excluded, in other words, the accuracy. In the calculation resource reduction amount 1645 at the time of selection, the description of the corresponding exclusion target in the calculation resource amount table 800 of FIG. 8 when one explanatory variable in the group is selected, that is, when the other explanatory variables are excluded. The sum of the values of the calculation time 802 required for the calculation for the variable name is displayed.

By checking the screen 1600, the expert 310 can know the names of the explanatory variables that may be excluded from the creation target, and the prediction impact (accuracy) and the amount of calculation resource reduction when the explanatory variables are excluded. Can be known. This allows the expert 310 to know the explanatory variables that have no or little effect on the alert importance prediction and may be excluded from the creation target. This allows the expert 310 to review the explanatory variable settings and, as a result, estimate the effect of reducing computational resource consumption for calculating log statistics.

FIG. 17 is a flowchart showing an example of the predicted impact degree evaluation procedure in the alert analyzer 135. When the alert analyzer 135 starts processing, it selects an entry for an alert to which "learning" is assigned to the data type 902 of the input data table 900. Then, the alert analyzer 135 extracts the items corresponding to the objective variable and the explanatory variable from those entries, and calculates the degree of correlation between the objective variable and the appearance frequency of the explanatory variable (S1701). However, the alert analyzer 135 may more simply calculate the degree of correlation between the objective variable and the explanatory variable. Next, the alert analyzer 135 creates a prediction model formula, and calculates the prediction value and the accuracy using the prediction model formula based on the learning alert and the test alert of the input data (S1702).

The alert analyzer 135 refers to the extraction method 1010 in FIG. 10 and determines whether or not the first method 1011 is selected (S1703). When the first method 1011 is selected, the alert analyzer 135 calculates the variance of each explanatory variable in the input data table 900, extracts the explanatory variable having a value of 0 (S1704), and proceeds to S1705. If the first method 1011 is not selected, the process proceeds to S1705 as it is.

Next, the alert analyzer 135 refers to the extraction method 1010 and determines whether or not the second method 1012 is selected (S1705). When the second method 1012 is selected, the alert analyzer 135 extracts an explanatory variable whose degree of correlation with the objective variable calculated in S1701 is equal to or less than the value of the uncorrelated threshold value 1021 (S1706). The alert analyzer 135 calculates the predicted influence degree when the explanatory variables extracted in S1706 are excluded from the creation target by the same procedure as in S1702 (S1707), and proceeds to S1708. If the second method 1012 is not selected, the process proceeds to S1708 as it is.

Next, the alert analyzer 135 refers to the extraction method 1010 and determines whether or not the third method 1013 is selected (S1708). When the third method 1013 is selected, the alert analyzer 135 calculates the degree of correlation between the explanatory variables of the input data table 900, that is, the degree of cross-correlation (S1709). Then, the alert analyzer 135 creates a group of explanatory variables whose cross-correlation degree is equal to or higher than the value of the strong correlation threshold value 1023 (S1710).

The alert analyzer 135 selects one explanatory variable for each created group, calculates the predicted influence degree when the other explanatory variables are excluded (S1711), and proceeds to S1712. .. If the third method 1013 is not selected, the process proceeds to S1712 as it is. Finally, the alert analysis device 135 transmits the information required for the output screen display 1600 to the integrated management device 133, and ends the process (S1712).

According to the first embodiment described above, the following effects can be obtained.
(1) The analysis system 1 uses an input data creation unit 303 that creates a plurality of explanatory variables using information about an event and creates input data including a plurality of explanatory variables and the result of the event, and a plurality of explanatory variables. The prediction unit 401 that creates a prediction model formula that predicts the value of the objective variable that is the result of the event, and the explanatory variable evaluation unit 402 that calculates the exclusion candidate variables that are candidates for the explanatory variables that do not need to be used in the prediction model formula. , An output unit 404 that outputs exclusion candidate variables. Therefore, the analysis system 1 can assist human beings in determining explanatory variables that are not used in the prediction model formula. It can be said that this further contributes to reducing the number of explanatory variables created, and as a result, it can contribute to reducing the consumption of computational resources for calculating log statistics.

(2) The explanatory variable evaluation unit 402 calculates an explanatory variable having a variance of zero. An explanatory variable with zero variance has no correlation with the objective variable, so it is most suitable as an explanatory variable not used in the prediction model formula. Therefore, outputting the explanatory variables with zero variance is useful for determining the explanatory variables that are not used in the prediction model formula.

(3) The explanatory variable evaluation unit 402 calculates an explanatory variable whose degree of correlation with the objective variable is equal to or less than a predetermined threshold value. An explanatory variable with a low degree of correlation with the objective variable is suitable as an explanatory variable that is not used in the prediction model formula. Therefore, it is useful to output an explanatory variable whose degree of correlation with the objective variable is equal to or less than a predetermined threshold value in determining an explanatory variable that is not used in the prediction model formula.

(4) The analysis system 1 includes information on the accuracy of the prediction model formula when the exclusion candidate variable is not used, that is, the prediction influence degree evaluation unit 403 for calculating the prediction influence degree 1632 in FIG. The output unit 404 outputs this predicted influence degree 1632. Therefore, an expert can see the value of the prediction influence degree 1632 and use it as a reference for determining whether or not to use the exclusion candidate variable in the prediction model formula.

(5) The output unit 404 outputs the amount of calculation resources that can be reduced by the prediction model expression that does not use the exclusion candidate variable with respect to the prediction model expression that uses the exclusion candidate variable, as shown by reference numerals 1623, 1633, and 1645 in FIG. .. Therefore, it is useful as auxiliary information for determining explanatory variables that are not used in the prediction model formula.

(6) The explanatory variable evaluation unit 402 extracts exclusion candidate variables that are candidates for explanatory variables that do not need to be used in the prediction model formula using one or more extraction methods, and the output unit 404 extracts identification information of the extraction method. Is output. Therefore, an expert who sees the output of the output unit 404 can know the correspondence between the extraction method and the exclusion candidate variable when the exclusion candidate variable is extracted by using a plurality of extraction methods.

(7) The explanatory variable evaluation unit 402 creates a group based on the degree of correlation between the explanatory variables, and the predictive model formula in the case where only one of the explanatory variables included in the group is used in the predictive model formula for each group. The predicted influence degree 1644 at the time of selection shown in FIG. 16, which is information on accuracy, is calculated, and the output unit 404 outputs this. Therefore, it can be used as a reference for judgments in groups with high correlation.

(Modification 1)
The output unit 404 may output only the exclusion candidate variables. Specifically, in the example shown in FIG. 16, the training and test data prediction result 1610 is not displayed at all, and only the explanatory variable name 1622, the explanatory variable name 1631, and the explanatory variable name 1642 in the exclusion candidate variable and the prediction influence degree 1620 are displayed. It may be output. Furthermore, it is not necessary to display each item name. As described above, the display example shown in FIG. 16 is linked to the setting shown in FIG. 10. Therefore, for example, when only the first method 1011 is selected in the extraction method 1010 of FIG. 10, the dispersion is dispersed in FIG. Only the explanatory variable name 1622, which is the explanatory variable for which is zero, is displayed. That is, as the output of the output unit 404, only "external threat information URL risk" may be displayed on the screen 1600.

(Modification 2)
The predictive impact evaluation unit 403 may output the accuracy calculated by using the learning alert or the test alert instead of the predictive impact. In this case, the accuracy calculated using the learning alert or the test alert is displayed instead of the predicted influence degree 1632 and the predicted influence degree 1644 at the time of selection in FIG. In this case, the expert may compare the accuracy displayed in place of the predicted impact 1632 and the predicted impact 1644 at the time of election with the value shown in the accuracy 1614 in FIG. Information can be obtained.

(Modification 3)
In the first embodiment described above, candidate explanatory variables that do not need to be used in the prediction model formula are presented. However, the analysis system 1 may change the prediction model formula according to a predetermined policy. For example, the analysis system 1 may be changed to a predictive model formula that does not use an explanatory variable having a variance of zero.

According to this modification 3, the prediction model formula can be automatically updated within a predetermined range. This is especially effective when a non-expert person sees the output of the analysis system 1 or when he / she wants to reduce the judgment by an expert.

(Modification example 4)
In the output to the screen 1600, the output unit 404 may output the predictive influence degree together with the explanatory variable 1621 in which the variance = 0 in order to make the output format uniform. Since the explanatory variable with zero variance has no effect on the objective variable, the predicted influence degree output in this case is "100%".

(Modification 5)
In the above-described embodiment, the analysis system 1 uses the interface shown in FIG. 10 to perform processing using an extraction method designated by an expert, that is, a human being, and a threshold value. However, at least one of the extraction method and the threshold value may be predetermined. For example, all of the extraction methods and thresholds are predetermined, and the tab of setting 1001 does not have to exist.

-Second embodiment-
A second embodiment of the analysis system will be described with reference to FIGS. 18-19. In the following description, the same components as those in the first embodiment are designated by the same reference numerals, and the differences will be mainly described. The points not particularly described are the same as those in the first embodiment. This embodiment is different from the first embodiment in that the installation position of the alert analysis device is changed and the function of the integrated management device is also used.

FIG. 18 is a diagram showing the configuration of the analysis system 1A according to the second embodiment. The analysis system 1A is SOC130A. The SOC 130A includes an alert management device 131, a log management device 132, an alert analysis device 1801, and a second network 134. Since the configurations and functions of the alert management device 131, the log management device 132, and the second network 134 are the same as those in the first embodiment, the description thereof will be omitted. The alert analysis device 1801 has the functions of the integrated management device 133 and the alert analysis device 135 according to the first embodiment.

FIG. 19 is a block diagram showing a functional configuration example of the alert analyzer 1801 according to the second embodiment. The alert analysis device 1801 includes an alert collection unit 301, a log collection unit 302, an input data creation unit 303, an output data display unit 304, a prediction unit 401, an explanatory variable evaluation unit 402, and a prediction impact evaluation unit 403. And an output unit 404. That is, FIG. 19 includes the configuration shown in FIG. 3 and the configuration shown in FIG. 4 in the first embodiment. The operation of the alert collection unit 301, the log collection unit 302, the input data creation unit 303, the output data display unit 304, the prediction unit 401, the explanatory variable evaluation unit 402, the prediction impact evaluation unit 403, and the output unit 404 is the first implementation. Since it is as described in the form of the above, the description thereof will be omitted.

In the second embodiment described above, the same effects as those in the first embodiment can be obtained.

Although the above disclosure is described for a representative embodiment, one of ordinary skill in the art understands that various changes and modifications can be made in form and detail without departing from the spirit or scope of the disclosed subject matter. Will do.

For example, the external threat information database 136 may be a failure information database or an external information database including these, the risk level of the alert may be the importance of the alert including the fault information database, or the security operation center. (SOC) may be an operations center that responds not only to security but also to a wider range of alerts such as failures.

It should be noted that the present invention is not limited to the above-described embodiment, and includes various modifications and equivalent configurations within the scope of the attached claims. For example, the above-described embodiments have been described in detail in order to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to those having all the described configurations. Further, a part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Further, the configuration of another embodiment may be added to the configuration of one embodiment. In addition, other configurations may be added, deleted, or replaced with respect to a part of the configurations of each embodiment.

In each of the above-described embodiments and modifications, the configuration of the functional block is only an example. Several functional configurations shown as separate functional blocks may be integrally configured, or the configuration represented by one functional block diagram may be divided into two or more functions. Further, a configuration in which a part of the functions of each functional block is provided in another functional block may be provided.

Each of the above-described embodiments and modifications may be combined. Although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other aspects considered within the scope of the technical idea of the present invention are also included within the scope of the present invention.

1 ... Analysis system 100a ... Monitoring target system 133 ... Integrated management device 135 ... Alert analysis device 301 ... Alert collection unit 302 ... Log collection unit 303 ... Input data creation unit 304 ... Output data display unit 401 ... Prediction unit 402 ... Explanatory variable evaluation Part 403… Predictive impact evaluation part 404… Output part

Claims (9)

An input data creation unit that creates a plurality of explanatory variables using information about an event and creates input data including the plurality of explanatory variables and the result of the event.
A prediction unit that creates a prediction model formula that predicts the value of the objective variable that is the result of the event using the plurality of explanatory variables.
An explanatory variable evaluation unit that calculates exclusion candidate variables that are candidates for the explanatory variables that do not need to be used in the prediction model formula.
An analysis system including an output unit that outputs the exclusion candidate variable. In the analysis system according to claim 1,
The explanatory variable evaluation unit is an analysis system that calculates the explanatory variables having a variance of zero. In the analysis system according to claim 1,
The explanatory variable evaluation unit is an analysis system that calculates the explanatory variables whose degree of correlation with the objective variable is equal to or less than a predetermined threshold value. In the analysis system according to claim 1,
Further provided with a predictive impact evaluation unit for calculating predictive impact information, which is information on the accuracy of the predictive model formula when the exclusion candidate variable is not used.
The output unit is an analysis system that further outputs the predicted effect information. In the analysis system according to claim 1,
The output unit is an analysis system that further outputs the amount of calculation resources that can be reduced by the prediction model formula that does not use the exclusion candidate variable with respect to the prediction model formula that uses the exclusion candidate variable. In the analysis system according to claim 1,
The explanatory variable evaluation unit uses one or a plurality of extraction methods to extract exclusion candidate variables that are candidates for the explanatory variables that do not need to be used in the prediction model formula.
The output unit is an analysis system that further outputs the identification information of the extraction method. In the analysis system according to claim 1,
The explanatory variable evaluation unit creates a group based on the degree of correlation between the explanatory variables, and predicts the case where only one of the explanatory variables included in the group is used in the prediction model formula for each group. Calculate predictive impact information, which is information about the accuracy of the model formula,
The output unit is an analysis system that further outputs the predicted effect information. An input data creation unit that creates a plurality of explanatory variables using information about an event and creates input data including the plurality of explanatory variables and the result of the event.
A prediction unit that creates a prediction model formula that predicts the value of the objective variable that is the result of the event using the plurality of explanatory variables.
An explanatory variable evaluation unit that calculates exclusion candidate variables that are candidates for the explanatory variables that do not need to be used in the prediction model formula.
An analyzer including an output unit for outputting the exclusion candidate variable. An analytical method performed by an analyzer,
Creating a plurality of explanatory variables using information about an event, and creating input data including the plurality of explanatory variables and the result of the event.
Creating a prediction model formula that predicts the value of the objective variable that is the result of the event using the plurality of explanatory variables.
To calculate exclusion candidate variables that are candidates for the explanatory variables that do not need to be used in the prediction model formula,
An analysis method comprising outputting the exclusion candidate variable.

Images