JP2021140675A - Performance analysis device, performance analysis method, and performance analysis program - Google Patents

Abstract

To make it possible to reduce a load on a human and to appropriately analyze performance in a system.SOLUTION: An abnormality detection device 100 analyzes performance using performance data including a plurality of entries including time information, specification information concerning a plurality of specifications indicating a context, and performance information. The abnormality detection device is configured to include: a data instance generation unit 110 for dividing performance data into a plurality of data instances 120-1 to 120-N based on the specification information related to at least one specification of an entry of the performance data; and a data instance label attaching unit 130 that evaluates data characteristics relative to the data instances 120-1 to 120-N, specifies a performance analysis method according to the evaluated data characteristics, performs performance analysis relative to entries belonging to the data instances 120-1 to 120-N by the specified performance analysis method to attach a label indicating performance analysis results.SELECTED DRAWING: Figure 1

Description

The present invention relates to a technique for analyzing the performance of a system by using performance data having time information.

In recent years, there has been increasing interest in automating IT operation management to improve the reliability, availability, and security of IT systems. The main task of IT operation management is to monitor and maintain the health of IT systems, including the tasks currently performed primarily by human operators.

If part of the IT system is malfunctioning, the human operator is obliged to find the current range as well as the cause of the problem. This can be a very time consuming process if done solely by manual search.

In recent years, the number of IT systems has steadily increased in organizations, and the use of large entities in IT systems, such as data centers, has become increasingly common. Therefore, rapid problem recognition becomes increasingly difficult, but even more important as humans become more dependent on IT systems in all areas of life. For example, automatically detecting IT system health problems by analyzing IT system data using a computational approach such as machine learning can alleviate the above problems.

Another important aspect is the reusability of the designed problem detection approach. As the number of IT systems grows, IT system health issues must be considered under many different problem contexts, but when new solutions are designed for each change in the problem context. Great human effort and knowledge of specific machine learning areas are required.
Therefore, an anomaly detection approach that can be easily replicated for different problem contexts provides relief for IT operations managers, especially when explicit knowledge of the machine learning domain is not required.

For example, Patent Document 1 discloses, as a related technique, a method of identifying a threat risk score for a group of clustered outliers. In this method, different features are used to get hints about outlier types, and outliers are identified and threat risk scores are assigned by applying data-dependent mathematical and ML models, respectively.

U.S. Patent Application Publication No. 2019/0260793

IT operations managers are trying to find problems with the performance of IT systems. Analysis of IT system performance data supports the detection of different problem contexts, namely anomalies related to IT system attributes and their initial causes. Currently, most proposed analysis methods require high human setup efforts for each change in problem context.

According to the technique of Patent Document 1, a two-step outlier detection method that can detect outliers and assign a score to each outlier for different problem contexts has been established, but features engineering, machine learning, or High human-based modeling efforts are required for each new problem context, including mathematical model selection, as well as decisions regarding model adjustments (eg, parameters). It is also necessary to secure sufficient training data.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique capable of reducing a load on a human being and appropriately analyzing performance in a system.

In order to achieve the above object, the performance analyzer according to one viewpoint uses time information, specification information regarding a plurality of specifications indicating a context, and performance data including a plurality of data elements including the performance information to perform performance. A performance analyzer for analysis, the data instance generation unit that divides the performance data into a plurality of data instances based on the specification information regarding at least one specification of the data element of the performance data, and the data instance. The data characteristics of the data are evaluated, the performance analysis method according to the evaluated data characteristics is specified, and the performance analysis is performed on the data elements belonging to the data instance by the specified performance analysis method, and the label showing the performance analysis result is shown. It has a labeled portion to which the data is attached.

According to the present invention, the load on humans can be reduced, and the performance in the system can be appropriately analyzed.

FIG. 1 is an overall configuration diagram of an abnormality detection system including an abnormality detection device according to an embodiment. FIG. 2 is a configuration diagram of a performance data database according to an embodiment. FIG. 3 is a flowchart of the data instance generation process by the data instance generation unit according to the embodiment. FIG. 4 is a flowchart of the data context selection process by the data context selection unit according to the embodiment. FIG. 5 is a data structure diagram of a data instance according to an embodiment. FIG. 6 is a flowchart of the data instance labeling process of the data instance labeling portion according to the embodiment. FIG. 7 is a block diagram of a method pool according to an embodiment. FIG. 8 is a configuration diagram of a data instance label database according to an embodiment. FIG. 9 is a flowchart of label data scoring processing by the label data scoring unit according to the embodiment. FIG. 10 is a configuration diagram of a context score database according to an embodiment. FIG. 11 is a flowchart of visualization processing by the visualization processing unit according to the embodiment. FIG. 12 is a hardware configuration diagram of the abnormality detection device according to the embodiment. FIG. 13 is a diagram showing a screen example of the GUI according to the embodiment.

The embodiment will be described with reference to the drawings. It should be noted that the embodiments described below do not limit the invention according to the claims, and all of the elements and combinations thereof described in the embodiments are indispensable for the means for solving the invention. Is not always.

In the following description, the information may be described by the expression of "AAA table", but the information may be expressed by any data structure. That is, the "AAA table" can be called "AAA information" to show that the information does not depend on the data structure.

Also, in the following description, the data context specifications are problem contexts defined in the form of several columns in the dataset of performance data to be analyzed.

In addition, the applied data context specifications are data context specifications that are selected for use in the process of performing anomaly detection and are used (applied) when the performance data is divided into data instances.

A data instance also means a dataset of performance data divided into smaller entities according to the configuration of the applied data context specifications.

FIG. 1 is an overall configuration diagram of an abnormality detection system including an abnormality detection device according to an embodiment.

The abnormality detection system includes an abnormality detection device 100 as an example of a performance analysis device, a performance data database (DB) 200, a console 300, and a display 400.

The performance data DB 200 defines a data set table 202 (see FIG. 2) including a data set (performance data set) of performance data analyzed by the abnormality detection device 100, and information about each attribute of each performance data set. It stores a data context table 201 including a data context 201a (an entry in the data context table 210). In the present embodiment, it is assumed that the performance data DB 200 is provided outside the abnormality detection device 100, for example, in a device connected via a network (not shown), but the abnormality detection device 100 You may prepare for it inside. Details of the performance data DB 200 will be described later with reference to FIG.

The abnormality detection device 100 identifies an abnormality in the performance data set of the performance data DB 200 by dividing the performance data set of the performance data DB 200 into data instances 120 (120-1 to N) according to the data context 201a, and each data. An event label is assigned to each data (data element) for the instance 120. Further, the abnormality detection device 100 calculates an abnormality score based on the labeled data and the data context in the data instance label DB 160, and identifies the abnormality of each event label.

The console 300 is an input device connected to the abnormality detection device 100. The console 300 enables management tasks by the administrator of the anomaly detection device 100. Specifically, the console 300 receives an input of the setting of the abnormality detection device 100 from the administrator, and receives a change of the visualized content (screen) on the display 400 from the user of the abnormality detection device 100.

The display 400 is an output device capable of visualizing the result of the abnormality detection device 100 using the GUI. In the present embodiment, the display 400 displays, for example, an abnormality detection result, for example, an abnormality score or the like by GUI. A display example of the GUI on the display 400 will be described later with reference to FIG.

The abnormality detection device 100 includes a data instance generation unit 110, a data instance labeling unit 130 as an example of a labeling unit, a recursive cause identification unit 140, a method pool 150, and a data instance label database (DB) 160. , And a context score database (DB) 170.

In the present embodiment, in the abnormality detection device 100, one data context (one entry: a target entry; in this example, the entry in the first row) of a certain data context table 201 and one performance corresponding thereto. A case where a data set (performance data table 202 corresponding to an entry) is acquired (received) as a processing target will be described.

The abnormality detection device 100 performs preprocessing on the performance data set received together with the data context 201a in the data instance generation unit 110. The data instance generation unit 110 has a data context selection unit 111 as an example of the selection unit.

This preprocessing step includes selection of the data context specifications in the data context selection unit 111 and formatting of the data of the data context specifications. Information regarding data formatting can be used with reference to Method Pool 150.

The applied data context specifications selected by the data context selection unit 111 are stored in the performance data DB 200. The applicable data context specifications are subsequently used to divide the performance data into several data instances 120. These details will be described later with reference to FIGS. 3, 4, and 5.

The retrieved data instances 120 are then further processed in the data instance labeling section 130, which has the task of assigning event labels to each data instance by selecting the best method from the method pool 150, and then labeling. The created data instance is stored in the data instance label DB 160. These details will be described later with reference to FIGS. 6, 7, and 8.

The labeled data instance from the data instance label DB 160 is further processed by the recursive cause identification unit 140. The recursive cause identification unit 140 includes a label data scoring unit 141 as an example of the scoring unit and a visualization processing unit 142. The recursive cause identification unit 140 uses the information of the applied data context specifications received from the data context selection unit 111 for aggregation, and the label data scoring unit 141 calculates the score for the abnormality for each event label, and the context. It has a task of storing score information in the score DB 170. Further, the aggregation from the context score DB 170 and the visualization of the score result are prepared by the visualization processing unit 142 for display on the display 400. When a new input is received from the user of the anomaly detection device 100 via the console 300, a new visualization in the visualization processing unit 142 or a recalculation of the score in the label data scoring unit 141 is triggered. Further details of these will be described later with reference to FIGS. 9, 10, and 11.

Next, the performance data DB 200 will be described.

FIG. 2 is a configuration diagram of a performance data database according to an embodiment.

The performance data DB 200 includes two types of tables, namely, a data context table 201 and a performance data table 202 (202-1 to n).

The data context table 201 contains entries for each data context. The entries in the data context table 201 include columns for data ID D20101, performance data table ID D20102, data context specifications D20103, applicable data context specifications D20104, format dictionary D20105, and labeling goal D20106.

The data ID D20101 stores a data ID which is a unique value associated with the performance data set (a data set stored in one table of the performance data table 202). The performance data table ID D20102 stores a pointer to any of the performance data tables 202-1 to n that stores the performance data set. In the data context specification D20103, the name of the column (specification: item) of the performance data table that stores the performance data set regarded as the information of the data context is stored.

The applied data context specification D20104 stores the name of the column (specification) of the performance data table that stores the performance data set as the information of the applied data context specification selected by the data context selection unit 111. The format dictionary D20105 stores a dictionary in which the name of the conversion program that performs the optional format and the name of the column (specifications) of the performance data table to be applied are associated with each other. This dictionary is defined, for example, by the user of the anomaly detection device 100. The labeling target D20106 stores a labeling target (labeling target) of the performance data set, which is required as information for selecting an accurate labeling method in the data instance labeling unit 130.

Each performance data table 202 (202-1 to n) stores a performance data set. The performance data table 202 has a different configuration depending on the type of performance data set to be stored. The performance data table 202 stores entries (rows: data elements) for each performance data in the performance data set. Here, in the following description of this table, a case where the performance data is performance data related to access to some websites will be described as an example.

The entries (data elements) of the performance data table 202 include, for example, columns at time D20201, URI D20202, source IP D2203, HTTPMord D20204, performance indicators 1-N D20205-D202N. In this example, the information of the URI D20202, the source IP D2203, and the HTTPmethod D20204 is an example of the specification information, and the performance indexes 1 to N D20205 to D202N are examples of the performance information.

The time D20201 stores time information (for example, year / month / day / hour / minute / second: an example of time information) for the data of each entry in the performance data set. The URI D20202 stores the URI (Uniform Resource Identifier: Web address) of the communication destination website indicated by the performance data corresponding to the entry. In this embodiment, this URI is an example of the specifications of the data context. The source IP D2203 stores the IP address (source IP) of the communication source indicated by the performance data corresponding to the entry. This IP address is an example of the specifications of the data context.

The HTTP method D20204 stores a request method in the http format at the time of accessing the website indicated by the performance data. This request method is an example of one of the specifications of the data context. Performance indexes 1 to N D20205 to D202N store performance indexes (metric values, generally numerical values) in performance data. The number of types of performance indicators may be arbitrary, and columns corresponding to the number of types will be prepared and used.

Next, the data instance generation process by the data instance generation unit 110 will be described.

FIG. 3 is a flowchart of the data instance generation process by the data instance generation unit according to the embodiment.

The data instance generation unit 110 receives a performance data set (one of the performance data tables) having the data context information (entry of the data context table 201) to be processed from the performance data DB 200 (S11001).

Next, the data instance generation unit 110 acquires the applied data context specifications from the data context selection unit 111 (S11002). In the data context selection unit 111, the applied data context specifications are selected by the data context selection process shown in FIG. 4, and the applied data context specifications are transmitted to the data instance generation unit 110. The data context selection process will be described later with reference to FIG.

Next, the data instance generation unit 110 creates a list including a combination of each unique value in the applied data context specifications (step S11003). For example, there are "URI1" and "URI2" as unique values in the specification "URI" included in the applied data context specification, and as unique values in the specification "source ID" included in the applied data context specification. , "10.0. *. *", The data instance generation unit 110 has two combinations "(URI1, 10.0. *. *), (URI2, 10. *) Combining these values. Create a list containing "0. *. *)".

Next, the data instance generation unit 110 executes the loop 1 process (steps S11004, S11005) for each combination included in the list. In this process, the combination of processing targets is called a target combination.

In the processing of loop 1, the data instance generation unit 110 makes an entry (row) including a value corresponding to the target combination from the performance data table 200 (performance data set) corresponding to the application data context specifications of the performance data DB 200. Extract and generate a data instance corresponding to the target combination (step S11004).

Next, the data instance generation unit 110 determines the ideal time window size for the data instance acquired in step S11004 by considering the sparseness of the data instance with respect to time (step S11005). Here, for data instances that are highly sparse (very sparse: for example, more sparse than a given threshold), the sparseness is less than or equal to the given in order to achieve more accurate labeling. Therefore, a large time window size is determined. This makes it possible to adjust the number of entries in the data instance to a number suitable for processing.

After processing loop 1 for one target combination, the data instance generation unit 110 processes loop 1 with the other unprocessed combinations as the next processing target, and targets all combinations in the list. When the processing of the loop 1 is performed, the loop 1 is exited and the data instance generation processing is terminated.

Next, the data context selection process (S11002) will be described.

FIG. 4 is a flowchart of the data context selection process by the data context selection unit according to the embodiment.

The data context selection unit 111 executes the processing of loop 2 (steps S11101 to S11105) for each of the data context specifications to be processed. Here, the data context specification to be processed is referred to as a target data context specification, and the specification to be processed in the loop 2 in the target data context specification is referred to as a target specification.

In the loop 2, the data context selection unit 111 sets a format rule for changing the format of the value of the target specification from the format dictionary D20105 of the entry corresponding to the target data context specification of the data context table 201 of the performance data DB 200. Acquire (S11101). In the present embodiment, the format rule is in a dictionary format in which pointers to applicable programs are associated with the names of the specifications.

Next, the data context selection unit 111 determines whether or not the rule acquired in step S11101 is available for the target specifications (S11102).

As a result, when the format rule is available for the target specifications (S11102: Yes), the data context selection unit 111 advances the process to step S11105.

On the other hand, when the format rule is not available for the target data context specifications (S11102: No), the data context selection unit 111 advances the process to step S11103.

In step S11103, the data context selection unit 111 acquires a format rule (here, an entry) corresponding to the target specifications defined in the context formatting table 152 (see FIG. 7) of the method pool 150.

Next, the data context selection unit 111 determines whether or not the value (data) of the target specification is the expected regular expression according to the expected regular expression format of the expected regular expression format D15202 of the context formatting table 152, thereby determining the target specification. It is determined whether or not the format rule is applied to the value of (S11104).

As a result, when the value of the target specification is the expected regular expression (S11104: No), it means that the data format does not need to be changed, so that the data context selection unit 111 loops the processing. Proceed to the end of 2.

On the other hand, when the target specifications are not the expected regular expression (S11104: Yes), it means that the data format needs to be changed. Therefore, the data context selection unit 111 starts from the format processing D15203 of the acquired entry. The pointer of the program (script) that executes the format is acquired, and the process proceeds to step S11105.

In step S11105, the data context selection unit 111 formats the values of the target specifications according to the format rule acquired in step S11102 or the program acquired in step S11104.

After finishing the processing of loop 2 for one corresponding specification, the data context selection unit 111 executes the processing of loop 2 with the other specifications as new processing targets, and processes all the specifications. After that, the loop 2 is exited and the process proceeds to step S11106.

In step S11106, the data context selection unit 111 displays all the data context specifications (list of specifications) of the data context specifications D20103 in the entry corresponding to the target data context specifications of the data context table 201 of the performance data DB 200. Copy this entry to the applicable data context specification D20104.

Next, the data context selection unit 111 divides the data of the performance data set based on all the specifications currently used as the applied data context specifications, and evaluates the sparseness of the data instance obtained by the division (). S11107).

Next, the data context selection unit 111 determines whether or not the data division for the data instance is too sparse (for example, there is no data instance exceeding a certain amount of data rows) (S11108).

As a result, when the data division is too sparse (S11108: Yes), the data context selection unit 111 advances the process to step S11109. On the other hand, when the data division is not too sparse (S11108: No), it means that the data division has been performed appropriately, so the data context selection unit 111 proceeds to the process in step S11110.

In step S11109, the data context selection unit 111 detects the specifications having the most non-uniform distribution from the applied data context specifications, drops (deletes) the detected specifications from the applied data context specifications, and processes them. To step S11107. For example, when the entry in the first row of the data context table 201 is targeted for processing, the specification "HTTPmethod" among the data context specifications has a non-uniform distribution in which most of the values are the method type "connect". Tends to have. In this case, in this step, the specification "HTTP method" will be dropped from the list of applicable data context specifications. As a result, specifications that are not suitable for the analysis process can be appropriately excluded.

By repeatedly executing the processes of steps S11107 to S11109 described above, it is possible to specify the applicable data context specifications that can generate a data instance that is not too sparse.

In step S11110, the data context selection unit 111 corresponds to the data context table 201 with the applicable data context specifications when the data division is not too sparse (S11108: No), that is, when the data division is properly performed. It is stored in the application data context specification D20104 of the entry to be applied.

According to the above-mentioned data context selection process, it is possible to appropriately select the specifications of the data context (applied data context specifications) in which the data division is appropriately performed.

Next, the data instance 120 will be described.

FIG. 5 is a data structure diagram of a data instance according to an embodiment.

The data instance 120 (120-1 to n) is obtained from the data instance generation unit 110. The number of data instances obtained from the data instance generation unit 110 varies depending on the applied data context specifications.

The data instance 120 stores an entry (row: data element) for each predetermined time interval for the same data context (that is, the value of each specification of the applied data context specification is the same). The entry for the data instance 120 includes columns for time D12001, time window size D12002, URI D12003, source IP D12004, performance indicators 1-N D12005-D120N.

The time D12001 stores time information (for example, year, month, day, hour, minute, second) corresponding to the representative time of the time window corresponding to the entry of the data instance. The time window size D12002 stores time difference information regarding the time window size (recommended time window size) to be used for labeling the data instance. The URI D12003 stores the URI of the communication destination website, which is one of the specifications of the data context corresponding to the entry. The source IP D12004 stores the IP address of the source of communication, which is one of the specifications of the data context corresponding to the entry.

Performance indexes 1 to N D12005 to D120N store performance indexes (metric values, generally numerical values) for data instances corresponding to entries. The number of types of performance indicators may be arbitrary, and columns corresponding to the number of types will be used.

Next, the data instance labeling process by the data instance labeling unit 130 will be described.

FIG. 6 is a flowchart of the data instance labeling process of the data instance labeling portion according to the embodiment.

First, the data instance labeling unit 130 acquires the labeling target from the labeling target D20106 of the target entry in the data context table 201 of the performance data DB 200 (S13001).

Next, the data instance labeling unit 130 receives the data instance 120 generated by the data instance generation unit 110 (S1302).

Next, the data instance labeling unit 130 executes the loop 3 processing (S13003 to S13006) for each data instance 120. Here, the data instance to be processed is referred to as a target data instance.

In the processing of the loop 3, the data instance labeling unit 130 calculates the statistical characteristics of the target data instance (S13003). For example, the statistical characteristics calculated by the data instance labeling unit 130 include at least one of the maximum and minimum values of the performance indicators of the entries contained in the data instance, the percentiles for the performance indicators, the standard deviation, or the number of entries. It may be included.

Next, the data instance labeling unit 130 applies a labeling method to be applied to the target data instance from the method pool 150 based on the statistical characteristics calculated in S13003 and the labeling target acquired in S13001. Select (S13004). Specifically, the data instance labeling unit 130 identifies an entry in which the labeling target is set to the labeling target D151N + 1 from the method pool 150 and the value of the statistical characteristic satisfies the conditions of the data attributes 1 to N D15102 to D151N. Then, the labeling method set in the labeling method D15101 of the entry is selected.

Next, the data instance labeling unit 130 assigns an event label to each data row (entry) of the data instance 120 according to the labeling method selected in S13004 (S13005). For example, when the labeling target is Outlier Identification, the data instance labeling unit 130 uses an outlier or non-outlier as an event label for each data row of the data instance 120, depending on the labeling method. Assign an event label that indicates the value. Here, the best labeling method tends to be different depending on the statistical characteristics of the data instance 120. Therefore, in the present embodiment, the labeling method to be used is selected according to the conditions for the statistical characteristics in the method pool 150. Statistical characteristics for choosing a labeling method can be generated based on the time, time window size, and performance values of the data instance.

Next, the data instance labeling unit 130 stores the data instance 120 together with the event label assigned to each data row as the data instance table 162 of the data instance label DB 160.

Next, the method pool 150 will be described.

FIG. 7 is a block diagram of a method pool according to an embodiment.

The method pool 150 includes two types of tables, namely a labeling method attribute table 151 and a context formatting table 152.

The labeling method attribute table 151 stores entries for each labeling method. The entry in the labeling method attribute table 151 includes columns for the labeling method D15101, one or more data attributes 1 to N D15102 to D151N, and a labeling target D151N + 1.

The labeling method D15101 stores the name of the labeling method corresponding to the entry and a pointer to the program that executes the labeling method. Data attributes 1 to N D15102 to D151N store conditions for statistical characteristics (attributes) that may be considered in order to select the best labeling method. The labeling target D151N + 1 stores one or more labeling targets that can use the labeling method corresponding to the entry. In the labeling target D151N + 1, for example, when performing abnormality detection (outlier detection) in the performance analysis, "Outlier Identification" is stored.

The context formatting table 152 stores entries for each type of data context specification (data context type). The entries in the context formatting table 152 include columns for data context specification type D15201, expected regular expression format D15202, and formatting D15203.

The data context specification type D15201 stores the name of the type of data context specification for which the format rule corresponding to the entry is provided. The expected regular expression format D15202 stores a regular expression that allows the values of all data context specifications that match the type of data context specification corresponding to the entry to be extracted. The format process D15203 stores a pointer to the program for reformatting the data of the data context specifications into a regular expression according to the rules defined in the program (including the script).

Next, the data instance DB 160 will be described.

FIG. 8 is a configuration diagram of a data instance label database according to an embodiment.

The data instance label data DB 160 includes two types of tables, namely the data instance management table 161 and the data instance table 162 (1621-1N).

The data instance management table 161 stores an entry for each data instance. The entries in the data instance management table 161 include columns for the data instance ID D16101, URI D16102, source IP D16103, and data instance table D16104.

The data instance ID D16101 stores a value (data instance ID) that identifies the data instance corresponding to the entry. The URI D16102 and the source IP D16103 are columns corresponding to the applied data context specifications, and are different columns depending on the specifications included in the applied data context specifications. The URI D16102 stores the value of the URI, which is the applicable data context specification for the data instance corresponding to the entry, that is, the URI (web address) of the website to communicate with. The source IP D16103 stores the value of the source IP, which is the applicable data context specification for the data instance corresponding to the entry, that is, the IP address (source IP) of the source of the communication. The data instance table D16104 stores a pointer to the data instance table 162 (any of 1621-1 to N) corresponding to the data instance corresponding to the entry.

Each of the data instance tables 1621-1 to N is provided for each data instance and stores an entry (data element) corresponding to each data instance. The entries in the data instance table 162 include columns at time D16201, performance indicators 1-N D16202-D162N, and event label D162N + 1.

Time D16201 stores time information (for example, year, month, day, hour, minute, second) for the data corresponding to the entry. Performance indexes 1 to N D16202 to D162N store performance indexes (metric values, generally numerical values) for the data corresponding to the entries. The event label D162N + 1 stores the event label assigned by the data instance labeling unit 130 for the entry data. For example, for the data instance targeted for abnormality detection, the event label D162N + 1 stores "-1" indicating normality when the entry data is normal, and indicates an abnormality when it is abnormal. "1" is stored.

Next, the label data scoring process by the label data scoring unit 141 will be described.

FIG. 9 is a flowchart of label data scoring processing by the label data scoring unit according to the embodiment.

The label data scoring unit 141 acquires the labeled data instance (data instance table 162) from the data instance label DB 160, and extracts all the unique event label values (label values) (S14101).

Next, the label data scoring unit 141 creates a list including all possible combinations (specification combinations) for each specification of the applied data context specifications of the data context used to generate the data instance (S14102). For example, if the applicable data context specifications are "URI", "Source IP", the label data scoring unit 141 will have ("URI", "Source IP"), ("URI"), ("Source"). Generate a list containing the three specification combinations of "IP").

If necessary, the label data scoring unit 141 resamples to the currently given aggregation time window (eg, by default or the aggregation time window given via the display screen described below) (S14103). .. For example, when the size of the aggregated time window is 1 hour, the label data scoring unit 141 reconverts the data of the 1-minute time window into the data of the 1-hour time window by adding or counting the data of the 1-minute time window. Sampling.

Next, the label data scoring unit 141 executes loop 4 processing (S14104 to S14108) for each label value extracted in step S14101. Here, the label value to be processed is referred to as a target label value.

In the processing of the loop 4, the label data scoring unit 141 executes the processing of the loop 5 (S14104 to S14106) for each specification combination acquired in step S14102. Here, the specification combination to be processed is referred to as a target specification combination.

In the processing of the loop 5, the label data scoring unit 141 aggregates the data of the data instance according to the target specification combination, the target label value, and the set of the given aggregation time windows (S14104). For example, when the target specification combination is "URI", the value of the specification of the target specification combination (value of the same URI) and the target without considering other specifications of the applied data context specifications. For data rows having label values, data is aggregated by adding the target label values or counting the number of data rows.

Next, the label data scoring unit 141 sets the score for the performance evaluation result (abnormal in this case) for the aggregated data (aggregated data) as the data in the current time window and the past (predetermined) for the same data. It is calculated by comparing the data in the time window before the hour (for example, one week before) (S14105). For example, the label data scoring unit 141 ranks based on the amount of change in the aggregated data from the past, and gives the highest score to the aggregated data having the largest change, for example. Specifically, for example, in the ranking, the smaller the amount of change, the lower the rank (the rank with the smaller numerical value), and the result of multiplying the amount of change and the rank is used as the score. In this embodiment, the larger the score, the higher the possibility that an abnormality has occurred.

Next, the label data scoring unit 141 stores the aggregated data in step S14104 and the score calculated in step S14105 in the context score DB 170 (S14106).

The label data scoring unit 141 executes the processing of the loop 5 for all the specification combinations, and exits the loop 5 when the processing of the loop 5 is completed for all the specification combinations.

After exiting the loop 5, the label data scoring unit 141 calculates the total score for the corresponding data context based on the score obtained for the aggregated data of each specification combination obtained in the processing of the loop 5 ( S14107). In the present embodiment, the total score is, for example, the total value of each score.

Next, the label data scoring unit 141 stores the total score calculated in step S14107 in the total score D17210 of the data context score table 172 of the context score DB170 (S14108).

The label data scoring unit 141 executes the processing of the loop 4 for all the label values, and when the processing of the loop 4 is completed for all the label values, exits the loop 4 and performs the label data scoring processing. finish.

Next, the context score DB 170 will be described.

FIG. 10 is a configuration diagram of a context score database according to an embodiment.

The context score DB 160 includes two types of tables, namely a data context aggregation table 171 and a data context score table 172.

The data context aggregation table 171 stores entries (data elements) for each aggregated data set aggregated at a predetermined aggregation time for each value of the applied data context specifications. The entries in the data context aggregation table 171 include columns for time D17101, data context (URI D17102, source IP D17103), and data context-based aggregation (aggregation URI D17104, aggregation source IP D17105, aggregation URI x source IP D17106). ..

Time D17101 stores time information (for example, year, month, day, hour, minute, second) about a representative time (for example, the first time of the aggregation time) that serves as a reference for the aggregation time of the aggregation data set corresponding to the entry.

The data context (URI D17102, source IP D17103) stores a value (data context value) for each applicable data context specification in the aggregated dataset corresponding to the entry. The URI D17102 stores the URI value for the aggregated data set corresponding to the entry, that is, the URI (web address) of the website to communicate with. The source IP D17103 stores the value of the source IP for the aggregated data set corresponding to the entry, that is, the IP address (source IP) of the source of the communication.

Data context-based aggregations (aggregate URI D17104, aggregate source IP D17105, aggregate URI x source IP D17106) include data for each combination of specifications for the applicable data context specifications in the dataset corresponding to the given aggregation time. The aggregated value is stored. The aggregated URI D17104 stores the number of data in which the URI values are common in the dataset corresponding to the entry. The aggregate source IP D17105 stores the number of data having a common source IP value in the dataset corresponding to the entry. Aggregate URI x source IP D17106 stores the number of data in which the URI value and the source IP value in the data set corresponding to the entry are common.

The data context score table 172 stores the entries for each aggregated data set aggregated at a predetermined aggregation time for each value of the applied data context specifications. The entries in the data context score table 172 are time D1721, URI D17202, source IP D1723, URI difference (rank) D17204, source IP difference (rank) D17205, URI x source IP difference (rank) D17206, URI score D17207, source IP. Includes columns with score D17208, URI x source IP score D17209, and overall score D17210.

The time D17201 stores time information (for example, year, month, day, hour, minute, second) about a representative time (for example, the first time of the aggregation time) that is a reference for the aggregation time of the aggregation data set corresponding to the entry. In the URI D17202 and the source IP D17203, the values of the applicable data context specifications of the aggregated data set corresponding to the entries are stored.

The URI difference (rank) D17204, the source IP difference (rank) D17205, and the URI x source IP difference (rank) D17206 include the current (eg, this week) and past (eg, this week) values for each specification combination in the aggregated dataset. The difference between the absolute value and the value of (last week) and the rank of the absolute value difference of each specification combination between the aggregated data sets are stored. The information in these columns can be used to calculate the score, for example, the absolute value difference, the rank of the absolute value difference, the difference in the absolute value of the rank between the current value and the past value, and the like. can.

The URI score D17207, source score D17208, and URI × source IP score D17209 store scores for each combination of applicable data context specifications. The total score D17210 stores the total score obtained by executing a predetermined calculation formula (for example, addition of scores) using the scores of columns D17207 to D17209.

Next, the visualization process by the visualization processing unit 142 will be described.

FIG. 11 is a flowchart of visualization processing by the visualization processing unit according to the embodiment.

The visualization processing unit 142 acquires aggregated data and score information from the context score DB 170 (S14201).

Next, the visualization processing unit 142 determines whether or not each score corresponding to the aggregated data has a score larger than a predetermined threshold value for each score (S14202).

As a result, when there is a score larger than the threshold value (S14202: Yes), the visualization processing unit 142 sends an alarm to the operator (S14203), and proceeds to the process in step S14204. On the other hand, if there is no score larger than the threshold value (S14202: No), the process proceeds to step S14204.

In step S14204, the visualization processing unit 142 visualizes the aggregated data and the score information acquired in step S14201. Specifically, the visualization processing unit 142 generates display screen data from aggregated data and score information, and displays the display screen (see FIG. 13) on the display 400.

Next, the visualization processing unit 142 waits for an input from the console 300 by the user of the abnormality detection device 100 (S14205). As the input to the console 300, there is an input of a change instruction of the display content such as a change of the aggregation period and a setting of the value of the specification not to be displayed.

Next, the visualization processing unit 142 determines whether or not the data necessary for displaying the screen corresponding to the change instruction input in step S14205 can be used in the context score DB 170 (S14206).

As a result, when the necessary data is available in the context score DB 170 (S14206: Yes), the visualization processing unit 142 acquires the necessary aggregated data and score information from the context score DB 170 (S14207) and processes them. To step S14204.

On the other hand, if the required data is not available in the context score DB 170 (S14206: No), the visualization processor 142 will use parameters related to aggregation and scoring (eg, time window size) based on user input. , Exclusion of display of values of specific data context specifications, etc.) (S14208).

Next, the visualization processing unit 142 transmits the parameters related to aggregation and scoring to the label data scoring unit 141, and causes the label data scoring unit 141 to execute the label data scoring process using the new parameters (S14209). , End the visualization process. After the label data scoring process using the new parameters is executed, the visualization process is newly executed and the display screen is displayed.

Next, the hardware configuration of the abnormality detection device 100 will be described.

FIG. 12 is a hardware configuration diagram of the abnormality detection device according to the embodiment.

The abnormality detection device 100 is, for example, a general-purpose computer, and includes a CPU (Central Processing Unit) 601, a memory 602, an auxiliary storage device 603, a communication interface 604, a medium interface 605, and an input / output interface 606.

The CPU 601 executes a program stored in the memory 602 or the auxiliary storage device 603, and executes various processes by using the data stored in the memory 602 or the auxiliary storage device 603. The memory 602 is, for example, a RAM (Random Access Memory), and stores a program executed by the CPU 601, data, and the like. The auxiliary storage device 603 is, for example, a hard disk drive, a flash memory, a RAM, or the like, and stores a program executed by the CPU 601 and data used by the CPU 601.

The communication interface 604 is an interface for communicating with another device via the network 608. The medium interface 605 is removable from the external storage medium 607 and mediates the input / output of data to and from the external storage medium 607. The input / output interface 606 can be connected to the console 300 or the display 400 operated by the administrator or the user of the abnormality detection device 100, and can input / output information to / from the console 300 or display the information on the display 400. do.

Each functional unit of the abnormality detection device 100 in FIG. 1 is realized, for example, by the CPU 601 executing a program (performance analysis program) stored in the memory 602 or the auxiliary storage device 603. Further, the information managed by the functional unit (method pool 150, data instance label DB 160, context score DB 170) is stored in the memory 602 or the auxiliary storage device 603, which is an example of the storage unit.

The program executed by the CPU 601 may be acquired from another device via the communication interface 604, or may be read from and acquired from an available storage medium via the medium interface 605, if necessary. The storage medium is, for example, a communication medium (that is, a wired, wireless, optical network, a carrier or digital signal propagating in the network) or an external storage medium 607 that can be attached to and detached from the medium interface 605.

Next, a GUI screen example will be described.

FIG. 13 is a diagram showing a screen example of the GUI according to the embodiment. The screen of FIG. 13 shows an example when the context tab 401-1 of the context 1 (URI) described later is selected.

The screen 1300 displayed on the display 400 includes a context tab 401 (401-1 to 401-7), a heat map 402 (402-1 in FIG. 13), and a top outlier list 403 (403-1 in FIG. 13). , Includes a threshold display area 404 (404-1 in FIG. 13). The display content on the screen 1300 is appropriately updated based on the information transmitted from the visualization processing unit 142.

The context tab 401 is a container for containing visualization contents for each specification combination in the applied data context specification, and tabs for the number of specification combinations are provided. For example, when the applied data context specifications include three specifications, the number of context tabs 401 is seven, that is, context tabs 401-1 to 401-7. In the example of FIG. 13, since the context tab 401-1 is selected, the context tab 401-1 is highlighted.

The heat map 402 is a map of scores for the values of some of the selected specification combinations of the applied data context specifications (ie, in the specification combinations corresponding to the selected context tab 401. In the example of FIG. 13, the heat map 402-1 is a map of scores for some URI values when the specification combination is URI. According to the heat map, any of the specification combinations In terms of value, it is possible to easily grasp whether or not the largest abnormality has occurred.

The high-ranking outlier list 403 is a list that visualizes the values in a predetermined number of specification combinations having high scores. In the example of FIG. 13, the high-ranking outlier list 403 is a list in which the scores are arranged in the order of URI1, URI2, URI3, ... As the high-ranking URI when the specification combination is URI. In addition, in the high-order outlier list 403 of FIG. 13, a check box 4031-1 and the like for excluding the values of specifications having a high score but not needing to be visualized from the list are prepared. Further, in the high-order outlier list 403, there is an area in which the start time and the end time to be considered and the aggregation time window size (Aggression) for visualization are displayed and can be set.

The present invention is not limited to the above-described embodiment, and can be appropriately modified and implemented without departing from the spirit of the present invention.

For example, in the above embodiment, an abnormality detection device for detecting an abnormality has been used as an example, but the present invention is not limited to this, and the present invention can be applied to a device for analyzing the performance of various devices.

Further, in the above embodiment, a part or all of the processing performed by the CPU may be performed by the hardware circuit. In addition, the program in the above embodiment may be installed from the program source. The program source may be a program distribution server or a storage medium (eg, a portable storage medium).

Further, in the above embodiment, the performance detection device assuming the use in the operation management of the IT system has been described, but the present invention is not limited to this, and the data is divided based on the data context to generate a data instance. A performance analyzer may be used in the case as well, and for example, a performance analyzer may be used in OT (Operational Technology).

100 ... Abnormality detection device, 110 ... Data instance generation unit, 111 ... Data context selection unit, 130 ... Data instance labeling unit, 140 ... Recursive cause identification unit, 141 ... Label data scoring unit, 142 ... Visualization processing unit, 150 ... Method pool, 160 ... Data instance label DB, 170 ... Context score DB, 200 ... Performance data DB

Claims (13)

A performance analysis device that analyzes performance using performance data that includes time information, specification information related to a plurality of specifications indicating context, and performance data including a plurality of data elements including performance information.
A data instance generation unit that divides the performance data into a plurality of data instances based on the specification information regarding at least one specification of the data element of the performance data.
Evaluate the data characteristics of the data instance, specify the performance analysis method according to the evaluated data characteristics, and perform performance analysis on the data elements belonging to the data instance by the specified performance analysis method. Labeled parts that label the results and
Performance analyzer with. Further having a method pool for storing the data characteristics in association with the performance analysis method used for performance analysis on the data instance having the data characteristics.
The performance analyzer according to claim 1, wherein the labeled portion specifies a performance analysis method corresponding to the evaluated data characteristics from the method pool. The performance analyzer according to claim 2, wherein the data characteristics are statistical characteristics for the data instance. A claim that further has a scoring unit that creates aggregated data that aggregates the same labeled data elements within a predetermined aggregation time in the data instance and calculates the score of the performance analysis result for the aggregated data. Item 1. The performance analyzer according to Item 1. The scoring unit is the difference between the number of aggregated data elements in the aggregated data and the number of aggregated data elements in the past aggregated data created for the predetermined aggregation time in the past at a predetermined time point. The performance analyzer according to claim 4, wherein the score in the aggregated data is calculated based on the rank of the difference in the plurality of data instances. The performance analyzer according to claim 4, further comprising a visualization processing unit that displays information on the specifications of the data instance aggregated in the aggregated data and the calculated score. The performance according to claim 6, wherein the visualization processing unit receives a change in the aggregation time from a user, causes the scoring unit to recalculate based on the changed aggregation time, and displays the result of the recalculation. Analysis equipment. For each combination of one or more specifications used when the data instance was generated, the scoring unit has the same combination specification value and the same label within a predetermined aggregation time. A claim that further has a scoring unit that identifies the number of data elements attached, calculates a score for each of all combinations, and calculates an overall score based on the calculated score for all combinations. Item 4. The performance analyzer according to Item 4. The performance according to claim 1, wherein the data instance generation unit applies the specification information of one or more of the plurality of specifications of the performance data when the performance data is divided into the data instances. Analysis equipment. Applying a plurality of specifications of the performance data, the performance data is divided into the data instances, the temporal sparseness of the data elements of the divided data instances is evaluated, and the time of the data elements of the divided data instances is evaluated. If it is too sparse, specify the most non-uniform specification among the plurality of applied specifications, and select one or more specifications excluding the specified specifications from the plurality of specifications in the data instance. The performance analyzer according to claim 9, further comprising a selection unit that determines one or more specifications to be applied by the generation unit. The data instance generation unit
When the performance data is divided into a plurality of data instances, the time window size for the data element of the performance data, which is the target of the data element of the data instance, is set so that the sparseness of the data element of the data instance is equal to or less than a predetermined value. The performance analyzer according to claim 1, which is determined in 1. It is a performance analysis method by a performance analyzer that analyzes performance using performance data including a plurality of entries including time information, specification information regarding a plurality of specifications indicating a context, and performance information.
The performance data is divided into a plurality of data instances based on the specification information regarding at least one specification of the data element of the performance data.
Evaluate the data characteristics of the data instance, specify the performance analysis method according to the evaluated data characteristics, and perform performance analysis on the data elements belonging to the data instance by the specified performance analysis method. A performance analysis method that labels the results. A performance analysis program that is run by a computer
On the computer
The performance data is obtained based on the specification information regarding at least one specification of the data element of the performance data including the time information, the specification information regarding a plurality of specifications indicating the context, and a plurality of data elements including the performance information. Divide into multiple data instances
The data characteristics of the data instance are evaluated, the performance analysis method according to the evaluated data characteristics is specified, and the performance analysis is performed on the data elements belonging to the data instance by the specified performance analysis method. A performance analysis program that executes a process that causes a label to indicate the analysis result.

Images