JP7276550B2 - Anomaly detection method, system and program - Google Patents

Description

The present invention relates to an anomaly detection method, system and program for detecting anomalies from logs.

A system running on a computer generally outputs a log containing event results, messages, and the like. When a system failure or the like occurs, the distribution of the number of log outputs often changes compared to normal times. This is because, for example, a log that is normally output is not output due to a system error, or a log that is not normally output is output due to a system error. Techniques for detecting anomalies using changes in the number of log outputs have been devised in the past.

The technology described in Patent Document 1 calculates the average and standard deviation from the distribution of the frequency of output of past logs (events), and calculates a theoretical distribution (normal distribution, Poisson distribution, etc.) from the calculated average and standard deviation. to generate The technique then determines whether an anomaly has occurred from the analyzed log based on the theoretical distribution. In addition, in Patent Document 1, a plurality of types of theoretical distributions are generated using different statistical methods such as normal distribution and Poisson distribution, and an optimum theoretical distribution is selected from a plurality of types of theoretical distributions. Detection is described.

JP 2005-236862 A

In general, there are multiple factors that affect the number of logs output, and the main factors that affect the number of logs output may change depending on the unit of time for aggregating logs (every hour, every day, etc.) . Therefore, when the average state obtained by averaging logs for a certain period of time in the past is used as a reference, there is a possibility that anomalies cannot be detected appropriately. Also, even if the devices are of the same type, the characteristics of each device are different, so the characteristics of the distribution of the number of log outputs for each device may also vary. Therefore, if the average state obtained by averaging the logs output from a plurality of devices is used as a reference, the characteristics of the distribution for each device and for each aggregation unit may be hidden, and anomalies may not be detected appropriately.

However, although Patent Document 1 describes that a distribution selected from distributions of different statistical methods is used as a reference for detecting anomalies, it is not assumed to use log distributions for different aggregation units and different devices. not

The present invention has been made in view of the above-mentioned problems, and is an anomaly detection method capable of performing anomaly detection with high accuracy using the distribution of the number of log outputs generated for different aggregation units and different devices. , the purpose of which is to provide systems and programs.

A first aspect of the present invention is an anomaly determination method that acquires a plurality of pieces of reference log information including information about the number of log outputs of a first reference device in a first time span, Acquiring analysis target log information including information about the number of log outputs of the first device, selecting the first reference log information from the plurality of reference log information based on similarity to the analysis target log information, The step of determining whether or not there is an abnormality in the first device based on the analysis target log information and the first reference log information.

A second aspect of the present invention is an abnormality determination system that acquires a plurality of reference log information including information on the number of log outputs of a first reference device in a first time span, Acquiring log information to be analyzed including information about the number of log outputs of a first device, and selecting first reference log information from the plurality of reference log information based on similarity to the log information to be analyzed An acquisition unit and a determination unit that determines whether or not there is an abnormality in the first device based on the analysis target log information and the first reference log information.

A third aspect of the present invention is an abnormality determination program that acquires a plurality of pieces of reference log information including information about the number of log outputs of a first reference device in a first time span, Acquiring analysis target log information including information about the number of log outputs of the first device, selecting the first reference log information from the plurality of reference log information based on similarity to the analysis target log information, A computer is caused to execute a step of determining whether or not there is an abnormality in the first device based on the log information to be analyzed and the first reference log information.

A fourth aspect of the present invention is an anomaly determination program that acquires a plurality of pieces of reference log information including information about the number of log outputs of a first reference device in a first time span, Acquiring log information to be analyzed including information about the number of log outputs of a first device, and selecting first reference log information from the plurality of reference log information based on similarity to the log information to be analyzed The computer functions as an acquisition unit and a determination unit that determines whether or not there is an abnormality in the first device based on the log information to be analyzed and the first reference log information.

According to the present invention, since anomalies are detected using a plurality of distributions generated for each device that outputs logs and for each log aggregation unit, the characteristics of the distribution for each device and each aggregation unit can be utilized to achieve high accuracy. Anomalies can be detected.

1 is a block diagram of an anomaly detection system according to a first embodiment; FIG. 4 is a schematic diagram of analysis target logs according to the first embodiment; FIG. 4 is a schematic diagram of a format according to the first embodiment; FIG. 1 is a schematic diagram of an abnormality detection method according to a first embodiment; FIG. FIG. 4 is a schematic diagram of exemplary aggregation units according to the first embodiment; 1 is a schematic configuration diagram of an abnormality detection system according to a first embodiment; FIG. It is a figure which shows the flowchart of the abnormality detection method which concerns on 1st Embodiment. 8 is a block diagram of an abnormality detection system according to a second embodiment; FIG. It is a schematic diagram of the abnormality detection method which concerns on 2nd Embodiment. FIG. 7 is a diagram showing a flowchart of an abnormality detection method according to the second embodiment; 1 is a block diagram of an anomaly detection system according to each embodiment; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings, but the present invention is not limited to these embodiments. In the drawings described below, elements having the same function are denoted by the same reference numerals, and repeated description thereof may be omitted.

(First embodiment)
FIG. 1 is a block diagram of an anomaly detection system 100 according to this embodiment. In FIG. 1, arrows indicate main data flows, and there may be data flows other than those shown in FIG. In FIG. 1, each block does not show the configuration in units of hardware (apparatus), but the configuration in units of functions. As such, the blocks shown in FIG. 1 may be implemented within a single device, or may be implemented separately within multiple devices. Data exchange between blocks may be performed via any means such as a data bus, network, or portable storage medium.

The anomaly detection system 100 includes a log input unit 110, a format determination unit 120, an analysis target distribution generation unit 130, a reference distribution acquisition unit 140, an anomaly detection unit 150, and a notification control unit 160 as processing units. The abnormality detection system 100 also includes a format storage unit 171 and a reference distribution storage unit 172 as storage units.

The log input unit 110 receives the analysis target log 10 to be analyzed and inputs it to the abnormality detection system 100 . The analysis target log 10 may be acquired from the outside of the anomaly detection system 100 or may be acquired by reading out what is previously recorded inside the anomaly detection system 100 . The analysis target log 10 includes one or more logs output from one or more devices or programs. The analysis target log 10 is a log expressed in any data format (file format), and may be binary data or text data, for example. Also, the analysis target log 10 may be recorded as a database table, or may be recorded as a text file.

FIG. 2 is a schematic diagram of an exemplary log 10 to be analyzed. The log to be analyzed 10 in this embodiment includes one log output from a device or program as one unit, and includes any number of logs equal to or greater than one. A log can be a single line string, or it can be a multi-line string. That is, the analysis target log 10 refers to the entirety of the logs included in the analysis target log 10, and the log refers to one log extracted from the analysis target log 10. FIG. Each log contains timestamps, messages, and so on. The anomaly detection system 100 can analyze not only a specific type of log but also a wide range of types of logs. For example, any log that records messages output from an operating system, an application, or the like, such as syslog and event log, can be used as the analysis target log 10 .

The format determination unit 120 determines which format (format) prerecorded in the format storage unit 171 matches each log included in the analysis target log 10, and uses the matching format to determine each log. Separate the log into variable and constant parts. A format is a type of log that is predetermined based on the characteristics of the log. The characteristics of logs include the property that similar logs are easily changed or difficult to change, and the property that a character string indicating easily changed parts in the log is described. The variable part is the part that can change in the format, and the constant part is the part that does not change in the format. A variable part value (including numeric values, character strings and other data) in an input log is called a variable value. The variable part and constant part are different for each format. So what is defined as a variable part in one format can be defined as a constant part in another format, and vice versa.

FIG. 3 is a schematic diagram of an exemplary format recorded in the format storage unit 171. As shown in FIG. A format includes a string representing the format associated with a unique format ID. The format is defined as a variable part by describing a predetermined identifier in the variable part in the log, and defines the part other than the variable part in the log as a constant part. As the identifier of the variable part, for example, "<variable: timestamp>" indicates a variable part that represents a timestamp, "<variable: character string>" indicates a variable part that indicates an arbitrary character string, and "<variable: numeric value >” indicates a variable portion representing an arbitrary numerical value, and “<variable: IP>” indicates a variable portion representing an arbitrary IP address. The identifier of the variable part is not limited to these, and may be defined by any method such as a regular expression or a list of possible values. Also, the format may consist of only a constant part without including a variable part, or may consist of only a variable part without including a constant part.

For example, the format determination unit 120 determines that the log on the third line in FIG. 2 matches the format with the ID of 1 in FIG. Then, the format determination unit 120 processes the log based on the determined format, and obtains a time stamp of "2015/08/17 08:28:37", a character string of "SV003", and a numerical value of " 3258” and the IP address “192.168.1.23” are determined as variable values.

In FIG. 3, the format is represented by a list of character strings for visibility, but may be represented by any data format (file format), such as binary data or text data. Also, the format may be recorded in the format storage unit 171 as a binary file or a text file, or may be recorded in the format storage unit 171 as a database table.

The analysis target distribution generation unit 130, the reference distribution acquisition unit 140, and the anomaly detection unit 150 extract the analysis target distribution generated from the analysis target log 10 and the analysis target distribution from the reference distribution storage unit 172 by the anomaly detection method described below. Detect anomalies using a reference distribution selected based on .

FIG. 4 is a schematic diagram of an abnormality detection method according to this embodiment. A reference distribution matrix A0 is recorded in advance in the reference distribution storage unit 172 . The reference distribution matrix A0 is a set of distributions A1 of the number of log outputs generated for each combination of aggregation units and devices. A log output count distribution A1 is generated for at least two aggregation units and at least two devices. The reference distribution matrix A0 may be generated using the number of log outputs of one format, or may be generated using the sum of the number of log outputs of multiple formats, or the number of log outputs of all formats. may be generated using the sum of That is, in this embodiment, the reference distribution matrix A0 is a set of distributions generated from the number of outputs of one or more predetermined format logs.

The aggregation unit defines the length of the time range to be aggregated to generate the distribution A1 of the number of log outputs and the conditions to which the time range applies. For example, the aggregation unit defines the length of the time range to be aggregated, such as minutes, hours, and days, and further defines that the time range satisfies the conditions of a specific day of the week or starting point. Each aggregation unit is assigned an aggregation unit number (aggregation unit No.) for identification.

FIG. 5 is a schematic diagram of exemplary aggregation units. The aggregation unit is associated with an aggregation unit number (aggregation unit No.) for identification and is defined in advance in the abnormality detection system 100 . The aggregation unit is a rule that defines the time range for generating the log output count distribution A1 included in the reference distribution matrix A0. That is, the distribution A1 of the number of log outputs is generated by aggregating the number of logs (format) that have been output in the past according to the unit of aggregation. For example, if the aggregation unit is "10 minutes", the time-series distribution of logs output in the past 10 minutes (that is, between time N and time N + 10 minutes, time N is arbitrary) among the logs output in the past. Let the average be distribution A1 of the number of log outputs. When the aggregation unit is "one day (Sunday)", the average of the time-series distribution of the logs output on Sunday for one day among the logs output in the past is taken as the distribution A1 of the number of log outputs. When the aggregation unit is "past one day", the time-series distribution of the logs output in one day from the present, among the logs output in the past, is defined as the distribution A1 of the number of log outputs. In this way, the aggregation unit defines the length of the time range for which logs are aggregated and the applicable conditions for the time range.

The aggregation unit shown in FIG. 5 is an example, and some aggregation units shown in FIG. 5 may be used, or other aggregation units not shown in FIG. 5 may be used. In FIG. 5, the units of aggregation are indicated by character strings for the sake of visibility, but actually the units of aggregation may be arbitrary binary data or text data indicating rules for aggregation.

A device is a device that outputs a log. Each device is assigned a device number (device No.) for identification. Even devices of the same type are assigned different device numbers if they are different entities. Also, when a plurality of pieces of software that output logs are executed in the same device, each piece of software may be assigned a different device number. That is, the software that outputs the log is also regarded as one device here.

The log output count distribution A1 is the time-series distribution of the log output count for each combination of aggregation unit and device. That is, the distribution A1 of the number of log outputs indicates the time-series transition of the number of logs output from the device within the time range of the aggregation unit for each combination of the aggregation unit and the device. The log output count distribution A1 is represented as a graph in FIG. good.

The analysis target distribution generation unit 130 generates the analysis target distribution A2 from the analysis target log 10 . The analysis target distribution A2 is a time-series distribution of the number of log outputs to be analyzed. That is, the analysis target distribution A2 indicates the time-series transition of the number of logs output in the analysis target period. The analysis target distribution A2 may be generated using logs output from one device, or may be generated using logs output from a plurality of devices. One or more formats to be aggregated in the analysis target distribution A2 are the same as one or more formats to be aggregated in the reference distribution matrix A0. Although the analysis target distribution A2 is represented as a graph in FIG. 4 for visibility, it may be generated as binary data or text data in which the numbers of log outputs are arranged in chronological order.

The reference distribution acquisition unit 140 acquires, as a reference distribution A3, the distribution A1 of the number of log outputs included in the reference distribution matrix A0 that is most similar to the analysis target distribution A2. Specifically, the reference distribution acquisition unit 140 reads the reference distribution matrix A0 from the reference distribution storage unit 172 when performing abnormality detection. Next, the reference distribution acquisition unit 140 calculates the degree of similarity between each distribution A1 of the number of log outputs included in the reference distribution matrix A0 and the analysis target distribution A2. As the degree of similarity, any index indicating the degree of similarity between the log output count distribution A1 and the analysis object distribution A2 can be used, and for example, a correlation coefficient can be used. The higher the correlation coefficient, the more similar the log output count distribution A1 and the analysis object distribution A2 are. Then, based on the calculated similarity, the reference distribution acquisition unit 140 determines the most similar distribution (for example, the most similar is high) is selected as the reference distribution A3.

The anomaly detection unit 150 detects an anomaly by comparing the reference distribution A3 selected by the reference distribution acquisition unit 140 and the analysis target distribution A2. Specifically, the abnormality detection unit 150 calculates the degree of abnormality of the analysis target distribution A2 with reference to the reference distribution A3. The abnormality detection unit 150 detects an abnormality when the calculated degree of abnormality is not within a predetermined normal range. As the degree of abnormality, any index indicating the extent to which the distribution A2 to be analyzed deviates from the reference distribution A3 can be used, and for example, a correlation coefficient can be used. The lower the correlation coefficient, the greater the deviation of the analysis target distribution A2 from the reference distribution A3.

The notification control unit 160 performs control to notify information indicating an abnormality detected by the abnormality detection unit 150 using the display 20 . The notification of an abnormality by the notification control unit 160 is not limited to the display on the display 20, but can be performed by any method that can notify the user, such as printing by a printer, turning on a lamp, or outputting sound by a speaker. good.

As described above, in this embodiment, the reference distribution acquisition unit 140 selects the reference distribution A3 closest to the analysis target distribution A2 from the distribution A1 of the number of log outputs of various aggregation units and devices, and the abnormality detection unit 150 selects Based on the reference distribution A3 thus obtained, it is determined whether or not the distribution A2 to be analyzed is abnormal. Therefore, an abnormality can be detected with high accuracy using a reference closer to the analysis target distribution A2.

In this embodiment, the analysis object distribution A2 is compared with all the log output number distributions A1 included in the reference distribution matrix A0. The comparison may be limited to . For example, the analysis target distribution generation unit 130 generates the analysis target distribution A2 in 10-minute aggregation units, and the reference distribution acquisition unit 140 obtains the log output number distribution A1 of aggregation unit number 1 (10 minutes) from the reference distribution matrix A0. Extract and calculate the similarity, and select the reference distribution A3 from among them.

In this embodiment, an abnormality is detected using one analysis target distribution A2, but an abnormality may be detected using a combination of a plurality of analysis target distributions A2. For example, the analysis target distribution generation unit 130 generates two analysis target distributions A2 from the analysis target log 10 in different aggregation units (for example, 10 minutes and 1 hour). Next, the reference distribution acquisition unit 140 calculates two degrees of similarity with respect to two distributions A1 of the number of log outputs in the same counting unit (that is, 10 minutes and 1 hour) for each device number. Then, the reference distribution acquisition unit 140 selects the two distributions A1 of the number of log outputs having the highest sum of the two similarities for one device number (that is, the most similar) as the two reference distributions A3. The number of analysis target distributions A2 is not limited to two, and may be any number.

FIG. 6 is a schematic configuration diagram showing an exemplary configuration of the abnormality detection system 100 according to this embodiment. The abnormality detection system 100 includes a CPU (Central Processing Unit) 101 , a memory 102 , a storage device 103 , a communication interface 104 and a display 20 . The anomaly detection system 100 may be a separate device or integrated with other devices.

The communication interface 104 is a communication unit that transmits and receives data, and is configured to be capable of executing at least one of wired communication and wireless communication. The communication interface 104 includes a processor, an electric circuit, an antenna, connection terminals, etc. necessary for the communication system. The communication interface 104 is connected to a network using the communication method according to a signal from the CPU 101 and performs communication. The communication interface 104 receives, for example, the analysis target log 10 from the outside.

The storage device 103 stores programs executed by the anomaly detection system 100, data of processing results of the programs, and the like. The storage device 103 includes a read-only ROM (Read Only Memory), a readable/writable hard disk drive, a flash memory, or the like. The storage device 103 may also include a computer-readable portable storage medium such as a CD-ROM. The memory 102 includes a RAM (Random Access Memory) for temporarily storing data being processed by the CPU 101 and programs and data read from the storage device 103 .

The CPU 101 temporarily records temporary data used for processing in the memory 102, reads a program recorded in the storage device 103, and performs various calculations, controls, determinations, etc. on the temporary data according to the program. is a processor that executes the processing operations of The CPU 101 also records the data of the processing result in the storage device 103 and transmits the data of the processing result to the outside via the communication interface 104 .

In this embodiment, the CPU 101 executes the program recorded in the storage device 103 to perform the following functions: the log input unit 110, the format determination unit 120, the analysis target distribution generation unit 130, the reference distribution acquisition unit 140, and the abnormality detection unit shown in FIG. 150 and notification control unit 160 . Further, in this embodiment, the storage device 103 functions as the format storage unit 171 and the reference distribution storage unit 172 in FIG.

The display 20 is a display device that displays information to the user. As the display 20, any display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or the like may be used. A display 20 displays predetermined information according to a signal from the CPU 101 .

Anomaly detection system 100 is not limited to the specific configuration shown in FIG. The anomaly detection system 100 is not limited to one device, and may be configured by connecting two or more physically separated devices by wire or wirelessly. Each unit included in the abnormality detection system 100 may be realized by an electric circuit configuration. Here, an electrical circuit configuration is a term that conceptually includes a single device, multiple devices, a chipset, or a cloud.

Also, at least part of the anomaly detection system 100 may be provided in a SaaS (Software as a Service) format. That is, at least part of the functions for realizing the anomaly detection system 100 may be executed by software executed via a network.

FIG. 7 is a diagram showing a flowchart of an abnormality detection method using the abnormality detection system 100 according to this embodiment. The flowchart of FIG. 7 is started, for example, by the user performing a predetermined operation for executing anomaly detection on the anomaly detection system 100 . First, the log input unit 110 receives the analysis target log 10 and inputs it to the abnormality detection system 100 (step S101). The format determination unit 120 determines which format recorded in the format storage unit 171 is compatible with each log included in the log to be analyzed 10 input in step S101 (step S102).

Next, the analysis target distribution generation unit 130 extracts a log of a predetermined format to be analyzed from the logs whose format was determined in step S102, and generates the time-series distribution of the number of outputs of the log as the analysis target distribution ( step S103). Here, the analysis target distribution may be generated for the entire period of the analysis target log, or the analysis target distribution may be generated for one or a plurality of predetermined aggregation units.

The reference distribution acquisition unit 140 reads out the reference distribution matrix from the reference distribution storage unit 172, and calculates the degree of similarity between each distribution of the number of log outputs included in the reference distribution matrix and the analysis target distribution generated in step S103. Calculate (step S104). Then, based on the similarity calculated in step S104, the reference distribution acquisition unit 140 obtains the most similar distribution (for example, the most similarity is selected as the reference distribution (step S105).

The anomaly detection unit 150 calculates the anomaly degree of the analysis target distribution based on the reference distribution selected in step S105. Then, the abnormality detection unit 150 detects an abnormality when the calculated degree of abnormality is not within a predetermined normal range (step S106).

When an abnormality is detected in step S106 (YES in step S107), the notification control unit 160 performs control to notify information indicating the abnormality detected in step S106 using the display 20 (step S108). After the notification in step S108, or when no abnormality is detected in step S106 (NO in step S107), the abnormality detection method is terminated.

The CPU 101 of the abnormality detection system 100 is the subject of each step (process) included in the abnormality detection method shown in FIG. That is, the CPU 101 reads a program for executing the abnormality detection method shown in FIG. Run the detection method.

In the conventional anomaly detection method, the averaged distribution of various aggregation units and devices was used as the standard for anomaly detection. was there. On the other hand, in the anomaly detection system 100 according to the present embodiment, the reference distribution acquisition unit 140 selects the reference distribution A3 that is closest to the analysis target distribution A2 from the distributions A1 of the number of log outputs of various aggregation units and devices. , the selected reference distribution A3 is used as a reference to determine whether or not the analysis target distribution A2 is abnormal. Therefore, it is possible to detect anomalies with high accuracy using a criterion close to the analysis target distribution A2 by taking advantage of the characteristics of the distribution for each aggregation unit and for each device.

(Second embodiment)
A first embodiment detects anomalies in the analyzed distribution by comparing the analyzed distribution to a reference distribution matrix. In contrast, the present embodiment detects abnormal distributions or abnormal devices by extracting outlier distributions from the reference distribution matrix.

FIG. 8 is a block diagram of an anomaly detection system 200 according to this embodiment. In FIG. 8, arrows indicate main data flows, and there may be data flows other than those shown in FIG. In FIG. 8, each block does not show the configuration in units of hardware (apparatus), but the configuration in units of functions. As such, the blocks shown in FIG. 8 may be implemented within a single device, or may be implemented separately within multiple devices. Data exchange between blocks may be performed via any means such as a data bus, network, or portable storage medium.

The anomaly detection system 200 includes a reference distribution acquisition unit 240, an anomaly detection unit 250, and a notification control unit 260 as processing units. The abnormality detection system 100 also includes a reference distribution storage unit 272 as a storage unit. The equipment configuration of the anomaly detection system 200 may be the same as in FIG. The reference distribution acquisition unit 240 and the abnormality detection unit 250 detect an abnormality from the reference distribution matrix by an abnormality detection method described below.

FIG. 9 is a schematic diagram of an abnormality detection method according to this embodiment. A reference distribution matrix B0 is recorded in advance in the reference distribution storage unit 272 . The reference distribution matrix B0 is a set of distributions B1 of the number of log outputs generated for each combination of aggregation units and devices. Aggregation units and device definitions are the same as in the first embodiment. Further, the log output number distribution B1 is generated by the same method as the log output number distribution A1 of the first embodiment.

The reference distribution acquisition unit 240 reads the reference distribution matrix B0 from the reference distribution storage unit 272 when performing abnormality detection. The abnormality detection unit 250 calculates a device average distribution by averaging the log output count distribution B1 included in the reference distribution matrix B0 for each device (device number). Next, the abnormality detection unit 250 calculates the degree of similarity between the calculated device average distributions, and generates a group of devices based on the degree of similarity (for example, a group of devices having a degree of similarity equal to or greater than a predetermined threshold). Devices may be grouped using known clustering methods. The abnormality detection unit 250 performs the following abnormality detection processing for each device group. Alternatively, the following abnormality detection processing may be performed on all devices without grouping the devices.

The abnormality detection unit 250 calculates an average distribution for each aggregation unit by averaging the distribution B1 for each aggregation unit, and calculates the degree of similarity between the calculated average distribution and each distribution B1 for that aggregation unit. Then, the abnormality detection unit 250 extracts the distribution B1 that deviates from the average distribution for each aggregation unit (for example, the similarity to the average distribution is equal to or less than a predetermined threshold) as an abnormal distribution B2. Alternatively, a predetermined number of distributions B1 may be extracted as abnormal distributions B2 in descending order of similarity to the average distribution.

As another method, the anomaly detection unit 250 calculates the similarity between the distributions B1 included in each aggregation unit, and calculates the similarity between the distributions B1 within the same aggregation unit, i.e., the sum of the similarities to the other distributions. A distribution B1 with a low value or average value is extracted as an abnormal distribution B2. Alternatively, a predetermined number of distributions B1 may be extracted as abnormal distributions B2 in descending order of total or average similarity with other distributions.

The method for extracting the abnormal distribution B2 from the reference distribution matrix B0 is not limited to the method shown here, and any method that can extract the distribution deviating from each aggregation unit may be used.

Furthermore, the abnormality detection unit 250 calculates the number or ratio of the abnormal distribution B2 in the distribution B1 related to each device (device number), and treats the devices for which the calculated number or ratio is equal to or greater than a predetermined threshold as abnormal devices. To detect.

The notification control unit 260 performs control for notifying the information indicating the abnormality detected by the abnormality detection unit 250 using the display 20 . The notification of an abnormality detected by the notification control unit 260 is not limited to the display on the display 20, but can be made by any method that can notify the user, such as printing by a printer, turning on a lamp, or outputting sound by a speaker. may be done.

FIG. 10 is a diagram showing a flowchart of an abnormality detection method using the abnormality detection system 200 according to this embodiment. The flowchart of FIG. 10 is started, for example, by the user performing a predetermined operation for executing anomaly detection on the anomaly detection system 200 . First, the reference distribution acquisition unit 240 reads and acquires the reference distribution matrix from the reference distribution storage unit 272 (step S201).

The abnormality detection unit 250 calculates a device average distribution by averaging the distribution of the number of log outputs included in the reference distribution matrix acquired in step S201 for each device (device number). Next, the abnormality detection unit 250 calculates the degree of similarity between the calculated device average distributions, and generates device groups based on the degree of similarity (step S202). The following processing is performed for each device group.

The anomaly detection unit 250 extracts, as an anomalous distribution, the distribution of the number of log outputs included in the reference distribution matrix acquired in step S201, which deviates from each aggregation unit (step S203).

Furthermore, the abnormality detection unit 250 calculates the number or ratio of the abnormal distribution extracted in step S204 for each device (device number), and treats the devices for which the calculated number or ratio is equal to or greater than a predetermined threshold as abnormal devices. Detect (step S204).

When an abnormal distribution or device is detected in steps S203-S204 (YES in step S205), the notification control unit 260 uses the display 20 to notify information indicating the abnormality detected in steps S203-S204. Control is performed (step S206). After the notification in step S206, or when no abnormality is detected in steps S203 and S204 (NO in step S205), the abnormality detection method ends.

The CPU 101 of the abnormality detection system 200 is the subject of each step (process) included in the abnormality detection method shown in FIG. 10 from the memory 102 or the storage device 103, and executes the program to control each part of the abnormality detection system 200 to detect the abnormality shown in FIG. Run the detection method.

As described above, the anomaly detection system 200 according to the present embodiment detects an anomaly by extracting the distribution B2 that deviates from the distribution B1 of the number of log outputs of various aggregation units and devices. Therefore, an abnormal distribution or device can be detected by making use of the characteristics of the distribution for each aggregation unit and for each device.

(Other embodiments)
FIG. 11 is a schematic configuration diagram of the abnormality detection systems 100 and 200 according to each of the above-described embodiments. FIG. 11 shows a configuration example for the anomaly detection systems 100 and 200 to function as devices that detect an anomaly using the distribution of the number of log outputs generated for different total units and different devices. Anomaly detection systems 100 and 200 include distribution acquisition units 140 and 240 that acquire a plurality of distributions generated for each device that outputs a log and for each time range unit for aggregating the logs, and using the plurality of distributions: and abnormality detection units 150 and 250 that detect an abnormality, and each of the plurality of distributions is a time-series distribution of the number of log outputs in the unit.

The present invention is not limited to the above-described embodiments, and can be modified as appropriate without departing from the gist of the present invention.

A program for operating the configuration of the embodiment so as to realize the functions of the embodiment described above (more specifically, an abnormality detection program for causing a computer to execute the processes shown in FIGS. 7 and 10) is recorded on a recording medium, A processing method in which the program recorded on the recording medium is read as code and executed by a computer is also included in the category of each embodiment. That is, a computer-readable recording medium is also included in the scope of each embodiment. In addition to the recording medium on which the above program is recorded, the program itself is also included in each embodiment.

For example, a floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, CD-ROM, magnetic tape, nonvolatile memory card, and ROM can be used as the recording medium. Further, not only the program recorded on the recording medium alone executes the process, but also the one that operates on the OS and executes the process in cooperation with other software and functions of the expansion board. included in the category of

Some or all of the above-described embodiments can also be described as the following additional remarks, but are not limited to the following.

(Appendix 1)
acquiring a plurality of distributions generated for each device that outputs logs and for each unit of time range for aggregating the logs;
detecting anomalies using the plurality of distributions;
including
The anomaly detection method, wherein each of the plurality of distributions is a time-series distribution of the number of log outputs in the unit.

(Appendix 2)
further comprising the step of generating an analysis target distribution, which is a time-series distribution of the number of log outputs included in the analysis target log;
The obtaining step selects a reference distribution that is most similar to the analysis target distribution from the plurality of distributions,
The anomaly detection method according to appendix 1, wherein the step of detecting detects the anomaly in the analysis target log by comparing the analysis target distribution with the reference distribution.

(Appendix 3)
In the obtaining step, a similarity between the analysis target distribution and each of the plurality of distributions is calculated, and among the plurality of distributions, a distribution whose similarity is equal to or higher than a predetermined threshold is selected as the reference distribution. The abnormality detection method according to appendix 2, characterized by:

(Appendix 4)
In the step of detecting, the degree of abnormality of the distribution to be analyzed is calculated based on the reference distribution, and the abnormality is detected when the degree of abnormality is not within a predetermined normal range, Supplementary Note 2 or 4. The abnormality detection method according to 3.

(Appendix 5)
The anomaly detection method according to appendix 1, wherein the anomaly is detected by extracting a distribution deviating from the plurality of distributions as an anomalous distribution.

(Appendix 6)
6. The anomaly detection method according to appendix 5, wherein mutual similarity of the plurality of distributions is calculated, and distributions deviating from the plurality of distributions are extracted based on the mutual similarity.

(Appendix 7)
Abnormality detection according to appendix 5 or 6, characterized in that the abnormality is detected by extracting, as an abnormal device, the device for which the number or ratio of the abnormal distribution for each device is equal to or greater than a predetermined threshold value. Method.

(Appendix 8)
8. The anomaly detection method according to any one of appendices 1 to 7, wherein the unit indicates a length of the time range and a condition to which the time range applies.

(Appendix 9)
to the computer,
obtaining a plurality of distributions generated for each device that outputs logs and for each unit of time range for aggregating the logs;
detecting anomalies using the plurality of distributions;
and
An anomaly detection program, wherein each of the plurality of distributions is a time-series distribution of the number of log outputs in the unit.

(Appendix 10)
a distribution acquisition unit that acquires a plurality of distributions generated for each device that outputs logs and for each unit of time range for aggregating the logs;
an anomaly detection unit that detects an anomaly using the plurality of distributions;
with
An anomaly detection system, wherein each of the plurality of distributions is a time-series distribution of the number of log outputs in the unit.

Claims (11)

First log information containing information about the number of log outputs of the first device in the first time span and second log information containing information about the number of log outputs of the second device in the second time span Acquired,
Acquiring analysis target log information including information on the number of log outputs of the analysis target device in the third time span,
determining the first log information or the second log information as reference log information based on the degree of similarity between the first log information and the second log information and the log information to be analyzed;
determining whether there is an abnormality in the analysis target device based on the analysis target log information and the reference log information;
Anomaly judgment method. 2. The abnormality determination method according to claim 1, wherein said first device is the same device as said analysis target device. 2. The abnormality determination method according to claim 1, wherein neither said first device nor said second device is the same device as said device to be analyzed. 4. The abnormality determination method according to claim 1, wherein said first duration is the same duration as said third duration. 4. The abnormality determination method according to claim 1, wherein said first time width and said second time width are not the same time width as said third time width. First log information containing information about the number of log outputs of the first device in the first time span and second log information containing information about the number of log outputs of the second device in the second time span Acquired,
Acquiring analysis target log information including information on the number of log outputs of the analysis target device in the third time span,
Acquisition for determining the first log information or the second log information as reference log information based on the degree of similarity between the first log information and the second log information and the log information to be analyzed Department and
An abnormality determination system, comprising: a determination unit that determines whether or not there is an abnormality in the analysis target device based on the analysis target log information and the reference log information. 7. The abnormality determination system according to claim 6, wherein said first device is the same device as said analysis target device. 7. The abnormality determination system according to claim 6, wherein neither said first device nor said second device is the same device as said device to be analyzed. The abnormality determination system according to any one of claims 6 to 8, wherein said first duration is the same duration as said third duration. 9. The abnormality determination system according to claim 6, wherein neither said first time width nor said second time width is the same time width as said third time width. First log information containing information about the number of log outputs of the first device in the first time span and second log information containing information about the number of log outputs of the second device in the second time span Acquired,
Acquiring analysis target log information including information on the number of log outputs of the analysis target device in the third time span,
determining the first log information or the second log information as reference log information based on the degree of similarity between the first log information and the second log information and the log information to be analyzed;
determining whether there is an abnormality in the analysis target device based on the analysis target log information and the reference log information;
A program for executing a process on a computer.

Images