JPWO2017085921A1 - Log analysis system, method and program - Google Patents

Abstract

The present invention provides a log analysis system, method, and program for outputting information suggesting the cause of an abnormality to a user even for a log in which a rule indicating the cause of the abnormality is not defined. A log analysis system 100 according to an embodiment of the present invention includes a format determination unit 120 as a variable extraction unit that extracts a value of a variable portion from an analysis target log 10 based on a format determined in advance based on log characteristics And a weighting unit 140 as a log analysis unit for weighting the analysis target log based on the value of the variable part.

Description

  The present invention relates to a log analysis system, method, and program for performing log analysis.

  Generally, in a system executed on a computer, logs including event results, messages, and the like are output from a plurality of devices and programs. The log analysis system detects an abnormal thing according to a predetermined standard from the outputted logs, and outputs it to a user (operator or the like) as an abnormal log.

  Since a plurality of devices and programs work together in the system, the cause of the abnormality may not be directly identified from a single abnormal log. In that case, the user needs to search the cause of the abnormality by referring to a large number of logs. In particular, a user with little experience or knowledge takes a long time to reach the cause of the abnormality from the log.

  Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for previously registering an event pattern and its cause and countermeasure method in association with each other based on past knowledge, and acquiring the cause and countermeasure method for the event pattern of the input log. By using the technique of Patent Document 1, the user can quickly know the cause for the registered event pattern.

  Further, in Patent Document 2, a location indicating a component is specified from a log, an influence corresponding to the component is determined from a predetermined influence list, and the importance of the log is determined based on the influence. A technique for setting is disclosed.

Japanese Patent No. 4318643 JP 2013-030092 A

  However, although the technique of Patent Document 1 can acquire a cause for a registered event pattern, it cannot acquire a cause for an unregistered event pattern. That is, since the technique of Patent Document 1 indicates the cause of abnormality by defining rules based on knowledge individually in advance, it is applied to a log that does not have knowledge (information) defined in advance regarding the cause of abnormality. Can not do it.

  Further, the technique of Patent Document 2 requires that the degree of influence for each component is registered in advance in the degree of influence list. For this reason, the technique of Patent Document 2 is premised on knowledge (information) defined in advance for each component, and cannot be applied to a log having no knowledge for each component.

  The present invention has been made in view of the above-described problem, and a log that outputs information indicating the cause of an abnormality to a user even for a log in which a rule indicating the cause of the abnormality is not defined An object is to provide an analysis system, method and program.

  A first aspect of the present invention is a log analysis system, comprising: a variable extraction unit that extracts a value of a variable part from an analysis target log based on a format determined in advance based on characteristics of the log; and A log analysis unit that weights the analysis target log based on the value.

  According to a second aspect of the present invention, there is provided a log analysis method, the step of extracting a value of a variable part from an analysis target log based on a format predetermined based on a characteristic of the log, and the value of the variable part. And a step of weighting the analysis target log.

  According to a third aspect of the present invention, there is provided a log analysis program, comprising: extracting a value of a variable part from an analysis target log based on a format predetermined based on a log characteristic; And a step of weighting the analysis target log based on the value of.

  According to the present invention, since weighting is performed based on the value of the variable part included in the analysis target log, the user searches for the cause of the abnormality by referring to the abnormal log or the value of the variable part included in the abnormal log. Becomes easier.

It is a block diagram of the log analysis system concerning a 1st embodiment. It is a schematic diagram of the analysis object log which concerns on 1st Embodiment. It is a schematic diagram of a format according to the first embodiment. It is a schematic diagram of the weighting result which concerns on 1st Embodiment. It is a schematic diagram which shows the display screen of the weighting result which concerns on 1st Embodiment. It is a schematic diagram which shows the display screen of the weighting result which concerns on 1st Embodiment. 1 is a schematic configuration diagram of a log analysis system according to a first embodiment. It is a figure which shows the flowchart of the log analysis method using the log analysis system which concerns on 1st Embodiment. It is a block diagram of the log analysis system concerning the modification of a 1st embodiment. It is a schematic diagram which shows the display screen of the weighting result which concerns on the modification of 1st Embodiment. It is a schematic diagram which shows the display screen of the weighting result which concerns on the modification of 1st Embodiment. It is a block diagram of the log analysis system concerning the modification of a 1st embodiment. It is a block diagram of the log analysis system concerning a 2nd embodiment. It is a schematic diagram of the distribution information which concerns on 2nd Embodiment. It is a schematic diagram of the distribution information which concerns on 2nd Embodiment. It is a schematic diagram which shows the display screen of the weighting result which concerns on 2nd Embodiment. It is a schematic diagram which shows the display screen of the weighting result which concerns on 2nd Embodiment. It is a schematic diagram which shows the display screen of the weighting result which concerns on 2nd Embodiment. It is a schematic diagram which shows the display screen of the weighting result which concerns on 2nd Embodiment. It is a block diagram of the log analysis system concerning a 3rd embodiment. It is a schematic diagram of the combination information which concerns on 3rd Embodiment. It is a schematic diagram which shows the display screen of the weighting result which concerns on 3rd Embodiment. It is a schematic diagram which shows the display screen of the weighting result which concerns on 3rd Embodiment. It is a schematic diagram which shows the display screen of the weighting result which concerns on 3rd Embodiment. It is a block diagram of the log analysis system concerning the modification of a 3rd embodiment. 1 is a schematic configuration diagram of a log analysis system according to each embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the embodiments. In the drawings described below, components having the same function are denoted by the same reference numerals, and repeated description thereof may be omitted.

(First embodiment)
FIG. 1 is a block diagram of a log analysis system 100 according to the present embodiment. In FIG. 1, arrows indicate main data flows, and there may be data flows other than those shown in FIG. In FIG. 1, each block shows a functional unit configuration, not a hardware (device) unit configuration. Therefore, the blocks shown in FIG. 1 may be implemented in a single device, or may be separately implemented in a plurality of devices. Data exchange between the blocks may be performed via any means such as a data bus, a network, a portable storage medium, or the like.

  The log analysis system 100 includes a log input unit 110, a format determination unit 120, a log abnormality analysis unit 130, a weighting unit 140, and an output unit 150 as processing units. In addition, the log analysis system 100 includes a format storage unit 161 and a model storage unit 162 as storage units.

  The log input unit 110 acquires the analysis target log 10 and inputs it to the log analysis system 100. The analysis target log 10 may be acquired from the outside of the log analysis system 100, or may be acquired by reading what is recorded in advance in the log analysis system 100. The analysis target log 10 includes one or more logs output from one or more devices or programs. The analysis target log 10 is a log expressed in an arbitrary data format (file format), and may be binary data or text data, for example. The analysis target log 10 may be recorded as a database table or may be recorded as a text file.

  FIG. 2A is a schematic diagram of an exemplary analysis target log 10. The analysis target log 10 in this embodiment includes one log output from the apparatus or program as one unit, and includes one or more arbitrary numbers of logs. One log may be a single-line character string, or may be a multi-line character string. That is, the analysis target log 10 indicates the total number of logs included in the analysis target log 10, and the log indicates one log extracted from the analysis target log 10. Each log includes a time stamp and a message. The log analysis system 100 is not limited to a specific type of log, and can analyze a wide variety of logs. For example, a log that records a message output from an operating system such as a syslog or an event log can be used as the analysis target log 10. In addition, a log of a security device on the network such as IDS (Instruction Detection System) or IPS (Intrusion Prevention System) can also be used as the analysis target log 10.

  The format determination unit 120 is a variable extraction unit, determines which format recorded in advance in the format storage unit 161 for each log included in the analysis target log 10, and selects a compatible format. To separate each log into variable and constant parts. The format is a log format determined in advance based on log characteristics. The log characteristics include a property that it is easy or difficult to change between logs that are similar to each other, and a property that a character string that can be regarded as a portion that easily changes in the log is described. The variable portion is a changeable portion in the format, and the constant portion is a portion that does not change in the log format. The value of the variable part in the input log (including numerical values, character strings, and other data) is called a variable value. The variable part and the constant part are different for each format. Therefore, a part defined as a variable part in one format may be defined as a constant part in another format, and vice versa. In this embodiment, since the log is analyzed using the format determined based on the characteristics of the log in this way, the cause of the abnormality can be determined without knowledge of the event pattern or the component that is the cause of the abnormality. Suggestive information can be provided.

  FIG. 2B is a schematic diagram of an exemplary format recorded in the format storage unit 161. The format includes a character string that represents the format associated with the unique ID. The format is defined as a variable part by describing a predetermined identifier in a variable part in the log, and a part other than the variable part in the log is defined as a constant part. For example, “<variable: timestamp>” indicates a variable portion representing a time stamp, “<variable: character string>” indicates a variable portion representing an arbitrary character string, and “<variable: numerical value”. ">" Represents a variable part representing an arbitrary numerical value, and "<variable: IP>" represents a variable part representing an arbitrary IP address. Variable identifiers are not limited to these, and may be defined by an arbitrary method such as a regular expression or a list of possible values. Further, the format may be configured only by the constant part without including the variable part, or may be configured only by the variable part without including the constant part.

  For example, the format determination unit 120 determines that the log in the third row in FIG. 2A is compatible with the format whose ID is 223 in FIG. 2B. Then, the format determination unit 120 processes the log based on the determined format, and includes “2015/08/17 08:29:59” as a time stamp, “SV008” as a character string, and an IP address. “192.168.1.23” is determined as a variable value.

  In FIG. 2B, the format is represented by a list of character strings for visibility, but may be represented in any data format (file format), for example, binary data or text data. The format may be recorded as a text file in the format storage unit 161 or may be recorded in the format storage unit 161 as a database table.

  The log abnormality analysis unit 130 determines whether the variable value in the log determined by the format determination unit 120 is abnormal based on a model recorded in advance in the model storage unit 162. A model is a definition of the normal behavior of a log. One or more models are recorded in the model storage unit 162 in advance. The model is, for example, that a numeric variable value is within a predetermined range in a certain format, or that a character string variable value is already registered in a certain format. The model is not limited to this, and any definition may be used.

  The log abnormality analysis unit 130 determines that the log is abnormal when the input log does not match any model in the model storage unit 162, and inputs the log to the next weighting unit 140 as an abnormality log. On the other hand, when the input log matches any model in the model storage unit 162, the log abnormality analysis unit 130 determines that the log is a normal log and does not input the log to the weighting unit 140.

  The weighting unit 140 is a log analysis unit, and weights the abnormality log output from the log abnormality analysis unit 130. In the present embodiment, the weighting unit 140 first extracts a variable value that is commonly included in two or more abnormality logs from among the variable values included in the plurality of abnormality logs input from the log abnormality analysis unit 130. A variable value that is commonly included in two or more abnormality logs is referred to as a common variable value. Furthermore, the weighting unit 140 calculates the number (frequency) at which each common variable value appears in the plurality of abnormality logs input from the log abnormality analysis unit 130. Then, the weighting unit 140 associates the common variable value with the calculated frequency, weights the common variable value with a higher frequency so as to be higher, and outputs the weighted result. In other words, in this embodiment, based on whether or not a variable value that is a part of the abnormality log is included in two or more abnormality logs, and the frequency of occurrence of the variable value in the abnormality log, Perform weighting. There is a high probability that the variable values included in the plurality of abnormality logs are the cause of the abnormality or have some relationship with the abnormality. Therefore, the log analysis system 100 can make it easier for the user to identify the cause of the abnormality by notifying the user of such a variable value.

  FIG. 2C is a schematic diagram illustrating an exemplary weighting result by the weighting unit 140. The weighting result includes a common variable value, a calculated frequency, and a ranking given in descending order of frequency. In FIG. 2C, the weighting result is represented by a list of character strings and numerical values for visibility, but may be represented in an arbitrary data format (file format), for example, binary data or text data.

  The output unit 150 outputs the weighting result from the weighting unit 140. In the present embodiment, the output unit 150 outputs the weighting result to the display device 20, and the display device 20 displays the weighting result as an image for the user. The display device 20 includes a display unit such as a liquid crystal display or a CRT (Cathode Ray Tube) display for displaying an image.

  FIGS. 3A and 3B are schematic diagrams illustrating exemplary weighting result display screens using the display device 20. Screen A shown in FIG. 3A displays the common variable values rearranged according to the weighting results by weighting section 140 and their ranks. The common variable value is displayed higher as the rank is higher, that is, as the frequency is higher. Furthermore, on the screen A, “SV008” which is the highest-order variable value is highlighted by bold and underline. Thereby, the user can easily know a variable value having a high probability of causing an abnormality. In order to emphasize the upper variable value, the character type, color, size, etc. of the variable value may be changed.

  Screen B shown in FIG. 3B displays the common variable values rearranged according to the weighting result by weighting unit 140 and their ranks, and the abnormality log output from log abnormality analysis unit 130 including the common variable value. To do. Further, on the screen B, “SV008” which is the highest variable value is highlighted in bold and underlined in the abnormality log. As a result, the user can easily know the variable value that is highly likely to cause an abnormality and the appearance location in the log. In order to emphasize the upper variable value, the character type, color, size, etc. of the variable value may be changed.

  The screens shown in FIGS. 3A and 3B are examples, and any display method may be used as long as the log or a variable value that is a part of the log can be displayed so that the weighting result by the weighting unit 140 can be understood by the user.

  The common variable value included in the weighting result is included in two or more abnormality logs. Therefore, the user can estimate that the probability that the common variable value displayed on the screens of FIGS. 3A and 3B is the cause of the abnormality is high. Furthermore, since the higher common variable value in the weighting result has a higher appearance frequency in the abnormality log, the user has a higher probability that the upper one among the common variable values displayed on the screen is the cause of the abnormality. Can be guessed.

  The output method of the weighting result is not limited to image display for the user. For example, the output unit 150 may output the weighting result as data, and the log analysis system 100 or other system may perform analysis processing, statistical processing, or the like on the weighting result data from the output unit 150. The log analysis system 100 or other system may automatically determine the cause of the abnormality based on the weighted result data from the output unit 150.

  FIG. 4 is a schematic configuration diagram illustrating an exemplary device configuration of the log analysis system 100 according to the present embodiment. The log analysis system 100 includes a CPU (Central Processing Unit) 101, a memory 102, a storage device 103, and a communication interface 104. The log analysis system 100 may be connected to the display device 20 via the communication interface 104 or may include the display device 20. The log analysis system 100 may be an independent device or may be integrated with other devices.

  The communication interface 104 is a communication unit that transmits and receives data, and is configured to be able to execute at least one communication method of wired communication and wireless communication. The communication interface 104 includes a processor, an electric circuit, an antenna, a connection terminal, and the like necessary for the communication method. The communication interface 104 is connected to a network using the communication method in accordance with a signal from the CPU 101 to perform communication. For example, the communication interface 104 receives the analysis target log 10 from the outside.

  The storage device 103 stores a program executed by the log analysis system 100, data of a processing result by the program, and the like. The storage device 103 includes a read-only ROM (Read Only Memory), a readable / writable hard disk drive, a flash memory, or the like. The storage device 103 may include a computer-readable portable storage medium such as a CD-ROM. The memory 102 includes a RAM (Random Access Memory) that temporarily stores data being processed by the CPU 101, a program read from the storage device 103, and data.

  The CPU 101 temporarily records temporary data used for processing in the memory 102, reads a program recorded in the storage device 103, and performs various calculations, control, discrimination, etc. on the temporary data according to the program It is a processor as a process part which performs these processing operations. In addition, the CPU 101 records processing result data in the storage device 103 and transmits processing result data to the outside via the communication interface 104.

  In this embodiment, the CPU 101 functions as the log input unit 110, the format determination unit 120, the log abnormality analysis unit 130, the weighting unit 140, and the output unit 150 in FIG. 1 by executing a program recorded in the storage device 103. . In the present embodiment, the storage device 103 functions as the format storage unit 161 and the model storage unit 162 in FIG.

  The log analysis system 100 is not limited to the specific configuration shown in FIG. The log analysis system 100 is not limited to a single device, and may be configured by connecting two or more physically separated devices in a wired or wireless manner. Each unit included in the log analysis system 100 may be realized by an electric circuit configuration. Here, the electric circuit configuration is a term that conceptually includes a single device, a plurality of devices, a chipset, or a cloud.

  In addition, at least a part of the log analysis system 100 may be provided in SaaS (Software as a Service) format. That is, at least a part of functions for realizing the log analysis system 100 may be executed by software executed via a network.

  FIG. 5 is a diagram illustrating a flowchart of a log analysis method using the log analysis system 100 according to the present embodiment. First, the log input unit 110 acquires the analysis target log 10 and inputs it to the log analysis system 100 (step S101). The format determination unit 120 determines whether one format included in the analysis target log 10 input in step S101 is a determination target, and is compatible with any of the formats recorded in the format storage unit 161 (step) S102).

  If the determination target log does not conform to any format recorded in the format storage unit 161 in step S102 (NO in step S103), steps S102 to S103 are performed with the next log of the analysis target log 10 as the determination target. repeat.

  If the determination target log matches any format recorded in the format storage unit 161 in step S102 (YES in step S103), the format determination unit 120 uses the format to change the determination target log to a variable. A part and a constant part are separated (step S104). The format determination unit 120 records the variable value in the determination target log.

  The log abnormality analysis unit 130 determines whether or not the determination target log matches any model recorded in the model storage unit 162 based on the variable value acquired in step S104 (step S105). When the determination target log does not match any model recorded in the model storage unit 162, the log abnormality analysis unit 130 determines that the determination target log is an abnormality log. On the other hand, when the determination target log matches any model recorded in the model storage unit 162, the log abnormality analysis unit 130 determines that the determination target log is a normal log.

  If the analysis has not been completed for all the logs in the analysis target log 10 (NO in step S106), steps S102 to S106 are repeated with the next log of the analysis target log 10 as a determination target.

  When the analysis is completed for all the logs in the analysis target log 10 (YES in step S106), the weighting unit 140 selects two or more of the variable values included in the abnormality log determined in step S105. The variable values (common variable values) included in the abnormality log are extracted, and the number (frequency) at which each common variable value appears in the abnormality log is calculated. Then, the weighting unit 140 associates the acquired common variable value with the frequency, weights the higher so that the higher the frequency, and outputs the weighted result (step S107).

  Finally, the output unit 150 outputs the weighting result acquired in step S107 to the display device 20 and displays it for the user (step S108).

  As described above, the log analysis system 100 indicates a variable value having a high probability of causing an abnormality to the user by performing weighting based on a variable value that is commonly included in a plurality of abnormality logs. . This makes it easy for the user to search for the cause of the abnormality.

  Next, a modified example of the log analysis system 100 according to the first embodiment will be described. FIG. 6 is a block diagram of a log analysis system 100-1 according to a first modification of the first embodiment. The log analysis system 100-1 includes a component extraction unit 131 and a configuration information storage unit 163 in addition to the configuration of FIG.

  The configuration information storage unit 163 records configuration information related to the system that outputs the analysis target log 10 in advance. The configuration information includes components included in the system that outputs the analysis target log 10 (for example, a physical device such as a server, a virtual device such as a virtual machine, various programs, and the like), and a dependency relationship between the components (a network connection relationship, Information indicating the master-slave relationship of virtual devices and programs).

  The component extraction unit 131 extracts a variable value that matches the component recorded in the configuration information storage unit 163 from among the variable values included in the abnormality log input from the log abnormality analysis unit 130. That is, the weighting unit 140 extracts and outputs the constituent elements included in the common variable value. Thereafter, the weighting unit 140 weights the abnormality log using only the variable value indicating the component extracted by the component extraction unit 131.

  7A and 7B are schematic diagrams illustrating exemplary weighting result display screens using the display device 20. A screen C shown in FIG. 7A displays a common variable value indicating a component extracted by the component extraction unit 131 and a rank table C1 including the rank. “SV008”, which is the highest variable value in the ranking table C1, is highlighted by bold and underline. Thereby, the user can easily know a variable value having a high probability of causing an abnormality. In order to emphasize the upper variable value, the character type, color, size, etc. of the variable value may be changed.

  Furthermore, the screen C displays a configuration diagram C2 showing each component and the relationship between them based on the configuration information recorded in the configuration information storage unit 163. In the configuration diagram C2, symbols (here, circles) indicating the respective components and lines connecting the components are shown. A character string indicating the component is displayed in the vicinity of the symbol indicating the component. On the configuration diagram C2, the constituent element of the highest common variable value is emphasized by a triple circle C3, and the constituent elements of other common variable values are emphasized by a double circle C4. Thereby, the user can easily know a component having a high probability of causing an abnormality. In order to emphasize the constituent element of the upper variable value, the type, color, size, etc. of the symbol or character string indicating the constituent element may be changed. Or you may blink the component of a high-order variable value. In order to clarify the relationship between the ranking table C1 and the configuration diagram C2, the color of the character string of the variable value in the ranking table C1 may be the same as the color of the character string of the variable value in the configuration diagram C2. .

  In addition to the same information as the screen C, the screen D shown in FIG. 7B highlights the lines connecting the components that correspond to the highly likely relationship that is the cause of the abnormality. The ranking table D1 displays a relationship in which both ends are common variable values in addition to the common variable values. That is, the weighting unit 140 extracts and outputs the relationship between the constituent elements related to the common variable value. For example, since “SV002” and “SV005” are common variable values, the relationship between them is also displayed as having a high probability of causing an abnormality. In particular, since network devices often fail without outputting a clear log, such a display can indicate to the user that there is a problem with the network connection.

  Further, in the configuration diagram D2, a line indicating a relationship in which both ends are common variable values is highlighted by a broken line D3. Thereby, the user can easily know the relationship between components having a high probability of causing an abnormality. In order to emphasize the relationship between such components, the line type, color, thickness, and the like may be changed. Further, a character string such as “abnormal” or a symbol indicating abnormality may be attached in the vicinity of the line.

  Screens C and D in FIGS. 7A and 7B may display all the components included in the weighting result, or display only a predetermined number of components (for example, only the first-ranked component) according to the weighting result. Also good.

  As described above, the log analysis system 100-1 performs weighting using only the variable values related to the components for weighting, and outputs the weighting result. In general, the cause of abnormality is often a component. Therefore, the log analysis system 100-1 can make it easier for the user to find the cause of the abnormality by outputting only the variable values related to the constituent elements as the weighting result.

  FIG. 8 is a block diagram of a log analysis system 100-2 according to a second modification of the first embodiment. The log analysis system 100-2 includes a format learning unit 171 and a model learning unit 172 in addition to the configuration of FIG.

  When the format determination unit 120 determines the format, the format learning unit 171 creates a new format if the determination target log does not match any format recorded in the format storage unit 161. Records in the storage unit 161.

  As a first method for the format learning unit 171 to learn the format, the format learning unit 171 accumulates a plurality of logs whose formats are unknown, and a variable part that changes statistically and a constant that does not change. By separating the part, it can be defined as a new format. As a second method for the format learning unit 171 to learn the format, the format learning unit 171 reads a list of known variable values and matches or resembles a known variable value in a log whose format is unknown. A new format can be defined by determining the part to be changed as a variable part and determining the other part as a constant part. As a known variable value, the value itself may be used, or a pattern such as a regular expression may be used. The format learning method is not limited to these, and any learning algorithm capable of defining a new format for the input log may be used.

  When the log abnormality analysis unit 130 determines a model, the model learning unit 172 creates a new model if the determination target log does not match any model recorded in the model storage unit 162. Record in the model storage unit 162.

  Normally, the log abnormality analysis unit 130 determines that a log that does not match any model recorded in the model storage unit 162 in advance is an abnormality log. However, even if the log is unknown, it may be a normal log. is there. In this case, when the user inputs an instruction that the log that does not match the model in the model storage unit 162 is a normal log via the input device, the model learning unit 172 creates a new model based on the format and variable value of the log. Is recorded in the model storage unit 162. The model learning method is not limited to this, and an arbitrary learning algorithm that can newly define a model from an input log may be used.

  As described above, since the log analysis system 100-2 includes the format and model learning unit, a new format and model can be generated and recorded from a log including an unknown format and model.

(Second Embodiment)
FIG. 9 is a block diagram of the log analysis system 200 according to the present embodiment. In FIG. 9, arrows indicate main data flows, and there may be data flows other than those shown in FIG. In FIG. 9, each block shows a functional unit configuration, not a hardware (device) unit configuration. Therefore, the blocks shown in FIG. 9 may be implemented in a single device, or may be separately implemented in a plurality of devices. Data exchange between the blocks may be performed via any means such as a data bus, a network, a portable storage medium, or the like. The log analysis system 200 may be implemented with the same or other device configuration as in FIG.

  The log analysis system 200 includes a log input unit 210, a format determination unit 220, a log abnormality analysis unit 230, a weighting unit 240, and an output unit 250 as processing units. The log analysis system 200 includes a format storage unit 261, a model storage unit 262, and a distribution information storage unit 263 as storage units.

  The configuration of the log input unit 210, the format determination unit 220, the log abnormality analysis unit 230, the format storage unit 261, and the model storage unit 262 are respectively the log input unit 110, the format determination unit 120, and the log abnormality analysis unit of the first embodiment. The configuration is the same as that of the format storage unit 161 and the model storage unit 162.

  The weighting unit 240 is a log analysis unit, and weights variable values included in an abnormality log output from the log abnormality analysis unit 230 based on distribution information recorded in advance in the distribution information storage unit 263. The distribution information includes a range in which a variable value can normally take in a certain format, that is, a distribution width. The distribution width of the variable value may be recorded in the distribution information storage unit 263 by extracting the variable value from a plurality of normal logs and statistically determining a range that the variable value can normally take. Alternatively, the distribution width of the variable value may be recorded in the distribution information storage unit 263 when the user inputs a value that the variable value can normally take.

  10A and 10B are schematic diagrams of exemplary distribution information recorded in the distribution information storage unit 263. The distribution information includes a distribution of possible variable values associated with a unique ID and a distribution width thereof. The distribution information in FIG. 10A shows a case where the variable values that can be taken are character strings. In this case, the number of types of variable values that can be taken may be used as the distribution width. In FIG. 10A, for example, the distribution with ID 1 includes five types of character strings, and thus the distribution width is 5.

  The distribution information in FIG. 10B shows the case where the variable values that can be taken are numerical values. In this case, the difference between the maximum value and the minimum value of the variable values that can be taken, the average value, the variance value, the chi-square value, etc. Any number that can represent the distribution of numerical values may be used as the distribution width. In the distribution information of FIG. 10B, a variance value is used as the distribution width. In FIG. 10B, for example, a distribution whose ID is 1 includes numerical values of 10, 20, 30, and 100, and thus the distribution width is 1250.

  10A and 10B, the character string distribution information and the numerical value distribution information are recorded separately, but may be recorded together. In this case, it is desirable to normalize the distribution width of the character string and the distribution width of the numerical values so that they can be compared with each other.

  The ID of the distribution information is associated with the variable part in the format by, for example, an association table that associates the variable part in the format with the distribution information. For example, if “<variable: character string>” in the format whose ID is 039 in FIG. 2B is associated with the distribution width whose ID is 1 in FIG. 10A, the variable portion is SV001 to SV005. It can be seen that any value is taken and the distribution width is 5.

  The weighting unit 240 acquires the distribution width from the distribution information storage unit 263 for each variable value included in the abnormality log output from the log abnormality analysis unit 230. Then, the weighting unit 240 associates the variable value with the acquired distribution width, weights the distribution value so as to be higher as the distribution width is smaller, and outputs the weighted result. In other words, in the present embodiment, the abnormality log is weighted based on the distribution width that can be taken by the variable values that are part of the abnormality log. In general, the variable part with a large distribution width is often assigned by a random number from the system or a large number of lists. It is considered that the probability of being abnormal is low. On the other hand, the variable part with a small distribution width has a limited range of fluctuation in the normal state, so if the variable value deviates from the normal time, the severity of the abnormality is large and the cause of the abnormality or abnormal It seems to be related. Therefore, the log analysis system 200 can make it easier for the user to specify the cause of the abnormality by notifying the user of such a variable value.

  As another method, the weighting unit 240 weights each variable value included in the abnormality log output from the log abnormality analysis unit 230 based on the amount deviating from the distribution width acquired from the distribution information storage unit 263. Also good. In this case, when the variable value is out of the range of the distribution width, weighting is performed so that the higher the difference (absolute value) between the maximum or minimum value of the distribution width and the variable value, the higher the value. Output as a result. Since a variable value whose normal variable value deviates from the distribution width is considered to be more serious as an abnormality, the weighting unit 240 can notify the user of such a variable value.

  The output unit 250 outputs the weighting result by the weighting unit 240 in the same manner as the output unit 150 of the first embodiment. 11A to 11D are schematic diagrams illustrating exemplary weighting result display screens. Screen E shown in FIG. 11A displays the variable values rearranged according to the weighting result by weighting section 240 and their ranks.

  A screen F shown in FIG. 11B displays an arrow F2 indicating the position of the variable value in the abnormality log, together with a bar graph F1 indicating the distribution width. The screen G shown in FIG. 11C displays an arrow G2 representing the position of the variable value in the abnormality log, along with the frequency distribution graph G1. A screen H shown in FIG. 11D displays a list of frequency distribution ratios, in which variable values in the abnormality log are highlighted with bold and underline. The frequency itself may be displayed instead of the frequency distribution ratio. In order to emphasize the variable value, the character type, color, size, etc. of the variable value may be changed.

  However, in order to display the screen G and the screen H, not only the values that the variable portion can take, but also the frequency thereof must be recorded in the distribution information storage unit 263 in advance. According to the screen F, the screen G, and the screen H, the user can easily recognize how much the variable value in the abnormality log deviates from the distribution width. Further, on the screen H, even if the value is a discrete value such as a character string, the relationship between the variable value in the abnormality log and the distribution width can be easily recognized. Not limited to these, any display method that can indicate the positional relationship between the variable value and the distribution width in the abnormality log may be used.

  The flowchart of the log analysis method using the log analysis system 200 according to the present embodiment is basically the same as that in FIG. 5, and only the weighting process in step S107 is different. The weighting unit 240 acquires the distribution width from the distribution information storage unit 263 for each variable value included in the abnormality log determined in step S105. Then, the weighting unit 240 associates the variable value in the abnormality log with the acquired distribution width, weights the distribution value so as to be higher as the distribution width is smaller, and outputs the weighted result (step S107).

  As described above, the log analysis system 200 weights the variable values included in the abnormality log based on the distribution range of the variable values, thereby giving the user variable values that have a high probability of causing an abnormality. Can show. This makes it easy for the user to search for the cause of the abnormality.

  As a modified example according to the present embodiment, a format learning unit 171 and a model learning unit 172 may be provided as in the log analysis system 100-2 of FIG.

(Third embodiment)
FIG. 12 is a block diagram of a log analysis system 300 according to the present embodiment. In FIG. 12, arrows indicate main data flows, and there may be data flows other than those shown in FIG. In FIG. 12, each block represents a functional unit configuration, not a hardware (device) unit configuration. Therefore, the blocks shown in FIG. 12 may be implemented in a single device, or may be separately implemented in a plurality of devices. Data exchange between the blocks may be performed via any means such as a data bus, a network, a portable storage medium, or the like. The log analysis system 300 may be implemented with the same or other device configuration as in FIG.

  The log analysis system 300 includes a log input unit 310, a format determination unit 320, a log abnormality analysis unit 330, a weighting unit 340, and an output unit 350 as processing units. The log analysis system 300 includes a format storage unit 361, a model storage unit 362, and a combination storage unit 363 as storage units.

  The configuration of the log input unit 310, the format determination unit 320, the log abnormality analysis unit 330, the format storage unit 361, and the model storage unit 362 are respectively the log input unit 110, the format determination unit 120, and the log abnormality analysis unit of the first embodiment. The configuration is the same as that of the format storage unit 161 and the model storage unit 162.

  The weighting unit 340 is a log analysis unit, and weights variable values included in an abnormality log output from the log abnormality analysis unit 330 based on combination information recorded in advance in the combination storage unit 363. The combination information includes a combination of variable values included in one log. The combination information may be recorded in the combination storage unit 363 by extracting the value of the variable part from the normal log. Alternatively, the combination information may be recorded in the combination storage unit 363 by inputting a combination of variable values that the user desires to be normal.

  FIG. 13 is a schematic diagram of exemplary combination information recorded in the combination storage unit 363. The combination information includes a combination of variable values associated with the unique ID. The weighting unit 340 determines whether or not a combination of variable values included in one abnormality log output from the log abnormality analysis unit 330 matches any combination registered in the combination storage unit 363.

  When the combination of the variable values included in the abnormality log output from the log abnormality analysis unit 330 does not match any combination in the combination storage unit 363, the weighting unit 340 makes the abnormality log higher. Weights and outputs as a weighting result. On the other hand, when the combination of the variable values included in the abnormality log output from the log abnormality analysis unit 330 matches any combination in the combination storage unit 363, the weighting unit 340 is in the lower order. Thus, the weighting is performed and the result is output as a weighting result. That is, when the combination of variable values in the abnormality log is not registered in the combination storage unit 363, the weighting unit 340 performs weighting so that the combination is higher than the case where it is registered in the combination storage unit 363. . Here, when the combination of the variable values in the abnormality log is registered in the combination storage unit 363, the lower rank is set. However, the abnormality log may be regarded as a normal log and excluded from the weighting result by the weighting unit 340. .

  In the present embodiment, even if the log abnormality analysis unit 330 determines that the log is abnormal because it does not conform to the model, a log including a combination of registered variable values is less likely to cause an abnormality. Is weighted. On the other hand, the weighting unit 340 weights a log including a combination of unknown variable values as having a high probability of causing an abnormality. As described above, the log analysis system 300 can make it easier for the user to identify the cause of the abnormality by ranking based on whether or not the combination of the variable values in the abnormality log is a predetermined combination. it can.

  14A and 14B are schematic diagrams illustrating exemplary weighting result display screens using the display device 20. A screen J shown in FIG. 14A displays a combination of variable values that do not match any combination in the combination storage unit 363 and that includes the abnormality log. Here, a combination of variable values that matches any combination in the combination storage unit 363 is not displayed, but may be displayed.

  Further, on the screen J, S2 and Sj that are the same type of variable value (that is, the variable value of the same variable part) are highlighted by bold and underline. In order to emphasize the variable value, the character type, color, size, etc. of the variable value may be changed. As a result, the same type of variable value can be easily discriminated. Further, when the user selects any variable value on the screen J, the screen may be rearranged so that the combination including the variable value is higher. Thereby, the user can easily know a log including a combination of variable values having a high probability of causing an abnormality.

  The screen K shown in FIG. 14B displays a combination table K1 in which variable value combinations and abnormality logs including the combinations are described. Furthermore, the screen K displays a map K2 that expresses a combination of variable values in two dimensions. The map K2 has a first variable part (here, a server name) on the horizontal axis and a second variable part (here, a component type) on the vertical axis, and each of these combinations is represented by a square symbol. A square K3 representing a combination of variable values that does not match any combination in the combination storage unit 363 is emphasized by black painting. On the other hand, a square K4 representing a combination of variable values that matches any combination in the combination storage unit 363 is emphasized by hatching. In order to emphasize a predetermined combination, the shape, color, and size of a symbol indicating a combination of variable values may be changed, or an operation such as blinking may be performed. With such a display, the user can easily know which combination of variable values is abnormal, and can estimate a variable value that is the cause of or related to the abnormality.

  On the screen K, variable portions to be displayed on the vertical axis and the horizontal axis of the map K2 may be selectable by the user. In this case, a list box or a check box including variable parts that can be displayed on the screen K may be displayed, and selection by the user may be accepted. Further, by selecting a square in the map K2, the abnormality log including the variable value represented by the square may be highlighted in the combination table K1, or the abnormality log may be displayed on a pop-up screen or the like. Good.

  The screens shown in FIGS. 14A and 14B are examples, and any display method may be used as long as the variable value that is a log or a part of the log can be displayed so that the weighting result by the weighting unit 340 can be understood by the user.

  FIG. 15 is a schematic diagram showing another weighting result display screen according to the present embodiment. The screen K shown in FIG. 15 displays variable value ranks K5 and K6 in addition to the screen K of FIG. 14B. As the ranks K5 and K6, one of the variable value combinations (Si in FIG. 15) is used, but the ranks for both combinations of variable values may be used. The rank K5 is described by being attached to the corresponding variable value in the combination table K1. The rank K6 is described in the square K3 of the corresponding variable value in the map K2. The rank of each variable value is calculated by at least one of the weighting methods of the first and second embodiments described above. That is, weighting by a combination of two variable values according to the present embodiment and weighting by one variable value according to the first and second embodiments may be used in combination.

  The flowchart of the log analysis method using the log analysis system 300 according to the present embodiment is basically the same as that in FIG. 5, and only the weighting process in step S107 is different. The weighting unit 340 determines whether or not the combination of the variable values included in the abnormality log determined in step S105 matches any combination registered in the combination storage unit 363. The weighting unit 340 is in the lower order when the combination of the variable values in the abnormality log matches any combination registered in the combination storage unit 363, and the weighting unit 340 applies any combination registered in the combination storage unit 363. If they do not match, weighting is performed so as to be higher, and the result is output as a weighting result (step S107).

  As described above, the log analysis system 300 can indicate a variable value having a high probability of causing an abnormality to the user by performing weighting based on a combination of variable values included in the abnormality log. . This makes it easy for the user to search for the cause of the abnormality.

  Next, a modified example of the log analysis system 300 according to the third embodiment will be described. In the log analysis system 300-1 according to the first modification of the third embodiment, the weighting unit 340 performs weighting of the abnormality log using a combination of variable values defined as vectors.

  In the combination storage unit 363, combination vectors of variable values are recorded in advance. A combination vector is defined as a multi-dimensional vector by treating one type of variable part as one dimension. In each dimension (each type) of the combination vector, 0 is set if a certain type of variable part exists, and if not, a numerical value that can uniquely define a value from a set observed in the past in the variable part of that type. Shall be taken. For example, when a certain type of variable portion has fluctuated with values of A to E in the past, numerical values of 1 to 5 are used corresponding to the respective values. Further, the distribution width of each variable value is calculated and recorded as the weight of each variable portion. The definition of the distribution width is the same as in the second embodiment.

  The weighting unit 340 generates the above-described combination vector from the variable values included in the abnormality log output from the log abnormality analysis unit 330. Then, the weighting unit 340 calculates the distance between the combination vector created from the abnormality log and each combination vector of variable values recorded in advance in the combination storage unit 363. Then, when the distance is equal to or greater than a predetermined threshold, the weighting unit 340 performs weighting assuming that the combination of variable values included in the abnormality log is higher. On the other hand, when the distance is less than a predetermined threshold, the weighting unit 340 performs weighting assuming that the combination of variable values included in the abnormality log is lower.

  As described above, the log analysis system 300-1 defines a combination of variable values as a vector and performs weighting using the distance between the vectors. Therefore, calculation and learning are easier than simply determining whether or not a combination of variable values is recorded in the combination storage unit 363. Further, by grouping vectorized information, only representative vectors (vectors indicating group characteristics, for example, average vectors) can be compared for each group, so that processing performance is improved.

  FIG. 16 is a block diagram of a log analysis system 300-2 according to a second modification of the third embodiment. The log analysis system 300-2 includes a format learning unit 371, a model learning unit 372, and a combination learning unit 373 in addition to the configuration of FIG. The configurations of the format learning unit 371 and the model learning unit 372 are the same as the configurations of the format learning unit 171 and the model learning unit 172 in FIG.

  The combination learning unit 373 accumulates the combinations of the variable values in the input log, and calculates the appearance frequency for each combination of the variable values. Then, the combination learning unit 373 registers the combination of variable values in the combination storage unit 363 based on the combination of variable values and the appearance frequency thereof. For example, the combination learning unit 373 may register the combination of variable values in the combination storage unit 363 when the appearance frequency of the combination of variable values is equal to or greater than a predetermined threshold. The appearance frequency may be a single value such as the sum of combinations included in the input log, or may be a plurality of values such as a time distribution in which combinations appear in the input log. When the time distribution is used as the appearance frequency, it is possible to determine whether or not the appearance time of the combination of the variable values in the abnormality log matches the past time distribution. The combination learning unit 373 may learn and record the combination vector of the log analysis system 300-1. In addition, any learning algorithm capable of learning a combination of variable values from an input log may be used for learning the combination.

  As described above, since the log analysis system 300-2 includes a learning unit for a combination of a format, a model, and a variable value, a new format, model, or variable value is created from a log that includes a combination of an unknown format, model, or variable value. Can be generated and recorded.

(Other embodiments)
FIG. 17 is a schematic configuration diagram of the log analysis systems 100, 200, and 300 according to the above-described embodiments. FIG. 17 illustrates a configuration example for the log analysis systems 100, 200, and 300 to function as an apparatus that performs log analysis based on variable values in the log. The log analysis systems 100, 200, and 300 include a format determination unit 120, 220, and 320 as a variable extraction unit that extracts a value of a variable part based on a predetermined format from an analysis target log, And weighting units 140, 240, and 340 as log analysis units that weight the analysis target log.

  The present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention.

  A program that operates the configuration of the embodiment so as to realize the functions of the above-described embodiment (more specifically, a program that causes a computer to execute the processing illustrated in FIG. 5) is recorded on a recording medium, and the recording medium A processing method of reading a recorded program as a code and executing it on a computer is also included in the category of each embodiment. That is, a computer-readable recording medium is also included in the scope of each embodiment. In addition to the recording medium on which the above program is recorded, the program itself is included in each embodiment.

  As the recording medium, for example, a floppy (registered trademark) disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, and a ROM can be used. Further, the embodiment is not limited to the processing executed by a single program recorded in the recording medium, and the embodiments that execute processing by operating on the OS in cooperation with other software and the function of the expansion board are also described in each embodiment. Included in the category.

  A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

(Appendix 1)
A variable extraction unit that extracts the value of the variable part from the analysis target log based on a predetermined format based on the characteristics of the log;
A log analysis unit that weights the analysis target log based on the value of the variable part;
A log analysis system comprising:

(Appendix 2)
The log analysis system according to supplementary note 1, wherein the log analysis unit weights values of the variable part included in common among a plurality of logs among the analysis target logs.

(Appendix 3)
A configuration information storage unit that records configuration information including information indicating the components of the system that outputs the analysis target log;
The log analysis system according to appendix 1 or 2, wherein the log analysis unit extracts the configuration information included in a value of the variable portion based on the configuration information read from the configuration information storage unit.

(Appendix 4)
The configuration information further includes information indicating a relationship between the components,
The log analysis system according to appendix 3, wherein the log analysis unit extracts the relationship included in the value of the variable portion based on the relationship read from the configuration information storage unit.

(Appendix 5)
The log analysis system according to supplementary note 1, wherein the log analysis unit weights the value of the variable part based on a distribution width of the variable part.

(Appendix 6)
The log analysis system according to appendix 5, wherein the distribution width is a numerical value indicating a range of values that the variable can take.

(Appendix 7)
A distribution information storage unit for recording the distribution width;
The log analysis system according to appendix 5 or 6, wherein the log analysis unit weights the value of the variable portion based on the distribution width read from the distribution information storage unit.

(Appendix 8)
The log analysis system according to appendix 1, wherein the log analysis unit weights the analysis target log based on a combination of values of a plurality of the variable portions included in one log of the analysis target logs.

(Appendix 9)
A combination storage unit for recording the combination;
The log analysis unit weights the analysis target log based on whether or not the combination of the values of the variable parts extracted by the variable extraction unit is recorded in the combination storage unit. The described log analysis system.

(Appendix 10)
A combination storage unit for recording a vector indicating the combination;
The log analysis unit is configured to perform the analysis based on a distance between the vector indicating the combination of the values of the variable parts extracted by the variable extraction unit and the vector recorded in the combination storage unit. The log analysis system according to appendix 8, which performs weighting of the target log.

(Appendix 11)
The log analysis system according to appendix 9 or 10, further comprising a combination learning unit that registers the combination of values of the variable part extracted by the variable extraction unit in the combination storage unit.

(Appendix 12)
The log analysis system according to any one of appendices 1 to 11, further comprising a format learning unit that generates the format based on the analysis target log.

(Appendix 13)
Extracting the value of the variable portion from the analysis target log based on a predetermined format based on the characteristics of the log; and
Weighting the analysis target log based on the value of the variable portion;
A log analysis method comprising:

(Appendix 14)
On the computer,
Extracting the value of the variable portion from the analysis target log based on a predetermined format based on the characteristics of the log; and
Weighting the analysis target log based on the value of the variable portion;
Log analysis program to execute

This application claims the priority on the basis of Japanese application Japanese Patent Application No. 2015-224500 for which it applied on November 17, 2015, and takes in those the indications of all here.

Claims (14)

A variable extraction unit that extracts the value of the variable part from the analysis target log based on a predetermined format based on the characteristics of the log;
A log analysis unit that weights the analysis target log based on the value of the variable part;
A log analysis system comprising:   The log analysis system according to claim 1, wherein the log analysis unit weights the value of the variable part included in common among a plurality of logs among the analysis target logs. A configuration information storage unit that records configuration information including information indicating the components of the system that outputs the analysis target log;
The log analysis system according to claim 1, wherein the log analysis unit extracts the configuration information included in a value of the variable portion based on the configuration information read from the configuration information storage unit. The configuration information further includes information indicating a relationship between the components,
The log analysis system according to claim 3, wherein the log analysis unit extracts the relationship included in the value of the variable portion based on the relationship read from the configuration information storage unit.   The log analysis system according to claim 1, wherein the log analysis unit weights the value of the variable part based on a distribution width of the variable part.   The log analysis system according to claim 5, wherein the distribution width is a numerical value indicating a range of values that the variable portion can take. A distribution information storage unit for recording the distribution width;
The log analysis system according to claim 5 or 6, wherein the log analysis unit weights the value of the variable portion based on the distribution width read from the distribution information storage unit.   The log analysis system according to claim 1, wherein the log analysis unit weights the analysis target log based on a combination of values of a plurality of the variable portions included in one of the analysis target logs. A combination storage unit for recording the combination;
The log analysis unit weights the analysis target log based on whether or not the combination of values of the variable part extracted by the variable extraction unit is recorded in the combination storage unit. Log analysis system described in. A combination storage unit for recording a vector indicating the combination;
The log analysis unit is configured to perform the analysis based on a distance between the vector indicating the combination of the values of the variable parts extracted by the variable extraction unit and the vector recorded in the combination storage unit. The log analysis system according to claim 8, wherein weighting of the target log is performed.   The log analysis system according to claim 9 or 10, further comprising a combination learning unit that registers the combination of the values of the variable parts extracted by the variable extraction unit in the combination storage unit.   The log analysis system according to claim 1, further comprising a format learning unit that generates the format based on the analysis target log. Extracting the value of the variable portion from the analysis target log based on a predetermined format based on the characteristics of the log; and
Weighting the analysis target log based on the value of the variable portion;
A log analysis method comprising: On the computer,
Extracting the value of the variable portion from the analysis target log based on a predetermined format based on the characteristics of the log; and
Weighting the analysis target log based on the value of the variable portion;
Log analysis program to execute

Images