JP2006259811A - Log creating device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce labor of log analyzing work. <P>SOLUTION: A log collecting means 21 collects log files from a plurality of log collection object servers at once via a network device 1. An intermediate log conversion means 22 converts the plurality of log files collected by the log collecting means 21 into an intermediate format which is a common format, and stores them in a collected log storing part 31. In an integrated log converting means 23, a conversion rule is fetched from a conversion table of a conversion table storing part 32, a pattern satisfying a conditional expression is retrieved from the plurality of log files converted into the intermediate format, and if there is a pattern satisfying conditions, the pattern in a result part of the conversion rule is outputted and stored in an integrated log storing part 33. All conversion rules existing in the conversion table are applied in the same manner, and by creating an integrative log of the whole system, a summary of log information is created. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a log creation device for creating a computer log and a program suitable for construction of the log creation device.

Management and analysis of system log information is very important from the viewpoint of improving the security and reliability of a computer system, and a system for the purpose of acquiring log information has been developed (for example, Patent Document 1). To 4).
However, the log information generally has a large amount of information and requires a lot of time for analysis work. In particular, when the log information of multiple computers must be tracked, the work becomes even more complicated, requiring a lot of experience and knowledge as a system administrator, and log analysis is complicated. There is a problem that it becomes difficult.

For example, Patent Document 1 discloses means for analyzing and displaying the relevance of log information. However, since only the relevance of a plurality of logs is displayed based on the shared counter value, their meanings are as follows. There is a problem that the system administrator has to judge.
In addition, by combining Patent Documents 2 to 4, it is possible to construct an invention for displaying a log of sessions (process flows) by a plurality of servers as a unified log, for example, as an integrated log By displaying the relevance and hierarchical structure of log patterns in a plurality of logs, it is possible to clarify the processing flow of the entire system.

However, what the displayed integrated log means must be determined from the experience of the system administrator himself, and there is a problem that it is not easy to use by a system administrator with little experience.
In addition, when displaying the integrated log, it is necessary to specify the server name, log file name, log output date and time, and select the log entry that is the starting point for creating the integrated log. There is.

JP 2001-184235 A JP 2002-244898 A JP 2002-318734 A JP 2004-227360 A

The present invention has been made in view of the above-described problems, and an object thereof is to reduce the labor of log analysis work.
Another object of the present invention is to enable even an inexperienced system administrator to easily perform log analysis.

According to the present invention, log collection means for collecting log files from a plurality of log collection target computers, and an intermediate log for converting a plurality of log files collected by the log collection means into a log file of an intermediate format that is a common format There is provided a log creation device comprising: conversion means; and integrated log conversion means for creating an integrated log by applying a predetermined conversion rule to the plurality of intermediate format log files. The
The log collection means collects log files from a plurality of log collection target computers. The intermediate log conversion unit converts the plurality of log files collected by the log collection unit into an intermediate format log file which is a common format. The integrated log conversion means creates an integrated log by applying a predetermined conversion rule to the plurality of intermediate format log files.

Here, it has conversion table storage means for storing the conversion table storing the conversion rule, and the integrated log conversion means is stored in the conversion table storage means for the plurality of intermediate format log files. You may comprise so that an integrated log may be obtained by applying a predetermined conversion rule and converting.
The intermediate format log file may include a computer name of a log collection target, an OS type of the computer, a time when a log entry is recorded on each log collection target server, and a message.

  And a collection log storage unit that stores the log file in the intermediate format; and an integrated log storage unit that stores the integrated log. The intermediate log conversion unit includes a plurality of logs received from the log collection unit. The file is converted into the intermediate format log file and recorded in the collected log storage means, and the integrated log conversion means stores the conversion table storage for a plurality of log files stored in the collected log storage means. The result obtained by applying the conversion rule stored in the means for conversion may be recorded in the integrated log storage means as an integrated log.

In addition, log information management means for processing a display / deletion request for the collected log and the integrated log received from the user interface may be provided.
Moreover, you may comprise so that the conversion rule management means which processes the request | requirement of the display of the conversion rule received from the user interface, addition, correction, and deletion may be provided.

  Further, according to the present invention, the computer collects log files from a plurality of log collection target computers, and the log file collected by the log collection unit is an intermediate format log file. Provided is a program characterized by functioning as an integrated log converting means for generating an integrated log by applying a predetermined conversion rule to the intermediate log converting means for converting to a plurality of intermediate format log files Is done.

  By executing a program, the computer collects log files from multiple log collection target computers, and converts the multiple log files collected by the log collection unit into a common format log file The intermediate log conversion unit that functions as an integrated log conversion unit that creates an integrated log by applying predetermined conversion rules to the plurality of intermediate format log files.

Here, the integrated log conversion means functions to obtain an integrated log by converting the plurality of intermediate format log files by applying a predetermined conversion rule stored in the conversion table storage means. You may comprise the said program so that it may make.
In addition, the intermediate log file includes the name of the computer that is the log collection target, the OS type of the computer, the time when the log entry is recorded on each log collection target server, and the message. May be.

  The intermediate log conversion unit converts the plurality of log files passed from the log collection unit into the log file of the intermediate format and records the log file in the collection log storage unit. The integrated log conversion unit includes the collection log A result obtained by applying the conversion rule stored in the conversion table storage unit and converting the plurality of log files stored in the storage unit is recorded as an integrated log in the integrated log storage unit. The program may be configured to function as described above.

Furthermore, it may be configured to function as a log information management unit that processes a display / deletion request for the collected log and the integrated log received from the user interface.
Further, it may be configured to function as a conversion rule management means for processing a request for display, addition, correction, and deletion of a conversion rule received from the user interface.

According to the log creating device of the present invention, it is possible to reduce the labor of log analysis work. Further, even an inexperienced system administrator can easily perform log analysis.
Further, according to the present invention, it is possible to provide a program suitable for constructing the log creation device.

  FIG. 1 is a block diagram showing a configuration of a log creation device according to an embodiment of the present invention. The log creation device includes a network device 1 for connecting the data processing device 2 to a network, a data processing device 2 that is a log creation device that operates by program control, and a storage device 3 as a storage unit that stores various types of information. It has. The data processing device 2 is configured by a computer having a central processing unit (CPU), a storage device, and the like, and is configured so that the CPU performs a process described later by executing a program stored in the storage device. Is possible.

The storage device 3 includes a collection log storage unit 31 as a collection log storage unit, a conversion table storage unit 32 as a conversion table storage unit, and an integrated log storage unit 33 as an integrated log storage unit.
The collected log storage unit 31 stores a log file group obtained by converting a plurality of log collection target computers (servers) (not shown) on the network system into a predetermined common format.

The conversion table storage unit 32 stores a conversion table storing conversion rules necessary for generating an integrated log from a plurality of log files. Here, the integrated log refers to a log obtained by collecting a plurality of logs, for example, a log obtained by summarizing logs of a plurality of computers connected to a network into one.
The integrated log storage unit 33 stores the integrated log created by the integrated log conversion unit 23.
The data processing device 2 includes a log collection unit 21, an intermediate log conversion unit 22, an integrated log conversion unit 23, a log information management unit 24, a conversion rule management unit 25, and a user interface 26.

The log collection unit 21 collects a specified log file group from the log collection target computer (server) on the network system and passes it to the intermediate log conversion unit 22.
The intermediate log conversion unit 22 converts the log file group passed from the log collection unit 21 into an intermediate format intermediate log that is a common format, and records it in the collection log storage unit 31.
The integrated log conversion unit 23 applies the conversion rule stored in the conversion table storage unit 32 to the intermediate format log information stored in the collected log storage unit 31, and integrates the output obtained as a result. The log is recorded in the integrated log storage unit 33.

The log information management unit 24 processes a display / deletion request for the collected log and the integrated log received from the user interface 26.
The conversion rule management means 25 processes the conversion rule display, addition, correction, and deletion requests received from the user interface 26.
The user interface 26 includes input means such as a keyboard and a mouse and output means such as a display unit. Upon receiving a log information collection request or a log information / conversion rule management request from the user, each of the means 21 to 21 is provided. The request is distributed to the corresponding means in 25, and the results obtained by the respective means 21 to 25 are displayed to the user.

2 to 6 are flowcharts showing processing of the log creation device according to the embodiment of the present invention, and are flowcharts showing processing of the data processing device 2. Note that these processes are performed when a computer constituting the data processing apparatus 2 executes a program.
FIG. 7 is a table showing the format of a common format intermediate log file used in the log creating apparatus according to the embodiment of the present invention. As shown in FIG. 7, the format of the intermediate log file is that the log collection target computer name (in this case, the server name), the OS (Operating System) type of the server, and log entries are recorded on each log collection target server. Time and message.

  FIG. 8 is a diagram showing the format of the conversion table used in the log creating apparatus according to the embodiment of the present invention. In FIG. 8, the conversion table is configured as a table having conversion rules as rows. A conversion rule is composed of a conditional expression and a result part. The conditional expression of the conversion rule is composed of a plurality of conditions and a time interval between these conditions. The condition includes a computer name (here, a server name), a log file name, and a log pattern.

  The server name is the host name of the server that is the target of log collection, and multiple server names can be specified. The log file name is a file name in which log information is recorded. A log pattern is a character string of log information that serves as a search condition, and enables partial search using a wild card and passing of a pattern to a result part using a variable. The time interval between conditions is the time interval of the time when the log pattern of each condition appears, and only the log pattern appearing within this time satisfies the conditional expression. The result part is an output character string after the conversion rule is applied, and enables embedding of a variable storing a server name, a server OS type, a log pattern appearance time, and a part of a condition log pattern.

Hereinafter, with reference to FIGS. 1 to 8, a log creation device and a program according to an embodiment of the present invention will be described in detail.
First, the log collection unit 21 and the intermediate log conversion unit 22 will be described. When the log collection unit 21 receives a log collection request from the user interface 26 (step A1 in FIG. 2), the log collection unit 21 transfers and receives log files from the log collection target server, which is a plurality of log collection target computers, via the network device 1. (Step A2), and the received log file is transferred to the intermediate log conversion means 22.

The intermediate log conversion means 22 converts the received plurality of log files into an intermediate format that is a common format (step A3), and then records the log as a group of intermediate log files in the collected log storage unit 31 (step A4). Collect files collectively.
Next, the integrated log conversion unit 23 will be described. When the integrated log conversion unit 23 receives the integrated log conversion request from the user interface 26 (step B1 in FIG. 3), the pointer indicating the conversion rule in the conversion table storage unit 32 is set in the first row of the conversion table (step B2). ), It is determined whether or not a predetermined conversion rule predetermined in the conversion table exists (step B21), and if it is determined that a predetermined conversion rule predetermined in the conversion table exists, the pointer indicates The conversion rule of the conversion table is read (step B3).

Next, the integrated log converting means 23 converts the output character string generated by applying the read conversion rule to the collected logs collected in the collected log storage unit 31 as an integrated log. (Step B4), and the pointer of the conversion table is set in the conversion rule of the next line (step B5).
Here, when the next conversion rule exists in the conversion table (step B21), the integrated log conversion unit 23 repeats steps B3 to B5.

  The integrated log conversion unit 23 outputs an integrated log as a conversion result when there is no conversion rule. At this time, the integrated log conversion means 23 determines whether or not an integrated log exists (step B22), and if the integrated log is not empty (in other words, if the integrated log exists, that is, for application of conversion rules). If successful, the recorded integrated log is sorted by the appearance time of the first log pattern as a condition and written to a file (step B6). On the other hand, when the integrated log is empty (when there is no log to which the conversion rule can be applied), the integrated log conversion unit 23 prompts the user to input the conversion rule from the user interface 26 (step B7). If there is an input, the above conversion process is executed again.

Next, the format of the conversion table and the conversion rule application process in process step B4 of FIG. 3 will be described in more detail.
With respect to the conversion rule application unit in process step B4 of FIG. 3, first, the integrated log conversion unit 23 acquires the first condition from the read conditional expression of the conversion rule (step C1 of FIG. 4).
Next, the integrated log conversion means 23 reads the log specified by the server name and log file name under the acquired condition from the collected log storage unit 31 (step C2), and searches for the log pattern specified in the condition. Then, it is determined whether or not a log pattern as a condition exists (steps C3 and C31). When a log pattern as a condition exists, the time (T11,..., T1N) at which the log pattern appears is stored (step C4).

Next, the integrated log conversion means 23 determines whether or not the next condition exists (step C41). If the next condition exists, the time interval between the condition and the previous condition ( TS1) is acquired (steps C5 and C6).
Next, the integrated log conversion means 23 reads the log specified by the server name and the log file name under the acquired condition from the collected log storage unit 31 (step C7), and the log pattern specified in the condition as the time Search is made from log entries between (T11 to T11 + TS1),..., (T1N to T1N + TS1) (step C8).

  Thereafter, the integrated log converting means 23 repeats steps C31 to C8 for all the conditions in the conditional expression, and if there is a log pattern specified by all the conditions, it is shown in the result part of the conversion rule. Is recorded as an integrated log together with the time, and the process returns to step B5 in FIG. If no log pattern is found in step C31, the process immediately returns to step B5.

Next, the log information management unit 24 will be described. The log information management unit 24 receives a log information management request from the user interface 26 (step D1), and performs processing that satisfies the request.
When receiving the collection log display request from the user interface 26 (step D11), the log information management unit 24 acquires the requested collection log file from the collection log storage unit 31 and displays it through the user interface 26 (step D11). D2).

When the log information management unit 24 receives a collection log deletion request instead of a collection log display request from the user interface 26 (step D12), the log information management unit 24 retrieves the collection log file designated by the user interface 26 from the collection log storage unit 31. It deletes and displays the result of the deletion process on the interface 26 (step D3).
When the log information management unit 24 receives a request for displaying an integrated log instead of a request for displaying a collected log or a request for deleting a collected log from the user interface 26 (step D13), the log information managing unit 24 first checks the integrated log storage unit 33 to check the integrated log. Is confirmed (step D14).

When there is an integrated log, the log information management unit 24 acquires the integrated log and displays it through the user interface 26 (step D4). When the integrated log does not exist, the log information management unit 24 sends a conversion request to the integrated log conversion unit 23 (step D5), and displays the conversion result of the integrated log conversion unit 23 on the interface 26 (step D6).
When receiving the request for deleting the integrated log (step D15), the log information management unit 24 deletes the integrated log from the integrated log storage unit 33 and displays the result on the interface 26 (step D7).

Next, the conversion rule management means 25 will be described. The conversion rule management means 25 receives a log information management request from the user interface 26 (step E1), and performs processing that satisfies the request.
When receiving a conversion rule display request from the user interface 26 (step E11), the conversion rule management unit 25 acquires a list of conversion rules from the conversion table storage unit 32 and displays it through the user interface 26 (step E2). .

When the conversion rule management unit 25 receives a conversion rule addition request instead of a conversion rule display request from the user interface 26 (step E12), the conversion rule management unit 25 adds the transmitted conversion rule to the conversion table storage unit 32 and performs processing. The result is displayed on the user interface 26 (step E3).
When the conversion rule management means 25 receives not a conversion rule display request or a conversion rule addition request from the user interface 26 but a conversion rule correction request (step E13), the conversion rule management means 25 has received the specified conversion rule. The processing result is displayed on the user interface 26 (step E4).

When receiving the conversion rule deletion request (step E14), the conversion rule management unit 25 deletes the specified conversion rule from the conversion table storage unit 32 and displays the processing result on the user interface 26 (step E5). .
Finally, the user interface 26 will be described. The user interface 26 receives a log collection request, an integrated log conversion request, a log information management request, and a conversion rule management request from the user, distributes each request to the corresponding processing means 21, 23, 24, and 25, and processes each means. Display the results.

  As described above, according to the log creation device according to the embodiment of the present invention, the log collection unit 21 that collects log files from a plurality of log collection target computers, and the plurality of log files that the log collection unit 21 collects. The intermediate log conversion means 22 for converting the log into an intermediate format log file, which is a common format, and integration of a plurality of log files by applying a predetermined conversion rule to the plurality of intermediate format logs It is characterized by comprising integrated log conversion means 23 for creating a log.

Here, the integrated log conversion unit 23 applies the predetermined conversion rule stored in the conversion table stored in the conversion table storage unit 32 to the plurality of intermediate format log files, thereby converting the log file. It is configured to obtain unified logs.
The intermediate log conversion unit 22 converts each of the plurality of log files delivered from the log collection unit 21 into the log file of the intermediate format and records it in the collection log storage unit 31, and the integrated log conversion unit 23 collects the log file. By applying the conversion rule stored in the conversion table storage unit 32 to a plurality of intermediate format log files stored in the log storage unit 31, the obtained integrated log is stored in the integrated log storage unit 33. It is configured to record.

  Further, the integrated log conversion means 23 retrieves the conversion rule from the conversion table, searches for a pattern satisfying the conditional expression from a plurality of log files converted into the intermediate format, and if there is a pattern satisfying the condition, conversion is performed. Configure the log information to be summarized by outputting the pattern in the result part of the rule, applying all the conversion rules that exist in the conversion table in the same way, and creating an integrated log for the entire system. Yes.

Therefore, since an integrated log (summary summarizing a plurality of logs) of the entire system can be created from a plurality of logs scattered on the network system, it is possible to reduce time and effort for system log analysis. Further, even an inexperienced system administrator can easily perform log analysis.
Also, by storing the analysis results of multiple log patterns as a conversion table, not only the display of relevance but also the automatic summarization of multiple logs (what the log patterns latent in multiple log files mean Can be displayed).

In addition, it is possible to convert and create a combination of log patterns (for example, a specific attack pattern) that is not related to one session (processing flow) by a plurality of servers.
In addition, by sharing the conversion table, it is possible to reduce the log analysis work of an inexperienced administrator.
Further, since the conversion locations can be listed by summarizing the entire log, there is an effect that it is possible to easily find the log pattern at the time of abnormality.
Further, according to the embodiment of the present invention, there is provided a program for constructing the log creating device by being executed by a computer.

  The present invention can be applied to a log creation device that performs log analysis of a plurality of computers in a system that uses a plurality of computers, such as a computer system in which a plurality of computers are connected to a network. Further, it can be applied to a log analysis support function in a system management support program.

It is a block diagram of a log creation device concerning an embodiment of the invention. It is a flowchart which shows the process of the log production apparatus which concerns on embodiment of this invention. It is a flowchart which shows the process of the log production apparatus which concerns on embodiment of this invention. It is a flowchart which shows the process of the log production apparatus which concerns on embodiment of this invention. It is a flowchart which shows the process of the log production apparatus which concerns on embodiment of this invention. It is a flowchart which shows the process of the log production apparatus which concerns on embodiment of this invention. It is a table which shows the format of the intermediate log file used with the log information summarization apparatus which concerns on embodiment of this invention. It is a figure which shows the format of the conversion table used with the log information summarization apparatus which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Network device 2 ... Data processing device 3 as log information processing means ... Storage device 21 as storage means ... Log collection means 22 ... Intermediate log conversion means 23 ... Integrated log Conversion means 24 ... log information management means 25 ... conversion rule management means 26 ... user interface 31 ... collection log storage section 32 ... conversion table storage section 33 ... integrated log storage section

Claims (12)

  Log collection means for collecting log files from a plurality of log collection target computers; intermediate log conversion means for converting a plurality of log files collected by the log collection means into a log file of an intermediate format that is a common format; An integrated log conversion unit that generates an integrated log by applying a predetermined conversion rule to the intermediate format log file. Conversion table storage means for storing a conversion table storing the conversion rule;
The integrated log conversion unit obtains an integrated log by applying a predetermined conversion rule stored in the conversion table storage unit to the plurality of intermediate format log files for conversion. The log creation device according to claim 1.   3. The intermediate format log file includes a computer name of a log collection target, an OS type of the computer, a time when a log entry is recorded on a log collection target server, and a message. Log creation device. A collection log storage means for storing the intermediate format log file; and an integrated log storage means for storing the integrated log;
The intermediate log conversion unit converts the plurality of log files passed from the log collection unit into the log file of the intermediate format and records the log file in the collection log storage unit,
The integrated log conversion unit applies a result obtained by converting a plurality of log files stored in the collected log storage unit by applying a conversion rule stored in the conversion table storage unit. 4. The log creation device according to claim 2, wherein the log creation device records the unified log in the unified log storage unit.   5. The log creation device according to claim 1, further comprising log information management means for processing a request for displaying and deleting the collected log and the integrated log received from the user interface.   6. The log creating apparatus according to claim 1, further comprising conversion rule management means for processing conversion rule display, addition, correction, and deletion requests received from the user interface.   Log collection means for collecting log files from a plurality of log collection target computers, and intermediate log conversion means for converting the plurality of log files collected by the log collection means into an intermediate format log file that is a common format; A program which functions as an integrated log conversion means for creating an integrated log by applying a predetermined conversion rule to the plurality of intermediate format log files.   The integrated log conversion unit functions to obtain an integrated log by applying a predetermined conversion rule stored in the conversion table storage unit to the plurality of intermediate format log files for conversion. The program according to claim 7, wherein   8. The intermediate format log file includes a computer name of a log collection target, an OS type of the computer, a time when a log entry is recorded on a log collection target server, and a message. 8. The program according to 8. The intermediate log conversion unit converts the plurality of log files passed from the log collection unit into the log file of the intermediate format and records the log file in the collection log storage unit,
The integrated log conversion unit applies a result obtained by converting a plurality of log files stored in the collected log storage unit by applying a conversion rule stored in the conversion table storage unit. The program according to claim 8 or 9, wherein the program is made to function so as to be recorded in an integrated log storage unit as an integrated log.   The program according to any one of claims 7 to 10, further functioning as log information management means for processing requests for displaying and deleting collected logs and integrated logs received from a user interface.   The program according to any one of claims 7 to 11, further functioning as a conversion rule management means for processing a request for display, addition, correction, and deletion of a conversion rule received from a user interface.

Images