JP5576834B2 - Log file collection system, server, log file collection method, log output method and program - Google Patents

Description

  The present invention relates to a log file collection system having a log file collection server for collecting log files, a server, a log file collection method, a log output method, and a program.

  As a method of collecting log files from a plurality of servers, there is a standard log output method (protocol) in UNIX (registered trademark) (for example, see Non-Patent Document 1). The purpose of collecting log files is to identify the cause of a problem and to manage performance when a problem occurs.

  In the log output method disclosed in Non-Patent Document 1, a log collection that collects a log file including a log output from an OS (Operating System) or an application executed by a log file collection target server is performed. Send to server via network. The log is written in a text format. The log file collection server that has received the log file stores the received log file in a standard file system.

  The log file collection server archives an old log file when the capacity of the stored log file increases. When the archived log file exceeds a certain amount, the log file collection server deletes it.

  Further, a technique for collecting log files in a large-scale environment is disclosed in Non-Patent Document 2, for example.

  In the technique disclosed in Non-Patent Document 2, a log file including a log described in a text format by a distributed file system is stored. Therefore, even when the number of servers from which log files are collected increases, it is possible to collect log files at a single server. Even when the capacity of the stored log file increases, the capacity of the log file that can be stored can be increased by adding a new server to the distributed file system.

C.Lonvick. RFC3164: The BSD syslog Protocol, August 2001.http: //www.ietf.org/rfc/rfc3164.txt Ariel Rabkin, Randy Katz. "Chukwa: A system for reliable large-scale log collection", 24th Large Installation System Administration Conference (LISA'10), San Jose, CA, USA, Nov 7-12, 2010.

  In the technologies disclosed in Non-Patent Document 1 and Non-Patent Document 2 described above, when the number of log file collection target servers increases, the log file collection server and the log file collection server There is a problem that the load on the network increases.

  Further, in the technologies disclosed in Non-Patent Document 1 and Non-Patent Document 2 described above, when the number of servers from which log files are collected increases, archiving, adding servers, and the like must be performed at a high frequency. In this case, there is a problem that a new log file cannot be stored.

  The present invention reduces the load on the network used when sending and receiving log files even when the number of servers from which log files are collected increases, and makes it impossible to store new log files. It is an object to provide a log file collection system, a server, a log file collection method, a log output method, and a program that can avoid the above.

In order to achieve the above object, a log file collection system according to the present invention includes a plurality of servers that execute an application having a log output function, and a log file collection server that collects log files including logs. A collection system,
Each of the plurality of servers is
Among the character strings described in the source code of the application, the predetermined character string used in the log output from the application is information associated with the predetermined character string, and the predetermined character string A first control unit for converting into character string information having a smaller capacity than
A first communication unit that transmits a log file including a log output from the application to the log file collection server,
The log file collection server
A second communication unit for receiving a log file transmitted from each of the plurality of servers;
A log file storage unit for storing the log file received by the second communication unit.

In order to achieve the above object, the server of the present invention is a server that executes an application having a function of outputting a log file,
Among the character strings described in the source code of the application, the predetermined character string used in the log output from the application is information associated with the predetermined character string, and the predetermined character string A control unit for converting to character string information having a smaller capacity than
And a communication unit that transmits a log file including a log output from the application to a log file collection server that collects the log file.

In order to achieve the above object, a log file collection method of the present invention includes a plurality of servers that execute an application having a function of outputting logs, and a log file collection server that collects log files including logs. A log file collection method in a log file collection system,
Information in which each of the plurality of servers is associated with a predetermined character string used in a log output from the application among character strings described in the source code of the application. A process of converting into character string information having a smaller capacity than the predetermined character string;
Each of the plurality of servers transmits a log file including a log output from the application to the log file collection server;
A process in which the log file collection server receives a log file transmitted from each of the plurality of servers;
The log file collection server storing the received log file.

In order to achieve the above object, a log output method of the present invention is a log output method in a server that executes an application having a function of outputting a log file,
Among the character strings described in the source code of the application, the predetermined character string used in the log output from the application is information associated with the predetermined character string, and the predetermined character string Processing to convert to character string information with a smaller capacity,
And a process of transmitting a log file including a log output from the application to a log file collection server that collects the log file.

In order to achieve the above object, the program of the present invention provides a server for executing an application having a function of outputting a log file.
Among the character strings described in the source code of the application, the predetermined character string used in the log output from the application is information associated with the predetermined character string, and the predetermined character string With the ability to convert to smaller string information
And a function of transmitting a log file including a log output from the application to a log file collection server that collects the log file.

  Since the present invention is configured as described above, even when the number of log file collection target servers increases, the load on the network used when sending and receiving log files is reduced, and new It becomes possible to avoid that it becomes impossible to memorize a log file.

It is a block diagram which shows the structure of one Embodiment of the log file collection system of this invention. It is a figure which shows an example of the conversion table with which the conversion table memory | storage part shown in FIG. 1 is provided. FIG. 2 is a diagram for explaining an operation in which the control unit shown in FIG. 1 converts a character string described in an application source code into a character string ID, and (a) shows an example of a state before conversion; b) is a diagram illustrating an example of a state after conversion. 4 is a flowchart for explaining an example of an operation when a log file stored in a log file collection server is searched by a user or the like in the log file collection system shown in FIGS. It is a figure which shows the other example of the conversion table with which the conversion table memory | storage part shown in FIG. 1 is provided.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an embodiment of a log file collection system of the present invention.

  As shown in FIG. 1, the log file collection system according to the present embodiment includes servers 10-1 to 10-n that execute each of a plurality of applications having a function of outputting logs, and a log file collection server 20. ing.

  Each of the servers 10-1 to 10-n includes a conversion table storage unit 11, a control unit 12 that is a first control unit, and a communication unit 13 that is a first communication unit.

  The conversion table storage unit 11 includes a conversion table. The conversion table identifies each of a plurality of predetermined character strings used in a log output from each of a plurality of applications and each of the plurality of predetermined character strings, and a character having a capacity smaller than that character string. A character string ID (IDentification) that is column information is stored in association with each other. The conversion table storage unit 11 includes a conversion table for each of a plurality of applications.

  FIG. 2 is a diagram illustrating an example of a conversion table provided in the conversion table storage unit 11 illustrated in FIG.

  As shown in FIG. 2, in the conversion table, for example, a predetermined character string (Recovering storage directory) used in a log is associated with a character string ID (2).

  In this embodiment, the character string ID is a number, but the character string ID is not limited to a number as long as the capacity is smaller than the corresponding character string.

  Referring to FIG. 1 again, the control unit 12 stores a plurality of applications. The control unit 12 selects the same character string as the character string stored in the conversion table of the application included in the conversion table storage unit 11 among the character strings described in the source codes of the plurality of applications. Is converted into a character string ID corresponding to. Then, the control unit 12 compiles and executes the converted source code. Even if the character string is converted into the character string ID in this way, there is no problem in executing the application. And the control part 12 produces | generates the log file containing the log output from each of a some application for every some application. Then, the control unit 12 outputs the generated log file to the communication unit 13. The log file is generated, for example, when a log output from the application reaches a predetermined capacity or at a predetermined time. The log file is given a log file name that can identify the application that has output the log included in the log file.

  FIG. 3 is a diagram for explaining an operation in which the control unit 12 shown in FIG. 1 converts a character string described in the application source code into a character string ID. FIG. 3A shows an example of a state before conversion. (B) is a figure which shows an example of the state after conversion. Here, the case where the conversion table provided in the conversion table storage unit 11 is the one shown in FIG. 2 will be described.

  Among the character strings shown in FIG. 3A, each of is not a directory, Recovering storage directory, from previous upgrade, Reducing replication for file, and .New replication is is the conversion table shown in FIG. Is a character string stored in

  Accordingly, each of these character strings is converted into a corresponding character string ID in the conversion table, as shown in FIG.

  Note that the predetermined character string stored in the conversion table in the present embodiment is, for example, a conventional character string quoted in the log as it is among the character strings written in the source code of each of a plurality of applications. . Since rootPath, Path, src, and replication shown in FIGS. 3A and 3B are character strings that change according to the execution state of the application when output as a log, conversion tables Not subject to conversion by.

  Referring to FIG. 1 again, the communication unit 13 receives the log file output from the control unit 12. Then, the communication unit 13 transmits the received log file to the log file collection server 20.

  The log file collection server 20 includes a conversion table storage unit 21, a communication unit 22 that is a second communication unit, a control unit 23 that is a second control unit, a log file storage unit 24, an input unit 25, And an output unit 26.

  The conversion table storage unit 21 includes a conversion table. The conversion table included in the conversion table storage unit 21 has the same configuration and contents as the conversion table included in the conversion table storage unit 11. Further, like the conversion table storage unit 11, the conversion table storage unit 21 includes a conversion table for each of a plurality of applications.

  The communication unit 22 receives log files transmitted from the servers 10-1 to 10-n. Then, the communication unit 22 outputs the received log file to the control unit 23.

  The control unit 23 receives the log file output from the communication unit 22. Then, the control unit 23 confirms the log file name of the received log file. And the control part 23 memorize | stores the received log file in the log file memory | storage part 24 for every some application based on the log file name of the received log file. Further, the control unit 23 receives the search information output from the input unit 25. The search information is information including a keyword for searching a log included in a log file stored in the log file storage unit 24 and a search target application name. A keyword is composed of one or more words. Then, the control unit 23 extracts a log corresponding to the keyword and application name included in the received search information from the log file stored in the log file storage unit 24. And the control part 23 produces | generates the log which converted the character string ID contained in the extracted log into the corresponding character string in the conversion table with which the conversion table memory | storage part 21 is provided. Then, the control unit 23 causes the output unit 26 to output the generated log. Note that details of the operation of generating a log by the control unit 23 will be described in an operation flow described later.

  The input unit 25 receives the input of the keyword and application name described above from the outside. Specifically, the input unit 25 includes an input device such as a keyboard and a mouse, and receives a keyword and an application name input by a user or the like operating the input device. Then, the input unit 25 outputs search information including the accepted keyword and application name to the control unit 23. Note that the input unit 25 may have a communication function and receive a keyword and an application name from the outside.

  The output unit 26 outputs the log generated by the control unit 23 to the outside. Specifically, the output unit 26 includes a screen, and displays the log generated by the control unit 23 on the screen. The output unit 26 only needs to have a function of outputting the log generated by the control unit 23 to the outside. For example, the output unit 26 has a printing function and prints the log generated by the control unit 23. There may be. Further, the output unit 26 may have a communication function, for example, and may transmit a log generated by the control unit 23 to a predetermined external device.

The operation of the log file collection system configured as described above will be described below.
Here, the operation of the log file collection server 20 when a log file stored in the log file collection server 20 is searched by a user or the like will be described.

  FIG. 4 is a flowchart for explaining an example of an operation when a log file stored in the log file collection server 20 is searched by the user or the like in the log file collection system shown in FIGS. FIG. 5 is a diagram illustrating another example of the conversion table provided in the conversion table storage units 11 and 21 illustrated in FIG.

  Here, a case where the conversion table is the one shown in FIG. 5 will be described. Note that the conversion table shown in FIG. 5 is a conversion table for application A among a plurality of applications.

  The input unit 25 receives an input of a keyword and an application name (Step S1). Here, it is assumed that the input unit 25 receives an application name indicating the application A.

  The input unit 25 that has received an input of a keyword and an application name outputs search information including the received keyword and application name to the control unit 23.

  Upon receiving the search information output from the input unit 25, the control unit 23 completely matches the keyword included in the received search information with the character string stored in the conversion table of the application A included in the conversion table storage unit 21. It is confirmed whether they match (step S2). Here, the case of complete matching means that if the keyword is composed of, for example, four words “ABCD”, each of “A”, “B”, “C”, “D”, “AB” And “C” and “D”, and character strings such as “ABCD” are stored in the conversion table. That is, the keyword includes a character string stored in the conversion table, and includes only the character string stored in the conversion table.

  As a result of the confirmation in step S2, if the keyword included in the received search information completely matches the character string stored in the conversion table of application A included in the conversion table storage unit 21, the control unit 23 The character string ID corresponding to the character string to be acquired is acquired from the conversion table (step S3). Specifically, if the keyword included in the received search information is “error”, for example, the control unit 23 sets the character string ID corresponding to “Error” or “error” in the conversion table shown in FIG. (100, 200) will be acquired.

  Next, the control unit 23 extracts a log including the acquired character string ID from the log file of the application A stored in the log file storage unit 24 (step S4).

  Next, the control part 23 produces | generates the log which converted the character string ID contained in the extracted log into the corresponding character string in the conversion table of the application A with which the conversion table memory | storage part 21 is provided (step S5).

  Then, the control unit 23 causes the output unit 26 to output the generated log (step S6).

  On the other hand, as a result of the confirmation in step S2, if the keyword included in the received search information does not completely match the character string stored in the conversion table of application A included in the conversion table storage unit 21, the control unit 23 It is confirmed whether or not the keyword included in the received search information partially matches the character string stored in the conversion table of application A included in the conversion table storage unit 21 (step S7). Here, in the case of partial matching, if the keyword is composed of, for example, four words “ABCD”, at least one of “A”, “B”, “C”, “D” This is a case where a character string such as “AB” is stored in the conversion table. That is, the keyword includes both a character string stored in the conversion table and a character string other than the character string stored in the conversion table.

  As a result of the confirmation in step S7, when the keyword included in the received search information partially matches the character string stored in the conversion table of application A included in the conversion table storage unit 21, the control unit 23 A character string ID corresponding to the matching character string is acquired from the conversion table (step S8). Specifically, when the keyword included in the received search information is, for example, “FileA is not a directory”, the control unit 23 sets the character corresponding to “is not a directory” in the conversion table shown in FIG. The column ID (1) is acquired.

  Next, the control unit 23 extracts a log including the acquired character string ID from the log file of the application A stored in the log file storage unit 24 as a first log (step S9).

  Next, the control unit 23 includes a log that includes words that do not match the character string stored in the conversion table of the application A included in the conversion table storage unit 21 among the words included in the keyword included in the received search information. The second log is acquired from the log file of the application A stored in the log file storage unit 24 (step S10). Specifically, when the keyword included in the received search information is, for example, “FileA is not a directory”, a word that does not match the character string stored in the conversion table of application A included in the conversion table storage unit 21 is “FileA”. Therefore, the control unit 23 acquires a log including “File A” from the log files of the application A stored in the log file storage unit 24 as the second log.

  Next, the control unit 23 extracts a log common to both of the acquired first log and second log (step S11). And it changes to operation | movement of step S5.

  On the other hand, as a result of the confirmation in step S7, if the keyword included in the received search information does not partially match the character string stored in the conversion table of application A included in the conversion table storage unit 21, the control unit 23 Then, a log including the keyword included in the received search information is extracted from the log file of application A stored in the log file storage unit 24 (step S12). Note that the case where the keyword included in the received search information does not partially match the character string stored in the conversion table of application A included in the conversion table storage unit 21 is, for example, the date and time of the keyword. This is the case such as “2011-03-22 08:10” indicating the minute.

  Next, the control unit 23 confirms whether or not the character string ID stored in the conversion table of the application A included in the conversion table storage unit 21 is included in the extracted log (step S13).

  As a result of the confirmation in step S13, when the character string ID stored in the conversion table of application A provided in the conversion table storage unit 21 is included in the extracted log, the process proceeds to operation in step S5.

  On the other hand, if the character string ID stored in the conversion table of the application A included in the conversion table storage unit 21 is not included in the extracted log as a result of the confirmation in step S13, the extracted log is output to the output unit 26. (Step S14).

  As described above, in the present embodiment, each of the servers 10-1 to 10-n receives a predetermined character string used in a log output from the application among character strings described in the source code of the application. It has the control part 12 which is the information matched with the predetermined character string, and converts it into character string ID with a capacity | capacitance smaller than the said predetermined character string.

  Each of the servers 10-1 to 10-n includes a communication unit 13 that transmits a log file including a log output from the application to the log file collection server 20.

  The log file collection server 20 includes a communication unit 22 that receives log files transmitted from the servers 10-1 to 10-n.

  Further, the log file collection server 20 includes a log file storage unit 24 that stores the log file received by the communication unit 22.

  As a result, even when the number of log file collection target servers increases, the load on the network used when sending and receiving log files is reduced, and new log files cannot be stored. Can be avoided.

  In the present embodiment, the case where the conversion table storage units 11 and 21 include a conversion table for each of a plurality of applications has been described. However, when there are a plurality of processes that output log files in one application, the conversion table storage units 11 and 21 may include a conversion table for each of the plurality of processes.

  Further, in the present embodiment, the case where the character string used in the log output from the application is the target to be stored in the conversion table has been described. For example, the character string used in the log output from the middleware is converted into the conversion table. It is also possible to set it as an object to be stored. If a character string in the source code can be converted into a character string ID and compiled, the character string used for the log output from the OS can be stored in the conversion table. is there.

  In the present invention, the processing in the server is recorded on a recording medium that can be read by the server, in addition to the above-described dedicated hardware, and the program for realizing the function is recorded on the recording medium. The program recorded in (1) may be read by a server and executed. The server-readable recording medium refers to a removable recording medium such as a flexible disk, a magneto-optical disk, a DVD, and a CD, as well as an HDD built in the server.

10-1 to 10-n server 11, 21 conversion table storage unit 12, 23 control unit 13, 22 communication unit 20 log file collection server 24 log file storage unit 25 input unit 26 output unit

Claims (2)

A log file collection system having a plurality of servers that execute an application having a log output function and a log file collection server that collects log files including logs,
Each of the plurality of servers is
Among the character strings described in the source code of the application, the predetermined character string used in the log output from the application is information associated with the predetermined character string, and the predetermined character string A first control unit for converting into character string information having a smaller capacity than
A first communication unit that transmits a log file including a log output from the application to the log file collection server,
The log file collection server
A second communication unit for receiving a log file transmitted from each of the plurality of servers;
A log file storage unit for storing a log file received by the second communication unit;
An input unit for receiving a keyword for searching for a log included in the log file stored in the log file storage unit;
A second control unit for confirming whether or not the keyword received by the input unit includes the predetermined character string;
When the keyword received by the input unit includes the predetermined character string, the second control unit corresponds to the predetermined character string from the log files stored in the log file storage unit. A log including character string information is extracted, character string information included in the extracted log is converted into a predetermined character string corresponding to the character string information, and a keyword received by the input unit is the predetermined When the character string is not included, the log including the keyword is extracted from the log file stored in the log file storage unit. When the character string information is included in the extracted log, the character string information is And a log file collection system for converting into a predetermined character string corresponding to the character string information . A log file collection method in a log file collection system having a plurality of servers that execute an application having a log output function and a log file collection server that collects log files including logs,
Information in which each of the plurality of servers is associated with a predetermined character string used in a log output from the application among character strings described in the source code of the application. A process of converting into character string information having a smaller capacity than the predetermined character string;
Each of the plurality of servers transmits a log file including a log output from the application to the log file collection server;
A process in which the log file collection server receives a log file transmitted from each of the plurality of servers;
The log file collection server stores the received log file;
The log file collection server receives a keyword for searching for a log included in the stored log file; and
When the received keyword includes the predetermined character string, the log file collection server extracts a log including character string information corresponding to the predetermined character string from the stored log file, and extracts the log Processing to convert the character string information included in the log to a predetermined character string corresponding to the character string information;
When the received keyword does not include the predetermined character string, the log file collection server extracts a log including the keyword from the stored log file, and the character string information is included in the extracted log. If included, a log file collection method comprising: converting the character string information into a predetermined character string corresponding to the character string information .

Images